Welcome to aviation, but beware-like anywhere else, there are lots of people peddling wares that you don't need. Kind of like the gift shops at Disney World.
For now get a good headset (David Clark, Lightspeed, etc), a small flight bag, an E6-B, and a couple of books. The FAA's Airplane Flying Handbook, the Pilots handbook of aeronautical knowledge, and the Gleim private pilot written guide are what I would get now. All of this should cost under $400. Leave the rest for later-you'll know when you need more.
The Jeppesen books are a mixed bag. They contain vast amounts of information and are good references later on, and have lots of pictures. The downside is they are very expensive, and are intended for use in a structured ground school. I liked them as a student, but disliked them as an instructor. Jeppesen likes to sell them in kits that contain lots of extras you will never use, so if you are going to buy the book look for a place that just sells the book.