Should I be trying to upgrade?

Short answer no. You know that you shouldn’t worry about this and are asking for reassurance. I will give it to you.

1. Be grateful you have a job at all. Also realize many people at those other shops might have their jobs cut. Especially in PE it’s starting to and about to be a bloodbath. Having a job at all at a good firm puts you in a unique class of individuals. This will become more true as people get laid off.
2. You have no idea what the job entails. You might love IB or hate PE. Eventually as you gain experience in your career you will be able to make a more informed decision on what the best path is. The optimization you are doing now is like a 3rd grader saying they want to go to the best college for space exploration. It could really be they become an astronaut, but likely they will learn a bunch between 3rd grade and college that will have them choosing an alternate career path. Just focus on learning and you will learn plenty in your current seat.
3. The hair splitting you are doing is like comparing across ivy league schools. Sure, Wharton might send a ton of kids to the “top shops” but ultimately really smart kids at Dartmouth do just fine. Most Firms aren’t so stupid that they only hire from Goldman and ignore all other resumes unless they have only that name. Also, if a firm really did hire like that, would you really want to work there? They probably are crap investors because they aren’t trying to hire the smartest people and are instead focusing exclusively on people that all have the same experience and background.

Personal choice is going to play a much larger role in your future earnings than the minor differences between the firm you are at and firms other people go. There are 2 ways to make a lot of money:

• Take risk and venture off to a new fund or strategy that is growing where you get in early
• Work hard for a very long time and move up the chain at an established firm.

Figure out which one you want to do and what you are willing to sacrifice to get your goal then work toward it. Do you really think MD’s at your bank wish they were at Goldman? Prob not. The key is optimizing for whatever you want and there are so many factors like location, hours, culture, politics, strategy, that it’s really not worth obsessing over comparing the prestige of your first job. Especially when the differences you are talking about are so close.