Is Harvard enough?

No. It is not enough. Even if you were a finance major and had a perfect GPA, and although coming from a target school helps, what banks are really looking for is previous internship experience. Also, while I am on the topic, I would say that you should either switch your major, or if you really want to keep doing the physics thing, and major in finance. You should def try to get an internship in finance, and yes PWM or AM are really the only things you can get into as a freshman, and even then you may have to wait until the summer of your sophomore year. To answer the last question, I would say that IB is a lot than physics on the math side, depending on what department you go into, but it is harder in that you will have long hours and may get treated pretty badly by superiors, due to the hierarchical nature if IB/PE. That said, you will have to develop a thick skin (if you don't already have one) to work in IB/PE. But the bright side to the long hours is that you get great pay. Last thing this is the right forum because you are talking about IB/PE.

Hope that helps and good luck!

Why are you calling it a fucking badge

How about you just get real and ask it like this: Hey WSO, I am a Harvard student and I want to know if my preftigious piece of paper alone is enough to get me into IB/PE because fuck networking and internships, right?