Is now the best time to become a career banker?

With the insanely high attrition rate and apparent max exodus out of IB into places like PE/VC, etc. is now the best time to become a career banker? Aren't career opportunities significantly better much higher if you're one of the few that stick around given that everybody has left? Won't these banks be desperate for more senior people in the relatively near future, making the possibility for a fast tracked career much higher if you stay? 

I get that banking sucks but isn't there an exponential amount to gain if you are willing to stick out for just a few more years or am I missing something.