Anyone go from IBD Analyst --> PE --> Back to Banking?

I'm currently a second year associate in PE at a big MM fund after spending two years in BB IBD. I'm staying on with my firm for a third year but have to leave after that (promote to VP is unlikely because my firm recently hired a few people at the mid-senior level). I've spent a lot of time thinking about what I want to do next and honestly going back to banking is sounding more and more attractive. I had a very good rep in my old group so I think they'd be happy to take me back if I reached out.

Am I crazy for even consdering this? I know this goes against the conventional wisdom on this forum (and maybe I'm just looking back with rose tinted glasses) but I enjoyed my time as an analyst and I'd think that by coming back as an associate I'd be positioned well compared to the rest of my class.