Worth it to lateral from MM?

Currently at a MM coverage group  (RJ/Stifel/Lincoln/Oppenheimer). I've been with my team for about 9 months after lateraling from another finance role - without going into too much detail, I strongly dislike my group culture and have had a pretty negative experience in IB so far. I'm looking to exit asap to a better wlb buy side job (looking at secondaries/co-investing/credit) and am wondering if it's necessary to lateral to a better bank before making the jump. I don't care about doing buyout at Carlyle/Apollo/etc because the culture sounds awful and I'm just not interested in that grind (even if I had the credentials to get looks from mega funds) - I'm really focusing on investing roles outside of PE. Frankly I'm pretty burned out and tired of banking, so I'd really only consider lateraling if it dramatically improves my long term career prospects.

Any thoughts or advice from individuals who have made the buy side switch from these banks or other MM platforms would be appreciated!