Staying in IB with good comp and W/L balance, too risk averse?

Look, I'm not trying to brag but genuinely want to get some thoughts on my current outlook and general advice on the question I'll pose below. But here's my situation (and fully aware that many in the industry have it differently):

I honestly have not found IB to be as brutal as so many have portrayed it to be. A1 comp was 175K all in and A2 comp from what I can tell will be only slightly below or flat (rough year). I'm on a pretty paved path to being promoted to associate (where comp is closer to 300-350K and less b*tch work) and while the work may be mind numbing at times, it's not all that bad (but ill have to do a third year as analyst first which is pretty much an associate role without the money). Here's the best part, I can confidently say I've worked (like actually worked) closer to 45-55 hours a week on average (with an occasional 80hr week now and then) while still being on 2-3 live deals at a time throughout my career so far. I think this had to do with me gaming the system with staffers to work with people I actually like and with VPs and MDs that have a clear vision of what they want up front so iteration is very limited. I get to WFH generally whenever I want (although they do expect me to come in a couple days a week, but still very flexible). Now on the hours worked, I still have to work late sometimes because the work doesn't actually hit until the evening so I'm not counting random idle hours. With the way things are, I get to have a good work life balance, take care of myself (I get to work out every day) and see my friends pretty often, go on dates, have hobby, etc. Even have a side businses where I get an extra ~30k (post tax) with minimal incremental hours; and tinker with various start up ideas with friends. 

I've gotten offers from a couple of MM/LMM PE firms (full honesty, my shop is solid MM but we don't place crazily into top PE) but hours seem worse with less comp so I'm not interested. From what I've heard too is that the work is pretty similar and the upside towards becoming principal is 10x more competitive. Again

What's been on my mind is that I feel like I am playing it so safe and not exploring different careers (i.e. startups and operation roles) that might be something I could actually enjoy / find more fulfilling, despite lower comp. Another thing is that I am financially pretty stable for my age (24m) with net worth of ~150K (very fortunate to have graduated with no debt and some investments doing well) so I can afford to take risks. The one thing I do know is that I don't see myself (skillwise and desire-wise) becoming a salesperson down the road as an MD / Director. 

If you were in my boat and knew that the peak role of this career is not something you want to do but have the option to benefit from a comp perspective for many years, would you stay or get out and try to succeed in another industry / role where you see yourself for the considerable long term and develop the relevant skillset earlier on?

tldr: Making good money right now with relatively low hours in IB with opportunity to make significantly more if I stay on, but worried I might be missing out (or will regret) on other opportunities / experiences and pigeonhole my skillset. What would you do?