MBB Fellow with an interesting opportunity

I am in a unique predicament and would truly appreciate any insights. I graduated in May from a non-target school, and I earned myself a spot in a rotational program at an MBB firm. For context, I studied finance and have always held financial internships in the PWM / WM industry since I was in high school. I am extremely passionate about the markets/investing, and it sucks not being exposed to the markets. I didn't anticipate going into consulting, was a tough job market, and I was between the MBB firm and BlackRock, and I ended up taking the safer and more "prestigious" name. 

I have a good mentor / mentee relationship with the Founder / CEO of a decently sized PWM ($10-12B AUM) shop where I have previously interned. There may be an opportunity to go work with him directly and learn the ropes of the PWM space after a portfolio that I had built for the firm at one of my internships did really well. 

Many family and friends have told me that the opportunity was too risky and that leaving such a "prestigious" firm would be a mistake. However, I feel that the opportunity in PWM may be too good to pass up. 

If I was to stay in this program I would want to do PWM after anyways, so is it worth making the jump now after being on the job for a month?