Wealth Management options

I am in the Wealth management space. I currently have a good job with a client book and am paid well. My concern is that the client book is not technically mine (think of us as advising other firms clients) so if that firm leaves I can lose a lot of my book quickly since the underlying clients would go as well. Our firms growth has also slowed. I have been recruited by a firm that has older Advisors that have significant books and will be retiring in the next 5 or so years. They want to bring me on to work new business and transition their clients through a succession plan over the long-term. The upside is significant as my pay could double or triple once they retire. Main downside risk is I would have to focus on closing new referral business for the first year or two. If it didn’t go well, there is always risk that the succession plan is altered since I haven’t worked with these people before. My current firm I’ve built a lot of political capital and respect so very little risk. Thoughts?