Is growth worth an industry change?

So a super quick snapshot of how I came to this intersection:

My first job out of college was at State Street Global Advisors in their AML group on contract. I hated it but did well. My boss at the time offered me a full time position but declined because I had been extended a job offer at a boutique Investment Management and Consulting firm. I really like my job and am compensated well. However, my old boss at State street reached out emphasizing how much value I could add to his fund accounting team, he wants to bring me on as a senior associate which would be a step up in regards to the corporate ladder. Additionally, the company that I'm at now is small, and as I see it (though I could be wrong) only employee turn over or fast growth could provide me the opportunity to move up, whereas I see this being less of a concern at a company as large as State Street, especially since my old boss seems to think well of me.

I've heard mixed reviews about fund accounting and generally good things about IM and Consulting. Is the opportunity to grow at State Street worth leaving IM/consulting?

All thoughts, questions and reccomendatons on how I should move forward are appreciated.

Thank you.