Career Switch? RE --> Finance

Should I stay with a RE Developer or make a transition to finance?

I've worked in RE for 2+ years, first in financing, and more recently at a prominent developer. Since joining the developer (6+ months), I have been chronically underworked and have not learnt as much as I anticipated. My initial plan was to join this developer specifically because it is a smaller shop and there would be large learning opportunities for myself. These learning opportunities have not come about to the degree I imagined. Every week / 10-days I make myself available to my team and ask for additional work. I am often met with a small administrative task to be completed, after which I spend most of the day browsing the internet or participating in online classes. That is not to say that I never do work or never learn things, but by and large, I have quite a bit of free time at work with no deliverables due. On a weekly basis, I barely scratch 40hrs/wk in the office, and am actually doing work-related tasks for maybe 20hrs/wk. Due to the small size of the shop, there is little room for career advancement (25 in company, 4 in group). The compensation package is quite good, especially considering my age / experience. Finally, at maybe of least importance, the company is well-respected in the market and has strong connections.

But, an opportunity has presented itself. It would be to join a MM bank (arguably the strongest bank where I live) as an institutional analyst. I previously worked with a PM there who was considering poaching me from my previous role before I left to work at the RE developer. The PM is bright and I believe I could learn a lot from him. The size of the bank and the group I would join would open doors/routes for advancement and career growth. But, the compensation package would likely be worse, and the level of responsibilities / tasks would likely be a step backward from where I am at currently. I am set to meet the PM to further discuss what the role could/would look like were I to take it.

Is this taking 1 step backward to take 2 steps forward? Or am I just being impatient/ungrateful at my current outfit? What details should I iron out when I meet the PM?