How much to prioritize coverage group for future career prospects?

I lateraled into IB recently and bc I wasn't in a position to be picky (I was gonna take any offer I was lucky enough to land), I landed in a coverage group (healthcare) that I don't necessarily want to be in long term. The firm is an elite boutique (Lazard/Gugg/PWP/etc) and I'm really happy currently. However, I'd rather be in a TMT type group. Now that I'd have a much easier time lateraling into a BB tech/media group, which are industries I actually want to pursue post-banking, is it worth moving again during my banking stint just to get into a group that might be more applicable to my long-term career goals? Or is it silly to put that much emphasis on something like this?