Why not GSE lending? Stigmas?

I'm 2 years out of college and have been working in GSE underwriting and now production. I read on here a decent bit about people advising to avoid the GSE groups at banks/lenders if possible. Why is that? My office does pretty much all GSE but other offices in the firm do balance sheet, Mezz, cmbs, etc... I don't have a ton of exposure to them but pay seems comparable, GSE business is half the debt market, and loan originators for GSE loans make very good money. So why do people advise to join other lending groups if possible?

If I had to guess it would be because GSE business is more vanilla and can get repetitive. Thoughts?