How do AM Firms with mutual funds work?

In general - there are a few broad categories of assets under management. Largely break down like this. 

- Retail vs. Institutional - these are the two categories. Retail is exactly what you think it is, assets for individuals, etc. Institutional includes things like pension plans, endowments, insurance companies, etc. From a top down level, they have to categorize and differentiate for reporting to the SEC, so you can relatively simply see the split. 

- Vehicle Type.. generally: Separate accounts vs. commingled funds (CIT's, Mutual funds, etc.). You'll see this generally as an 'option' for how you can invest with them. You can pick a mutual fund they have and simply add money to that strategy, maybe you are big enough and can open up a separately managed account, etc. You might have the same strategy offered in a variety of different vehicles - all for a variety of reasons, many of which have to do with investment policies, types of assets, preferences, etc. 

- Strategy type - bringing it all together, you may have a manager who runs a Core fixed income strategy, for institutional investors, that's available as a separate account or commingled fund (note: i'm just using this as a blanket term to differentiate for illustrative purposes here). 

In most cases, following along on the above, you'll see a mix of these within a firm. To your third question - generally, it works more like: $10b client puts out a mandate search for a given strategy - firm, maybe wellington, will bid on that strategy (either RFP or through a consultant... sometimes both, which sucks). Then you will get, say, a $300m allocation from that $10b to a given strategy. You might use a mutual fund, you might use an SMA - but regardless, I'd focus more on that level than what literal vehicle they place clients in. 

More than you wanted - but that's how I would start to parse out an AM firm and what they do. Then you should consider who their client base is (pull up their ADV - quick, easy) and that will help you track back to a lot of other things. How they may manage assets, what they do, etc. 

I'd make one other note - there is a bit of contention over the 'put it in your own funds' especially if you have a large client who you manage an entire portfolio for. If you look at things like an OCIO model (where you advise on an entire, or potion sometimes of a porftolio - and allocate it to various managers, strategies, etc. for the client) you'll have differing opinions. Some like Vanguard may place the passive portions in their funds - logical... cheapest out there for that stuff - and then for 'actively' managed strategies, they may use their own funds but they are sub-advised (i.e. it's a vanguard mutual fund - but wellington manages it for them)... in theory, removing the conflict of interest of putting your client in your own funds. I'm vastly off your topic - or your questions - but those are some other considerations that start to come up as you go down the rabbit hole.