How easy is it to switch from Wealth Management to Asset Management?

not great, but also not impossible. here's what I'd do, since I work in PWM and before my business took off I daydreamed about moving to Cali and working at PIMCO, Brandes, Oaktree, or TCW.

• get your CFA, you don't have to finish it, but at least pass L2 and be working on L3 when you're recruiting
• crush it at your current job. if you're a financial advisor, you need to leave that ASAP, that's a sales role (esp in the early years) and will not transfer well. whatever you're in, crush it, a turd on the resume will stink up anything else that's good
• work on your investment prowess. join things like sumzero and value investors club, have your thesis torn apart and sharpen your skills. this will mean many nights and weekends refining this
• take the GMAT, crush it (over 700 at least, preferably higher). this will make sense in a moment
• network, network, network. meet with anyone who will listen, learn about their asset class, learn about their work, ask for help, you'll get told no a lot but you might get a hit with a boutique AM shop and it's a helluva lot easier getting into a big name from a small name in the same industry than it is to get into a big name from PWM
• meet with ER analysts from your firm. don't go for the MD, but if you work at Merrill, you have access to their entire directory, use it. if I was in your shoes, I'd plan a couple trips to NYC if you're not there already (outside of earnings season), and say "hey I'm gonna be in the area and I've been reading your reports a lot, would love to pick your brain about X over coffee or lunch (probably coffee). can we visit for a bit at the office on 42nd st?
• if all else fails, use that GMAT to recruit for a top MBA.
• after you graduate b school, go into IB if you can't get an AM job (have heard anecdotally that it's hard to go straight from MBA to buyside if you weren't already buyside)
• from there, try to lateral to buy side as if you were anyone else

best of luck