Why is CAPM used to estimate Cost of Equity when Beta only measures systematic risk?

Beta isn't perfect and it is a misleading measure of risk, which is one of the explanations for why the factor anomaly called "Betting against Beta" exists. It's still used though because it's theoretically correct. If every investor were completely diversified, Beta would be the measure of risk to use (assuming you could properly calculate it). Since no one is that diversified, then beta has its flaws.

The main point about beta you seem to be missing or not appreciating, is the fact that beta assumes investors are completely diversified.

edit: too much to explain

in a portfolio of 100000 stocks, a biotech stocks is less risky because it's less correlated with the other stocks in that portfolio :)

The beta calculation is a measure of covariability between market returns and the returns of a single stock. A short step away from the beta calculation is the calculation for the coefficient of determination between the two, or R^2. R^2 tells you what percent of variability in the stock is due to variability in the market. Variability in the market minus the risk free rate (the market risk premium) is THE measure of systemic risk, so beta tells you how much systemic risk is in any one stock's historical returns.

Firm-specific risk can be harder to quantify, because it can include acts of god, CEO plane crash or drug use, shady accounting, etc. You can't just say, "R^2 = 55% of the variability in this stock is systemic risk, so 45% is firm-specific risk."

You should probably be asking: what use is CAPM for doing anything useful with the stock market?

The CAPM assumes that investors are sufficiently diversified such that the unsystematic risk (firm-specific risk) becomes negligible as diversification will help "cancel out" the randomness from unsystematic risk. E.g. if you hold a portfolio of 30 assets, then the firm-specific factors for each company will sometime be positive, sometimes negative - due to the number of stocks held, such "random" variation should be cancelled out (same idea as for "random error" in statistics).

It of course relies heavily on that people are sufficiently diversified etc. etc. etc.

Because in theory, an investor should only be rewarded for holding systematic risk.

Having worked in fundamental research positions on the buyside for 3 years, I can tell you I've never used CAPM once.

Investors differ in their willingness to accept risk for a greater return. But if investors are willing to invest in the stock market, then they are willing to assume some risk. The capital asset pricing model provides is a consistent means to price risk premiums. If you are willing to accept higher risks to get higher returns, then it makes sense to demand a higher return for a higher risk; otherwise, why take the higher risk. By comparing the beta of a stock and its historical return with that of the general market, you can determine whether the return of a stock is worth its risk.