How to think about benefits of taking up debt with respect to debt tax shield

Hey guys, not sure if there is a better place to ask technical questions since I am new to the website. One concept I'm having trouble with is how to think about benefits of tax savings from taking on debt in different situations. I understand that taking on debt will lead to a debt shield in the form of tax savings as interest is tax-deductible, and due to lower cost of capital if the D/E ratio goes up, assuming that interest rate on debt is lower than cost of equity.

However, isn't there another side to this, which is that you are actually spending money on the interest on the debt? I'm trying to think about how to weigh the two sides of the equation properly when assessing different situations.

For example, I was reading up on LBOs and saw that one major cost savings when a LBO firm acquires a company using debt is the tax savings due to the debt tax shield. However, interest payments must be higher than the savings in taxes so I don't really understand how this is a major reason of synergy. Sure, debt tax shield would increase your returns compared to a situation in which interest payments aren't tax deductible... What am I missing?

Thanks,
by a noob