Print Money (immediate, but causes inflation and can wreck your economy if you do it too much/fast)
Sell bonds (debt) (Immediate, but need to be paid back with interest)
Bonds by default have a period (10Y, 20Y, etc) on them, after which the issuer pays the whole amount back (and they've been paying interest the entire time)
The public debt is really just a bunch of these bonds, and they're constantly cycling through. If the government doesn't want to use tax money or print money to pay them off, they just issue new bonds, and use that money to pay for the previous bonds.
Sometimes the government wants to push the debt down, so they use taxes or printed money to buy the bonds and "cancel" them. Governments don't always run deficits so this happens automatically when they have a surplus.
So to answer your original question, the reason they may not want to, is that they may want to spend the tax money elsewhere, or they may not want to deal with the problems of printing so much money regarding inflation.