Correct. If the WH is in conditioned space, the "waste heat" isn't really wasted during the heating season, at least. However, it may be heating a space that maybe doesn't need the heat (such as a closet or unfinished basement). Although the heat will eventually help out where you need it, you may be better off trying to reduce the heat loss there and adding the heat where you really need it.
It also depends on what type of WH you have. If electric, modern tanks are well insulated and really wouldn't see much benefit to a blanket. It depends on the electric rate, but typically these electric tanks might have $20-$30 per YEAR in standby losses, so you can see that it is hard to justify spending much to try to reduce that.
If it is a gas WH, a lot of your losses are up the flue as well as gas WHs typically have less insulation than the electric models. The blanket may help more on these units, but you are still losing a bunch up the flue.
Next, you have to think about your heating sources and the cost to run. If an electric WH, that is really just electric resistance heat. If the rest of your house had electric baseboards or an electric furnace, than the waste heat from the WH costs exactly what the space heating costs (BTU for BTU). If your heating was a heat pump or perhaps nat. gas with a high % furnace, then your space heating might be cheaper than the waste electric heat.
So, it depends on the situation. In most cases, if newer electric WH in conditioned space, then probably don't worry about it. If gas, maybe some benefit.
What you can do, that will help in any situation, is to insulate all of the hot water piping that you can get to. 3/8" foam stuff will help, but 1/2" or 3/4" wall material is better. Doing this helps reduce amount of water wasted (and energy wasted) during repeated hot water draws (don't have to wait around as much waiting for decently warm/hot water to arrive). Water will still cool off over long periods (such as overnight), but can really help keep the water temp up in the hot water lines during the day.