Electrical code requires that all metallic piping be bonded to the electrical ground system. That includes both hot and cold potable water if they're copper or iron. You could run a ground wire from the breaker box to both cold and hot water pipes, but it's also pretty common to run a ground just to the cold and then jumper across the hot water heater to bond the hot to the cold. Some electricians prefer to do it the second way because it's very obvious to the inspector. Your water heater doesn't count as a ground path because it can be disconnected, but you wouldn't want it to be anyway since it would corrode from carrying stray ground currents. (I discovered exactly this in a house I had long ago, when prior homeowner had disconnected the bonding jumper. The pipe corrosion eventually blocked water flow).
Electrolysis is the circular current through the steel tank, copper pipe, and then back through the water. A hot water heater should be connected to copper pipe using dielectric fittings on both inlet and outlet to prevent electrolysis -- dielectric nipples, flexible tubing, and dielectric unions are all fine (although it's easy to screw up the unions so I don't like them). Electrolysis happens at a single hot water heater connection; that bonding jumper doesn't stop it.