What's your water pressure? Are the supply valves to the house and WH fully open?
It sounds like somewhere the pipes aren't clamped down well. The plumber may have removed or loosened one or more to get the new WH installed, then failed to replace or tighten them. Just like when you take your foot off the accellerator in your car it doesn't immediately stop, the water, once moving tries to keep going. It's sort of like hitting a brick wall when a valve is turned off quickly..the water tries to keep going, and can't through the now closed valve, so the pipe moves to disipate the energy. You need to either clamp the lines so they can't jump and bang into things, or install hammer arrestors. The whole thing gets worse if the pressure is high...it's like hitting a wall at 60mph verses 1mph.