Leave the water on otherwise, you'll only get gravity rather than water pressure to help flush out crud. Pluse, if you didn't open a valve somewhere else in the house, you'd have the equivalent of a big straw that you've got your finger over the end. Remember, the water will be hot. Some hoses can't handle heat, plus the metal part of the nozzle could get too hot to hold, so run it where it won't hurt anything. As noted, if you do end up flushing out a bunch of calcium deposits, it can make it hard to get the drain valve to seal if some of it gets caught under the washer. this is something that you probably should do on a regular basis (depending on the water hardness), maybe every 6-months to a year. If it hasn't been done, and it is old, you might not get anything out since it could have solidified into a big hunk. You can often tell when that happens, since the tank will make sort of a burping sound as the deposit creates hot spots and little steam explosions when the burner or elements are on.
On a new tank, it's not a bad idea to have the (usually very) cheap valve replaced with a ball valve that you can be sure it will seal again.