It's probably a good idea to shut of both the water and the gas supply to it.
There is a knob on the WH, which usually has three positions: ON, PILOT, and OFF. To get it to go to off, you usually have to press down on it while turning.
Usually, but not always, there's a shutoff valve on the cold pipe comiing into the WH. You can tell which is which if you carefully touch them...especially if you have a faucet running to get hot water...one will be cold and the other hot. If you have a shutoff, turn that off AFTER you turn the gas off. You don't want the tank leaking and the gas still on.
Then, if there is a good place where you could drain some water, and you have a hose, you could hook up a hose to the drain valve near the bottom of the WH. It may have a bunch of sediment in it, so it might not actually drain through there, but try it. Run the hose somewhere it can drain by gravity, then open the drain valve. If you open a faucet on hot somewhere in the house, the tank will drain. That's probably better than slowly dumping it on the floor around the tank. If you can't run the hose somewhere, just shut the water and gas off going in, then you won't get a constant flow out of it. Depending on the size of the leak, you still could get 40 or more gallons on the floor, so draining it away has advantages.
Next time, consider putting an emergency drain pan under the new WH, it will contain and channel the runoff to a drain somewhere.
Now, if the water is leaking from the valve on the top of the heater, and coming out of the pipe directed towards the bottom, you might just need a new valve. Often, you can put a pail under that discharge. But, if it's coming from the actual tank, the tank is shot...time for a new one.