Two things to look at. First, when you drained the tank, did you sponge out the water that always remains after flushing? If not, this water will drain from the supply line when you loosen it. If the tank is dry, then I'd suspect a bad cut off valve on the supply line. Shut off the main water supply, install a new valve, and see if that doesn't fix the problem. "New" water should not come into the tank with the valve off.