The thing to do is put a gauge on the well so that you can measure that actual pressure the water is producing. If it's only a few pounds, you can raise the casing high enough to keep it from flowing. The formula for that is for every pound, you would have to raise the casing .433 feet. If that don't work, and your in a freezing climate, you may want to put the casing underground in a pit and use a good Well Seal to stop the flow. If the pressure isn't too great and you are sure your casing is driven in tightly or grouted properly, there should not be a problem.
One word of caution. If this well was drilled poorly and water could be forced up along the casing once you shut off the flow, you could end up with a mess of great proportions. You don't want the water to start coming up along the casing washing a larger hole which can increase the flow which will enlarge the hole even more and the beat goes on.
I know of a few occurrances like that and it was very costly to stop the flow.