Depends....
If the faucet is screwed on, maybe the easiest thing would be to replace it. If it is soldered on, then, depending on how comfortable you are, you could solder a new one on. If you don't want to worry about it for awhile, shut off the internal valve and forget about it. Any water trapped in the short pipe may freeze and mess things up, but it shouldn't create a mess if the inside shut-off works.
Do you know if this is a frostfree sillcock? These have a long stem and the actual valve is inside the warm(er) house/wall rather than being outside. If you are going to replace it, it may be worthwhile using one of these instead of a straight valve.
Usually, by opening the valve slightly to relieve the pressure on the seat (the internal seal), you can then unscrew the nut holding the stem in place. Once that is out, you can see all of the guts. Typically, there is a washer on the end of the stem held in by a screw. If it is missing, then it is stuck inside of your faucet. The suggestion to then turn on the water momentarily was to dislodge that and expell it from the faucet. Once you have that out, take a look at the seat (the part where the washer hits). If it is smooth, then just replacing the washer and probably the screw that holds it in if it is corroded will probably solve your problem. If the seat is not smooth, if it is replaceable, you can remove it and replace it. This typically takes a special (inexpensive) tool. If all else fails, you can replace the faucet entirely. Note, I'm not a pro. My unprofessional experience.