Exterior Faucet Shut Off Valve Leaking

I just bought this house a couple of months ago, in February. While the home inspector was there he shut off the valves to the exterior faucets, apparently the PO didn't care about freezing pipes. Any way, I went to turn them back on, the one on the front did not have a problem, the one in the rear of the house immediately began spraying water. When I shut it off all the way there is no leaking. Upon further inspection it appears to be spraying from drain valve / bleeder valve. I think thats the correct term. It looks like a little cap that screws off with a small hole in the side of it. Any ideas what would be causing this to leak? If I shut the valve and remove the cap no water comes out.

The little cap is likely missing its little rubber sealing washer. You can make your own to replace it, or any decent hardware store
should have replacements. They come in different sizes, so make a note of the size of the threaded stub the cap screws onto.

Great. Yeah when I completely took it off there was no washer that came with it. Would it help to take the cap to the store to compare sizes?

As an update. I got the parts. You were right, it was a little rubber washer. That did the trick, turned it on, no leaking. Thanks.