It sounds like one of two things:
Also, the pipe in the wall was not anchored properly so the pressure when you shut off the flow pushes the spout out. If you see the silicon foaming, you are spraying water into the wall like a water jet...stop until you fix this or you'll have a much bigger mess. If the pipe is no longer round, you'll have a tough time getting a push-on spout to seal.
- if it is a slip-on spout, you deformed it enough when tightening the set screw to allow it to leak when there's any pressure (like when you use the diverter to take a shower). Normally, there's no backpressure, so it flows out the spout. The setscrew just keeps it in place and from turning, it has nothing else to do with the actual water seal.
- you damaged the o-ring seals when you installed it. The end of the pipe can't have any burrs or sharp edges, or you risk this.
Now, if this is a screw-on spout, you probably didn't use teflon tape or pipe dope to seal the threads. That kind of joint can't be made waterproof by just tightening it up without something to seal around the threads (like the tape or dope).