We would like a picture of where you're at.
Two things come to mind:
> diverter stems have a back-seat washer, and those sometime get splayed out and can be tight. Usually just needs a lot of wiggle.
> if the wall is tiled, and the tile guy was "too cute" there may not be a hole large enough in the tile for the largest diameter of the stem to come through. If that is the case, you can "worry" the hole out carefully with a small cold chisel.