How do I remove diverter handle???

OK, I talked to a local Plumber about the diverter I mentioned on a post yesterday. According to him the diverter should completely shut off the flow to the tub spout.

He said to take off the handle and bring in the internal stuff to his faucet shop.

Heres the issue, I took off the little screw that holds on the handle, but the handle does not come off. It does rock a little bit. I am afraid to use too much force when I rock it, for fear I will crack what I think is pot metal.

Does anyone have any suggestions???

If you could find a handle puller,that should do it......