There's a tool made for this - it's called a nipple extractor. You insert it into the broken part, and use a wrench to unscrew things. It has a cam that expands to hold and turn the fitting from the inside.
Sometimes you can take something small and pry the thing out of there - it is usually easier if you can take a hacksaw blade and cut through it from the inside. If you try that, make sure you don't go deep enough to mess up the threads of the fitting.
This extractor has had the end sawed off so that it won't bottom out in the fitting.