The brass ferrule is the reason why they could make a solid connection to the CI, and you can't make one at the other end which is only lead. WIth it cut off there, you could take a hacksaw blade and cut a notch through it on the inside in a couple of places, then peel it out rather than drilling out the lead on the outside. It's a pain, but if you make the lead into swiss cheese, you will be able to pry it out of there.
Important note - I'm not a pro
Retired Defense Industry Engineer; Schluter 2.5-day Workshop Completed 2013, 2014