It sounds like whomever installed your flange used an internal mount flange on a 3" pipe. This in itself is a problem, as good practice only allows an internal mount flange on a 4" pipe. Narrowing it further is asking for problems as the waste exits the toilet, it is generall in a turn, and needs room to make that turn without slowing. Narrowing it more is just asking for problems.
Hindsight, but the proper time to have addressed this was before the new floor was installed. The proper place for a flange is on TOP of the FINISHED floor, not burried in it. I'd definately go with spacers rather than further narrowing the hole. If I had the option from below, I'd consider replacing the flange and moving it where it is supposed to be and then ensure I used a flange that didn't narrow what sounds like your 3" drainage pipe.