Assuming leaving the primer as the top coat is 'to code' then I'd inspect it really well in different light to assure it looks good all the time.
Also, you won't get as good surface protection with a primer as you will with a top coat (w the poss exception of flat top coats).
Tops and primers are different beasts. One preps the surface and is optimized for the substrate, the other protects the surface and is optimized for the light and contact it gets.
If it were me, I'd spring for a standard white top coat. it won't cost that much, and my instinct is in the long run you'll be glad u did.