In short, although others may have convinced you already--DONT use primer as a finish coat.
Primer is formulated for good adherence. IF you used a primer-SEALER then it would be more acceptable.
Paint boundries are blurring, in the old (oil) days primer was often just finish paint thinned out. Now formulations are a lot of genuine chemical wizardry and a healthy dose of baloney marketing.
One key is the quality of your primer--if it was cheap (I mean the stuff home despot sells by the tanker load) standard drywall primer, you will be sorry if you don't finish coat it. Good primer has more binder and fewer volatiles so leaves a more durable finish.
For a pretty good discussion of paint tech look at:
www.zinsser.com
However, the short answer is probably that you will be happier in the long run if you use a finish coat.