If it's what I think you have, it may be defective. When operating properly, it should only open when the pressure in the closed boiler loop drops below its setting. It sounds like it is leaking a bit.
Two things, an autofill valve like that will mask a system problem with a leak, but it will keep the boiler operating because it should never get below the low-pressure cutout safety switch. Second, many times, they'll install a ball valve before the autofill, so they can shut off its function, and it only really gets used when you need to drain and then refill the system after maintenance or repair.
It is not good to regularly be adding fresh water to a boiler loop...fresh water has oxygen and dissolved minerals in it. Once the water has been there awhile, it becomes fairly inert, but that extra oxygen means it can react with any metals that can rust...you really want to keep that out!