If the expansion tank isn't sized properly or precharged to the needed level, it could allow the pressure to rise more than desired. But, it's more likely that the autofill valve is potentially leaking or incorrectly set. FWIW, an autofill valve can mask system problems, and you may want to just readjust the pressure to design, then (hopefully you have one) shut the water off to the autofill valve. If you have a leak, having it open can mask that to you. Constantly adding fresh water to a boiler is problematic, and you really want to know about it. IF you don't have any leaks, shutting the autofill valve will have no effect on the system operation. If you do have a leak, instead of creeping up, it will gradually drop as water leaks out, eventually tripping the boiler's low-limit safety device interlock.
If it's something else, it's still a good idea to shut the autofill valve and you'd have eliminated one of the possibilities.