You're on target. Near where the wire is pushed in, there will be a small slot with a brass spring visible down in the slot. That spring digs into the wire to make the "back-stab" connection. Usually there's a stamped marking of "Press to release" or "Press spring thru slot to release wire" or something like that. With a small screwdriver, do just that -- press the brass spring, to move it away from the wire and it will release. Sometimes it's easier if you first push the wire in a bit to allow the spring to "un-dig" itself from the wire.