With the volume you have, and 150cfm fan, you'll get about 6 air exchanges per hour, which isn't too bad. Some of the Panasonic fans will work with either 4 or 6" ducts, but the distance you can vent it will be shorter with the 4" pipe - you need to read the specs very carefully. Some of them also allow you to customize the output (i.e, slow it down) should you need or want to for the best operation.
Panasonic does may a through-the-wall fan (don't remember the size) that I'd prefer over going out a soffit. FWIW, I put a Panasonic fan in my mother's house this spring and the outlet was actually stepped so that you could put either 4" or 6" ductwork on it...but, the outlet of the fan was actually 4". By expanding immediately to 6", the overall restriction isn't a big deal. I ended up using 6" since with the elbows I needed along with the length, that was what was called for.