You would really be much better off using a 6" wall cap. Using the 4" wall cap will definitely result in a reduced air flow and more noise than using a 6" wall cap. The short 8" section of 4" round duct isn't really that much of a problem, but the 4" wall cap itself will be a major factor. What is the added effort of trying the 4" wall cap and using the results to determine whether or not to change out the 4" wall cap for the 6"? If it isn't too much effort, I say go for it. It may prove to be educational.
According to the Panasonic site, a 4" wall cap is equivalent to 30' duct length and a 6" wall cap is equivalent to 40' duct length. Using one of the calculators at http://efficientcomfort.net/jsp/DuctCalculator_Web.jsp yields:
30' of 4" round duct @ 150CFM = 0.9732" wc friction loss
40' of 6" round duct @ 150CFM = 0.146934" wc friction loss
Assuming you have a FV15VQ, it is rated for 150 CFM at 0.1" WC static pressure and 123 CFM at 0.25" WC static pressure. What is the fan going to do at 0.9732" WC+ of static pressure?
This is the loss for the wall cap itself. You'll need to factor in the rest of the ducting too.
Now here is strange thing. The 110 CFM version of that fan has a 4" outlet. Using a 4" wall cap with the 110 CFM gives a friction loss of 0.525033 "wc. It seems that you may want to use a 6" wall cap with that fan too even though it has 4" outlet.