To be effective, you might need a larger duct. If I remember correctly, a combustion device should have at least 1sqin/1K BTU or so. Now, it is currently pulling that air through cracks in the house, and yes, pulling it from outside directly or closer to where it is being used would make the whole house feel more comfortable IF you can prevent that cold air from chilling off the basement when the thing isn't running. A 3" duct is only about 7sqin, and every elbow or other restriction cuts its effective ability down. A closed combustion system can get by with a small pipe because it is fan assisted and the outlet is directly coupled, which helps to pull it in as well. A naturally vented device can't handle much restriction, so the air must be provided very freely.