Depending on how fine of particles the filter is designed to trap, a visual inspection won't tell you all that much. Once you have a lot of visually observable crud, the filter is long past changing. How much of a restriction the fan and heat exchangers can take depends on the design. For example, it might be designed to only allow a maximum of 15% reduction with the filter. A new filter might be 10% out of the box (verses no filter), and you might not be able to see an accumulation of crud on the filter until it dropped to 40%, but by design, you'd been overstressing the system for quite awhile. While few people change their filters every month, that is the interval some suggest. What you're seeing may be normal. Now, most people don't replace their filter that often and their systems don't die, but they may not be working as efficiently as they could for much of their lives.