UPDATE (for the proverbial record):
The problem was, indeed, that galvanized reducer. The plumber removed it and the end of it was very rusty.
This was a brand new reducer (and he only put it in because he didn't have brass at the moment he needed it). It was a pain to get it out, but he agreed that he really shouldn't have used it. It was just the edge (not the interior)... and I guess sometimes the galvanization isn't so good. It was amazing how much rust could be produced in a mere 12 hours. I've attached a photo of it; it was brand new just 3 months ago! Also interesting, was the rust would drift down the supply to a lower spot such that the first few inches of water would be mostly clear (which is why I thought there might be a screw in that pipe a bit further back closer to the valve).
Anyway, it was replaced with a proper brass one and there you are, no rust.
Thanks to all for the feedback.