I think you're misunderstanding what the regression model was for. The regression model ended up worth less than one crappy poll in most states. It wasn't for picking up trends the polls didn't pick up. It was for picking up the slack when a state wasn't being polled. For instance, the assumption was the Dakotas -- being highly similar -- would move together. So if SD didn't have a poll for a month, but ND had a few, SD would be adjusted to move along with ND. In some cases, like Indiana, Silver's model was skeptical because the demographics suggested that it shouldn't be close. But eventually the IN regression came into step, and even when it was out of step, it was the difference of a point and a half at most. Why? Because Indiana did have polls, and the model ceded to them.
Silver also provided a raw, trend-adjusted polling model. That included only subsequent national poll movement (downweighted), pollster quality (see previous caveat), sample size. True, how much sample size is weighed is arbitrary. But we do that kind of things in our mind, and it always ends up more arbitrary that way.
The RCP average is really the same thing as the Atlas, a "dumb" average.
Ah, it's all coming back to me now. Point accepted on non-poll factors. But I think was a bit sceptical of the weighting of the polls. I guess it was a general impression of a lot of weights and stuff that didn't seem solid enough. I like to keep as close to raw data as possible.