Also, don't forget that Silver's been putting most if not all of the polls into his tracking method, including the comically useless We Ask America and a few of the more terrible Rasmussen ones. That might be skewing things a bit.
Silver's models weights polls based on their reliability and includes adjustments for house effects, IIRC.
Did Silver include the NYT/WaPo polls of Virginia and Ohio? Neither of those appeared in his list of polls that got factored into the model, but the (R)asmussen polls and ARG ones did.