Yeah, but if you look at the polls in 2004 they were (IIRC) fairly accurate, predicting Kerry and Edwards ahead of the others. Of course, what makes things really bad is not the things you mention but the fact that people are allowed to change their vote when their candidates get knocked out in their specific caucus, etc.
The very last round of polling in 2004 (as in, the polls taken within about a week of the caucus) did tend to show Kerry slightly ahead, but they grossly underestimated the margin by which Kerry and Edwards finished ahead of Dean and Gephardt. I remember going into caucus day, people were saying that any of the four main candidates (Dean, Edwards, Gephardt, Kerry) could plausibly win, because they were all packed so closely together in the polls.
And of course, for months in advance of the caucus, it was Dean and Gephardt who were leading all the polls there. It was only at the end that Kerry and Edwards surged.
This is something that it's important to remember. IA and NH tend to have a big impact on the race, and they can also be very unpredictable. There's often a lot of movement in both states just days before the election (of course, some of the movement in NH is caused by fallout from the IA results).