I mean part of the reason incumbents usually win reelection is they're usually at least somewhat popular (48%+ of the country likes them). We've just happened to have fairly popular presidents in the past 100 years, but if you look back into the 1800's, a lot of them didn't get reelected.
It's hard to say how important incumbency is because we don't know the approval ratings of all of the presidents who lost in the 1800's.
But yeah as for factors for/against
- Are they better today than they were 4 years ago?
- Accomplishments
- National Security
- Strength of America 4 years before
- Money (this can work both for or against -- Incumbent presidents will usually have a much easier time raising money, but a very unpopular incumbent can mobilize the other side).
The President has a great deal of power to control people's views as to the above. Folks might vote for a President and, afterward, come to think they've been had. That doesn't matter, however, once the votes are counted, other than the ill will it would generate for an incumbent President's second term.
