Well, in the 20th century, the following lost re-election:
Taft: Third party challenge split the vote, though he was also somewhat unpopular.
Hoover: Worst economic crisis in our nation's history occurred.
Ford: Doesn't really count, as he had never been elected.
Carter: Poor economy, foreign affairs failures, Reagan was a much better campaigner.
Bush: Poor economy, third party splitting the vote, Clinton was a much better campaigner.
So, I think for Obama to lose, the economy would have to stay very bad, and I just don't think that's likely. It may still be going poorly by the 2010 midterms, but by 2012 it'll have been five years since the recession started. Things should at least have started to get better by then.
Those conditions were pretty much the ones I was thinking that would cause Obama to lose, although I won't start guessing where things will be at the next election. I think you can add to Ford the whole Watergate thing had blown up since the previous election and probably tarnished the Republican brand, additionally, pardoning Nixon may have translated to failure in the eyes of the electorate. In the case of both Bush and Ford, they followed a two-term president of the same party... the party's fourth consecutive win is hard, regardless of the benefits of incumbency and even though it hasn't been the same president consistently throughout the terms.
