It was Iraq and the Republicans moving too far to the right. In 2006, it was mostly moderate Republicans in swing/Dem states who lost. Republicans lost 3 seats in NY, 2 in CT, 4 in PA, 2 in NH, 2 in FL, 2 in IA, and 1 in CO. None of these are conservative states - nor were the districts patricularly conservative or Republican. A couple of conservative Dems unseated Republicans, but they were the exception, not the rule. The conservative base turned out, and Republicans actually did well in conservative states like GA, AL, and OK; they were just overwhelmed by the anti-GOP vote of independents and Democrats.
As for worst Pres candidate, it's Goldwater. What happened with conservatives later didn't matter; he was a terrible candidate for 1964, and was on the wrong side of several major issues that year. Even with the Deep South backlash against civil rights, he still managed to do worse than Mondale or McGovern. He didn't win a single county north or east of Pennsylvania.