Not much. The timing was just bad, because the economy improved by late 1983/early 1984. Had the election been in 1982, it would have been a toss-up.
What sunk Mondale as well as his ineffectual debate performance was the fact that Geraldine Ferraro's husband's ethics got the best of the campaign. Mondale would had to dump Ferraro and thus he would have angered women voters and they got him through the primaries.
Actually I disagree on that one. The only really strong debate moment of Reagan was the age joke in the foreign policy discussion. His performance in the domestic debate was awful. Probably Reagan was already in the early stages of Alzheimer's by 1984. He often seemed confused and it took him relatively long to give a response. He was far more vital and agile in the 1980 debate with Jimmy Carter.