1984 was very much a popularity contest for Reagan...there were a lot of people who came out just to vote for him and nobody else. This explains the wide downballot disparity (Democrats actually won the popular vote in both houses despite Reagan’s 18-point landslide). My guess is that the overwhelming majority of Reagan-only voters in 1984 probably didn’t vote at all in 1980, thus explaining the spike in turnout.
This could be partially true. Another theory added- although Reagan did win the overall women’s vote, Geraldine Ferraro on the Democratic ticket likely caused somewhat of an uptick in turnout among women overall perhaps adding to higher turnout over 1980.