Not every midterm will be a massive wave for one party or the other. It also depends on other factors such as the map (which was the reason Democrats did so poorly in the Senate races in 2014), candidate quality, polarization, etc.
I mean, maybe I'm wrong and every midterm election will be an easy win for the party that doesn't control the White House, but I kinda doubt it. 2018 will be very telling, in all honesty.
The midterms are a referendum on the incumbent president. Not all will favour the other party as sometimes the incumbent president can be popular (2002 and 1998), but that won't be the case this time apparently.