Jackson had the stain of the Trail of Tears so to many that is irredeemable regardless of anything else he did. To argue his anti-central banking, limited government stance as a positive does not mean I admire him overall and his treatment of non-whites was deplorable.
Like certain historical figures both of the distant and recent past he has a very mixed record. To some of these same people who view history through today's racially sensitive lens any history a particular white American or European has with slavery makes it hard to value any contribution they made. Thus Jefferson and Madison are now under fire and any and all pronouncements and actions these pre-Civil War slave-holding statesmen made are considered worthless and hypocritical by certain figures. Thus even if a person promoted militarism, government waste, and greed that they were not a slaveholder means they are more likely than not considered to come out on top in any particular discussion on the merits of a historical issue - even on one where slavery was on the outside of the main discussion.
The cutoff right now stops at Woodrow Wilson, so far FDR, Teddy Roosevelt - who is widely admired by the left and right alike is a Rushmore deity - have been spared the wrath.
Many of the same defenders of Lincoln also vehemently defend TR and FDR - regardless of party despite all three having huge liabilities from the perspective of individual liberty and racially motivated acts of a oppression committed against their own country's residents and others overseas in the name of our empire.
I (and many others) am not the biggest fan of TR, Jefferson, and Madison as well, for reasons other than racist policy.