Take this with a grain of salt, since I don't remember which poll measured this, but I recall reading in 2012 that for every two Obama 08 voters that he lost, Obama gained one McCain voter.
Who would those have been? They couldn't have all been PUMA's.
Presumably more fiscally conservative voters who saw that Obama was actually very moderate and didn't appreciate the direction that the Tea Party was taking the GOP.
In 2008, it was still seen as acceptable for well-educated people to support the Republicans. A great deal of damage was done to the party's credibility with the rise of birtherism and the rhetoric that the GOP used in the lead up to 2012.