Tainted as a loser. Plus, unlike Palin, he was never particularly loved by the GOP base. Ryan's base is the Very Serious People in the beltway media, which isn't a particularly strong demographic.
I personally don't find Ryan charismatic. He can be incredibly awkward. To me, he comes off as this geek who spends his time memorizing a bunch of stats. Not to mention, he has never won a statewide election. He's fine in a related role in the House, but I don't see him as much more than that.
I agree with all of you. He has no "base", sadly.