No, Rand isn't running as a Libertarian. Literally zero chance of that happening. And Sanders probably isn't running either, and if he did, it wouldn't simply be to counterbalance Trump.
Regarding Trump: if he runs a third-party campaign and gets good numbers (10%+), Hillary's the president. Most of the voters Trump will get will be stolen from Republicans (combined with right-leaning voters who probably weren't going to vote for either main party, but still, it would take away far more Republicans than Democrats), so every swing state would surely be a win for the Democrats.
Hillary Clinton (D) 40% 269 EV
Jeb Bush (R) 35% 235 EV
Donald Trump (I) 24% 34 EV
Not sure about that. If Trump gets enough votes to win any state, he'll split the vote in most others and Clinton would win places like Florida, North Carolina, Indiana, Arizona, Georgia, Missouri, Montana, NE-2, et cetera.