It's because of connotations that have developed over time, mostly influenced by media and academia. Left is associated with compassion and equality, so far-left is considered naive, not evil. Right is associated with authority (especially outside the US, where liberty is core to our history, therefore leading conservatives to have a liking for libertarian appeals), so far-right is thought of as totalitarian and evil.
The left-right divide is in and of itself absurd, and brought about by the "us vs. them" nature of political parties (specifically in the US and other two-party systems). And it makes the right more of an ideological hodge-podge. The idea that libertarians inherently have things in common with nationalists, fascists, or the religious right is laughable, when they have completely different core values/motives.
In reality, there are many more than two political ideologies. Unfortunately, organized politics leads these sides to form alliances, sometimes strange ones, in order to win majorities, thus blurring the lines between the nuanced or even fundamental differences they have. This is compounded further when you have these "coalitions" being two whole parties, thus leading to a duopoly of politicians parroting one of two Accepted Views (which sucks, but I'm not going to rant about that again
).
