Why are libertarians associated with the right?

[100% pro-life no matter what]

David Azzerad and other conservative thinkers have made the argument that, while it can be beneficial to team up with them to accomplish economic goals and win elections, their ideology actually has more in common with the left.  Basically, they support the same sort of culture that the left supports, with the only difference being who will pay for it.  For example, neither group would say that transgenderism harms society, but they would only disagree on whether their sex change should be funded.  So, at the core, their values and principles have far more in common with the left.  He could articulate this far better than I, I'm sure, but I consider libertarianism an ideology of the left.

[100% pro-life no matter what]

Why are libertarians associated with the right?

Dude, it's because they are on the right.  

For example, consider this the Political Matrix scores, which can be linked to with the PM button above.  1045 posters have taken the quiz, and here are the average economic scores by party affiliation:

    Democratic        -4.13      
    Republican         +4.39
    Independent        -0.65
    Libertarian         +5.14
    Green                  -5.94
    Constitution        +4.76
    Other                -3.87
    Decline to State   -0.48

Bear in mind that positive scores are further right, and negative scores left.  Sure, it's a small (and self-selected) sample, but it should be obvious that at least among the posters in this forum, and by the standards of the Political Matrix test, Libertarians are the rightmost group.  Even Constitution party people, who are considered right-wing nuts, are not as far right as Libertarians.  This is only one data set, but I encourage to find other data and you'll see a strong correlation of self-identification with the LP with economically conservative benchmarks.

Where do you see the averages, and how different are they for the social side (what I really associate conservatism with)?