"If a tree falls in the forest but nobody hears it..."
My favorite version of that phrase is "If a man speaks in the forest, and there is no woman there to hear him, is he still wrong?"
I agree with what MODO said, it would make more sense in my opinion to keep your avatar with whatever party you identify with (or with no party, as is the case here), though I can understand your logic.
Having your political matrix scores right below your avatar would be sufficient capsulation of your political views for someone who wants to know this immediately without reading your posts indepth.