Despite everything that had been said, the brilliant people who animated Charlie Hebdo knew all well the legitimate limits of freedom of expression. Their drawings were often crude and provocative, but never hateful. Despite their old slogan, they were anything but "dumb and mean". They were animated by values of universal solidarity and peaceful coexistence. Yes, they were militant anticlericals, but frankly I don't think that's the main point. They opposed organized religions because they saw them as forces of hatred and oppression. That's their legacy IMO.
Those limits being?
Hate speech. Charlie Hebdo never engaged in hateful drawings or commentary, and had a policy of attacking ideas and beliefs rather than groups of people.
What do you mean by "hate speech"? I don't the OP or anybody else would suggest that their legacy was one of racism or hatred (well, some people might, but not anyone who supported the cartoonists), though of course people should be free to express those things as well.