I think people are capable of recognizing that a writer can have both good and bad ideas. Charles Murray has made some interesting and original contributions, unlike his perspective on race and IQ which are neither.
While I don't disagree with this, it is also true that it's more than likely that Mr Murray used his credentials and standing to promote b.s racist theories. Given that it seems he did this on the one occasion it does raise serious questions as to whether any of his scholarship is valid.
Are you accusing him of just making things up? The fault is with the interpretation of the evidence. Herrnstein probably deserves more credit for The Bell Curve than his co-author Murray does, and students still study his work in behavioral psychology.
It's obviously far more a question of interpretation than facts, as Murray's facts were quite solid for the most part.
I generally think that discussing race and IQ should be avoided generally, with instead an emphasis on attributes like hard work for the betterment of all individuals.