Even without any genetic reasons, one can make a case for banning cousin marriage as a way of preventing a group of people from becoming too insular and isolated from the rest of society. It's not merely genetics that makes close marriage a bad thing when it is the norm.
I have heard this argument before, it usual come from the anti-Muslim right in Europe. I'm personal not a fan of it, through it make more sense that the genetic one, the latter suffer under the fact, that groups which practice cousin marriage to an limited extent, tend to be more healthy than one which doesn't practice it all. As for the cultural ones, people do have a right to isolate themselves and keep their cultural aspect, as long as they're not a burden on the rest of society it's not something the state should deal with.