Not a big deal. A stupid comment based on the idea that he felt shadowed and that he apologized for. Time to move on. Unless one wants to grasp at straws, of course, because their candidate is losing. And a person can do that, fine, that's politics. But to say something like this makes him "not qualified" is the bitterest partisanship of bitter partisanship. Come on. And if that's the case, okay, whatever you are doing, whoever you are, as soon as an insensitive thought or comment or something flashes through your head, you're not qualified to be doing whatever you're doing because chances are you are interacting with people on a daily basis. Because that's how I read the partisans' reaction to this - it's not the comment itself as much as the principle that allegedly behind it.
It's not that this means he's a terrible Senator. However, this is the kind of behaviour that sends up a red flag for me. Someone in a position of power can't have a disposition like this. What he was doing, plain and simple, was singling out the person in the room who disagreed with the majority and making fun of him like a schoolyard bully. A less than competent one, but still.
He's harmless, but he knows that he had the crowd behind him. It's not a quality I find particularly attractive in a human being.