I see the same issue in both north and central Florida. Plenty of people I know will have no problem whatsoever working with or for individual black people, voting for them, or even having relationships with them but at the same time will flat out refuse to consider going to black parts of town and consider blacks in general as a group to be criminal and uneducated. I'd say this very phenomenon is the predominant kind of racism in the south today; you really don't get very many old school racists anymore.
People in this situation aren't necessarily racist persay; they don't dislike black people in general or think less of them as individual people. However, they still would not like to be living in a black neighborhood, surrounded by only black people. It's kind of intimidating being the only white person in a black neighborhood (and I expect the same is true for the only black person in a white neighborhood).
By definition, if it is motivated mostly by race that's a racist attitude.