I would much rather be around automobiles operated by machines than automobiles operated by people. How many people each year do you think are killed by automobiles as a result of human error?
~95% of them. Mechanical failure and extremely bad luck make up the rest. Driverless cars will save many thousands of lives a year just driving drunks home.
I suspect when the majority of cars have some of the better modern safety features on them (like the auto emergency braking, stay in your lane assist, adaptive cruise control, etc), we'll see traffic fatalities drop even faster than they have been (and they've been plummeting for forty years).
Now if we can automatically get people out of the passing lane who aren't passing!