Sorry, but we're voting for the president of the united states, not the president of California and New York. We need the electoral college for the other 48 states to have any real voice.
Since when do states like Wyoming and Idaho command actual attention and drive policy from presidential candidates? The electoral college puts a lot of focus on a small set of states, and it's likely a popular vote system would do the same but with different areas and types of areas. If both have shortcomings like this, we should at least go with the one that actually represents what most people want and not let politicians who can't even gather at least plurality support slither into office.
No one's stopping you from moving out of DC, assuming you have the finances to do so.
And nothing would be preventing people from moving out of Iowa, Kentucky, Wyoming (...) if we were to switch to PV.