2008 Nationwide Precinct Map Project - (Mostly) Complete!

[Oldiesfreak1854]

I think Stanford has a page like this: http://web.stanford.edu/~jrodden/jrhome_files/electiondata.htm

But thanks anyway!  I'm really looking forward to this.

[Adam Griffin]

Are you using the 2000 or 2010 voting district lines for your map?

What I can't figure out is what Dave Gardow uses as a voting district.  If Dave's lines are the same as the Census Bureau's VTDs, coming up with a nationwide map would be as simple as downloading the 2008 or 2010 (depending on which you are using) VTD shapefiles for each state, using MapWindowGIS or a similar mapping program to load all 50 states, and creating a map at whatever zoom level you want.  There might even be a way to do this using Google Maps online so that you could zoom in or out to your heart's content, though I don't have experience in using Google's online mapping app.  

Edited to add: Not all states have VTD shapefiles.  Dave Gardow uses something else in the states that don't have them.

VTDs don't directly match precinct lines, though, so if Dave is using VTDs, I don't know where he got the VTD-level data from.  (On the other hand, it's not necessarily easy to find population data for actual precincts, if he is using precinct maps, though I suppose it can be done through aggregating census blocks).

The Harvard Election Data Archive has aggregated precinct-level results for recent elections and put the spreadsheets online.  The problem, as always, is finding the corresponding precinct maps for each year when trying to map the data.  Harvard's archive has posted some state precinct shapefiles, but not all.

Using the 2010 ones. In either case, I foresee inaccuracies. I'm not even sure how Gardow makes it work considering that he does use VTD that inevitably had to be changed between 2008-2010.

Here are some problems I (thought) I'd run into with the GIS/CSV/shapefile route. The first one is what I already referenced: precinct boundary changes. I know that in my own home state, it's not uncommon for 10 counties to change their boundaries in a given year. Something like 20% of the counties in GA now have different precinct boundaries from 2008. Finding a shapefile for a particular state and at a particular moment in time is challenging; if you're picky, it very well may be impossible (unless they're buried somewhere in Census data). Now, accumulating 50 states' shapefiles down to the precinct-level/VTD for an exact year...unless there is already a national precinct shapefile out there that I'm not aware of, I don't see how it can be done.

As far as piecing them together...presumably that can be done by loading each one up individually and then saving it once you've loaded them all up in ArcGIS or something similar? I've never bothered to try this for the reasons I mentioned above. I figured for better or worse, using Gardow's data in whatever way possible would be the easiest route in terms of overall accuracy.

I do have some basic experience with GIS and I actually use Google Fusion Tables quite regularly to combine shapefiles/CSV data with election results to make maps that way. One issue with this approach (and presumably it's similar no matter whether you use a GIS program or not) is getting a spreadsheet with election data to correspond with precincts in the shapefile. I know that whenever I tried to do this for GA in Fusion Tables, the columns/names of precincts in Harvard's and the Guardian's files were not configured in such a way that they'd correspond with each other. They could be named/numbered the same and correspond or be named/numbered differently but they'd need to be in the same order in order to correspond shapefile precinct data to election results precinct data.

How are you extracting data from DRA?

[Adam Griffin]

I think Stanford has a page like this: http://web.stanford.edu/~jrodden/jrhome_files/electiondata.htm

But thanks anyway!  I'm really looking forward to this.

Do they have an actual map? I saw their map with the dots; that's sort of what I initially referenced as being not to my standards as far as a map project goes.

[Miles]

How are you extracting data from DRA?

If you go under "File" on the top left, you can save the precinct/CD data in CSV format.

[cinyc]

You'll probably get a better map by removing precinct boundaries. At such a small scale, what purpose do they serve?

I would also recommend finding a feature set that includes coastlines so that you can run a clip on the precinct features. This should address the aesthetic concern that homely brought up on the first page.

Of course, to do this right, you'd need either a plotter or (better yet) one of those walls of seamless screens...

I have water features on in California and Utah, the only two states with significant water.  My California water shapefile doesn't include Lake Tahoe.  I must have left it out.  I put the precinct boundaries (but not colors) on top of the water because I think it looks better.  Of course, other people don't agree on that.

Unfortunately, I can't find a way to get rid of the boundaries with the version of MapWindow GIS I am using (an older one because newer versions don't support my color schemes).  Changing width to 0 does nothing.  Changing the lines to dotted is barely noticeable at this scale.  Edited to add: but putting the same shapefile on top of the other one with 100% transparency and line size 0 seems to work for some bizarre reason.  That's counterintuitive.

[cinyc]

Here are some problems I (thought) I'd run into with the GIS/CSV/shapefile route. The first one is what I already referenced: precinct boundary changes. I know that in my own home state, it's not uncommon for 10 counties to change their boundaries in a given year. Something like 20% of the counties in GA now have different precinct boundaries from 2008. Finding a shapefile for a particular state and at a particular moment in time is challenging; if you're picky, it very well may be impossible (unless they're buried somewhere in Census data). Now, accumulating 50 states' shapefiles down to the precinct-level/VTD for an exact year...unless there is already a national precinct shapefile out there that I'm not aware of, I don't see how it can be done.

Using VTDs and block groups instead of the precinct shapefiles has to be inexact.  I'm not sure how Gardow did it, unless there was some requirement by Census that result data be sent to it when creating VTDs.  I doubt it.

You must be logged in to read this quote.

The =sumif (or sumifs) function is your friend here.  Make a second sheet with the most descriptive column copied from the shapefile spreadsheets as the first column.  Then write =sumif (Sheet1!$A$2:$A$1000,$A2,Sheet1!C$2:C$1000) in cell B2, where Sheet1! $A$2 etc is the corresponding column for the descriptive data on the results page and $A$1000 is the assumed end of the data.  It's easier to create the function by mousing over the data you want than typing the numbers out, then putting the dollar signs in. C$2 etc is the column with the results data you want.  You can copy and paste this function in cells below and to the right of the cell due to the dollar signs.  It keeps the search area constant, changing only the results column when you paste in cell C2 and the lookup cell when you paste into B3, etc.  The =sumif function only works with results data that are numbers, not text.

[Adam Griffin]

Here are some problems I (thought) I'd run into with the GIS/CSV/shapefile route. The first one is what I already referenced: precinct boundary changes. I know that in my own home state, it's not uncommon for 10 counties to change their boundaries in a given year. Something like 20% of the counties in GA now have different precinct boundaries from 2008. Finding a shapefile for a particular state and at a particular moment in time is challenging; if you're picky, it very well may be impossible (unless they're buried somewhere in Census data). Now, accumulating 50 states' shapefiles down to the precinct-level/VTD for an exact year...unless there is already a national precinct shapefile out there that I'm not aware of, I don't see how it can be done.

Using VTDs and block groups instead of the precinct shapefiles has to be inexact.  I'm not sure how Gardow did it, unless there was some requirement by Census that result data be sent to it when creating VTDs.  I doubt it.

You must be logged in to read this quote.

The =sumif (or sumifs) function is your friend here.  Make a second sheet with the most descriptive column copied from the shapefile spreadsheets as the first column.  Then write =sumif (Sheet1!$A$2:$A$1000,$A2,Sheet1!C$2:C$1000) in cell B2, where Sheet1! $A$2 etc is the corresponding column for the descriptive data on the results page and $A$1000 is the assumed end of the data.  It's easier to create the function by mousing over the data you want than typing the numbers out, then putting the dollar signs in. C$2 etc is the column with the results data you want.  You can copy and paste this function in cells below and to the right of the cell due to the dollar signs.  It keeps the search area constant, changing only the results column when you paste in cell C2 and the lookup cell when you paste into B3, etc.  The =sumif function only works with results data that are numbers, not text.

Well this is just making me feel dumb, because I can't figure out neither the correct way to use the formula nor how it helps sort it all out. Comparing my 2012 shapefile data of Georgia alongside the exported data from DRA, however, already shows me that there's going to be problems: there are 105 fewer precincts as of 2012 when compared to DRA's data.

[cinyc]

Well this is just making me feel dumb, because I can't figure out neither the correct way to use the formula nor how it helps sort it all out. Comparing my 2012 shapefile data of Georgia alongside the exported data from DRA, however, already shows me that there's going to be problems: there are 105 fewer precincts as of 2012 when compared to DRA's data.

It works by only adding amounts from column C on the data spreadsheet if the GEOID on the shapefile spreadsheet equals the GEOID on the data spreasheet.  Here's an example from my AZ spreadsheet.  First, the data page from Dave Gardow's .csv file:



Next, worksheet from my shapefile spreadsheet with the =sumif function to sort.  I copied Column A directly from the AZ VTD shapefile spreadsheet.  Note the =sumif function in the box:



The data is properly sorted.  You can "prove" this by eyeballing the ultimate results on the map and making sure the precincts look like they are colored right.  For example, blue precincts in South Phoenix would tell you something's wrong.

[cinyc]

I replaced the 2008 Southwestern US "precinct" map in the gallery with one that shows county lines instead of "precinct" boundaries and a correct key.

[Adam Griffin]

Thanks for the explanation - I'll go through it later and see if I can get it this time.

Earlier, I decided to try and do 2012 for Georgia (seeing as how the 2012 Harvard data-set was much more organized and I had a 2012 shapefile for the state). Very few alterations overall had to be made to the spreadsheet, but I've run into a bit of a problem with Fusion Tables:

First attempt at Georgia 2012 presidential by precinct



Maybe there is too much data in the spreadsheet for FT to handle overall. After about 100 entries have copied, it stalls out - a rare problem that has to be corrected manually by Google, apparently. I'm going to try to remove a few columns when I get back home and/or attempt the upload again, and if that doesn't work I'll play around in ArcGIS. The precincts that did manage to get copied are functioning properly, though.

[cinyc]

I've added the South Central States, plus Kansas and Missouri.  Here is the updated map (click for a larger view):



I changed the projection to Michigan since the last iteration.

Here is a somewhat closer-up map showing just the South Central states (again, click for a larger view):

[Adam Griffin]

Very nice! I'm going to get the hang of this sooner or later, but you very well may beat all of us to it. If possible, though, I'd like to get something functioning in an online, zoomable format.

[Bandit3 the Worker]

Obama didn't win ANY precincts in Jefferson City MO? For a central city - especially a state capital - that's humiliating.

[cinyc]

Very nice! I'm going to get the hang of this sooner or later, but you very well may beat all of us to it. If possible, though, I'd like to get something functioning in an online, zoomable format.

If MapWindowGIS doesn't crash from having about 125 shapefiles open at once, when all states are added.  Otherwise, I'll have to piece together an image by regions.

I have no experience with Google Fusion tables.  So that's all on you.  I'll be able to send you the necessary shapefiles and spreadsheets when I'm done, though.

Obama didn't win ANY precincts in Jefferson City MO? For a central city - especially a state capital - that's humiliating.

That's not true.  These maps tend to overemphasize rural areas at the expense of urban areas.  You probably just can't see the VTDs Obama won in Jefferson City because they're too small.  See the blown-up map below.  Jefferson City is contained within the centered yellow shape, which straddles two counties (black lines):

[SNJ1985]

I've added the South Central States, plus Kansas and Missouri.  Here is the updated map (click for a larger view):

Wow. Nice work!

[cinyc]

Nationwide map (Click on image for larger view):



Three states are missing.  Dave's Redistricting App doesn't have data for Oregon or Montana.  I can't find the VTD shapefile for Kentucky.  Should I fill in those states with county data instead of leaving them blank?

I change the projection to Google Mercator, which doesn't bend as much.

Regional close-ups plus Alaska and Hawaii to come.

[Adam Griffin]

!!!

[○∙◄☻¥tπ[╪AV┼cVê└]

I like how that Democratic area around the 4 corners doesn't really follow state boundaries.

[World politics is up Schmitt creek]

I like how that Democratic area around the 4 corners doesn't really follow state boundaries.

That's the Navajo Nation.

[Flake]

!!!

[Linus Van Pelt]

Nice work, cinyc.

[cinyc]

Three states are missing.  Dave's Redistricting App doesn't have data for Oregon or Montana.  I can't find the VTD shapefile for Kentucky.  Should I fill in those states with county data instead of leaving them blank?

Using a more neutral color (gray?) that doesn't attract the eye so much as white does might be best.

I'm not sure if MapWindowGIS is compatible with layer files (.lyr) or external map servers but ESRI has a few options of varying quality that might serve as a base map. If you were using Desktop 10 I'd recommend light gray canvas.

I don't think MapWindowGIS is compatible with .lyr files.  But I could always add a grey county map layer underneath the other shapefiles so that it shows up only where there is no data. 

[cinyc]

Regional closeups are below.  Since I started with California, I generally split the states along the border of Northern California stretched eastward.  I had to split the Southeast into Mid- and Deep South due to shapefile size.  As usual, click on the image for a larger view. Clockwise from the Northeast:

Northeast:


Mid-South:


Deep South:


South Central:


Southwest:


Northwest:


Upper Midwest:


Great Lakes:


I can zoom into any map level, so I can make maps of cities or metro areas by request.

[Flake]

Do you have a full size image of the map available for viewing/download? Like, zooming into metro areas within one image?

[cinyc]

Do you have a full size image of the map available for viewing/download? Like, zooming into metro areas within one image?

Not yet.  It's not even one shapefile for MapWindowGIS purposes.  The files are probably too large.  Adam Griffin and I are working it, though.

We're trying to upload the image to Google Maps, but are running into size limitations.  I was able to put Missouri online as a test here, but when I tried to put the Southwest region on the maps, things crashed.