Political preferences in the US are highly correlated with population density, at national, state, and metropolitan-area scales. Using new data from voter registration records, we assess the extent to which this pattern can be explained by geographic mobility. We find that the revealed preferences of voters who move from one residence to another correlate with partisan affiliation, though voters appear to be sorting on non-political neighborhood attributes that covary with partisan preferences rather than explicitly seeking politically congruent neighbors. But, critically, we demonstrate through a simulation study that the estimated partisan bias in moving choices is on the order of five times too small to sustain the current geographic polarization of preferences. We conclude that location must have some influence on political preference, rather than the other way around, and provide evidence in support of this theory.