Last week, Joe Biden was elected the forty-sixth President of the United States, defeating Donald Trump by what is likely to be several percentage points nationwide. The margins in crucial swing states, however, were closer than polling averages suggested that they would be. Though Biden won the three Midwest battlegrounds—Michigan, Wisconsin, and Pennsylvania—where Trump eked out victories in 2016, and the former Vice-President is likely to emerge victorious in other previously red states such as Georgia and Arizona, major networks took several days to call the election results, and what looked like a landslide for much of the campaign ended in another scramble for electors.
I recently spoke by phone with Nate Cohn, a domestic correspondent at the New York Times who spearheaded the newspaper’s polling this cycle. (Full disclosure: Cohn and I worked together at The New Republic, and are close friends.) During our conversation, which has been edited for length and clarity, we discussed how the pandemic affected polling, the role of data in election coverage, and the Times’ contentious “election needle.”
What was the biggest surprise to you when the results came in, or as they have come in?
The Hispanic vote. The swing towards Trump in Hispanic areas across the country is extraordinary. It was hinted at in the preëlection polls. The polls always showed the President faring better among nonwhite, and particularly Hispanic, voters than he did four years ago, but the magnitude of the shift was way beyond expectations. We learned that early in the night in Miami-Dade County, where no one had the President doing as well as he did. And it has proven true, as far as I can tell, basically everywhere in the country among Latino voters, to varying degrees. It’s true down-ballot. It’s not like this was just about the President. And I think it’s a huge and important political story.
And the second thing that really surprised me is the white, rural, Midwestern vote. The preëlection polls said that Joe Biden was doing much better than Hillary Clinton was four years ago among white voters without a degree. And those gains simply did not materialize. The results looked quite a bit like 2016 across most of rural America, and there were many areas where Donald Trump did better in white working-class areas than he did in 2016.
Because the definition of “Hispanic” seems to change from poll to poll, and because exit polls are unreliable, I had trouble finding the exact numbers for what percentage of Hispanic voters Trump won in 2016 and what percentage he won this time. What is your sense of the size of that change across the country?
I think it could easily be a double-digit swing in the President’s direction. I have not crunched these numbers conclusively, and it’s still too early to do that. But that would be my initial gut sense, yes.
The most obvious reason for this, I would assume, is the education divide in our politics manifesting itself across racial lines. Do you have a different theory?
I think that’s right. And I think this was not an election on immigration. Immigration was a major theme of the 2016 election, from Donald Trump’s announcement that he was seeking the Presidency all the way to the policy proposal to build a wall. The President’s position on immigration and his attitude toward Mexican-Americans were a central theme of that campaign. And I would say that it has not been a central theme of this Presidency, and it was certainly not a central theme of the 2020 campaign. So it makes sense to me that if we stop talking immigration and Hispanic voters start assessing the President without that in mind, that they might begin to shift in ways that are fairly similar to demographically similar white voters, but four years later. That strikes me as a possibility.
And it would also perhaps suggest that it was more Hispanic men than Hispanic women, correct?
There is a lot of preëlection survey data that suggests that the Hispanic gender gap got very, very large in this election.
It’s interesting you said that it wasn’t a big theme of the campaign or his Presidency. It feels to me like it was a big theme of this Presidency, but not so much the reëlection campaign.
If you were to choose a year that was dominated by immigration during his Presidency, what year would that be?
I think 2018. Family separation and then the caravan.
Yeah. I think that’s right. I think there was a period in late 2018 where immigration was at the forefront of our politics. I would absolutely say that was untrue of 2020, and I think it was also untrue of a lot of the Trump Administration. And I would say that immigration was always near the top in 2016.
In a lot of preëlection polling, Trump was running ahead of Republican Senate candidates. But Republicans seemed to do better than Trump in the generic House vote, and in some of these Senate races. Why do you think that was?
I don’t know why that turned out to be the case. I think one of the most interesting parts of the polling error this year is that it was greater down-ballot than it was as at the top of the ticket, and in 2016 it was the opposite. And so while a lot of our explanation for what went wrong in 2016 was searching for things that were mainly about the President, I’m not sure that the pollsters would be right to suppose that this year’s polling error is unique or specific to the President.
In 2016, the national polls were off a couple points, and then Midwestern polls, especially polls that didn’t weight by education, were off more. This year with your polling for the New York Times, with other polling, there were a lot of misses. And these were the polls that really took weighting by education seriously and took a lot of steps after 2016 to fix errors, and in your case nailed the 2018 midterms. Do you know what happened this time?
I don’t. I can offer you some theories.
Yeah, please.
But, before I say that, I do want to agree that this was a much bigger polling miss, in important ways, than in 2016. It was a bigger polling miss in the national surveys. It was a bigger polling miss for the industry’s most prominent and pricey survey houses. The state polling error will be just as bad, even though, as you mentioned, many state pollsters took steps to increase the number of white voters without a degree in their surveys. And state polls look a lot like they did in 2016.
But, if the state polls are just as bad as they were in 2016, despite steps that we know improved the President’s standing in the surveys, we can say with total confidence—and I know this was true in our data—that the underlying survey data has to be worse than it was in 2016. Or, if you prefer, if all the pollsters were using the 2016 methodology, the polls would have been far worse this year than they were in 2016. And that is really interesting. As I said, I can list a bunch of theories for you.
Yeah. What are your theories?
Well, the key framing is what’s changed since 2016. What would make the polls worse now than they were then? So one possibility is that it’s four more years of Trump, and that as American politics grew more and more defined along the lines of your attitudes about the President, and as old political allegiances sort of fell away, that non-response bias in polling became more and more correlated with Presidential vote choice.
Another possibility is that “the resistance” is what broke the polls. Think of all of the political engagement on the left, the millions of dollars that were spent to help Jon Ossoff in 2017 or to help Jaime Harrison in 2020. This portrays a tremendous increase in the level of political participation on the part of progressives. We know that politically engaged voters are more likely to respond to surveys. And so it may be that as the Trump Presidency has totally energized the Democratic base, it has also led those same kinds of voters to increase their propensity to respond to political surveys.
Another possibility is the high turnout this year. We in the polling world have tended to assume that higher turnout makes polling easier, because we think of turnout as something that’s an additional variable that the polls have to get right beyond just taking a nice sample of the population. This year, though, we have this huge increase in turnout, and most people have supposed that it was good for Joe Biden. Maybe it was good for Joe Biden. But I think we also have to be open to the idea that it was not good for Joe Biden. In Florida, where we were collecting turnout data live on Election Day, I can tell you with certainty that the electorate was more Republican than it was in 2016, more than our polls projected, no matter your likely-voter methodology. That may be true elsewhere in the country. I don’t know. It may not be. We just don’t have that data yet.
But I would note that many of the late polls stopped showing a gap between the preferences of registered and likely voters, and in some cases it went into reverse, where the Democrats were faring better among likely voters and registered voters. A few late examples—the CNN poll showed Biden up ten in Pennsylvania, but only up five among registered voters. I believe the final ABC News/Washington Post poll in Pennsylvania showed Joe Biden doing better among likely voters than registered voters.
And then a final thing I would raise is that maybe it was the coronavirus. You may recall that, one year ago, at the time we published a series of polls that showed Biden narrowly ahead of Trump and Elizabeth Warren trailing Trump, and those polls were a lot more accurate than the polls that we did since then.
They were more accurate, you mean, in that they were closer to the final Biden-versus-Trump results.
Yeah, they were a lot closer to the final results in 2020 than our final 2020 polls were. And that’s interesting to me because our 2018 polls were also quite accurate. And one thing that’s happened since then is the coronavirus. And, when I started looking at the election results on Tuesday night, one of the first things that stood out to me was that Biden didn’t do well in the coronavirus hot spots. Remember how Biden was supposed to do great in Wisconsin, in part because of coronavirus? That didn’t happen.
Wisconsin is probably the swing state that had the biggest polling error, showing Biden with huge leads on the eve of the election.
Yeah. But part of the rationale for why Biden might have had that growing lead in Wisconsin was that he was doing better because coronavirus was hitting there hard, right? You might recall that over the summer there were these studies that said, “When coronavirus hits, Biden does better.” What if when coronavirus hits Biden does better in the polls but not in reality, because a bunch of Democrats take it seriously and stay home while Republicans do not? I’m not saying that’s true, but we know that the survey response rates went up after coronavirus hit. There were plenty of articles about that. And if we stipulate that it was the Democrats who, on balance, took the coronavirus more seriously, maybe that helps explain it. And it would also do a better job of explaining the chronological order of changes in polling error. That said, I don’t think that theory would explain the extent to which the geographic distribution of the polling error is reminiscent of 2016.
So Florida and the Midwest being seriously wrong and other places not as terrible?
Right. But maybe it does help explain the national polls, on average, being much more biased, even though some of these underlying biases still exist at the state level.
You said that your Biden-versus-Trump polls in 2019 ended up close to the actual results. But isn’t another possibility that those polls were wrong, too, and that in 2019 Trump was actually leading Biden, and coronavirus made it possible for Biden to come back and win?
Yeah, that is absolutely possible. And I think that if you take the final results and the final polls seriously, you have to sort of rethink a lot about the course of the Trump Presidency. One of the things about the polls being wrong, Isaac, is that then you’re never sure when they ever were right or wrong. Now, to be clear, that uncertainty always exists. When people get the polls right in an election, they assume that they’re right all the time, until they’re next wrong. But you never know that’s true. And I would say the same thing this year. It’s possible that the polls reflected attitudes about the President at many points over the last four years. And it’s very hard to tell when they were right or when they were wrong.
To give an example, everyone knows that the final Des Moines Register/Selzer poll showing Trump ahead in Iowa over the final weekend was one of the very few polls that were accurate in this election. But, the prior two Selzer polls showed an extremely close race in Iowa, with [the Democratic Senate candidate] Theresa Greenfield ahead. What are we supposed to make of that? Are we supposed to believe that the race shifted hugely toward the President at the end and it was just largely missed? Are we supposed to believe that Selzer was just lucky, and that if they had done five surveys at the end, three of them would have shown Greenfield ahead just like the prior two, and she just got lucky on the last? I have no idea. It’s very hard to try and make sense of all the information you’ve gotten once you know that they were wrong at the end.
What is it that you wanted to convey to readers during an election about the race, and about politics, through polling? And have you had to rethink that, given that we’ve had so much polling error for the last few elections?
That’s a great question, and I think everyone has different purposes for political surveys. There are some people who use surveys primarily to predict election results, and there are some people who use them primarily to understand the attitudes of the country. I think that this is a very challenging time to use polling to predict the outcome of elections, because we live in a very polarized era. The range of Presidential election results that I have seen in my lifetime ranges from Bush winning by two in 2004 to Clinton winning by eight in 1996. And when you have such a narrow range of possible outcomes or realistic outcomes and the polls are never going to be exactly accurate, then there’s basically no election where the polls will offer a definitive and conclusive answer to which party is going to win the Presidency.
The FiveThirtyEight model, for instance, is mainly just an exercise in quantifying how wrong the polls might be. And there will never be a Presidential election in this era where, on Election Day, the FiveThirtyEight model tells you there is no chance for the other side—at least not a competitive national election. That’s been true for a while. What hasn’t been true for a while is that the polls also got the underlying story of this election wrong. Most of the articles that I write are about demographic trends, for instance. And I think those demographic trends are really, really important, because they have a huge effect on the way our democracy works. Political campaigns, activists, politicians—they all try to decide on what policies they want to support and what strategies they should adopt based on what they perceive as the electoral advantages or disadvantages of those choices.
I’ll just take my favorite example. I think that Barack Obama and establishment Republicans like Marco Rubio and Jeb Bush and so on supported immigration reform after the 2012 election because they thought Hispanic voters decided it in Barack Obama’s favor based on exit polls. And so hitting those demographic trends right and telling the stories of these elections accurately has a huge effect on the course of politics in our country. And the polls in this election did not tell that story accurately. They said that Joe Biden was doing way better among white voters than Hillary Clinton. They were wrong about that. He did somewhat better, but the final polls from some of our most prominent survey houses had an all-but-tied race among white voters nationwide. It didn’t happen.
And if you can’t tell the story of an election at the end of it, then the democratic process has some serious problems. Because, in a democracy, politicians need to reflect the will of the electorate, and if you cannot do a good job of interpreting the will of the electorate at any given time, our politicians won’t either. And you end up in a position where the public may not be happy with what politicians try and do on their behalf. And so I think it’s a serious problem that the polls were wrong to the extent that they were this year.
On the other hand, the alternatives to polling are not particularly good. And so that puts us in a tough position where you have something that you think is deeply flawed, and perhaps even so flawed that it could mislead people into making decisions that they shouldn’t make. But, on the other hand, the alternatives don’t really exist.
One counter to that is that we are a republic, and politicians do plenty to try and reflect what their voters think—and they could stand to do a little less.
I think there’s a fine argument that if we could live in a world where the will of the electorate was ignored by elected officials as long as they were in office, and we had a true republican form of government (where the representatives did what they thought was right regardless of the electoral consequences), I think it’s quite possible that would be a better country. I just don’t know.
But the second point I would make is that the foundation of our government is based on democratic legitimacy, and it’s based on the idea that people believe that their views are being represented in Washington and state capitols across the country. And part of the reason why Donald Trump was elected President was because a huge number of people didn’t feel that way.
My own concern about getting rid of polls in an election campaign is that the coverage would not be less horse-race-y, but would instead just switch to even less scientific metrics. If we had no polls and we were going by yard signs and social-media engagement and crowd size, they would feel more misled, because Joe Biden’s going to win by about five percentage points, rather than lose horribly. And you’d get from all these other metrics that Trump was going to win in a landslide.
I don’t think there is a good alternative to public polling. I don’t think we have other ways to measure the attitudes of a really diverse country. I think that without polling we would mainly consider the views of ourselves and our neighbors and our like-minded friends. And so we need tools to reach out to people who are very different from us in order to understand our country well. I don’t think that on-the-ground reporting cuts it. Face-to-face polling exists, and it doesn’t work all that much better anyway.
But I think the bigger question maybe is about horse-race coverage. I used to work in the world of public policy. I worked at a think tank called the Stimpson Center. And I used to hold the sort of naïve belief that data journalism would reduce horse-race coverage, because reporters or whoever might come to understand that so much of the results of our elections are based on the underlying fundamentals, like economic growth or something. And, if that were true, then we didn’t need to focus on every gaffe of the day because they don’t matter that much. Because, as far as I saw it, the only justification for caring about the gaffe of the day in the way that we do is if there is an electoral consequence to it.
And, if we came to that conclusion, then we would talk about the stuff that we thought really did matter and was in the interest of the reader, like talking about policies. I don’t think that’s how it has worked out. I think that it has been an appendage to traditional orchestrated journalism, which, whatever its other strengths and weaknesses, I don’t think was exceptional or any better than polling at anticipating the results of elections. And a lot of that journalism was based on reporting what internal polls said anyway, which were just as bad in this election as the public polls, as far as I can tell.
So should we have polls, but fewer of them?
I have long been of the view that media organizations should coöperate on polls rather than everyone having their own polls. We do it for the exit polls. And we don’t need to have six different national surveys a month coming out from each of the major news organizations saying more or less the same thing, with most of the deviation from the average being attributable to noise, not meaningful methodological choices. And I also feel like that would facilitate better analysis, and get larger sub samples, and so on. I don’t think we need to have a nationwide poll every day to understand public opinion. I do think that the demand to know how things have changed in our political moment is so great that I think it’s hard to believe we would go to a world where you only got the polls once a month or something.
The New York Times “needle,” which this year tracked the odds of each candidate in Florida, North Carolina, and Georgia, elicited, as it always seems to, much criticism and angst and anger. How do you think it performed?
I think the Needle performed spectacularly. I thought it performed spectacularly in 2016. If you just think about it strictly from a journalistic standpoint, in 2016, it scooped the world on Trump being favored to win this election. I think in 2020 it more or less scooped the world on Trump being the conclusive winner of Florida, the favorite in North Carolina even as he trailed by nearly double digits in the tabulated result, and Joe Biden being the favorite in Georgia even though it came later in the night.
People’s anger over things like the needle, or polls being wrong, seems to me to be greater than it is over other kinds of journalistic failures, in many cases. And so I’m wondering if we should have second thoughts about it because, even if serving readers is not the only value of journalism, it’s one value of journalism.
Yeah. I am totally open to that idea, if for no other reason than that I think the needle is the single best live forecast that has ever been published, and is responsible for breaking some of the biggest stories—electoral stories, at least—of the last few cycles. And so, if that’s true, maybe we’re doing something wrong. I don’t know if it’s the probability. I don’t know if it’s P.T.S.D. from 2016 on behalf of some Clinton supporters. I don’t know if it’s about the visual display, not about the numbers. But I do think there is something about the way people perceive the needle that isn’t the way I would have it.
I mean, look, we put a ton of work into it, and I think we’re extraordinarily proud of it. It is difficult to explain how proud we are of how it has performed. And, when other people don’t recognize that, that’s really frustrating, and I would love to change that. And that may involve changing its representation, whether it’s graphically or in terms of the numbers, I don’t know. But I am certainly open to it.
I think there are two ways to look at Joe Biden’s candidacy. The first is that even if he is going to win by a healthy-ish margin in the popular vote, the margin in key states was narrow, and so maybe another Democrat would have lost this election, despite the pandemic. But the second way of looking at it is that even with the pandemic, Trump ended up getting all of these Republican voters, and the country is, as you say, fairly set in certain ways. What do you think?
From my point of view, the Biden candidacy always rested on the assumption that a majority of the public did not want to reëlect Donald Trump as President, and so, therefore, you should give them the most inoffensive person possible, and you will win the election, and that you can only risk the election by choosing someone more controversial. I think that this election has complicated the first half of that. There was not a overwhelmingly clear majority of the electorate that did not want to reëlect Trump as President.
The second part of that formulation, that the only way to maximize your chance of winning was to choose the least controversial person, has not necessarily been contradicted by this election. We can never run the counterfactual to learn whether Elizabeth Warren or Bernie Sanders would have cost Democrats six-tenths of a point in Wisconsin, in the traditionally Republican suburbs of Milwaukee, which will win this election for Joe Biden. We’ll never know. But I don’t think that the election results do anything to prove or disprove that possibility.
This election was much closer than I thought in terms of the popular vote, and it’s very clear that Donald Trump was close to winning it. That being said, I also think it’s really hard when you’re a journalist and you are supposed to be engaged with this idea of talking about what “the people” want, to forget that Joe Biden winning by a greater percentage than, say, Obama beat Mitt Romney, is an impressive achievement in our partisan environment. How do you balance that?
That’s a great question, and I think that the actor is really important here. If you are a politician who is trying to win your congressional district, then you would be wholly justified in focussing on the views of your congressional district. If you are a President who is seeking reëlection, you focus on the states that decide the Electoral College. If you’re a political movement, like progressives or alt-right conservatives, or whoever, and your goal is to obtain political power in this country, then you have to focus on winning political power with the system that we have. If you’re talking about the will of the public and just in broad strokes what Americans believe, then you’re talking about something very different.
To me, most political journalism is focussed on the pursuit of power when we cover progressives or whatever, and we’re asking whether Democrats should support Medicare for All. The question is not what the American people think. The question is: is this something that will help Democrats seize power in order to enact their preferred agenda? In which case we’re justified in focussing on the electoral outcomes or on the attitudes of voters in the most important districts and states. But I think you’re right that if we’re just trying to characterize the American people more generally, that the national vote tells a different story.
Read More About the 2020 Election
- Joe Biden has won, and American democracy survives Donald Trump, David Remnick writes.
- Trump has survived impeachment, twenty-six sexual-misconduct accusations, and thousands of lawsuits. His luck may well end now that Joe Biden will be the next President.
- What happens if Trump fights the election results?
- Here is what you can do if Trump stages a coup.
- For the G.O.P., the future may look like Florida, which is becoming less white but not much less conservative. Can Republicans become a multiracial working-class party?
- Sign up for our daily newsletter for insight and analysis from our reporters and columnists.
