The signal and the noise. Election analyst Nate Silver used this phrase when he released his book The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t after he successfully predicted the winner in all fifty states in the 2012 presidential election. The main lesson from Silver’s analysis of the art and science of statistical predictive modeling is that the signal is the “truth” and the noise is all that which distracts us from the truth. The signal, or the truth if you prefer, regarding the upcoming congressional midterms was cast in stone on November 9th, 2016. That was the day on which the 48.2% of Americans that voted for Hillary Clinton (and the roughly 3% of “progressives” that cast protest ballots) in the 2016 presidential election woke up to face their first day in Donald Trump’s America. Clinton’s loss and Trump’s victory cemented an enthusiasm gap in favor of the Democrats at least until 2020. And with the retirement of sometime-swing vote Justice Kennedy, the Supreme Court is sure to issue judicial decisions that are popular among conservatives, but jarring to liberals for at least the next decade. This enthusiasm gap has already produced a net gain of 43 state legislative and congressional seats for the Democrats in regular state legislative elections in New Jersey and Virginia as well as in a slate of special state and federal elections across the country. And it’s important to point out, Democrats are not just winning the competitive elections. Several of their pickups come from districts in which the Republicans held double digit advantages such as Pennsylvania House District 18 or, more recently, Wisconsin’s 1st state senate district where the Democrat won that open seat despite the fact that Trump carried that district by nearly 18 pts over Clinton. Perhaps even more notable, the 1st district contains a 2016 “pivot” county: a county that voted for Donald Trump after voting for Barack Obama in 2012.
The proliferation of political media outlets such as FiveThirtyEight, The Cook Political Report, RealClear Politics, and Sabato’s Crystal Ball over the past decade, combined with exponential growth in the polling industry, has dramatically increased both the coverage of elections as well as the amount of data available to us to study them. In doing so, analysts have brought once obscure theories of electoral behavior formerly consigned to academic journals to the mainstream. Members of the public who approach me after events or lectures are often well-versed in political phenomena such as the midterm effect and even in more complex concepts such as partisan gerrymandering. Much of the research put out by data journalists resembles, at least methodologically, research published by some of the top political science journals. Major outlets such as The New York Times can afford the staff and infrastructure needed to publish election analysis in real time, and without the time constraints imposed by peer review.
One byproduct of the contemporary environment is that analysis relies heavily on assumptions and theories of political behavior, much of which was produced in the pre-polarized era. Most Americans understand that American politics has become increasingly tribal and that the two political parties have grown increasingly acrimonious over the past two decades via negative partisanship. Indeed, I often find references to the DW-NOMINATE scores (which have evolved into Alpha-NOMINATE scores) that allow for unbiased estimates of ideology for each member of Congress. These can be used to compare individual members with each other or to demonstrate the increase in ideological extremism in the House and Senate over time. Less well understood are the theoretical mechanisms that have led us to this point. Some pundits point to the 1994 Republican Revolution that ushered Newt Gingrich into power as the starting point for the decline of civility in American politics. However, the issues that fuel polarization reach much further back, to the America that was created after the passage of the Civil Rights Act of 1964, the Voting Rights Act of 1965, the women’s liberation movement; which eventually led to the Supreme Court’s decision in Roe v. Wade, the imposed secularization of the country by a series of Court decisions that limited the influence of religion in the public sphere, the movements for civil rights for other minority groups such as gay Americans, and liberalized immigration policy that expanded immigration from non-European countries such Asia, Africa, and South America. These changes, combined with the emergence of the modern media environment and then the internet, have profound impacts on American political culture.
One of the most relevant impacts of these changes to American political behavior is the phenomenon known as “party sorting,” which refers to the ideological sorting of conservatives into the Republican Party and liberals into the Democratic Party. Largely gone are the days of liberal Republicans and despite some hangers-on, largely in the South and Midwest, conservative Democrats. Party sorting left each party ideologically homogeneous and that homogeneity has pushed out the bounds of the ideological spectrum. Today base voters in both parties believe in corrupt party “Establishments” and an all-powerful “deep state.” Donald Trump’s presidency has led to the mainstreaming of conspiracy theorism among Republican rank and file voters.
So why go into all of this background about polarization in a post that purports to offer predictions for the upcoming midterm elections? I do so because a central theoretical tenet serving as the basis for my predictions is that “this ain’t your granddaddy’s electorate” anymore. That is to say, contemporary elections are largely driven by the aforementioned negative partisanship. Although partisanship has always been an important driver of electoral behavior, the influence of partisanship on vote choice is immense in the polarized era. What matters most to the vote decision is party identification. For most voters, in most places, and in most elections, even judicial elections, this consideration overrides all others. Despite the rise of self-identified Independents over the past few decades split-ticket balloting, which refers to the decision a voter makes to vote for a presidential candidate of one party and at least one congressional candidate of another, as well as the number of states where the winning presidential candidate is from the opposite party of the winning Senate candidate, has collapsed since the mid 20th century. In 2016, 34 out of 34 states chose the same party for president and Senate and in 2016 only 35 House districts of the 435 total districts voted for a presidential candidate of one party and a House candidate of the other.
One need look no further than the nomination of Donald Trump by the Republican Party in 2016 to understand how high the stakes of partisanship of the polarized era affect voter behavior. Despite serious considerations of a brokered convention and the history-making Never Trump movement, on Election Day 88% of Republican identifiers cast ballots for Donald Trump despite the fact that even Republican voters felt he lacked the temperament or behavior to serve as President of the United States. This is right in-line with other presidential elections where about 90% of partisans cast ballots for the candidate of their own party. The same power of partisanship was displayed again in the special senate election in Alabama to fill Jeff Sessions’ vacant seat. In that race, Republican primary voters selected for their nominee Roy Moore, who had twice been removed from the Alabama Supreme Court by fellow conservatives for conduct unbecoming a justice and more problematically, for whom credible allegations of sexual abuse of minors emerged. Yet on Election Day, 90% of self-identified Republicans still cast ballots for this flawed nominee because he was the Republican. Republican voters interviewed about their support for Moore either expressed concerns about the allegations but justified their support by citing aspects of negative partisanship such as the high policy stakes of the seat going to a Democrat and general acrimony they held towards Democrats, or more common in the Trump Era, simply refused to believe the allegations. This type of behavior in Republican politics presents a sharp departure from the past, when scandal ended political careers.
The normal laws of gravity still applied to Republican politics as recently as 2012. That cycle, incumbent senator Democrat Claire McCaskill (MO) was one of the most endangered incumbent Democrats. Yet, McCaskill ended up winning reelection handily because Republican voters defected in large numbers from their party’s nominee Todd Akin after he made widely mocked claims about the capabilities of the female reproductive system in response to rape. Akin ended up winning just 79% of Republicans and lost Independents by 12 points, which produced a 16 point trouncing. Yet in the same state this year Missouri’s Republican governor Eric Greitens had to be forced out of office by prosecutors via a plea deal after he refused to resign over sexual abuse allegations. Although many of Greitens’ GOP colleagues in Missouri failed to rally behind him, Republican voters in the state largely stood by their man. At the height of the scandal Greitens still maintained a 63% approval rating among Republicans largely by relying on what has become a go-to tactic for the modern, scandal-ridden politician: cast aspersions on the investigation to get your own party’s voters to question the legitimacy of the claims against you. So a lot has changed in the last 5 years in terms of what the electorate is willing to tolerate and that is being driven in part by increasing hyperpartisanship and polarization in the public, particularly the part of it that votes, which in recent midterm elections constitutes between 35% and 40% of eligible voters and in presidential elections, about 55%.
The way we understand the electorate needs to be reexamined for the polarized era. The traditional view sees the electorate as an ocean that flows from left to right depending on the movement of Independent voters from Republican to Democratic party candidates, which is largely predicated on major factors such as how the economy is performing and whether there are any large, salient issues moving voters toward one party or the other. Take the aforementioned midterm effect, for example. The midterm effect is the longstanding tradition of the president’s party losing seats in the subsequent congressional elections two years later, midway through the president’s term. The midterm effect is really a referendum effect and it supposedly measures the amount of “buyer’s remorse” the electorate, particularly Independents, have after the preceding cycle’s presidential election. This may well have been the case in earlier decades, when partisans were more ideologically heterogeneous, Independents fewer in number, and “Reagan Democrats” still roamed the Earth. But my preliminary analysis of voter files indicates that the modern midterm effect may be misunderstood. The data suggests that the rise and fall of the incumbent party’s fortunes may not be driven by the movement of Independent voters from one party to the other, but instead, by the entrance (and exit) of partisan voters who are activated or deactivated by negative partisanship. Keep in mind, midterm elections are low turnout elections. The 2014 midterms produced the worst turnout rates in the modern era, only 36.4% of eligible voters cast ballots that cycle. And what drives turnout at the margins in off year and midterm elections is negative partisanship fueled by being locked out of power in Washington, particularly the big, white house at 1600 Pennsylvania Avenue.
I’ll come back to this shortly but first I want to explain a very important, but largely ignored, fact about the American electorate. In many elections, even competitive ones, Independents are not always the decisive factor determining who wins and who loses an election. You are likely scoffing at this claim because it contradicts the way we understand elections but consider the evidence. Although Barack Obama won the majority of Independents in his 2008 presidential race (primarily because the economy was quite literally collapsing on Election Day) he did not win the majority of Independents in his 2012 reelection bid. Given the conventional wisdom of elections, such a thing should not be possible. And it’s not just that he failed to carry Independents nationally, he failed to carry Independents in critical swing states such as Ohio that he still won. In fact, Obama lost Independents in that decisive swing state by a staggering 10 points, but he still won the state because the impressive turnout operation established by the Obama campaign managed to produce an electorate that was 38% Democrat. And as I show in my unfortunately titled book The Unprecedented 2016 Presidential Election, Democrats lose Independents quite often, and in elections they win and they lose because they have a population advantage in many places and when their partisans turn out in high numbers, it trumps the combined loss of Republicans and Independents, assuming they don’t lose the latter group by wide margins.
The fortunes of the Republican and Democratic parties seem to rock back and forth every few cycles, creating the appearance of neurotic electorate that can’t quite figure out what it wants. But what we are really seeing, especially in the first midterm under a new president, is backlash from negative partisanship from voters of the party that lost the presidency looking for electoral revenge coupled with complacency from voters of the ruling party. Out of power partisans vote because fear is an excellent motivator. Especially the kind of fear that comes from seeing the opposition party enacting policies you don’t support and stacking the federal courts with judges with the “wrong” ideology.
Think about it. When we look at the impressive gains made by Republicans in the 2010 and 2014 congressional midterms, as well as the 1000+ state legislative seats they gained over the course of the Obama presidency, partisan gerrymandering only accounts for part of their electoral success. And in the case of the 2010 midterms, the current district lines that strongly advantage Republicans in many states are the product of the big gains Republicans made in state and federal elections, not the cause of it. So the electoral success of Republicans is more than a story of partisan gerrymandering, which didn’t take effect until the 2012 election. Instead, much of their electoral prowess over the past 8 years was largely driven by backlash to Obama and Republican strategists’ success at tapping into this “fear factor” by nationalizing elections. For Republicans, elections in the Obama era, both big and small, were framed as a referendum on Barack Obama and Nancy Pelosi. This brilliant messaging, combined with a complacent Democratic electorate, allowed Republicans to over perform their share of the electorate by 5 points in the 2010 midterms and 10 points in 2014 in midterms. It is negative partisanship among opposition party voters that drives the midterm effect, not movement of independent voters back and forth between the parties.
This updated theory of electoral behavior led to my successful prediction of the Blue Wave in the 2017 elections in Virginia (at the 20 minute and 32 minute marks). All told, we ran 5 surveys on the gubernatorial race between Democrat Ralph Northam and Republican Ed Gillespie over the course of the general election and they were remarkably stable, predicting that Northam would win the election handily. This worried my colleague, who had spent the past decade making a close study of the Virginia electorate because the elections in 2013 and 2014 had turned out to be far more competitive than expected. Indeed, this was a reason the national punditry herded around a close and competitive election the final week heading into Election Day. But by applying my theory of negative partisanship’s electoral effects in the polarized era, I suspected that Ralph Northam’s victory was cemented on November 9th, 2016 when Donald Trump won the presidency. Trump’s victory created a different Virginia electorate from the electorates of 2010, 2013, and 2014. Because Democrats lost the 2016 presidential election, especially considering the way they lost it and to whom, I expected a turnout surge among the Democratic portion of the electorate and this is exactly what happened. Despite predictions of a close race by other pundits, Northam ended up winning by 9%. And he did it by a surge in Democratic Party participation, not by winning over Virginia’s right-leaning Independents. In 2013, 37% of the electorate were Democrats and in 2017 that percent increased to 41%, which is enough to turn a average 2-3 point advantage for statewide Democrats into a 9 point route that also allowed Democrats to flip 15 House of Delegate seats when even the most ambitious predictions, including my own, predicted a gain of just 7 or 8 seats due to gerrymandering. The point I want to hammer home is that the determinate factor driving voter behavior in this election was negative partisanship because had Hillary Clinton won in 2016, Virginia may well be currently governed by the Gillespie Administration despite the growing demographic advantage Democrats hold among the overall population of the state and the increasing influence of Northern Virginia on statewide election outcomes.
So let me come back to Silver’s concept of the signal and the noise. Because of negative partisanship Democrats will have a significant enthusiasm advantage in turnout in elections so long as Donald Trump sits in the White House. In places where there are large pools of untapped Democratic voters, the party is going to win marginal seats as well as some seats that have not been competitive since at least 2006. Case in point, the special election in Pennsylvania CD 18. Although the narrative of Connor Lamb’s unexpected victory points to a well-run, highly funded campaign (it was) and Lamb’s centrist platform attracting Independents (it did) Lamb’s narrow 1 point victory would not have possible without massive Democratic turnout. In a largely rural district with an 11 point Republican Party advantage (PVI) that Trump carried by 19 points Democrats managed to make up a plurality of the the electorate, 46% compared to just 41% for Republicans. And that was driven by large turnout among Democratic voters in the Pittsburgh suburbs motivated to the polls by negative partisanship and backlash to Trump.
My analysis of special elections since Trump was elected reveals that Democratic Party candidates are over-performing Hillary Clinton’s share of the two-party vote by an average of 7.36 points while Republican Party candidates have under performed Trump’s vote share by an average of -3.47 for a net improvement advantage for Democrats of 10.83 points. This advantage is especially pronounced in two regions of the country: the Midwest (D+ 19.27) and the South (D+13). And while it is true that low turnout elections such as special elections benefited Republicans over the Obama years, since the election of Trump, Democrats have flipped 27 seats in special elections and in these elections the Democrats improved their share of the two-party vote over Clinton’s share by an average of 13 points, including 45 points in a special election in Kentucky.
Although statistical analysis informs the predictions I offer here, it is important to draw a distinction between what I am doing, and the predictions that are derived purely from forecasting models. Contemporary midterm forecasting models do what they do quite well: estimate a range of potential seat gains/losses for each party by utilizing the highly predictive indicators of generic ballot advantage and presidential approval to run thousands of election simulations which are updated constantly with new data to produce a real-time forecast for a range of outcomes for control of the chamber and overall seat share. Forecasters have built on earlier predictive models by refining their polling aggregators and introducing some district-specific factors. The predictions offered here do not endeavor to “reinvent the cart” or even to replicate it. These parsimonious models have established a long track record of efficiency and accuracy and I expect that the actual performance of the Democrats on Election Day will closely mirror their predictions, especially because the models are updated continuously. The future becomes clearer when it is only a week or a day away. Right now, the advantage for Democrats on the generic ballot hovers on average around 7 points and produces a seat gain for Democrats anticipated to be somewhere between 12 and 33 seats, depending on the forecaster. As Election Day approaches that “cone of uncertainty” will narrow and I expect the aggregate models will be producing seat gain forecasts closer to the predictions I offer here. Indeed, this is exactly what has happened since I initially revealed my predictions on July 1st. In the run up to that day, the generic ballot was in the midst of a two week narrowing leaving many analysts to become less bullish on Democrats’ probability of winning enough seats to gain control of the House. Aggregate forecasting models at the time put Democrats winning the House at less than a .5. Fast forward two months and the aggregate models are bullish again, putting the probability of control switch at nearly .8. So what changed? The forecasters and handicappers will tell you the generic ballot changed. It widened again and as a result the models are predicting a stronger showing for Democrats. But really nothing has changed. The factors that will largely determine House election outcomes are exactly the same now as they were in June. Donald Trump is still the president and Democrats are still riled up compared to their Republican counterparts via negative partisanship.
In order to understand the factors that produced the Blue Wave in Virginia, I took a deep dive into the demographic composition of the districts that Democrats flipped. It should be noted that the most important predictor that a district flipped was whether it was a so-called Clinton district. A Clinton district is a district with a Republican representative that broke in favor of Hillary Clinton in the 2016 presidential election. In Virginia, there were 17 such districts and 14 of them flipped on Election Day. In addition, one non-Clinton district also flipped. It should probably be noted that an additional Clinton district that did not ultimately flip was exceedingly close to flipping. Indeed, after a recount in the 94th district, the Democratic challenger and the Republican incumbent were tied, with 11,608 votes each. The election was decided by “drawing for lots” with the Republican winning the draw. And in another district, not carried by Clinton, 147 were given ballots for the wrong race in a contest in which the Republican incumbent won by just 73 votes. All in all, the probability of a Virginia Clinton district flipping was .67, a data point I will return to when I analyze the upcoming congressional midterms.
To better understand the impact of district-level characteristics on the two-party vote share for Democrats I specify a linear regression model to predict the two-party vote share of Democratic candidates competing in one of Virginia’s contested House of Delegate districts. Of Virginia’s 100 districts, 61 produced a nominee for both parties where the losing candidate achieved at least 30% of the vote. Based on observations from my polling data and on my theory of the polarized electorate I expect that certain district specific demographic features made districts more likely to produce the type of electorate Democrats needed to topple Republican incumbents. I expect that the percent of the population with a college education, the level of diversity within the district, and the level of urbanization of the district will have a significant effect on Democratic candidate vote share. I also hypothesize that vote share will be affected based on whether the seat was open or being defended by a Republican incumbent. Finally, I expect that the level of partisan competition will significantly affect Democratic candidate vote share. To test for this, I construct Cook PVI (partisan voter index) scores following the methodology they use for federal congressional districts for the 61 Virginia districts included in my analysis. As expected, the level of college education, racial diversity, and urbanization in each district, as well as the district’s PVI are statistically significant predictors of Democrats’ share of the two-party vote. Interestingly, the presence of a Republican incumbent has no influence on Democrats’ share of the two-party vote. However, post-regression diagnostics reveal an issue with multicollinearity largely driven by the strong correlation between racial diversity and urbanization and the PVI scores (VIF=4.37). The correlation between the constructed PVI scores and racial diversity was .86 and between PVI and urbanicity .71. Because PVI scores appear to capture much of a district’s diversity and urbanization, these two factors are dropped from the final analysis. The results of the model predicting two-party vote share of Democratic candidates in Virginia’s 2017 elections are displayed in Table 1 and Figure 1. The level of college education, the district’s PVI score, and whether or not the Democrat matched or outspent their competitor are each statistically significant predictors of the two-party vote share for Democratic candidates. Interestingly, incumbency did not help Republicans in the Virginia elections.
Table 1: Predictors of Democratic Party Vote Share, VA Predictors
The partial effect of the percent of residents in the district with a college degree while controlling for the other factors affecting the two-party vote is positive and .10. For each 1% increase in the percent of residents with a college degree, the two-party vote share for Democratic candidates increases by a tenth of a percent. While significant, it should be noted a 10% increase in college education is required to increase Democratic vote share by 1%. However, in close elections, a full percent or even less can be decisive like we see in the Pennsylvania 18 and Ohio 12 special elections. As expected, the largest partial effect comes from a district’s partisan competitiveness. When controlling for the other factors each 1 point increase in the district’s PVI score, decreases Democratic candidate vote share by a full percent. Given their statistical power in this model it is no surprise that handicappers rely on them extensively with great success. The model explains nearly 96% of the variance in Democratic candidate vote share in the 2017 Virginia elections.
Having isolated the factors impacting the two-party vote share of Democratic candidates, as well as the predictive power of Clinton district designation for a district to flip from Republican to Democrat, I apply these factors to create estimates of the two-party vote share for Democrats running to unseat Republican incumbents or running in an open seat previously held by Republicans in the upcoming elections for the House of Representatives. My goal is to try to identify the districts that are most likely to flip from Red to Blue well in advance of the start of the general election season. Once I have baseline estimates of the predicted two-party vote share for each analyzed district based on the linear regression model, I then consider other factors expected to influence the election outcome such as open seat status, Trump’s performance in the district relative to Mitt Romney’s performance in 2012, Trump’s approval in the state and region the district resides in, and primary turnout data. Below I show three categories of races based on the predicted two party. The first list shows the 30 most competitive races, the bulk of which have two-party vote shares for the Democrats at or above 50%.
This second group of races are races for which the predicted vote share for Democrats falls short of 50% but are open races where incumbency will not matter. Many of these offer competitive predicted vote share, but not all. However, in the 2010 cycle, Republicans nearly swept open races. As such, I consider districts with a predicted vote shares above 45% as at least toss ups.
This final set of districts are districts that are largely off of the radar in other forecasts due mainly to the fact that many have what appears to be insurmountable PVI advantages for the Republican incumbent. Still, due to factors like a large college education percent, a high level of diversity in the district (or a combination of those factors) many of these races produce predicted two party vote shares that indicate they may be competitive.
The race ratings below are based on a combination of the Education/PVI model and on fundraising data from the primaries. This list is the same list I posted on July 1. Because my analysis was originally posted on July 1, only 1st quarter fundraising data was available. I will be updating the 2nd quarter in the coming week or so. Now that the general election season has officially begun, I will be adding data points to the model as they become available. This data may move races up or down in the ranking. I will also be releasing my long-awaited Senate and gubernatorial race ratings so stay tuned for that. However, on my initial launch in July I identified that Texas would be competitive and since then, the state has indeed become competitive by some raters. Just remember folks, you heard it here first!
But here’s where it gets really interesting. Although the analysis of the 2017 election in Virginia provides robust evidence that the two party vote for Democrats can be estimated via the percent of college educated residents in the district, the district’s PVI, and the Democrats’ ability to compete financially analysis of the 59 special elections since Trump finds less clarity of the factors driving Democratic performance. A lot less. Table 2 shows that little predicts Democratic vote share in the specials. The only significant variable is college education and that is only significant when it is “dummied” out to create a two category variable that separates districts into those with higher than 40% college education rates and lower than 40% college education rates. Any statistician reading this will recognize this immediately as “p-hacking,” which refers to the manipulation of data to produce significant results.
Table 2: Predictors of Democratic Party Vote Share in Special Elections
Of the 59 special elections in my data 18 of them flipped to the Democrats. Of those 18, the smallest PVI advantage for Republicans was 6.59 and the largest was 14.2. In half of the races the Democrat was able to meet or exceed the spending of the Republican but the other half did not. The average college education rates of the 18 flippers was 40% but the lowest was 30% and the highest was 66%. On average the districts had a white population of 78% but several had nearly 100% white populations. 9 of the 18 were urbanized but 2 were quite rural. What this suggests is that predictions based purely on college education rates, PVI competitiveness, and financial competitiveness are likely to underestimate the performance of Democrats in the 2018 midterms. As such, I utilize a mixed methods forecast, which combines statistical data with qualitative assessments and additional data points that would overwhelm a predictive model. The goal is to accurately predict the seats that will flip with as much lead time as possible.
The original race analysis appears below.
My analysis currently predicts Democrats will pick up 42 House seats as well as holding onto Senate seats in Florida, West Virginia, Montana, Indiana, and Missouri. My analysis also predicts that the Democrats will likely pick up the Nevada senate seat, while Arizona and Texas, (yes Texas) are currently toss ups. The senate seat I predict is most vulnerable for the Democrats is Heidi Heitkamp’s seat in North Dakota, and even with the support she is receiving from the Koch Brothers, this race is currently coded as Lean Republican. Although many pundits code Tennessee as a tossup, given the strength of the Republican nominee, the lack of the factors most likely to produce a surge of Democratic voters, and the lack of any competitive House races, Tennessee is currently Lean Republican.
I can use the Senate map to further illustrate my point about the polarized electorate because if Hillary Clinton was currently president the Democrats would likely lose every one of these races with the exceptions of West Virginia, where split ticket balloting is still common due to the issue of coal, and Florida, which would have been extremely tight with two well-known, well-financed statewide incumbents facing off. With a Clinton Administration the edge in that race would go to Rick Scott, because Democratic turnout would be lackluster while Republicans would be galvanized after Trump’s loss and hatred of Clinton. Despite Scott’s considerable assets, flipping Nelson’s seat this cycle will be an uphill battle because the electorate that made him a two term governor has been replaced by an electorate that will favor Democrats while Trump is in office. Unfortunately for the Democrats the timing of this wave falls on the Class 1 map, which structurally restrains Democrats significantly, even under scenarios where they win the national popular vote by a wide margin.
In terms of the House I am able to identify 12 specific seats that will flipto the Democrats, most of which are coded as toss up races by other forecasters. My analysis also produces an additional 12 seats that are highly likely to flip. Although many of these districts are Clinton districts, not all are. I include in the list of likely pickups districts like Virginia’s 7th district and California’s 21st, which aren’t even considered toss ups by other outlets. At least not yet.
Like with Virginia in 2017, I expect these predictions to be met with skepticism at this point in the cycle. But when you look at the data, Democratic Party vote share collapsed between 2006 and 2010 and 2014. Where did all of those Democratic voters go? Complacency depressed their turnout while Republicans were highly motivated due to negative partisanship. I predict the 2018 electorate will look more like the 2006 electorate than the 2010 and 2014 electorates in terms of its partisan composition. A few percentage point increase in Democratic turnout has a large effect on Democratic candidates’ two party vote share. In Virginia Democrats improved their share of the electorate by 3 points and in doing so transformed a modest demographic advantage into a wave.
We can learn a lot regarding the potential gains for Democrats in 2018 by looking back at how Republicans performed in the 2010 midterm. In that cycle, Republicans not only won 29 of the 40 toss up races, they also won 5 of the 25 districts classified as “Lean Democrat” and even won 2 of the 13 races classified as “Likely Democrat.” All told the Republicans netted 63 House seats, the difference coming from a host of districts whose PVIs advantaged Democrats enough that they were largely left out of consideration as competitive races by forecasters until the final weeks of the cycle (the strong performance of my friends over at Crystal Ball in their final forecast issued on election eve should be noted). Like in 2018 the beginning of the 2010 cycle was marked by a high number of retirements, 37 in the House. But unlike 2018, retirements in 2010 were fairly evenly split between the parties: 17 Democrats to 20 Republicans. Members of Congress are good at anticipating tough electoral environments and Republicans clearly recognized that 2018 would favor Democrats because of the 65 retirements this cycle, 46 of them are Republicans. A handful of these retirements are due to appointments in the Trump administration but 17 of them are either from scandal or strategic retirements to avoid a loss. Republicans might have learned from watching the Democrats who stayed in 2010 get shellacked. 52 Democrat incumbents lost election in 2010 and 14 open seats held by Democrats switched to Republicans while just 1 seat switched to the Democrats. Potential challengers certainly smell blood in the water. The 2018 cycle brought out an unprecedented number of House candidates powered by arecord number of female candidates.
Of course, there are key differences between 2018 and 2010. Part of the reason the Republicans gained so many seats in 2010 is because Democrats were deeply over extended from their own wave election in 2006 and by additional gains made from increased Democratic turnout in the 2008 presidential election. One rationale behind the more conservative seat gain predictions for Democrats in 2018 by other analysts stems from the fact that in the 2010 cycle Democrats had to defend 40 seats carried by McCain in the 2008 presidential election, but in this cycle, there are only 25 so-called Clinton Districts. Add to that the partisan gerrymandering that occurred after the 2010 census and there is no doubt that Democrats are more structurally constrained than Republicans in 2010. Right now most of the conversation centers around whether the Democrats can win the 24 seats they need to control the House, not on a wave that will compare in scope to 2010. But the Democrats have an asset that can do much to negate these structural disadvantages. In many districts, there are simply more Democrats than Republicans or in the cases of “red” districts, there are enough Democrats to offset the Republican advantage if Democratic voters have strong turnout (as seen in PA 18 and a host of special elections that strongly favored Republicans). In elections where 30-40% of eligible voters participate, an outnumbered party has a lot of opportunity to offset their number disadvantage.
Based on my analyses I predict that nearly all, if not all, of the Clinton districts will, in fact, flip to Democrats. What makes them so vulnerable is what makes them Clinton districts in the first place. They have 3 elements that my analysis suggests will be strongly predictive of strong Democratic Party performance this cycle. One of these factors is the percent of the district that is college educated because it is from this group that any “pink” surge of college educated women will come from. The average percent of college educated residents in my 12 “will flip” districts is 43%. And that is the college education rate of the overall population, it will be much higher among actual voters. Virginia’s 2017 state legislative elections are a good barometer of a potential pink turnout surge from a highly educated populace. Of the 17 Clinton districts in Virginia, Democrats won nearly all of them. They even picked up a seat in a district that broke for Trump. And the most significant factor explaining districts that flipped from those that didn’t (other than the district’s partisan advantage and challenger spending relative to incumbent spending) is the percent of college educated residents residing within the district. In districts prone to a Democratic surge, Democrats won races they shouldn’t have been able to win. Another key factor is Trump’s performance in the district, relative to Mitt Romney’s performance in 2012. In some districts Trump outperforms Romney, but in other key districts Trump trailed Romney. And what factor accounts for this? The college education level of the district. In the 12 districts I predict will flip, the average under performance score for Trump is -8 points. In the 12 “likely flip” districts it’s -6.2 points.
Each of these 24 districts will draw a wealth of investment from both parties and because of their competitiveness, have produced strong challengers (this fall I will be adding candidate and campaign quality metrics for the 60 or so competitive districts to the analysis). I have included ratings from other forecasting outlets in the table below (in the case of Morris, the predicted probability of the Democrat winning the district) along with my ratings to serve as points of comparison. Although we identify similar districts as competitive, there are significant differences in our ratings. Districts coded here as “will flip” or “likely flip” are largely coded as toss ups by Cook, Inside Elections, RealClear Politics, and Crystal Ball. Although some of my 12 “will flip” districts display high probabilities of flipping based on the Morris scores, others do not. My toss up districts are those districts that are likely to be competitive, but lack factors that make them susceptible to a surge of Democratic voters large enough to overcome the either the structural advantage for Republicans in that district, a strong incumbent, or in some cases both of these factors. Of particular interest are CA 45 & 48, IL 6, and VA 7 which are seen as Lean or Tilt R (or in the case of the Morris scores < 50%) by other outlets, but which are all but certain to flip to the Democrats under my analysis.
The 23 districts coded as toss ups here are coded as toss ups, lean R or even likely R by other outlets. I don’t anticipate much change in the “will flip” categories, but the “likely flip” and “toss up” races will be refined as additional data points become available. One of the key data points will be investment from the RNC, DCCC, and outside entities. Assuming the Democratic Party can marshal the resources needed to compete on a wide map, and also assuming that foreign interference is not a factor, the negative partisanship referendum effect in 2018 should mirror its strength in 2010 midterms, although the scope may be limited by the structural disadvantages that will constrain gains by Democrats. Most analysts agree that Democrats need to win the national popular vote by at least 6% to make significant inroads towards the majority in the House and probably by at least 8% to guarantee picking up the needed 24 seats to control the chamber. Of course, as Hillary Clinton will tell you, it’s not merely the size of the national vote margin, but how that margin gets distributed that matters. This is why the Democrats can win the majority of votes and still fail to make much headway in their seat share. The size of the enthusiasm gap among likely voters in specific competitive districts around the nation will tell us a lot about the ceiling for Democrats in November and there will be more public polling of these midterms than in any midterm in history including polling in Virginia’s 4 competitive districts by the Wason Center. Based on our initial survey in March, I anticipate double digit enthusiasm gaps between Democrats and Republicans among likely voters unless the Republicans decide to hold off filling Kennedy’s vacancy on the Supreme Court.
I should also point out that there is a significant unknown right now. How will Independent voters behave in the Trump Era? Although Independents generally favored Republicans in the 2010 and 2014 midterms history suggests that Independents are not immune to the midterm effect, particularly when the incumbent president has low favorability. In the 2006 midterms when George Bush’s overall favorability was similar to President Trump’s approval rating now (an average of 40%) the Republicans lost Independents by a stunning 18 points, largely due to backlash over the Iraq War as the economy was still largely stable at that point. It is possible that after favoring Republicans during the Obama Era, Independents will favor Democrats in 2018. The recent (modest) uptick in Trump’s favorability is being driven largely by Republicans and right-leaning Independents “coming home” and even among right-leaning Independents his approval rating still lags, coming in at just 71%. His rating among left-leaning Independents is virtually equal to Democrats, at 7% and 8% respectively. More telling may be his favorability rating among “true” Independents. Gallup reports a favorability rating of just 26% among this group. It should probably be noted that Obama spent most of 2010 with a positive favorability rating, an asset that Trump has never had. Still, that didn’t stop him from getting hammered on Election Day because negative partisanship drove Republicans to the polls. If Independents break for Democrats and Democratic voters surge their turnout this cycle it is possible that even the ambitious predictions offered here will prove to be too modest. Preferences on the generic ballot among true Independent, likely voters will be one of the most important data points from survey data this fall.
I will be refining my predictions over the course of the general election as more data, including data on candidate and campaign quality become available. In marginal races I will be examining the electoral strategies deployed by candidates in specific races and how they might impact results and will share those insights here. It should also be noted that the predictions offered here are based on the assumption that Donald Trump will be president on November 6th 2018, no major national security events occur between now and Election Day, and that Democrats successfully exploit their advantages. Stay tuned!