Many different forecasters and prognosticators are trying to predict the results of the 2026 election, using polls and sometimes on-the-ground knowledge. Can we trust their predictions based on their historical performance? Michael Franz goes all the way back to the 1960s, finding that the handicappers have been surprisingly good (even without all of the modern tools). They rarely picking the wrong winner and their “lean” ratings closely matching the eventual results. The old-fashioned experts usually agree with one another and largely match more complicated models. And not that much has changed.
Guest: Michael Franz, Bowdoin
Study: “Predicting Congressional Elections From Expert Ratings”
Transcript
Matt Grossmann: How well can we predict elections? This week on The Science of Politics. For the Niskanen Center, I’m Matt Grossmann.
Many different forecasters and prognosticators are trying to predict the results of the 2026 election, using polls and models and sometimes, on the ground knowledge. Can we trust their predictions? How well have they done historically, and are they getting better or worse? It turns out that election handicapping has a long history we can learn from.
This week, I talked to Michael Franz of Bowdoin about his new co-authored Social Science Quarterly article, Predicting Congressional Elections from Expert Ratings. Going back to the 1960s, he finds that handicappers have been surprisingly good, rarely picking the wrong winner, and closely matching the eventual results. The old-fashioned experts usually agree with one another and largely match more complicated models, and not that much has changed. We also discussed what election handicapping success suggests about the importance of polls, candidate quality, ads, and money, and whether anything we’re seeing in 2026 might change the picture. If you’re waiting for the results this year, I think you’ll enjoy the big picture look back and ahead.
So, tell us about the major takeaways from your recent article on expert ratings in congressional elections. What’d you find?
Michael Franz: Well, we looked at an analysis of the congressional elections over a long period of time, back into the 1960s, comparing pre-election ratings of these races by expert raters, predominantly a congressional quarterly, which did them for many decades back between the 1960s and the early 2000s. And then we added in some subsequent other folks who’ve rated races since then, and we compared those ratings pre-election, right before the election rating, whether the race is leaning Democrat, leaning Republican, or a toss-up race, to the outcome to see how accurate those forecasts were. These were not probabilistic forecasts like we sometimes see from folks like Nate Silver, these were expert ratings or qualitative forecasts to some extent. And we found that they were quite accurate, actually, in predicting the lean of the race, whether it went Democrat or Republican, and often how close the race actually was.
Matt Grossmann: So, let’s start with the broad analysis. So you have these categories that they used, like safe and favored. What did these mean in practice for the vote margin in these races? And how often were they getting these things way off, versus pretty on the mark?
Michael Franz: Yeah, so it’s probably no big surprise that most congressional races are not at all competitive. And so, to say that the bulk of races are safe Dem or safe Republican is not that fantastic of finding.
But what we did find is that when raters said, or the CQ said for instance that this race was going to be safe Dem or safe Republican, it often ended up, in fact, most of the time ended up to be a race where yes, the Democrat would get 60 plus percent of the vote. But they also had sort of gradations, they would say this is a sort of lean Dem race or a Democratic favored race, which suggests a tighter election. And on average, that’s what they would find. So if a race was ranked as lean Dem, then the average outcome in those races over the many years we looked at the data would be about a Democratic win, maybe say 52, 53% of the vote, instead of a safe Democratic win of 60 or 65 plus percent of the vote. And so, they were pretty good at picking up gradations across these races in terms of how close they were going to be.
They very rarely actually would get them wrong in the sense that a race ranked as lean Dem would end up being won by the Republican. That would sometimes happen, but very rarely. The biggest source of variation we found was in the toss-up races. A place like CQ or Roll Call might establish an election to be a toss-up race. And there are many right now that we’re all following in this election that would end up being decided by a bigger margin than four or five or six points. And so in some ways, these experts were more conservative in deciding what was a toss-up race than maybe ended up actually being the case.
Matt Grossmann: So one version of this is that this is just what we did when we didn’t have better data, but now we have polling averages, we have fundamentals, we have all this data available for all of these House and Senate districts. We have different kinds of prediction models and simulations that we can use, and that this is just the old, old version.
On the other hand, you might say these things are still picking up, these race specific factors, that might matter to the outcome. So what were they relying on to get this close, and how much of it was these district specific qualitative factors versus just knowing the evidence before others did?
Michael Franz: Yeah, I think if you go way back, if you go into the ’60s or ’70s and you go into that period where we didn’t have great data, then I think what the raters were doing, these are all experts. So Congressional Quarterly, these are experts, right? These are people that know what they’re talking about in terms of different dynamics.
And one of the things that I thought was kind of interesting was to say, how good were these experts back when they didn’t have all this information that they could plug into models, and so forth? And so that was maybe the intellectual interest that I started with and maybe expected, I think I probably expected to find a lot more variation. To find places where they would look at an election in Missouri or something and say, “Oh, this is a likely Dem seat,” but then get it wrong and actually be won by the Republican. And maybe thinking that they were too reliant on news reports or on the impact potentially of scandals, but seeing that on average they were pretty good.
So then the secondary thing that motivated me in this study was to, not in a sense wade into the sort of forecasting wars that we oftentimes see election to election, but to figure out if we could say that human beings who just accumulate information, some of it’s quantitative, some of it’s qualitative, but who otherwise are just nerds about elections and learn a lot about what’s happening in these campaigns, can they do a good job at baking together a sort of forecast for the outcome? And how does that stack up to the nerd wars about the forecasting models? And that I think was the motivating piece here.
And forgetting for a moment the period of time today where the sort of qualitative forecasts probably are relying on a lot of quantitative data, and we can talk a little bit about that, and looking instead at the past where they definitely were relying on cross tabs of polls, some fundraising numbers, but basically instead a lot more sort of on the ground qualitative information, they got it pretty good. And so, I think that that to me suggests that throwing that out, and I’m not saying forecasters want to throw that out, but throwing that out and saying all of it can be predicted on the basis of a couple of big sources of data and some really good modeling choices and computational work, misses the fact that human beings are actually pretty good at learning about stuff and then making predictions based in line with the thing that they’re experts about.
And I don’t want to go too far down that road into some of the forecasts that come out of other areas outside of politics. I like to stick in my world of politics and say that actually, these folks, they’re pretty good at learning whether or not a particular thing will matter or not.
Matt Grossmann: So in addition to all of the quantitative models, we also have more qualitative forecasters now as well, or expert raters, and you were able to compare across those expert raters a little bit. So, what did you find and do you think that this is sort of a consensus effect where they’re all watching one another, or do you think that they’re all responding to the same information and getting similar results?
Michael Franz: Yeah, that’s a great question. I guess my sense would be they probably do a little bit of both, in the sense that there’s no question because these ratings are so publicized and you can go to Inside Elections or you can go to Larry Sabato and you can look at them. And if I were doing my own, it might be the case that I would go take a look at what other people are doing just to see how close that I would be getting and maybe almost benchmark my own assessments against others. Maybe in fact, I would even tweak my own in the basis of what the others… I don’t know if that’s actually happening or not, so I assume some of that’s happening. So it’s really hard to know whether there’s a herd effect happening with these sort of expert rating sources online, or whether or not they’re each doing their own work and generally coming to similar conclusions.
But when we compared say Roll Call against Cook or Inside Elections against CQ back when they were doing those, we tended to find a fair amount of overlap in the assessments, but not a perfect overlap. And so, sometimes you might see one rating source like Cook Political Report would have a particular House race at safe Dem, but another one would have it at likely Dem. Now, that’s not a huge difference, but they did have a difference, and that happened in a small subset of cases. There would be more disagreement on the competitiveness, in the sense that one ranking might have a race at competitive and another might have it at likely Dem or even lean Dem. And so we did see some variation, but very few instances where one place would say it was lean Dem and the other would say it was lean Republican. And so they all tended to have a similar assessment of how the race was likely going to end up. And that I think was also pretty interesting.
But you still can’t answer that question of how much they’re watching each other and shifting their assessments to be in line with one another. But it still, regardless then, I think gives you generally a good sense if you go to one of these sites or another, of getting a sense of what potentially the outcome of the race is likely to be, in terms of who’s going to win control of the House or Senate.
Matt Grossmann: You also have an analysis where you look at change over the course of a campaign. You generally use the very last rating. And so obviously this section, the forecasters have been predicting more of a Democratic wave than the expert raters, but the expert raters are moving toward the Democratic side as we get closer to election day. So, how much does that pattern generalize? And what did you find about how they are changing over the course of a campaign?
Michael Franz: Yeah, this was a little harder to assess, because getting the overtime, the change is not as straightforward as we might think. So CQ, I went back into the old reports, basically the printed CQ weeklies back into the ’60s and would go closest rating to the election, record those. And my assessment there was right at the end of the campaign. Some of these were mid-October, some of these were late October, but basically, once they had as much information as possible, how close did they get to the outcome? But I would have to go back to the weekly ones pretty systematically and see when they offered pre-election, maybe September, maybe August, maybe earlier in the year, ratings. And they didn’t do that as consistently, and so it was really hard to get the over within cycle change.
But Inside Elections did have, they just changed their website, to my chagrin, but they did have a really good archive piece that if you went to their archive section, you could easily get electronically, you couldn’t download it, but you could record them pretty simply, the ratings on all the different dates. And so we were able to look at within cycle change for several cycles from that one source. And it’s just a reconnaissance, if you will, into the broader question of how much change we see in these ratings. And there’s a lot of stability, as you might expect, but there are changes in Senate and House ratings that are the ones we looked at, we looked at some interesting changes, but they might change for a variety of different reasons, but they do change.
And so, one of the things that might be fun to look at is to go back into the historical data and see if we can get some old within cycle changes, say for like a 1970 election or something, to see if they changed more dramatically than the more recent ones did. Because in recent years, there’s so much data coming across the news services and the internet and the fundraising. It’s so much more present in our consciousness that we would expect a little more bumps along the way, changes, given those changes in information. And maybe that was a little harder to assess back 40, 50, 60 years ago.
But we found some interesting changes nonetheless, and they tended to largely move in one direction. If it was moving in the Democratic direction, moving in the Republican direction, that was reflected. And then other changes would show up based off of the outcome of a primary, or in some cases late redistricting changes.
Matt Grossmann: So, just like the race handicappers can now use quantitative information, the modelers can now also use this qualitative information, and many of them do incorporate these race ratings into their models. But the assessment has normally been that the ratings help early in the campaign when we don’t have a district specific polling or before campaign finance has reacted, but by the time you get to election day, they’re not adding a whole lot to the final results. So, and part of the reason is because they’re slow to react to the national environment, but they finally do. So I guess, what do you make of those kinds of analysis and what does that tell you?
Michael Franz: Yeah, no, I mean, I think it’s a great interesting point about the sort of interplay between these qualitative assessments versus the quantitative data. The quantitative data, if you’re modeling something, the fundraising numbers, the campaign financing numbers, they’re slow to really fold in, I suspect because of reporting deadlines and so forth. So really, a lot of things can be more dynamic because of polling changes. And then you can, I suppose, leverage national polling numbers to infer changes at the race level. But I think polling is probably driving a lot of the more dynamic changes within these models.
And to that point, I mean, we’ve had all sorts of questions about the accuracy of the polling. And so, I think the responsiveness of the quantitative models to changes on the ground, to some extent rely on good-
[inaudible 00:16:00] on the ground to some extent rely on good data, and that good data is not always there if the polling is systematically off in one cycle or another. And so that doesn’t say to me, or doesn’t necessarily say to me, that the statistical quantitative approach is picking up more actual change in an election than is actually ongoing in that election, but it might be picking up the changes that are happening as a result of all the different polls that are entering into and affecting the model estimates.
Whereas the expert raters, yes, they’re slower, they’re not as issued as fast. So if you go and look at, say, Inside Elections, they will change their expert ratings every seven to 10 or 14 days. And so right now, I just checked the other day, the most recent assessment up on their website is I think October 1st, could have changed since I looked. And we’re at October 13th right now. And so that’s pretty old and so doesn’t necessarily have the most current thing that might be happening in a particular election. But at the same time, I’m not as wowed by changes day to day that might say, oh, now the Democrats have a 58% chance of winning the House instead of a 54% chance because I’m not necessarily sure that the inputs into those models are as reliable.
Matt Grossmann: So you mentioned that the mistakes are infrequent, but you also do tally them up and try to learn from them. So when do these bigger mistakes happen? Is it getting a whole cycle wrong as too Democratic or too Republican? Is it overestimating the race specific factors compared to the national trends? Are there any patterns in when this goes wrong?
Michael Franz: Yeah, I think probably what I would say from what I’ve been able to see is that it seems to me that maybe one of the biggest sources of error in these ratings might be underestimating the actual wave. And nobody knows if it’s going to be a wave. Nobody knows if it’s going to be big. We thought we had a pretty good handle on this to some extent several years ago when we’d sort of model aggregate level seat change. Forget about race level seat change or race level outcomes, but take some basic inputs like the popularity of the president or change in economic conditions and then make a prediction on the underlying wave.
And that, it would be pretty good. But actually if you look, it doesn’t really get you substantially to it. It could be really under or really, in some cases, over perhaps, but often it’s under. And so underestimating the potential wave. And so to that extent, I think the expert forecasters in approaching these elections understand, and let’s use ’26 as an example, that this is a pro-Democratic year, but they might not necessarily have a clean handle on whether or not that is going to be a 50 seat, it’s not going to be, or a 20 seat change. And so to that extent, they might end up seeing some races as leaning towards the opposing party, or leaning for that matter towards the wave party, in ways that don’t actually turn out to be the case.
But again, they’re not often wrong. If anything, they’ll say that races are more competitive than they really are, and that’s a sort of safe way to avoid getting it wrong to some extent. And so they’re more often in that situation where they say a race is close and it’s not. And I’ll use up here in Maine as an example. I mean, right now Maine is ranked as pretty much a toss-up race, and it may very well end up being a toss-up race. But it wasn’t in 2020. It was a safer win for Susan Collins. And a lot of people maybe relied on the polling data to suggest it was going to be more competitive, maybe the broader national conditions, which were favorable to Democrats, but she ended up winning that race pretty comfortably.
And so that would be a case where, in this election, it’s rated as competitive, but if she were to win by five points, it wouldn’t be that surprising. And knowing what I know about Maine politics, and the raters would’ve gotten that one wrong. And so technically in terms of calling it competitive when it wasn’t. So those are those sources I think of the major sources where things deviate.
Matt Grossmann: We should mention that you looked at House and Senate ratings, and even though Senate results in aggregate follow less from the generic ballot changes historically, they’re relatively similar in their ability. Anything we should make from that?
Michael Franz: Well, it could be, and it might be that because these races historically, and even in today, just have way more information that the experts are able to assess those fairly confidently. I mean, the Congressional ones, some of these races are, I mean, there’s a lot of interesting information, but there’s just a lot more races.
So you and I’ll meet people, and I’m not one of these people because I just don’t have a great memory sometimes, but people will just remember things about particular candidates or Congressional districts that will be really interesting. So they’ll say, oh, in New York 21, and they’ll talk about a scandal, or they’ll talk about the history of that Congressional district, and they’ll have a pretty good sense of what’s going to happen there given the sort of dynamics that have happened in previous elections. And so those experts are pretty good at those Congressional seats.
Well, it’s easier to be that kind of expert to some extent, I think, at the Senate level because they’re bigger, they’re higher profile. And so in that sense, they might provide even a sort of, I’ll call them regular journalists, but someone who’s not super duper interested in politics can become a pretty good expert about a Senate race or a gubernatorial race, and maybe handicap that race a little bit easier than it would be to do the more idiosyncratic Congressional seats of which there’s less information and there’s many more of them.
Matt Grossmann: So as you know, a lot of the factors that used to matter for these individual races are decreasingly mattering, things like incumbency and potentially candidate experience and ideology, and that’s all in align with the increasing role of base district partisanship and a presidential vote and national trends. So in other words, the kind of baseline model has improved compared to the expert ratings. If you just have district partisanship and not very much else, then you’re able to do pretty well. So what should we make of that? Their results are improving, but they’re up against a baseline that should be easier.
Michael Franz: Yeah. No, I think that’s absolutely right. I was looking at these last year for a talk I was giving about this upcoming midterm. I was talking about what will happen for the Democrats in ’26, but talking about it back in ’25. The first thing you do is you go look at some data on how many Congressional districts were won by Harris, but that are held by a Republican, and those kind of places that the Democrats might be able to think about targeting to pick up. And then sort of the reverse. And what you see is that just doesn’t happen as much anymore. And so when Congressional races line up pretty well with presidential outcomes, and that partisanship is all stacking up together, then yeah, those become easier to predict to some extent.
And so I think what ends up happening as a result is we’re all talking about… And because the House is so close, we’re all talking instead about the 15 or 20 Congressional races that, in the presidential race anyway, we’re also very, very close. And so we’re all narrowing in on a small subset of races that will determine control because control is close already. But then the vast majority of the other seats are pretty easy to predict.
I’ll say a couple things on that though. Predicting the outcome, easier today maybe than it was before, picking up those gradations is still super fascinating. So knowing which races will be 52 to 55% versus 56 to 58% is still an interesting exercise. Can we still do that on the basis of just the information we have about election, about the dynamics in a given district?
But also it’s still then, to me, I still get really sort of amazed at how well CQ did this back in the ’60s and ’70s when there wasn’t as much of a lineup between the partisanship of a district and the outcome in the presidential race. And yes, they were relying a lot on incumbency effects. They were relying a lot on candidate quality and assessments of candidate quality, but it still allowed them to pick up with some degree of accuracy that difference again between a 52% seat and a 56% seat, which is kind of notable.
Matt Grossmann: This is also one of those cases where the people being studied can change their behavior in response to the results, and there’s a case we made that we’re all now more aware of the same information. And so it’s less likely to be the case that we have a House district where Republicans way outspend Democrats because there’s a pickup opportunity that only one side knew about.
And I guess this leads me to how much do you think is still a sort of a special sauce that we need on the ground judgments for, things about the specific patterns of local history, or the candidates and their specific touch with voters versus just we just have more data on the things that matter and we’ve outlined those things that matter pretty well by now.
Michael Franz: Yeah. So do you mean in terms of with all that data, can we more comfortably rely on the quantitative modeling estimates? Is that what you’re suggesting?
Matt Grossmann: I think that’s certainly one side, but I think the other side I guess would leave open more possibility for political actors to kind of develop different views. If we’re all basically relying on the same set of variables, we’re probably going to come to the same conclusions in terms of including in things earlier like candidate recruitment and primary influence.
Michael Franz: Yeah. I mean, I still think to some extent the sort of, well, I will say, I think that even with all that similarity of information, it does seem to be the case that, and I don’t want to draw this out too, too strongly, but it seems to be the case that different political actors still will make somewhat different assessments. And it might be just on the margins. I mean, in the most competitive House races and Senate races, obviously that’s where all the money is going.
But in terms of how much to spend, and even for that matter, how to spend it, I still think there’s some strategizing that might differ across different political actors, even with all that similar information. And so for example, yes, we all have the same information, generally, the same broad access to information today about economic conditions in the district or the popularity even the president in that part of the country that the Democratic Party or Republican Party can both say, “This may be a winnable seat,” or, “It may not be a winnable seat,” or, “It’s a stretch for us,” so on and so forth.
But then after you kind of have those baseline assessments, if you’re going to spend money in a particular place, you have to figure out what is going to be the best way to spend that money. And that may be where you get a lot of variation. So it used to just be if you weren’t in a major city, broadcast spots would be what you would spend your money on, is you might have a ground game, but then you’d also spend a lot of money on political ads.
And now you can do that, or and you can do streaming applications, you can do radio buys, you can do satellite television, you can do digital advertising. You can target or you can broadcast. You can do both. And so what’s the best approach to try to win a marginal seat? There’s a lot more options in the presence of a lot more information. So I think that’s where the strategy might end up causing political actors to make different decisions on how to allocate funds that might end up being different within a district or even across districts.
Matt Grossmann: So political actors might also try to influence these ratings. And in fact, there’s been one debate between the forecasters and the expert raters has been that expert raters have access to internal polling data that the campaigns give them. Forecasters may say that that’s basically just access to spin, and might sort of lead them astray. Where would you come down on that? Is there value, especially where we don’t have a whole lot of public polling of individual House districts, in this kind of access to internal data, or does that just mean that they’re going to be influenced by the campaigns?
Michael Franz: Yeah, yeah. Actually, I don’t know, and I think it would be an interesting thing to dig into a little bit. My hunch, at least my initial suspicion, would be that that kind of critique of the expert rating sort of strategy, I don’t necessarily call it a critique, but just to say that these experts are a little more susceptible to information that might be strategically passed along to them, that that to me sounds a little like concerns about voter fraud in the sense that doing that at scale would be really challenging.
So if you’re a Cook Political Report, or you’re Larry Sabato’s Crystal Ball or something, or Roll Call or whatever, and somebody emails you a sort of, “Here’s what our internals are showing, we’re way more competitive than we used to be or than you are assuming we are.” My reaction if I were in that position would be, “Well, that’s useful information, but I’m going to wait and see how it shows up in subsequent other information.” In other words, I’m going to act like an expert in the sense that I’m not going to just assume that this piece of information is going to move my assessment. I want to sort of triangulate that and see what happens with other sources of data.
I would suspect that that’s what is often happening. And so I’m thinking up here again in Maine in the second district, which we’ve had-
I’m thinking up here again in Maine in the 2nd District, which we’ve had hugely competitive 2nd District race for several cycles now. And it’s looking less competitive this time around, primarily because the former governor is the candidate and the former secretary of state is the Democratic candidate and he’s not exactly super exciting. And so if Matt Dunlap, the Democrat were to say, “Oh no, no, we’re not down by 10, we’re actually down by two,” I would be like, “I don’t know, let’s see what else we can find out.” And I would lean heavily instead on conditions on the ground. I might be relying on news reports or debate performance, fundraising numbers are going to qualitatively impact how I’m thinking about things, quality of the ads, looking at those sorts of things and then making my assessment from there. So I’m not as worried about the sort of ways in which these experts are potentially influenced by information that can’t show up in the more quantitative models.
Matt Grossmann: So one reason we’re talking about this right now is we’re obviously in the middle of a 2026 cycle, but I think one where there is still a difference between how the handicappers of individual races see things versus the quantitative models, especially the ones based more on national polls that show something like a 12 point shift in the generic ballot, which would be very large compared to 2024. So having reviewed all these expert ratings, do you have a lean as to who should be trusted there? And I’ve also noticed that this is also true in even state legislative races and things, people who are on the ground doing ratings will still think that the ones that were the most competitive at the beginning are still mostly the most competitive. They won’t necessarily be thinking, well, if there’s a 12 point shift in the vote, what districts would that open? So is there just a general conservative bias, not towards the Republicans, but toward less of a wave that we should suspect this time?
Michael Franz: Yeah, I mean, I think this election’s super interesting for those reasons. So you go back to those, where are the congressional districts that Trump won and that Harris won and by how much and compare that to who’s in that seat now. And once you do that, you get to the same spot that you might look at from the expert assessments, which is a pretty close House contest that will be decided probably by 16 to 20 toss-up seats, unless we see some surprises or a wave, as we mentioned before, that’s been sort of unseen in the qualitative ethos. And so I think they are very conservative in that sense in assigning races to the district that seems to have the partisan alignment and then holding back a rating or a lean in the small number of districts where things are less clear.
And so that would say, okay, well, the Democrats may end up winning the House. And if I give the national condition boost here to where these competitive seats will go and give not just half to the Dems and half to Republicans, but the bulk or a good chunk to the Democrats giving them that sort of national condition bonus, then Democrats should win control of the House, but they might only get to 225 and not 240 or they might get to 222 or something like that. And so it’s going to still be close.
On the other hand, as we mentioned before, if these forecasters or these experts are missing the sort of wave that might show up, then that 12 point generic ballot lead could move a lot of the seats where Trump won a district at 56% of the vote could make that district very competitive or even move to the Democrat. And if it happens systematically, there’ll be some of these lean Republican seats that end up in the Democratic column. And so that’ll certainly be one of the things I’m interested in the day after the election is how many of these ratings were wrong in that direction.
But the other thing that gives me pause, and this is in October, we’ll be way more critical and confident about where things went off the rails in November, is if the congressional ballot, the generic ballot is plus 12 or 14 for the Dems, if you believe that the polling bias of late has tended to be to the Republican favor in the sense that these polls have been a little more Democratic than the results, then that generic ballot advantage is smaller than it really is in the polling data. And that would lean me back towards the experts as being more appropriately cautious in moving some seats into the Democratic column.
So I know it’s a long answer and it also ends up basically saying we won’t know until the election is over, but I have reasons to believe that both sides could be hanging onto something that may prove to be true. We just don’t know yet.
Matt Grossmann: So one other thing that’s interesting about this year is we have a lot of mid-cycle redistricting changes. Those originally were thought to suggest that Democrats would have to win the generic ballot by more than a couple of points in order to gain control. Since then, I think of the 14 or 15 seats that Republicans supposedly gained through redistricting, about 11 or so of them are considered in play by someone now. So maybe that opens things up more than we expected. How well have the handicappers done with redistricting before? Do they do worse in redistricting years, normal redistricting years because there’s just more change in the actual districts? And have we seen anything like this before where mid-cycle redistricting leads to a bunch of changes over the course of the campaign?
Michael Franz: Well, I will say that in the paper itself, we didn’t actually compare, we should have, but we didn’t actually compare the cycle where districts have been redrawn to the other cycles where we’ve had… So that’s actually a pretty interesting thing to think about and I could probably look at those numbers. In the aggregate, because there aren’t a lot of places where they often got it wrong, I suspect it isn’t the case that they do worse in those years, 1994, 2004, things like that. But there’s a hypothesis there that they would be a little more volatile in those elections. So that’s something to follow up on.
When we did look at within cycle change using inside elections, we were able to see that when redistricting would happen in an election year, and we only looked at 2018, 2020 and 2022 for that sort of within cycle change. When we looked at that, those district lines would often change because of a court mandate. It would throw out a map or something like that in a state and require the districts to be redrawn. It wasn’t a partisan thing or at least it wasn’t a political thing driven through process. It was a challenge or something and probably a partisan challenge, but that ended up being successful or something. And what was neat was that you would see that once the lines were redrawn and you have the ratings for, say Pennsylvania 5 or something, before or after the rating would change completely in line with what you would expect given the new lines, given the new boundaries.
So the experts were pretty good at doing what the media and others would do after these court challenges is they’d look at the new map or the map that was adopted and they would say, “Oh, well look, now this district, which was in a heavy Democratic area has now got a lot of rural areas to it. That’s going to pull in a lot of Republican votes and it’s going to make that race more competitive.” And so you’d follow along, then the court would issue its ruling and then the new map would come out and then the race would just change in its rating as a result of that.
So I think the raters are pretty good at making those adjustments. I’ve been collecting the changes this cycle as we move along because I’m kind of curious at looking at that after the election and I haven’t done that yet, but I suspect that in these states, Texas or California, where these changes have happened, we’ll see changes that fall in line with the new expected district lines. But yeah, we’re in an uncharted world here where this is happening in this way and in such an open way. We’re doing this to win seats. We’re not just doing this under the auspice of challenging majority minority districts or anything like that. No, we’re doing this to pick up seats basically. And so we don’t have a good analogy in the history of that. But the experts, they are good at seeing what the new lines look like and shifting the adjustments to be in line with that.
Matt Grossmann: So we’re also having wide debates about the extent to which primary winners matter and candidate characteristics. And it seems to me that these race handicappers are all in, I guess what I would consider the traditional political science view that, for example, more moderate candidates do better, more experienced candidates do better. And so when they shift after a primary, it is often because of that, because they think the candidate that was either worse positioned or better positioned surprisingly won. So what should we take from that? Is that maybe evidence that the traditional view is correct or just evidence that maybe these folks are still operating in an old political environment and in the contemporary environment that doesn’t matter as much?
Michael Franz: Yeah. Well, that’s a great question. I think to make the, or I guess to rely on the assumption that an incumbent or a more moderate candidate has a better chance of winning is I think what an expert would probably rely on as a sort of good baseline. And they might very well be wrong in this environment today, especially if Democrats, and for that extent Republican Trump voters, would prefer a candidate more on the extreme end of each party’s distribution. Then I think the experts might be slow to pick that up because of the assumptions they’re making that the electorate in a general election is more moderate than the electorate in a primary election. And it might just be that the dynamic candidates, the candidates who have the most charisma tend to come more from the left or the right of each party’s distribution and therefore generate a lot more media buzz and everything else. And so the experts, if they rely on that assumption a lot, which it seems like they do, would be missing that.
I do think that the quantitative assessments though don’t necessarily have an easy way of picking that up unless they’re able to rely on ideological estimates of where these candidates are. And we don’t really have necessarily conditions, political conditions, and I don’t know what you think about this, but political conditions to suggest that more extreme candidates are going to win more often. I know that they’re having a debate in that in political science, but it’s a little different today. I don’t know, it feels different today when you have candidates who are very liberal, but who are also very charismatic. And you combine those two things, that might give them a better chance of winning than a candidate who’s just, say, more liberal, but as charismatic as a moderate. And so I don’t really know if the quantitative assessments would be able to pick that up very well unless you are able to take a candidate quality measure and an ideological extremism measure and say those two things together are really going to be successful and we’re going to sort of try to model that.
So I don’t know. I don’t think that to that point that the expert raters are completely off base in making a very traditional assumption about what will be successful in a general election. And I don’t think the quantitative assessments have an easy way of handling the current environment that we’re in.
Matt Grossmann: In addition to candidate characteristics, obviously we’re both interested in the campaign characteristics and we might think that the expert ratings give us some sense of the continuing importance of actual campaign content, not just how much money is spent, but what messages it is delivering, how hard the candidates are working and who they are reaching. Should we see the expert ratings as a proxy for maybe these characteristics of the campaigns still matter or is that taking it too far?
Michael Franz: My initial impression is it might be taking it a little too far just because the expert ratings are going to lean heavily, I think, on some big macro factors. It could be district specific, it could be unemployment in the district, it could be whether there’s an incumbent or not, it could be a debate performance that was particularly compelling. And that’s sort of a campaign effect, but also reflects the sort of candidate characteristics that might make a candidate do well in that kind of environment and so forth.
And they don’t shift or change all that much. You might see a race go… And we’re talking here about expert ratings that sometimes fall on a five point scale, sometimes fall on a seven point scale. So going from a tilt Dem, which is sort of a competitive race to a lean Dem, to get over that and over there would require a decent amount of evidence to suggest that conditions have changed in the direction of the Democratic candidate. And I’m not totally sure that traditional measures of ad effects or effects themselves are going to do it enough to convince the experts to move the race in that direction. I think there’d have to be a sort of preponderance of evidence which might show up in a variety of different ways.
So yeah, campaigns can have an effect in moving these assessments, but I think they’re not the primary driver. And I’m someone who is a believer in-
… driver. And I’m someone who is a believer in campaign effects. I think campaigns do matter, and I think political science has become a little too convinced that campaigns don’t matter. But I mean, no one would run for office and follow that advice. Nobody would say, “I want to run for the United States Senate, and political science suggests that I shouldn’t run any ads, so I’m not going to do that. I’m not going to spend any money. In fact, campaigns don’t matter at all, so I’m actually not even going to campaign.” Nobody would do that. So they clearly think that campaigns matter to some extent, but I’m a believer that they do just matter on the margins. And so that’s why they might not show up in these… They might not compel the experts to move the numbers on their own, but the campaign can still move two or 3% of the vote, depending on what we’re talking about, and can still determine the outcome in a close race if the election turns on two or three percentage points.
And so there’s a lot to say for campaigns putting their best foot forward and trying to persuade voters and mobilize voters. But how that might impact dynamically over the course of, say, a fall, the rating that that election has is it might be a little bit harder to pick up.
Matt Grossmann: These ratings are also sometimes used in political science for investigating governing. So for example, there are early indications that candidates might face a difficult campaign and then might change their behavior, maybe going to their district more often or voting differently on district concerns. How do you think about that role of ratings? Are they giving members of Congress important information about their electorate and should they be thinking about this?
Michael Franz: Yeah. I don’t know if they’re giving any information that otherwise wouldn’t be already available to, say, the incumbents. So thinking again about second district here in Maine, Jared Golden has hung on against all odds in some cases for many cycles. But I think after the election is over, I mean, he’s not running this year, but last time around after the election was over, I don’t think he needed anybody to tell him that it was going to be another tight race in 2026. And so I think the sort of same thing that might be impacting how the experts assess these races is also something that I think the incumbent would have a pretty good handle on already.
I think maybe a case where it might not would be if you’re sort of a longstanding incumbent, maybe say like a Susan Collins, and you had a somewhat competitive race last time, but you beat it and you beat it good and you start to feel like, okay, I’m kind of maybe invincible. And then the ratings come out early in an election cycle and put you on the list of competitive races. Maybe that raises some eyebrows for yourself and you’re like, what am I missing here? And you start to think a little bit about how, okay, it’s possible that I don’t have this locked up.
But my hunch would be that these campaigns already have a good sense of when they might be in trouble. And so the ratings might reinforce that. Certainly is going to drive some maybe media coverage if in fact early ratings show you to be in a less than safe position and that might compel you to act. But I think you pretty much already have a sense of it and are probably trying to strategize.
Matt Grossmann: So I won’t make you give your individual expert ratings, but I thought it was worth at least commenting a little bit on the Senate races because this is a cycle in which we have a surprisingly broad field. We still have Michigan and Maine on the board, but we’ve expanded to places like Iowa and Kansas as states that the parties are playing in. And you might’ve expected from just national political trends, from the decreasing divisions, decreasing split states between presidential vote and senator that what we’d see is a narrowing. Certainly that’s what we’ve seen in the electoral college, is a narrowing of competitive states. So what do you make of this year? Does it suggest that maybe that this is why race ratings can still matter in this era is because these individual candidate differences and campaign differences or maybe do not all trends continue in the same direction they’ve been heading?
Michael Franz: No, it’s great. And I think when you think about the question we were talking about before in terms of moderates versus, I don’t want to call them extremists, but more sort of liberal or conservative candidates, we’ve seen places where things have moved both directions, but to the extent that, let’s say the Democrats in this case, which are trying to take advantage of the national conditions, have tried to put more places in play. They’ve leaned more onto the traditional assumption that a candidate not of the mold of their party might be more successful. I wouldn’t put Sherrod Brown in that category since he’s a former incumbent who’s well known in the state. But you can’t beat a Sherrod Brown in Ohio in terms of the Democrats finding another candidate. So to that extent, I think they’re leaning towards candidates that they think are broadly favorable to the electorate in those places, and even in some cases not even running candidates and leaning on independents to be the alternative to the Republicans.
And if you look at the ratings, at least in the Senate, as you suggest, the places where we might see some change are Republicans losing possibly to Democrats. And so I think that… And you see in some cases three or four incumbent seats for Republicans are competitive. And if we wanted to not believe this, if we wanted to say, I don’t think this is really what’s going to happen, I don’t think Democrats are going to win the Senate, you might say, then what you would end up leaning on is a little bit that these experts are being cautious. They’re saying conditions aren’t really good for Republicans, so there’s no way for them to expand the map. So conditions are good for Democrats and where are the places that Democrats might have the best chance? And those become the competitive seats. And there are polling data to kind of reinforce some of this, but compelling candidates in Texas, compelling candidates in Alaska, Democratic candidates.
And so hey, if they’re going to win anywhere, if they’re going to win in Alaska, they’re going to win this way or they’re going to win with Talarico in Texas. And so they might be wrong, the experts… It might be that on election day, Paxton and Texas wins by six. And people have gotten Texas wrong. We’re waiting for Texas to turn. And Democrats have offered up all different types of candidates. And so it might be that this is one of those situations where the experts are reading the national conditions, places that Republicans are going to get hurt are these places where Democrats have been strategic. But the electorate might just be the electorate and it might end up kind of sticking with its lean and Democrats might not get more than one or two. And so we’ll see.
But there’s clearly, like if you said, again, I’m going to look at the competitive races and I’m going to give the advantage to the Dems in these places because of national conditions, a decent guess, then the Democrats will win North Carolina, they’ll win Ohio, maybe they’ll win Maine, but that’s it. And so they could get to 49 or 50, who knows if they’ll get to 51. So I think that that’s how I would handicap things based on what the experts have said. There’s no way Republicans are going to gain seats, I don’t think. And so that is, I think, where the experts generally are going to be right is that the national conditions will inform the way the election goes even if they don’t necessarily get all the races right.
Matt Grossmann: So you mentioned that you’re collecting data already in this campaign. So anything you want to tell us about where you think the next big questions are or what other projects you’re working on that you want to tell?
Michael Franz: Yeah. I mean, I’m always interested in… For these data, I don’t have a big thing. So this is sort of, in many ways, kind of a side project. When I was writing my dissertation, I mean, I’m going to study campaigns, when I was writing my dissertation, I was interested in how PACs allocated their money differently to competitive seats versus to safe seats and whether or not traditional corporate PACs or labor PACs made different strategic decisions over time as competitive seats shifted. And so I needed to get assessments of competitive seats and so I relied on CQ. And so I’ve been collecting the data ever since, partially to be like, well, I’m going to update that analysis at some point. But you know how that goes. So now I have all these, I have the PAC data, if PACs are even relevant anymore, and I have the CQ data, but who knows if I’ll ever update the dissertation in that way.
But what I am interested in though is in understanding the allocation of resources. So we have good data on political advertising spending within districts and markets across different media outlets. So I’m particularly interested in the streaming allocation. So as more people watch TV on their smart TVs and less on live television, there’s more need to go there. And so I’m kind of curious to see how the spending sort of enters different types of races at different points in time. So if you’re a competitive race, you might expect an allocation of resources across medium, you kind of go everywhere. But if you’re a lean Dem race, do the Republicans or Democrats allocate their resources any differently either in an attempt to boost turnout by targeting your ad buys or what happens? So I’m still trying to develop the expectations I would have. But sort of a relationship between the expectation of a seat and the allocation of spending in that race. And we’ll see where that goes, but that’s sort of the next sort of project in that regard.
Matt Grossmann: There’s a lot more to learn. The Science of Politics is available biweekly from the Niskanen Center, and I’m your host, Matt Grossmann. If you like this discussion, here are the episodes you should check out next all linked on our website. How Much Are Polls Misrepresenting Americans? Can We Believe The Polls? The Past and Future of Polling. Previewing 2024: How Voters Judge Presidents, and Class, Race, Gender in the 2024 Election. Thanks to Michael Franz for joining me. Please check out Predicting Congressional Elections from Expert Ratings and then listen in next time.
