Republicans and Democrats dislike and misunderstand each other and anti-democratic attitudes are on the rise. But some strategies are effective for reducing polarization and animosity. Robb Willer tested 25 short interventions like videos and informational messages from across the scholarly and practitioner community, finding that many reduced partisan animosity and some reduced support for antidemocratic practices. The effects lasted and provided some ideas for real-world tactics to tackle polarization. The results offer some good news for a change.
Guests: Robb Willer, Stanford University
Studies: “Megastudy identifying effective interventions to strengthen Americans’ democratic attitudes.”
Transcript
Matt Grossmann: How to Reduce Partisan Animosity, this week on The Science of Politics. For the Niskanen Center, I’m Matt Grossmann. Republicans and Democrats dislike and misunderstand each other, increasingly distancing themselves from those in the other camp. In tandem, anti-democratic attitudes are on the rise, especially in defense of each partisan is side, but some strategies are effective for reducing polarization, animosity, and anti-democratic attitudes. This week I talked to Robb Willer of Stanford University about the Strengthening Democracy Challenge and a recent mass collaboration he co-authored, Megastudy Identifying Effective Interventions to Strengthen Americans’ Democratic Attitudes. He tested 25 short interventions like videos and informational messages, finding that many reduced partisan animosity and some reduced support for anti-democratic practices. The effects lasted and provided some ideas for real world tactics to tackle polarization. The results offer some good news for a change, I think you’ll enjoy our conversation.
So your team recently tested 25 interventions to reduce partisan animosity and support for undemocratic practices in the American public, how did you go about doing that?
Robb Willer: Yeah. So what we did was a few years ago we had started doing research on theoretical mechanisms, techniques that could improve levels of partisan animosity and anti-democratic attitudes in the American mass public and pretty quickly we saw that we had some ideas, maybe even a good idea or two, but that these were really, really difficult outcomes to try to treat. And so we thought, “Well, if we really take these outcomes seriously, we would try to do the study that would generate the most knowledge as quickly and efficiently as possible.” So we sort of flipped it around and said, “What are…” Rather than working in the conventional mode of asking ourselves what our best idea was, we sort of went out and asked the field, “What are all of your best ideas?”
And we crowdsourced the problem, we put out a call for submissions that we could test for intervention ideas, anything we could embed in a survey experiment in less than eight minutes that we could administer to a representative sample of Americans. And we said, “If you’ve got an idea, we’ll also work with you to help get it over the walls,” that we tried to create as low as possible. And we got tons of submissions. We actually made this very specific effort to recruit submissions from the practitioner community, people working on democracy and partisan animosity in the nonprofit sector and the activist space in the US. And we wound up with way more interest and submissions than we had expected, about 250 different submissions from about 400 people, actually well beyond the US, 17 countries on four continents, just a way larger amount of interest in submitting to the challenge than we expected.
And then from this, we identified the 25 interventions we thought were most promising for test. And then we tested those in a massive 32,000 person survey experiment that we conducted in spring of last year of 2022. And those 25 interventions were tested against a couple different control conditions. And this was an effort to figure out which of these interventions moved the needle on parts and animosity, anti-democratic attitudes and a bunch of other outcomes and then which of these interventions were best relative to one another.
Matt Grossmann: And what were the biggest findings and takeaways?
Robb Willer: Well, we found a lot of really interesting stuff. So we found for all of the outcomes that we focused on, there were a couple different strategies that united the most successful interventions. So just thinking about partisan animosity or what political scientists call affective polarization, thinking about that first, what we found was that first, that overall the interventions we tested were very good at reducing partisan animosity, much better than we had expected and much better than academic forecasters predicted. So 22 out of the 25 interventions significantly reduced partisan animosity, which is pretty striking. Also, reductions of partisan animosity were greater in terms of standardized effect size than reductions of any of the other attitudes that we’ve measured. I mean social distance preferences from rival partisans, which is really, really similar, had had a similar kind of effect size, but it’s also just a very similar outcome.
And the two strategies that stuck out there as most effective were these. So one was presenting people with sympathetic exemplars of folks from the other political side, the rival partisans who were engaging respectfully or disconfirming stereotypes or just being sympathetic in some sort of fashion. And then the second strategy for this outcome that stuck out was invoking some sort of overarching common identities that would connect people across party lines. For example, the common identity of we are all Americans. And it’s interesting because these are kind of like, if you were to survey a bunch of social psychologists about how to reduce animosity across lines of group conflict, those might be the first two things they would say is, present people with a really sympathetic exemplar and also invoke an overarching common identity. So we kind of came out with a pretty intuitive finding there.
I think that the various anti-democratic attitudes that we measured, things were maybe more novel, at least to me, the findings were not necessarily what I would expect. So one was we found that the most effective strategy for improving Americans’ democratic attitudes was to correct their misperceptions of the views of their rival partisans, to correct misperceptions of how much rival partisans supported eroding democratic norms, how much rival partisans dehumanized members of their party and so on. So it was a kind of mini literature that’s emerged on these so-called meta percepts, in this case, partisan’s perceptions of their rival partisan’s views on things, views towards them, views on the world, et cetera. And because people have really, really inaccurate partisan stereotypes at baseline, this turns out to be a quite efficacious way to intervene. So that strategy stuck out.
And then I’ll highlight maybe one other strategy that was effective for intervening on anti-democratic attitudes, and that was to make more salient the risk of democratic collapse and just how bad democratic collapse looks. So this was an intervention that was submitted by Katie Clayton and Mike Toms a political science PhD student and faculty member here at Stanford. And here it leveraged the comparative politics literature and just video from other countries, places like Turkey, Venezuela, video showing the disorder and police repression on the streets that you see in settings where they’re dealing with some sort of major democratic collapse.
And it was narrated to emphasize democratic collapse may seem like it can’t happen in a place like America, but it can and it can happen fast when things get… When things start to slip away, they can get bad very quickly and an unimaginable reality can set in. So it was very compelling and really quite scary. And it also culminated in footage of the Capitol riot, which helped to connect the dots of, no, this really could happen in America. So these are the strategies that stuck out the most to me as most effective.
Matt Grossmann: So let’s go through a few more of the individual interventions to give people an idea of what we have in mind. I think one of them was a Heineken commercial and there were other ways of creating positive contact or common identity. What were some of your favorite effective ones?
Robb Willer: Yeah, the number one intervention just in terms of absolute effect size for reducing partisan animosity was an intervention that it had a few components but it most centrally featured this video that Heineken’s UK chapter put together in the wake of the 2017 Brexit vote. And this video, it’s a very cool video. It’s definitely worth watching for anybody who hasn’t seen it. It’s called Worlds Apart. And in it, Heineken brought in pairs of ideologically divergent folks from Britain, British folks, to have conversations while they built a bar together and then drank a pint of Heineken beer. And despite the quality of the beer, these interactions actually went really, really well. I don’t have to tell you Matt, you run a bar yourself. But all shade on Heineken aside, they really did a great job crafting this video so that they’ve selected these interactions that went really, really well and were really respectful and even kind of funny and heartwarming.
And there’s a critical point in the video where these pairs of folks, conservative and liberal, have found a lot of common ground, they’re getting along great and then they’re shown video, together they’re shown video of them being interviewed about their views, which are very strong and ideological from before the interaction and now they have to talk through that. And that’s, I think actually a really critical part of it is that it’s one thing to make nice with strangers, but then when you’re really accountable to what your views are when you’re in private, when you express them with co-partisans, let’s say, now you really need to talk through the tough stuff that separates you and they do a good job of it. So this video is very effective. It’s really interesting to me that this is a video that was effective five years later in another country.
Now the ideological differences map really cleanly to the US. It’s trans rights, it’s immigration and so on, climate change, so it is very relatable. And then it might be slightly less threatening as well to see this. It’s a proxy battle. It’s not the same. It’s not the US partisan divide so you don’t map directly to anybody in it, it might be less likely to elicit defensiveness.
Then the second most effective intervention for reducing partisan animosity was an intervention that was submitted by a team of scholars mostly at NYU, social psychologists primarily. And it used a whole bunch of techniques from the social psychology literature. So they really took advantage of the fact that we didn’t say you had to just use one theoretical mechanism or technique at once. And so they were like, “Great, we’re going to win this thing. We’re going to do three or four things.” But if I was to highlight, and so it included corrections of misperceptions of the views of rival partisans. It included some, I believe elite cues from trusted in-group leaders I believe, if I recall correctly. But then the most central element I think was the invocation of a common American identity, which it emphasized a lot. And the scholar submitting it were steeped in social identity theory, so this would be a technique they would be quite good at, and that was a very effective intervention as well.
Matt Grossmann: And what about some of the ones that did not work as well? What did we learn about what doesn’t work?
Robb Willer: Yeah, so this is I think as interesting as what worked, one thing that did not work as well as you might think is invoking overlap in the views of Democrats and Republicans. I think I’ve seen this work in other research and I think that it could work, so it might just be the implementation. It definitely is one of the most striking things when you look at patterns of polarization in the American public that American’s attitudes, especially less politically knowledgeable Americans, are not that terribly polarized and are not trending to get that much more polarized.
They’re sorting to parties of course, but they’re not, like the actual distribution of attitudes, it’s not strikingly polarized, but an affective polarization, steadily growing, steadily growing. That’s probably because sorting is more related to affective polarization than attitudinal polarization is per se. But it also looks to me as an intervention opportunity of, oh, if the affective polarization’s outstripping the attitudinal polarization, but that people think of their party identity as rooted in their attitudes, then let’s leverage that gap. And so an intervention called Party Overlap tried to do that and wasn’t that successful. So maybe there’s another way to do it, I don’t know. It was a very intuitive approach.
Another intervention that didn’t have any effect on partisan and animosity, but which I thought could, was one that tried to invoke a common identity as the following, a social class-based identity. And it said, “Look, Democrats or Republicans are divided politically, but really almost all of them have much more in common with one another in that they’re getting screwed economically while a very small percentage of folks are taking the lion’s share of profits in the American system.
And this has been accelerated during the pandemic when the super rich are getting really rich and normal Americans are getting unemployed,” and so on and so on. Just for me, with my own political background, I was like, “Yes, that’s exactly right.” And so I thought this would be potentially compelling and it didn’t move the needle on partisan animosity, even though it did try to invoke a common identity. And I think that fits pretty well with the literature on how hard it is to craft class-based identities for political mobilization in the US, that hasn’t generally worked, Seymour Martin Lipset and others on this topic. That’s a tough thing to make work, or at least across party lines it is.
Then one last thing that didn’t work very well that I would’ve thought was, an intervention that sought to ameliorate feelings of threat that each party might have on the basis of their parties standing and power in the country. So it told Democrats, “Hey, if you think about it, you have lots and lots of power in the system,” Democrats have the presidency and both Houses of Congress, which was true at the time, and they have demographic advantages in the future and they should not feel threatened at all by Republicans. And then did something similar with Republicans with the Supreme Court and so on, and geographic advantages in the Senate and the electoral college. So it just made a case that one need not feel threatened if one is a Democrat or did the same thing for Republicans and it didn’t move the needle at all, but it had a logic to it, it had a clear theoretical logic. It could have worked conceivably. So I think it’s insightful, the ones that don’t work as well.
Matt Grossmann: It seemed harder to move anti-democratic practice support than partisan animosity, but it also seemed like most of the interventions were geared more towards partisan animosity. So have we learned anything about how connected those attitudes and their sources are and whether it is just harder to move the anti-democratic support attitudes than the partisan attitudes?
Robb Willer: Yeah. Well, it’s a great observation and I think… I was actually talking to Brendan Nyhan recently about the research and he said that he thought this was the most interesting finding, that the partisan animosity was so much more malleable than the democratic attitudes or really everything else that we studied. And I think it is an interesting finding that we don’t entirely understand the causes of this pattern. So the first thing I would highlight as a potential cause is just the field has been way more focused on reducing partisan animosity than on improving democratic attitudes. Democratic attitudes have only become a focus in the last couple years, partisan animosity has a decade long literature.
It also is something that’s been studied across disciplines more. So partisan animosity is immediately legible to social psychologists who study intergroup conflict. Often in settings like Israel and Palestine, they say, “Aha, we have a whole suite of ideas and methods for attacking this. Let’s go after it.” Whereas the anti-democratic attitudes, to even understand how they work, you have to understand democratic institutions and so it’s just something that hasn’t commanded as much attention outside of political science yet, whereas partisan animosity definitely has.
Also, the practitioner space is much more robustly focused on partisan animosity than on anti-democratic attitudes. I don’t know exactly why that is, but I mean that’s itself sort of phenomenologically interesting that you have thousands of activists that are really interested in partisan animosity and reducing it that feel their communities and their families being cleaved apart by these divisions, but affectively cleaved apart that are less concerned about democratic stability. And I think democratic stability has emerged with Trump as a major American concern, whereas the partisan animosity has been something we’ve been thinking about and writing about since at least the early 2000s or even the ’90s people have been saying, it seems like polarization’s getting worse and more heated for decades now. So there’s just more momentum for people to be upset about that, have thought about it and done research on how to reduce it or activism on how to reduce it.
So that would be the main reason that I would highlight. But I even think, having said that, even if we had a similar knowledge base for both problems, that it would be harder to achieve the same effect sizes for anti-democratic attitudes because at least with the way we measured anti-democratic attitudes, we juxtaposed the defense of democratic norms against partisan interests. So here we were heavily influenced by Levitsky and Ziblatt, and Graham and Svolik who have an excellent recent APSR article where we said, to ourselves, we implicitly said, okay, it’s not hard to get people to enforce democratic norms on their rivals, they’re going to vote against the rival partisans most of the time anyway. So getting them more motivated to do so doesn’t move the needle a lot. Ideally, if you’re going to really defend these norms, it’s going to be getting the other 45 to 50% of Americans to enforce those norms on their own partisans, when it hurts, when it’s against your interest. That’s the key thing, especially if it’s going to be a norm, it’s going to actually breach some consensus level.
And so that’s really tough. So we ask people questions like, “If somebody from your party didn’t acknowledge the results of the last election, how likely would you be to vote for them? If they shut down polling places in areas that benefit the other party, how likely would you be to vote for them? Or what would you think of that action?” Those kinds of questions are tough. If you asked me that question, I have a party I prefer, would I actually stay home or even vote against someone from my party if they did one, two, three of these actions? I don’t know. I’d like to think I would because I care about democratic norms a lot, but I also think there’s a bunch of other morally loaded stuff at stake that differentiates my party from the other party. And so you can easily see how people who care a lot about politics would have trouble defending these norms and enforcing them on their own party. And that’s a powerful force to overcome. And I think that’s part of why the treatment effects were smaller.
Matt Grossmann: So we call these interventions, but they were, and attitudes, but they were on a survey where at most you can show people a video or give them some text and images. We didn’t show up at their house with their friends and make that big of an impact. And we ask them afterwards some questions, but as you know, anytime we try to take the findings from survey experiments outside of that context, we find that the effects delay quickly and might not necessarily match real world outcomes. So how likely do you think these interventions would work in the real world, both if we got people together and somehow put them through this, but then how likely would they be to stay with the participants in the face of say, a year long election campaign with lots of partisan fighting designed to bring them back to their sides?
Robb Willer: Yeah, it’s a great question. So yeah, you’re right. We refer to them as interventions, for lack of a better expression, but we don’t confuse them with real world field interventions. And we do think that that is the next step to test how to apply this knowledge in the field and whether it makes a difference and under what conditions. One thing we’re doing to that end is we’re organizing now a grant competition where we have people, researchers and practitioners, pair up, work together to make a proposed field experiment where they take one of the insights from the challenge, implement it out in the world with the help of a partnering organization and then evaluate it in the field experiment that’s headed up by the academic part of the team. And we’re helping to form some of these teams, pairing practitioners and academics since folks don’t necessarily know one another, and then also some folks do know one another. And we got something like 30 grant proposals that we’re going through now, 35.
At every stage of this project when we run into a thorny problem, we try to crowdsource it basically. So at first it was like, “We need good ideas, we don’t have enough of them. Let’s crowdsource that problem and let’s try to get everybody involved.” “Here we have a problem of the survey experimental evidence is, I think, compelling and insightful, but it definitely doesn’t directly relate to the field. How do we close that gap? Let’s crowdsource that as well.” “Oh, the academics and practitioners don’t know one another, and that’s a big inefficiency. Can we intervene on that to help them know each other so that they can work together?” So we’ve tried to get as many minds on every difficult problem as we can.
So not knowing yet what will or won’t work in the field, and really the five or six projects we’re able to fund won’t give us a final word on that either, it’ll just be the beginning of my answer, to speculate a bit, I agree that interventions of the sort we test in survey experiments often would decay in the field. We have a lot of reason to think that a lot of our political interventions don’t make a difference out in the world.
So what can you do? The most efficacious way to intervene would be to fundamentally change the structure of people’s information environments such that they chronically would feature the mechanisms reflected in the more successful mechanisms from the challenge. So is there a way that people’s information environments could be invoking overarching identities or featuring those at a higher rate relative to partisan identities? Is there a way that people’s information environments could give them corrective information or correct information about the views of out-partisans rather than inaccurate information, which I think is what people get on balance now. Is there a way that people can get real life exposures to people from the other side rather than only these kind of simulated caricatured ones they get from co-partisans and from themselves?
The most obvious way to do that would be through social media platforms, which have a great deal of influence over people’s information environments. They don’t have a lot of incentive to intervene on this problem because it generally trades off with engagement for them, which basically means lower profits because the opposite kind of content does very well on social media and keeps people coming back to social media and keeps the social media companies making money. And what our approach has been to go to the more motivated, the less powerful actors in this space, which are the bridging organizations, nonprofits, activist groups that work all the time on these problems, try to support their efforts and get some evidence-based clarity on what of the things they’re doing makes a bigger difference and what makes a smaller difference or maybe even has backfire effects. That’s the easier place for us to intervene because the motivations there.
But if you look at the lay of the field, you have these unmotivated but very powerful actors, the social media platforms, you have these very motivated but much less powerful actors, these bridging organizations, and you have no highly powerful, highly motivated actors. And so I’m tempted to be very sad about that, but then I also think, “Well, that’s sort of in the nature of the ecology of social problems in a society, that if they elicited a bunch of powerful motivated actors, they would probably stop being social problems and would exit the space. So I don’t think it’s an unusual lay of the land for a problem, but macro social problems are hard to work on and we’ll do our best. But I mean, I’m encouraged by the fact that there are literally hundreds of thousands of people who work to some extent or another in these bridging type organizations, people that are out doing something about affective polarization because they care a lot about it. That’s the beginning of taking effective action, and then we’re trying to bring some discerning data to help them be as strategic as possible about their actions.
Matt Grossmann: So I guess, let me take it the other way and say maybe you’ve found some easy interventions. Maybe all those people on the ground don’t need to be working and we can just have the funders run the Heineken commercial as much as the political ads in the advertising season. Do we think that it might be easier than people think, it’s just a question of getting the message through?
Robb Willer: Well, it’s a really good question. I think that we found that for some outcomes like the democratic outcomes, it is no easier than we think, those are hard to treat. Probably the most important outcome in the democratic space is, I mean this would be my read on it, my normative read would be reducing support for undemocratic candidates in the mass public seems like the most important space to intervene. I think that improving the quality of techniques we have there is good, elections are won often by very small margins. You can bring a whole bunch of potentially monied or motivated volunteers, moneyed actors, motivated volunteers to the table motivated by the democracy part that wouldn’t normally work on an election, and then this is helping provide them more techniques for intervening.
So I think that that’s a space where a little bit could go a long ways. That isn’t a very… Because it overlaps with electoral politics in complicated ways, that’s a not fully activated space. Right now the people who care a lot about democracy are Democrats, and the Republicans don’t care much about democracy, or at least they focus more on winning elections as do the Democrats and so we haven’t yet gotten a third force motivated that says, “Democrat or Republican, we’re coming in to support the pro-democracy candidates because that’s actually becoming a threat to the country.” But as that emerges, and it may, we would have some techniques. And the effect sizes don’t have to be enormous to be influential there because of how elections work.
On the flip side, we did have some pretty big effect size interventions, like the Heineken commercial, which did have a persistent effect two weeks later on partisan animosity from just viewing this video. And so I think it would help for every American to see the Heineken commercial or more content like it, I think it could only help. I think that for the activists in this space that are more focused on getting people to break bread together or sit down in a living room and have a conversation together, that their instinct that real in-person interaction’s even more effective, I think they’re probably exactly right about that. And some of our own experience facilitating diotic Zoom conversations across party lines suggest you can achieve even higher effect sizes that way.
And so I think that we can take some of these techniques and potentially improve how we structure those interactions, some of the features we add to them, how we prep people for them, how we guide people through the interaction, because I think that that… I still think that they’re right, that that’s a very good way to intervene. So I wouldn’t say living room conversations, you should shut down your operation and just show people the Heineken ad or spend all your money on placing Heineken ads on TV or something. I think, yeah, even based on our own data the cross-partisan conversations are great, and the question is how do you scale them? And that’s a challenge.
Matt Grossmann: So you’ve targeted the American public with these interventions, but I think most people would agree that political elites are at least as responsible for bringing them about. I know you have done some work on trying to influence the opinions of legislators. So what do you think, I guess, about the chance to change views among people who are directly involved in this process with the same kinds of interventions work with political elites? Is there more or less opportunity to change their views?
Robb Willer: Yeah, it’s a great question. We have a short paper that’s forthcoming now at Proceedings, the National Academy of Sciences, where we recruited a sample of mostly state level politicians to go through a meta perceptions correction experiment where they give their guesses about what levels of support for partisan violence, anti-democratic attitudes and partisan animosity that the American public reports and specifically rival partisans report. And then we correct that. And that correction’s not very big for partisan animosity, people tend to estimate that their rivals have high levels of cold feelings towards them, and they’re right. So that correction’s pretty small, but people tend to really overestimate how much their rivals support political violence and overestimate to a good extent, to a significant extent, how anti-democratic they are.
And so we found that this intervention reduced… That people were inaccurate in similar ways to the general public. They were a little less inaccurate than the general public, which you might expect since politicians need to know something about what the general public thinks in order to keep getting elected and so on, and that they think about it all day, every day. But we found that even this group that you might think would have really stable, well considered views on these things, that they even had some malleability in particular on their own support for undemocratic practices. They supported undemocratic practices less after this correction.
Which I think, thinking about the psychology of these folks, it fits well with the sort of… That a lot of the escalation that we’re seeing around consideration of undemocratic moves comes from perceiving that they have to do it because the other side’s going to do it, that kind of don’t bring a knife to a gunfight logic. And that when you say, “Oh, actually that’s a pretty big misperception, the other side doesn’t support that as much as you think.” They turn it down a little bit. So that’s encouraging because, as you said, a way more powerful and influential set of actors to treat. And interesting that even though they knew more about public opinion and their levels on all these things were a little lower, a little bit healthier, I would say, than the general public’s, they were still treatable with this technique. They were still wrong enough to where correcting was helpful.
Matt Grossmann: And you’ve also done some research showing that these elite attitudes, if you can highlight them, can actually change public attitudes, especially among say, Republican elites changing public attitudes in the COVID space or in the anti-democratic tendencies. So I guess if that’s the case, they don’t seem to believe that it will work, or they seem to believe that they can’t go against the initial base attitude. So what could actually change the direction of expert cues that people are getting from elites and why do elites seem to have the perception that they really can’t affect those views?
Robb Willer: Yeah, it’s such a good question because… And the way we approach this is if there’s some sort of, we would say deleterious pattern of public opinion related to polarization where there are trusted in-group or in-party leaders that are going away from the apparent partisan consensus, leverage them, that’s going to be your highest leverage way to intervene. So what I mean is, you have a vaccination problem, but Donald Trump actually happens to be pro-vaccine even though most Republican leaders are anti-vax or are not commenting on it, leverage Donald Trump. Present people with Donald Trump endorsing vaccination, that’ll help. And we found that, at least for the survey experimental data, that was just about the most effective intervention in the early vaccine days of 2021 of anything that we tested or that we saw tested. And then it was replicated in a field experiment speaking to that there is a path from survey experiments to field experiments, a field experiment that was run by Brad Larsen and colleagues and shown that you could make ads that had this Trump endorsing vaccination content and increase vaccination rates out in America.
We did the same thing with election legitimacy. Most Republicans early in 2021 are questioning the results of the 2020 presidential election. Some of them are not commenting on it, but then a small number like Mitch McConnell are saying, “Actually, this was a totally legitimate election and there’s no credible evidence for fraud here.” So leverage that, promote that person, get that person and others like them in front of Republicans and that will increase their faith in elections. And we found that it did.
So the interesting thing to me is your question of, if that works, and if we had a perfect information system, which we clearly don’t, that wouldn’t make a difference because people would already know, “Oh, 80% of Republicans think the election was fraudulent, but Mitch McConnell and a significant number of dissenters say the opposite.” They would already know. But I think the biggest problem here is they don’t already know. So when we would show people Donald Trump promoting vaccination, for example, most people didn’t know that Donald Trump supported vaccination even though he had publicly supported vaccination on several occasions, it had gotten national news and so on. But within the conservative media bubble, this was not being promoted, Trump supporting vaccination was not being platformed. And likewise, Mitch McConnell supporting the 2020 election was not platformed. A lot of Republicans didn’t know that there was significant dissent early on on that view.
So you could think of it as going against the media prioritization of voices that are being heard in this case by conservative Americans, that that’s why the intervention works, that’s why the information is salient and novel because they’re just not hearing it within a very biased media ecosystem. And I’m sure you could find equivalents on the Left as well. Yeah. Yeah. And so that makes it hard to scale as well because you’re going up against that apparatus.
Matt Grossmann: So you’ve been working obviously with a large team, but in particular have worked across disciplines with the political scientists and the psychologist on these projects. So reflect a little bit on how the disciplines think about polarization and what their contributions are and what their blind spots are, maybe in the context of running it by one another.
Robb Willer: Oh yeah, that’s a fun question. Yeah, and I am fortunate… Or I enjoy having a multidisciplinary background, sociology itself’s a kind of multiparadigm, multidisciplinary field of sorts where we have to read outside of the discipline as well as within. So for me, I enjoy sort of code switching with my political science colleagues and my psychology colleagues and thinking like they do and then trying to bring that back to other spaces. And so for me, it’s very, very profitable to, it’s just… And it’s really fun to engage with people who approach these problems really differently.
At a high level, the difference I noticed between psychological political psychologists and political science political psychologists, these are the folks I am most often collaborating with these days, the psychological political psychologists, they want to find human universals as much as possible. They would like to find just fundamental aspects of human cognition and behavior that will shape political attitudes and behavior in any context. And if they have a fault, it’s celebrating the discovery of these prematurely. Holding up, “Yay, we found another human universal great news,” and that it just doesn’t even apply in a parliamentary system, let’s say.
Then the political scientists are very, very good at identifying political psychological dynamics that are operant in a particular context that really are mattering for, let’s say, contemporary American politics. They drill down on those questions really, really well. And then if they perhaps have a weak spot, it’s maybe not trying enough to find those trans situational mechanisms or develop general theories that would articulate the political behavioral dynamics and then the contextual moderation, what aspects of political institutions and systems moderate that political behavioral dynamic such that we would see it in this slightly different way, in this other context? Just, I think sometimes a willingness just be like, “Well, that’s very hard,” which is true, “So we’ll just do it in the American political setting.” But it would be great to do the more general theoretical project as far as we can do it.
I am no role model for doing the best version of either of these. If anything, I probably do it a bit more like the political scientists of saying, “Well, I’m just going to try to do as well as I can in the American politics setting.” But yeah, if I had to highlight strengths and weaknesses, that’s probably how I would do it. And this line of analysis is heavily influenced by my colleague Neil Malhotra, who I think does a really nice job of laying out this terrain and articulates all this better than I do.
And one thing he said is he was like, “Lord, Ross and Lepper,” this original confirmation bias paper that Lee Ross and colleagues led back in the late ’70s is just a perfect work of psychological political psychology. It’s a phenomenon, very new to show at the time that is true in a lot of contexts and it’s not the final word at all. In particular, Bayesian models of political cognition are really substantially revising how we think about this. But that is the psychological approach at its best, a new thing that you’ll see in communist Russia, but you’ll also see it in the US and you’ll see it in 1970 and you’ll see it in 2023, and that the psychologists, they’re arguably better at doing that because they’re trying really hard to do exactly that. But then if I was to make a list of my 10 favorite political psych findings of the last few years, the majority would be probably from political science.
Matt Grossmann: So your study also contrasts a bit with the replication literature where we have lots of mega studies, but they tend to find pretty small effects. You have done what a lot of people are suggesting, which is that you’ve been collaborative, you’ve taken proposals from lots of different people and work together in a big research team to evaluate them. So reflect a little bit on whether, I don’t want to say you found a solution, but at least you’ve highlighted some good trends in the social psychology world and maybe what the downsides were to trying to do this global cooperation.
Robb Willer: Yeah, thanks for that question. Yeah, and it is a striking contrast, of you read Nosek, et al, 2015, you’re like, “Oh, these social psych experiments anyway are replicating, at a very low level… Well, just a concerning level, 40%, maybe a little less.” You even read Camerer, et al’s, science paper and you’re like, “Oh, it’s better in experimental economics. It’s not radically better. It’s not like 80%, that’s sad.” We don’t like that either. So it’s not just the psychologist that need to improve. And so you might have this dismal view. And I think that was, going into our study of how much success these interventions will have. And that was reflected in the forecasting where the academics were pretty pessimistic about the interventions relative to reality. And actually, interestingly, relative to the practitioners who maybe were, it was less salient to them, these recent replication problems.
But then we get 22 to 23 out of 25 interventions being successful, on partisan animosity anyway, that’s the positive pattern. There were other ones where the interventions weren’t that successful and you’re like, “Oh, well, what’s going on?” Sorry, I’m just recapitulating your question here. And I think one answer to that is political science. Political science has been really rigorous about publishing code and publishing data sets and making it possible for people to check their work. They have worked with fewer self-generated data sets than psychologists, so they’re more likely to use the ANES, for example, which anybody can download and check your work on very easily.
Political scientists also have been better about publishing null effects and making light of them being important. And so there’s a different culture that I think would lead you to expect the average political science idea in 2021 when we were collecting these ideas is going to be more replicable than social psychology and also your average just social science discipline perhaps because it had good practice, it had above average practices and had for a while. So the fact that there’s a substantial role of political science here is helpful.
I also think that most of the social psych work in this space was from the last five, six to seven years, and social psych got a lot better in that period. You have a great book on this on how social science has gotten so much better, which I really appreciate that optimistic take on it. Having been primarily a social psychologist over the last 10 to 15 years, it’s just striking the terrain and how it incentivizes good work that’s valid and replicable, it’s just changed enormously in a way that radically changes the incentive structure for researchers. And so at that same time, polarization’s emerging as a problem and so it’s just kind of set up perfectly for the social psych knowledge in this space to be unusually replicable.
And I think is a great sign for things to come in support of the Grossmann thesis that social science is doing a lot better and that it’s a great time to be a social scientist because of it. Because our, more than arguably ever before, our interests are aligned with the interests of our science. We have set up much, much improved curbs and barriers that keep us doing our work in a way that’s good for the collective scientific product. So that’s my main thought on why we had good replication rates or why our research looked so different from Nosek, et al, at all and the many lab studies.
I will say on the flip side, that we also let people submit pilot data with their intervention and that we selected the 10% of submissions we found most promising. So it’s also this, it’s not directly comparable to Nosek, et al, where yeah, they’re replicating actual published studies randomly selected. In our case, we took the ones we thought were most likely to work, so that’s a huge factor as well.
Matt Grossmann: So what’s next for the Strengthening Democracy Challenge and maybe this framework that you have going where you take ideas from lots of different researchers and try to test them in a similar format?
Robb Willer: Yeah, so we’re working on a number of extensions of this work with a group of folks we’re working on potentially crowdsourcing, get out the vote tactics to see if we can identify even more efficacious ways to intervene and promote voter turnout in the United States. We’re also working on a collaboration with researchers in the Israeli/Palestinian context to see if some of the insights from the challenge can be ported there, combined with intervention ideas from the Israeli/Palestinian context and some original crowdsourcing to try to do something ideally better for the Israeli/Palestinian context, which of course the conflict is worse there, so we’ll need to do it better to contribute knowledge. We are also working on a variety of other ideas where, I guess, the way I would put it is that we’re trying not to just let the sort of acceptance of polarization and intergroup conflict as a sort of consensus normative bad determine the outcomes we’re working on.
So for example, I think that there’s good reason to think that union density in the US should be higher, that working Americans would do better with higher unionization rate and so we’re working on a challenge now to see how you can motivate Americans to be interested in joining unions at a higher rate. What kinds of appeals are more or less effective there? That’s a real departure from working on something like democratic attitudes or voter turnout where you poll people in 80, 90% of people would say, “Yeah, that’s good.” Union support in the US right now is really high, it’s 70% or something. You pull people on whether getting more folks interested in joining unions is a good thing, you’d probably get, I don’t know, 60, 65% of people saying yes. So it’s a little bit more of a contested public good, but I am trying to think of the most effective ways to put this kind of methodology and technology to good ends. And so we don’t want to just work on the high consensus public goods because you’re not necessarily targeting the biggest social problems.
Matt Grossmann: So you mentioned that some people aren’t necessarily on board with polarization as the universal bad. There has been some recent pushback to people saying that if one side is operating anti-democratically or one side is operating against fundamental rights, then people should be upset about that side. So I guess to what extent do you take that critique and to what extent does it matter, in your case the meta percepts off, but we might have a case where anti-democratic attitudes actually grow and are quite strong on one side. And what are the implications of there being a reason why people might have these negative attitudes about one of the sides?
Robb Willer: Yeah, I think it’s a great question. Really complicated and hard to say simple definitive things. I mean, here are some thoughts on this. A critique I hear from some of my most valued colleagues who are skeptical of polarization research or some polarization research is that there’s just too much of it, as a portion of the American politics research space, it’s too big relative to other problems that we have in American politics. And that that dynamic of we work on the very, very high consensus stuff, and when we greet it in peer review we are friendly to it. Because we say, “Oh, it checks the box of working on a social problem that’s largely uncontested. We all agree this is a bad thing. Moving on.”
Whereas other work on racial justice or on gender inequality or poverty or what have you, we might be less likely, the papers may struggle in the ecology of publishing American politics work because that box for some people doesn’t get checked and holds them up in review and they say, “Wait, actually this isn’t a consensus social problem, or not everybody would agree, therefore, this needs to prove itself in a way that the polarization papers don’t have to or has to justify itself.”
I think that I agree with that critique. I think we have an overfocus on polarization. I think it stems in part from this dynamic of normative consensus being… That normative consensus social problems get studied without much trouble in peer review, without being held up on that point. I also think that those longstanding norms around normative consensus were invented for good reason and to stop the primarily Left wing social scientists from just doing highly ideologically biased work that they like, but isn’t as scientific as it could be. And I don’t know what the best way is to find a new set of rules, but I think that it would be great to elevate that conversation and have it in a really intentional way so that we could adjust the rules to make sure we’re producing the most valuable knowledge. So in that way, I really agree with the polarization critique.
I often think that, as you alluded to, some of the sorts of interventions that are suggested by the polarization literature are really hard to debate with, like correcting misperceptions of what other people think. How could you defend that we ought to misperceive what other people think? That seems like a hard ground to hold in a debate. It seems like we should want to be calibrated to reality. So some of the interventions I think, evade this critique pretty well.
But I think the toughest one is the point that a lot of this work is done in a symmetrical fashion as though the anti-democratic energy in American society is symmetrical, but it’s not. And here I think the truth is really interesting and nuanced. If you go out and measure anti-democratic attitudes in the mass public, which I think are clearly a part of the problem, you don’t see massive differences between the Left and Right, you see a latent anti-democratic willingness on the [inaudible 00:54:17]. I’m just quoting the public opinion data here, just go out and ask people how willing you would be to vote for somebody who denied the results of an election that disadvantaged their party, and Republicans report higher levels of that willingness. Democrats report concerning levels of that willingness. And you can think about conspiracy theorizing after Kerry lost in 2004, there are examples. And that’s not being capitalized on by Republican elites.
So right there, we get some helpful data that there is an anti-democratic latent sentiment on the Left and Right it’s worse on the Right. And then the bigger difference is the behavior of the elites, which is massively worse on the Republican side in 2023 than on the Democrat side. Now to me, that helps us understand our problem and where to intervene on it and where it’s going to be hard to intervene on it. So I think what we do with all this polarization knowledge we’re generating, that’s where a lot of the normative considerations come in and I think the data’s really helpful.
So in our study, it might seem like we’re suggesting some sort of symmetry between Democrats or Republicans, anti-democratic attitudes just because we studied both of them. But I think the alternative of not testing the interventions on Democrats because they’re not raiding the capitol right now and maybe won’t be, I certainly hope, I don’t agree with that because I think by testing these interventions on Democrats as well as Republicans, we find out, for example, that pretty much all the interventions have similar effects across party lines and I think that’s helpful scientific knowledge. I think that a practitioner should walk up to these data and definitely consider that the anti-democratic moves are happening on the Right far more than on the Left in 2023, and think about that in allocating scarce time and money to intervening on the problem and then hopefully we’re offering them helpful tools. But I think it was the scientifically better move to study this on both sides, even if I would not advise a practitioner to make a lot of use of the Democrat data in 2023, if that makes sense.
Matt Grossmann: It does. Anything we didn’t get to that you wanted to include or any take home message you want to leave us with?
Robb Willer: No, I just really appreciate these questions and the engagement of the work. And for us working in this multidisciplinary space is so rewarding because we get access to so many cool new methods and ideas. And so, in particular, just having a background in social psychology and reading more and more political science over the years, it’s just very enriching and it helps us confront weaknesses in our work and try to improve them. Also, I want to say, Matt, thanks for doing this podcast for all these years. I’ve assigned episodes of this on a graduate syllabi and shared episodes with colleagues and so on. I mean, it’s really a great resource. It’s a tremendous resource.
Matt Grossmann: There’s a lot more to learn. The Science of Politics is available biweekly from the Niskanen Center. I’m your host, Matt Grossmann. If you like this discussion, here are the episodes you should check out next, linked on our website, Reducing Polarization with Shared Values, How Marriage and Inequality Reinforced Partisan Polarization, Compromise Still Works in Congress and with Voters, How Political Values in Social Influence Drive Polarization, and How Much are Polls Misrepresenting Americans? Thanks to Robb Willer for joining me. Please check out, Megastudy Identifying Effective Interventions to Strengthen Americans’ Democratic Attitudes, and then listen in next time.