We fear that legislatures are gridlocked and performative, rather than focused on actually passing new laws. But some legislators are consistently more effective than others, at the state and national levels. Can we train legislators to be more effective at advancing their bills? Mackenzie Dobson finds that training programs for state legislators don’t make them more effective than their untrained colleagues. Most of the factors that make legislators more effective are structural: the competition between the parties and the positions they hold. Some of the common advice, find bipartisan partners and become an issue area expert, may not work everywhere.
Guests: Mackenzie Dobson, the University of Tennessee
Study: Does Legislator Training Improve Policymaking?
Transcript
Matt Grossmann: Can we make legislators more effective? This week on the Science of Politics. For the Niskanen Center, I’m Matt Grossmann. We fear that legislatures are gridlocked and performative rather than focused on actually passing new laws, but some new legislators are consistently more effective than others at the state and national levels. Can we train legislators to be more effective at advancing their bills? And if not, is it because most of the determinants are out of their control?
This week, I talked to Mackenzie Dobson of the University of Tennessee about her research on legislative effectiveness. Her new work finds that training programs for state legislators don’t make them more effective than their untrained colleagues. Most of the factors that make legislators more effective are structural, the competition between the parties and the positions they hold. Some of the common advice fined bipartisan partners and become an issue area expert may not work everywhere. We reflect on the lessons for state and national policymaking and for the reformers trying to coax legislators in a policymaking direction. I think you’ll enjoy our conversation.
So what is state legislative effectiveness? Why do we care about it? And how did we create a measure that we can rely on across states and time?
Mackenzie Dobson: So legislative effectiveness broadly, so this is beyond just state legislative effectiveness, at least in the political science community, is a way that we think about legislators being able to pass public policy. So it is a procedural measure that documents or quantifies how successful legislators, whether they’re members of Congress or state legislators, are at seeing their bills through the legislative process.
At the state level, this can be a bit more tricky because states often have various rules that shape the way the game is played, so to speak. So many state legislatures have bill introduction limits or germaneness provisions that act as constraints on the types of bills they pass, but we’re still interested, nonetheless, in which of those bills ultimately become law. And so quantifying that is what state legislative effectiveness is and it’s a really useful benchmark to be able to document how productive and how representative state houses are.
So as I mentioned, there are many rules that surround the legislative process across states, and so that can make this sort of comparability point a little more nuanced at the state level just because one state chamber may be very different than a neighboring state chamber. And so we’ve taken, as scholars of legislative effectiveness, many attempts at reckoning with this to be able to make indices that are comparable over space and over time.
Matt Grossmann: And what do we have now? How many states and how long of a period and what’s the scope of what we’re able to say now?
Mackenzie Dobson: Sure. So at the subnational level, the Center for Effective Lawmaking has made great gains in quantifying state legislative effectiveness, and I’ve been fortunate enough to be a part of that process. In the general sense, if we’re talking about all policies put together and we’re interested in how productive a legislator is at seeing her bills through the law making process, regardless of what policy area she might be active in, we have been able to document that across the state houses from around 1993 to present.
In earlier time series, there have been some nuances across specific state chambers like Nebraska or Kansas where there have been different rules that have made documenting these sorts of gains difficult, but at present we now have State Legislative Effectiveness Scores for all state houses here in the US and we are working now on backfilling some of those scores with issue area effectiveness scores.
So I’m working on a project with colleagues at Florida State, at TCU, and at UVA and Vanderbilt to create the State Legislative Effectiveness Score across 18 different policy areas. So we can not only document how successful legislators are in general, but start honing in on issue specialization and to be able to say something about how successful lawmakers are at getting their bills through particular policy domains. And that is an effort that is a work in progress, but we are able to do that as of now from around 2009 to 2010 up through the most recent legislative terms.
Matt Grossmann: So what are the common critiques of this measure? How much do they apply more or less to state legislatures to like houses or senates? And in what sense have we sort of reached agreement that this is the primary thing that legislators are trying to do or should be trying to do?
Mackenzie Dobson: Yeah, great question. So legislative effectiveness generally, like any measure, is subject to critique, and across both the federal and subnational levels, one of the biggest critiques of both the concept and the subsequent way we measure or operationalize it is it’s only getting at one dimension of what legislators are responsible for doing. And so this is tracking policy productivity. Legislators engage in a lot of other important roles too though, right? Things like oversight or constituency service, and at present, these are at least not elements that are baked into traditional measures of legislative effectiveness. So that provides an opportunity moving forward for scholars to think about ways to incorporate other dimensions of legislators’ jobs into measures. But as it stands currently, I think I view that as one of the larger limitations of legislative effectiveness, broadly.
Specific to the state level, though, we have far more limitations in the data availability space as well as comparability across state chambers, and so I’ll say something about each of those. So in terms of data availability in the US Congress, we have the Library of Congress, which maintains congress.gov and its corresponding API. So it makes tracking legislation a lot easier. There’s a common source we can go to and it’s well-documented. At the state level, that’s not the case. So oftentimes we see piecemeal efforts to collect data on state legislative outputs, whether these are bill introductions or roll call votes or let alone complete bill text, and that just makes our job as scholars of state politics a lot more nuanced because we’re often looking at legislative archives specific to a state or specific to a state chamber, sort of piecemealing together other scholars’ data sets for different time series that are often incomplete or across very few state chambers over time.
And so one of the larger efforts I see in the field now that I’m happy to be a part of is trying to create some open data initiatives that are working toward bringing those data together and have them in one public repository so that scholars, moving forward, will have complete bill introduction information, have complete roll call data across the states and not just within a particular state or within a particular time series. So data availability is a huge hurdle at the state level. Resources like LegiScan have made it possible to at least collect some data from 2010 to present, but backfilling from that point on is something that we’re still actively working on and I imagine will take a great deal of time to do.
In terms of State Legislative Effectiveness Scores, another limitation is that we can’t directly make comparisons between a legislator surveying in state A and a legislator serving in state B’s score. So for instance, I’ll use two example states, a legislator who has a value of, let’s say, 1 in Michigan and a legislator who has a value of 2.5 in Tennessee aren’t actually comparable. So the legislator, despite having a score of 2.5 in Tennessee, isn’t necessarily more effective. So it’s all relative to their co-members within that chamber. And so we have to get creative as scholars about thinking about ways to rescale these measures to make them comparable. And we see people doing that, but the scores themselves are not directly comparable. So I view that as a limitation that we don’t have to grapple with in the US Congress.
Matt Grossmann: So what do we know so far about the major correlates of legislative effectiveness in Congress and how well they also apply to the states? And what about those things that we would have liked to know, but there’s not enough variation in Congress to know that we now have a chance to investigate?
Mackenzie Dobson: Yeah. So there are definitely some similarities in terms of legislator and institutional characteristics between Congress and the subnational level that are predictive of effectiveness across both institutions, and then there are some important differences and differences that we’re starting to see people leverage in important ways to teach us new things.
So in terms of the shared predictors of legislative effectiveness, this is probably unsurprising, but being in the majority party helps a lot. So when you are advantaged, whether that’s through seat share or through positions of leadership, you typically have a higher effectiveness score. So legislators who have served for longer periods, they’re more senior members, they’ve sort of learned the ropes, so to speak, they are typically more effective than freshmen counterparts. Members who are chairing committees or serving on what we call “power committees”, these are committees like ways and means or rules, appropriations, they typically are in a position to engage in gatekeeping and to sort of see their initiatives through the process in a way that a rank and file member might not be able to, and that all contributes to just being more productive in the policymaking space.
In Congress, one interesting difference, though, is that we see bipartisanship really matters. Members who are in the majority and the minority parties who are working across the aisle are more effective lawmakers and this is true in earlier, less polarized times as well as now, if not, arguably, more now. So competition in Congress is increasingly high. We see majority parties operating under a lot of uncertainty within each midterm and presidential election cycle. It is truly a flip of a coin in terms of who is going to control a particular chamber in the subsequent election. And because of that, that fundamentally restructures how legislators think about their relationships. And legislators are relational actors. They are aware of the decisions they make in this current term affecting their relationships and their ability to pass legislation in future terms. And so because of that, bipartisanship is a strategy that is adopted by a lot of legislators, perhaps more than we would expect in Congress, when we se policymaking actually occur.
At the state level, though, my research would tell us that this is not the case. And it’s not that bipartisanship itself is less valuable or is no longer an ingredient for effective lawmaking, but that absent party competition, bipartisanship is just not as useful and it’s not a predictor of effectiveness anymore. And I think that is one of the most important lessons that I’ve learned in the research that I’m doing in the bipartisanship space, and that’s that its benefits, whether it be for effectiveness, for democratic representation, for policy responsiveness is not uniform. So bipartisanship really helps legislators in both the majority and the minority party when the majority is relatively slim, when they need one another to govern.
And at the state level, we often see one party control states, whether we see Hawaii, for example, that has been exclusively controlled by the Democratic Party since statehood. In 2017, they didn’t have a single Republican Senator in their upper chamber. We can look at many Republican-dominated states where Democrats are not represented, North Dakota, Wyoming. These are very different than the US Congress, and because of that, it fundamentally restructures what being an effective lawmaker means. And I think that that’s really consequential for how we think about prescribing bipartisanship as being a desirable attribute. It is helpful and is conducive to effective lawmaking if and only if there’s competition, and we see that in Congress. And so being a bipartisan legislator is really important for being effective, but that doesn’t translate to the subnational level.
Matt Grossmann: So ideally we would be able to teach people how to be more effective legislators, and I know that the groups that you’ve been a part of, and my group as well, tried to do that, but you have some new evidence looking at whether these legislative training programs are working, at least to increase legislators legislative effectiveness. So walk me through how you were able to do that analysis and what you found.
Mackenzie Dobson: Sure. My team of colleagues are interested in determining whether or not some programs put on by good governance institutions can help act as another type of institutional reform and make legislators better at their jobs. And here we’re interested in better being more effective. So we’re using these Legislative Effectiveness Scores to test this and we were able to collect data from the Council of State Governments who hosts four regional training programs not only for legislators, although legislators are the majority group across these different regional programs, but also elected and appointed officials of other offices. And they hold these generally four to five-day long training orientations where they invite political scientists, they-
Training orientations where they invite political scientists, they invite lobbyists, they invite former elected officials as well as representatives from other good governance organizations to speak to them and try to inform them about the importance of different aspects of their job as a public official. And despite some of the curricula indicating that they are taught things like some legislative skill or the value of compromise or how to negotiate, we don’t actually observe differences in their legislative effectiveness outputs after treatment.
So we were able to test this through collecting data on alumni records for these various programs and across the board as well as looking at the individual programs themselves, we don’t see an observable difference. And so we’re hoping that while this is not necessarily the result that we would hope for in terms of good governance and making legislators better at their jobs, that this can be a learning opportunity as we see more good governance organizations and centers developing with this shared goal of making lawmakers better.
And so the lesson we’re drawing from this is, it’s not that all programs don’t have to work, it’s just that we need to fine tune them to make sure that the skills that are being taught are aimed at the technical craft of lawmaking and not some of these broader leadership themes which aren’t translating into effective legislating.
Matt Grossmann: So because you know who attended these programs you’re also able to try to figure out who is actually trying to take advantage of these. And I think you also have some access to thinking about how legislators are selected to attend these and why they say they are there. So, what do we know from that at least for this analysis? I know that it would be a concern for the analysis if they were already selecting in the most effective legislators, but they don’t seem to be. So what are they selecting for?
Mackenzie Dobson: Yep, that’s right. And part of that is because these programs are geared toward lawmakers who are freshmen, that there isn’t this opportunity to sort of become rising stars. In our model specification we control for this using legislative effectiveness scores. But in terms of looking at just descriptively who’s attending, these programs are aimed at legislators in their first few years. In fact, three of the four programs that we look at explicitly say that legislators cannot have served for more than four years with preference for their second and third terms in office. So they have at least had some legislative experience through a single term, but they’re still sort of getting their footing, so to speak, with the hopes that these training interventions can then help them succeed in future terms.
And so these are freshmen legislators through and through and so we don’t have this concern that they are necessarily rising stars. And then in terms of what other characteristics these legislators have or maintain, these programs at least based on the public information that we’ve been able to look through suggests party parity. So we see at least aims in having an equal share of Democrats and Republicans being admitted to these programs, and then specific to the legislature a balance between upper and lower chamber representation as well.
Legislators do have to go through quite the process to be selected though. It’s unclear to us if they are responsible for paying for these trips out of pocket. It doesn’t say that these are fully funded by the institution, so that’s something we’re interested in learning a little bit more about as well in terms of travel arrangements and so forth, but legislators have to, for many of these programs, write four to five short essay responses, submit letters of recommendation from either legislators that are in their chamber or in the sister chamber, other leaders in elected government in some cases alums from the program that they might know, as well as governors or judges within that particular state. They also have to write a cover letter and in a few cases they have to also provide a personal statement that suggests how they view the program as being an intervention that could help them in some particular way.
So it’s a very involved application process. This isn’t simply checking a box and saying, “Hey, I would like to be considered as an applicant in the coming years.” These are legislators who are at least through the application process documenting their interest in participating in these programs with the hopes of getting something out of it. And the hopes, at least as they’ve been documented through a series of videos put forth by these institutions, interview legislators who’ve participated and suggest that it is fundamentally reshaped how they think about their job. And so we’re not observing that change in their effectiveness score, but these legislators report feeling as though they’re better equipped to do their job and passing public policy. So it’s not only on dimensions that aren’t observed in effectiveness scores, but on being a responsive legislator and addressing societal problems. And so they respond as though they have sort of internalized this treatment and they think that it’s going to be observed in their work, but we’re just not finding evidence as of now that that’s actually the case.
Matt Grossmann: So I know you don’t have full access to all of these training programs, but some of your co-authors have attended and you have these videos and some materials about what is there. So what does a typical training program look like? And I know that there’s some things that might be geared toward being an effective legislator, but what are the other goals that it seems like they’re trying to achieve?
Mackenzie Dobson: Sure. So some of the curricula that we’ve been able to look at do include sessions on sort of learning the ropes. These are often taught by political scientists, learning the importance of negotiation, how to navigate the committee process, things that are more closely aligned with sort of the technical craft of legislating, but we’ve also seen sort of larger, more obtuse language about being a better leader or being someone who is willing to work with others. And it’s unclear to us what skills are being taught at these programs that are aimed at these sort of like broader, more obtuse terms.
We also know that lobbyists have a presence at these programs. And so to what extent are these programs aimed at also connecting legislators with lobbyists which might influence the way they think about making policy in the future? Some of these programs are supported by lobbying firms. And so that’s something that we don’t have a ton of inferential leverage on as scholars, but something that my team is at least aware of. And so I think there is ample room to dive into that further to see if there are other ways of sort of testing effects of lobbying presence at these events.
Matt Grossmann: So you mentioned that a lot of the characteristics of effective legislators that we know about are these stable or hard to change factors like they’re in the majority party or they’re in a position of power, they’re on a particular committee. I know that obviously if it was 100% that, then it would be hard to show that you’re able to make people better legislators. But I know that there’s also quite a bit of variation within that. So beyond these kinds of positions or kind of institutional factors, what stands out in terms of making a legislator who may not have access to those positions more influential than their peers?
Mackenzie Dobson: So relational lawmaking is really important. And so I think even if an individual legislator isn’t serving on a power committee or isn’t in majority party leadership, connecting themselves to those individuals does matter and there can be downstream effects. So in work that others are doing, we’ve documented that there are sort of favors or points that are given to these individuals who are setting themselves up to be connected with otherwise more powerful legislators. So even if we don’t observe a lawmaker as being more effective through her effectiveness score, if she is in the same room, so to speak, as some of these individuals who are better positioned to be effective lawmakers, there may be downstream awards for her involvement with a particular group of better connected legislators, whether this is eventually being appointed to serve on a committee or having a better connected legislator sign on as a co-sponsor to your bills. There are other things beyond just passing your own sponsored legislation that can help a legislator become more connected and have downstream consequences for being an effective legislator.
Matt Grossmann: As you mentioned, you also have a lot of research on bipartisanship. So remind us when that is helpful and when it’s les helpful and if it means that minority partisans are just sort of at the mercy of what the majority party wants to do if they can moderate ideologically to gain effectiveness and sort of what is still up to them.
Mackenzie Dobson: Sure. So you’re right that bipartisanship is not uniformly helpful and so I argue in my work, not uniformly desirable. So absent party competition bipartisanship is a less useful legislative strategy and something that I sort of caution against prescribing as sort of integral to small D democracy. So when we have party competition, meaning that the majority and the minority seat share is closer in balance, the majority party fundamentally relies on the minority party to pass legislation and that minority party knows that they are useful, that they have a voice and that they’re heard. Now this isn’t to suggest that the majority and minority party members are going to get along on everything and that concessions are going to be given, but it at least provides an avenue that when certain pieces of legislation are really important, bipartisanship is a productive path forward.
And we can look at substantive policy gains in both the US Congress and subnational legislatures where we see greater competition, like Virginia or Colorado, to see evidence of this playing out. But when we move into subnational chambers where party competition is seemingly non-existent and in fact we may see instead that there are factions within the same party, that is where bipartisanship is of less utility. These are cases where minority party legislators are so sidelined that they aren’t seeing their bills become law even if they are supporting majority party endeavors.
So published work that I have demonstrates that minority party legislators in these secure states are consistently co-sponsoring majority party bills in hopes of sort of building up this reputational capital, but this reputational capital isn’t translating into their own policy gains in subsequent terms despite building up this image of being a team player. And so while it’s not allowing them to cash in on policy gains, it may be helping them in other ways. So my suspicion, and this is in line with work that others are doing, there are rewards that the majority party provides minority party legislators that are beyond the scope of making public policy. So as I mentioned earlier, this may be something like allowing them to serve on a committee. We see this in Texas where at least until the most recent session, it was not uncommon for minority party members, particularly in the state Senate to serve on committees. And so there are other favors that we may see majority party legislators providing minority party members with.
But in terms of passing public policy, it’s a grim finding. When competition doesn’t exist, bipartisanship is not a productive strategy for minority party legislators and we’re not seeing evidence that they’re any more likely to pass public policy that’s important to them if they are a bipartisan legislator or if they aren’t.
Matt Grossmann: So is there anything that does stand out among the legislators that are able to still succeed? Is moderation mattering? Is expertise mattering? Is there anything that we can tell those folks?
Mackenzie Dobson: Sure. So earlier we talked a little bit about individual legislators characteristics that predict effectiveness and policy moderation or ideological moderation is something that even in these uncompetitive environments continues to be a predictor. It’s not as strong the predictor in non-competitive or less competitive chambers, but it still matters. And in ongoing work that I’m doing, I’m actually looking at how majority parties continue to build coalitions when they are a super majority because the idea of coalition building is just as important in these cases. Now, it’s not necessarily a bipartisan coalition that legislators have to prioritize building, it’s building a heterogeneous coalition within your own party. And so this is where moderation continues to really matter. In states where the majority has a super majority or a more sizable seat chair, there are more heterogeneous preferences within their own party and those are preferences that have to be organized. And so policy moderation becomes particularly useful when-
And so policy moderation becomes particularly useful when a legislator is in a majority party in a state chamber where the majority enjoys a sizeable sea share. So looking at Utah is a great example of this where there are essentially two Republican parties and those two parties have to reconcile oftentimes competing preferences to pass public policy. The votes that they are trying to pick off aren’t among Democrats. Those are our sideline minority party members, but ideologically distant members that share the same party label are the legislators that they have to try to vote by with. And that’s what we see as being useful for being a productive and effective majority party member.
Matt Grossmann: So you mentioned that one measurement concern is that state legislators operate very differently in terms of how centralized the bill writing is or the assignment process or any of these kind of institutional characteristics. I think we assume that state legislatures are centralizing because we see that in Congress and thus we might expect further concentration of these kinds of measures. Is that true? And what are the kind of institutional differences that really matter in the size of the majority party advantage or the size of the key leadership position’s advantage?
Mackenzie Dobson: Yep. So you’re right that on average we continue to see centralization occur at the subnational level. There are institutional characteristics that certainly matter. I think the most important is not structural, it’s electoral and that is how competitive the chamber is, how much turnover are we observing and is that majority party a permanent majority? Beyond that chamber size matters a lot and this is just related to having an opportunity for more heterogeneous preferences. If we think about the number of seats in the Alaska Senate, 20 individuals, they’re far fewer preferences that have to be reconciled to pass policy and it can be far more centralized. In state chambers where the chamber size is larger and competition is greater, centralization is less evident at least on average. It’s a bit too premature to speak on now, but colleagues and I are working on creating issue specific state legislative effectiveness scores. So being able to quantify how effective state legislators are across 18 different policy areas and we’re hoping to have a bit more insight on whether or not it’s also issue specific.
It could be the case that centralization and issue complexity play a role in who’s involved, who the key riders are, which degree of expertise or specificity goes into a particular policy proposal. And one could imagine that that varies in important ways by policy issue area.
Matt Grossmann: Since you mentioned it, I know we’ll keep the caveat with it that it’s still in progress, but can we say yet whether the same legislators are effective across the board or if we have some that are really effective in particular areas and if effectiveness is really mostly measuring some specific subset of issue areas or if it’s kind of across the board?
Mackenzie Dobson: So it is a bit too early to say whether or not legislators are more effective if they are a specialist or a generalist, but that is a substantive area that we hope to soon be addressing. What I can say though is that there seems to be opportunities to be an effective legislator beyond some of the institutional positions that we’ve spent some time chatting about. These are things like being a committee chair, being in the majority party, being a more senior member and that is being a descriptive representative. And so some of our preliminary work now in the measurement paper that we’re working on is interested in documenting a host of policy areas that we are labeling as women’s issues. So these are issues that we expect women legislators to be more involved in relative to their male counterparts. And we see that they are not only more active in those policy areas, but that they’re more successful in those policy areas too.
And so this is at least initial findings that suggest that being an effective legislator is not something that is totally predicted by institutional positionality and that having a particular background or identity or occupation might matter in ways that enable legislators to be better representatives and more responsive to specific societal problems that we’re not able to currently observe with the traditional measure of effectiveness.
Matt Grossmann: So with the traditional measure, what do we know about the characteristics of individual legislators like race and gender, or do we know anything about previous office holding and is there kind of a trajectory where the longer you serve, the more effective you are? How regular is that kind of a pattern?
Mackenzie Dobson: Yeah. So we know that more senior members are more effective. They’ve had more time to sort of learn the ropes. So seniority is very much a predictor of effectiveness as well as being in the majority party. In terms of legislator specific identities, we see that it is typically the case that a legislator’s identity is not predictive of effectiveness in a linear way, but often it’s the case that interacted with some other characteristic or occupation. It might matter in important ways. So among women legislators, for instance, we know that they generally are more effective when they’re in the minority party. So that’s some nuance that is important in the US Congress. You also mentioned occupation. So work that I have published with two other co-authors, we find that political experience is something that’s important, not only for being less polarizing but also more effective.
So work that I have in the US Congress finds that individuals who are not political amateurs, so these are individuals with prior elective experience, they bring those skills with them to Congress and it helps them build relationships with other legislators so they’re more likely to be bipartisan and engage in cross-party co-sponsorship and that translates into effectiveness gains in the US House.
Matt Grossmann: So we’re also obviously speaking at a time of polarization and the data here cover fairly extreme polarization across state legislatures over this time period. So do we know anything yet about how that’s affecting who is effective or how much the other characteristics matter?
Mackenzie Dobson: Yes. So across the time series that we have at the subnational level, being an ideological moderate continues to predict effectiveness in periods of lower as well as higher polarization periods. So it’s perhaps unsurprising that being a more moderate legislator enables a legislator to be able to gain support for her legislation and see that through the lawmaking process. So that’s something that we continue to see. My broader concern is less about ideological moderation and more about a lack of competition and that’s related to polarization, that’s related to things like redistricting and gerrymandering, which are producing these outcomes. So at the subnational level, we don’t see very competitive individual districts. So not only are chambers not necessarily competitive where the minority party has hopes of becoming a majority in the future, but we see state legislative seats as existing as essentially family heirlooms. So we see states occupied by the same person for term after term.
In many cases, we see a family member occupy that seat after the fact and that’s concerning. Polarization is certainly a concern in terms of being able to moderate on a policy issue area, but more importantly, competition is restructuring an ability to even consider passing a particular piece of legislation and that to me is a little bit more concerning, particularly what it means for a minority party member being involved and not only the minority party, but by extension their constituencies and sidelining those voices I think have dire implications for how we think about State House democracy.
Matt Grossmann: So we have found so far, you, well, not me, have found so far that the training programs that we can study don’t necessarily increase legislative effectiveness, but a lot of what we’ve discussed might help explain why in the sense that there might not be as much we can tell them as we think. Some of the characteristics are stable characteristics of individuals. Some might require changes in views that they don’t want to make or can’t make easily. And some of the strategies that we might teach them don’t work everywhere depending on the environment that they find themselves in. So I guess how big of a problem do you see as that? If you have the legislators in a room, do you still think you have a set of things that you can tell them that will sort of reliably help across state legislatures or is part of the problem that we actually don’t have that curriculum to teach?
Mackenzie Dobson: Yeah. That’s a great question. So I’m going to answer this in two parts. So sort of stepping out of this room with legislators, I think one aim of future training programs should be focused on candidate recruitment. So what we are observing are the consequences of a lack of competition and one way we can bring about competition is ensuring that the electoral processes are more competitive. And so encouraging more people to run for office is an aim that I view as certainly an ambitious one but an important one.
Beyond that, when we are talking about reforming existing legislative behavior, it’s unclear to me at this point whether training programs can change some legislative behavior if it’s so determined by these sort of structural forces that we spent some time talking about and electoral forces. But I do think that moving forward training programs that are geared toward reforming existing legislative behavior should be aimed at the technical processes of the lawmaking process and ensuring that legislators, particularly these freshmen targeted legislators that are participating in these training programs are taught the value of relational legislating, of how to be a co-sponsor, of what attracting a co-sponsor means, how to write technical legislative policy and which networks that they should make sure to take advantage of to be able to network with already effective legislators in their chambers to hopefully generate some of these spillover effects.
And so that would be my goal moving forward is to hopefully take these results from the training paper that my colleagues and I currently have and really push for program reform among some of these other institutions that are in centers that are active in the training space to really hone in on the legislative craft. It seems like that’s missing in a lot of the programs now and they’re more geared toward leadership skills and in these more obtuse themes that are certainly important, right? Compromise, negotiation, those are important things, but I think we need to sort of hone in from the aerial view and focus more on specific legislative skills.
Matt Grossmann: So you also do work on Congress. So talk to me about how your national work informs your state work and vice versa. And I know that most state legislative researchers are paying some attention to Congress, but it’s not always true in the other direction. So why should people interested in national politics and Congress be interested in understanding these changes and variation in state legislatures?
Mackenzie Dobson: So you’re right that the foundation of my research actually came from being dissatisfied with some of the conclusions that we’ve drawn from congressional research and congressional politics. And that was that the idea that bipartisanship is this inherently valuable thing just didn’t make sense to me when we thought about the conditions under which these theories of congressional policymaking take place, particularly the notion of insecure majorities and that incentive structure doesn’t exist at the subnational level. And so that was sort of the springing board for me to start thinking about ways that states are different in important ways that might inform how we think about legislative politics broadly. And so you’re right that I’m active in both the Congress and the state legislative space, but I’m always mindful that the institutions, while related, are fundamentally different arenas and I think focusing on the differences between the two chambers is a real opportunity to learn more about each of them simultaneously. And so for me-
… to learn more about each of them simultaneously. And so, for me, that meant wanting to understand why legislators who were working together in competitive environments like the U.S. Congress were better at their jobs and that we didn’t see that at the subnational level. And so, that motivated that research agenda.
In terms of why congressional scholars and state politics scholars should be paying attention to what one another are doing, we have ambitious state legislative officeholders, and so it is true that state legislators often are using their statehouse as their stepping stone. And whether that means they want to serve in a statewide position or if they are hoping to eventually be on Capitol Hill, that provides an opportunity for us to learn a little bit about how state legislative behavior can potentially translate to the U.S. Congress in ways that are important for representation. And so, I hope to continue seeing people be interested in legislative politics broadly and not putting themselves in either a Congress or a state legislative camp.
Matt Grossmann: You’ve also been able to observe up close how having a focus on measurement and getting measures that everyone wants to use are able to stimulate research agendas. And I know that you’re now transitioning to link some of the two most successful measurement strategies in American politics, linking this legislative effectiveness data to the Policy Agendas Project issue category. So I know that you’re steeped in these measurement debates as well. So talk to me a little bit about the importance of having these standardized measures that we all can use and how that simulates research, but also the difficulties that come with it.
Mackenzie Dobson: So first and foremost, being able to quantify the political phenomena that motivates so much of our research is incredibly important for the continuance of these research agendas. These pressing political problems that we want to be able to answer or to better understand rely on our ability to apply the scientific process to them, and that requires thoughtful measurement. But as with any measurement, there are always a host of limitations or concerns, and that’s something we have to take seriously and guard against to the best of our ability.
And so, in the legislative effectiveness space, some of the most pressing issues at both the congressional and the state level are, well, are we capturing all dimensions of legislative effectiveness? And at present, the answer to that question is no. We think we’re capturing the most important, and that is policy responsiveness and policy productivity, but there are other dimensions of a legislator’s job, such as constituency service or oversight, that we’re not capturing in these metrics. And we hope that scholars moving forward will take those issues seriously in their own work and think creatively about ways to better incorporate those other tasks in potentially a multidimensional measure of legislative effectiveness.
At the state level, that problem is compounded greatly because we have different institutional rules and norms that structure what the policymaking process looks like. So even within the dimension of policy productivity that we do measure using legislative effectiveness, that policy process is not linear, even within a state, let alone across states. State legislative rules fundamentally reshape the policy process across our 99 chambers in ways that are important, and we have to not only think carefully about the computational efforts that go into creating these measures, but being a true substantive expert and understanding what those rules mean and how they shape some of the patterns that we observe.
So we’re really excited to be able to be moving into the issue area space, but one limitation of that is the data that we are using start in 2010. And so, we are already having conversations about are there opportunities to expand beyond 2010 and backfill these data so that we can make over-time comparisons by issue area for more than the current time period.
Matt Grossmann: One of the, I don’t know if it’s a criticism, but one of the conversations that comes up when I talk to state legislators about this kind of measure is that maybe some people either don’t want to pass as many bills or think it’s actually a problem that the legislature is passing too many bills. So how should we think about that in terms of this, especially if there’s an ideological component, where most things under consideration tend to be new things government could do and a lot of conservatives will say, “What I want to do is stop things from moving forward.” How should we think about that?
Mackenzie Dobson: Yeah. That’s a great question. Importantly though, there’s not an extra point that legislators are given for passing government-enhancing or government-expanding policy. If they are passing a restrictive policy, that is a substantive piece of legislation, and so that is something that’s captured in these measures. Now, if an individual legislator chooses not to be active in the public policymaking space and that is their prerogative, they are going to have a lower effectiveness score, because the way we think about effectiveness is a legislator’s ability to shepherd legislation through the lawmaking process. But this idea of policy valence is not something that the measure is susceptible to in that a permissive and a restrictive piece of public policy, should it be passed, is something that would be baked into a legislator’s effectiveness score.
Matt Grossmann: Is there anything we didn’t get to that you wanted to include or anything you want to tout about what is next?
Mackenzie Dobson: Yeah. So since I’m going to be given an opportunity to share some new projects, I am primarily active in the state legislative space, and I think the ISLES, the issue-specific legislative effectiveness scores, is the project that I’m the most excited about. So there are a host of future papers that the team that’s worked so hard on getting ISLES in in its current form are excited to write about. So one is teaming up with a few other scholars to write a best practices guide on how to think about using automated computational tools and large language models in legislative research.
In terms of the substantive side of things we want to be able to answer with the issue-specific scores, we’re really interested in identifying issue generalists and issue specialists, and identifying avenues in both camps to facilitate broader effective lawmaking. And whether or not there are important patterns based on legislative subgroups, whether that is gender, race, past occupation, majority party status, even party ownership in terms of issue areas, that might enlighten how we think about policymaking at the subnational level by issue area.
We’re also interested in whether or not some issue areas are better conduits for bipartisan lawmaking, and not only are just these perhaps less salient issue areas easier to be a cross-party lawmaker in, but whether or not building a reputation as a bipartisan lawmaker in one issue area further enables that lawmaker to be more successful at being a bipartisan legislator in other issue areas, particularly ones that we might deem as more of a party-owned issue area space or an issue area where we generally see less bipartisanship on average. We’re also interested in a policy diffusion paper, so using our issue area scores to understand policy transfer and policy learning across states beyond just legislator’s own activities in those spaces, just to see if there is space for a policy diffusion paper to come out of this work.
Moving into Congress though, I am working with a collaborator at the University of Cincinnati, Jacob Lawless, on what is our first R package that we’ve developed, along with a public repository of congressional bipartisanship scores from 1983 to present. And so, we have documented bipartisanship scores in the aggregate, so these are across all issue areas for both the U.S. House and the U.S. Senate, as well as across 34 policy issue areas, as labeled by the CRS. And we are in this process right now of releasing these data publicly through an R package called By Party, which will enable those who are familiar with R to immediately access our scores, both weighted and unweighted versions of those, depending on their own research goals, as well as a host of crosswalks that we have compiled to make merging with our dataset and other congressional datasets an easier feat. And then, we’ve included some co-variates that we think might be of interest to those scholars.
This will all be available on the Portman Center for Policy Solutions website in the coming months, and then, of course, the R package is hosted on my personal site. So we’re excited to see how others are going to use these data in substantive papers to come.
Matt Grossmann: There’s a lot more to learn. The Science of Politics is available biweekly from the Niskanen Center, and I’m your host, Matt Grossman. If you liked this discussion, here are the episodes you should check out next, all linked on our website. Can state politicians be held accountable to the public? Legislators are raising money instead of making policy. Are divided governments the cause of delays and shutdowns? Is democracy declining in the American states? And do congressional committees still make policy? Thanks to Mackenzie Dobson for joining me. Please check out her work and then listen in next time.
