Does Any Of This Work?

Political tools and minimal campaign effects

Photo by Emily Morter on Unsplash

One thing you can learn from the HGL Political Tech report is that Democrats are so, so good at campaigning. Every cycle, we manage to sink more and more time, money, and staffing into the mechanics of campaigns, while also building new tools and new organizations. The number of people employed in this industry continues to grow, as do the numbers of contacts and ads run. As far as the mechanics go, we have got it down.

Since the start of this week, I have been thinking about something that Micah Sifrey brought up.

“While Democrats believe they were able to reach out to 70% of registered voters in battleground states and more than half elsewhere, “actual engagement dropped significantly—especially among younger voters and key demographics that we had previously engaged more effectively.” There’s a logical conclusion that could resolve this conundrum, which is that the meta-machine Dems have learned to build using texts, calls, and door-to-door visits at massive scale is delivering declining, zero, or even negative returns.”

Which is to say, does any of this work????

And further, does it even make sense to expect that the impact of campaigning would infinitely scale with effort?

Personally, I land on 1. Probably some and 2. Absolutely no.

It is clear from the report that Democrats are doing more than ever before. In fact, every cycle I’ve worked in this industry, we’ve done more than ever before. Some of this is due to the rise of completely new methods of campaigning (like SMS, and relational) that enable far more contacts with voters than were otherwise possible. Some of it is just massive scaling of traditional programs. Can we know if this is working?

Testing and Its Limits

One way to measure the impact of your political intervention of choice is the RCT. This lets you hold out a control group and (post-election, or in survey) get an estimate of how your intervention impacted people. It also usually lets you back out the much-hallowed Cost Per Vote metric, which tells you for every N dollars you dump into your intervention, you will get Y additional votes out. This is all good and well if you are the only intervention in town, but that is never, ever the case. Every election, you are going to be operating in a complicated environment where you are not the only one talking to voters. Do our RCT estimates hold up in that situation? What do they tell us about the *aggregate* impact of our programs?

For TV ads, at least, we have a sense. There’s a Sides, Vavrek, and Warshaw paper on TV ads that includes the following:

“Assuming that the effects of ads are linear, our findings imply that moving from three standard deviations below the average advertising advantage to three standard deviations above the average (a 6 standard deviation shift) within border pairs would lead to a 0.5-point change in two-party vote share”

So a massive, enormous ad advantage would give you....half a point in voteshare. This is better than not having that half a point, elections can turn on half a point, but it’s not a ton of return for your 11 billion dollars. This doesn’t override the results of RCTs, of course, and it’s highly compatible for each ad to have a small effect and the aggregate of all advertising to also have a small effect. It just puts up some warnings that we shouldn’t expect the effects of our programs to simply add on top of each other.

What about our other programs? We have less information. Pretty much none of this is public, and most of it is RCTs. I have no idea what the aggregate impact of all the texts we sent in 2024 were. There’s just not research on it. I can make an informed guess, based on the findings that traditional advertising is still unusually effective compared to other strategies, and even in the maximum case that only gets to less than a percent of vote share changed. It is not a very hopeful guess.

None of these findings tell you where the point of diminishing returns is. If one text gets you 1/10,000th of a vote, does 200,000 texts get you 20 votes? Does this scale with 20 million texts? Probably not, but we don’t know where that drop off is.

Multiple Problems At Once

There are two separate anxieties running through my comments here. One is that a lot of our specific, individual programs just don’t work at all, and that we’re lighting money on fire. The other is that *in aggregate* we are getting no more than an average campaign benefit from all this spend and work.

The first problem is resolvable by really good testing and monitoring of programs. If I could force every single entity involved in Dem politics to agree on some control geographies and then produce reports of their contacts and the spend they used to get those contacts, and had infinite money to run analysis, I might be able to tell you if those specific programs are “working”. I could look at the people they contacted and find a higher rate of turnout, or a higher Democratic vote share in the non-holdout geographies. In the absence of dictatorial power, I can look at people’s RCTs and survey experiments and get an okay sense of if their program works. (This is less satisfying than having dictatorial power, but probably easier)

The aggregate question I really can’t answer. I can squint at the differences between battlegrounds and non-battleground states for a potential campaign effect, I can look at overall high turnout as a comforting sign, but I can’t know. Depressingly, I can certainly say that it hasn’t assured us wins. I am perfectly comfortable being in a profession where all your hard work doesn’t matter at all if the national environment meant you were going to lose by >2 points. I am less comfortable pouring increasingly huge amounts of money and time into that effort for no additional return.

“The Long Term”

One response to these anxieties, and to questions about campaign effectiveness, has been a push for more “power building” or “organizing” or “long term work”. This probably sounds arcane if you don’t work here. Here’s an example from Arena just to demonstrate what I’m talking about.

The specifics vary a lot, but generally, the critique being leveled is that campaigns are too short term and building a consistent winning coalition requires a return to the sort of civic infrastructure we had before our local institutions all imploded under the weight of the internet and individual atomization. This means that our RCT-driven measurement-obsessed industry is missing the real opportunities to ensure we win, which would require us to put money into stuff whose effectiveness cannot be measured in an RCT.

Which like. Yeah, that would be nice! Big if true!

However. It has been long enough now that we ought to be able to see some examples of this working. We have poured money into “power-building” organizations for at least 2 cycles now. Has civic infrastructure in battleground states recovered? Are those organizations even *around* anymore? Not that I have seen. I would love to be wrong- please, feel free to show me your civic engagement org that has existed for >4 years and has people voluntarily turning up to meetings in off years. But what I have observed is that this is mostly another way for us to light money on fire, just one that happens to make you feel a bit better about yourself.

The impulse to organizing/power building is, imo, a longing for a time when the Democratic party was a functional institution, with local branches that served a significant community function. This is a yearning for a time before our culture became Like This. It sounds great, but “simply undo the direction of our entire culture” is not a compelling pitch for a way to win elections. Plus, this sort of work is doubly subject to the problems of a rapid-turnover industry that is constantly running out of money. No one really cares if the org running ads has a different name than the org that did it 2 years ago. If you set up community meetings and got volunteers engaged, it’s a bad look when a completely different organization tries to do that again after you vanish.

So, yes, I would love us to live in a different and better political culture. I would love local civic institutions to be a thing again. I strongly believe that change would need to come from the Democratic party itself, rather than the shifting constellation of associated organizations. And I don’t see much hope of that happening when, as usual, the whole DNC evaporates every 4 years. No amount of funding one off local groups can fix that.

Horse Blinders

Staring at the mechanics of campaigning for a long time gives you a peculiar sort of blindness. When you are Just Some Guy responsible for a tiny slice of an overall picture, you have to work with what you’re given. You can spend all your time hyper optimizing the appearance of a button on a donation page, but you don’t get any sort of say in what the candidate does. Of course not! This is way above your pay grade!

It makes sense that this would lead you to exclude the most obvious factor in winning or losing a campaign: The candidate’s policy positions. Reading things like the HGL report, you might believe that Harris’ policies came down from a mystical on-high in an unchangeable format. They are utterly outside the domain of the texters and the emailers and the ad-testers. They certainly are not to be brought up as something that went wrong (doing this is the benefit over being an internet loudmouth). This makes discussions of the state of political tech just a bit....awkward. It seems like we are, in fact, very good at running campaigns! Perhaps something else went wrong, but we can’t really discuss that.

I am a data person, so I am always going to turn to polls. And looking at those polls, it seems pretty reasonable that a bigger factor in the Harris loss than the optimization of our phone calls might have been the fact that voters really hate our position on immigration. Not to pick on immigration, this is true of a variety of policy areas, including climate, guns, trans issues, and so on. Voters disagree with democrats on plenty of things. Which is not *inherently* bad, but is a factor if you are, for example, struggling to win elections.

All of this is to say that we as a party have bigger levers to pull than the ones available to folks executing your campaign tactics. Given the limitations of campaign effects, it is extremely possible that just getting even better at tactics will not be enough. I’m not thrilled about doing more and more forever and ignoring the wide open options of bringing ourselves more in line with the voters. There’s a whole separate argument to be had about how exactly to do that, but I think it’s relevant to the tools conversation to note that that argument *exists* and *should be had*.

And to any anxious staffers, chill out, it’s fine, your poorly constructed call list is not the reason we lost. You’re fine. There are bigger things at play than campaign tactics.