Shame them, shun them, ban them, beat them!

OR: crock pots and lightning bolts

photo cred: my dad

1.

Say what you will about the Union of Soviet Socialist Republics, its 1936 constitution was a banger.

It guaranteed freedom of speech, the press, assembly, and protest. It extended equal rights to all citizens, regardless of race or gender. It shortened the working day to seven hours, affirmed “the right to rest and leisure”, and offered free education and free health care to all, including a “wide network of health resorts for the working people.” You gotta admit this is a lot better than certain other constitutions that, say, count slaves as three-fifths of a person.

In the two years after the constitution was adopted, however, Stalin purged something like a million people. Eighteen million Soviet citizens would be forced into gulag camps and colonies over the next three decades. This was all clearly in violation of Articles 103, 111, and 127, and many others besides. How could one of the greatest tragedies in human history happen when it was very explicitly not allowed?

The uncomfortable answer is that a critical mass of people found prisons and purges palatable. We say that “Stalin” did all these things as if held the gun himself¹, but obviously you can’t run a gulag without guards, secretaries, accountants, engineers, architects, doctors, drivers, quartermasters, mid-level managers, kangaroo court judges, and all manner of flunkies, patsies, and stool pigeons. Apparently, hundreds of thousands of regular people were willing to commit their own personal portion of an atrocity.

The lesson here is obvious: rules don’t matter unless people act like they matter. Writing down laws does not endow them with physical force or psychic potency. We all know this. We all believe this.

So why don’t we act like it?

2.

Ever since the replication crisis began over a decade ago, most folks have agreed that the problem is the laxness of our rules, and that the solution is to tighten them. We should mandate replication, preregistration, public data, bigger studies and tinier p-values. The more we can reduce researcher degrees of freedom, the thinking goes, the better our science will be.

Vindicating this theory, a big group of researchers published a paper in 2023 showing that it’s possible to achieve high rates of replicability as long as you follow a set of “rigor-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency.”

One year later, the paper was retracted because it...failed to follow those rigor-enhancing practices. The journal editors claim the authors were not transparent about their methods, did not abide by their own preregistration, and cherry-picked their results.²

Clearly, we’re not missing the right regulations. We’re missing the right motivations. If you want to discover true things about the world, you’ll be interested in the guidelines that help you do that, and you’ll be thankful to the people who develop them. That’s how the replication crisis could have played out: someone demonstrates that our sample sizes are too small, and we all go, “oh wow we should make our sample sizes bigger because want to know what’s real and what’s not”.

But if you’re not actually seeking the truth, no amount of “rigor-enhancing practices” will ever cause you to find it. That’s why our revolution in scientific regulation has mostly failed. We require researchers who conduct clinical trials to post the results on a public website, but only 45% of them do. We tell people to specify their primary outcomes beforehand, but if their studies don’t work as planned, they just sneak in different analyses—one study on anesthesiology experiments found that 92%(!) of them did this. We make researchers end their papers by saying “data available on request” and then only 17% of them actually make their data available on request.³

You can’t turn a cheat into a scientist by making a rule against cheating. The most important “rigor-enhancing practice” is caring about getting things right, and without that, nothing else matters.

3.

Every couple will eventually have some version of the “Let’s Make a Rule” fight, where they try to solve some interpersonal issue through legislation. “You think I don’t take enough interest in your life, so let’s make a rule: I have to ask you three things about your day before I start telling you about mine.” The theory behind the Let’s Make a Rule fight is that we could live in harmony with one another if we could just compile all of our expectations into one big Google Doc.

The Let’s Make a Rule fight never leads to a satisfying conclusion because nobody actually wants their partner to follow the rules. They want their partner to care. Being asked “How was your day, dear?” through gritted teeth because that’s what our Relationship Handbook says to do is probably worse than not being asked at all.

You want your partner to realize that your preferences are not silly affectations that can be belittled, ignored, or disputed until they go away, that they are, in fact, load-bearing parts of your personality, and to reject them is to reject you. In return, you have to realize that some of your preferences are more malleable than you thought, that maybe they don’t all have to be foundational to your sense of self, and that some of them can be bent or jettisoned in the interests of coexistence.

This is the work of love, and it takes a lifetime. You can’t speedrun it by filling out a spreadsheet or signing a contract. The frictions of a lifelong relationship can be made intelligible—that is, understandable to the people involved, but they cannot be made legible—that is, understandable to everyone else.

The best couples I know have all sorts of arrangements and accommodations that make zero sense to me but perfect sense to them, and that’s exactly why they work well together. A successful relationship is nothing more than a package of haphazard remedies and rickety fixes that people would only ever devise and maintain when they really, really want to be together.

4.

Sometimes police officers break the rules, and the solution is obvious: we just need to police the police.

If you strap a camera to every officer’s chest and record everything they do, then they won’t misbehave in the first place. And even if they do, we can discipline or fire them. This is the most foolproof plan of all time, supported by an unheard-of 89% of Americans. Police departments all over the country agreed—now every single large precinct has body cameras, and a majority of smaller precincts do too. The technology is out there and the data is in; how is our foolproof plan working out?

Not well. According to a 2022 Department of Justice report, body cams might not have any meaningful effect on police behavior. One meta-analysis found a moderate drop in citizen complaints, but didn’t find any difference on any other outcomes: use of force, assaults against police officers, number of incident reports, etc. Another meta-analysis finds no evidence of downstream outcomes like conviction rates. The DoJ warns that we need more research, but this is enough to rule out gigantic effects. If body cams do anything at all, they probably don’t do much.

Maybe our plan just didn’t go far enough. Bad cops can simply turn their cameras off or cause the footage to mysteriously go missing. Even if the video exists, it’s only one version of events, and it’s from the officer’s point of view. Maybe what we really need is a swarm of drones to follow every police officer, recording every action from every angle, and they never shut off and their footage automatically gets uploaded to a public database.

Or maybe we misdiagnosed the problem in the first place. We assumed that the justice system was eager to hold bad cops accountable and that all it was missing was the necessary evidence. It turns out the justice system is actually rather ambivalent about holding bad cops accountable, and so it handles additional evidence as halfheartedly as it handled all of the evidence it already had. A camera can allow you to see, but it can’t make you look.

5.

Everywhere—from politics to science, from love to law—we are constantly asking: how can we get people to do the right thing?

And the answer comes to us so naturally: prod them, push them, shame them, shun them, ban them, beat them!

But every time we try to do this, we run into the same paradox that the Roman poet Juvenal pointed out 2,000 years ago: quis custodiet ipsos custodes? Who watches the watchmen?⁴ If you try to police the bad actors, you will soon find that some of your police are bad actors as well. And if your solution to bad police is to police the police, then who will police the police that police the police?

At some point, there has to be an Unwatched Watchman, someone who will do the right thing not because they are forced to, but because they want to. Instead of asking, “How we can get people to do the right thing,” we should ask, “How can we get people to want the right thing?”

6.

I think there are only two ways that people’s desires change: they either get put in a crock pot, or they get hit with a lightning bolt. That is, people either change so slowly that they never notice it, or they change so suddenly that they never forget it. There is no in between.

7.

Here’s what a lighting bolt looks like.

One day in seventh grade, my teacher—let’s call him Mr. L—told me and two other kids to put up the American flag outside the school. While another boy undid the knot on the flag pole, I absent-mindedly draped the flag over my shoulders like a cape, as I had seen Olympic athletes do on TV.

Mr. L, who was a Vietnam War veteran, apparently saw me do this, and when we came back, he gave me a dressing down in front of the whole class. I had disrespected the flag of our country by using it as a costume, he said. My behavior was shameful and inexcusable, and I must sit at my desk silently and meditate on the wrongness of my action.

That might sound like nothing, but I was a Good Kid. I never got in real trouble in school before, and so this experience was new and humiliating. In that moment, I silently vowed to never, ever be like Mr. L. As far as I was concerned, the Stars and Stripes were a symbol of unconscionable cruelty against the most innocent and well-meaning of people, namely, me.

Mr. L got me to do the right thing (at least the thing he thought was right)—I never draped myself in the flag again. But he didn’t get me to want the right thing. Before Mr. L, I told people I wanted to be a soldier when I grew up. After Mr. L, I told people I wanted to be a writer. One cruel remark undid a lifetime of saying the Pledge of Allegiance.

8.

Here’s what a crock pot looks like.

Most scientists learn their trade from scratch in young adulthood. They do it not so much by sitting through lectures, but by hanging around in labs, watching what other people do, and doing rinky-dink versions of their own experiments that get critiqued by people with more experience. Therefore, the kind of scientist you become depends a lot on the crock pot you steep in.

Many of those crock pots, it turns out, are filled with rancid juices. If your professor hands you a dataset and tells you to “Go on a fishing expedition for something—anything—interesting” and then praises you when you return with significant p-values, of course you’re going to internalize the wrong lessons. If you hang around people who bury their inconvenient findings, or who stuff their reference lists with self-citations, or who take other people’s work without crediting them, you’re going to grow into that kind of person yourself. You won’t feel like you’re becoming a villain; you’ll feel like you’re becoming an adult.

Once you’ve spent a few years in the wrong crock pot, you’re cooked, both literally and figuratively. No amount of legislation can turn you raw again or re-cook you the right way. You are immune to any “rigor-enhancing practices” because you never learned to care about rigor. When you encounter a rule like “No excessive self-citations!”, it barely even registers, because of course you don’t think your self-citations are excessive. You think they’re normal!

As Richard Feynman once put it:

But this long history of learning how to not fool ourselves—of having utter scientific integrity—is, I’m sorry to say, something that we haven’t specifically included in any particular course that I know of. We just hope you’ve caught on by osmosis.

I think Feynman was right. The most important lessons—in science, or in anything—are not learned. They are absorbed. And if you’re steeping in dirty water, you’ll absorb the wrong lessons, and then it’s almost impossible to get them back out again.

9.

When you think in terms of crock pots and lightning bolts, you may or may not come up with the right theory of change. But you’ll at least realize that you need a theory of change.

We don’t seem to do this by default. Instead, we assume that other humans are lumps of Silly Putty that can be stretched or compressed to fit whatever container is convenient. Change is easy—who needs a theory?

As long as that’s your model of human nature, you’ll believe that winning hearts and minds is just a matter of updating the bylaws, and you’ll keep wondering why it never seems to work.

10.

I understand why people are obsessed with rules: it’s fun to pretend that we can control the future. It’s pleasing to think all this messiness is temporary, and that once we articulate what each person should and shouldn’t do, then finally we can live in peace with each other.

And yet, when the future arrives, the people in it always end up doing whatever they want. The only way to have some influence over the future, then, is to have some influence over those wants.

It’s hard to build the right crock pots; it’s hard to fire the right lightning bolts. It’s much easier to edit Google Docs full of rules, to install more security cameras, and to enumerate ever-longer lists of rigor-enhancing practices. And as long as we do that, we can expect more broken hearts, bad cops, and retracted papers. You can outlaw war and you can mandate love, you can ban falsehood and demand truth, but you won’t get anything you want unless you can convince other people to want it, too.

Anyway, that’s how my day was, dear! How was yours?

1. Lenin could have done it single-handedly, of course, because according to Soviet TV he was a magic mushroom.

2. The authors and the critics/editors disagree on what happened, exactly. The former say their paper was retracted because they included one misleading sentence, and the latter say it was because the entire paper was based on ad hoc analyses that were implied to be preregistered but really weren’t.

Regardless, the authors eventually explained that the project wasn’t originally about enhancing rigor per se—it was conceived ten years ago to investigate the “decline effect”, where previously robust findings mysteriously shrink or disappear in subsequent investigations. Some people on the team apparently believed in the standard explanations for this phenomenon: sampling error, selective reporting, questionable research practices, etc. But others suspected voodoo was at play: maybe the mere act of observing a phenomenon causes it to wither, for spooky reasons beyond our understanding and perhaps beyond the laws of physics. Suffice it to say, this backstory didn’t make it into the manuscript.

3. By the way, that response rate was no different between journals with mandatory data-sharing statements and those without.

4. This comes from Juvenal’s Satire VI, which sounds a lot like a standup routine from the 1980s:

SatVI:25-59 You’re Mad To Marry!

SatVI:200-230 The Way They Lord It Over You!

SatVI:286-313 What Brought All This About?

SatVI:Ox1-34 and 346-379 And Those Eunuchs!

SatVI:592-661 It’s Tragic!

See also this synopsis of the introduction, via Wikipedia:

lines 6.25-37 – Are you preparing to get married, Postumus, in this day and age, when you could just commit suicide or sleep with a boy?