“26% think that the sun goes around the Earth. For these people, talking to a chatbot is likely to leave them with a MORE accurate view of reality.”
This is a great example of a common phenomenon that needs a name. Does it already have a name in the literature? Similarly, Sesame Street didn’t have much effect on the literacy of children of parents who talked to their kids a lot, read to them every night, etc. (Watching TV might have even been mildly negative for some of these kids, as the real-world alternative was available and better.) But it had a powerful positive effect on kids whose parents hardly spoke to them, didn’t read to them, etc. This effect – negative at the top, neutral in the middle positive at the bottom – tends to wash out to nothing statistically. But it’s actually having lots of different effects! And arguments about it are extremely frustrating, because people can always just point at one end of the distribution, and ignore the other.
Anyway, great post, as ever. Full of juicy informational goodness.
Good point. And the same things goes for pills, programs, etc.––if it helps some people but makes other people worse off, it looks ineffective on average, when in fact it's both harmful and helpful, depending on who you are. Right now, the main way we find phenomena like this is by studying categories that we already know: does the treatment effect differ by gender, by whether you have Gene X or not, etc. But this relies on the hope that we've carved up the world with the right categories, which we almost certainly have not.
Slime Mold Time Mold and I were kicking around ideas of how to capture this kind of thing, but then we got busy with other things. Maybe it's time to take it back off the shelf...
Let's also remember the "Lizardman Constant", which is not typically as high as 26% but is certainly not a trivial amount, either. The following three paragraphs are a quote from the Substack post linked at the bottom of this comment:
Jensen (2013) asked in a survey of Americans “Do you believe that shape-shifting reptilian people control our world by taking on human form and gaining political power to manipulate our societies, or not?”. 4% of respondents answered yes, leading to headlines like this in the Atlantic “12 Million Americans Believe Lizard People Run Our Country”
An additional 7% answered “not sure”, which if you take it at face value means 21 million Americans consider it possible that lizard people run the USA.
Blogger and nerd-royalty Scott Alexander coined the term “Lizardman Constant” as a handle to equip us with some scepticism around reports like these. The idea is that all survey questions will get a baseline level of endorsement, regardless of how ridiculous they are...
Loved this post, thanks. #19 - apologies for being pedantic but (writing from a country that has a monarch to one that theoretically doesn't have one) 'reigning' is what kings do and 'reining in' is how you restrain an unruly horse or a thuggish agent.
Re #11 - Using the simplest measure of evaluating Spotify's spin, the number isn't 13,800 *artists* getting paid more that $100k, it's 13,800 rightsholders. Meaning for 13,800 artists' work, Spotify pays that much, but that payout then gets split based on the label/distributor/songwriter/artist split. They also seem to switch between "artist" and "rightsholder" in that Loud and Clear site in a way that obscures where and how the money actually flows. And they argue they are increasing the revenue pool for royalties, but if that is accomplished by including more artists without any increase in payment from users then the growth is essentially from an increasingly diluted, commoditized pool of music. It might still be true that Spotify does increase payouts to artists, and that complaints about streaming revenues are motivated more by philosophical objections or perceived lack of fairness in payouts, but that report still seems like it chooses the noisy way to prove their point than the good way. Which, I've read, means they probably couldn't prove it the good way to begin with.
This was wonderful! I followed many of the links and learned so many interesting things (of course everything can be traced back to The Simpsons). Your wife's article in The Atlantic was interesting and potentially helpful.
“A computer scientist named Tuhin Chakrabarty claims that the empty metaphors that plague the prize-winning story—and that plague all AI writing—often come from human-written fanfiction. For instance, the rare phrase ‘the patience of a reptile,’ which appears in the allegedly AI-generated story, also occurs in this fanfiction based on the Japanese anime/manga Naruto.”
I’ve been seeing bits of writing lately that seem stuck in a meme-y, 2010s Tumblr register—“choose your fighter,” “just three XYZ in a trench coat,” etc. Publishing a book traditionally takes too long to risk use language that might get uncool that quickly, but fan fiction gets published instantly, so you can say whatever you want. If only I had known how much power I had as a teen writing bad fiction on the internet in the 2010s
The blogger Linch claims that Pope Leo’s first encyclical, which is about AI, was substantially written by AI. I believe that it’s a mortal sin to pass off computer-generated text as your own, so if this is true, I hope the pontiff confesses and repents."
Thanks for the shoutout! Quick clarification re: framing. All we can observe externally is traces of possible (As my article argues, imo *highly probable*) AI use in different sections. We can't observe *who* used AI, if the essay is "written" by multiple people.
Because papal encyclicals tend to be multi-person projects with dozens of writers, anybody or group of people within the Vatican could've contributed the AI writing. My best guess is that Pope Leo himself was not involved and it's rather some other senior official(s) at the Vatican.
(You didn't explicitly say the Pope himself used AI but I think most people reading your summary might get the wrong impression of what I claimed)
Per Pope-Plexity: I don’t know whether to laugh or be offended on the pontiff’s behalf. BUT, it does remind me of the first time I taught Roman history to 3rd graders. When I introduced the person of Caesar, several of the kids professed to being amazed and amused that he had been named after the pizza chain, Little Caesars. Similarly, I think Mr. Linch also has things reversed, a gentleman of the pontiff’s generation who first studied with the Augustinian Fathers, then attended Villanova where he studied Mathematics and Philosophy, completed his theological training at the Catholic Theological Union in Chicago and then rounded it all off by studying canon law in Rome at the Pontifical University of St. Thomas Aquinas hardly needs help writing! I think, like my old 8 year old students, Mr. Linch has things reversed e.g., the machines have been trained, among other things, on the libraries that the Pope and his peers studied before the machines ever existed. Nothing against Claude and its abilities but in a room with the Pope and Claude sitting at a desk, it would be the pontiff teaching Claude how to think not the other way around.
Turning down $175,000 because three days at Yale would cost more than the money is worth. Not because she doesn’t need the money. Because she needs the time more. That is the most insane description of what serious writing actually costs, not in hours logged, but in everything else you refuse in order to keep the hours clear. In peace
#5 - this article from New York Magazine (5/15/26) https://archive.is/3ixJc talks about how social media is a lot of subtle advertising, whether in music or other areas
Cases like McHugh's are unsettling in the best way, because they show how thin the line is between who we are and a particular arrangement of tissue. As a biologist I read it as a reminder that personality is a phenotype too, one a stroke can rewrite. Does a story like this make you think the creative drive was always latent in him, or that the injury genuinely built something new?
Each of these stories turns on a boundary where a system flips — a physiological limit, a misread signal, a cultural rule. My work lives at the same scale of mechanism, just stretched across physics, biology, and narrative, so I always pay attention when small boundaries show their teeth.
“26% think that the sun goes around the Earth. For these people, talking to a chatbot is likely to leave them with a MORE accurate view of reality.”
This is a great example of a common phenomenon that needs a name. Does it already have a name in the literature? Similarly, Sesame Street didn’t have much effect on the literacy of children of parents who talked to their kids a lot, read to them every night, etc. (Watching TV might have even been mildly negative for some of these kids, as the real-world alternative was available and better.) But it had a powerful positive effect on kids whose parents hardly spoke to them, didn’t read to them, etc. This effect – negative at the top, neutral in the middle positive at the bottom – tends to wash out to nothing statistically. But it’s actually having lots of different effects! And arguments about it are extremely frustrating, because people can always just point at one end of the distribution, and ignore the other.
Anyway, great post, as ever. Full of juicy informational goodness.
Good point. And the same things goes for pills, programs, etc.––if it helps some people but makes other people worse off, it looks ineffective on average, when in fact it's both harmful and helpful, depending on who you are. Right now, the main way we find phenomena like this is by studying categories that we already know: does the treatment effect differ by gender, by whether you have Gene X or not, etc. But this relies on the hope that we've carved up the world with the right categories, which we almost certainly have not.
Slime Mold Time Mold and I were kicking around ideas of how to capture this kind of thing, but then we got busy with other things. Maybe it's time to take it back off the shelf...
Let's also remember the "Lizardman Constant", which is not typically as high as 26% but is certainly not a trivial amount, either. The following three paragraphs are a quote from the Substack post linked at the bottom of this comment:
Jensen (2013) asked in a survey of Americans “Do you believe that shape-shifting reptilian people control our world by taking on human form and gaining political power to manipulate our societies, or not?”. 4% of respondents answered yes, leading to headlines like this in the Atlantic “12 Million Americans Believe Lizard People Run Our Country”
An additional 7% answered “not sure”, which if you take it at face value means 21 million Americans consider it possible that lizard people run the USA.
Blogger and nerd-royalty Scott Alexander coined the term “Lizardman Constant” as a handle to equip us with some scepticism around reports like these. The idea is that all survey questions will get a baseline level of endorsement, regardless of how ridiculous they are...
https://tomstafford.substack.com/p/the-lizardman-constant
This will work if the LLM proprietor doesn't have an axe to grind. Like a certain wealthy, Robert Heineken fan, who is also a very strange man
Loved this post, thanks. #19 - apologies for being pedantic but (writing from a country that has a monarch to one that theoretically doesn't have one) 'reigning' is what kings do and 'reining in' is how you restrain an unruly horse or a thuggish agent.
mortifying! thanks for the catch, fixed
Re #11 - Using the simplest measure of evaluating Spotify's spin, the number isn't 13,800 *artists* getting paid more that $100k, it's 13,800 rightsholders. Meaning for 13,800 artists' work, Spotify pays that much, but that payout then gets split based on the label/distributor/songwriter/artist split. They also seem to switch between "artist" and "rightsholder" in that Loud and Clear site in a way that obscures where and how the money actually flows. And they argue they are increasing the revenue pool for royalties, but if that is accomplished by including more artists without any increase in payment from users then the growth is essentially from an increasingly diluted, commoditized pool of music. It might still be true that Spotify does increase payouts to artists, and that complaints about streaming revenues are motivated more by philosophical objections or perceived lack of fairness in payouts, but that report still seems like it chooses the noisy way to prove their point than the good way. Which, I've read, means they probably couldn't prove it the good way to begin with.
This is helpful context, thank you!
#12 DeWitt's Last Samurai is a delightful novel to read.
It was my absolute favourite book as a younger chap! I loved reading #12
Will Sheff (of Okkervil River) has a song about Tommy McHugh!
This was wonderful! I followed many of the links and learned so many interesting things (of course everything can be traced back to The Simpsons). Your wife's article in The Atlantic was interesting and potentially helpful.
“A computer scientist named Tuhin Chakrabarty claims that the empty metaphors that plague the prize-winning story—and that plague all AI writing—often come from human-written fanfiction. For instance, the rare phrase ‘the patience of a reptile,’ which appears in the allegedly AI-generated story, also occurs in this fanfiction based on the Japanese anime/manga Naruto.”
I’ve been seeing bits of writing lately that seem stuck in a meme-y, 2010s Tumblr register—“choose your fighter,” “just three XYZ in a trench coat,” etc. Publishing a book traditionally takes too long to risk use language that might get uncool that quickly, but fan fiction gets published instantly, so you can say whatever you want. If only I had known how much power I had as a teen writing bad fiction on the internet in the 2010s
It would be very funny if all of our early internet teenage personas lived on forever inside the machines.
Thanks for the shoutout.
"(2) POPE-PLEXITY
The blogger Linch claims that Pope Leo’s first encyclical, which is about AI, was substantially written by AI. I believe that it’s a mortal sin to pass off computer-generated text as your own, so if this is true, I hope the pontiff confesses and repents."
Thanks for the shoutout! Quick clarification re: framing. All we can observe externally is traces of possible (As my article argues, imo *highly probable*) AI use in different sections. We can't observe *who* used AI, if the essay is "written" by multiple people.
Because papal encyclicals tend to be multi-person projects with dozens of writers, anybody or group of people within the Vatican could've contributed the AI writing. My best guess is that Pope Leo himself was not involved and it's rather some other senior official(s) at the Vatican.
(You didn't explicitly say the Pope himself used AI but I think most people reading your summary might get the wrong impression of what I claimed)
Per Pope-Plexity: I don’t know whether to laugh or be offended on the pontiff’s behalf. BUT, it does remind me of the first time I taught Roman history to 3rd graders. When I introduced the person of Caesar, several of the kids professed to being amazed and amused that he had been named after the pizza chain, Little Caesars. Similarly, I think Mr. Linch also has things reversed, a gentleman of the pontiff’s generation who first studied with the Augustinian Fathers, then attended Villanova where he studied Mathematics and Philosophy, completed his theological training at the Catholic Theological Union in Chicago and then rounded it all off by studying canon law in Rome at the Pontifical University of St. Thomas Aquinas hardly needs help writing! I think, like my old 8 year old students, Mr. Linch has things reversed e.g., the machines have been trained, among other things, on the libraries that the Pope and his peers studied before the machines ever existed. Nothing against Claude and its abilities but in a room with the Pope and Claude sitting at a desk, it would be the pontiff teaching Claude how to think not the other way around.
Turning down $175,000 because three days at Yale would cost more than the money is worth. Not because she doesn’t need the money. Because she needs the time more. That is the most insane description of what serious writing actually costs, not in hours logged, but in everything else you refuse in order to keep the hours clear. In peace
This is wonderful, but honestly? I'd have read it just to see the bearded tomb stone. Priceless.
#5 - this article from New York Magazine (5/15/26) https://archive.is/3ixJc talks about how social media is a lot of subtle advertising, whether in music or other areas
Cases like McHugh's are unsettling in the best way, because they show how thin the line is between who we are and a particular arrangement of tissue. As a biologist I read it as a reminder that personality is a phenotype too, one a stroke can rewrite. Does a story like this make you think the creative drive was always latent in him, or that the injury genuinely built something new?
Each of these stories turns on a boundary where a system flips — a physiological limit, a misread signal, a cultural rule. My work lives at the same scale of mechanism, just stretched across physics, biology, and narrative, so I always pay attention when small boundaries show their teeth.
“While doing a poo.” Seriously, who speaks like that?