AI Labelers Found Online Arguments Turning Toxic. They Were Counting “You”

Do online arguments get nastier the further down the replies you go? Researchers who look for such “laws” of conversation across millions of comments have long counted toxicity with an automatic labeler, usually Google’s Perspective API. Perspective stopped taking new requests in February and goes out of service at the end of 2026, so the next measurements will be made with large language models. Liang Yue and Eric A. Vance, statisticians at the University of Colorado Boulder, tried that on 1.25 million comments posted under English-language news articles, each already judged by a panel of crowd raters. By the raters’ majority, toxicity falls slightly as replies go deeper: 0.12 percentage points per level. Labeled by Claude Haiku 5.5, it rises by 0.93 points per level. From the first comment to the fifth level of replies, Haiku adds 4.7 points of toxicity, in a corpus where 7.6 per cent of comments are toxic. Under Claude Sonnet 5.5, tested on a 5 per cent sample, the rise is steeper still.

Most of the difference comes down to one word. The more often a comment says “you”, or your, yours, yourself, the more often Haiku calls it toxic when the raters did not. Among the comments the majority found harmless, it flags 12.1 per cent of those without the word and 50.9 per cent of those that use it four times or more. A single human rater, judged against the others, goes only from 5.9 to 10.0 per cent; Detoxify, an open classifier the authors use as a stand-in for Perspective, from 2.6 to 8.4. And deeper replies talk to someone far more often: among harmless comments, 23 per cent of those that open a thread contain a second-person word, against 61 per cent of those six or more replies down. Hold the number of “you”s fixed, and the rise in Haiku’s false alarms with depth disappears. Among the comments that four raters out of four had found harmless and Haiku flagged: “You’re lecturing a fence post.” “Dude, are you stalking him? Kinda creepy.” “Clearly, you ate lead paint chips as a child.” Rewritten as “They’re lecturing a fence post” and so on, they passed.

So far that is a correlation; then the authors changed the words. A fixed rule that turns you into they or them, and your into their, applied to the 24,116 comments in the 5 per cent sample that address someone, removes 67 per cent of Haiku’s false rise, and all of it when only comments in the same thread are compared. Paraphrases written by Sonnet to keep each comment’s meaning, tone and politeness give the same result. The edit works in reverse too: turning “they” into “you” in clearly harmless comments that had no “you” raises Haiku’s flags by about five points, at the top of a thread as far down it. Nor is this a subtle hostility that the crowd missed. Among harmless comments on which no rater cast a single toxic vote or marked an insult, Haiku’s false alarms still climb from 8.3 per cent without a second-person word to 42.9 with four or more, and raters who were shown the comment being answered, and so knew whom a reply addressed, report no rise with depth. “Read alone,” the authors write, “a comment is toxic to the LLMs partly for whom it addresses, not only for what it says.”

The obvious fixes make it worse. Showing Haiku the comment being answered, so that it can see a reply is a reply, enlarges its distortion among replies from 1.29 to 1.59 points per level. Telling it in the prompt that addressing someone is not by itself toxic raises the distortion by 0.26; letting it think at length before answering raises it by 0.83, to 2.07. All seven language models tested, the two Claude models and five open ones including Llama, Qwen and Gemma, flag harmless comments with four or more “you”s 3.3 to 5.1 times as often as those with none, when set to flag as many comments as the raters do; in the open models the effect on the depth trend is smaller, and at a fixed number of “you”s it vanishes. Haiku over-flags “you” on Reddit, on Gab and on Wikipedia talk pages too, even when the question is specifically about attacks on the person addressed. The one place it does not is in dialogues between people and chatbots.

The usual check would not have caught it. A labeler is validated by how well it agrees with human labels, but the authors prove that agreement barely constrains a law: on these comments, a labeler that agrees with the raters almost perfectly, a Cohen’s kappa of 0.9, and flags as many comments as they do could still report any depth trend from a fall of 1.30 to a rise of 1.65 points per level. Here the models’ agreement was low, below that of a single human rater, and it did not even rank them: of the seven, the one that agreed best with the raters, Sonnet asked for a score from 0 to 100, inflated the escalation most. The textbook correction for a labeler’s measured error rates more than doubles Haiku’s rising law, to 2.02. In samples of 10,000 to 100,000 comments, the size of many studies, Haiku’s verdict on whether toxicity rises with depth differs from the raters’ in 70 to 100 per cent of samples and reverses it in up to 48 per cent. Of the 74 pairs of news publications that the raters clearly rank by their share of toxic comments, Haiku puts 16 the wrong way round.

Yue and Vance do not say the published laws are wrong: Detoxify, their stand-in for Perspective, does not manufacture escalation on these comments. Their warning is about the next round. Measured with Haiku, discussions escalate, and longer threads are much more toxic: its law for thread size is 12.7 times the raters’. Their remedy is to certify the law rather than the labeler. A method they call Threadsafe uses the model’s scores to choose which comments people should label, and the human labels to correct the rest; in their simulations a thousand human labels are enough to rule out Haiku’s rising law. But the shape of the mistake is what lingers. The further down a thread, the more people answer one another directly. A machine that mistakes being spoken to for being attacked will find, in thread after thread, people turning on each other, and will write it down as a law.

What I checked, and what I did not. The authors released Haiku’s labels with the structure of the threads; the comments and the raters’ verdicts are in the public Civil Comments release. From these I recomputed the main figures: 1,251,487 comments, 7.62 per cent toxic for the raters and 22.33 per cent for Haiku, a depth trend of −0.12 points per level for the raters and +0.93 for Haiku, Haiku’s false alarms by number of second-person words (12.1, 19.7, 30.5, 39.1 and 50.9 per cent for none, one, two, three, and four or more), and the share of harmless comments that address someone at the top of a thread and six or more levels down (23 and 61 per cent). All match the paper. I also found the three quoted comments in the data: each was rated by four people, none of whom called it toxic, and Haiku flagged each one. I did not query any model myself, since the labels are the authors’ outputs, and I did not check the rewritten versions, the other forums, the prompt variants, the single rater’s figures or the proofs. The authors list their own limits: the corpora are English; Claude is the only commercial model tested; the rewritten comments were scored against the labels the raters gave the originals, and no person judged the edits; and the human labels are a majority vote with its own threshold, not ground truth. An AI assistant helped them write code and draft text. The paper is a preprint, posted on 16 October and not peer-reviewed. Perspective’s end of service is confirmed on its own FAQ page.