Uyghur Topped a Count of State Media in AI Training Data. Its Biggest Source Was Writing Kazakh

Which voices does an AI hear when it learns a language with little text online? Researchers at the University of Pennsylvania and four other institutions went looking for one kind, state media, in the web pages used to train language models. Their dataset, released on 18 October, matches 640 outlets owned, run or heavily influenced by governments against FineWeb and FineWeb-2, two open collections of web text that are widely used for training. Of the 114 languages they examined, Uyghur came out on top: 17.8 per cent of all the pages filed under it come from state media, most of them, by the 2025 ratings of the watchdog State Media Monitor, from outlets under state control. Read quickly, the number seems to say something obvious about Xinjiang. The released data says something else.

The 17.8 per cent is 29,479 pages out of 165,637, and it comes from three countries. The largest share, 12,225 pages or 41.5 per cent, is from Kazinform, Kazakhstan’s national news agency. Then come 9,814 pages from TRT, Turkey’s state broadcaster, and 7,377 from Radio Free Asia, which is funded by the United States government. Not one page comes from China. Radio Free Asia counts as state-controlled only under the 2025 ratings, after the US agency that oversees it was reclassified; under the 2024 ratings, which still called it independent, the controlled share of these pages would be 33.5 per cent instead of 58.

And Kazinform is not writing Uyghur. Its pages in the collection come from the sections where it publishes Kazakh in Arabic script, the script Uyghur is written in, and the program that sorted FineWeb-2 by language filed them as Uyghur. A crude test of my own, counting the word for “and”, which is جانە in Kazakh and ۋە in Uyghur, puts 8,683 of Kazinform’s 12,225 pages on the Kazakh side and none on the Uyghur side; the rest are too short to tell. One Kazinform page begins its dispatch with “Astana. Qazaqparat”, the agency’s name in Kazakh. Across the whole Uyghur collection, the same test leans Kazakh for 46,175 pages, more than a quarter.

The Chinese state’s Uyghur, meanwhile, is there but uncounted. The same collection holds 13,498 pages from Tianshan Net, the Xinjiang news portal set up in 2001 by the regional Communist Party’s publicity department with People’s Daily Online, 7,601 of them from its Uyghur edition and 5,150 from its Kazakh one, plus 427 pages from the Uyghur edition of Xinhua and 284 from that of People’s Daily. They are missing for mundane reasons. Tianshan Net is not on the watchdog’s list, and the method apparently matches only the web addresses listed for each outlet, while the Uyghur editions of Xinhua and People’s Daily live at other addresses. Radio Free Asia’s Uyghur service is undercounted the same way: another 10,955 pages with the same layout, collected between 2019 and 2023 from what appears to be a second address of the service, were not matched.

None of this is something the authors hide. They write that their counts inherit the coverage and filtering of the collections they draw on and that the 2025 ratings are applied to every year, and they do not say whose state media fills the Uyghur pages; their summary is that in several such languages “the outlets of one or a few states supply 5–18%” of the web data. Their other headline figure rests on similar ground, as they explain themselves: in English, the share of state-media pages coming from controlled outlets rose from 39 per cent in 2013 to 93 in 2025, partly because US government broadcasters were relabeled in 2025, and partly because since 2024 independent public media have been shutting out the programs that collect pages for training, which controlled outlets have done far less.

For a big language these seams are rounding errors. For a small one, each moves thousands of pages: which outlets are on a list, which address counts as an outlet, which language a classifier thinks it is reading, which year’s label applies. The same release shows how far the pattern goes. In Tibetan, 95.7 per cent of the state-media pages are Radio Free Asia’s. In Belarusian, the second-largest source, Belsat, a channel the dataset files under Poland, has no rating at all. Ask who speaks Uyghur to a machine, and the answer depends less on the machine than on the bookkeeping that decides what a page is.

What I checked, and what I did not. I downloaded the dataset’s lists of matched pages for Uyghur, Tibetan, Belarusian and nine other languages, its table of outlets, and the Uyghur training file of FineWeb-2. From these I recomputed the Uyghur figure (29,479 of 165,637 pages, 17.80 per cent), its breakdown by outlet and country, and its share under the 2025 and 2024 ratings; my 58.3 per cent controlled share differs slightly from the paper’s 58.5, a gap I cannot explain. I counted the uncounted outlets by web address in the same file. The Kazakh test is my own rough heuristic, not a language identifier, and I make no claim about the pages it could not classify. Tianshan Net’s origins are from People’s Daily Online’s announcement of its launch in 2001; I did not confirm who owns the second Radio Free Asia address, and I did not check why Kazinform has no rating yet is counted. I did not recompute the English trend, the Russia–Ukraine case study or the paper’s experiment on retrieval, and I do not know which models, if any, were trained on this particular file. The paper is a preprint, posted on 18 October and not peer reviewed; I did not contact the authors.