The Case of a Disease That Never Existed: How Fake Research Entered AI Systems

 

Almira Thunström, a physician at the University of Gothenburg, designed an experiment as elegant as it was unsettling. She invented an entirely fictitious medical condition—bixonimania, supposedly a skin disorder affecting the eyelids and linked to blue-light exposure—and uploaded two fabricated preprints to an academic repository. Within weeks, major large language models were presenting the condition to users as established medical fact. 

The experiment was deliberately constructed to be transparently fraudulent. The fictional lead author, Lazljiv Izgubljenovic, was listed as affiliated with the non-existent Asteria Horizon University in the equally imaginary Nova City, California. The acknowledgements thanked “Professor Maria Bohm at The Starfleet Academy” and cited funding from “the University of Fellowship of the Ring.” One paper explicitly stated that it had been entirely fabricated; another described its study population as “fifty made-up individuals.” Yet none of these obvious warning signs prevented the fabricated research from being absorbed into AI-generated medical knowledge. https://www.nature.com/articles/d41586-026-01100-y

Less than three weeks after the fabricated preprints appeared, major AI systems were already presenting bixonimania as a legitimate medical condition. Microsoft’s Copilot described it as “an intriguing and relatively rare condition.” Google’s Gemini advised users to consult an ophthalmologist. Perplexity AI even estimated its prevalence at one case per 90,000 people, while OpenAI’s ChatGPT offered symptom comparisons. The experiment suggests that the very architecture of academic publishing can become a vehicle for misinformation when AI systems are trained to treat these signals as proxies for credibility. More troubling still, the false research did not remain confined to AI-generated outputs. One of the fabricated preprints was later cited in a peer-reviewed article published in Cureus, part of the Springer Nature group, which described bixonimania as “an emerging form of periorbital melanosis linked to blue light exposure.” After inquiries from Nature, the journal retracted the paper on 30 March 2026—demonstrating how quickly fabricated research can move from preprint repositories to AI systems and, ultimately, into the scientific record itself.

PS - Yet the most obvious failure in this case was not the language models. It was the near-total absence of upstream verification. These papers were not sophisticated forgeries. They cited funding from “the University of Fellowship of the Ring,” listed an author affiliated with a university that did not exist, and located that institution in an imaginary city. And still, they passed through the submission process. Preprint repositories such as bioRxiv and medRxiv continue to accept submissions tied to institutions that may not exist, often with less scrutiny than a commercial web form. A basic automated check against established research registries such as the Global Research Identifier Database would have flagged “Asteria Horizon University” and “Nova City, California” in seconds. This is not a technical limitation, nor a question of cost. It is a question of institutional priorities. If platforms feeding the scientific record cannot detect references to fictional universities, imaginary cities, or funding bodies borrowed from fantasy literature, they should not be surprised when fabricated research flows downstream into AI systems, citation networks, and eventually the scientific literature itself.

Putativas elites mediáticas e a humilhação sistemática da ciência portuguesa

https://pacheco-torgal.blogspot.com/2019/12/a-maquina-de-estupidificar-os.html

Depois da indigência intelectual perpetrada pela revista Visão em 2019, que nas suas célebres "100 figuras da década" conseguiu o prodígio de incluir 29 Portugueses "notáveis", entre os quais apenas um único cientista, vide post acessível no link supra. Depois da deplorável lista do semanário Expresso de 2023, que elegeu os 100 Portugueses que "marcaram e vão marcar o país e o mundo", onde apareciam apenas 14 Académicos, quase o mesmo número de desportistas, como se o contributo de quem produz conhecimento, inovação e progresso civilizacional pudesse ser colocado no mesmo plano do de quem se limita a proporcionar espectáculo e distracção das massas. https://19-pacheco-torgal-19.blogspot.com/2023/08/a-deploravel-lista-do-expresso-que.html Eis que agora a revista Sábado, decidiu assinalar os seus 22 anos de existência com a publicação de uma nova lista, “22 figuras que estão a mudar Portugal”. 

A inclusão de três cientistas, Rui Costa, Fábio Rosa e Gonçalo Castelo-Branco, representa um avanço claro face às vergonhosas listas anteriores. Igualmente positiva é a presença dos fundadores da Tekever, da Sword Health e da Feedzai, três tecnológicas portuguesas com projeção internacional genuína e bastante meritória, cujos líderes, em particular o CEO da Sword Health, elogiei em diversos posts, incluindo num com elevado potencial para envergonhar alemães  https://pachecotorgal.com/2025/11/30/germanys-underperformance-and-the-rising-ai-obsession-among-portuguese-researchers/

Na lista há ainda dois políticos, uma reitora, um médico, dois cantores, um artista visual, um escritor, uma humorista, uma chefe de cozinha, um empresário de restauração e cinco desportistas. O que significa que uma vez mais, a equação do país revela-se a mesma, a ciência que move o mundo continua a valer menos do que o entretenimento que distrai o povo. O único registo positivo é o facto de desta vez haver apenas um único representante modalidade do pontapé e da cabeçada no esférico. Ou melhor dizendo, uma. 

PS - A ironia mais devastadora talvez nem esteja na lista em si, mas no facto de que, dos três cientistas distinguidos, nenhum deles trabalha em Portugal. Um encontra-se nos Estados Unidos e os outros dois na Suécia. Em suma, o nosso país continua a fazer aquilo em que se tornou especialista: financiar talento, formar talento e depois exportá-lo para que outros países colham os dividendos científicos, tecnológicos e económicos desse investimento. Vide post anterior onde abordei essa tragicomédia de título, Uma "brilhante" estratégia Portuguesa - Empobrecer orgulhosamente ao mesmo tempo que ajuda ricos a ficarem mais ricos  https://19-pacheco-torgal-19.blogspot.com/2026/04/uma-brilhante-estrategia-portuguesa.html

Problems with a well-intentioned but naive manifesto about sustainable scientific research in the era of big science and AI

 

In a manifesto published this week in Nature Communications, 21 scientists from universities across the United States, United Kingdom, Ireland, Italy, Germany, and Switzerland argue that science is losing valuable knowledge through unpublished negative results, inadequate documentation, and fragmented preservation practices. The concern is plausible—and in many ways overdue. Yet the paper’s central weakness appears immediately: it treats the scale of the problem as self-evident rather than demonstrated. How often are failed experiments unknowingly repeated? What proportion of irreproducibility is actually caused by missing documentation, rather than underpowered studies, unstable reagents, or flawed experimental design? What are the financial, scientific, or human costs of such losses? The paper offers no empirical answers. Instead, it relies largely on other perspective pieces to justify major cultural and infrastructural reforms. Without quantification, it is difficult to judge whether the proposed solutions are proportionate to the problem being described. https://www.nature.com/articles/s41467-026-72667-3?utm_source 

The paper is equally reluctant to confront the political economy of scientific publishing—or its own place within it. Commercial publishers benefit from scarcity of access, novelty bias, and the prestige economy of high-impact journals. The authors acknowledge that publishers have “a role to play,” but avoid naming the structural conflict: many of the reforms they advocate run directly against the financial incentives of the institutions through which scientific prestige is currently distributed. This creates an uncomfortable tension. The paper is written by active researchers whose careers depend on the existing system, published in one of the journals that helps sustain that system, while calling for reforms that would require parts of it to be fundamentally restructured. Yet there is no acknowledgment of this positionality, and no reflection on what it means for the credibility—or political feasibility—of the recommendations. The most honest version of this manifesto would have confronted a harder question: are we, as authors, actually willing to do what we are asking others to do?

The paper is also surprisingly optimistic about AI. Large language models are trained on the published scientific literature—the same positivity-biased and methodologically incomplete record the authors criticize. They cannot recover failed experiments that were never documented, nor reconstruct tacit laboratory knowledge that was never written down. What they can do is reorganize an already distorted archive and return it with extraordinary fluency. But there is an additional danger the paper never addresses: as AI-generated content increasingly enters the scientific and online knowledge ecosystem, future models may be trained not only on incomplete human records, but on synthetic outputs generated by previous models. As recent research on “model collapse” has shown, systems trained recursively on AI-generated data can progressively lose diversity, accuracy, and fidelity to the original human knowledge base. In that scenario, AI would not merely reproduce the biases of the scientific record—it could amplify and recursively entrench them. Trustworthy AI in science therefore depends not only on better data curation, but on preserving human authorship, human judgment, and human-generated knowledge as the foundation of scientific communication.

PS - Unfortunately, the paper never acknowledges that American science — still one of the institutional pillars of the global research enterprise it claims to protect — is facing an existential threat. The Trump administration has cancelled thousands of grants, gutted federal science agencies, subjected peer-reviewed journals to Department of Justice scrutiny, and defunded research areas on ideological grounds. These are not background conditions; they are direct attacks on the very infrastructure of knowledge production this paper is trying to reform and, in many cases, defend. A manifesto about the fragility of scientific knowledge that finds no space for that reality is not apolitical — it is politically evasive. It reads like a document preoccupied with ventilation standards while the building is already on fire.

Update after 1 day - Blogger analytics indicate that the majority of views for this post come from the USA (19%), Germany (18%) and Finland (10%).