LLMs & Retracted Research Papers

Large language models should not be used to weed out retracted literature, a study of 21 chatbots concludes. Not only were the chatbots unreliable at correctly identifying retracted papers, they spit out different results when given the same prompts. On average, the 21 chatbots correctly identified fewer than half of the retracted papers. More at Retraction Watch

24 Articles about AI & Academic Scholarship

Evaluating AI guidelines in leading family medicine journals: a cross-sectional study – BMC

Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers? – arXiv

University of Hong Kong probes non-existent AI-generated references in paper; prof. says content not fabricated – Hong Kong Free Press  

An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers - arXiv  

AI ‘Godfather’ hits record 1 million citations on Google Scholar - Semafor

Large language models in peer review: challenges and opportunities – Springer  

Authors self-disclosed use of AI in research submissions to 49 biomedical journals: A cross-sectional study - MedRxiv

arXiv Changes Rules After Getting Spammed With AI-Generated 'Research' Papers – 404 Media 

Letters to scientific journals surge as ‘prolific debutante’ authors likely use AI – Science.org 

Academic misconduct and artificial intelligence use by medical students, interns and PhD students in Ukraine: a cross-sectional study - BMC

From Language Barrier to AI Bias: The Non-Native Speaker’s Dilemma in Scientific Publishing – Scholarly Kitchen

Will AI + OA be OK? - Cabells 

Why AI transparency is not enough - Centre for Science and Technology Studies at Leiden University

AI tools combat paper mill fraud in scientific publishing as peer review system struggles – Chemistry World

AI-powered fraud: Chinese paper mills are mass-producing fake academic research - South China Morning Post

AI bots wrote and reviewed all papers at this conference – Nature  

Low-quality papers are flooding the cancer literature — can this AI tool help to catch them? – Nature

The chemistry community should ban drawing chemical structures with generative AI, chemists warn – Chemistry World

How ChatGPT-5 redefines scientific reproducibility.” – Elephant in the Lab

AAAI Launches AI-Powered Peer Review Assessment System - Association for the Advancement of Artificial Intelligence

Research commissioner appears to cite discredited study in AI speech – Science Business

AI in peer review: where to draw the line? – Research Professional News  

MIT takes down article on an AI platform for churches -MIT Technology Review 

Journal defends work with fake AI citations after Hong Kong university launches probe - South China Morning Post

22 Articles about AI & Academic Scholarship

Fraud, AI slop and huge profits: is science publishing broken? (a podcast) – The Guardian

AI-generated ‘participants’ can lead social science experiments astray, study finds – Science

AI tools could reduce the appeal of predatory journals – Nature

Fake microscopy images generated by AI are indistinguishable from the real thing. – Chemistry World

The Machines Finding Life That Humans Can’t See – The Atlantic

Can researchers stop AI making up citations? - Nature 

AI models are using material from retracted scientific papers – MIT Tech Review

AI tool detects LLM-generated text in research papers and peer reviews – Nature

Prestige over merit: An adapted audit of LLM bias in peer review – Cornell University arXiv

Chatbots and large language models are being used to fact-check scientific work, but how effective are they? – Q.space 

Far more authors use AI to write science papers than admit it, publisher reports – Science  

What do researchers acknowledge ChatGPT for in their papers? – London School of Economics  

The rising danger of AI-generated images in nanomaterials science and what we can do about it – Nature

ChatGPT Fails to Flag Retracted and Problematic Articles – The Scientist  

Beyond ‘we used ChatGPT’: a new way to declare AI in research – Research Professional News  

Study looks at how biomedical journal editors-in-chief feel about AI use in their journals. – Springer

AI-generated medical data can sidestep usual ethics review, universities say – Nature

AI could be used for a Research Excellence Framework, says Royal Society president – Research Professional News  

Can Generative AI Restore Hope or Result in a Decline in the Quest for Academic Integrity – Sage  

When AI rejects your grant proposal: algorithms are helping to make funding decisions - Nature  

We risk a deluge of AI-written ‘science’ pushing corporate interests – here’s what to do about it – The Conversation

Far more authors use AI to write science papers than admit it, publisher reports – Science

17 Articles about AI & Academic Scholarship

Can generative AI replace humans in qualitative research studies? - Techxplore

The recent reduction in spelling error rates in academic papers could be due to an increased use of LLMs – OSF Preprints  

AI linked to explosion of low-quality biomedical research papers - Nature 

Flood of AI-assisted research ‘weakening quality of science'” – Times Higher Ed

Shoddy study designs and false findings using a large public health dataset portend future risk of exploitation by AI and paper mills – PLOS Biology

Is it OK for AI to write science papers? Nature survey shows researchers are split - Nature

MIT Says It No Longer Stands Behind Student’s AI Research Paper – Wall Street Journal  

Meta releases new data set, AI model aimed at speeding up scientific research – Semafor

Experiment using AI-generated posts on Reddit draws fire for ethics concerns – Retraction Watch

AI-Reddit study leader gets warning as ethics committee moves to ‘stricter review process’ – Retraction Watch  

Why misuse of generative AI is worse than plagiarism – Springer

Science sleuths flag hundreds of papers that use AI without disclosing it - Nature

Google engineer withdraws preprint after getting called out for using AI – Retraction Watch

Scientific Data Fabrication and AI—Pandora’s Box – JAMA Network

AI summary ‘trashed author’s work’ and took weeks to be corrected – Times Higher Ed

AI language models increasingly shape economics research writing, study finds – Phys.org

Artificial intelligence in vaccine research and development: an umbrella review – Frontiers

23 Articles about AI & Academic Scholarship

AI bots are overwhelming some journals – C&EN 

AI research summaries ‘exaggerate findings’, study warns – Times Higher Ed

Ethics in Academic Research: Who Is Responsible for Unethical Practices—AI, Scholars, Editors, or Institutions? – PrePrints

A Scanning Error Created a Fake Science Term—Now AI Won’t Let It Die - Gizmodo

GenAI Footprint in Scholarly Publications Reflects Complex Issues of Ac. Integrity Post-Plagiarism (video) - PUPP 

AI is transforming peer review — and many scientists are worried – Nature

A Shortcut or a Level Up? Harvard Faculty Debate Generative AI in Academia – The Crimson

Publishers Embrace AI as Research Integrity Tool – Inside Higher Ed 

AI tools are spotting errors in research papers: inside a growing movement – Nature

AI search summaries cannibalise academic publishers’ web traffic – Times Higher Ed

An academic paper written by AI passed peer review — but it’s a bit more nuanced than that – Tech Crunch

Trying to Write an Academic Paper with LLM Assistance – Scholarly Kitchen  

Academic publishers warn against AI copyright plans - Research Professional News – Research Professional News  

AI detectors are poor western blot classifiers: a study of accuracy and predictive values – PeerJ  

Will AI jeopardize science photography? – Nature

Can AI Solve the Peer Review Crisis? - IZA Institute of Labor Economics 

Publishers need to provide guidelines on use of AI in research, says Wiley – Chemistry World  

ChatGPT to help peer review scientific studies in UK Government trial – Telegraph

Generative artificial intelligence usage guidelines for scholarly publishing: a cross-sectional study of medical journals – BMC Medicine  

Retractions Increase 10-Fold in 20 Years - and Now AI is Involved – AAPS News

A viral video reveals how an AI-generated mistake led to nearly two dozen flawed research papers – Economic  Times  

Is AI the new research scientist? Not so, according to a human-led study. – University of Florida  

The Use of Artificial Intelligence (AI) in Academic Writing and Publishing Papers – Research Gate

An Arms Race of Research Misconduct

Retractions are rising in medical research literature, even as more eyes examine peer-reviewed papers for accuracy. AI is powering an arms race in the world of research misconduct, making it easier for scientific fraud to occur, and for editors to identify and root out. In 2002, 1 in 5,000 papers were retracted, Oransky said. In 2023 retractions increased to 1 in 500 papers. -AAPS news magazine

18 Articles about AI & Academic Scholarship

What are the best AI tools for research? Nature’s guide - Nature

Exploring the Impact of Generative AI on Peer Review: Insights from Journal Reviewers – Springer

OpenAI unveils a new ChatGPT agent for ‘deep research’ – TechCrunch

AI-Generated Junk Science Is a Big Problem on Google Scholar, Research Suggests – Gizmodo 

What happens when you let ChatGPT assess impact case studies? – London School of Economics  

Generative AI in the research process – A survey of researchers’ practices and perceptions – Science Direct 

Springer Nature offers to sell authors “AI Summaries of Their Own Work” – Futurism

Teens Are Doing AI Research Now. Is That a Good Thing? - Chronicle of Higher Ed

How is content generated by ChatGPT infiltrating scientific papers published in premier journals? – Wiley

Elsevier denies AI use in response to evolution journal board resignations – Retraction Watch  

Springer Nature reveals AI-driven tool to 'automate some editorial quality checks' – The Bookseller 

Nvidia unveils $3,000 desktop AI computer for home researchers - ArsTechnica 

Generative artificial intelligence and academic writing: friend or foe? - Elsevier

Detecting Research Misconduct in the Age of Artificial Intelligence – The Scientist

Can AI-generated podcasts boost science engagement? – Nature

AI-Authored Abstracts ‘More Authentic’ Than Human-Written Ones – Inside Higher Ed

Scholars Are Supposed to Say When They Use AI. Do They? - Chronicle of Higher Ed 

Will ChatGPT Get Tenure? - Leiden Vladtrice

16 Articles about AI & Academic Scholarship

16 Articles about AI & Academic Scholarship

Google's new AI tool transforms dense research papers into accessible conversations - try it free - ZDnet 

Optimizing Large-Scale AI Model Pre-Training for Academic Research: A Resource-Efficient Approach – MarTech Post 

A group of experienced editorial board members struggled to distinguish human versus AI authorship – AHA Journals

AI can carry out qualitative research at unprecedented scale – London School of Economics  

Can AI be used to assess research quality? Chatbots and other tools are increasingly being considered, but people power is still seen as a safer option. – Nature  

Is AI the Answer to Peer Review Problems, or the Problem Itself? – Scholarly Kitchen 

Is Detecting genAI in Scholarly Research Beside the Point? – Clear Skies Adam

Unleashing the power of AI in science-key considerations for materials data preparation – Nature 

UK Research and Innovation tells reviewers they must not use generative AI – Research Professional News 

In which fields can ChatGPT detect journal article quality – ARXIV

Overcoming Skepticism Through Experimentation: The Role of AI in Transforming Peer Review – Scholarly Kitchen

If generative AI accelerates science, peer review needs to catch up - London School of Economics   

Some Thoughts on the Promise and Pitfalls of Innovation and Technology in Peer Review - Scholarly Kitchen

Is AI the Answer to Peer Review Problems, or the Problem Itself? - Scholarly Kitchen 

Do AI models produce more original ideas than researchers? – Nature  

How Gen AI Could Transform Scholarly Publishing: Themes and Reflections from Interviews with Industry Leaders - Scholarly Kitchen

18 Articles about AI & Academic Scholarship

Do AI models produce more original ideas than researchers? - Nature

Strengths, Weaknesses, Opportunities, and Threats: A Comprehensive SWOT Analysis of AI and Human Expertise in Peer Review – Scholarly Kitchen

How Are AI Chatbots Changing Scientific Publishing? – Science Friday

New academic AI guidelines aim to curb research misconduct – Global Times

Generative AI-assisted Peer Review in Medical Publications: Opportunities Or Trap – JRIM Publications

GPT-fabricated scientific papers on Google Scholar: preempting evidence manipulation – Harvard

AI Editing: Are We There Yet? - Science Editor – Science Editor  

AI tool claims 94% accuracy in telling apart fake from real research papers – Deccan Herald  

AI firms must play fair when they use academic data in training – Nature

AI Scientists Have a Problem: AI Bots Are Reviewing Their Work ChatGPT – Chronicle of Higher Ed 

A list of more than 500 papers with clear evidence of generative AI use - Academ-AI

Is AI my co-author? The ethics of using artificial intelligence in scientific publishing – Taylor & Francis Online 

Is ChatGPT a Reliable Ghostwriter? – The Journal of Nuclear Medicine

A new ‘AI scientist’ can write science papers without any human input. Here’s why that’s a problem – The Conversation

Could science be fully automated? A team of machine-learning researchers has now tried. - Nature

How AI tools help students—and their professors—in academic research – Fast Company  

AI-Generated Junk Science Research a Growing Problem, Experts Say – PYMNTS  

Did a criminal Russian academic paper mill use AI to plagiarize a BYU professor and his student? – Deseret News

A Dozen Quotes about AI & Academic Scholarship

AI chatbots have thoroughly infiltrated scientific publishing. One percent of scientific articles published in 2023 showed signs of generative AI’s potential involvement, according to a recent analysis - Scientific American 

The journey from research data generation to manuscript publication presents many opportunities where AI could, hypothetically, be used – for better or for worse. - Technology Network

Is ChatGPT corrupting peer review? There are telltale words that hint at AI use. A study of review reports identifies dozens of adjectives that could indicate text written with the help of chatbots. - Nature 

Should researchers use AI to write papers? This group aims to release a set of guidelines by August, which will be updated every year - Science.org

Generative AI firms should stop ripping off publishers and instead work with them to enrich scholarship, says Oxford University Press’ David Clark. - Times Higher Ed 

Here are three ways ChatGPT helps me in my academic writing. Generative AI can be a valuable aid in writing, editing and peer review – if you use it responsibly - Nature 

New detection tools powered by AI have lifted the lid on what some are calling an epidemic of fraud in medical research and publishing. Last year, the number of papers retracted by research journals topped 10,000 for the first time. - DW News (video) 

Estimating the prevalence of ChatGPT "contamination” in the scholarly literature: It is estimated that at least 60,000 papers (slightly over 1% of all articles) were LLM-assisted - ArXiiv 

AI-Generated Texts from LLM has infiltrated the realm of scientific writing? We confirmed and quantified the widespread influence of AI-generated texts in scientific publications across many scientific domains - BioRxiv 

Georgetown found that American scholarly institutions and companies are the biggest contributors to AI safety research, but it pales in comparison to the amount of overall studies into AI, raising questions about public and private sector priorities. - Semafor 

Google Books is indexing low quality, AI-generated books that will turn up in search results, and could possibly impact Google Ngram viewer, an important tool used by researchers to track language use throughout history. - 404Media

The Association of Research Libraries announced a set of seven guiding principles for university librarians to follow in light of rising generative AI use. - Inside Higher Ed

18 Articles about AI & Academic Research

Could AI Disrupt Peer Review?  Publishers’ policies lag technological advances - Spectrum

The Use of Artificial Intelligence in Writing Scientific Review Articles - Springer

‘Obviously ChatGPT’ — how reviewers accused me of scientific fraud - Nature

AI could accelerate scientific fraud as well as progress - Economist  

Researchers plan to release guidelines for the use of AI in publishing - Chemical & Engineering News

ChatGPT use shows that the grant-application system is broken - Nature   

Detecting fraud in scientific publications: the perils and promise of AI - Science Pod 

The Science family of journals is adopting the use of Proofig, an artificial intelligence (AI)–powered image-analysis tool- Science Magazine  

Can ChatGPT and Other AI Bots Serve as Peer Reviewers? - ACS Publishing  

AI Use in Manuscript Preparation for Academic Journals - Cornell University 

As scientists face a flood of papers, AI developers aim to help New tools show promise, but technical and legal barriers may hinder widespread use - Science Magazine  

Is AI leading to a reproducibility crisis in science? – Nature  

Affiliation Bias in Peer Review of Abstracts by a Large Language Mode - JAMA

AI copilots and robo-labs turbocharge research - Axios 

Editing companies are stealing unpublished research to train their AI - Times Higher Ed 

How journals are fighting back against a wave of questionable images - Nature  

Can ChatGPT evaluate research quality? - Cornell University   

The JSTOR Daily Sleuth - Jstor

Science has been in a “replication crisis” for a decade

In an attempt to test just how rigorous scientific research is, some researchers have undertaken the task of replicating research that’s been published in a whole range of fields. And as more and more of those attempted replications have come back, the results have been striking — it is not uncommon to find that many, many published studies cannot be replicated.

A decade of talking about the replication crisis hasn’t translated into a scientific process that’s much less vulnerable to it. Bad science is still frequently published, including in top journals.

Kelsey Piper writing in Vox

Verification bias

Verification bias refers to a stubborn resistance to accepting the null hypothesis – the assumption that there is no inherent relationship between the variables being studied. The null hypothesis is the default position in experiments. This is what the researcher is attempting to eliminate through experimental investigation. For example, continuing to repeat an experiment until it “works” as desired, or excluding inconvenient cases or results may make the hypothesis immune to the facts. Verification bias amounts to the repression of negative results. 

Augustine Brannigan, The Use and Misuse of the Experimental Method in Social Psychology

Solo Performance

A mountain of studies has shown that face-to-face brainstorming and teamwork often lead to inferior decisionmaking. That’s because social dynamics lead groups astray; they coalesce around the loudest extrovert’s most confidently asserted idea, no matter how daft it might be.

What works better? “Virtual” collaboration—with team members cogitating on solutions alone, in private, before getting together to talk them over. As Susan Cain (who wrote Quiet: The Power of Introverts in a World That Can’t Stop Talking) discovered, researchers have found that groups working in this fashion generate better ideas and solve problems more adroitly. To really get the best out of people, have them work alone first, then network later.

Sounds like the way people collaborate on the Internet, doesn’t it? With texting, chat, status updates, comment threads, and email, you hash over ideas and thoughts with a pause between each utterance, giving crucial time for reflection. Plus, you can do so in private.

(The) overall the irony here is pretty gorgeous. It suggests we’ve been thinking about the social web the wrong way. We generally assume that it has unleashed an unruly explosion of disclosure, a constant high school of blather. But what it has really done is made our culture more introverted—and productively so. Now if we could just get some doors on those cubicles.

Clive Thompson writing in Wired Magazine