Claim Extraction In Journalism: Why AI Should Find Claims, Not Write Stories
My position: newsrooms should aim AI at claim extraction in journalism, meaning finding, isolating and organizing the checkable statements in a source, before they let it anywhere near writing the story. The reason is simple. When AI extracts a claim, an editor can compare it to the exact words in the source in seconds. When AI writes a story, the same editor has to hunt for things that might not exist. This piece lays out the evidence for that view, the strongest objections and a workflow you can use.
What claim extraction in journalism actually does
Claim extraction takes a document, such as a speech, a press release, a transcript or a draft, and pulls out the individual statements that can be checked. Full Fact, the UK fact-checking charity, defines a claim in this field as an assertion about the world that can be checked. Tools that do this are often described as AI claim detection, and they usually add a second step: deciding which claims are worth checking.
ClaimBuster, built by researchers at the University of Texas at Arlington and the University of Mississippi, shows the idea clearly. It sorts sentences into three groups: non-factual ones such as opinions, unimportant factual ones, and check-worthy factual ones. Its own papers report recall of 74% and precision of 79% in detecting check-worthy claims, and a positive correlation with which claims professional fact-checkers at CNN, PolitiFact and FactCheck.org chose to check. That work is older and was tuned on US debates, but the structure of the task has not changed.
The point is that extraction is a narrow job with a checkable output. That is the heart of my argument.
The case against letting AI write the story
The best recent evidence on AI and news accuracy comes from a study coordinated by the European Broadcasting Union and led by the BBC. Journalists at 22 public service media organizations in 18 countries and 14 languages evaluated more than 3,000 responses from ChatGPT, Copilot, Gemini and Perplexity to news questions. They found that 45% had at least one significant issue, 31% had serious sourcing problems such as missing, misleading or incorrect attributions, and 20% had major accuracy problems, including hallucinated details and outdated information.
A caution: that study tested consumer chatbots answering news questions, not newsroom writing tools built on verified sources. I am not claiming newsroom tools fail at the same rate. I am saying the failure types, invented details and weak sourcing, are the ones that matter when AI writes.
The organizers of the CLEF 2026 CheckThat! Lab, a research program on fact-checking technology, describe the same pattern in a narrower setting. They report that LLM-generated fact-checking articles lag far behind expert-written ones, especially in building coherent, evidence-based arguments that explain whether a claim is true. They also note that verifying numerical claims is harder for LLMs than verifying non-numerical ones.
The track record of AI claim detection
Claim extraction is not hypothetical. Full Fact’s tools have been used in more than forty countries. In Nigeria, a Full Fact product manager said that before a Google.org grant the system detected around 100 claims a day, and afterward it could identify and review 100,000 a day, a 1,000-fold increase. Full Fact’s own description is that each article is tagged by topic, each claim is labeled by type and by who is making it, and fact-checkers then decide which to check. Its team says the tools are meant to assist fact-checkers, not replace them, and a newsletter profile reports that Full Fact avoids asking AI to determine truth or falsehood.
Microsoft Research’s Claimify shows how careful extraction can be designed. It works in three stages: selecting sentences with verifiable content, resolving ambiguity, and breaking sentences into standalone claims. If ambiguity cannot be resolved from the context, it labels the sentence “cannot be disambiguated” instead of guessing, and the researchers found that removing the selection stage caused the largest drop in performance. Claimify was built to check LLM outputs rather than news, so I cite it as a design lesson, not as proof about newsroom use.
Why extraction is the safer job
This is my reasoning, not a finding from those sources. An extracted claim is anchored to the original text. A reviewer can read the claim, open the source and see in seconds whether the words match. A generated paragraph has no such anchor. The reviewer must decide, sentence by sentence, whether each statement is real, sourced and current, which is slow and easy to get wrong under deadline.
Extraction can still fail. Claimify’s own evaluation lists missing, incomplete and inaccurate claims as failure modes to be measured. The difference is that these errors are visible against the source. They are omissions and distortions you can catch, not fabrications you may never notice.
The strongest objections
“Extraction misses things.” True. A Reuters Institute factsheet on automated fact-checking noted that ClaimBuster tended to miss implied claims and claims embedded in complex sentences, which humans recognize easily, and that it performed worse on tweets than on debates. A claim list can give a false sense that everything was covered. So treat the list as a starting point, and tell editors what it cannot see.
“Check-worthiness is a judgment call.” Also true. Researchers separate whether a statement is a checkable claim from whether it is worth checking, and the CheckThat! Lab has run check-worthiness tasks since 2018 because the question is hard. AI can rank, but an editor decides what matters for the audience.
“AI writing help is useful under review.” I agree it can be. My claim is about order, not a ban: extract and verify first, then draft from confirmed claims. AI as first author of a reported story is the part I would avoid.
“Humans still have to verify extracted claims.” Yes, and that is a feature. The gain is that AI does the monitoring and sorting at a scale no desk could manage, and people spend their time on the checking that needs judgment.
A simple claim extraction workflow
Extract. Run AI claim detection on the source or draft and pull out numbers, quotes, attributions and cause-and-effect statements.
Anchor. Keep each claim next to the exact words it came from, so it can be checked against the source.
Resolve ambiguity. Add who, what, when and where. If the meaning is unclear, mark it as unresolved instead of guessing.
Prioritize. Put high-stakes, central claims first.
Verify. A person checks each priority claim against a primary source and records the evidence.
Sort. Mark each claim confirmed, unconfirmed or conflicting in your fact pack.
Draft from confirmed claims. Write, with or without AI help, using only what has passed.
Log it. Record what AI extracted and what people changed in your AI audit trail.
A minimal claim log needs six columns: claim, source and exact words, type (number, quote, cause, attribution), priority, evidence link and status.
Check independence too: several articles repeating one claim may be one source and many copies.
Where NewsBolts fits
At NewsBolts, we believe AI should help editors see what is claimed and where it came from, while people decide what is confirmed and what is published. Claim-level checking is one stage in the workflow described in our AI newsroom operating system guide.
FAQ
What is claim extraction in journalism? It is the process of pulling checkable statements, such as numbers, quotes and attributions, out of a document so they can be verified one by one.
What is AI claim detection? It is software that scans text, audio or video transcripts, finds statements that could be checked and often ranks them by how worth checking they are.
Can AI fact-check claims automatically? AI can help find claims and surface possible evidence, but a person should decide whether a claim is true. Even leading fact-checking organizations use AI for detection and monitoring, not as the final judge.
How accurate is automated claim detection? It varies by tool, language and type of text. ClaimBuster’s own papers reported 74% recall and 79% precision on US debate sentences, and later work shows it can miss implied claims, so human review is still needed.
Does claim extraction replace fact-checkers? No. It helps them find and organize claims at scale. Verification and judgment stay with people.
Conclusion
The safest, most useful first job for AI in a newsroom is finding claims, not writing the story. Extraction produces output you can check against the source, while generation produces text you must audit for things that may not exist. Start with claim lists, keep a person on every verification step, and draft from what has been confirmed.
Comments