The number of new papers on the pandemic is doubling every two weeks, and shows no signs of slowing down. Many of these papers are first published on preprint servers, meaning they are made public before peer review. This makes it difficult to judge their qualifications.
Now, a start-up company says its platform — called Scite.ai — can automatically tell readers whether papers are supported or contradicted by subsequent academic work.
Unlike traditional citation-matrix tools, Scite.ai tells users how many times a paper has been supported or refuted by those studies, as well as how often it has been cited. The resulting reports display citations in the context in which they are cited, allowing users to assess for themselves how the paper is being cited.
So far, Scite.ai has analyzed more than 16 million full-text scientific articles from publishers, such as BMJ Publishing Group in London and Karger in Basel, Switzerland. But this is only a fraction of the scientific literature. “They are limited by the literature they can capture and machine-learning algorithms,” notes Jody Schneider, an information scientist at the University of Illinois at Urbana-Champaign.
Still, the tool — accessible through a searchable website and as a Chrome and Firefox browser plug-in — can provide clarity. In March, the site’s developers pointed its artificial intelligence (AI)-based engine to a database, which at the time included 30,000 papers of various types, to help provide context how much weight each article could carry. (see go.nature.com/35nchkp).
They found that a February 22 preprint1, which indicated that high levels of certain immune-signaling molecules were associated with more-severe cases, was supported by a preprint2 from the second group five days later (see scikit.ai report . nature.com/2ztuokb).
Conversely, users who search Scite.ai for a preprint that suggests HIV may have contributed to the formation of the new one will find that the report was refuted by two follow-ups and supported by none. was not (see go.nature.com/2vtdfxd). (The authors of the preprint have withdrawn this for revision in response to the researchers’ comments on the work.) At the moment, the analysis of Scite.ai’s database of papers is not fully automated, so it is sometimes delayed that How quickly is the preprint is analyzed by the tool.
Scite.ai receives about 1,000 visitors a day and has some 2,700 registered users, a number rising since the site required users to register to view the full citation analysis for the paper it delivered on March 20. it occurs.
Adding perspective
Citation numbers have traditionally been viewed by researchers as measures of impact. But just because a paper is highly cited doesn’t mean it’s a good thing, says Elizabeth Suelzer, a reference librarian at the Medical College of Wisconsin Library in Milwaukee.
Former physician Andrew Wakefield’s infamous 1998 study claiming a link between autism and vaccines is over-cited, she notes, but most of those quotes are negative. Without a complete citation analysis, “it would be hard to tell why the article was cited so heavily”, Suelzer explains. That’s why a tool like Scite.ai can be helpful, she says. Other examples include articles retracting the Retraction Watch plug-in flags for Zotero reference-management software.
Scite.ai co-founder and chief executive Josh Nicholson first recognized the need for such a tool in 2012. Nicholson was pursuing his PhD in cell biology at Virginia Polytechnic Institute and State University in Blacksburg when he read a Nature commentary. Making waves about scientific reproducibility 3. In this, a researcher formerly at the biotechnology company Amgen in Thousand Oaks, California, reveals that scientists there can reproduce the findings of 47 out of 53 ‘landmark’ cancer studies. have been unable to.
This prompted Nicholson and biologist Yuri Lezebnik, then at Yale University in New Haven, Connecticut, to propose a new citation metric to indicate that a given study or its findings have been verified by subsequent reports. Or not. The duo launched Scite.ai in April last year.
At the heart of Scite.ai is a machine-learning algorithm that scans research articles to identify which papers they cite, and to determine whether they support, contradict, or merely cite those papers. mention. The algorithm mines the text of articles from publisher partners including Rockefeller University Press in New York City and Wiley in Hoboken, New Jersey. Scite.ai also has preliminary talks with Springer Nature in Heidelberg, Germany, which publishes Nature, Nicholson says.