The bibliographic database reviews the journals on its list to ensure that it indexes high-quality titles that are not involved in questionable publishing practices. Credit: Hisaka Osawa/Getty
Researchers have identified three warning signs that a journal might be involved in suspicious publishing activity: a doubling of its annual output, at least 70% of its articles coming from one country, and at least 15% of its articles being flagged by the free online tool Questionable Paper Screener. Applying these criteria to journals in the OpenAlex database revealed potential problems with dozens of titles indexed by the major academic database Scopus.
The researchers identified 3,753 journals that met at least one of the criteria. After cross-referencing those titles with journals indexed in Scopus, they found that 5% had subsequently been delisted. The findings were published in a preprint1 on October 20.
Researchers and research-integrity experts say the signals can identify journals that merit further investigation, but they do not prove that a title is problematic. Achar Agrawal, a co-author of the preprint and a freelance data scientist in Raipur, India, describes them as “rudimentary flags”. They also illustrate the challenges facing bibliographic databases as questionable publishing practices become more widespread, he says.
How Scopus reviews journals
Bibliographic databases such as Scopus play an important role in the global research ecosystem because they provide rankings used to evaluate individual researchers and institutions. Scopus is run by the Dutch publisher Elsevier, and publication in the database usually indicates that a journal is considered trustworthy.
Scopus regularly reviews the journals on its list, using tools that look for signs of suspicious activity suggesting that titles are producing low-quality research or failing to meet accepted academic standards. These signs include “changes in publication volume and journal scope, unusual patterns in author and author collaboration, unusual citation behavior, and signs that the journal is becoming vulnerable to questionable content”, an Elsevier spokesperson told Nature.
For journals that receive reports, Scopus “carefully and individually evaluates all available information, including considering the journal’s own response”, the spokesperson added. Journal entries that do not meet the threshold for a specific metric are deleted. Scopus delisted 156 journals last year and has removed 100 so far this year.
An Elsevier spokesperson said: “No single signal or fixed threshold determines the outcome.” The Scopus Content Selection Committee said that signals should prompt an investigation and that, if necessary, journals would be submitted for re-evaluation.
Signs of trouble
The study found that approximately 3,500 journals that had not been delisted collectively published more than 500,000 articles during the years in which they were flagged by at least one signal. Nearly 300 journals were flagged by two signals, while 28 were flagged by all three.
Anna Abarkina, a research-integrity researcher and social scientist at the Free University of Berlin, says that each symptom warrants further investigation but is not by itself a “clear decision for delisting”. “There are many reasons why a journal might double in publication volume,” she says. Such growth can occur, for example, as part of a new magazine or publisher’s business plan. However, Abarkina believes that the presence of all three red flags indicates suspicious activity that is difficult to justify.
Álvaro Cabezas Clavijo, a researcher in research evaluation at La Rioja International University in Logroño, Spain, also says that the three signals should be interpreted cautiously. For a small journal, for example, a large majority of papers from one country might not be a “sign of questionable practices”.
Source: www.nature.com
