KEY TAKEAWAY

• More than 100 institutions have declared their support for eLife’s reviewed preprint model, following the journal’s loss of impact factor.

Rather than only accepting papers recommended for publication by peer reviewers, eLife publishes all reviewed research as reviewed preprints. However, Clarivate, the provider of Web of Science, only indexes peer reviewed content, resulting in the loss of eLife’s impact factor for 2025. Rather than changing their publishing model, eLife agreed to be partially indexed in Web of Science’s Emerging Sources Citation Index (ESCI). But how has this been received?

As reported in Research Information, eLife surveyed over 100 institutions and funders to assess how their publishing model is viewed. Over 95% of respondents endorsed non-traditional publishing approaches like eLife’s, confirming publications will continue to be factored into hiring, promotion, and funding decisions.

Promoting integrity or outdated metrics?

Dr Nandita Quaderi, Senior Vice President and Editor-in-Chief of the Web of Science at Clarivate, stressed that policies must be applied universally to protect research integrity. Quaderi warned that “cover-to-cover indexing of journals in which publication is decoupled from validation by peer review risks allowing untrustworthy actors to benefit from publishing poor quality content”.

On the other hand, Ashley Farley, Senior Officer of Knowledge & Research Services at the Gates Foundation, believes Web of Science’s policy “reinforces outdated publishing metrics that hinder innovation”, while Damian Pattinson, Executive Director at eLife, noted that with increasing emphasis on open science, “eLife remains confident that its model represents the future of scholarly publishing – one that prioritises scientific quality, transparency, and integrity over outdated prestige metrics”.

“eLife remains confident that its model represents the future of scholarly publishing – one that prioritises scientific quality, transparency, and integrity over outdated prestige metrics.”
– Damian Pattinson, eLife

As debates over the future of the impact factor continue, Farley believes that “indexers must evolve to support responsible, transparent models like eLife’s”.

—————————————————

KEY TAKEAWAYS

• Publication extenders make articles more accessible, increase citations, and aid reader comprehension.
• Lack of adoption by publishers or and difficulty finding publication extenders on journal platforms highlights opportunities for improvements.

Publication extenders, such as plain-language summaries (PLSs), infographics, and video abstracts, have become incredibly useful tools for reaching a wider audience, enhancing the impact of research. Yet evidence supporting their use has not been readily available. To address this, the International Society for Medical Publication Professionals (ISMPP) Digital/Visual Communications Committee developed the Publication Extenders Evidence Resource. In an article published in The MAP Newsletter, Kelly Soldavin and colleagues discuss key findings, demonstrating the value of publication extenders.

Publication extenders improve article metrics

Publication extenders can lead to increased downloads, Altmetric scores, and citations. The authors point to studies that found:

• Articles with video abstracts had a 1.206 higher citation rate than those without.
• 62% of articles with a text-based PLS were downloaded significantly more than similar articles without PLSs.

Audience preferences for publication extenders vary

The authors highlight several studies assessing the format of publication extenders preferred by different groups. Patients report that publication extenders, and specifically PLSs with plain text and infographics, make articles easier to understand. On the other hand, some health care professionals prefer plain-text PLSs over graphical formats.

Uptake of publication extenders could be improved

“While the benefits of publication extenders are clear, uptake by authors and journals is low.”

While the benefits of publication extenders are clear, uptake by authors and journals is generally low: one study found only 11 of 30 haematology journals allowed PLSs. Even when publication extenders are used, they are often difficult to find on journal platforms.

Looking to the future

The authors conclude that research consistently shows publication extenders enhance the impact of research, making it more accessible across a wide audience. Publishers should consider ways to make publication extenders more discoverable, while authors and publication planning professionals should consider platforms other than those offered by journals to improve the accessibility of digital content.

—————————————————

KEY TAKEAWAYS

• eLife adopted a ‘reviewed preprint’ publishing model in 2023, publishing all reviewed papers, regardless of reviewer recommendation.
• Clarivate has since updated its policies to no longer provide impact factors for journals that publish papers that are not endorsed by peer review.

As reported in Research Professional News, the non-profit research journal eLife will not receive an impact factor rating from Web of Science in 2025, following implementation of a new policy by Web of Science provider Clarivate. Under eLife’s ‘reviewed preprint’ model adopted in 2023, all submitted research papers that undergo peer review are published, regardless of whether reviewers recommended them for publication. In response to the growing trend of journals decoupling publication from peer review, Clarivate introduced its policy to index only content that is validated by peer review. ​

“[Clarivate’s policy] reflects our commitment to support the integrity of the scholarly record through curation and selectivity in the Web of Science.” – Nandita Quaderi, editor-in-chief, Web of Science

eLife, a signatory of the Declaration on Research Assessment, opposes the reliance on metrics like the impact factor and has reiterated its commitment to meaningful research assessment, stating that its model is closer to the ideal of how scientific discourse should work.

This development will inevitably spark discussion about the pros and cons of traditional metrics in research assessment. eLife’s innovative model challenges the conventional take on peer review, prompting the scientific community to reconsider how best to measure research impact and quality. As the peer review and publishing landscape evolves, this case underscores the need for ongoing dialogue about practices that best serve the advancement of science.

————————————————–

KEY TAKEAWAYS

• Traditional metrics like impact factor still influence journal selection and engagement, despite their limitations in assessing research quality.
• Enhanced publication content, including graphical abstracts and plain language summaries, improves healthcare professional engagement.

In an article for The MAP newsletter, Alexa Holland, Hamish McDougall, Radhika Bhatia, and Sarah J Clements highlight the importance of adopting novel metrics and enhanced publication content (EPC) to improve healthcare professional (HCP) engagement with scientific publications.

Re-evaluating journal metrics

In the evolving landscape of scientific publishing, traditional metrics like impact factor continue to dominate journal selection and readership decisions, despite their well-documented limitations. A survey conducted by Clements and colleagues, presented at the 2024 European Meeting of the International Society for Medical Publication Professionals, revealed that 57.9% of HCPs still rely on impact factor when selecting articles to read or choosing where to publish. However, such metrics do not always reflect the true value or reach of research.

57.9% of HCPs still rely on impact factor when selecting articles to read or choosing where to publish.

To move away from this outdated method of research assessment, publication professionals must advocate for a shift towards more diverse and transparent metrics, as outlined by the Declaration on Research Assessment. Additionally, efforts should focus on encouraging HCPs to engage with a more varied pool of publications, select appropriate target journals, and promote open access.

The EPC effect: more engagement, more impact

Elsewhere, the survey identified that graphical abstracts, plain language summaries, video summaries, and other forms of EPC are powerful tools for boosting engagement. 38.8% of HCPs are more likely to read publications with EPC, and research has also shown that articles featuring EPC tend to receive higher Altmetric scores and experience increased social media engagement than those without.

Despite the benefits, barriers such as development time, strict journal guidelines, and the fact that not all journals offer EPC options continue to hinder their broader implementation.

To enhance EPC impact and adoption, the authors recommend:

1. Educating HCPs on the importance of EPC and how to create it
2. Encouraging journals that have yet to implement EPC to adopt it
3. Advocating for standardised inclusion of EPC across journals

The authors identified future areas for exploration, including how EPC can influence clinical decision-making and patient education. By prioritising a more rigorous research evaluation process and promoting opportunities to implement EPC, the medical publishing industry can better support knowledge dissemination, ultimately improving patient outcomes.

————————————————–

KEY TAKEAWAYS

• The ‘diversity factor’ has been proposed as a new, more equitable metric for assessing journal quality and the impact of health research.
• The index takes into account the diversity of the authors, study participants, and departmental affiliations to promote a wider range of perspectives in research.

The impact factor remains the dominant metric among researchers for assessing journal and (indirectly) research paper quality, despite multiple calls for it to be superseded by alternative measures. Recently, a novel metric claimed the spotlight in an MIT News article. The article describes a study by Dr Jack Gallifant et al., published in PLOS Global Public Health, which suggests that the impact factor misses the mark in capturing a paper’s impact on health. The researchers argue that, for a more accurate understanding of impact, journal metrics should take into account the diversity of the authors and of the study participants. They propose a novel metric, termed the ‘diversity factor’.

The index is comprised of 3 key components:

• author demographics: the gender and geographic location of the authors
• participant demographics: the gender, ethnicity, race, language, geographic location, and age of the individuals enrolled in the study
• departmental affiliation: papers with authors from different disciplines (eg, doctors, nurses, and engineers) score more highly than papers with authors from a single field.

After settling on the metric’s components, the group used the database OpenAlex to extract metadata relating to the authors of over 100,000 medical papers, from around 7,500 journals, published in the last 20 years. A considerable number of the papers retrieved were not open access, meaning that participant demographics could not be included in the final analysis. However, as the researchers predicted, most papers did not perform well against the new metric, even when considering author information alone. Specifically, there was significant underrepresentation of female authors and of authors from low- or middle-income countries. The group hope that by quantifying and tracking diversity in this way, over time, those working in health research would be prompted to drive progress against these measures.

So, why exactly is a lack of diversity a problem for global health outcomes? Ultimately, it boils down to ‘blind spots’ in medical knowledge, explains Dr Leo Anthony Celi, senior author of the paper:

“What happens when all of the authors involved in a project are alike is that they’re going to have the same blind spots. They’re all going to see the problem from the same angle. What we need is cognitive diversity, which is predicated on lived experiences.”

Dr Celi believes that stakeholders within medical publishing — including journals, academic institutions, funding bodies, and even the media — are accountable for the inequity seen in health research. As such, each must play their part in diversifying medical research publications. To this end, Dr Celi calls for the diversity factor to prompt discussions within the medical research community and provide a first step towards a more equitable evaluation of the true impact of research.

————————————————

KEY TAKEAWAYS

• Journal impact factor is a poor predictor of the quality of peer review.
• Implementing open peer review and reviewer training would facilitate the assessment and quality of peer review.

Journal impact factor is often considered an indicator of quality, but does a high impact factor necessarily equate to high quality peer review? In a recent Nature News Q&A article, Dr Anna Severin discussed how her team used artificial intelligence to investigate whether there is any link between the two. Their results were also reported in a recent preprint on arXiv.

As Dr Severin explained, ‘quality’ peer review is difficult to define. Authors want suggestions for improvement, while editors want recommendations on whether to publish. As a result, Dr Severin and her team developed the following proxy categories for quality:

• Thoroughness: comments made on the materials and methods, presentation and reporting, results and discussion, or the importance of the article.
• Helpfulness: comments relating to praise or criticism of the article, giving suggestions for improvement, or providing further examples.

The team randomly selected 10,000 peer review reports submitted to more than 1,600 medical or life science journals. A random sample of 2,000 sentences were then coded to the above categories. The team used this information to train machine learning models to predict categories for over 187,000 sentences across the reports.

Overall, peer reviewers from higher impact factor journals focused on methodology, but spent less time suggesting improvements and providing examples than reviewers from lower impact factor journals.

Reports in higher impact factor journals tended to be longer, and reviewers were more likely to be from Europe or North America. Although there seemed to be a link between impact factor and peer review quality, any differences were modest, and variability was high. As such, the team concluded that impact factor is “a bad predictor for the quality of review of an individual manuscript”.

The team concluded that impact factor is “a bad predictor for the quality of review of an individual manuscript”.

Dr Severin recognised the limitations of using artificial intelligence, but considered this to be a first step in assessing peer review quality in a systematic and scalable way, and called for the following actions:

• Reviewers should be trained and given clear guidelines.
• Peer review should be open rather than confidential.

—————————————————–

A new journal metric is being introduced that will rank academic journals based on their commitment to research transparency and reproducibility, rather than traditionally used citation counts. The TOP Factor, launched by the Center for Open Science, will assess journal policies according to their compliance with the eight standards of the Transparency and Openness Promotion (TOP) Guidelines, for which there are three increasing levels of stringency.

The eight standards of the TOP Guidelines are designed to reflect behaviours that can improve transparency and reproducibility of research, covering the following elements:

• transparency of data, materials, code, and research design
• preregistration
• replication.

The new metric rewards journals that publish articles that adhere to these core scientific values. As noted by Simine Vazire, Professor of Psychology at UC Davis and Editor-in-Chief at the psychology journal Collabra:

“Transparency is not the same thing as quality, but it is a necessary precondition for evaluating quality”

In addition to the eight TOP Guideline indicators, journals will also be assessed on whether they offer Registered Reports, which aim to reduce publication bias against negative findings by allowing study design peer review before data are analysed, and Open Science Badges that mark the availability of shared data, materials and preregistered study designs.

So far, over 250 journals publishing in the fields of psychology, economics, education and general science have been evaluated and are presented on the TOP Factor website. One of the key differentiators of the TOP Factor from journal impact factor is that the former provides a searchable, modular set of indicators of journal policies promoting good research practices, rather than a single number. Another distinguishing feature is that, in principle, all journals could achieve the top score, with TOP Factor envisaged as a rating system rather than a way of ranking journals. Indeed, the goal is that releasing journal TOP factors will encourage journals to improve their open science policies.

Note: There is a poll embedded within this post, please visit the site to participate in this post's poll.

——————————————————–

Summary by Julianna Solomons PhD, CMPP from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

Many academic researchers still consider the journal impact factor a key metric in their career progression, despite long-standing concerns over its use in assessing journal quality.

A recent survey of 55 universities in the US and Canada showed that 36% of researchers consider the impact factor to be ‘very valued’ for promotions and tenure, and 27% said they were ‘very important’ when deciding where to submit their research. Younger and non-tenured researchers put more emphasis on impact factors when deciding where to publish, which the authors link to the perceived tenure process.

In what appears to be misconstrued peer pressure, researchers tended to perceive that their colleagues were more driven by the impact factor than them, and that they valued readership and open access less. In comments made to Nature Index, the lead author, Meredith Niles, calls for more honest conversations in academia: “If we don’t actually care about the journal impact factor as much as factors such as readership…then let’s stop pretending we care…”

But change needs to come at an institutional level. As reported in Nature Index,  a previous study from the same project showed that 23% of the North American universities studied referred to impact factors or related phrases in at least one of their review, promotion, and tenure documents, and nearly 90% of the institutions supported their use. The study authors note this is probably the tip of the iceberg, and call for further research to support efforts such as Humane Metrics in the Humanities and Social Sciences (HuMetricsHSS) and the San Francisco Declaration on Research Assessment (DORA) in changing the values underlying academic evaluation.

Note: There is a poll embedded within this post, please visit the site to participate in this post's poll.

——————————————————–

Summary by Robyn Foster PhD from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd