KEY TAKEAWAYS

• The key goals of reforming research assessment include reduced reliance on counterproductive, citation-based metrics and promotion of open science.
• New metrics designed to incentivise open science risk undermining initiatives to improve research evaluation.

Wider adoption of open science and reduced reliance on counterproductive, citation-based metrics are both key goals in the push to reform research assessment. However, in an article for Research Professional News, Ulrich Herb argues that flooding the market with open science metrics designed to incentivise researchers undermines the very reforms they are meant to promote.

Incentivising open science

Herb reports that while open science aims to improve transparency, accessibility, and collaboration in research, initiatives have struggled to gain traction with researchers. In a bid to push open science forward, advocates, research institutions, and funders have designed myriad new metrics to incentivise openness, including:

• counting outputs such as open access publications, preprints, Findable Accessible Interoperable and Reusable (FAIR) datasets, data management plans, replication studies, and pre-registrations
• measuring attention from downloads, citations, and media coverage
• analysing social dimensions via collaborations, diversity, and citizen science activities.

New metrics are already the subject of extensive research and development in Europe.

Open science metrics undermine research assessment reform

Herb believes that open science metrics are experimental, fragmented, and lacking standardisation. Their dependence on quantitative measurement conflicts with the key principles of research evaluation reform, which promote qualitative, holistic assessment. Further, because open science metrics are used both to measure behaviour and influence it, they can encourage ‘metric-driven’ activities, such as using multiple data cuts to generate high numbers of FAIR-licensed datasets, or selecting diamond open access in favour of more appropriate journals. Finally, Herb argues, the current lack of clarity around precisely what open metrics are measuring renders them as counterproductive for research assessment as the citation-based metrics they are designed to replace.

“Because open science metrics are used both to measure behaviour and influence it, they can encourage ‘metric-driven’ activities.”

Using open science metrics as a force for good

Herb suggests that, if standardised, open science metrics could promote open science practices. At present, they risk creating a culture of incentivised behaviours that contradict the very ideals of open, fair, and meaningful research evaluation. The task ahead is to ensure that open science involves a genuine shift in how research is assessed.

————————————————–

KEY TAKEAWAYS

• Current academic metrics can be manipulated, leading to unethical practices, such as self-citation and citation cartels.
• The addition of clinical guidelines and Bluesky tracking to the Altmetric Attention Score could help drive a cultural shift toward recognising genuine forms of research impact.

A recent article by Dan K Pearson on the London School of Economics’ Impact of Social Sciences blog sheds light on the growing concern over the gamification of academic metrics. Pearson highlights how the pressure to publish in high-impact journals has led to unethical practices, such as excessive self-citation, citation cartels, and even the emergence of a citation black market where researchers can purchase citations to boost their profiles.

The pressure to publish creates an environment where researchers focus on quantifiable outputs rather than the actual outcomes and societal impacts of their work.

This environment encourages researchers to focus on quantifiable outputs rather than the actual outcomes and societal impacts of their work. Pearson argues that this system not only undermines the integrity of academic research but also discourages public engagement and collaboration. He suggests a cultural shift is needed whereby impact-oriented activities, such as public outreach, are emphasised over citation counts.

Two recent developments to the Altmetric platform could address this need by better reflecting the real-world applications of scholarly work. As reported by Research Information, Altmetric now tracks citations in clinical guidelines, providing insights into how research informs clinical practice and patient care. This addition allows researchers and institutions to assess the practical applications of medical research, thereby informing funding decisions and publication strategies.

Altmetric has also expanded its tracking to incorporate Bluesky, a social media platform favoured by the research community. This inclusion offers a more comprehensive view of research conversations, helping to understand the broader engagement and influence of research.

By embracing these developments and promoting wider recognition of scholarly contributions, the academic community can move beyond the pitfalls of gamified metrics and toward a more authentic assessment of research impact.

————————————————–

KEY TAKEAWAYS

• Traditional metrics like impact factor still influence journal selection and engagement, despite their limitations in assessing research quality.
• Enhanced publication content, including graphical abstracts and plain language summaries, improves healthcare professional engagement.

In an article for The MAP newsletter, Alexa Holland, Hamish McDougall, Radhika Bhatia, and Sarah J Clements highlight the importance of adopting novel metrics and enhanced publication content (EPC) to improve healthcare professional (HCP) engagement with scientific publications.

Re-evaluating journal metrics

In the evolving landscape of scientific publishing, traditional metrics like impact factor continue to dominate journal selection and readership decisions, despite their well-documented limitations. A survey conducted by Clements and colleagues, presented at the 2024 European Meeting of the International Society for Medical Publication Professionals, revealed that 57.9% of HCPs still rely on impact factor when selecting articles to read or choosing where to publish. However, such metrics do not always reflect the true value or reach of research.

57.9% of HCPs still rely on impact factor when selecting articles to read or choosing where to publish.

To move away from this outdated method of research assessment, publication professionals must advocate for a shift towards more diverse and transparent metrics, as outlined by the Declaration on Research Assessment. Additionally, efforts should focus on encouraging HCPs to engage with a more varied pool of publications, select appropriate target journals, and promote open access.

The EPC effect: more engagement, more impact

Elsewhere, the survey identified that graphical abstracts, plain language summaries, video summaries, and other forms of EPC are powerful tools for boosting engagement. 38.8% of HCPs are more likely to read publications with EPC, and research has also shown that articles featuring EPC tend to receive higher Altmetric scores and experience increased social media engagement than those without.

Despite the benefits, barriers such as development time, strict journal guidelines, and the fact that not all journals offer EPC options continue to hinder their broader implementation.

To enhance EPC impact and adoption, the authors recommend:

1. Educating HCPs on the importance of EPC and how to create it
2. Encouraging journals that have yet to implement EPC to adopt it
3. Advocating for standardised inclusion of EPC across journals

The authors identified future areas for exploration, including how EPC can influence clinical decision-making and patient education. By prioritising a more rigorous research evaluation process and promoting opportunities to implement EPC, the medical publishing industry can better support knowledge dissemination, ultimately improving patient outcomes.

————————————————–

The 20th Annual Meeting of the International Society for Medical Publication Professionals (ISMPP) took place once again in Washington, DC, from 29 April – 1 May with the theme ‘Storytelling: Its Art and Power’. Incorporating stories into medical publications can clarify complex ideas, build empathy and establish trust, ultimately dispelling stigmas and misinformation. Over 700 attendees discovered the power of storytelling to unlock greater understanding in medical communications.

A summary of the first day of the meeting is provided below to benefit those who were unable to attend the meeting, and as a timely reminder of the key topics covered for those who did.

You can read our summary of Day 2 of the meeting here, and our summary of Day 3 here. 

Summaries of Day 1

Opening remarks and presidential address

KEY TAKEAWAY

• ISMPP celebrates the 20^th year of ISMPP and the 15^th year of CMPP. ISMPP continues to improve and grow year-on-year.

Robert Matheis (President and CEO, ISMPP) opened the meeting by celebrating 20 years of ISMPP, highlighting the growth year-on-year, from 12 founding members to an impressive current 2,580 members! Matheis emphasised that the ultimate goal in medical publications is to optimise patient care and noted that as we are entering a new era in publications, embracing our history is the key to our future. Key ISMPP achievements from the past and upcoming year include the establishment of the artificial intelligence (AI) task force; updated website with eLearning; launch of a patient scholarship programme; improvements in accessibility (for example, live captioning was available for presentations at the meeting); new Certified Medical Publication Professional (CMPP) software (to be rolled out later this year); and the transition from ISMPP West to ISMPP Academy. The strong productivity of the restructured committees was also highlighted; over 300 members are involved with ISMPP committees and the introduction of two task forces – artificial intelligence (AI) and patient engagement – has led to multiple accomplishments and activities. Matheis rounded up his address with a call to action to volunteer with ISMPP!

Dana Fox (Caudex) provided an update on the CMPP programme. Fox congratulated CMPP on 15 years of professional excellence in medical publications, and discussed how CMPPs have demonstrated expertise, proficiency, commitment and leadership. There are over 1,662 CMPPs worldwide, represented across six continents, with several new countries included this year. The importance of CMPP mentors was emphasised and Fox encouraged CMPPs to consider participating in the mentor program. Fox finished by summarising what 2024 holds for CMPPs, noting the availability of the remote online exam option and the new certification platform to come later this year.

Keynote: The brain science of story and its impact on communication

KEY TAKEAWAY

• The most successful stories start with an ‘unexpected middle’ that lead the reader or audience to ask themselves ‘why’ and find out more; AI focuses on the ‘what’ rather than the ‘why’, and therefore cannot create an effective narrative.

Angus Fletcher (Ohio State University) delivered the first keynote presentation of the meeting, which focused on the brain science of storytelling, and the associated challenges and opportunities for effective communication.

Stories versus data

Fletcher began the session by highlighting that stories are far more powerful than data or information alone. The art of storytelling was championed 2,000 years ago by orators such as Cicero, but has perhaps been forgotten in the current age of data AI.

Evolution of narrative cognition

During the Cambrian explosion 525 million years ago, the brain evolved to identify complex patterns in the environment (computation) and to facilitate creative actions (narrative cognition) as a means of competition and survival. Fletcher emphasised that storytelling therefore evolved not for the purposes of communication, but rather for thinking.

Storytelling in the brain

Fletcher noted that storytelling in the brain is triggered by an unexpected event, which leads it to ask ‘why’ the event happened (causal thinking), followed by ‘what if’ the event continues or happens again (counterfactual thinking). Thus, effective stories begin with an ‘unexpected middle’ that leads the reader or audience to ask themselves ‘why’, before moving back to the beginning, and then to the end. Notable examples of successful storytelling that follow this trajectory include Star Wars: Episode IV – A New Hope, and Nike’s 1988 Just Do It campaign.

Unexpected middles

Unexpected middles compel the reader to ask themselves ‘why’ by using threats to stimulate fear, or by using opportunities to stimulate curiosity. While fear is most effective at inducing behavioural changes in the short term, curiosity is more powerful than fear over a longer period. The ‘Know Your Lemons’ campaign, for example, successfully uses curiosity rather than fear to increase awareness of early symptoms of breast cancer.

Unexpected middles compel the reader to ask themselves ‘why’ by using threats to stimulate fear, or by using opportunities to stimulate curiosity. While fear is most effective at inducing behavioural changes in the short term, curiosity is more powerful than fear over a longer period.

Fletcher highlighted that because AI uses computation rather than narrative cognition, it cannot ask ‘why’ or notice unexpected middles, and therefore will never be able to replace humans to develop original communication.

Storytelling in medical writing

Within the field of medical writing, Fletcher mentioned two famous works that both start in the middle of a story: ‘Merck’s 1899 Manual’ by Merck & Co., and ‘The Principles and Practice of Medicine’ by Sir William Osler that was first published in 1892 – the former stimulating fear in the reader with the opening sentence ‘Memory is treacherous.’, and the latter inspiring curiosity with incomplete information. Fletcher concluded that although modern medical writing publications may be restricted in terms of structure and format, the writer can still introduce an unexpected middle at the beginning of the introduction, thus motivating the reader to continue on to the data, or the ‘what’. Medical writers should aim to be storytellers when developing publications, rather than simply providing the reader with information and data.

Humanising the patient experience with insights derived from novel measures of QoL

KEY TAKEAWAY

• Novel non-traditional approaches are useful for assessing and measuring patient experience and QoL and include social media listening and the use of AI; however, it is crucial to ensure that these approaches are truly reflective of the patient experience.

During this plenary session, Mary James (Real Chemistry), Theresa Schmidt (Real Chemistry), Michael Mittelman (patient advocate), and Mary Michael (Otsuka) discussed non-traditional approaches to assessing and measuring patient experience and quality of life (QoL).

Mittelman opened the session by sharing his experience of being a kidney transplant recipient from a living donor and his subsequent role as caregiver to the donor (his mother). Although there is no shortage of data on the QoL of transplant recipients, Mittelman highlighted that important evidence gaps exist, such as the psychosocial impact of being a recipient from a living donor, and the experiences of living donors themselves.

Defining patient experience

In order to measure patient experience, it is important to be clear about the concepts in question, which broadly encompass patient experience of care and health-related QoL. According to the Agency for Healthcare Research and Quality, patient experience of care “encompasses the range of interactions that patients have with the healthcare system, including their care from [providers and payers]”. The US Food and Drug Administration (FDA) defines health-related QoL as “a multidomain concept that represents the patient’s general perception of the effect of illness and treatment on physical, psychological, and social aspects of life”.

The ‘whys’ and ‘hows’ of measuring patient experience

It was highlighted that the measurement of patient experience and QoL is not only important for research purposes (for example, to assess the efficacy of an intervention in improving patients’ lives) but also to understand barriers to access to care (such as discrimination and marginalisation), and thereby improve clinical care, inform value-based care, and help ensure that communication strategies are as effective as possible.

Traditional methods of measuring patient experience and QoL have relied on clinician-administered assessments, such as Activities of Daily Living, and patient-reported outcome measures. These typically comprise structured surveys and questionnaires to capture information on, for example, symptoms, experience of care, treatment satisfaction, and QoL. Qualitative methods may also be used, such as interviews, focus groups and advisory boards. However, being designed by researchers, these traditional methods do not necessarily reflect what is most important to patients. Non-traditional approaches are therefore required to illuminate the “patient as a person” and help clarify more precise and meaningful care outcomes.

Non-traditional approaches encompass social media listening, AI and natural language processing (NLP), and the generation of synthetic patient cohorts (based on claims data) and AI patient personas. The panel noted that social media listening can be particularly valuable in helping to understand the language/lexicon and sentiment of a particular patient community, highlighting that “we in the [medical publishing] community have an opportunity and obligation to become culturally humble to the community we serve”. Generative AI can also be helpful in producing imagery to convey patient feelings and sentiments that are difficult to articulate into words, which may be useful in the medical communications setting. A live poll of the audience revealed that some attendees are already using non-traditional data sources and methods, particularly social media listening and NLP.

Social media listening can be particularly valuable in helping to understand the language/lexicon and sentiment of a particular patient community.

A note of caution

Although the panel members acknowledged the many potential benefits of non-traditional measures (including the ability to capture unsolicited patient perspectives, address data scarcity, and amplify previously unheard voices), they also highlighted important challenges and risks. These include lack of transparency, standards for data quality and methods, and regulation regarding privacy, and the potential for ‘AI hallucination’, whereby AI may create anomalous inferences that are unrelated to real human experience. In particular, it is crucial that patients are themselves involved in the development and use of these new approaches; an issue being championed by The Light Collective, which has initiated the Patients AI Rights Initiative to advance the rights, interests, and voices of patients in health AI. 

Roundtable sessions

Attendees then had the opportunity to participate in roundtables, which covered the following topics:

• Going back to the start: rethinking the scientific communication platform for better evidence storytelling
• Exploring the role of consensus methods in biomedicine
• Where are the data? Challenges and controversies in data sharing
• Catching the AI wave: balancing innovation to harmonise digital omnichannel storytelling
• Promoting diversity, equity, and inclusion in the global age of medical communications
• Leveraging actionable insights for effective medical communication strategies
• Open forum discussion for newer medical publication professionals
• Local, regional, global publication plans – challenges to effective alignment
• Tackling author challenges in the changing world of publications
• Science friction: using conflict to enrich stories in HEOR publications
• Evoking patient trust in rare disease: utilising digital opinion leaders (DOLs) to drive global clinical trial enrolment
• How will increased use of real-world evidence by regulators influence medical publications?
• Digital engagement in publications​: optimising reach, engagement, and impact​
• Driving your career and setting yourself apart

My voice, my story: embracing the patient experience to elevate publications

KEY TAKEAWAY

• Patient engagement and patient experience data are essential to ensure that patient perspectives are incorporated into evidence generation and dissemination.

In this moderated panel discussion, Wilson Joe (Lumanity), Yaswant Dayaram (Bayer), Hayley Chapman (Patient Focused Medicines Development [PFMD]), and Jill Feldman (patient advocate) explored strategies to integrate patient perspectives into the development and execution of evidence generation and dissemination.

Feldman began by sharing her experience as a lung cancer patient and advocate, highlighting the importance of remembering that “behind every data point is a patient and family whose lives are profoundly impacted”. Those involved in research and evidence dissemination must therefore engage with patients and caregivers to obtain reliable information on the impact of a disease, condition or treatment on patients’ lives. Moreover, the language used to disseminate information is crucially important; for example, when describing side effects, terms such as ‘tolerable’ or ‘acceptable’ minimise the patient perspective by not truly reflecting the lived experience of patients.

How can we effectively engage with patients?

Chapman explained that PFMD is a non-profit, global, disease-agnostic organisation that focuses on amplifying the voice of patients, enabling them to provide a meaningful and valuable contribution to healthcare decision making. The organisation provides free tools and resources to facilitate effective patient engagement. The benefits of patient engagement are far-reaching: in the clinical trial setting, it can speed up recruitment, decrease the need for protocol amendments, and help optimise retention; most importantly, patient engagement can result in better health outcomes for patients and the wider community. Consequently, investors and other stakeholders are increasingly questioning companies about how they engage with patients to identify unmet needs and what is most important to the patient community.

Patient experience data (PED)

PED provide information about patients’ experiences, perspectives, needs, and priorities, and can include data from patient-reported outcome measures, patient preference/satisfaction studies, and QoL measures. It was stressed that PED and patient engagement are distinct but go hand-in-hand: as the PFMD states, “Patient engagement can support PED to ensure patient-centricity of measurement, contextualisation of data, increased understanding of the evidence generated, and the co-creation of the design, generation, collection, and analysis of PED”.

Patient engagement across the lifecycle of drug development

The panel highlighted that patient engagement is needed from the beginning of product development and throughout its lifecycle, something that is becoming increasingly recognised within the medical publications industry. The recent American Association for Cancer Research (AACR) Annual Meeting 2024 included a session entitled ‘Strategies to effectively communicate science to the public‘. Moreover, the importance of using patient-sensitive, respectful and inclusive language is increasingly being advocated by scientific organisations, such as the International Association for the Study of Lung Cancer (IASLC Language Guide) and the National Comprehensive Cancer Network (NCCN Language Guidance), as well as medical journals, such as The Lancet Oncology, which recently published an editorial entitled ‘Patient first; person first’.

Case study: plain language summary publications (PLSPs)

Bidirectional dialogue between patients and industry stakeholders is essential, and the generation of PLSPs was used to illustrate the importance of patient engagement from the earliest stages of the publication process. Potential patient authors can be identified via patient organisations and advocacy groups. Once identified, open dialogue between the patient authors and publication team is required, so that patient authors are fully aware of the importance of their involvement to the wider community, what to expect from the publication process (including the time commitment), and the tenets of good publication practice.

Embracing the paradigm shift

It was highlighted that recent years have seen a paradigm shift towards viewing patients as research partners rather than passive trial participants, something that should be embraced to ensure that patient perspectives and experiences are incorporated into every aspect of evidence generation and dissemination. Language is powerful: how you tell a story is important and words matter.

Language is powerful: how you tell a story is important and words matter.

Transformative dynamics: the vital role of scientific platforms in humanising science and shaping omnichannel evolution through collaborative partnerships within publications and medical affairs teams

KEY TAKEAWAY

• SCPs are a vehicle for leadership and change, designed to ensure a clear and accurate story is communicated, and provide cross-functional connection and alignment.

Scientific communication platforms (SCPs) may be defined as a strategic framework to ensure the consistent communication of a product’s evidence-based data and scientific narrative. During this session, Charlie Buckwell (IPG Health Medical Communications), David Hogben (Complete HealthVizion-IPG Health), Marie-Ludivine Chateau-Degat (Pfizer) and Jennifer Ghith (Pfizer) discussed the increasingly important role of SCPs in an evolving landscape.

Humanisation

Humanisation focuses on bringing the information held within a SCP to life and ensuring that it is connected with the overall scientific narrative. Understanding the audience is also a key factor, with respect to their persona, educational needs and how they want to access the information. This should allow the scientific statements to be connected to the target audience in an optimal approach and maximise the ability of the SCP to drive the overall medical communications plan. Consequently, the SCP will be more accessible to internal and external audiences, with the aim of ultimately shortening or closing the clinical care gap by aligning decision making by healthcare professionals (HCPs) to the scientific data.

Integration

The SCP needs to be integrated across the whole organisation to achieve a strong and unique voice across all stakeholders. Audience opinion indicated that SCPs are perceived to be important by publications and medical affairs teams, but perhaps less so by other industry functions (eg, commercial, medical information, R&D, and local market teams). The medical affairs and publications teams consider SCPs to be a foundational and guiding principle. The SCP can help position medical affairs and publications team members as leaders in communicating/sharing key medical information, making them integral members of cross-functional teams. It is important for medical affairs and publications teams to ensure that the global narrative is sufficiently cascaded to local teams to provide consistent communication. SCPs should be considered the indisputable centre of medical information and the anchor for achieving external acceptance of the narrative by HCPs.

It is important for medical affairs and publications teams to ensure that the global narrative is sufficiently cascaded to local teams to provide consistent communication.

Innovation and transformation

SCPs are generally developed within PowerPoint or digital platforms; it is important to think about possible unmet needs and develop the right tool for the right question. Technology is constantly providing newer capabilities that can be used and leveraged to facilitate greater understanding. AI and technology have the potential to change SCPs in several ways, such as improving speed/efficiency, integration and standardisation. However, some people may have less understanding of AI and technology and attach less value in their potential utility for SCPs. The adoption of AI and technology may depend on the overall needs and position in the lifecycle of clinical development. To help introduce technology, practical solutions may include working with cross-functional colleagues to allow collective ownership. Ultimately, it’s most important to understand the audience and provide the information they need in the most accessible method. Building SCPs can be time consuming, costly and intense. Technology can help people to find information within SCPs faster and more easily, leading to improved experiences.

The foundations and practical experience of using publication metrics

KEY TAKEAWAY

• Multiple types of publication metrics are currently available, which can be collectively measured and discussed cross functionally to help define the overall impact of publications.

Tomas Rees (Oxford PharmaGenesis), Doreen Valentine (Bristol Myers Squibb), Jonathan Lee (Takeda) and Sam Cavana (Taylor & Francis Group) discussed currently available publication metrics and their potential applicability.

Fundamentals of metrics

Metrics can help us understand the impact of publications and such learnings can be used to help drive strategic decisions and improvement. Publication metrics should be considered at the publication planning phase. Publication metric types include quantitative (eg, downloads and citation counts), qualitative (eg, sentiments, preferences and behaviours), mixed methods (eg, social media and omnichannel), and indicators associated with journals and congress targets.

Dimensions of impact

Engagement with a publication can be defined by reach, readership, activation and translational impact. To enhance reach (the number of people with the potential to read a publication), journal readership or congress attendance and features such as open access should be considered. To enhance readership (the number of people who have read a publication), we should consider multimodality options to enhance readability (eg, graphical abstracts, research in context). To enhance activation (the number of people who have interacted with a publication), multiple social media and audio/video channels should be taken into account. To enhance the translational impact of a publication on clinical practice, such as incorporation into practice guidelines or pathways, awareness initiatives for healthcare (eg, ad boards) or patient communities (eg, advocacy groups) should be considered. Thinking about these factors may help to make a publication more enduring and potentially impact clinical practice.

How to track and use metrics

Metrics should be used to inform journal selection strategy, proactively plan additional features/extenders and consider potential leverage from similar/complementary datasets. Article metrics may help to contextualise the impact and demonstrate the performance of a publication. Metrics can be considered as a tool and framework that are essentially a continuous loop – the outputs (post-publication impact of previous publications) can be used to inform the inputs (new publication tactics).

Metrics: which ones

Citation impact factor is the gold-standard metric and carries a lot of weight, but alternative metrics (eg, digital eminence algorithms) may also be of value. Overall metrics must be fit for purpose – a metric for measuring success may differ for different article types (eg, health economics versus Phase III clinical trial data). From the publisher’s perspective, downloads are probably the most important metric, in addition to citation metrics such as Altmetrics and CiteScore. The timeframe is also important – 12 months is probably most useful, especially for publication extenders which may be introduced at staggered timepoints.

Overall metrics must be fit for purpose – a metric for measuring success may differ for different article types.

Actionable insights: practical application of metrics

Publishers are keen to receive feedback on which metrics are important, eg, for different article types and different therapy areas. The panel advised publication teams to speak to publishers when planning a publication and not simply rely on tools which provide measures such as acceptance rate or time for review.

It is important to establish a process to share and discuss metrics cross functionally. Within the cross-functional framework, it is important to tailor how and what you communicate to the audience. Additionally, although metrics can be dry, they can help inject life into a story and emphasise how a specific publication has made a difference. Metrics are not a single point in time – to help measure the success of a publication, it is important that different pharmaceutical functions and publishers consistently collect all possible metrics data to help define a baseline – without a baseline it’s difficult to measure success. 

Guided poster tour: Trust and Accessibility

Attendees on Day 1 had the opportunity to attend a guided poster tour of the following posters that focus on Trust and Accessibility:

• Are HCPs leaving X? Analysing the evolving social media landscape
• Designing inclusive and accessible patient materials
• How does the inclusion of supplemental materials impact delegate engagement metrics? Lessons from a QR-coded poster database
• It takes many good deeds to build a good reputation 1: biopharma company attributes that are important to HCPs and consumers

Why not also read our summaries of  Day 2 and Day 3 of the meeting? 

——————————————————–

Written as part of a Media Partnership between ISMPP and The Publication Plan, by Aspire Scientific, an independent medical writing agency led by experienced editorial team members, and supported by MSc and/or PhD-educated writers.

——————————————————–

KEY TAKEAWAY

• As the volume of online content created by generative AI grows, altmetrics must adapt to this changing environment or risk becoming redundant.

Altmetrics, which measure the online attention that scholarly literature receives, have proliferated in recent years. Measures such as the Altmetric Attention Score (AAS) are now commonplace on journal webpages, alongside traditional citation analysis. However, as the digital landscape shifts towards content developed by generative artificial intelligence (AI), will altmetrics need to adapt or perish? A recent article by David Stuart for Research Information examined this conundrum.

Generative AI: the challenges for meaningful metrics

Stuart points to various challenges associated with the rapid increase in use of generative AI. Its capabilities can be harnessed to make previously time consuming tasks straightforward. Alongside the benefits this brings, there is an increased risk of:

• purposeful metric manipulation – generative AI could be used to generate large volumes of online mentions, each appearing to originate from a separate user
• AI being used to create and endorse work produced by paper mills
• mentions becoming misattributed by AI to derivative, rather than original, works, skewing metrics data.

Adapt or perish

While it could be argued that this points to the possible demise of altmetrics, we can also look to the words of Okakura Kakuzo: “the art of life lies in constant readjustment to our surroundings”. To this end, Stuart suggests altmetrics will need to adapt if they are to keep pace with a new and changing online environment. For example, as more and more online content is developed by generative AI, one possibility is that social media will move towards the more widespread use of subscription models and verified accounts, with restrictions placed on content generated by other sources. Such changes could mean that a modified information base is available to feed into new or adapted metrics. As the online world continues to adjust to the capabilities of generative AI, so too must the way we use metrics.

Altmetrics will need to adapt if they are to keep pace with a new and changing online environment.

————————————————

KEY TAKEAWAYS

• Altmetrics offer valuable insights beyond numerical scores.
• Researchers can use them to discover suitable platforms to enhance research visibility, promote research sharing, connect with experts, and uncover opportunities for collaboration.

Altmetrics offer an alternative approach to evaluating research impact that extends beyond traditional citation-based metrics, such as journal impact factor and the h-index. A high Altmetric Attention Score (AAS) indicates that research is actively discussed and shared across diverse online platforms, such as news outlets, social media, or in policy documents. However, there is more to altmetrics than assessing impact alone.

In a recent post on the LSE Impact Blog, Andy Tattersall explores 5 ways researchers can leverage altmetrics to gain useful insights into the audiences for their research and find channels to extend the reach of their work.

1. Connect with the right audiences: Altmetrics can help identify blogs covering particular research topics to aid in the selection of suitable platforms for guest post contributions.
2. Incentive to take action: A low AAS can signal that a research output is receiving little online attention. Authors may use this as motivation to promote their work.
3. Encourage open access: Altmetrics provide a compelling reason to share research through open access repositories to increase the likelihood of media coverage and blog discussions.
4. Connect with experts: Altmetrics can be used to identify suitable expert speakers for academic events by gauging which potential speakers receive media or blog coverage for their research outputs.
5. Create opportunities for collaboration: Altmetric tools like Overton and BMJ Impact Analytics allow users to discover if and how their research is cited in policy documents and clinical guidelines, paving the way for collaborations with entities such as charities or governmental organisations.

Although altmetrics have been around for some time now, Tattersall believes that they may still hold untapped potential. Researchers are encouraged to capitalise on this potential to maximise the impact of their work and uncover new opportunities in today’s dynamic academic landscape.

Although altmetrics have been around for some time now,
Tattersall believes that they may still hold untapped potential.

————————————————

Altmetrics

1. Pharmacovigilance: Social media monitoring of adverse events could complement traditional surveillance methods and has already been explored in the WEB-RADR.
2. Patient experience: Providing digital and social media tools to better engage with patients and carers may be the future as pharma becomes more patient centric. However, it is suggested that this content and interaction will need to be assessed as accurately and transparently as any other activity in drug development and delivery.
3. Product marketing: Social media ‘influencers’ may be engaged as advertising partners to promote new drugs or medical devices to their followers.
4. Clinical practice: In addition to using digital networking communities for sharing clinical opinion, physicians are also active on social media, offering an opportunity for pharma to engage with physicians through the use of social media tools.
5. Regulatory compliance: Some regional codes of practice exist to ensure pharma companies who do engage in social media activities remain compliant with industry regulations.
6. Drug development: Social media can influence the drug development process in orphan and rare diseases by allowing study of disease progression at earlier stages, increasing awareness both of the disease and of orphan drug developers, and by facilitating patient recruitment, retention and monitoring during the clinical trial process.
7. Scientific meetings: The use of social media by physicians increases substantially during scientific meetings, providing an opportunity to facilitate real time scientific discussion and immediate dissemination of key information to a large global audience.

While there are many potential applications for social media monitoring, the pharmaceutical industry remains reluctant to engage in these activities.

While there are many potential applications for social media monitoring and measurements, the pharmaceutical industry remains reluctant to engage in these activities, citing the risk of circulating fake or fraudulent research and misinterpretation or oversimplification of genuine findings as concerns. Nevertheless, Delahunty identified some priority areas for future engagement, including increased interaction in rare diseases and harnessing social media to facilitate clinical trial recruitment, but conceded that existing barriers are likely to prevent social media from supplanting more traditional methods currently used for these activities.

—————————————————–

—————————————————–

In academia, decisions regarding hiring, promotion, and tenure are often based on traditional evaluation metrics, such as citation counts and grant allocations, which have been found to disproportionately favour men. Altmetrics, on the other hand, measure the amount of online attention a paper receives on digital platforms, such as news sites and social media, and many journals now publish the Altmetric Attention Score (AAS) for individual journal articles.

Thanks to the broad nature of online audiences, altmetrics could effectively democratise research evaluation. To test this theory, the study by Fortin et al, published in Scientometrics, analysed the AAS of over 200,000 articles published between 2011–2018 in 7 major journals/preprint servers:

• Science
• PNAS
• PLOS One
• Nature
• Cell
• NEJM
• bioRχiv.

The authors found that articles by male and female authors generally received equal online attention one year after publication. The exception to this was in 2018, when marked gender bias against women was found in Science, the reasons for which are unknown.

By using altmetrics together with traditional citation-based metrics, a fairer, more equitable system for performance evaluation could be established for academics, whatever their gender.

The authors state that further research is required to determine why altmetrics did not show evidence of substantial gender bias for most journals. Understanding the underlying reasons may help to identify mechanisms that could be applied to eliminate bias from other performance assessment tools. The authors conclude that by using altmetrics together with traditional citation-based metrics, a fairer, more equitable system for performance evaluation could be established for academics, whatever their gender.

—————————————————–

—————————————————–

Preprints are used to expedite research findings into the public domain, but their inherent uncertainty is not well understood outside the scientific community. During the COVID-19 pandemic, the need for credible and accessible heath information caused a dramatic increase in the use of preprints by media outlets. However, a recent study published in Health Communication found that almost half of 457 news articles from 15 media outlets that cited COVID-19-related preprints did not frame the preprint research as uncertain in any way.

Using two preprint servers (medRxiv and bioRxiv), Altmetrics and coding, Dr Alice Fleerackers and colleagues analysed the use of COVID-19 preprints in the media coverage of the early months of the pandemic, and assessed if and how the uncertain nature of this type of research was explained. Although almost all the news stories hyperlinked to a preprint, there was great variation in how the preprint content was covered. While 80.5% identified the content mentioned as research, nearly 20% included a hyperlink with no explanation of where it linked to or indication that it linked to a preprint.

Regardless of the use of hyperlinks, more than half of stories highlighted the scientific uncertainty associated with preprints, using a variety of framing devices, such as:

• explaining that the content was unreviewed
• identifying the content as a preprint
• adding that further verification was needed
• noting that the work was preliminary.

The authors speculate that media outlets may avoid adding explanations around preprints because simply using the word ‘research’ gives credibility to the reported content, and there may be reluctance to emphasise any uncertainty. They may also want to avoid alienating audiences who are not familiar with evaluating scientific content. However, it is encouraging that several outlets in the study reported on preprint research with adequate explanation on the uncertainty surrounding science and peer review. As the pandemic has reminded us, it is critical that the public have access to scientific research, and that it is reported by media outlets in an accurate and transparent way.

——————————————————–

——————————————————–