KEY TAKEAWAYS

• The journal eLife has abolished accept/reject decisions and established a new model for research dissemination.
• All papers assessed by the journal will now be published as reviewed preprints, featuring open peer reviews and a further, standardised assessment.

Following eLife’s 2021 policy to make preprinting a mandatory requirement ahead of submission, the journal has recently stopped accepting or rejecting papers after peer review. Instead, eLife will publish ‘reviewed preprints’, which they hope will bring together the diligence of peer review and the fast pace of preprint posting.

The reviewed preprint format will see all papers that are accepted for review posted online. They will appear alongside open peer reviews and an eLife assessment, which summarises editor and peer reviewer opinions on the paper’s impact and strength of evidence. These assessments are tailored to non-experts and use standardised language, akin to a grading scale. Authors also have the opportunity to include a response to the reviewer comments and assessment. After posting, authors can choose to take no further action, submit a revised version of the paper, or deem the article the final version of record to be indexed on PubMed.

eLife have shared various benefits that they believe this approach will offer:

• increased transparency around author–reviewer communication
• improved communication of intricate reviewer thought processes
• greater accessibility for non-expert readers
• greater autonomy for authors
• increased emphasis on paper content, as opposed to publisher title
• faster mobilisation of scientific content.

Editor-in-Chief Michael Eisen has described the major shift in approach as “relinquishing the traditional journal role of gatekeeper” in favour of a system tailored to scientists rather than publishers.

eLife editors expressed their hopes that this transparent review model will encourage readers to judge a paper by its content as opposed to the journal it is published in.

In a recent editorial, eLife editors expressed their hopes that this transparent review model will encourage readers to judge a paper by its content as opposed to the journal it is published in, and that it will “become the norm across science”.

—————————————————–

KEY TAKEAWAYS

• Journal impact factor is a poor predictor of the quality of peer review.
• Implementing open peer review and reviewer training would facilitate the assessment and quality of peer review.

Journal impact factor is often considered an indicator of quality, but does a high impact factor necessarily equate to high quality peer review? In a recent Nature News Q&A article, Dr Anna Severin discussed how her team used artificial intelligence to investigate whether there is any link between the two. Their results were also reported in a recent preprint on arXiv.

As Dr Severin explained, ‘quality’ peer review is difficult to define. Authors want suggestions for improvement, while editors want recommendations on whether to publish. As a result, Dr Severin and her team developed the following proxy categories for quality:

• Thoroughness: comments made on the materials and methods, presentation and reporting, results and discussion, or the importance of the article.
• Helpfulness: comments relating to praise or criticism of the article, giving suggestions for improvement, or providing further examples.

The team randomly selected 10,000 peer review reports submitted to more than 1,600 medical or life science journals. A random sample of 2,000 sentences were then coded to the above categories. The team used this information to train machine learning models to predict categories for over 187,000 sentences across the reports.

Overall, peer reviewers from higher impact factor journals focused on methodology, but spent less time suggesting improvements and providing examples than reviewers from lower impact factor journals.

Reports in higher impact factor journals tended to be longer, and reviewers were more likely to be from Europe or North America. Although there seemed to be a link between impact factor and peer review quality, any differences were modest, and variability was high. As such, the team concluded that impact factor is “a bad predictor for the quality of review of an individual manuscript”.

The team concluded that impact factor is “a bad predictor for the quality of review of an individual manuscript”.

Dr Severin recognised the limitations of using artificial intelligence, but considered this to be a first step in assessing peer review quality in a systematic and scalable way, and called for the following actions:

• Reviewers should be trained and given clear guidelines.
• Peer review should be open rather than confidential.

—————————————————–

KEY TAKEAWAYS

• As preprint publications increase in popularity, mechanisms to encourage transparent public review are needed.
• The FAST principles provide a framework for use by authors, reviewers, and the wider community to foster engagement in preprint discussions.

The publication of preprint articles has gathered pace in recent years, accelerating rapidly during the COVID-19 pandemic. A key advantage of preprints is that they can be scrutinised by a diverse audience ahead of submission to traditional scholarly journals. Despite the benefits of public feedback, just 5–10% of preprint articles on bioRxiv and medRxiv receive publicly accessible comments, with many reviewers preferring to provide feedback privately.

To help facilitate rapid and constructive preprint feedback, ASAPbio established a Working Group to develop best-practice guidance for public commentary and engagement with open preprint discussions. In their guest post on The Scholarly Kitchen, Sandra Franco Iborra, Jessica Polka, and Iratxe Puebla of ASAPbio summarised the FAST principles for preprint feedback that were developed.

The 14 FAST principles are grouped into 4 central themes:

• Focussed: comments and feedback should focus on the scientific content and not the suitability for potential target journals.
• Appropriate: reviewers should reflect on their potential biases and engage in scientific discourse respectfully and with integrity.
• Specific: feedback should be candid, assess a study’s claims against the data presented, and be clear on whether issues identified are major or minor.
• Transparent: reviews should be as open and transparent as possible and credit any co-reviewers. Those not comfortable signing their review can disclose their background or expertise alongside their comments.

The authors highlight that the unique features of the FAST principles mean that they are relevant to all involved in feedback, including journals, authors, and the wider community. Importantly, they are not intended to replace the reviewer guidance already provided by traditional scholarly journals, but rather complement it, to facilitate communication between authors and peer-reviewers and help promote positive behaviours for peer-reviewers.

The authors hope that the FAST principles will contribute to a broader conversation on the review process, helping produce a more positive and diverse culture.

The authors propose that encouraging public review of preprint articles could help journals expand and diversify their reviewer pool by identifying junior researchers and those located across broader geographical regions.

There has already been a move by some journals to incorporate preprint reviews into their editorial processes. Both Review Commons and Peer Community In Registered Reports (PCI RR) provide journal-independent preprint review, which is accepted by several affiliated journals. The FAST principles could be used to support this process, defining the expectations for preprint reviews that will ultimately be acceptable to scholarly journals.

The authors hope that the FAST principles will contribute to a broader conversation on the review process, helping produce a more positive and diverse culture.

—————————————————–

KEY TAKEAWAYS

• Disclosing reviewer identities can improve peer review transparency, but its risks and benefits remain unclear.
• Signed peer review offers scholars an opportunity to claim credit for their work, but will not in itself remove bias from the process.

Despite the growing support for improving transparency in the peer review process, attitudes towards revealing reviewer identities remain mixed. In their article for the Scholarly Kitchen, Véronique Kiermer and Alison Mudditt explore the arguments for and against disclosing reviewer identities alongside published articles, and discuss learnings from publishers who have implemented different approaches to open review.

The benefits of open-identities peer review have been described in terms of:

• accountability: increasing the transparency of any competing interests
• credit: recognising reviewers’ contribution to this important academic activity
• quality: encouraging more thorough reviews.

However, Kiermer and Mudditt cautioned that the following risks should be considered:

• Bias: masking author and reviewer identities has traditionally been used to counteract biases in peer review.
• Impact on vulnerable researchers: early-stage researchers may be more susceptible to retaliatory actions such as negative subsequent peer reviews.
• Rigor and candour: researchers may be less willing to provide critical assessments or may trade positive reviews with other academics.
• Halo/horn effect: readers might overlay their perceptions of the credibility of the reviewer on the research article, instead of judging the research on its merit.

Various journals have adopted versions of open review. The BMJ identified a small improvement in constructive feedback following the release of reviewer identities; however, this was coupled with decreased willingness to review manuscripts. In contrast to the BMJ data, PeerJ reported that reviewers choosing to disclose their identity gave more subjective and positive reviews, which could suggest that candour is influenced by social pressures.

Nature journals offer their referees the option to be named on published papers (with authors’ permission), but restrict access to the peer review reports. Overall, 80% of Nature articles name at least one reviewer, which likely reflects a desire amongst reviewers to receive credit for their efforts.

Since 2019, authors publishing in PLOS journals have had the option to release their article’s peer review history, including peer reviewers’ names. Subsequently, in 22% of PLOS journal articles at least one reviewer has chosen to reveal their identity. PLOS also observe a strong desire for credit and have enabled reviewers to add certified proof of review to their ORCID records.

Kiermer and Mudditt believe that all stakeholders can benefit from greater transparency in peer review, although there is some ambiguity as to the value of naming individual reviewers, and a limited understanding of its impact. They stress that on its own, disclosure of reviewer identities is unlikely to overcome the bias inherent in the peer review process, and that greater diversity and representation among reviewers and editors is needed. Further exploring the impact of social identities could help reduce bias and increase trust and transparency in peer review.

—————————————————–

At the annual meeting of ISMPP this year, the way that publishers engage with preprints was identified as a key trend for the future of medical communications. In their article for the Scholarly Kitchen, Roger Schonfeld and Oya Rieger discuss this subject in more detail, drawing on a recent publication on developments in this area for Ithaka S+R.

Preprints are viewed by many as the path towards a more efficient and accelerated publishing machinery. It is now observed that not only are publishers increasingly willing to engage with preprint servers, they are also establishing integrated preprint platforms into their manuscript submission workflows.

An interesting new dimension of journal-integrated preprint sharing is the function to link the preprint with its publication status. For example, Springer Nature’s In Review service allows the progress of a manuscript to be tracked through per review, after being shared on the preprint server Research Square. If the manuscript is not accepted for publication, the preprint remains on the platform but is disassociated from Springer Nature and the submission process. While if accepted, a link from the preprint to the published article is included.

Wiley journals are offering a similar preprint service (Wiley Under Review) that enables authors to deposit and track their manuscript on the Authorea preprint server in parallel to the journal editorial review process. This service was introduced at the start of the year as a pilot and is now being expanded to 37 journals with a focus on expediting the dissemination of pandemic research.

As part of its move into preprints, Elsevier has acquired the SSRN eLibrary and developed the SSRN-compatible First Look service, which enables journals to create a branded space for preprints on SSRN. Taylor & Francis has taken the preprint process one step further with its acquisition of F1000 Research, in which a preprint is ‘published’ and then subject to open peer review and ‘post-publication’ author revision.

Challenges to this cross-platform integration include the requirement for interoperability, whereby the publisher’s submission platform(s) needs to be compatible with the preprint system. This also raises interesting ethical questions when either submission platforms or preprint servers have investment from competing publishing houses. A case in point is the Elsevier purchase of the Aries submission platform that is utilised by Springer Nature. Encouragingly, Aries now supports deposition of manuscripts via the Springer Nature Research Square functionality, showing that cross-publisher interoperability is possible.

While there is much enthusiasm for the potential of preprints to expedite research findings into the public domain and to level the financial playing field of scholarly research, some have raised concerns about potential harm in the sphere of biomedical publishing. Concerns cited include lack of identifying/tracking metadata, or indeed the implications of assigning digital object identifiers (DOIs) to preprints, the publication of unvetted claims and the need to emphasise the preliminary nature of preprints so that they are not reported by news media as established information. The introduction of a more rigorous code of practice may be a prudent next step for the preprint revolution.

——————————————————–

Summary by Julianna Solomons PhD, CMPP from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

In their own words, Cochrane systematic reviews are “the benchmark for high-quality information about the effectiveness of health care.” However, Cochrane has been criticised over its lack of transparency in the past, and Hilda Bastian, one of its founding members, thinks there is room for improvement in minimising bias.

The protocol is a crucial component of a systematic review, and is believed to be the key to minimising bias in the results. However, Bastian argues that there is potential for bias in the protocols themselves. For example, there is a section called ‘how the intervention might work’ in the background for Cochrane protocols, which is intrinsically biased by assuming the intervention could work, without actively considering if it could do harm. And even if a perfectly unbiased protocol did exist, if the results from the study contradict the preconceived beliefs of its readers (especially around a controversial topic), then the integrity of the protocol is likely to be criticised anyway.

Bastian proposes that researchers should engage with these potential controversies and be open about the process. Taking inspiration from various systematic review bodies, such as the Agency for Healthcare Research and Quality (AHRQ) and the Cochrane Handbook, her proposed solutions include:

• justifying methodological and analytical choices within the protocol itself, rather than in the discussion (or not at all)
• planning sensitivity analyses on contested aspects
• keeping resulting data open and detailed
• ensuring that every stage of the systematic review process has involvement and oversight from impartial people with methodological expertise.

Focusing on the last step, Bastian is in the process of establishing an Independent Advisory Group for an ongoing update to a Cochrane review. She sees this as a commitment to genuine accountability and responsiveness, and a mechanism for public and open discussion of the protocol. She believes that these steps are critical to ensure ongoing confidence in the integrity of the systematic review.

——————————————————–

Summary by Robyn Foster PhD from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

Despite peer reviewers being an integral cog in the biomedical publication machinery, there is little consensus on what their function involves. To address this question, Ketevan Glonti and colleagues, examined articles and grey literature for statements describing the roles, tasks and competencies of peer reviewers of biomedical journals.

The authors’ findings, which were published in BMC Medicine, revealed that:

peer reviewers were most commonly described as performing the role of unbiased and ethical professionals, and skilled critics.

Other statements identified peer reviewers as upstanding members of the scientific community who could be relied upon to uphold standards of scientific communication, as highlighted by the following themes:

• reliable professionals
• dutiful/altruistic towards scientific community
• respectful communicators
• self-critical professionals
• proficient experts in their field
• advocates for author/editor/reader
• gatekeepers
• educators

The remaining statements described responsibilities associated with maintaining journal standards, falling under the themes of ‘being familiar with journal’, and ‘advisors to editors’.

Statements relating to peer reviewers’ tasks were grouped into the following six categories, listed in order from the most to the least common:

• assess and address content for each section of the manuscript
• organisation and approach to reviewing
• make general comments
• assess manuscript presentation
• provide recommendations
• address ethical aspects

The authors conclude that the large number of roles and tasks expected of peer reviewers warrants further clarification to avoid overburdening these critical contributors. The study findings indicate heterogeneity and a lack of clarity in journal expectations during peer review, and the authors speculate that a consensus in this area could lead to an improvement in journal article quality. Indeed, the authors point out that some peer reviewer tasks are duplicative of those of the journal editor, while the number of low-quality publications in the literature suggests that certain aspects of quality control are being missed by both parties. The authors point to the lack of formal training received by peer reviewers as a flaw in the system and advocate open peer review as a valuable potential training opportunity.

——————————————————–

Summary by Julianna Solomons PhD, CMPP from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

Firstly, the authors identify 7 key forms of open review:

1. Open identities: identities of authors and reviewers are known to one another;
2. Open reports: peer review comments are published alongside an article;
3. Open participation: involvement of the wider community;
4. Open interaction: discussions between authors and reviewers;
5. Open pre-review manuscripts: manuscripts are available prior to peer review;
6. Open final version commenting: the final version can still be reviewed and commented on;
7. Open platforms: the review is conducted by an organisation other than the publisher.

They go on to describe how each form of review is associated with both advantages and disadvantages. For example, ‘open identities’ may increase accountability, lead to a more thorough review and allow potential conflicts of interest to be identified, but could also discourage reviewers from voicing strong criticisms.

Regarding specific guidance, which was created in close consultation with a group of experts, Ross-Hellauer and Görögh advise editors to identify which elements of open review they wish to utilise and recommend that editors set clear goals outlining what they hope to achieve by implementing such systems. The authors highlight the need to consider the technological feasibility and cost implications of open peer review and suggest that new processes are made optional initially or introduced in a phased manner. Communication and engagement with the community, and identification of academics to advocate open peer review are also recommended. In addition, publishers are advised to evaluate performance, by measuring review quality and acceptance rates.

The authors note that open peer review processes are becoming adopted by a growing number of journals and it is their hope that these guidelines will ‘prove useful in setting expectations and guiding best-practice’.

—————————————————–

Summary by Debbie Sherwood BSc from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

The authors assessed referee behaviours at five Elsevier journals that published peer review reports as articles (anonymous or signed) and compared these with behaviours at five journals with traditional review processes. Encouragingly, referees’ willingness to review, recommendations and turnaround times were unaffected by the open process, although younger and non-academic reviewers were slightly more likely to accept an invitation to review and returned reports that were more positive and objective. These findings are broadly in keeping with those published in a previous report from Genome Biology, which found that transparent review did not compromise the peer review process following a year-long trial. Of note, only 8.1% of reviewers agreed to sign their reports in the Elsevier study, with writers of more positive reports keener to do so, leading the study authors to conclude that the “the veil of anonymity is key for open peer review”.

In another recent pilot, the publisher Wiley launched an automated, scalable, transparent peer review platform in collaboration with Publons and ScholarOne, aiming to increase accountability, provide insight into the editorial process, and recognise the work of editors and reviewers. Clinical Genetics was the first journal to join the initiative, in which authors can opt for open peer review when submitting their article and reviewers can choose whether to disclose their names alongside their reports. Each article’s peer review history is openly available, linked to the article, and citable. Since the start of the pilot, there has been a strong demand for open peer review, with 83% of authors selecting this option. However, similar to the findings reported by Elsevier, only a small number of reviewers (19%) have signed their reports.

Ten more journals are joining the Wiley pilot, seven of which previously operated a single- or double-blind rather than transparent review process, reflecting the movement in scholarly publishing to make research more open. Data from such expanding pilots will provide further evidence for the utility of open peer review. However, it looks like peer review can be transparent without compromising the process – with reviewers much more willing for their reports to be published than their names.

——————————————————–

Summary by Beatrice Tyrrell, DPhil from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

The 2019 European Meeting of the International Society for Medical Publication Professionals (ISMPP) was held in London on 22–23 January and attracted more than 300 delegates; the highest number of attendees to date. The meeting’s theme was ‘Scientific Communications in a Fast-Paced World: Fighting Fit for the Future’ and the agenda focused on innovations in data publishing, open access, patient involvement in publications, and the expanding role of the publication professional. Industry newcomers had the opportunity to attend a satellite training session and all delegates were treated to two keynote addresses, lively panel discussions, interactive roundtables and parallel sessions. Delegates also had the chance to present their own research in a poster session.

A summary of the second day of the meeting is provided below for those who could not attend. A summary of the first day of the meeting can be found here.

Early rise evidence boot camp

In one of three parallel sessions, Richard Macaulay (PAREXEL) discussed how the use of real world evidence is beginning to change regulatory procedures, allowing certain therapies to be approved at an earlier stage of the drug development process. Macaulay highlighted the different requirements needed from a payer’s (ie to reimburse) or regulatory perspective (ie to approve a drug). While a regulatory agency’s decision to approve is largely based on efficacy and safety in randomised clinical trials, the payer also requires evidence that it is cost effective. Payers also want to see evidence that a new therapy provides a benefit over the current standard of care, but demonstrating this can be challenging. For example, a greater number of patients may be required to power a trial to show superiority over an active control, compared with placebo. Therefore, other strategies that leverage real world evidence are being employed. For example, real world evidence from early access programmes may be utilised in submissions. Under one such programme, the Early Access to Medicines Scheme (EAMS), patients in the UK can access treatments that have not yet received marketing approval, if there is a clear unmet medical need. Payers are also developing alternative ways to approve drugs, such as a temporary period of reimbursement, during which time real world evidence can be generated to support full approval. The Cancer Drugs Fund, for example, provides interim funding for cancer drugs in England, until sufficient data can be collected for routine commissioning. Another strategy that payers may consider is linking reimbursement to real world outcomes through performance-based pricing. Macaulay advised that companies not only develop processes to generate real world evidence, but also consider how such information should be optimally disseminated when designing publication plans.

In the second half of the session, Matthew Booth (PAREXEL), on behalf of Melinda Daumont (Bristol Myers Squibb), discussed I-O Optimise, a collaborative initiative, that aims to generate scientific insights based on real world data within the areas of non-small cell lung cancer, small cell lung cancer and mesothelioma. The programme consists of three phases covering pre-launch, early post-launch and established treatment use for each indication. Indication-specific protocols have been standardised to enable the cross-evaluation of complementary real world data from geographically diverse regions. Collaboration is instrumental to the success of the initiative. An External Scientific Committee has a key advisory role, guiding agenda and strategy, while a cross-functional core team comprising clinical, development, commercial, and publication leads, meet regularly to provide ongoing feedback. It was noted that dealing with such large datasets presents certain challenges, such as prioritising publication of the data. Another potential issue is how to accommodate delays with data availability while ensuring the timely publication of results. A further consideration is the extent to which different information needs to be conveyed for each indication versus the need to publish unified information for all. To overcome these potential difficulties, it was recommended that publication plans are in place before data are analysed, and are reviewed and updated regularly.

Innovations in data publishing: hitting the ground running, and making each step count

A second parallel session was moderated by Jan Seal-Roberts (Adis, Springer Healthcare) and examined recent trends in data publishing and data engagement through innovation.

Caroline Halford (Adis, Springer Healthcare) began by highlighting that recent key trends in publication planning seem to be focused on presubmission, production, and post-publication initiatives to increase speed, agility and impact. For example, more authors are starting to look beyond impact factor when considering which science, technology & medicine journals to submit their publications to. A recent survey of over 30,000 authors conducted by Springer Nature revealed several key factors driving journal choice, including reputation of journal, readership and reach, speed of publication, and open access availability. Halford illustrated this point with the Springer Healthcare journal Infectious Diseases & Therapy, which does not have an impact factor but is PubMed indexed and open access. Compared with the flagship Springer Healthcare journal Advances in Therapy, Infectious Diseases & Therapy has a higher acceptance rate, and similar citation and readership rates. Another recent trend is that more authors are choosing the rapid publication options now offered by an increasing number of journals.

Halford moved on to provide some tips to ensure a successful publication. Common reasons for a manuscript to be rejected by a journal include poor English/grammar, confusing/unclear structure, and unclear methodology. Many healthcare professionals have limited time so clarity — particularly within the title and abstract of the article — is key. Several tools are available to help increase the clarity of a manuscript; for example, visual abstracts and plain language summaries can aid understanding for patients, non-specialists, or members of the public. If a trial is particularly complicated, publication of a protocol/trial in progress article could aid understanding before the primary data are reported. The use of digital additions to publications are on the increase with many journals offering cascade initiatives (such as video abstracts, educational videos, slide sets, video articles, surgical/practical videos, audioslides, animations, infographics, and podcasts).

Halford ended her talk by noting that an increasing number of authors are now sharing their research on social media. Some top tips for social media success include: linking with hashtags for conferences, world disease days and disease (where appropriate); tagging author affiliations; sharing on affiliation pages; authors cross-posting (retweeting).

Anne-Clare Wadsworth (Envision Pharma Group) went on to discuss data engagement through innovation. As a starting point, Wadsworth emphasised that medical congresses remain a focal point for healthcare professionals in their medical education journey, but noted that there is an increasing expectation that meeting materials will be available online. As such, medical societies are adopting innovative approaches during their events to aid engagement. Wadsworth illustrated her point with a case study of digital content engagement at the annual European Association for the Study of Diabetes meeting, noting that online viewings of 14 recorded presentations reporting data from phase III studies were equivalent to 10–50% of the physical attendees. In addition, third-party sources are building content platforms featuring data from medical congresses; for example, ecancertv hosts video interviews from the key international oncology congresses including those of the American Society of Clinical Oncology and the European Cancer Organisation. Moving away from traditional methods for circulating data (detail aids, conferences, journals), multiple touchpoints are now available to healthcare professionals which are changing the way they access content, such as: infographics, eJournals, eForums, portals, social networks, instant messaging, eLearning, eDetails, mobile apps, and blogs. However, it is important to note that, despite technological advances and changing habits, long-form content is still popular, with in-depth articles being shared the most on social media.

Wadsworth discussed the various ways in which presentations can be enhanced to speed up access, augment understanding and increase engagement, through the use of audio visual additions, mobile-friendly content and infographics. An online survey conducted by Envision Pharma Group in December 2018 suggests that readers are starting to engage by accessing enhanced content at congresses or additional content via QR codes and results of a recent ISMPP members’ poll revealed that 51.2% of responders (n=207) plan to use digital enhancements in 2019.

As well as the evolution of audience engagement at congresses, Wadsworth noted that the patient perspective is becoming increasingly important to consider. Patients will occasionally attend a medical congress to raise awareness of treatment developments via social media; for example, Julie Flygare blogged about her attendance at the 2018 SLEEP meeting to raise awareness of treatment developments for narcolepsy. Moreover, Wadsworth highlighted that plain language summaries can enhance science communication; not only for trial participants and patients seeking scientific information, but also for managers/decision makers, and scientists (both within and outside the field of research).

Leveraging medical data – ways to maximize impact for the long run

In a third parallel session, Theodora Bloom (The BMJ) spoke about preprints and open peer review. Preprint servers have been around since the early 1990s starting with the physics platform arXiv, followed by bioRxiv for biology research and soon to be joined by MedRxiv for medical research. These sites offer a platform for researchers to deposit non-peer reviewed manuscripts in return for a citable digital object identifier number. The BMJ supports the use of preprint servers prior to publication, recognising their importance in speeding up citability and access to medical research. In response to audience concerns around posting non-peer-reviewed information, Bloom highlighted the importance of clearly marking preprints as preliminary findings, and further explained how additional safeguards in the screening process at bioRxiv means an article is not posted if doing so is considered to put public health at risk.

Bloom then spoke about The BMJ’s commitment to fully open peer review, in terms of both identity and comments. Open peer review has been shown to improve the quality of comments received and decrease the rate of rejection without affecting the time taken for review. Critically, although slightly more peer reviewers need to be invited to complete reviews, the initiative has mostly been well received by the scientific community. Bloom also stated support for post-publication peer review through comment posting, although recommends that responses are moderated.

Richard Ashdown (Caudex Medical) and Simon Fry (Springer Healthcare) then presented a more abstract concept to reform the medical publishing industry. The majority of the audience were already at least partly familiar with Bitcoin, which intends to disrupt traditional financial systems through its decentralised digital currency. Ashdown and Fry propose that the blockchain principle underlying Bitcoin could solve issues in the ‘data ecosystem’ such as data ownership, identifying source data and tracking the processing of data. Under the blockchain principle, data generated from research would be held in a decentralised digital database or ‘ledger’, where any modifications/usage of the data would be trackable, and control of the data would be ‘democratised’. Under this new spirit of verifiable data processing, public perception and trust in publishing brands would be a key factor for successful journals, underpinned by transparent processes such as open peer review. Interestingly, the ‘openness’ of the science blockchain could be variable, allowing certain users different access rights. One exciting envisaged outcome is that patients could control their own health data and how it is used. On the Gartner Hype Cycle, which tracks new technologies with the potential to transform human lives, blockchain is currently descending from the ‘peak of inflated expectations’ into the ‘trough of disillusionment’. Nevertheless, as blockchain gains traction in other areas, such as the music industry, Ashdown and Fry may well be forecasting the next revolution in medical research.

Maintaining our core strength: driving publication integrity through leadership

Gavin Sharrock (Wiley) introduced the panel session by describing how publication ethics has become a major focus for the publishing industry over the last 25 years. Advances have included the formation of the Committee on Publication Ethics (COPE), the development of authorship guidelines, such as the Good Publication Practice (GPP3) guideline, and global congresses held on the topic. Sharrock introduced the panellists who described the integrity issues they face in the world of medical publishing.

The author perspective came from Edwin Gale (University of Bristol) whose long career as a researcher has seen the replacement of ’eminence-based medicine’, where the treatment a patient receives is primarily driven by the physician’s own clinical experience, with evidence-based medicine and the emergence of randomised controlled trials as the gold standard in clinical research. Gale’s opinion was that market-based medicine is now an additional factor. He argued that companies perform and publish results from clinical studies to provide evidence for product approval, but that this leaves evidence gaps and unanswered questions once the product has launched.

Robert Wright (Sanofi) then spoke from the pharmaceutical industry perspective, noting that pharmaceutical companies have always been under a great degree of scrutiny regarding how they carry out and report the clinical studies they sponsor. He highlighted that this scrutiny has driven the industry to take massive strides over the last two decades to meet the escalating demands for clinical trial transparency and data and information sharing. Voluntary adoption of ‘Principles for responsible clinical trial data sharing’ from the Pharmaceutical Research and Manufacturers of America (PhRMA) and the European Federation of Pharmaceutical Industries and Associations (EFPIA) and improvements in company systems and processes, such as the introduction of corporate integrity agreements, has meant that pharmaceutical companies are now amongst the most compliant with reporting standards. With regards to the future of open access, Wright believes that there is general agreement across the industry that “knocking down the paywall” is a good thing for both healthcare professionals and patients, but questions remain about how this can be broadly implemented.

Next up, Simon Stones (patient advocate and consultant) shared his views on publication integrity from the patient perspective. Stones stated that while we carry out research to improve the lives of patients, this fact is frequently forgotten when it comes to communicating the outcomes of research. Patients are often keen to learn about their condition and potential treatments, but many say it is hard to find accessible, credible and accurate information. While there are some journals that have the patient audience in mind, and the use of plain language summaries is becoming more popular, Stones concluded that there is still more that can be done to engage patients.

Matt Hodgkinson (Hindawi) then provided an overview of the ways in which publishers maintain publication integrity and the associated challenges. These challenges include misconceptions around declaring conflicts of interest. Hodgkinson highlighted that conflicts of interest are not just financial conflicts, but can relate to anything that might affect how the research was done or reported and should not be considered an admission of bias. The increased use of reporting guidelines has benefited medical publishing, but Hodgkinson noted that this has brought about a ‘tick-box mentality’ which authors need to move beyond. Requirements for data sharing statements within papers are also becoming more common but Hodgkinson suggested that requests for data are not always accepted and monitoring this is not straightforward. Confusion remains regarding what data needs to be made available for sharing and exactly whose responsibility it is. Hodgkinson also highlighted several authorship integrity challenges for publishers, such as verifying author identities and who qualifies as a legitimate author. Lastly, he remarked that questionable or ‘predatory’ publishers are also still a problem and advised researchers to think, check, submit to identify trusted journals.

Finally, Gordon Muir-Jones (Porterhouse Medical US; ISMPP Ethics and Standards Committee) described the leadership role ISMPP has taken in driving and ensuring integrity in medical publishing. Muir-Jones began by giving a brief history of ISMPP and explained how the vision and mission of the society has evolved since it was formed in 2005. He also highlighted the many initiatives that ISMPP has introduced, such as ISMPP University, the CMPP certification program and the mentoring scheme, and indicated that more developments are on the horizon. While the session highlighted the amount of work required to maintain publication integrity, Muir-Jones concluded on a positive note declaring that it was an achievable aim, especially when stakeholders collaborate, and it is tackled one step at a time.

Speed work: research presentations

The number of poster presentations by ISMPP members continued to rise this . The meeting abstract committee, chaired by Laura McGovern (Nucleus Global), selected three of the best abstracts for oral presentation during a ‘Speedwork’ session at this year’s meeting.

Heather Lang (Oxford Pharmagenesis) kicked off the first, preprint-focused presentation with the statement that “preprint servers are definitely here, and they’re here to stay”. In order to address the question ‘Is pharmaceutical industry research posted as preprints?’, the group searched the bioRxiv biology preprint server for authors affiliated with the top 50 pharmaceutical companies. They found that half of these companies have published preprints, with corresponding authors from 10 of the top 50 companies. The majority of these preprints reported basic research. Although most preprints were not commented on or revised, the team found this seemed to be the norm for preprints uploaded to bioRxiv. Importantly, two-thirds of preprints posted for at least a year had subsequently been published in a peer reviewed journal. Lang concluded with a number of suggestions for the pharmaceutical industry: 1) To evaluate which types of research are appropriate for preprints; 2) To consider running pilots to explore the use of preprint servers; 3) To develop and communicate policies on appropriate use.

Next up, Caroline Shepherd (CMC) reported her team’s work on the ‘Evaluation of plain-language summaries (PLS): Optimising readability and format’. Selected for this study were three articles published in top-tier journals between May 2016–May 2018, reporting on phase III randomised controlled trials in one of three indications (psoriasis, multiple sclerosis [MS] or rheumatoid arthritis [RA]). Four plain language summaries were developed for each article: one in infographic format and three written with a low, medium or high degree of complexity. These plain language summaries for psoriasis, MS and RA were then disseminated to patient groups alongside a 20-question online survey. Shepherd highlighted the unexpected finding that, of 167 respondents, approximately half held a higher education degree; a population that Shepherd postulated may better represent individuals likely to seek out and read plain language summaries. Across all three indications, weighted-average preference scores demonstrated infographics and medium-complexity (reading age 14–17 years) text plain language summaries to be the first and second most popular first-choice preferences, respectively. Shepherd advised that these preferences be accounted for when preparing plain language summaries to accompany original research articles, and shared some recommendations for their preparation. For infographic plain language summaries, these included effective and consistent use of colours, layout and icons, and an overall focus on the ‘so what?’. For text-based plain language summaries, tips included the use of short sentences and readability scores.

The session drew to a close with a presentation from Anisha Mehra (Cello Health Communications) on her team’s ‘Evaluation of data reporting in real-world publications in clinical versus health economic journals’. This study aimed to decipher trends in RWD publication across four therapeutic areas, and to assess the quality of data reporting according to International Society for Pharmacoeconomics and Outcomes Research (ISPOR) recommendations. Literature searches were performed using PubMed for real world datasets on diabetes, psoriatic arthritis, haemophilia and wet macular degeneration published in the top five therapeutic area-specific clinical and health economics journals between 2015–2017. Those in psoriatic arthritis were selected for a more in-depth analysis using the ISPOR retrospective database checklist. The team found that real world data were commonly published in clinical journals; a total of 1,003 and 75 articles reported real world data in clinical and health economics journals, respectively. In data quality analyses, health economics journals appeared to more consistently publish articles adhering to ISPOR checklist criteria compared with clinical journals. Mehra suggested this may reflect a lack of awareness of good reporting guidelines for these types of studies in journals outside of the health economic community, and concluded that consistent and transparent reporting in real world publications will require ongoing efforts from authors and journals.

Keynote: no more robowars

In day 2’s keynote address, Simon Fry (Springer Healthcare) discussed artificial intelligence (AI) and its potential role in healthcare and medical publishing. Fry noted that a wide spectrum of views exists with respect to the potential benefits and downsides to the increasing prevalence of AI and referred to futurist Raymond Kurzweil’s prediction that the rate of change in technology will occur at an exponential rate, with machine intelligence surpassing that of humans within decades.

In the past, the progress of AI was limited by an inability to amass sufficient computer power to carry out particular tasks. However, this issue has now largely been solved, leading to an explosion of AI use across many different fields. Several examples exist within healthcare of AI being used to complement and enhance human expertise. Microsoft’s Project InnerEye uses AI for radiological imaging and precision surgery. The surgery is driven by humans, but its accuracy is improved through the utilisation of AI. In this way humans and machines work together, to achieve optimal results. IBM’s Watson Oncology, is a question-answering computer system, designed to support physicians with cancer care decisions. The system, which can analyse vast quantities of RWD, was trained by physicians from Memorial Sloan Kettering to interpret clinical information from cancer patients and determine individualised treatment recommendations. In this example AI is used to assist clinicians with evidence-based decision making. Deontics is another example of an AI-powered tool designed to support clinicians with decision making. AI is also being used within medical publishing. Synchrogenix, a consultancy of regulatory and medical writers, reports the ability to generate 80% of the first draft of a regulatory document such as a clinical study report within 24 hours, using AI, based on documents such as a study protocol and statistical analysis plan.

While machines may be able to take over data-driven tasks, Fry argued that healthcare requires human delivery. He suggested that increased use of technology may raise the value of processes that cannot be automated, such as doctor–patient interactions. Fry also noted that the increasing use of AI is likely to raise a number of legal, emotional, socioeconomic and financial questions.

Staying ahead of the game: news, views & controversies

In a world where Kim Kardashian, with her 59.5M Twitter followers arguably has more influence than, the Editor in Chief of The BMJ (who has 12.6K), how do we address the challenge of health-related ‘fake news’? This was the intriguing question at the beginning of a fascinating session moderated by Andy Powrie-Smith (EFPIA), with a panel comprising Theodora Bloom (The BMJ); Dhruv Ghulati (Factmata), Nicole Rapior (Boehringer Ingelheim), Stuart Spencer (The Lancet), and Chris Winchester (Oxford PharmaGenesis; Chair, ISMPP Board of Trustees).

The first step in addressing the challenge of health-related ‘fake news’ is identifying where inaccuracies in reporting arise. The panel felt that press releases from the pharmaceutical industry and respected journals are generally balanced but that inaccuracies are frequently introduced once news has been translated for mainstream audiences; journalists frequently over-simplify scientific findings in their desire for an attention-grabbing headline. Bloom noted that The BMJ highlights potential limitations of research in its press releases to help journalists to contextualise news; Spencer felt that the industry could go further by including ‘downsides’ in press releases – for example, information about potential adverse events and reproducibility of results. Ghulati noted that social media presents a particular problem as the small fragments of information presented often have no context.

Powrie-Smith asked the panel to think of one way to fight ‘fake news’ in healthcare. Winchester, from the medical communications standpoint, suggested that anyone should be able to access high-quality information so open access should be mandatory for pharmaceutical industry-funded research. Rapior felt that the industry should think more broadly in terms of the potential audiences for medical publications. From the publisher’s perspective, Spencer proposed that balanced summaries and narrative meta analyses would help, while Bloom’s view was that, firstly, every press release should state how long it would be until the drug/technology would be available to patients, and secondly, that post-trial surveillance should be routine to ensure that information is updated as new evidence becomes available. Ghulati had the last word from the panel and felt strongly about the need for individual scientists and researchers to be involved in scientific communication, checking claims and evidence and involving the public. “You’re trusted as a scientist: do your job”, he urged.

After a successful and thought-provoking 2 days the 2019 European meeting of ISMPP drew to a close, with Al Weigel (ISMPP President/Chief Operating Officer) thanking delegates, the meeting faculty, the Programme Committee, and the meeting sponsors and exhibitors. Presentations will be posted to the ISMPP European Meeting Archive in due course.

———————————

By Aspire Scientific, an independent medical writing agency led by experienced editorial team members, and supported by MSc and/or PhD-educated writers

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd