KEY TAKEAWAYS

• AI tools have the potential to streamline, optimise, or speed up various stages of the clinical trial process.
• This could accelerate drug development and potentially reduce the number of participants needed in clinical trials.

While advances in the field of artificial intelligence (AI) have been making headlines recently, drug development has been slowing down. Taking more than a billion dollars and a decade to complete increasingly complex clinical trials, the majority of investigational drugs never reach the market. Could AI help reverse this trend?

How can AI aid clinical research?

In a recent article for Nature, Matthew Hutson discusses how researchers have already begun to investigate the potential for AI to optimise clinical trial processes:

• Clinidigest claims to simultaneously access dozens of clinical trial records to create summaries, allowing researchers to gain a quick overview of existing trial data.
• HINT aims to predict a drug’s success during the trial design stage to ensure resources are not wasted.
• Trialpathfinder seeks to optimise eligibility criteria by testing whether broadening criteria would have any effect on risk to patients; its developers believe this would also allow for more inclusive trials.
• DQueST seeks to match patients looking to participate in research with suitable clinical trials.
• SDQ is used to extract, analyse, and clean datasets.
• Some AI tools aim to predict missing data points and identify relevant clinical subgroups.
• Others aim to monitor adherence to medication, so that investigators will not have to.

Pharmaceutical companies are now experimenting with software that completes tasks within a couple of days, which previously took 2 months.

Can AI help patients too?

As well as facilitating recruitment and including more populations in trials, AI tools could decrease the number of patients required for successful research. Hutson highlights examples such as Unlearn.AI, which aims to achieve this through creating ‘digital twins’, predicting a patient’s results had they been given a placebo. The makers claim this reduces the sample size required and allows a greater proportion of patients to take investigational drug rather than placebo.

Technologies such as ChatDoctor answer patients’ questions, which could help retain participants in clinical trials, as well as being useful in clinical practice.

Should we be worried?

As Hutson points out, there are concerns about AI and research integrity. Xiaoyan Wang, co-developer of AutoCriteria, highlights the risk of biased data and confidentiality issues when providing tools with a huge training data set. The WHO has published guidelines on making sure  AI is used ethically. As is currently widely acknowledged, AI outputs may always need to be checked by a human expert.

While AI has the potential to optimise clinical trials, researchers and clinicians need to be mindful of its limitations.

————————————————–

KEY TAKEAWAYS

• Mendelian randomisation studies are increasingly used to analyse the causal effects of exposures and outcomes in diseases, but are often inadequately reported.
• The STROBE-MR reporting checklist has been developed to improve the reporting of Mendelian randomisation studies.

Mendelian randomisation (MR) is a popular method in epidemiology that uses genetic variation as a proxy for environmental exposures to understand the cause of diseases. Despite the increasing use and relevance of the technique, there is a lack of consensus on how to report MR studies. To address this need, a new STROBE guideline, Strengthening the Reporting of Observational Studies in Epidemiology using Mendelian Randomisation (STROBE-MR), has been published in the Journal of the American Medical Association (JAMA), with the explanation and elaboration paper published in The BMJ.

Associations between exposures and health outcomes reported in observational epidemiology studies are not always reliable due to confounding, or other forms of bias. Advances in technology have enabled identification of associations between genetic variations and exposures, traits, and health outcomes. MR uses these genetic variations as instrumental variables to analyse the effects of exposures on diseases. MR studies can therefore help understand the influence of exposure on various health, social and economic outcomes. As genetic variation is randomly inherited and cannot be affected by certain confounding factors, bias associated with other epidemiological study designs can be avoided using MR.

Mendelian randomisation studies provide an opportunity to study the causal association between an exposure and an outcome while reducing the risk of certain biases associated with traditional observational study designs.

Items in the existing STROBE checklist were either too general or did not apply to MR studies, while items specific to MR studies were missing. To help improve reporting on MR studies, a separate checklist was developed based on the STROBE guidelines, with 20 items covering specific recommendations for publications reporting MR studies. The checklist is also available on the EQUATOR Network website.

It is hoped that adoption of the STROBE-MR checklist by authors, reviewers and journal editors will improve the quality of MR study reports and enable consistent and transparent communication in this field.

After reading the article, click here for a brief survey and to receive your authorization code for your Credit Tracker. This serves as documentation for the activity.

—————————————————–

While there is increasing awareness of the need to tackle inequities in scientific publishing, research, and medical communications, and to improve diversity and representation in clinical trials, significant work remains if we are to account for sex and gender dimensions in medical research. Following her presentation at the 2022 European Meeting of the International Society for Medical Publication Professionals (ISMPP), The Publication Plan found out more about Shirin Heidari’s experiences of how sex and gender dimensions are currently considered in medical research, and her perspectives on how research conduct and reporting could be improved in the future.

Please could you share a little about your background and how you started advocating for greater consideration of sex and gender in medical research?

“I’m a virologist and experimental oncologist by training: my PhD was on polyomaviruses, and then I shifted into HIV research. I didn’t pay much attention to sex differences when I was doing basic science research – I was using a lot of animal studies, and interestingly enough, we mostly used female mice as they fought less. When I moved into the HIV field, concerns around sex and gender dimensions were much more pronounced, particularly the gender aspects. That’s when I started to wonder why we hadn’t been looking at differences based on biological sex, as well as gender differences, in our research.

At that time, I’d moved away from basic science research and was heading the research promotion department at the International AIDS Society, working with research but not conducting it directly. I was also editor of the peer reviewed Journal of the International AIDS Society (JIAS), and it grabbed my attention that few of the many submissions I was reviewing took into account sex and gender. I thought that was weird, given that we had sufficient knowledge that there were sex and gender differences in terms of HIV transmission, including vertical transmission during pregnancy, and drug metabolism and access. I thought that I could help address the gap and introduced a policy asking authors to disaggregate their data by sex, and conduct gender-based analyses when possible. The society also worked on other gender aspects in the field of HIV, including around human rights, access to services, gender-based violence, and a whole range of issues related to sexual and reproductive health, all of which are gendered. We tried to bring these issues together and raise them through any work we did with researchers, regulatory agencies, and pharmaceutical companies.

As I joined the European Association of Science Editors, I thought that this was a great opportunity to try to introduce sex and gender reporting guidelines to other journals. We established the Gender Policy Committee and developed the Sex and Gender Equity in Research (SAGER) guidelines, which we’ll talk about in more detail later.”

Why is it important to collect and present health data disaggregated by sex?

“We have a lot of knowledge, even if we do not often look at it systematically, that there are important biological differences between sexes. There are also a number of differences in terms of experiences, preferences, and health-seeking behaviours between women, men, and non-binary people. If we’re not looking at these issues, we might sometimes overlook very important differences, which can result in inequitable access to health services, or simply greater harm to one sex over others.

“If we’re not looking at sex and gender dimensions, we may overlook very important differences, which can result in inequitable access to health services, or simply greater harm to one sex over another.”

Since research has historically focused on the white male model, disadvantages are often experienced by women and people of non-white ethnicities. This tends to impact women due to historical under-representation in studies and clinical trials. By not considering sex and gender dimensions, I think we are missing out on important findings with regards to disease presentation, symptoms, drug metabolism, immunological responses, side effects, and adverse events, but also other patterns of consumption, health-seeking behaviours, accessing particular health services, and vaccine hesitancy, for example. That’s why it’s so important to systematically look at these issues.”

How well have sex and gender variables been considered in research related to COVID-19, including in vaccine development? How much consideration has there been for the interplay of sex- and gender-related factors with other variables?

“Recently we conducted two systematic reviews: one of observational studies and clinical trials regarding COVID-19 and another of COVID-19 vaccine trials. Despite the fact we knew about important sex and gender differences related to COVID-19, such as the risk of infection, presentation of symptoms and severity of disease, both systematic reviews showed that the majority of studies did not report how they considered sex and gender aspects in the research. They rarely reported outcome data by sex, or other variables, or discussed the implications of the research for different populations.

“Despite the fact we knew about important sex and gender differences related to COVID-19…both systematic reviews showed that the majority of studies did not report how they considered sex and gender aspects in the research. They rarely reported outcome data by sex, or other variables, or discussed the implications of the research for different populations.”

As vaccines were rolled out, we could observe that there were different patterns of adverse events between women and men. It’s important to underline that these adverse events are rare, but nevertheless, particular adverse events or side effects seem to be predominant among women, and some others predominant among young men. This really indicates that we should pay more attention to sex and gender aspects during clinical trials, and ensure that disaggregated information is captured in follow-up after clinical trials and in post-marketing data, and that data are analysed more regularly. Sometimes the vaccine trials did disaggregate the data by sex only, but they didn’t do this across different study arms, different sites, different dosing strategies, or different intervals – and there could be important findings there, which may lead to innovations too. I have co-authored two papers in my capacity as a senior technical consultant with the WHO, around how to consider sex and gender dimensions in COVID-19 vaccine development from very early on in clinical trial design all the way to vaccine delivery and roll-out.

In our systematic review of COVID-19 vaccines trials, we tried to look at how sex- and gender-related factors intersect with other variables in our systematic reviews, but the data are poorly reported. We looked at the intersection of sex and age, but very rarely saw concurrent disaggregation by sex and age. I think it’s important to keep in mind that we need to look at these different variables concurrently. We barely saw any data around people of diverse sexual orientations or gender identities, for example, even with regards to vaccine hesitancy and vaccine uptake. As far as I know, we also haven’t seen much data around COVID-19 vaccination in racial or ethnic minorities or migrant or forced displaced populations, although there have been some reports showing important inequities. For example, in one report from the UK we could see differences in the risk of COVID-19 infection and severity among different ethnic minorities, with gender differences within those ethnic minorities as well. This really highlights those intersectional aspects.

There have of course also been issues related to other aspects of the COVID-19 pandemic, such as delay in getting sufficient data on safety and efficacy of vaccines during pregnancy, and gender-based violence affecting women and people of diverse sexual orientation and gender identity. As such, there are many sex and gender aspects of COVID-19 which need to be looked at from many angles.”

You were the lead author of the SAGER guidelines, published in 2016. For any readers who may not be familiar with these guidelines, please could you outline what the main recommendations are?

“The SAGER guidelines try to guide authors to consider and report on sex and gender dimensions in scientific publications. We’re not forcing people to do a sex- and gender-based analysis, but we are trying to encourage them to be transparent in terms of reporting much more completely what they have done. For example, we want to make sure that people use the words ‘sex’ and ‘gender’ carefully – using ‘sex’ when they are referring to biological differences, and when they are reporting gender, providing information about how the data were collected.

We encourage authors to be clear in the title and abstract and state if they have, for example, only included one sex or gender in the study, so that the title and abstract are not misleading. We also encourage authors to do their due diligence, doing a literature search and including in the introduction whether we would expect any sex or gender differences related to their topic of study. If so, the authors should present in the methodology how they have considered the anticipated differences, or if these have not been considered, the authors should justify why these were not factored into the study design.

The results should be clear about reporting baseline, outcomes and other data by sex and/or gender. One of the misconceptions has been that we are forcing authors to conduct a sex- and gender-based analysis, even when studies haven’t been designed with this in mind and might be underpowered. That is by no means our intention: we encourage people to do a sex- and gender-based analysis if the design and study size allow. Nevertheless, we do believe that reporting outcome data by sex and/or gender (or making disaggregated data available as an appendix or in the supplementary materials) is really important to allow for future pooled analysis and meta-analysis. It can also give indications as to whether there are signals that need to be investigated in additional studies better designed for this purpose.

“One of the misconceptions has been that we are forcing authors to conduct a sex- and gender-based analysis, even when studies haven’t been designed with this in mind and might be underpowered. That is by no means what we intend…Nevertheless, we do believe that reporting outcome data by sex and gender…is really important to allow for future pooled analysis and meta-analysis.”

Finally, in the discussion it is really important to consider what the data show with respect to sex and gender differences or similarities, or if the study hasn’t looked at this, noting it as a limitation. The discussion should also consider what the implications of the findings are in terms of what we already know and what we should look at in the future. We’re just trying to raise awareness so that when people publish their findings, they are much more transparent and complete in terms of reporting what they have or have not done.”

How well have the SAGER guidelines been received to date, in terms of their incorporation into journal and other policies? Has this translated into implementation of the recommendations in research reporting?

“That’s a very good question. Initially, there was a very slow uptake, and we even saw some resistance in a survey we conducted before launching the guidelines. Now, more than 5 years later, we are very pleased to see rather widespread uptake of the SAGER guidelines by most leading journals and many publishers, like Elsevier, Springer, and Nature. They have all taken up the SAGER recommendations and tried to incorporate them into their systems and instructions for authors. We are also very pleased to see that more and more non-academic organisations that conduct research, for example non-profit organisations, are adopting the SAGER guidelines in their publication policies (the recommendations have been incorporated into FIND’s publication policy, the WHO Polio Eradication Strategy, and GENDRO’s consensus statement regarding the Pandemic Treaty, and have been discussed with the Drugs for Neglected Diseases initiative). Furthermore, the SAGER recommendations have been translated into several different languages and have been the topic of a TEDx Talk.

“More than 5 years later, we are very pleased to see rather widespread uptake of the SAGER guidelines by most leading journals and many publishers.”

It’s still too early to know to what extent these policies are being implemented. The big publishers who are incorporating the guidelines have better resources to monitor progress than we do, so we are hoping that more data will become available over time. At this point, I think that the SAGER guidelines have led to a much greater awareness among authors, funding agencies, and the research community at large about these issues. I’m hopeful for the future.”

Could you tell us about any barriers to the uptake of the SAGER guidelines? How could these be addressed?

“There are always barriers, but there is no barrier that cannot be overcome. One of the challenges we often hear is journals not wanting to require authors to consider the SAGER guidelines, given the many reporting guidelines out there already. Considering the guidelines address a problem and promote complete and transparent reporting, I find this a poor excuse, as journals have a key role in influencing reporting practices. Another potential barrier is the additional burden on editorial teams to determine whether submissions comply with the guidelines. Having been an editor myself, I recognise that this is a slightly cumbersome process. At the same time, the responsibility can be distributed across the different actors in the peer review process. For example, at one of my previous journals, I included a tick-box within the submission system, giving the authors the responsibility to declare that they have read the SAGER guidelines, have disaggregated their data by sex, and complied with the guidelines. You can also ask peer reviewers to pay attention to this.

“Journals may not want to make the SAGER guidelines mandatory, given the many reporting guidelines out there already. I find this a poor excuse as journals have a key role in influencing publishing behaviour, given the guidelines address a problem and promote complete and transparent reporting.”

As I mentioned, there are also misconceptions around the idea that the guidelines are mandating a sex- and gender-based analysis regardless of design, and that’s really not the case – it’s important that authors mention in their articles whether they have considered these issues, and if not, to justify that.”

While the SAGER guidelines tackle the reporting of sex and gender information in publications, achieving gender-balanced research will require changes to how research is conducted. Do you have any recommendations for how sex and gender should be considered in clinical trial design, conduct, and analysis to increase the value of the information generated? How can ethics committees influence this process?

“This is a really, really important point, and there are several answers. One of our first ambitions was to make sure that the key principles of the SAGER guidelines are incorporated into other reporting guidelines to exert another layer of pressure, since clinical trial reporting guidelines are much more complied with at this point, yet don’t address gender bias adequately. At the same time, I think it’s really important to think beyond research protocols, to the logistics involved in trial participation. We rarely consult research participants – women, men, and non-binary people – about why they do or don’t participate in clinical trials. We rarely look at issues related to income loss, childcare, and respectful participation. Are researchers trained to recognise their own gender biases and other biases, and how these influence the dynamics and interactions between researchers and research participants? Do we analyse our dropout and discontinuation data with a gender lens to, for example, see whether there are any underlying reasons that are gender- or biological sex-related, such as adverse event experiences? All these aspects need to be considered with an intersectional gender lens in mind, and everyone involved in the research system – as gatekeepers or as actors – has an important role to play.

“I think it’s really important to think beyond research protocols, to the logistics involved in trial participation. We rarely consult research participants – women, men, and non-binary people – about why they do or don’t participate in clinical trials. We rarely look at issues related to income loss, childcare, and respectful participation.”

Similarly, we are currently working on developing SAGER guidelines adapted for research ethics committees (RECs) and institutional review boards (IRBs): some of the most influential gatekeepers since no research is conducted without ethical approval. We’re trying to make sure that RECs and IRBs consider sex and gender dimensions when they evaluate research protocols. There are a lot of ethical aspects that ethics committees look at, in terms of ensuring equity, justice, fair participation, and beneficence, that can be influenced by sex and gender. There are other gender-related factors with ethical implications too, such as the inclusion of pregnant women in studies. Another issue is that ethics committees are often neither gender-balanced, nor usually include anyone with gender expertise to evaluate the applications with that lens. Hopefully the adapted SAGER guidelines can further shed light on how RECs can support further changes to the system.”

Looking ahead, how would you like to see sex- and gender-related factors considered in medical research in 5 years’ time? What one action would you suggest stakeholders in the pharmaceutical industry and in medical publishing start taking now, to help achieve these goals?

“There is already a positive trend which makes me hopeful that in 5 years’ time we will have sex- and gender-based reporting guidelines that are also considered by RECs, funding agencies, and other key gatekeepers in the research system. If all of the gatekeepers develop, incorporate, implement, and enforce these guidelines, I believe that we will be in a very different place in 5 years. Along these lines, I think that regulatory agencies, pharmaceutical companies, diagnostic companies, and any other entities that develop interventions used by humans should also take more responsibility. I’ve worked a lot with pharmaceutical companies in the past, and I believe that they have a moral obligation towards transparency of their data, in particular around clinical trials. They should ensure that the design of clinical trials takes into account sex and gender dimensions, and use models that include pregnant women, to make sure that drugs or vaccines are safe and can be used during pregnancy, for instance. I think that there is very little communication with pharmaceutical companies, and the next step is to have more open and frank discussions around their role and responsibilities towards sex- and gender-balanced research. Regulatory agencies could be one partner applying pressure to make these changes. It’s also not just about efficacy and safety data, but also other aspects like pricing and preferential modes of drug delivery.

“I believe that pharmaceutical companies have a moral obligation towards transparency of their data, in particular around clinical trials. They should ensure that the design of clinical trials takes into account sex and gender dimensions, and use models that include pregnant women.”

Other than these issues being important from gender equality, scientific rigour, and transparency perspectives, looking at sex and gender dimensions is also a very interesting area of investigation in terms of opening up new findings, new learnings, and possibilities for innovations. I think we should look at these analyses not just as something that we must do, but something that we want to do to give us new ways of understanding our world in different ways.”

Dr Shirin Heidari is the Founding President of GENDRO, Senior Research Fellow at the Global Health Centre at the Graduate Institute of International and Development Studies in Geneva, and Senior Technical Consultant on gender to the World Health Organization Gender, Equity and Human Rights Unit. Shirin is founding chair of Gender Policy Committee and Chair of the SAGER Project Group of the European Association of Science Editors. You can contact Shirin at s.heidari@gendro.org or follow her on Twitter at @heidari_s.

—————————————————–

Artificial intelligence (AI) is a rapidly evolving field that has many potential health applications, including systems that can be used for screening and triage, diagnosis, decision support and treatment recommendation. Concerns regarding the design and reporting of recent trials evaluating such systems have led to the development of an extension to the SPIRIT (The Standard Protocol Items: Recommendations for Interventional Trials) 2013 statement, an international guideline aiming to standardise the quality and completeness of published trial protocols. Details of the extension and the methods used in its development were recently published by Samantha Cruz Rivera et al in the British Medical Journal.

The SPIRIT-AI extension, which complements the CONSORT-AI guidelines for the reporting of AI trials, provides consensus-based, AI-specific guidance on what should be included in the trial protocol, and should be used alongside SPIRIT 2013 and other relevant extensions. Items recommended for inclusion in the new checklist include:

• algorithm version
• input and output data
• expertise of users
• plans to identify and analyse performance errors
• prior level of evidence for validation of the AI intervention.

The latter two points were added to address safety concerns raised by the consensus group in recognition that that AI systems may unpredictably produce errors that are not easily detectable or explainable by human judgment, which could have unintended but harmful consequences.

Furthermore, as some AI systems continuously learn from new data their safety performance can evolve and change over time. While the issue of continuously evolving systems has not been covered in the current SPIRIT-AI extension, the authors suggest that software changes should be identified and documented and note that this topic may be addressed in future versions of SPIRIT-AI. Due to the relative newness of the field, there were few published AI trials and no published AI trial protocols at the time that the SPIRIT-AI guidance was developed. The authors acknowledge the importance of re-evaluating and updating SPIRIT-AI as AI technology continues to evolve and as its application expands to include therapeutic uses in addition to detection and diagnostics.

After reading the SPIRIT-AI article, click here for a brief survey and to receive your authorization code for your Credit Tracker. This serves as documentation for the activity.

After reading the CONSORT-AI article, click here for a brief survey and to receive your authorization code for your Credit Tracker. This serves as documentation for the activity.

——————————————————–

Summary by Alice Wareham PhD, CMPP from Aspire Scientific

——————————————————–

With thanks to our sponsor, Aspire Scientific Ltd

Although randomised controlled trials (RCTs) are the cornerstone of evidence-based medicine, studies that evaluate multiple crossovers between intervention and comparator in a single patient, or ‘n-of-1 trials’, can help to inform personalised treatment decisions. N-of-1 trials rigorously measure intervention effects for individuals and can generate valuable data in situations where RCTs are not feasible or appropriate, such as in rare diseases or when treatment effects are heterogeneous. Well conducted n-of-1 trials can also provide a source of population data for meta-analyses, important for conditions where there is little RCT evidence.

In 2013, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement was released, providing evidence-based recommendations for the minimum content required in a published clinical trial protocol. An extension to these guidelines has recently been published in The BMJ by Dr Antony J Porcino and colleagues. The SPIRIT extension for n-of-1 trials (SPENT) provides guidance on the development and publishing of n-of-1 trial protocols to improve completeness and transparency.

The SPENT guidelines present the minimum content needed for complete and transparent reporting of n-of-1 trial protocols.

SPENT contains 14 extension items specific to n-of-1 trials, a checklist for n-of-1 trial protocol abstracts, and additional guidance for eight SPIRIT items providing information tailored to the n-of-1 trial context. To ensure concordance between the reporting of trial protocols and results, SPENT was developed in line with the CONSORT extension for n-of-1 trials (CENT) reporting guidelines.

The authors highlight that many aspects of n-of-1 trials are the same as parallel-group RCTs and, as such, many aspects of SPIRIT apply to SPENT. Stakeholders and reviewers are strongly encouraged to adopt SPENT to increase the standardisation, completeness and comparability of n-of-1 trial protocols. The authors conclude that increased uptake of SPENT will improve the quality of these protocols, their subsequent review and use, and ultimately, the resulting trial.

——————————————————–

Summary by Alice Wareham PhD, CMPP from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd