Post

Bayesian clinical trials and the FDA

Posted
By Witold Więcek 8 min read

This is a short note on the current stance of the FDA to use of Bayesian methods in clinical trials and what people mean when they talk about “Bayesian trials”. These are just loose and basic notes that I compiled out of my own interest in early 2023 and for colleagues who are not statisticians. I’m not a clinical researcher, but for many years I worked on statistical consulting for drug makers. One time I even went to the FDA to talk about a particular trial, but all I remember from it is that they expect you to wear a tie to these things.

This was written in February 2023 and, as you can see below, this is a pretty dynamic area, so keep that in mind.

What is a Bayesian clinical trial?

First of all, this is a very wide and flexible term. Bayesian methods can be used at many stages of designing and conducting a clinical trial. In the current context this term usually pops up in the context of what FDA calls “complex innovative designs” or in relation to adaptive designs (more on both below). But first I want to introduce this concept broadly and in a non-technical way.

The point of using Bayesian methods is that they allow evaluation of drugs using all of available data, as it becomes available. Very roughly that means two things: (1) building information from other/prior trials directly into the statistical analysis, (2) “live updating” of trial results throughout the trial.

To illustrate these two points, imagine a placebo-controlled randomised trial of effectiveness (preventing infections) of a vaccine in children, conducted after it was already proven to work in adults . (1) above means that some information about vaccine’s effect in adults can be extrapolated into children, for example using immunological modeling (with appropriate uncertainty that reflects the differences between children and adults). This could allow us to link immune responses to effectiveness OR to increase our prior confidence in effectiveness of the vaccine in children.1 (2) means that we evaluate vaccine’s effectiveness after each individual in the trial becomes infected.

Ultimately, the point of (1) and (2) both is that the trial can run shorter/more efficiently than a “traditional” trial, but give us the same level of confidence in the results. (2) also allows us to modify the trial using some pre-defined rules as we learn more about the treatment (more on that below).

In contrast, a traditional trial uses prior information only at the stage of designing the trial (e.g. if a vaccine developer and regulator are both highly confident the vaccine will work, they may agree to design a smaller/simpler trial). It then waits to reach a pre-defined endpoint (based on calculations of statistical power) before reporting results.

If Bayesian trials have those advantages, why were they not used more widely? One of the reasons is that development of this type of applied Bayesian statistics was slower because their implementation require more computational power. With advances in computing power in the last three decades, Bayesian methods have become accesible to researchers. Second reason, related to this, is that Bayesian inference was not typically taught as part of statistics degrees (let alone to clinical researchers). Third reason is that there are valid methodological concerns about how these methods should be applied and they are typically seen as more subjective.

Good general references for this topic are Bayesian clinical trials and Bayesian Approaches to Clinical Trials and Health-Care Evaluation.

Currently there is no FDA guidance on “Bayesian trials”.2 As part of FDA’s commitment to investigate new trial designs, in 2025 the agency will issue a guidance on use of Bayesian statistics in clinical trials (source). Until then, there is an existing Bayesian stats guidance for trials of medical devices (from 2010), but each division issues their own guidances, so that document applies only within work supervised by Center for Devices and Radiological Health.3

However, there is more guidance on “complex” designs and adaptive designs guidance. Adaptive trials allow for modifications to be made to a clinical trial as it is ongoing, based on accumulating data.4 FDAs stance is captured in these docs:

I cover the basics of adaptive approaches in one of my recent papers (Clinical trials for accelerating pandemic vaccines, see section III). I also found this summary to be very good for basics of the FDA’s current approach: FDA announces continuation of program to support ‘complex’ and ‘innovative’ trial designs (Agency IQ). There is also a new (Dec 2022) article from FDA staff: Bayesian Methods in Human Drug and Biological Products Development in CDER and CBER.

Lastly, in a very broad sense the turn toward Bayesian methods can be seen as part of the (also very broad) debate about overreliance of decision makers on p-values. There is a statement from Center for Drug Evaluation and Research (CDER) at FDA that “recognizes the limitations and the usefulness of p-values in the drug development setting”. Bayesian methods are quite different in this regard (although it’s hard to summarise the difference in a sentence or two).

Is this going to make a difference?

Are those “complex” designs a better way of researching drugs in general? In some cases, almost definitely yes. I gave description of their theoretical advantages earlier, but if you want practical examples, look up I-SPY trial or follow the linked papers in previous section, they have some actual examples.

However, a more bleak view (especially if we think of the relationship between drug developers and regulators as a kind of game-theoretic problem) is that adaptive and Bayesian trials may give developers additional degrees of freedom in “proving” that their drugs work. I’m not familiar with that area enough to have an opinion, but it does not sound implausible.

More specifically, does FDA issuing extra guidance documents make a difference? Why are not there more Bayesian clinical trials? A mix of pharma, CRO, and regulatory respondents usually say that it’s due to lack of familiarity with Bayesian methods and in my limited experience this is completely true. However, lack of guidance is consistently listed as the second factor and there is an obvious feedback loop between training and regulatory expectations.

Odds and ends

A few more things I noted, mostly for myself:

Footnotes

  1. I made up this vaccine example, but extrapolating from adults to children is a quite common problem. See here for practical example in Guillian-Barre syndrome. Or recent work on systemic lupus erythematosus (SLE), described as Example 1 here. ↩

  2. Just to be clear on this rather obvious point: as the name suggests, guidance is there to outline FDA’s interpretation of existing regulations. So they only constitute a recommendation, not a binding obligation. However, given the regulatory burden, we can probably expect all of producers to follow what’s set out in guidance documents closely in order to stay on FDA’s good side. Also, there is a lot of them: 137 for conduct of clinical trials alone, although many are subject-specific things like, say, how to design a clinical lactation study etc. ↩

  3. Why is there a separate guidance on devices? See this short presentation from 2016. The presentation makes a point that devices have more prior information (they evolve slower and can be less variable across individuals). The adoption of Bayesian methods for devices dates all the way back to late 1990s, which I found really interesting. What are the recommendations in the guidance? According to the presentation there are two: one is on using information from previous studies as a prior (or using Bayesian hierarchical model, which is probably a bit too complex to explain here, but the idea driving this is similar); another is building adaptive designs into trials. More on that in the main text. ↩

  4. Here is a generic defintion: “Adaptive design can include changes to sample size, treatment allocation, endpoints, or statistical analysis, among other factors. Adaptive designs can help to reduce the number of patients needed for a trial, increase the likelihood of success, and shorten the time to completion, while still maintaining the scientific integrity of the trial. However, they require careful planning and implementation to ensure that they are valid, reliable, and transparent.” Why are they mentioned together with Bayesian trials? First of all, they do not have to be Bayesian, but they often make use of the Bayesian property I mentioned earlier, which is (simplifying here) continuous reassessment of the probability of various hypotheses with each piece of data. ↩

This post is licensed under CC BY 4.0 by the author.