Single binary endpoint

This vignette describes the analysis of a proof-of-concept (PoC) study with one binary endpoint (Kang, Yamaguchi and Han, 2026). vignette("overview") introduces the decision criteria.

Model

In group $k$ ($k = t$ for treatment, $c$ for control), $y_k$ of $n_k$ subjects respond, $y_k \sim \textrm{Binomial}(n_k, \pi_k)$, and the response rate has the prior distribution $\pi_k \sim \textrm{Beta}(a_k, b_k)$ (prior_beta(); prior_beta(0.5, 0.5) is the Jeffreys prior). The posterior distribution is \[\pi_k \mid y_k \sim \textrm{Beta}(a_k + y_k, b_k + n_k - y_k),\] and the treatment effect is $\theta = \pi_t - \pi_c$. The posterior probability $\Pr(\theta > q \mid D)$ is computed by numerical integration (pbetadiff()).

For the predictive criterion, $m_k$ future subjects of group $k$ have $\tilde y_k$ responders, whose posterior predictive distribution is beta-binomial. The future effect is $\tilde\theta = \tilde y_t / m_t - \tilde y_c / m_c$, and its predictive probabilities are computed exactly by summation (pbetabinomdiff()).

Decision for observed data

A study with 12 subjects in each group, Jeffreys priors and a posterior criterion with $\theta_{TV} = 0.2$, $\theta_{MAV} = 0.1$, $\gamma_{go} = 0.5$ and $\gamma_{nogo} = 0.8$:

jeffreys <- prior_beta(0.5, 0.5)
crit <- criterion_posterior(theta_tv = 0.2, theta_mav = 0.1,
                            gamma_go = 0.5, gamma_nogo = 0.8)
des <- design_binary(n_t = 12, n_c = 12, prior_t = jeffreys,
                     prior_c = jeffreys, criterion = crit)
dec <- qdm_decide(des, data_t = data_binary(12, 9),
                  data_c = data_binary(12, 3))
dec
#> Decision: Go
#>   Go:   P(theta > 0.2 | data) = 0.9304 (threshold 0.5, met)
#>   NoGo: P(theta <= 0.1 | data) = 0.02338 (threshold 0.8, not met)
#> Posterior distributions
#>   Treatment: Beta(a = 9.5, b = 3.5)
#>   Control:   Beta(a = 3.5, b = 9.5)

The probabilities of the Go and NoGo regions are the posterior probabilities $\Pr(\theta > 0.2 \mid D)$ and $\Pr(\theta \le 0.1 \mid D)$, which pbetadiff() gives from the posterior parameters:

post_t <- dec$posterior_t
post_c <- dec$posterior_c
pbetadiff(0.2, a_t = post_t[["a"]], b_t = post_t[["b"]],
          a_c = post_c[["a"]], b_c = post_c[["b"]], lower.tail = FALSE)
#> [1] 0.9303746
pbetadiff(0.1, a_t = post_t[["a"]], b_t = post_t[["b"]],
          a_c = post_c[["a"]], b_c = post_c[["b"]])
#> [1] 0.02338322

With 9 and 3 responders, the decision is Go.

Forms of the control group

External control data

External data enter through a power prior: with $n_e$ subjects, $y_e$ responders and weight $a_0$ (from 0 to 1), the posterior distribution of the response rate is \[\textrm{Beta}(a + a_0 y_e + y, \; b + a_0 (n_e - y_e) + n - y).\] With $a_0 = 0$ the external data are ignored, and with $a_0 = 1$ they are pooled with the current data. Borrowing half of 20 external control subjects with 5 responders:

des_ext <- design_binary(
  n_t = 12, n_c = 12, prior_t = jeffreys, prior_c = jeffreys,
  criterion = crit,
  external_c = borrow(data_binary(n = 20, y = 5), a0 = 0.5)
)
dec_ext <- qdm_decide(des_ext, data_t = data_binary(12, 9),
                      data_c = data_binary(12, 3))
dec_ext$posterior_c
#>  a  b 
#>  6 17

The external data add 2.5 responders and 7.5 non-responders to the posterior distribution of the control group, and the probability of the Go region becomes 0.955 (without them, 0.93).

Without a concurrent control group (n_c = 0), the control group can be described by the external data alone:

des_ext0 <- design_binary(
  n_t = 12, n_c = 0, prior_t = jeffreys, prior_c = jeffreys,
  criterion = crit,
  external_c = borrow(data_binary(n = 40, y = 10), a0 = 1)
)
qdm_decide(des_ext0, data_t = data_binary(12, 9))
#> Decision: Go
#>   Go:   P(theta > 0.2 | data) = 0.9684 (threshold 0.5, met)
#>   NoGo: P(theta <= 0.1 | data) = 0.006469 (threshold 0.8, not met)
#> Posterior distributions
#>   Treatment: Beta(a = 9.5, b = 3.5)
#>   Control:   Beta(a = 10.5, b = 30.5)

External treatment data (external_t) can be used in the same way with any form of the control group.

Hypothetical control

For an uncontrolled study, Kang et al. (2026, Section 4.1.4) describe the control response rate by a hypothetical control, $\pi_c \sim \textrm{Beta}(a_t + z, b_t + n_t - z)$, where $z$ is the number of responders expected among $n_t$ control subjects. This is obtained with prior_c equal to prior_t and hypothetical(data_binary(n_t, z)):

des_hyp <- design_binary(
  n_t = 12, n_c = 0, prior_t = jeffreys, prior_c = jeffreys,
  criterion = crit,
  hypothetical_c = hypothetical(data = data_binary(n = 12, y = 3))
)
qdm_decide(des_hyp, data_t = data_binary(12, 9))
#> Decision: Go
#>   Go:   P(theta > 0.2 | data) = 0.9304 (threshold 0.5, met)
#>   NoGo: P(theta <= 0.1 | data) = 0.02338 (threshold 0.8, not met)
#> Posterior distributions
#>   Treatment: Beta(a = 9.5, b = 3.5)
#>   Control:   Beta(a = 3.5, b = 9.5)

Fixed control

A single-arm study is often compared with a known benchmark, such as the response rate of the standard of care reported in the literature. With fixed_c, the control response rate is a known value $\pi_{c0}$, the treatment effect is $\theta = \pi_t - \pi_{c0}$, and prior_c is not used (it must be NULL). The probability of the Go region is then the Beta probability $\Pr(\pi_t > \pi_{c0} + \theta_{TV} \mid D)$. A study with 20 subjects compared with a benchmark of 0.2:

crit_fix <- criterion_posterior(theta_tv = 0.2, theta_mav = 0.05,
                                gamma_go = 0.8, gamma_nogo = 0.8)
des_fix <- design_binary(n_t = 20, n_c = 0, prior_t = prior_beta(1, 1),
                         criterion = crit_fix, fixed_c = 0.2)
dec_fix <- qdm_decide(des_fix, data_t = data_binary(20, 11))
dec_fix
#> Decision: Go
#>   Go:   P(theta > 0.2 | data) = 0.9151 (threshold 0.8, met)
#>   NoGo: P(theta <= 0.05 | data) = 0.001687 (threshold 0.8, not met)
#> Posterior distributions
#>   Treatment: Beta(a = 12, b = 10)
#>   Control:   fixed value 0.2
pbeta(0.2 + 0.2, 1 + 11, 1 + 20 - 11, lower.tail = FALSE)
#> [1] 0.9150757

With fixed_c = 0, the effect is the response rate itself, and the thresholds are response rates. For the predictive criterion, the future effect is $\tilde y_t / m_t - \pi_{c0}$, and m_c is not used.

Predictive criterion

The predictive criterion asks whether a future study with $m_t$ and $m_c$ subjects would observe an effect larger than $\theta_{NULL}$. With 40 subjects per group in the future study and $\theta_{NULL} = 0$:

des_pred <- design_binary(
  n_t = 12, n_c = 12, prior_t = jeffreys, prior_c = jeffreys,
  criterion = criterion_predictive(theta_null = 0, m_t = 40,
                                   gamma_go = 0.9, gamma_nogo = 0.7)
)
dec_pred <- qdm_decide(des_pred, data_t = data_binary(12, 9),
                       data_c = data_binary(12, 3))
dec_pred
#> Decision: Go
#>   Go:   P(future theta > 0 | data) = 0.9835 (threshold 0.9, met)
#>   NoGo: P(future theta <= 0 | data) = 0.01647 (threshold 0.7, not met)
#> Posterior distributions
#>   Treatment: Beta(a = 9.5, b = 3.5)
#>   Control:   Beta(a = 3.5, b = 9.5)
post_t <- dec_pred$posterior_t
post_c <- dec_pred$posterior_c
pbetabinomdiff(0, m_t = 40, a_t = post_t[["a"]], b_t = post_t[["b"]],
               m_c = 40, a_c = post_c[["a"]], b_c = post_c[["b"]],
               lower.tail = FALSE)
#> [1] 0.9835266

The Go and NoGo regions of the predictive criterion are complementary, so their probabilities add up to 1. A Gray decision is then possible only when $\gamma_{go} + \gamma_{nogo} > 1$.

Operating characteristics

qdm_oc() computes the probabilities of Go, NoGo, Gray and Miss decisions under scenarios for the true response rates. All outcomes $(y_t, y_c)$ of the study are enumerated and weighted by their binomial probabilities, so the results are exact.

oc <- qdm_oc(des, data.frame(pi_t = seq(0.2, 0.8, by = 0.05), pi_c = 0.2))
oc
#> Operating characteristics (exact enumeration of the outcomes)
#>  pi_t pi_c     Go      NoGo    Gray Miss
#>  0.20  0.2 0.0984 0.4451679 0.45643    0
#>  0.25  0.2 0.1739 0.3215025 0.50463    0
#>  0.30  0.2 0.2678 0.2236534 0.50854    0
#>  0.35  0.2 0.3741 0.1497882 0.47614    0
#>  0.40  0.2 0.4853 0.0963433 0.41831    0
#>  0.45  0.2 0.5943 0.0592599 0.34640    0
#>  0.50  0.2 0.6947 0.0346440 0.27063    0
#>  0.55  0.2 0.7819 0.0190887 0.19905    0
#>  0.60  0.2 0.8531 0.0098010 0.13713    0
#>  0.65  0.2 0.9076 0.0046167 0.08774    0
#>  0.70  0.2 0.9466 0.0019515 0.05145    0
#>  0.75  0.2 0.9722 0.0007164 0.02710    0
#>  0.80  0.2 0.9874 0.0002169 0.01240    0
plot(oc)

The vertical lines mark the effects $\theta_{MAV}$ and $\theta_{TV}$. For the uncontrolled design, the horizontal axis is the response rate of the treatment group, and the lines are shifted by the mean of the hypothetical control distribution:

oc_hyp <- qdm_oc(des_hyp, data.frame(pi_t = seq(0.2, 0.8, by = 0.05)))
plot(oc_hyp)

Decision boundary

qdm_boundary() shows how many responders lead to each decision. Unlike the operating characteristics, it does not depend on the true response rates. For the design with a concurrent control group, it gives the numbers of treatment responders that lead to Go, NoGo and Gray decisions for each number of control responders, together with the observed response rates of the treatment group and the observed differences in response rates:

b <- qdm_boundary(des)
b
#> Decision boundary of a design with one binary endpoint
#>   n_t = 12, n_c = 12
#> 
#> Responders in the treatment group (y_t)
#>   y_c   NoGo   Gray    Go
#>   0     0      1-2     3-12
#>   1     0-1    2-3     4-12
#>   2     0-1    2-4     5-12
#>   3     0-2    3-5     6-12
#>   4     0-3    4-6     7-12
#>   5     0-4    5-7     8-12
#>   6     0-5    6-8     9-12
#>   7     0-6    7-9     10-12
#>   8     0-7    8-10    11-12
#>   9     0-8    9-11    12
#>   10    0-9    10-12   -
#>   11    0-11   12      -
#>   12    0-12   -       -
#> 
#> Observed response rate (y_t / n_t)
#>   y_c   NoGo             Gray              Go
#>   0     0.0%             8.3% to 16.7%     25.0% to 100.0%
#>   1     0.0% to 8.3%     16.7% to 25.0%    33.3% to 100.0%
#>   2     0.0% to 8.3%     16.7% to 33.3%    41.7% to 100.0%
#>   3     0.0% to 16.7%    25.0% to 41.7%    50.0% to 100.0%
#>   4     0.0% to 25.0%    33.3% to 50.0%    58.3% to 100.0%
#>   5     0.0% to 33.3%    41.7% to 58.3%    66.7% to 100.0%
#>   6     0.0% to 41.7%    50.0% to 66.7%    75.0% to 100.0%
#>   7     0.0% to 50.0%    58.3% to 75.0%    83.3% to 100.0%
#>   8     0.0% to 58.3%    66.7% to 83.3%    91.7% to 100.0%
#>   9     0.0% to 66.7%    75.0% to 91.7%    100.0%
#>   10    0.0% to 75.0%    83.3% to 100.0%   -
#>   11    0.0% to 91.7%    100.0%            -
#>   12    0.0% to 100.0%   -                 -
#> 
#> Observed difference in response rates (y_t / n_t - y_c / n_c)
#>   y_c   NoGo                  Gray                Go
#>   0     0                     0.08333 to 0.1667   0.25 to 1
#>   1     -0.08333 to 0         0.08333 to 0.1667   0.25 to 0.9167
#>   2     -0.1667 to -0.08333   0 to 0.1667         0.25 to 0.8333
#>   3     -0.25 to -0.08333     0 to 0.1667         0.25 to 0.75
#>   4     -0.3333 to -0.08333   0 to 0.1667         0.25 to 0.6667
#>   5     -0.4167 to -0.08333   0 to 0.1667         0.25 to 0.5833
#>   6     -0.5 to -0.08333      0 to 0.1667         0.25 to 0.5
#>   7     -0.5833 to -0.08333   0 to 0.1667         0.25 to 0.4167
#>   8     -0.6667 to -0.08333   0 to 0.1667         0.25 to 0.3333
#>   9     -0.75 to -0.08333     0 to 0.1667         0.25
#>   10    -0.8333 to -0.08333   0 to 0.1667         -
#>   11    -0.9167 to 0          0.08333             -
#>   12    -1 to 0               -                   -
plot(b)

The plot shows the regions of the decisions with the boundary values written on them, and the axis on the right gives the observed response rate of the treatment group. With 3 control responders, a Go decision needs at least 6 treatment responders, an observed difference in response rates of 0.25.

Without a concurrent control group, the boundary is shown for several sample sizes of the treatment group, here from 10 to 40 subjects for the design with a fixed control:

b_fix <- qdm_boundary(des_fix, n_t = seq(10, 40, by = 5))
b_fix
#> Decision boundary of a design with one binary endpoint
#> 
#> Responders in the treatment group (y_t)
#>   n_t   NoGo   Gray   Go
#>   10    0-1    2-5    6-10
#>   15    0-2    3-7    8-15
#>   20    0-3    4-9    10-20
#>   25    0-4    5-11   12-25
#>   30    0-5    6-14   15-30
#>   35    0-6    7-16   17-35
#>   40    0-7    8-18   19-40
#> 
#> Observed response rate (y_t / n_t)
#>   n_t   NoGo            Gray             Go
#>   10    0.0% to 10.0%   20.0% to 50.0%   60.0% to 100.0%
#>   15    0.0% to 13.3%   20.0% to 46.7%   53.3% to 100.0%
#>   20    0.0% to 15.0%   20.0% to 45.0%   50.0% to 100.0%
#>   25    0.0% to 16.0%   20.0% to 44.0%   48.0% to 100.0%
#>   30    0.0% to 16.7%   20.0% to 46.7%   50.0% to 100.0%
#>   35    0.0% to 17.1%   20.0% to 45.7%   48.6% to 100.0%
#>   40    0.0% to 17.5%   20.0% to 45.0%   47.5% to 100.0%
#> 
#> Observed difference from the fixed control rate (y_t / n_t - 0.2)
#>   n_t   NoGo               Gray          Go
#>   10    -0.2 to -0.1       0 to 0.3      0.4 to 0.8
#>   15    -0.2 to -0.06667   0 to 0.2667   0.3333 to 0.8
#>   20    -0.2 to -0.05      0 to 0.25     0.3 to 0.8
#>   25    -0.2 to -0.04      0 to 0.24     0.28 to 0.8
#>   30    -0.2 to -0.03333   0 to 0.2667   0.3 to 0.8
#>   35    -0.2 to -0.02857   0 to 0.2571   0.2857 to 0.8
#>   40    -0.2 to -0.025     0 to 0.25     0.275 to 0.8
plot(b_fix)

With 20 subjects, a Go decision needs at least 10 responders, and the decision is NoGo with at most 3 responders. The data frame returned by qdm_boundary() has one row per decision interval, so such values can be extracted from its columns from and to.

Smaller effects as favorable

For a safety endpoint, such as the occurrence of an adverse event, a smaller rate in the treatment group is favorable. With direction = "less" in the criterion, the Go region is $\theta < \theta_{TV}$ and the NoGo region is $\theta \ge \theta_{MAV}$, with $\theta_{TV} < \theta_{MAV}$. Here the decision is Go if the rate of the adverse event is likely to be reduced by more than 0.1, and NoGo if it is likely not to be reduced:

crit_ae <- criterion_posterior(theta_tv = -0.1, theta_mav = 0,
                               gamma_go = 0.6, gamma_nogo = 0.7,
                               direction = "less")
des_ae <- design_binary(n_t = 12, n_c = 12, prior_t = jeffreys,
                        prior_c = jeffreys, criterion = crit_ae)
dec_ae <- qdm_decide(des_ae, data_t = data_binary(12, 2),
                     data_c = data_binary(12, 6))
dec_ae
#> Decision: Go
#>   Go:   P(theta < -0.1 | data) = 0.8852 (threshold 0.6, met)
#>   NoGo: P(theta >= 0 | data) = 0.04018 (threshold 0.7, not met)
#> Posterior distributions
#>   Treatment: Beta(a = 2.5, b = 10.5)
#>   Control:   Beta(a = 6.5, b = 6.5)
post_t <- dec_ae$posterior_t
post_c <- dec_ae$posterior_c
pbetadiff(-0.1, a_t = post_t[["a"]], b_t = post_t[["b"]],
          a_c = post_c[["a"]], b_c = post_c[["b"]])
#> [1] 0.8852471
pbetadiff(0, a_t = post_t[["a"]], b_t = post_t[["b"]],
          a_c = post_c[["a"]], b_c = post_c[["b"]], lower.tail = FALSE)
#> [1] 0.04018446

The probability of a Go decision now decreases with the true effect, and a Go decision needs few treatment subjects with the adverse event:

plot(qdm_oc(des_ae, data.frame(pi_t = seq(0.1, 0.6, by = 0.05),
                               pi_c = 0.4)))

qdm_boundary(des_ae)
#> Decision boundary of a design with one binary endpoint
#>   n_t = 12, n_c = 12
#> 
#> Responders in the treatment group (y_t)
#>   y_c   Go     Gray    NoGo
#>   0     -      0       1-12
#>   1     -      0-1     2-12
#>   2     0      1-3     4-12
#>   3     0-1    2-4     5-12
#>   4     0-2    3-5     6-12
#>   5     0-3    4-6     7-12
#>   6     0-4    5-7     8-12
#>   7     0-5    6-8     9-12
#>   8     0-6    7-9     10-12
#>   9     0-7    8-10    11-12
#>   10    0-8    9-10    11-12
#>   11    0-9    10-11   12
#>   12    0-10   11-12   -
#> 
#> Observed response rate (y_t / n_t)
#>   y_c   Go              Gray              NoGo
#>   0     -               0.0%              8.3% to 100.0%
#>   1     -               0.0% to 8.3%      16.7% to 100.0%
#>   2     0.0%            8.3% to 25.0%     33.3% to 100.0%
#>   3     0.0% to 8.3%    16.7% to 33.3%    41.7% to 100.0%
#>   4     0.0% to 16.7%   25.0% to 41.7%    50.0% to 100.0%
#>   5     0.0% to 25.0%   33.3% to 50.0%    58.3% to 100.0%
#>   6     0.0% to 33.3%   41.7% to 58.3%    66.7% to 100.0%
#>   7     0.0% to 41.7%   50.0% to 66.7%    75.0% to 100.0%
#>   8     0.0% to 50.0%   58.3% to 75.0%    83.3% to 100.0%
#>   9     0.0% to 58.3%   66.7% to 83.3%    91.7% to 100.0%
#>   10    0.0% to 66.7%   75.0% to 83.3%    91.7% to 100.0%
#>   11    0.0% to 75.0%   83.3% to 91.7%    100.0%
#>   12    0.0% to 83.3%   91.7% to 100.0%   -
#> 
#> Observed difference in response rates (y_t / n_t - y_c / n_c)
#>   y_c   Go                   Gray                  NoGo
#>   0     -                    0                     0.08333 to 1
#>   1     -                    -0.08333 to 0         0.08333 to 0.9167
#>   2     -0.1667              -0.08333 to 0.08333   0.1667 to 0.8333
#>   3     -0.25 to -0.1667     -0.08333 to 0.08333   0.1667 to 0.75
#>   4     -0.3333 to -0.1667   -0.08333 to 0.08333   0.1667 to 0.6667
#>   5     -0.4167 to -0.1667   -0.08333 to 0.08333   0.1667 to 0.5833
#>   6     -0.5 to -0.1667      -0.08333 to 0.08333   0.1667 to 0.5
#>   7     -0.5833 to -0.1667   -0.08333 to 0.08333   0.1667 to 0.4167
#>   8     -0.6667 to -0.1667   -0.08333 to 0.08333   0.1667 to 0.3333
#>   9     -0.75 to -0.1667     -0.08333 to 0.08333   0.1667 to 0.25
#>   10    -0.8333 to -0.1667   -0.08333 to 0         0.08333 to 0.1667
#>   11    -0.9167 to -0.1667   -0.08333 to 0         0.08333
#>   12    -1 to -0.1667        -0.08333 to 0         -

For the predictive criterion with direction = "less", the Go region is that the future effect is below $\theta_{NULL}$, and the NoGo region that it is at least $\theta_{NULL}$.

Calibration of the thresholds

qdm_calibrate() chooses $\gamma_{go}$ as the smallest value of a grid for which the probability of meeting the Go criterion under scenario_go is at most target_go, and $\gamma_{nogo}$ in the same way for the NoGo criterion under scenario_nogo. Here the Go criterion should be met with probability at most 0.05 when the treatment has no effect, and the NoGo criterion with probability at most 0.2 when the effect is 0.3:

cal <- qdm_calibrate(des,
                     scenario_go = data.frame(pi_t = 0.2, pi_c = 0.2),
                     scenario_nogo = data.frame(pi_t = 0.5, pi_c = 0.2),
                     target_go = 0.05, target_nogo = 0.2)
cal
#> Calibration of the decision thresholds (exact enumeration of the outcomes)
#>   gamma_go   = 0.58: P(Go criterion met | scenario_go) = 0.03558 (target 0.05)
#>   gamma_nogo = 0.39: P(NoGo criterion met | scenario_nogo) = 0.1708 (target 0.2)
#> Decision probabilities with the chosen thresholds
#>       scenario pi_t pi_c      Go   NoGo   Gray Miss
#>    scenario_go  0.2  0.2 0.03558 0.7810 0.1835    0
#>  scenario_nogo  0.5  0.2 0.52502 0.1708 0.3041    0
plot(cal)

The chosen thresholds are $\gamma_{go} =$ 0.58 and $\gamma_{nogo} =$ 0.39. The design with these thresholds is cal$design:

qdm_oc(cal$design, data.frame(pi_t = c(0.2, 0.35, 0.5), pi_c = 0.2))
#> Operating characteristics (exact enumeration of the outcomes)
#>  pi_t pi_c      Go   NoGo   Gray Miss
#>  0.20  0.2 0.03558 0.7810 0.1835    0
#>  0.35  0.2 0.21479 0.4424 0.3428    0
#>  0.50  0.2 0.52502 0.1708 0.3041    0