CausalSmith · seminar slides

Measuring Full Adoption from Partial Rollout

When an experiment stops before everyone is treated, choosing when to measure can control the uncertainty from estimating full adoption.

Motivation

  • With interference, a unit’s outcome can depend on other units’ treatment assignments.
  • We want the mean outcome change between nobody treated and everybody treated.
  • The rollout reaches only a treated fraction q<1q<1.
  • Subtracting the baseline from the final measurement estimates the partial-rollout contrast.

Cortez et al. (2022) show how bounded-order interactions can make full-adoption effects identifiable through polynomial extrapolation.

How should we measure a partial rollout to estimate full adoption precisely?

Research Question

Suppose the mean response is a polynomial of degree at most β\beta, where β\beta is the maximum order.

A natural approach measures at β+1\beta+1 equally spaced fractions and extrapolates the fitted polynomial:

peq(β,q)j=qjβ,j=0,…,β. p^{\mathrm{eq}}(\beta,q)_j=\frac{qj}{\beta}, \qquad j=0,\ldots,\beta .

Here pjeqp^{\mathrm{eq}}_j is the treated fraction at measurement jj; qq is the final fraction.

  • Interpolation gives an unbiased estimate. That alone does not control how strongly its weights amplify noise.
  • Cortez-Rodriguez et al. (2024) study clustering to reduce variance for polynomial rollout estimation.
  • Choosing measurement fractions is an optimal-design problem, as in Kiefer and Wolfowitz (1959).

Which fractions control the variance cost of extrapolation?

Key Idea

Place more measurements near both endpoints of the observed interval.

These Chebyshev-Lobatto fractions control polynomial growth between measurements. Extrapolation beyond the interval still has an unavoidable cost.

  • Choose weights that recover the target for every polynomial of degree at most β\beta.
  • Among those weights, minimize the sum of their absolute values.
  • With more than the minimum number of measurements, Chebyshev-Lobatto placement attains the unavoidable exponential variance amplification up to constants.

This conclusion uses a static polynomial mean, correct round expectations, a common variance bound, and a budget bounded away from full adoption.

Half-rollout Example

At q=0.5q=0.5, measurements cover only [0,0.5][0,0.5], while the target includes the response at one.

The Chebyshev-Lobatto measurement fractions are

pjCh(k,q)=q2{1−cos⁡(πjk)}. p^{\mathrm{Ch}}_j(k,q) = \frac{q}{2}\left\{1-\cos\left(\frac{\pi j}{k}\right)\right\}.

Here kk is the number of rollout intervals, giving k+1k+1 measurements; j=0,…,kj=0,\ldots,k indexes them. The superscript Ch\mathrm{Ch} names this schedule.

  • At this budget, the schedule starts at zero, ends at 0.50.5, and concentrates measurements near both endpoints.
  • The variance-amplification base is approximately 5.835.83.
  • The matching bounds raise that base to 2β2\beta, up to constants.

The design controls what happens inside the observed half; it cannot eliminate extrapolation to one.

Model

For an outcome law PP, let mP(u)m_P(u) be the expected population mean at treated fraction uu.

mP(u)=∑ℓ=0βaP,ℓuℓ m_P(u)=\sum_{\ell=0}^{\beta} a_{P,\ell}u^\ell

The coefficients aP,ℓa_{P,\ell} describe the response curve. This restriction holds throughout [0,1][0,1], including the unobserved endpoint.

Our target is

τP=mP(1)−mP(0)=∑ℓ=1βaP,ℓ. \tau_P=m_P(1)-m_P(0)=\sum_{\ell=1}^{\beta} a_{P,\ell}.

Thus τP\tau_P adds the nonconstant coefficients; the intercept cancels.

At measurement fraction pjp_j, we observe the population mean Yˉj\bar Y_j. We require Eπ[Yˉj]=mP(pj)\mathbb E_\pi[\bar Y_j]=m_P(p_j), where π\pi denotes rollout randomization.

Static consistency: outcomes depend on current assignments, including others’ assignments, with no separate dependence on earlier rollout steps.

Assumptions

Common variance envelope: every round satisfies

Var⁡π(Yˉj)≤σ02n. \operatorname{Var}_\pi(\bar Y_j)\le \frac{\sigma_0^2}{n}.

Here nn is population size, and σ02≥0\sigma_0^2\ge0 sets the common variance scale. It incorporates outcome magnitudes and dependence across units.

  • Cross-round covariances are otherwise unrestricted, provided they form a valid covariance matrix.
  • Admissible schedules, denoted Sk,qS_{k,q}, start at zero, increase strictly, and end at qq.
  • Budget cap: 0<q≤qmax⁡<10<q\le q_{\max}<1, keeping full adoption outside the observed interval.
  • Oversampling: k≥cβk\ge c\beta, with fixed c>1c>1, gives more than the minimum β+1\beta+1 measurements.

Let Pβ\mathcal P_\beta denote laws satisfying static consistency, polynomiality, correct round expectations, and the variance envelope.

For the half-rollout example, polynomiality must hold beyond 0.50.5, all the way to one. Bounded-order interference motivates this restriction; we impose it.

Unbiased Estimation

Estimate the full-adoption contrast with

τ^w,p=∑j=0kwjYˉj, \hat\tau_{w,p}=\sum_{j=0}^k w_j\bar Y_j,

where wjw_j is the weight on measurement jj, and pp is the schedule.

The weights must recover the contrast for every polynomial term:

Wβ(p)={w∈Rk+1:∑j=0kwjpj0=0 and ∑j=0kwjpjℓ=1 for every ℓ=1,…,β}. W_\beta(p) = \left\{ w\in\mathbb R^{k+1}: \sum_{j=0}^k w_j p_j^0=0 \ \text{and}\ \sum_{j=0}^k w_j p_j^\ell=1 \ \text{for every } \ell=1,\ldots,\beta \right\}.

Here Wβ(p)W_\beta(p) is the set of polynomial-exact weights.

  • The first equation removes the intercept; pj0p_j^0 means one.
  • Each remaining equation recovers one nonconstant coefficient’s contribution to τP\tau_P.
  • Distinct nodes and k≥βk\ge\beta guarantee that these weights exist.

These equations deliver unbiasedness. What determines their variance?

Variance Cost

Large positive and negative weights can cancel in the expected estimate while magnifying noise.

informal · Theorem T-1 Under the rollout law restrictions, 1≤β≤k1\le\beta\le k, σ02≥0\sigma_0^2\ge0, and an admissible schedule, polynomial-exact weights exist, are unbiased, and obey a variance bound sharp over the covariance envelope.

For every polynomial-exact weight vector,

Var⁡π ⁣(∑j=0kwjYˉj)≤σ02n(∑j=0k∣wj∣)2. \operatorname{Var}_\pi\!\left(\sum_{j=0}^k w_j\bar Y_j\right) \le \frac{\sigma_0^2}{n}\left(\sum_{j=0}^k |w_j|\right)^2 .

Estimator variance is bounded by the common round variance times the squared sum of absolute weights.

Why absolute values? The envelope permits covariance signs that align with the weight signs. A valid covariance matrix attains this bound.

Unbiasedness constrains signed sums; variance control requires small absolute sums.

Design Objective

First choose the least costly polynomial-exact weights at a fixed schedule:

Aβ(p)=inf⁡w∈Wβ(p)(∑j=0k∣wj∣)2. A_\beta(p) = \inf_{w\in W_\beta(p)} \left(\sum_{j=0}^k |w_j|\right)^2.

Here Aβ(p)A_\beta(p) is the smallest worst-case variance multiplier available at schedule pp.

Then choose measurement fractions:

Mβ,k,q=inf⁡p∈Sk,qAβ(p). M_{\beta,k,q} = \inf_{p\in S_{k,q}} A_\beta(p).

Here Mβ,k,qM_{\beta,k,q} is the best multiplier across all admissible schedules with k+1k+1 measurements and budget qq.

Multiplying by σ02/n\sigma_0^2/n gives the optimal worst-case variance over all valid covariance matrices satisfying the round variance envelope.

Main Result

informal · Theorem T-2 For β≥1\beta\ge1, k≥cβk\ge c\beta, fixed c>1c>1, and 0<q≤qmax⁡<10<q\le q_{\max}<1, Chebyshev-Lobatto schedules attain the smallest worst-case variance amplification up to constants depending only on c,qmax⁡c,q_{\max}.

C−(qmax⁡)((1+1−q)2q)2β≤Mβ,k,q≤C+(c,qmax⁡)((1+1−q)2q)2β. C_-(q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta} \le M_{\beta,k,q} \le C_+(c,q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta}.

The positive constants C−C_- and C+C_+ are independent of q,β,kq,\beta,k; their arguments show what they depend on.

  • Every admissible schedule faces the lower bound.
  • Chebyshev-Lobatto placement attains the upper bound.
  • Chebyshev-Lobatto placement attains the unavoidable exponential rate up to constants.

At q=0.5q=0.5, with a fixed cap qmax⁡≥0.5q_{\max}\ge0.5 below one, the common base is approximately 5.835.83.

Why does measurement placement control this cost?

Hidden Polynomial Growth

The weight problem has an equivalent polynomial interpretation:

inf⁡w∈Wβ(p)∑j=0k∣wj∣=sup⁡{∣r(1)−r(0)∣:r∈R[x], deg⁡(r)≤β, max⁡0≤j≤k∣r(pj)∣≤1}. \inf_{w\in W_\beta(p)} \sum_{j=0}^k |w_j| = \sup\left\{ |r(1)-r(0)|: r\in\mathbb R[x],\ \deg(r)\le \beta,\ \max_{0\le j\le k}|r(p_j)|\le 1 \right\}.

Here rr is a test polynomial of degree at most β\beta; R[x]\mathbb R[x] denotes real polynomials.

The right side asks: How large can the full-adoption contrast be while every measured value stays within one?

  • A large hidden contrast forces large absolute weights.
  • Choosing fractions therefore means controlling polynomial growth that measurements fail to constrain.

In the half-rollout example, we bound rr at measurement fractions in [0,0.5][0,0.5], but its contrast involves r(1)r(1).

Between Measurements

Measurements must first control the polynomial throughout the observed interval.

Using the Chebyshev-Lobatto mesh inequality, oversampling with k≥cβk\ge c\beta, c>1c>1, gives

∣R(x)∣≤K(c)max⁡0≤j≤k∣R ⁣(−cos⁡ ⁣(πjk))∣. |R(x)| \le K(c) \max_{0\le j\le k}\left|R\!\left(-\cos\!\left(\frac{\pi j}{k}\right)\right)\right|.

Here RR is a polynomial of degree at most β\beta, x∈[−1,1]x\in[-1,1], and K(c)K(c) is a finite constant depending only on the oversampling ratio.

  • Rescaling [0,q][0,q] to [−1,1][-1,1] sends the measurement fractions to the nodes inside the maximum.
  • Small values at those nodes imply small values everywhere in the observed interval, up to K(c)K(c).
  • Oversampling supplies control between measurements.

For the half-rollout example, this step controls the entire interval [0,0.5][0,0.5]. How much growth remains outside it?

Beyond the Observed Interval

Under the same rescaling, full adoption becomes the exterior point xq=2/q−1>1x_q=2/q-1>1.

Its growth factor is λ(q)=xq+xq2−1\lambda(q)=x_q+\sqrt{x_q^2-1}.

For polynomials bounded throughout the rescaled interval,

for every real polynomial R with deg⁡(R)≤β and sup⁡x∈[−1,1]∣R(x)∣≤1,∣R(xq)−R(−1)∣≤C(qmax⁡) λ(q)β, \text{for every real polynomial }R\text{ with }\deg(R)\le \beta \text{ and }\sup_{x\in[-1,1]}|R(x)|\le 1,\qquad |R(x_q)-R(-1)|\le C(q_{\max})\,\lambda(q)^\beta,

Here R(−1)R(-1) corresponds to baseline and R(xq)R(x_q) to full adoption. The constant C(qmax⁡)C(q_{\max}) depends only on the budget cap.

  • The mesh inequality controls the observed interval.
  • This bound controls the remaining endpoint extrapolation.
  • The dual identity converts endpoint growth into absolute-weight size; squaring gives variance amplification.

The growth factor equals the main result’s base, (1+1−q)2/q(1+\sqrt{1-q})^2/q.

Unavoidable Amplification

A polynomial bounded throughout [0,q][0,q] satisfies the measurement restriction for every schedule.

The Chebyshev polynomial TβT_\beta, of degree β\beta, supplies such a polynomial after rescaling. Its endpoint contrast is at least Tβ(xq)−1T_\beta(x_q)-1.

Tβ(xq)−1≥c(qmax⁡) λ(q)β. T_\beta(x_q)-1\ge c(q_{\max})\,\lambda(q)^\beta .

This lower bound uses a positive constant c(qmax⁡)c(q_{\max}), distinct from the oversampling ratio cc.

  • Every placement inside the rollout interval admits this test polynomial.
  • The dual identity forces absolute-weight size at least proportional to λ(q)β\lambda(q)^\beta.
  • Squaring gives the matching variance-amplification lower bound.

The unavoidable cost comes from extrapolation. Chebyshev-Lobatto placement prevents additional uncontrolled growth between measurements.

Rollout Variance Guarantee

The envelope solution also supplies an upper bound when covariance comes from the rollout itself.

informal · Lemma L-12 Under the rollout law restrictions, σ02≥0\sigma_0^2\ge0, β≥1\beta\ge1, c>1c>1, 0<q≤qmax⁡<10<q\le q_{\max}<1, and k=⌈cβ⌉k=\lceil c\beta\rceil, Chebyshev-Lobatto placement supplies the following worst-case rollout variance upper bound.

inf⁡w∈Wβ(pCh(k,q))sup⁡P∈PβVar⁡π ⁣(∑j=0kwjYˉj)≤σ02n C+(c,qmax⁡)((1+1−q)2q)2β, \inf_{w\in W_\beta(p^{\mathrm{Ch}}(k,q))} \sup_{P\in\mathcal P_\beta} \operatorname{Var}_\pi\!\left(\sum_{j=0}^k w_j\bar Y_j\right) \le \frac{\sigma_0^2}{n}\, C_+(c,q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta},

This bounds the infimum worst-case rollout variance over polynomial-exact weights on the Chebyshev-Lobatto schedule. The ceiling rounds the interval count upward.

The reason is direct: rollout covariance satisfies the envelope, so the envelope upper bound applies.

Open Questions

  • Covariance-specific optimality: Does Chebyshev-Lobatto placement minimize worst-case variance under the covariance restrictions of a specified nested rollout? A matching lower bound remains open.
  • Clustered rollouts: How should measurement placement account for cluster size and dependence within clusters?
  • Mixed designs: How should fractions be chosen jointly with saturation, encouragement, or exposure-based randomization?

Takeaways

  • A static degree-β\beta mean curve over [0,1][0,1], with correct round expectations, makes the full-adoption contrast estimable through polynomial-exact weights.
  • Under the common variance envelope, the squared sum of absolute weights is the sharp worst-case variance multiplier.
  • With k≥cβk\ge c\beta, c>1c>1, and 0<q≤qmax⁡<10<q\le q_{\max}<1, Chebyshev-Lobatto placement controls growth between measurements and attains the unavoidable exponential amplification up to constants.

Appendix: Variance Envelope Design

Polynomial-exact weights are unbiased, and their squared absolute-weight sum gives the sharp worst-case variance over the diagonal covariance envelope.

Theorem T-1 (TV Envelope Design Bound)

Let n,k,βn,k,\beta be integers, let q∈Rq\in\mathbb R, and let σ02∈R\sigma_0^2\in\mathbb R. Fix a finite assignment space Ω\Omega, a finite design π\pi on Ω\Omega, rollout assignment vectors Z0,…,ZkZ_0,\ldots,Z_k, round means Yˉ0,…,Yˉk\bar Y_0,\ldots,\bar Y_k, a rollout mean curve mPm_P, polynomial coefficients aP,ℓa_{P,\ell}, and a schedule p=(p0,…,pk)p=(p_0,\ldots,p_k). Suppose that

  • (Order.) 1≤β≤k1\le \beta\le k.
  • (Envelope scale.) σ02≥0\sigma_0^2\ge 0.
  • (Schedule.) p∈Sk,qp\in S_{k,q}, in the sense of Definition P-1.
  • (Law class.) The design, rollout variables, round means, mean curve, coefficients, envelope scale, and schedule satisfy the rollout law-class restrictions Pβ\mathcal P_\beta in Definition P-2.

Then Wβ(p)W_\beta(p), as defined in Definition P-3, is nonempty. Moreover, for every w∈Wβ(p)w\in W_\beta(p), Eπ ⁣[∑j=0kwjYˉj]=mP(1)−mP(0) \mathbb E_\pi\!\left[\sum_{j=0}^k w_j\bar Y_j\right]=m_P(1)-m_P(0) and Var⁡π ⁣(∑j=0kwjYˉj)≤σ02n(∑j=0k∣wj∣)2. \operatorname{Var}_\pi\!\left(\sum_{j=0}^k w_j\bar Y_j\right) \le \frac{\sigma_0^2}{n}\left(\sum_{j=0}^k |w_j|\right)^2 . For every such ww, there also exists a positive semidefinite (k+1)×(k+1)(k+1)\times(k+1) matrix Γ\Gamma such that Γjj≤σ02nfor every j=0,…,k, \Gamma_{jj}\le \frac{\sigma_0^2}{n}\quad\text{for every }j=0,\ldots,k, and ∑i=0k∑j=0kwiΓijwj=σ02n(∑j=0k∣wj∣)2. \sum_{i=0}^k\sum_{j=0}^k w_i\Gamma_{ij}w_j = \frac{\sigma_0^2}{n}\left(\sum_{j=0}^k |w_j|\right)^2 . Consequently, with Aβ(⋅)A_\beta(\cdot) and Mβ,k,qM_{\beta,k,q} as in Definition P-5, inf⁡{v: ∃w∈Wβ(p) with v=σ02n(∑j=0k∣wj∣)2}=σ02nAβ(p), \inf\left\{ v:\ \exists w\in W_\beta(p)\text{ with } v=\frac{\sigma_0^2}{n}\left(\sum_{j=0}^k |w_j|\right)^2 \right\} = \frac{\sigma_0^2}{n}A_\beta(p), and inf⁡{v: ∃p′∈Sk,q with v=σ02nAβ(p′)}=σ02nMβ,k,q. \inf\left\{ v:\ \exists p'\in S_{k,q}\text{ with } v=\frac{\sigma_0^2}{n}A_\beta(p') \right\} = \frac{\sigma_0^2}{n}M_{\beta,k,q}. Finally, if p⋆∈Sk,qp^\star\in S_{k,q} satisfies Aβ(p⋆)=Mβ,k,q, A_\beta(p^\star)=M_{\beta,k,q}, then for every p′∈Sk,qp'\in S_{k,q}, σ02nAβ(p⋆)≤σ02nAβ(p′). \frac{\sigma_0^2}{n}A_\beta(p^\star) \le \frac{\sigma_0^2}{n}A_\beta(p').

Appendix: Chebyshev Minimax Amplification

With oversampling and budgets bounded away from one, Chebyshev-Lobatto schedules attain matching minimax amplification bounds up to constants.

Theorem T-2 (Chebyshev Minimax Amplification)

Fix a budget cap qmax⁡∈(0,1)q_{\max}\in(0,1). There exists a constant C−(qmax⁡)>0C_-(q_{\max})>0 such that, for every oversampling ratio c>1c>1, there exists a constant C+(c,qmax⁡)>0C_+(c,q_{\max})>0 with the following property. For every pair of integers β,k\beta,k, every rollout budget qq, and every schedule pp, suppose that

  • (Order and oversampling.) β≥1\beta\ge 1 and k≥cβk\ge c\beta.
  • (Low-budget regime.) q>0q>0 and the low-budget cap condition of Assumption A-4 holds for (q,qmax⁡)(q,q_{\max}).
  • (Admissible schedule.) When pp is used below, p∈Sk,qp\in S_{k,q} as in Definition P-1.

Then, with Aβ(⋅)A_\beta(\cdot) and Mβ,k,qM_{\beta,k,q} as in Definition P-5 and pCh(k,q)p^{\mathrm{Ch}}(k,q) as in Definition P-4, Aβ(p)≥C−(qmax⁡)((1+1−q)2q)2β, A_\beta(p) \ge C_-(q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta}, and Aβ ⁣(pCh(k,q))≤C+(c,qmax⁡)((1+1−q)2q)2β. A_\beta\!\left(p^{\mathrm{Ch}}(k,q)\right) \le C_+(c,q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta}. Moreover, the minimax amplification satisfies the matching two-sided bound C−(qmax⁡)((1+1−q)2q)2β≤Mβ,k,q≤C+(c,qmax⁡)((1+1−q)2q)2β. C_-(q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta} \le M_{\beta,k,q} \le C_+(c,q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta}.

Appendix: Rollout Variance Feasibility

Under the rollout law restrictions, Chebyshev-Lobatto placement supplies the stated exponential upper bound for worst-case rollout variance.

Lemma L-12 (Feasible Chebyshev Rate)

Fix an oversampling constant cc and a budget cap qmax⁡q_{\max} such that c>1c>1 and 0<qmax⁡<10<q_{\max}<1. Then there exists a constant C+(c,qmax⁡)>0C_+(c,q_{\max})>0 such that, for every population size nn, polynomial order β\beta, number of rollout intervals kk, finite assignment space Ω\Omega, finite design π\pi on Ω\Omega, rollout budget qq, and variance-envelope scale σ02\sigma_0^2, the following conditions imply the stated bound:

  • (Order.) β≥1\beta\ge 1.
  • (Budget.) q>0q>0 and qq satisfies the low-budget cap with qmax⁡q_{\max} in Assumption A-4.
  • (Variance scale.) σ02≥0\sigma_0^2\ge 0.
  • (Chebyshev size.) k=⌈cβ⌉k=\lceil c\beta\rceil.
  • (Schedule.) p=pCh(k,q)p=p^{\mathrm{Ch}}(k,q) is the shifted Chebyshev-Lobatto schedule of Definition P-4.

Under these conditions, inf⁡w∈Wβ(pCh(k,q))sup⁡P∈PβVar⁡π ⁣(∑j=0kwjYˉj)≤σ02n C+(c,qmax⁡)((1+1−q)2q)2β, \inf_{w\in W_\beta(p^{\mathrm{Ch}}(k,q))} \sup_{P\in\mathcal P_\beta} \operatorname{Var}_\pi\!\left(\sum_{j=0}^k w_j\bar Y_j\right) \le \frac{\sigma_0^2}{n}\, C_+(c,q_{\max}) \left(\frac{(1+\sqrt{1-q})^2}{q}\right)^{2\beta}, where Wβ(pCh(k,q))W_\beta(p^{\mathrm{Ch}}(k,q)) is the unbiased weight set of Definition P-3, and the supremum ranges over rollout laws P∈PβP\in\mathcal P_\beta satisfying Definition P-2 at the schedule pCh(k,q)p^{\mathrm{Ch}}(k,q) with variance-envelope scale σ02\sigma_0^2.