CausalSmith · seminar slides
Who Benefits when Outcomes Are Missing?
We bound the fraction who benefit from treatment among people whose receipt responds to an offer and whose outcome would be observed either way.
slides for Sharp Ordinal Benefit Bounds for Survivor Compliers under Selection and Noncompliance
Research Question
- A training offer changes treatment receipt, while wage categories are observed only for employed people.
- Comparing employed recipients with employed nonrecipients mixes treatment effects with differences in who is employed.
- Survivor compliers respond to the offer and would be observed under either treatment. Kennedy et al. (2019) study this group.
- Dong and Heiler (2026) recover selected-complier information when selection directions vary with covariates.
- Lu et al. (2018) bound ordinal benefit from complete outcome margins. Here, the selected groups can differ in size, and survivor membership is unknown.
What fraction of survivor compliers would have a strictly higher outcome under treatment?
Key Idea
An outcome bin supplies a capacity: the selected-complier probability mass available at that level.
Selection monotonicity makes the two selected groups nested within each covariate cell. Their smaller total is exactly the survivor-complier mass.
Pair that amount across the outcome lists, allowing unused capacity in the larger group.
Under the IV and selection assumptions, ordered outcomes give closed formulas for the smallest and largest possible benefit shares. Both endpoints are attainable.
The unknown pairing remains uncertain; its total mass becomes known.
Example
One covariate cell x⋆, three outcome levels, and full compliance: receipt equals instrument assignment. Each assignment has probability 1/2.
Entries are joint probabilities of selection and outcome, conditional on assignment.
| Outcome status | Untreated arm | Treated arm | |---|---:|---:| | Selected, level 0 | 0.075 | 0.10 | | Selected, level 1 | 0.10 | 0.25 | | Selected, level 2 | 0.075 | 0.15 | | Not selected | 0.75 | 0.50 |
Assume treatment can increase selection without removing anyone. The untreated selected mass 1/4 consists entirely of survivors; the treated selected mass is 1/2.
ΘI(Pobs⋆)=[0,107].
Here Pobs⋆ denotes these data, and ΘI is the set of compatible benefit probabilities. The same data permit both zero benefit and 70% benefit.
Which people and outcome pairs make these endpoints possible?
Model
X indexes finitely many covariate cells. The instrument Z, receipt D, and selection S are binary. The observed outcome Y has ordered levels Y={0,…,K−1}, with K≥3.
Potential receipts D0,D1 vary instrument assignment:
D=DZ
Potential selections S0,S1 vary received treatment:
S=SD
Potential outcomes Y0,Y1 also vary received treatment:
Y=YD
The last equation applies when S=1. Together, these equations impose consistency and exclude direct instrument effects on selection and outcomes.
Compliers are C={D0=0,D1=1}. Our target is θ=P(Y1>Y0∣S1=S0=1,C): strict improvement for the same survivor complier.
Assumptions
(D0,D1,S0,S1,Y0,Y1)⊥Z∣X.
IV independence: within each covariate cell, assignment is independent of all potential receipts, selections, and outcomes.
- Consistency and exclusion: the three model equations connect observed variables to potential variables through received treatment.
- Overlap: εZ≤π(x)≤1−εZ, where π(x)=P(Z=1∣X=x) and 0<εZ<1/2. Both assignments occur in each supported cell.
- Treatment monotonicity: D1≥D0. Nobody takes treatment only under the lower instrument state.
- Cellwise selection monotonicity: among compliers within a cell, either everyone satisfies S0≤S1, or everyone satisfies S1≤S0. The direction may differ across cells.
- Positive survivor mass: the aggregate survivor-complier probability M is positive.
In the example, everyone complies and S0≤S1: treatment adds selected people without removing anyone.
Recovering the Capacities
Use joint events containing receipt, selection, and outcome. Condition on assignment and covariates—not on receipt.
For untreated level i, define:
ℓi(x)=P(D=0,S=1,Y=i∣Z=0,X=x)−P(D=0,S=1,Y=i∣Z=1,X=x).
For treated level j, reverse the subtraction:
hj(x)=P(D=1,S=1,Y=j∣Z=1,X=x)−P(D=1,S=1,Y=j∣Z=0,X=x).
Instrument independence makes the groups whose receipt never changes cancel. Treatment monotonicity leaves compliers:
ℓi(x)=P(Y0=i,S0=1,C∣X=x)
This identifies untreated selected-complier mass at level i.
hj(x)=P(Y1=j,S1=1,C∣X=x)
This identifies treated selected-complier mass at level j. Add each arm’s capacities:
q0(x)=i∈Y∑ℓi(x),q1(x)=j∈Y∑hj(x),
The totals count selected compliers under each treatment.
Δq(x)=q1(x)−q0(x),m(x)=min{q0(x),q1(x)}.
The gap Δq(x) measures the selection change. Nested selected groups have intersection mass m(x).
In the example, q0=1/4, q1=1/2, and m=1/4; another 1/4 is selected only under treatment. Equal totals imply equal selection status among compliers.
We know how many survivors there are, while their membership in the larger group remains unknown.
Pairing Exactly the Survivors
Let γij,x be survivor-complier mass with untreated outcome i and treated outcome j, conditional on cell x.
Γx∗=⎩⎨⎧γx≥0:j∈Y∑γij,x≤ℓi(x), i∈Y∑γij,x≤hj(x), i,j∈Y∑γij,x=m(x)⎭⎬⎫.
The set Γx∗ contains feasible pairing arrays: each row and column respects its available capacity, and exactly the survivor mass is paired.
Benefit mass is bx(γx)=∑i<jγij,x: count pairs with a higher treated outcome.
In the example, every untreated capacity must be used, while only 1/4 of the treated capacity is used.
Separately normalizing the two selected lists would pair different populations. These constraints retain the uncertainty about which treated selected people are survivors.
Main Result
Let BL(x) and BU(x) be the minimum and maximum feasible benefit masses in cell x.
Write px=P(X=x) for its population weight and M=∑xpxm(x) for aggregate survivor-complier mass.
informal · Theorem T-6 In the finite ordered-outcome model, IV independence, consistency/exclusion, overlap, treatment monotonicity, cellwise selection monotonicity and M>0 make the threshold interval exactly the feasible benefit probabilities.
ΘI(Pobs)=[M∑x∈XpxBL(x),M∑x∈XpxBU(x)].
Here Pobs is the observed-data law. Each endpoint adds population-weighted benefit masses and divides by population-weighted survivor mass.
Sharpness: every value in this interval is compatible with the data and assumptions.
For the example, BL=0, BU=7/40, and M=1/4, giving [0,0.7].
How do we find—and attain—the two cell endpoints?
Upper Threshold Bound
Choose an outcome threshold t. Write ℓ<t(x)=∑i<tℓi(x) for untreated mass below it and h>t(x)=∑j>thj(x) for treated mass above it.
Every beneficial pair i<j uses at least one of these two capacities: if i≥t, then j>t.
BU(x)=min{m(x), t∈Ymin[ℓ<t(x)+h>t(x)]}.
Maximum benefit mass is limited by survivor mass and by the tightest threshold capacity.
In the example, t=2 leaves only untreated levels 0 and 1 available for improvement. Their mass is 7/40.
The bound is necessary. Why is it sufficient?
Ordered Pairing
To maximize benefit, serve untreated levels from highest to lowest: higher starting outcomes have fewer improving partners.
- For the current untreated level i, find the lowest treated level with unused capacity and j>i.
- Pair as much mass as those two bins allow. Advance whenever a bin is exhausted.
- If no improving partner remains, leave that row’s residual for completion. Continue to lower untreated levels.
- Stop the benefit allocation if it reaches survivor mass m(x).
A lower untreated level can use every improving partner available to a higher level. Exchanging partners therefore protects the hardest-to-match rows without losing benefit.
For rows i≥t, the combined improving partner set is j>t. Any collection’s partner set is determined by its lowest level; adding all higher rows increases demand without expanding that set.
Thus consecutive upper groups give the strongest shortages. Rows below t contribute at most ℓ<t(x); the rest share at most h>t(x).
The ordered pairing exhausts these shortages. Complete to mass m(x) with residual capacities; no available improving pair remains unless the benefit allocation already reached m(x).
Example: Attaining 70%
Apply the ordered pairing to the same three-level capacities.
| Untreated Y0 | Treated Y1 | Survivor mass | Role | |---:|---:|---:|---| | 1 | 2 | 1/10 | Improving pair | | 0 | 1 | 3/40 | Improving pair | | 2 | 0 | 3/40 | Residual completion |
Level 2 cannot improve. Match level 1 to level 2, then level 0 to level 1. Improving mass is 7/40; completion brings survivor mass to 1/4, giving benefit share 7/10.
Assign unused treated capacity to the 1/4 selected only under treatment. The remaining 1/2 are never selected. This reproduces the observed arms.
More generally, put unused selected capacity into the one-sided selected-complier group allowed by selection monotonicity, put remaining compliers into the never-selected group, and preserve people whose receipt is always zero or always one.
informal · Theorem T-3 Under IV independence, consistency/exclusion, overlap, treatment monotonicity, cellwise selection monotonicity and M>0 in the finite ordered-outcome model, both endpoint pairings extend to full causal laws reproducing the observed data.
Combine cell constructions using px. Mixing lower and upper compatible laws preserves the data and assumptions and produces every intermediate benefit probability.
Sharpness describes complete causal explanations, as well as feasible pairing arrays.
Lower Threshold Bound
Write ℓ≤t(x)=∑i≤tℓi(x) and h≤t(x)=∑j≤thj(x) for capacities through threshold t.
Survivors starting at or below t can avoid improvement only by using treated outcomes at or below their starting level.
BL(x)=max{0, t∈Ymax[ℓ≤t(x)−h≤t(x)]+min{Δq(x),0}}
The largest shortage of low treated outcomes forces benefit. When untreated selected mass is larger, the correction min{Δq(x),0} subtracts untreated capacity that can belong to nonsurvivors.
To attain minimum benefit, maximize pairs that do not improve, j≤i:
- Serve untreated levels from lowest to highest.
- Use the highest remaining treated level satisfying j≤i, allocating until a bin is exhausted.
- Complete to survivor mass m(x) with residual capacities.
Lower starting levels have fewer nonimproving partners. For any collection of rows, those partners are determined by its highest level; adding all lower rows adds demand without adding partners. Only lower-prefix shortages can force improvement.
After maximizing nonimproving mass, every residual pair needed for completion improves.
In the example, every untreated prefix fits within the treated prefix. Diagonal pairs with masses 0.075,0.10,0.075 fill survivor mass: BL=0, and nobody benefits.
Estimation
Take independent observations from the same observed-data law.
Replace the observable conditional probabilities with empirical frequencies. Set negative estimated capacities to zero, then apply the same totals and threshold formulas.
Screen out cells with very small estimated contributions to survivor mass:
rx=pxm(x),Ix=1{rx>ηn}.
Here px estimates the cell weight, m(x) estimates survivor mass, and Ix indicates retention. Choose ηn>0, with ηn→0 and nηn→∞.
Ψn={(NL,n/Mn, NU,n/Mn),(0,1),Mn>0,Mn=0.
The denominator Mn adds retained, cell-weighted survivor masses. The numerators NL,n and NU,n add retained, cell-weighted benefit masses. Their ratios estimate the interval endpoints.
Guarded Inference
Maintain the structural assumptions, independent sampling, fixed finite supports, common overlap εZ, and a known bound M≥m⋆>0. Use the screening sequence above and 0<α<1.
C1−α,n=[max{0,Ψn,L−gn,α}, min{1,Ψn,U+gn,α}].
The interval C1−α,n expands the estimated endpoints by deterministic padding gn,α, then restricts them to probabilities.
The padding controls sampling error plus mass lost through screening. The positive mass bound m⋆ controls division by survivor mass.
informal · Theorem T-9 Under the structural assumptions, independent sampling, fixed finite supports, common overlap, M≥m⋆>0 and the prescribed screening sequence, the guarded interval covers the entire sharp interval with probability at least 1−α, uniformly.
Thus P{ΘI(Pobs)⊆C1−α,n}≥1−α throughout this class, at every sample size.
Practical width: with three levels, one cell, εZ=1/4, α=0.05, and m⋆=1/4, the current guard gives [0,1] below roughly 1011 observations.
Further Results
informal · Theorem T-7 For nonnegative capacities on finite covariate support with K≥3, both threshold endpoints and sparse attaining pairings can be computed with work bounded by O(∣X∣K).
informal · Theorem T-8 At a fixed law, under the structural and sampling assumptions, fixed finite supports, prescribed screening, and the multinomial and directional delta-method inputs, the estimator is consistent and has a directional limit.
A directional limit describes how the endpoint changes under small perturbations; tied thresholds can make that response nonlinear.
Open Questions
- Useful confidence widths: sharper concentration bounds and simulations are needed to make the uniform guard practical.
- Calibrated inference: can resampling handle simultaneous threshold ties and changing retained cells when selection gaps stay bounded away from zero?
- Extensions: empirical reanalysis, continuous outcomes, sensitivity to selection monotonicity, and restrictions linking the two potential outcomes.
Takeaways
- Joint instrument-arm contrasts identify selected-complier capacities. Selection monotonicity identifies their common survivor mass.
- Pair exactly that mass. Ordered partner sets make threshold shortages the only obstacles, and ordered pairing constructs both endpoints.
- The resulting interval is sharp. Empirical formulas estimate it; uniform whole-interval coverage is available with currently impractical padding.
Appendix: Sharp Exact-mass Interval
The threshold formulas give exactly the feasible benefit probabilities, including equal-total and zero-survivor cells.
Let Pobs be the observed-data law generated by a five-node potential-outcome slate with ordered outcome support of size K≥3. Suppose the latent law satisfies the structural restrictions in Definition P-1 for the instrument-overlap constant εZ and weak-selection direction d(x). Let c be the observable capacity vector from Definition P-3, identified from Pobs as in Theorem T-1, and suppose that M=x∈X∑pxm(x)>0. Then the following statements hold.
- (Sharp cellwise projection.) For every cell x with px>0, the set of survivor-complier couplings generated by feasible full laws with observed law Pobs is exactly Γx∗={γx≥0:j∑γij,x≤ℓi(x), i∑γij,x≤hj(x), i,j∑γij,x=m(x)}.
- (Exact-mass branch.) For every cell x with px>0, Δq(x)>0⇒Γx∗=Γx+,Δq(x)<0⇒Γx∗=Γx−,Δq(x)=0⇒Γx∗=Γx0.
- (Threshold endpoints.) For every cell x with px>0, γx∈Γx∗infbx(γx)=BL(x),γx∈Γx∗supbx(γx)=BU(x). Writing γxL and γxU for the lower and upper cellwise threshold-flow couplings from Definition P-7, γxL∈Γx∗,bx(γxL)=BL(x),γxU∈Γx∗,bx(γxU)=BU(x).
- (Zero-mass cells.) For every cell x with m(x)=0, Γx∗={0}.
Consequently, the sharp identified interval for the benefit probability over feasible full laws is ΘI(Pobs)=[M∑x∈XpxBL(x),M∑x∈XpxBU(x)].
Appendix: Endpoint Law Attainment
Both endpoints are realized by full potential-outcome laws preserving the observed data and structural restrictions.
Let Pobs be the observed-data law induced by a five-node potential-outcome slate system with covariates X, binary instrument Z, treatment D, selection S, and ordered outcome Y∈{0,…,K−1}, where K≥3. Let d:X→{−1,+1} encode the covariate-specific weak selection-monotonicity direction, with d(x)=+1 corresponding to S0≤S1 and d(x)=−1 corresponding to S1≤S0. Let εZ be the instrument-overlap constant. Suppose that:
- (Structural restrictions.) The latent slate system satisfies Assumption A-1, Assumption A-6, and Assumption A-7.
- (Tie-safe survivor model.) The same law satisfies Assumption A-2, Assumption A-3, Assumption A-4, Assumption A-5 at εZ, Assumption A-6, and Assumption A-7 in direction d.
- (Positive aggregate mass.) For the observable capacity vector c formed from Pobs as in Definition P-3, the cell probabilities px, and the survivor-complier masses m(x), M=x∑pxm(x)>0.
Then there exists a baseline law compatible with Pobs and c: it induces Pobs, has observable capacities c, has conditional complier mass in every cell at least max{q0(x),q1(x)}, and carries the same never-taker and always-taker components used by the latent completion. For this baseline, the cellwise threshold-flow construction in Definition P-7 returns two full laws PL and PU over (Y0,Y1,S0,S1,D0,D1,X,Z). These laws are endpoint witnesses for the endpoint map Ψ(Pobs)=(θL,θU): each induces exactly Pobs, satisfies the structural restrictions with instrument-overlap constant εZ and direction d, and attains PL(Y1>Y0∣S1=S0=1,C)=θL,PU(Y1>Y0∣S1=S0=1,C)=θU. Moreover, for every covariate cell x, PL realizes the lower threshold latent completion generated from the baseline at x, and PU realizes the upper threshold latent completion generated from the baseline at x.
Appendix: Branch-free Endpoint Limit
At a fixed law, screening recovers the positive-survivor cells and the estimated endpoints have a directional limit.
Fix a slate potential-outcome system with finite covariate support and ordered outcome support of size K≥3. Let Pobs be its observed-data law, let O1,…,On denote the observed cells, let d(x)∈{−1,+1} be the weak selection-monotonicity direction, let εZ be the instrument-overlap constant, and let ηn be the screening threshold from Definition P-9. Suppose that:
- (Sampling.) For every n, the first n observations satisfy Assumption A-11 with common law Pobs.
- (Overlap.) The instrument propensity satisfies Assumption A-5 with constant εZ.
- (Structural restrictions.) The latent law satisfies Assumption A-1, Assumption A-2, Assumption A-3, Assumption A-4, Assumption A-6, Assumption A-7, Assumption A-9, with weak selection monotonicity in direction d.
- (Screening rate.) For every n, ηn>0, with ηn→0 and nηn→∞.
- (Limit inputs.) The finite multinomial central limit theorem applies to the empirical observed-cell vector, and the Hadamard directional delta method applies to tangentially directionally differentiable maps.
Let c be the observable capacity vector from Definition P-3. Let M=x∑pxm(x), where m(x) is the exact survivor-complier mass in cell x, and let X+(P)={x:pxm(x)>0}. Let Ψ(Pobs)=(θL,θU) be the endpoint map identified by Theorem T-1, and let Ψn be the projected screened plug-in estimator from Definition P-9. Then P({x:rx>ηn}=X+(P))→1 and Ψn→PΨ(Pobs). Moreover, there exist a Gaussian law ZPobs on observed-cell score functions and a map DΨ such that ∫zdZPobs(z)=0 and, for observed cells a,b, ∫z(a)z(b)dZPobs(z)={Pobs(a){1−Pobs(a)},−Pobs(a)Pobs(b),a=b,a=b. The map DΨ is the Hadamard directional derivative, at the observed-law mass vector and on the recovered support X+(P), of the reduced-support endpoint map. With this derivative, n{Ψn−Ψ(Pobs)}⇝DΨ(ZPobs).
Appendix: Uniform Deterministic Guard
The guarded confidence interval contains the entire sharp identified interval uniformly at every sample size.
Fix a family of finite-slate potential-outcome systems indexed by λ∈Λ, with nonempty finite covariate support X, ordered outcome support of size K, and K≥3. For each λ, let Pobs,λ be the observed-data law of (X,Z,D,S,Y), where Y=Y on selected units and is missing off selection. Let Oλ,1,Oλ,2,… be observations taking values in the observed-data cells, let ξλ be the auxiliary process used by the guarded interval in Definition P-10, let dλ(x) be the weak selection-monotonicity direction, and let μλ be the sampling law. Assume:
- (Nominal level.) The nominal error level satisfies 0<α<1.
- (Overlap and mass constants.) The instrument-overlap constant and aggregate survivor-complier mass lower bound satisfy 0<εZ<21,m⋆>0.
- (Screening sequence.) The screening thresholds satisfy ηn>0 for every n, ηn→0,nηn→∞.
- (Uniform law class.) For every law λ and every n≥1, the system belongs to the uniform finite-slate law class of Definition P-2 at index n, with overlap constant εZ, aggregate survivor-complier mass bound m⋆, direction dλ, sampling law μλ, and observations Oλ,1,…,Oλ,n.
Let ΘI(Pobs,λ) be the sharp identified interval for the benefit probability from Definition P-6, and let Ψ(Pobs,λ)=(θL,λ,θU,λ) be its endpoint map. Let Ψn,λ(ω) be the projected screened plug-in endpoint estimator from Definition P-9. Let C1−α,n,λ(ω) and gn,α be the guarded confidence interval and deterministic endpoint-padding guard from Definition P-10. Then, for every n≥1, λ∈Λinfμλ{ω:ΘI(Pobs,λ)⊆C1−α,n,λ(ω)}≥1−α. Moreover, λ∈Λsupμλ{ω:gn,α<max{θL,n,λ(ω)−θL,λ,θU,n,λ(ω)−θU,λ}}⟶0, where Ψn,λ=(θL,n,λ,θU,n,λ), and gn,α=O(n−1/2+ηn). Finally, for every deterministic sequence Ln>0 such that Ln→∞,Ln=o(n), the screening choice ηn=n−1/2Ln satisfies, for every n≥1, ηn>0,ηn→0,nηn→∞, and the corresponding deterministic guard radius obeys gn,α=O(n−1/2Ln).
The padding is computed from:
bn,α=4nα∣E∣.
Here bn,α bounds empirical frequency error across the finite event collection E.
An,α=εZ264K∣X∣bn,α+∣X∣ηn,
The envelope An,α combines sampling error and screening error; ∣X∣ is the number of covariate cells.
gn,α=min{1,m⋆4An,α}.
The padding gn,α converts that envelope to endpoint error using the known mass bound m⋆.
The events are {X=x}, {X=x,Z=z}, and {X=x,Z=z,D=d,S=1,Y=k}, where Y records the outcome when selected.
∣E∣=∣X∣+2∣X∣+4K∣X∣=∣X∣(4K+3).
This counts all cell, assignment-arm, and selected-outcome events used by the guard.