Stat.Sample.Stratified.TreatmentRegression
This module collects finite-sample design, occupancy, residual-noise, and clipped-risk bounds for within-cell treatment regression under arbitrary finite categorical covariate distributions.
Basic 14 core · 9 supporting Counts, empirical treatment fractions, the residualized treatment Gram sum, and the real-outcome regression estimator for an arbitrary finite cell type. ★ observed_design_pushforward
Categorical within-cell regression statistics
Counts, empirical treatment fractions, the residualized treatment Gram sum, and the real-outcome regression estimator for an arbitrary finite cell type. Empty cells contribute zero and the estimator is zero when the Gram sum is zero. These definitions are independent of any paper or causal model.
The cell count of a cell k in a sample of n cell labels x is the number of labels equal to k.
Definition (Lean source)
The treated count of a cell k in a design d of n cell-treatment pairs is the number of observations whose cell is k and whose treatment is true.
The repeat count of a sample of n cell labels x sums, over all cells, the cell count minus one (truncated at zero): the number of observations beyond the first in each occupied cell.
Definition (Lean source)
The empirical treatment fraction of a cell k in a design d of n cell-treatment pairs is the cell's treated count divided by its count, with value zero at an empty cell.
The Gram contribution of a cell with total count m and treated count t is t(m − t)/m, with value zero when the cell is empty; the difference m − t is taken in the natural numbers (truncated at zero).
Definition (Lean source)
The Gram denominator of a design d of n cell-treatment pairs is the sum over all cells of the cell Gram contributions t(m − t)/m, where m is the cell count and t the treated count (an empty cell contributes zero).
Definition (Lean source)
The residualized treatment weight of observation i in a design d of n cell-treatment pairs is its treatment indicator minus the empirical treatment fraction of its own cell.
Definition (Lean source)
The regression numerator of a design d of n cell-treatment pairs with real outcomes y is the sum over observations of the residualized treatment weight times the outcome.
Definition (Lean source)
The guarded inverse Gram of a design d of n cell-treatment pairs is the reciprocal of its Gram denominator when that denominator is positive, and zero otherwise.
The un-clipped within-cell regression coefficient of real outcomes y on the treatment in a design d of n cell-treatment pairs is the regression numerator divided by the Gram denominator when the latter is positive, and zero otherwise.
Interval clipping of a real estimate t at radius R is max(−R, min(R, t)); for nonnegative R this confines t to the symmetric interval from −R to R.
The clipped observed estimator reads the cell label X, the Boolean treatment A and the real outcome Y off each draw of a sample z of n observations, computes the within-cell regression of the outcomes on the treatment, and clips it at radius R.
Definition (Lean source)
The regression residual coefficient runs the guarded within-cell regression on a sample z of n observations, with design given by the cell label X and the Boolean treatment A, and with outcome the real outcome Y minus an arm-and-cell centre evaluated at the observation's treatment and cell; it is zero when the Gram denominator is zero.
Definition (Lean source)
For n independent draws from a probability law carrying measurable cell and Boolean-treatment labels, the law of the observed design — the vector of the n cell-treatment pairs read off the draws — is the n-fold product of the joint law of one draw's cell and treatment.
Formal statement
Proof (Lean source)
9 supporting declarations (lemmas, instances)
-
sum_cellCountlemma — For a sample of n cell labels x, the cell counts summed over all cells equal the sample size n.Proof (Lean source)
lemma sum_cellCount {n : ℕ} (x : Fin n → κ) : ∑ k, cellCount x k = n := by classical unfold cellCount rw [Finset.sum_comm] simp -
treatedCount_lelemma — In a design d of n cell-treatment pairs, the treated count of a cell k never exceeds that cell's total count.hypothesesconclusiontreatedCount d k ≤ cellCount (fun i => (d i).1) kProof (Lean source)
lemma treatedCount_le {n : ℕ} (d : Fin n → κ × Bool) (k : κ) : treatedCount d k ≤ cellCount (fun i => (d i).1) k := by classical unfold treatedCount cellCount apply Finset.sum_le_sum intro i hi by_cases hx : (d i).1 = k <;> by_cases ha : (d i).2 = true <;> simp [hx, ha] -
cellGram_nonneglemma — For any total count m and treated count t, the cell Gram contribution is nonnegative, including the empty-cell value zero.Proof (Lean source)
-
gram_nonneglemma — For every design d of n cell-treatment pairs, the Gram denominator is nonnegative.Proof (Lean source)
lemma gram_nonneg {n : ℕ} (d : Fin n → κ × Bool) : 0 ≤ gram d := Finset.sum_nonneg fun _ _ => cellGram_nonneg _ _ -
regression_zerolemma — If the Gram denominator of the design d is zero, then both the guarded regression of any outcomes y on d and the guarded inverse Gram of d are zero.hypothesesProof (Lean source)
lemma regression_zero {n : ℕ} (d : Fin n → κ × Bool) (y : Fin n → ℝ) (h : gram d = 0) : regression d y = 0 ∧ inverseGram d = 0 := by simp [regression, inverseGram, h] -
measurable_designStatisticlemma — With finitely many cells whose singletons are measurable, every real-valued statistic f of a design of n cell-treatment pairs is measurable.Proof (Lean source)
lemma measurable_designStatistic [Finite κ] {n : ℕ} (f : (Fin n → κ × Bool) → ℝ) : Measurable f := measurable_of_countable f -
measurable_numeratorlemma — With finitely many cells, for every sample size n the regression numerator is jointly measurable in the design and the real outcomes.hypothesesκ :sharedType u_1Finite κn :ℕconclusionProof (Lean source)
@[fun_prop] lemma measurable_numerator [Finite κ] {n : ℕ} : Measurable (fun v : (Fin n → κ × Bool) × (Fin n → ℝ) => numerator v.1 v.2) := by unfold numerator apply Finset.measurable_sum intro i hi exact ((measurable_designStatistic (fun d => residualWeight d i)).comp measurable_fst).mul ((measurable_pi_apply i).comp measurable_snd) -
measurable_regressionlemma — For every sample size n, the un-clipped guarded regression coefficient is jointly measurable in the design and the real outcomes.hypothesesκ :sharedType u_1n :ℕconclusionProof (Lean source)
@[fun_prop] lemma measurable_regression {n : ℕ} : Measurable (fun v : (Fin n → κ × Bool) × (Fin n → ℝ) => regression v.1 v.2) := by unfold regression have hg : Measurable (fun v : (Fin n → κ × Bool) × (Fin n → ℝ) => gram v.1) := (measurable_designStatistic gram).comp measurable_fst exact Measurable.ite (measurableSet_lt measurable_const hg) (measurable_numerator.div hg) measurable_const -
measurable_clippedEstimatorlemma — If the cell label X is measurable, the treatment A is measurable and the outcome Y is measurable, then for every sample size n and clipping radius R, the clipped regression estimator is a measurable function of the sample.hypothesesconclusionProof (Lean source)
@[fun_prop] lemma measurable_clippedEstimator {n : ℕ} (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (R : ℝ) (hX : Measurable X) (hA : Measurable A) (hY : Measurable Y) : Measurable (fun z : Fin n → Ω => clippedEstimator X A Y R z) := by change Measurable (fun z : Fin n → Ω => max (-R) (min R (regression (sampleDesign X A z) (fun i => Y (z i))))) have hd : Measurable (sampleDesign (n := n) X A) := Causalean.Stat.measurable_sampleDesign X A hX hA have hy : Measurable (fun z : Fin n → Ω => fun i => Y (z i)) := measurable_pi_lambda _ fun i => hY.comp (measurable_pi_apply i) have hg : Measurable (fun z : Fin n → Ω => gram (sampleDesign X A z)) := (measurable_designStatistic (gram (κ := κ) (n := n))).comp hd have hH : Measurable (fun z : Fin n → Ω => numerator (sampleDesign X A z) (fun i => Y (z i))) := by unfold numerator apply Finset.measurable_sum intro i hi exact ((measurable_designStatistic (fun d : Fin n → κ × Bool => residualWeight d i)).comp hd).mul (hY.comp (measurable_pi_apply i)) have hr : Measurable (fun z : Fin n → Ω => regression (sampleDesign X A z) (fun i => Y (z i))) := by unfold regression exact Measurable.ite (measurableSet_lt measurable_const hg) (hH.div hg) measurable_const exact (measurable_const (a := -R)).max ((measurable_const (a := R)).min hr)
TreatmentMoments 2 core · 4 supporting Explicit enumeration of independent Boolean assignments at fixed categorical labels. ★ assignment_gram_mean_lower
Finite conditional treatment moments
Explicit enumeration of independent Boolean assignments at fixed categorical labels. The expected cell Gram is (N-1)_+ times pi(1-pi), including empty and singleton cells. Bernoulli design moments in Causalean provide the independent one- and two-coordinate expectations; these statements expose the required count and denominator algebra.
The conditional weight of a treatment assignment a of n Boolean treatments at fixed cell labels x, under cell propensities p, is the product over observations of p at the observation's cell when it is treated and one minus that value when it is not.
Given fixed labels x and cell propensities p, if ε is an overlap margin and every occupied cell has propensity between ε and 1 − ε, then the assignment-weighted sum of the Gram denominator is at least ε(1 − ε) times the repeat count of the labels.
Formal statement
Proof (Lean source)
4 supporting declarations (lemmas, instances)
-
sum_assignmentWeightlemma — For arbitrary real cell weights p and fixed labels x of n observations, the conditional weights of all Boolean treatment assignments sum to one.hypothesesconclusionProof (Lean source)
lemma sum_assignmentWeight {n : ℕ} (p : κ → ℝ) (x : Fin n → κ) : (∑ a : Fin n → Bool, assignmentWeight p x a) = 1 := by have h := assignment_prod_mean p x (fun _ _ => 1) simpa using h -
assignment_treatedCount_meanlemma — For arbitrary real cell weights p and fixed labels x of n observations, and a cell k, the assignment-weighted sum of the treated count of k equals the cell count of k times p at k.hypothesesconclusion= (cellCount x k : ℝ) * p kProof (Lean source)
lemma assignment_treatedCount_mean {n : ℕ} (p : κ → ℝ) (x : Fin n → κ) (k : κ) : (∑ a : Fin n → Bool, assignmentWeight p x a * (treatedCount (fun i => (x i, a i)) k : ℝ)) = (cellCount x k : ℝ) * p k := by classical have hc (a : Fin n → Bool) : (treatedCount (fun i => (x i, a i)) k : ℝ) = ∑ i, if x i = k then (if a i then (1 : ℝ) else 0) else 0 := by simp only [treatedCount, Nat.cast_sum] apply Finset.sum_congr rfl intro i _ by_cases hx : x i = k <;> by_cases ha : a i = true <;> simp [hx, ha] simp_rw [hc, Finset.mul_sum] rw [Finset.sum_comm] have hm (i : Fin n) : (∑ a : Fin n → Bool, assignmentWeight p x a * (if x i = k then (if a i then (1 : ℝ) else 0) else 0)) = if x i = k then p k else 0 := by by_cases hx : x i = k · simpa [hx] using assignment_coord_mean p x i (fun b => if b then (1 : ℝ) else 0) · simp [hx] simp_rw [hm] simp only [cellCount, Nat.cast_sum, Nat.cast_ite, Nat.cast_one, Nat.cast_zero, Finset.sum_mul, ite_mul, one_mul, zero_mul] -
assignment_treated_control_meanlemma — For arbitrary real cell weights p and fixed labels x of n observations, and a cell k, the assignment-weighted sum of the treated count of k times its control count equals N(N − 1)·p(1 − p), where N is the cell count of k, p is the weight at k, and N − 1 is truncated at zero; it is zero in empty and singleton cells.hypothesesconclusion(∑ a : Fin n → Bool, assignmentWeight p x a * ((treatedCount (fun i => (x i, a i)) k : ℝ) * (cellCount x k - treatedCount (fun i => (x i, a i)) k : ℕ)))= (cellCount x k : ℝ) * (cellCount x k - 1 : ℕ) * p k * (1 - p k)Proof (Lean source)
lemma assignment_treated_control_mean {n : ℕ} (p : κ → ℝ) (x : Fin n → κ) (k : κ) : (∑ a : Fin n → Bool, assignmentWeight p x a * ((treatedCount (fun i => (x i, a i)) k : ℝ) * (cellCount x k - treatedCount (fun i => (x i, a i)) k : ℕ))) = (cellCount x k : ℝ) * (cellCount x k - 1 : ℕ) * p k * (1 - p k) := by classical let S : Finset (Fin n) := Finset.univ.filter (fun i => x i = k) have hs : S.card = cellCount x k := by simp [S, cellCount] have hx {i : Fin n} (hi : i ∈ S) : x i = k := (Finset.mem_filter.mp hi).2 have ht (a : Fin n → Bool) : (treatedCount (fun i => (x i, a i)) k : ℝ) = ∑ i ∈ S, if a i then (1 : ℝ) else 0 := by simp only [treatedCount, Nat.cast_sum, Finset.sum_filter, S] apply Finset.sum_congr rfl intro i _ by_cases hi : x i = k <;> by_cases ha : a i = true <;> simp [hi, ha] have hc (a : Fin n → Bool) : (cellCount x k - treatedCount (fun i => (x i, a i)) k : ℕ) = (∑ i ∈ S, if a i then (0 : ℝ) else 1) := by rw [Nat.cast_sub (treatedCount_le (fun i => (x i, a i)) k), ht, ← hs] have hcard : (S.card : ℝ) = ∑ i ∈ S, (1 : ℝ) := by simp rw [hcard, ← Finset.sum_sub_distrib] apply Finset.sum_congr rfl intro i _ by_cases ha : a i = true <;> simp [ha] simp_rw [ht, hc, Finset.sum_mul, Finset.mul_sum] rw [Finset.sum_comm] have hm (i : Fin n) (hi : i ∈ S) (j : Fin n) (hj : j ∈ S) : (∑ a : Fin n → Bool, assignmentWeight p x a * ((if a i then (1 : ℝ) else 0) * (if a j then (0 : ℝ) else 1))) = if j = i then 0 else p k * (1 - p k) := by by_cases hij : j = i · subst j have hz (a : Fin n → Bool) : (if a i then (1 : ℝ) else 0) * (if a i then (0 : ℝ) else 1) = 0 := by by_cases ha : a i = true <;> simp [ha] simp [hz] · simpa [hx hi, hx hj, hij] using assignment_pair_mean p x i j (Ne.symm hij) (fun b => if b then (1 : ℝ) else 0) (fun b => if b then (0 : ℝ) else 1) have hiMean (i : Fin n) (hi : i ∈ S) : (∑ j ∈ S, ∑ a : Fin n → Bool, assignmentWeight p x a * ((if a i then (1 : ℝ) else 0) * (if a j then (0 : ℝ) else 1))) = (S.card - 1 : ℕ) * (p k * (1 - p k)) := by calc _ = ∑ j ∈ S, if j = i then (0 : ℝ) else p k * (1 - p k) := Finset.sum_congr rfl (hm i hi) _ = _ := by rw [← Finset.sum_erase_add _ _ hi] simp only [ite_true, add_zero] calc _ = ∑ _j ∈ S.erase i, p k * (1 - p k) := Finset.sum_congr rfl (fun j hj => if_neg (Finset.ne_of_mem_erase hj)) _ = _ := by simp [Finset.card_erase_of_mem hi] calc _ = ∑ i ∈ S, ∑ j ∈ S, ∑ a : Fin n → Bool, assignmentWeight p x a * ((if a i then (1 : ℝ) else 0) * (if a j then (0 : ℝ) else 1)) := by apply Finset.sum_congr rfl intro i hi rw [Finset.sum_comm] _ = ∑ i ∈ S, (S.card - 1 : ℕ) * (p k * (1 - p k)) := Finset.sum_congr rfl hiMean _ = _ := by simp [hs]; ring -
assignment_gram_meantheorem — For arbitrary real cell weights p and fixed labels x of n observations, the assignment-weighted sum of the Gram denominator equals the sum over cells of (N − 1)·p(1 − p), where N is the cell count, N − 1 is truncated at zero, and p is the weight at that cell — the repeated counts times the Bernoulli variances.hypothesesconclusion= ∑ k, (cellCount x k - 1 : ℕ) * p k * (1 - p k)Proof (Lean source)
theorem assignment_gram_mean {n : ℕ} (p : κ → ℝ) (x : Fin n → κ) : (∑ a : Fin n → Bool, assignmentWeight p x a * gram (fun i => (x i, a i))) = ∑ k, (cellCount x k - 1 : ℕ) * p k * (1 - p k) := by classical simp only [gram, Finset.mul_sum] rw [Finset.sum_comm] apply Finset.sum_congr rfl intro k _ by_cases hm : cellCount x k = 0 · simp [cellGram, hm] · have hmR : (cellCount x k : ℝ) ≠ 0 := Nat.cast_ne_zero.mpr hm simp only [cellGram, hm, if_false] simp_rw [← mul_div_assoc] rw [← Finset.sum_div, assignment_treated_control_mean] field_simp
CategoricalLaw 5 core · 6 supporting An arbitrary joint law of a finite cell and a Boolean treatment is disintegrated by explicit finite sums. ★ conditional_gram_mean_lower
Categorical conditioning, including null cells
An arbitrary joint law of a finite cell and a Boolean treatment is disintegrated by explicit finite sums. Treatment propensity is guarded at zero cell mass; zero-mass label vectors have zero product weight. The fixed-label Gram identity is consequently a genuine conditional expectation statement on every occupied label fiber, without requiring overlap in null cells.
The cell probability of a cell k under a joint law ν of a cell and a Boolean treatment is the mass ν assigns to that cell, summed over both treatment values, as a real number.
The treatment propensity of a cell k under a joint law ν of a cell and a Boolean treatment is the mass of the treated pair (k, true) divided by the cell probability of k, with value zero in a null cell.
Definition (Lean source)
The product weight of a fixed vector x of n cell labels under a joint law ν of a cell and a Boolean treatment is the product of the cell probabilities of its labels.
Definition (Lean source)
The label fiber of a vector x of n cell labels is the set of designs of n cell-treatment pairs whose cell coordinates equal x.
Under the n-fold product of a joint probability law ν of a cell and a Boolean treatment, if a fixed label vector x has positive product weight, ε is an overlap margin and every cell of positive probability has propensity between ε and 1 − ε, then the conditional mean of the Gram denominator given the labels x is at least ε(1 − ε) times the repeat count of x.
Formal statement
Proof (Lean source)
6 supporting declarations (lemmas, instances)
-
singleton_probability_factorlemma — Under a joint probability law ν of a cell and a Boolean treatment, the mass of each pair of a cell k and a treatment value a equals the cell probability of k times the guarded propensity of k when a is true, or times one minus it when a is false; both sides vanish in null cells.hypothesesconclusion(ν {(k, a)}).toReal= cellProbability ν k * (if a then propensity ν k else 1 - propensity ν k)Proof (Lean source)
lemma singleton_probability_factor (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (k : κ) (a : Bool) : (ν {(k, a)}).toReal = cellProbability ν k * (if a then propensity ν k else 1 - propensity ν k) := by have hsubset (b : Bool) : ({(k, b)} : Set (κ × Bool)) ⊆ {v | v.1 = k} := by rintro v rfl rfl have hle (b : Bool) : (ν {(k, b)}).toReal ≤ cellProbability ν k := ENNReal.toReal_mono (measure_ne_top ν _) (measure_mono (hsubset b)) have hsum : cellProbability ν k = (ν {(k, false)}).toReal + (ν {(k, true)}).toReal := by have hs : {v : κ × Bool | v.1 = k} = {(k, false)} ∪ {(k, true)} := by ext v rcases v with ⟨j, b⟩ cases b <;> simp rw [cellProbability, hs, measure_union (by simp) (measurableSet_singleton _), ENNReal.toReal_add (measure_ne_top ν _) (measure_ne_top ν _)] by_cases hk : cellProbability ν k = 0 · have ha : (ν {(k, a)}).toReal = 0 := le_antisymm (by simpa [hk] using hle a) ENNReal.toReal_nonneg simp [ha, hk] · cases a <;> simp only [Bool.false_eq_true, if_false, if_true, propensity, hk] · field_simp linarith [hsum] · field_simp -
propensity_mem_Icclemma — Under a joint probability law ν of a cell and a Boolean treatment, the guarded propensity of every cell k lies between zero and one, including at null cells.hypothesesconclusionpropensity ν k ∈ Icc (0 : ℝ) 1Proof (Lean source)
lemma propensity_mem_Icc (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (k : κ) : propensity ν k ∈ Icc (0 : ℝ) 1 := by have hle : (ν {(k, true)}).toReal ≤ cellProbability ν k := by apply ENNReal.toReal_mono (measure_ne_top ν _) apply measure_mono rintro v rfl rfl by_cases hk : cellProbability ν k = 0 · simp [propensity, hk] · have hp : 0 < cellProbability ν k := lt_of_le_of_ne ENNReal.toReal_nonneg (Ne.symm hk) simp only [propensity, hk, if_false, mem_Icc] exact ⟨div_nonneg ENNReal.toReal_nonneg hp.le, (div_le_one hp).mpr hle⟩ -
integral_labelFibertheorem — Under the n-fold product of a joint probability law ν of a cell and a Boolean treatment, the integral of any real statistic F of the design over the label fiber of a fixed label vector x equals the product weight of x times the sum, over all Boolean treatment assignments, of the assignment's conditional weight under the propensities of ν times F evaluated at the design pairing x with that assignment.hypothesesconclusion(∫ d in labelFiber x, F d ∂Measure.pi (fun _ : Fin n => ν))= labelWeight ν x * ∑ a : Fin n → Bool, assignmentWeight (propensity ν) x a * F (fun i => (x i, a i))Proof (Lean source)
theorem integral_labelFiber (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (x : Fin n → κ) (F : (Fin n → κ × Bool) → ℝ) : (∫ d in labelFiber x, F d ∂Measure.pi (fun _ : Fin n => ν)) = labelWeight ν x * ∑ a : Fin n → Bool, assignmentWeight (propensity ν) x a * F (fun i => (x i, a i)) := by classical let g : (Fin n → Bool) → (Fin n → κ × Bool) := fun a i => (x i, a i) have hg : Injective g := by intro a b h funext i exact congrArg snd (congrFun h i) let s := Finset.univ.image g have hs : labelFiber x = (s : Set (Fin n → κ × Bool)) := by ext d constructor · intro hd apply Finset.mem_image.mpr refine ⟨fun i => (d i).2, Finset.mem_univ _, ?_⟩ funext i exact Prod.ext (hd i).symm rfl · intro hd obtain ⟨a, _, rfl⟩ := Finset.mem_image.mp hd intro i rfl rw [hs, setIntegral_finset s IntegrableOn.finset] rw [Finset.sum_image (fun a _ b _ h => hg h)] simp only [measureReal_def, Measure.pi_singleton, ENNReal.toReal_prod, smul_eq_mul, g, singleton_probability_factor] simp only [labelWeight, assignmentWeight, Finset.prod_mul_distrib, Finset.mul_sum, mul_assoc] -
probability_labelFiberlemma — Under the n-fold product of a joint probability law ν of a cell and a Boolean treatment, the mass of the label fiber of a fixed label vector x equals the product weight of x, the product of its cell probabilities.hypothesesconclusionProof (Lean source)
lemma probability_labelFiber (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (x : Fin n → κ) : ((Measure.pi (fun _ : Fin n => ν)) (labelFiber x)).toReal = labelWeight ν x := by simpa [sum_assignmentWeight, measureReal_def] using integral_labelFiber n ν x (fun _ => (1 : ℝ)) -
integral_gram_labelFibertheorem — Under the n-fold product of a joint probability law ν of a cell and a Boolean treatment, the integral of the Gram denominator over the label fiber of a fixed label vector x equals the product weight of x times the sum over cells of (N − 1)₊·π(1 − π), where N is the cell count in x and π the propensity of the cell; this holds also for null label fibers.hypothesesconclusion(∫ d in labelFiber x, gram d ∂Measure.pi (fun _ : Fin n => ν))= labelWeight ν x * ∑ k, (cellCount x k - 1 : ℕ) * propensity ν k * (1 - propensity ν k)Proof (Lean source)
theorem integral_gram_labelFiber (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (x : Fin n → κ) : (∫ d in labelFiber x, gram d ∂Measure.pi (fun _ : Fin n => ν)) = labelWeight ν x * ∑ k, (cellCount x k - 1 : ℕ) * propensity ν k * (1 - propensity ν k) := by rw [integral_labelFiber, assignment_gram_mean] -
conditional_gram_meantheorem — Under the n-fold product of a joint probability law ν of a cell and a Boolean treatment, if a fixed label vector x has positive product weight, then the conditional mean of the Gram denominator given the labels x — its integral over the label fiber divided by the fiber's weight — equals the sum over cells of (N − 1)₊·π(1 − π), where N is the cell count in x and π the propensity of the cell.hypothesesconclusion= ∑ k, (cellCount x k - 1 : ℕ) * propensity ν k * (1 - propensity ν k)Proof (Lean source)
theorem conditional_gram_mean (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (x : Fin n → κ) (hx : 0 < labelWeight ν x) : (∫ d in labelFiber x, gram d ∂Measure.pi (fun _ : Fin n => ν)) / labelWeight ν x = ∑ k, (cellCount x k - 1 : ℕ) * propensity ν k * (1 - propensity ν k) := by rw [integral_gram_labelFiber] field_simp [ne_of_gt hx]
Concentration 3 core · 3 supporting In a regression of an outcome on a binary treatment with cell fixed effects, the denominator of the treatment coefficient is the within-cell treatment variation ∑ₖ tₖ (mₖ − tₖ) / mₖ, where cell k holds mₖ observations of ★ observed_gram_lower_tail
Lower-tail bound for within-cell treatment variation
In a regression of an outcome on a binary treatment with cell fixed effects, the denominator of
the treatment coefficient is the within-cell treatment variation ∑ₖ tₖ (mₖ − tₖ) / mₖ, where
cell k holds mₖ observations of which tₖ are treated. For n i.i.d. draws of (cell,
treatment) with at most n cells and propensity in [ε, 1 − ε] in every cell of positive
probability (0 < ε ≤ 1/2), this denominator has mean at least ε(1 − ε) n / 4 when n ≥ 2, and
falls below ε(1 − ε) n / 8 with probability at most exp(−2 (ε(1 − ε)/8)² n), hence at most
C(ε)/n with C(ε) = 1 / (2 (ε(1 − ε)/8)²) for every n ≥ 1. The bounds are uniform over all
such laws; the tail bound is McDiarmid's inequality, since replacing one observation changes the
denominator by at most one.
Main definitions and results
* gramThreshold, gramTailConstant — the constants ε(1 − ε)/8 and C(ε).
* integral_gram_lower — the lower bound on the expected denominator (n ≥ 2).
* gram_lower_tail_exp — the exponential lower-tail bound (n ≥ 2).
* gram_lower_tail — the C(ε)/n bound for every n ≥ 1; at n = 1 it holds because
C(ε) ≥ 1.
* observed_gram_lower_tail — the same bound for the design observed from any i.i.d. sample with
a measurable cell label and Boolean treatment; no overlap condition is imposed on null cells.
McDiarmid's inequality is taken from Causalean.Stat.Concentration.TailBounds.McDiarmid
(formalized in FoML, github.com/auto-res/lean-rademacher).
The good-design Gram threshold per observation at overlap margin ε is ε(1 − ε)/8.
Definition (Lean source)
The uniform Gram lower-tail constant at overlap margin ε is the reciprocal of twice the squared good-design threshold, 1/(2·(ε(1 − ε)/8)²).
Definition (Lean source)
Draw n iid observations from a probability law μ carrying a measurable cell label X and a measurable Boolean treatment A. If there is at least one observation, the number of cells is at most n, the overlap margin ε is positive and at most one half, and every cell of positive probability under the joint law of (X, A) has propensity between ε and 1 − ε, then the probability that the Gram denominator of the observed design falls below the good-design threshold times n is at most the uniform Gram lower-tail constant divided by n.
Formal statement
Proof (Lean source)
3 supporting declarations (lemmas, instances)
-
integral_gram_lowertheorem — Let the design be n iid draws from a joint probability law ν of a cell and a Boolean treatment. If there are at least two observations, the number of cells is at most n, the overlap margin ε is positive and at most one half, and every cell of positive probability has propensity between ε and 1 − ε, then the expected Gram denominator is at least twice the good-design threshold times n, that is ε(1 − ε)n/4.hypothesesκ :sharedType u_1n :ℕν :hn :2 ≤ nhcard :card κ ≤ nepsilon :ℝhepsilon :0 < epsilonhhalf :epsilon ≤ 1 / 2hoverlap :∀ kif0 < cellProbability ν kthenepsilon ≤ propensity ν k ∧ propensity ν k ≤ 1 - epsilonconclusion2 * gramThreshold epsilon * nProof (Lean source)
theorem integral_gram_lower (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (hn : 2 ≤ n) (hcard : card κ ≤ n) (epsilon : ℝ) (hepsilon : 0 < epsilon) (hhalf : epsilon ≤ 1 / 2) (hoverlap : ∀ k, 0 < cellProbability ν k → epsilon ≤ propensity ν k ∧ propensity ν k ≤ 1 - epsilon) : 2 * gramThreshold epsilon * n ≤ ∫ d : Fin n → κ × Bool, gram d ∂Measure.pi (fun _ : Fin n => ν) := by classical let P := Measure.pi (fun _ : Fin n => ν) let Q := ν.map Prod.fst have hQ : IsProbabilityMeasure Q := Measure.isProbabilityMeasure_map measurable_fst.aemeasurable have hfiber (F : (Fin n → κ × Bool) → ℝ) : (∫ d, F d ∂P) = ∑ x : Fin n → κ, ∫ d in labelFiber x, F d ∂P := by simp_rw [← integral_indicator (Set.to_countable _ |>.measurableSet)] rw [← integral_finsetSum _ (fun _ _ => Integrable.of_finite)] apply integral_congr_ae filter_upwards with d have hx (x : Fin n → κ) : d ∈ labelFiber x ↔ x = fun i => (d i).1 := by simp only [labelFiber, mem_ofPred_eq] exact ⟨fun h => funext (fun i => (h i).symm), fun h i => congrFun h.symm i⟩ simp [indicator, hx] have hmass (k : κ) : (Q {k}).toReal = cellProbability ν k := by dsimp only [Q] rw [Measure.map_apply measurable_fst (measurableSet_singleton k)] rfl have hrepeat : (∫ x : Fin n → κ, (repeatCount x : ℝ) ∂Measure.pi (fun _ : Fin n => Q)) = ∑ x : Fin n → κ, labelWeight ν x * (repeatCount x : ℝ) := by rw [integral_fintype Integrable.of_finite] simp [Measure.real, Measure.pi_singleton, ENNReal.toReal_prod, hmass, labelWeight] have hlower (x : Fin n → κ) : epsilon * (1 - epsilon) * (labelWeight ν x * (repeatCount x : ℝ)) ≤ ∫ d in labelFiber x, gram d ∂P := by have hw : 0 ≤ labelWeight ν x := Finset.prod_nonneg (fun _ _ => ENNReal.toReal_nonneg) by_cases hx : 0 < labelWeight ν x · have h := conditional_gram_mean_lower n ν x hx epsilon hoverlap have h' := (le_div_iff₀ hx).mp h simpa [P, mul_assoc, mul_left_comm, mul_comm] using h' · have hz : labelWeight ν x = 0 := le_antisymm (le_of_not_gt hx) hw simp [P, integral_gram_labelFiber, hz] have hoc := integral_repeatCount_lower n Q hn hcard have hp : 0 ≤ epsilon * (1 - epsilon) := mul_nonneg hepsilon.le (by linarith) calc 2 * gramThreshold epsilon * n = epsilon * (1 - epsilon) * ((n : ℝ) / 4) := by unfold gramThreshold ring _ ≤ epsilon * (1 - epsilon) * (∫ x : Fin n → κ, (repeatCount x : ℝ) ∂Measure.pi (fun _ : Fin n => Q)) := mul_le_mul_of_nonneg_left hoc hp _ = ∑ x : Fin n → κ, epsilon * (1 - epsilon) * (labelWeight ν x * (repeatCount x : ℝ)) := by rw [hrepeat, Finset.mul_sum] _ ≤ ∑ x : Fin n → κ, ∫ d in labelFiber x, gram d ∂P := Finset.sum_le_sum (fun x _ => hlower x) _ = ∫ d, gram d ∂P := (hfiber gram).symm -
gram_lower_tail_exptheorem — Let the design be n iid draws from a joint probability law ν of a cell and a Boolean treatment. If there are at least two observations, the number of cells is at most n, the overlap margin ε is positive and at most one half, and every cell of positive probability has propensity between ε and 1 − ε, then the probability that the Gram denominator falls below the good-design threshold times n is at most exp(−2·(ε(1 − ε)/8)²·n), uniformly over all such laws.hypothesesκ :sharedType u_1n :ℕν :hn :2 ≤ nhcard :card κ ≤ nepsilon :ℝhepsilon :0 < epsilonhhalf :epsilon ≤ 1 / 2hoverlap :∀ kif0 < cellProbability ν kthenepsilon ≤ propensity ν k ∧ propensity ν k ≤ 1 - epsilonconclusion≤ exp (-2 * gramThreshold epsilon ^ 2 * n)Proof (Lean source)
theorem gram_lower_tail_exp (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (hn : 2 ≤ n) (hcard : card κ ≤ n) (epsilon : ℝ) (hepsilon : 0 < epsilon) (hhalf : epsilon ≤ 1 / 2) (hoverlap : ∀ k, 0 < cellProbability ν k → epsilon ≤ propensity ν k ∧ propensity ν k ≤ 1 - epsilon) : ((Measure.pi (fun _ : Fin n => ν)) {d | gram d < gramThreshold epsilon * n}).toReal ≤ exp (-2 * gramThreshold epsilon ^ 2 * n) := by classical have : Nonempty (κ × Bool) := nonempty_of_isProbabilityMeasure ν have hnpos : (0 : ℝ) < n := by exact_mod_cast (by omega : 0 < n) have hgpos : 0 < gramThreshold epsilon := by unfold gramThreshold exact div_pos (mul_pos hepsilon (by linarith)) (by norm_num) let P := Measure.pi (fun _ : Fin n => ν) have hmean := integral_gram_lower n ν hn hcard epsilon hepsilon hhalf hoverlap have ht : (1 / (n : ℝ)) * ∑ _ : Fin n, (1 : ℝ) ^ 2 ≤ 1 := by simp [ne_of_gt hnpos] have hmc := mcdiarmid_inequality_neg (μ := P) (fun (i : Fin n) (d : Fin n → κ × Bool) => d i) (fun i => measurable_pi_apply i) (pi_comp_eval_iIndepFun (μ := ν) (ι := Fin n) measurable_id) gram (fun _ => (1 : ℝ)) (fun i d v => gram_update_oscillation d i v) (measurable_designStatistic gram) (gramThreshold epsilon * n) (mul_nonneg hgpos.le hnpos.le) (1 / n) ht have hmc' : (P {d | gram d - (∫ d, gram d ∂P) ≤ -(gramThreshold epsilon * n)}).toReal ≤ exp (-2 * (gramThreshold epsilon * n) ^ 2 * (1 / n)) := by exact hmc have hsub : {d : Fin n → κ × Bool | gram d < gramThreshold epsilon * n} ⊆ {d | gram d - (∫ d, gram d ∂P) ≤ -(gramThreshold epsilon * n)} := by intro d hd dsimp only [mem_ofPred_eq] at * change 2 * gramThreshold epsilon * n ≤ ∫ d, gram d ∂P at hmean linarith calc _ ≤ (P {d | gram d - (∫ d, gram d ∂P) ≤ -(gramThreshold epsilon * n)}).toReal := ENNReal.toReal_mono (measure_ne_top P _) (measure_mono hsub) _ ≤ exp (-2 * (gramThreshold epsilon * n) ^ 2 * (1 / n)) := hmc' _ = _ := by congr 1 field_simp -
gram_lower_tailtheorem — Let the design be n iid draws from a joint probability law ν of a cell and a Boolean treatment. If there is at least one observation, the number of cells is at most n, the overlap margin ε is positive and at most one half, and every cell of positive probability has propensity between ε and 1 − ε, then the probability that the Gram denominator falls below the good-design threshold times n is at most the uniform Gram lower-tail constant divided by n, including the one-observation case.hypothesesκ :sharedType u_1n :ℕν :hn :0 < nhcard :card κ ≤ nepsilon :ℝhepsilon :0 < epsilonhhalf :epsilon ≤ 1 / 2hoverlap :∀ kif0 < cellProbability ν kthenepsilon ≤ propensity ν k ∧ propensity ν k ≤ 1 - epsilonconclusion≤ gramTailConstant epsilon / nProof (Lean source)
theorem gram_lower_tail (n : ℕ) (ν : Measure (κ × Bool)) [IsProbabilityMeasure ν] (hn : 0 < n) (hcard : card κ ≤ n) (epsilon : ℝ) (hepsilon : 0 < epsilon) (hhalf : epsilon ≤ 1 / 2) (hoverlap : ∀ k, 0 < cellProbability ν k → epsilon ≤ propensity ν k ∧ propensity ν k ≤ 1 - epsilon) : ((Measure.pi (fun _ : Fin n => ν)) {d | gram d < gramThreshold epsilon * n}).toReal ≤ gramTailConstant epsilon / n := by have hnpos : (0 : ℝ) < n := by exact_mod_cast hn have hgpos : 0 < gramThreshold epsilon := by unfold gramThreshold exact div_pos (mul_pos hepsilon (by linarith)) (by norm_num) by_cases hn2 : 2 ≤ n · have hu : 0 < 2 * gramThreshold epsilon ^ 2 * (n : ℝ) := by positivity have hexp : exp (-(2 * gramThreshold epsilon ^ 2 * (n : ℝ))) ≤ 1 / (2 * gramThreshold epsilon ^ 2 * (n : ℝ)) := by rw [Real.exp_neg, ← one_div] apply one_div_le_one_div_of_le hu linarith [Real.add_one_le_exp (2 * gramThreshold epsilon ^ 2 * (n : ℝ))] calc _ ≤ exp (-2 * gramThreshold epsilon ^ 2 * n) := gram_lower_tail_exp n ν hn2 hcard epsilon hepsilon hhalf hoverlap _ ≤ 1 / (2 * gramThreshold epsilon ^ 2 * (n : ℝ)) := by simpa only [neg_mul] using hexp _ = gramTailConstant epsilon / n := by unfold gramTailConstant rw [div_div] · have hn1 : n = 1 := by omega subst n have hgsmall : gramThreshold epsilon ≤ 1 / 8 := by unfold gramThreshold nlinarith [sq_nonneg epsilon] have hdpos : 0 < 2 * gramThreshold epsilon ^ 2 := by positivity have hdsmall : 2 * gramThreshold epsilon ^ 2 ≤ 1 := by nlinarith calc _ ≤ 1 := by exact ENNReal.toReal_mono ENNReal.one_ne_top (prob_le_one) _ ≤ gramTailConstant epsilon / (1 : ℕ) := by simp only [Nat.cast_one, div_one, gramTailConstant] exact (le_div_iff₀ hdpos).mpr (by simpa using hdsmall)
Residual 2 core · 4 supporting Arm-cell residuals need only be centered and square integrable with their conditional second moments bounded by M². ★ conditional_regression_error_sq_le
Conditional regression residual second moments
Arm-cell residuals need only be centered and square integrable with their conditional second moments bounded by M². Arbitrary nonnegative functions of the finite observed design can weight the residual regression square. Cross-coordinate terms vanish, and the sum of squared residualized treatment weights equals the Gram, leaving the guarded reciprocal Gram envelope. Conditional moments are stated as normalized positive-mass design-fiber integrals, rather than choosing a conditional-expectation version on null fibers.
The residual assumptions on an observation law μ with cell label X, Boolean treatment A, real outcome Y, arm-and-cell centres and second-moment envelope M require that the label, the treatment and the outcome are measurable, that on each arm-cell event the residual Y minus its centre is square-integrable, integrates to zero, and has integral of its square at most the event's mass times M². Null arm-cells are allowed; outcomes need not be bounded.
Definition (Lean source)
If the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, on every cell of positive probability the treated centre minus the control centre equals a common contrast θ, and a design d of n cell-treatment pairs has positive Gram denominator D and positive probability of being observed, then the conditional mean squared error of the un-clipped regression coefficient about θ, given that the observed design equals d, is at most M²/D.
Formal statement
Proof (Lean source)
4 supporting declarations (lemmas, instances)
-
residualRegression_memLplemma — If the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, then for every sample size n the guarded residual regression coefficient has a finite second moment under the iid product law.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝM :ℝh :ResidualAssumptions μ X A Y center MconclusionProof (Lean source)
lemma residualRegression_memLp (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (M : ℝ) (h : ResidualAssumptions μ X A Y center M) : MemLp (residualRegression (n := n) X A Y center) 2 (Measure.pi (fun _ : Fin n => μ)) := by have hm := designWeightedResidual_memLp n μ X A Y center h.X_measurable h.A_measurable h.residual_L2 (fun d i => inverseGram d * residualWeight d i) convert hm using 1 funext z exact residualRegression_eq_designWeightedResidual X A Y center z -
integral_weighted_residualRegression_sq_letheorem — If the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions and W is a nonnegative weight on designs of n cell-treatment pairs, then under the iid product law the expectation of W at the observed design times the squared residual regression coefficient is at most M² times the expectation of W times the guarded inverse Gram.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝM :ℝh :ResidualAssumptions μ X A Y center MhW :∀ d, 0 ≤ W dconclusion(∫ z : Fin n → Ω, W (sampleDesign X A z) * residualRegression X A Y center z ^ 2 ∂Measure.pi (fun _ : Fin n => μ))≤ M ^ 2 * ∫ z : Fin n → Ω, W (sampleDesign X A z) * inverseGram (sampleDesign X A z) ∂Measure.pi (fun _ : Fin n => μ)Proof (Lean source)
theorem integral_weighted_residualRegression_sq_le (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (M : ℝ) (h : ResidualAssumptions μ X A Y center M) (W : (Fin n → κ × Bool) → ℝ) (hW : ∀ d, 0 ≤ W d) : (∫ z : Fin n → Ω, W (sampleDesign X A z) * residualRegression X A Y center z ^ 2 ∂Measure.pi (fun _ : Fin n => μ)) ≤ M ^ 2 * ∫ z : Fin n → Ω, W (sampleDesign X A z) * inverseGram (sampleDesign X A z) ∂Measure.pi (fun _ : Fin n => μ) := by have hb (d : Fin n → κ × Bool) : (∑ i, (inverseGram d * residualWeight d i) ^ 2) = inverseGram d := by simp only [mul_pow, ← Finset.mul_sum, sum_residualWeight_sq] unfold inverseGram by_cases hd : 0 < gram d · simp only [hd, if_pos] field_simp [ne_of_gt hd] · simp [hd] have hv := integral_designWeightedResidual_sq_le n μ X A Y center M h.X_measurable h.A_measurable h.residual_L2 h.residual_centered h.residual_second_moment (fun d i => inverseGram d * residualWeight d i) W hW simpa only [← residualRegression_eq_designWeightedResidual, hb] using hv -
conditional_residualRegression_sq_letheorem — If the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, and a design d of n cell-treatment pairs has positive Gram denominator D and positive probability of being observed, then the conditional second moment of the residual regression coefficient given that the observed design equals d is at most M²/D.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝM :ℝh :ResidualAssumptions μ X A Y center MhD :0 < gram dhd :conclusion(∫ z in {z | sampleDesign X A z = d}, residualRegression X A Y center z ^ 2 ∂Measure.pi (fun _ : Fin n => μ)) / ((Measure.pi (fun _ : Fin n => μ)) {z | sampleDesign X A z = d}).toReal≤ M ^ 2 / gram dProof (Lean source)
theorem conditional_residualRegression_sq_le (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (M : ℝ) (h : ResidualAssumptions μ X A Y center M) (d : Fin n → κ × Bool) (hD : 0 < gram d) (hd : 0 < ((Measure.pi (fun _ : Fin n => μ)) {z | sampleDesign X A z = d}).toReal) : (∫ z in {z | sampleDesign X A z = d}, residualRegression X A Y center z ^ 2 ∂Measure.pi (fun _ : Fin n => μ)) / ((Measure.pi (fun _ : Fin n => μ)) {z | sampleDesign X A z = d}).toReal ≤ M ^ 2 / gram d := by classical let S : Set (Fin n → Ω) := {z | sampleDesign X A z = d} have hS : MeasurableSet S := (measurable_sampleDesign X A h.X_measurable h.A_measurable) (measurableSet_singleton d) have hw := integral_weighted_residualRegression_sq_le n μ X A Y center M h (fun e => if e = d then 1 else 0) (by intro e; split_ifs <;> norm_num) have hleft : (fun z : Fin n → Ω => (if sampleDesign X A z = d then (1 : ℝ) else 0) * residualRegression X A Y center z ^ 2) = S.indicator (fun z => residualRegression X A Y center z ^ 2) := by funext z by_cases hz : sampleDesign X A z = d <;> simp [S, hz] have hright : (fun z : Fin n → Ω => (if sampleDesign X A z = d then (1 : ℝ) else 0) * inverseGram (sampleDesign X A z)) = S.indicator (fun _ => inverseGram d) := by funext z by_cases hz : sampleDesign X A z = d <;> simp [S, hz] rw [hleft, hright, integral_indicator hS, integral_indicator hS, setIntegral_const, smul_eq_mul] at hw apply (div_le_iff₀ hd).2 simpa [S, Measure.real, inverseGram, hD, div_eq_mul_inv, mul_assoc, mul_comm, mul_left_comm] using hw -
regression_sub_eq_residualRegressionlemma — For a sample z of n observations with cell label X, treatment A, outcome Y and arm-and-cell centres, if the observed design has positive Gram denominator and at every sampled cell the treated centre minus the control centre equals a common contrast θ, then the un-clipped regression coefficient minus θ equals the residual regression coefficient.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕX :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝtheta :ℝz :Fin n → ΩhD :0 < gram (sampleDesign X A z)conclusionregression (sampleDesign X A z) (fun i => Y (z i)) - theta= residualRegression X A Y center zProof (Lean source)
lemma regression_sub_eq_residualRegression {n : ℕ} (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (theta : ℝ) (z : Fin n → Ω) (hD : 0 < gram (sampleDesign X A z)) (hcontrast : ∀ i, center true (X (z i)) - center false (X (z i)) = theta) : regression (sampleDesign X A z) (fun i => Y (z i)) - theta = residualRegression X A Y center z := by classical let d := sampleDesign X A z have hbase : (∑ i, residualWeight d i * center false (X (z i))) = 0 := by calc _ = ∑ k : κ, (∑ i, if (d i).1 = k then residualWeight d i else 0) * center false k := by simp_rw [Finset.sum_mul] rw [Finset.sum_comm] apply Finset.sum_congr rfl intro i hi simp [d, sampleDesign] _ = 0 := by simp [sum_residualWeight_cell] have hc (i : Fin n) : center (A (z i)) (X (z i)) = center false (X (z i)) + (if (d i).2 then (1 : ℝ) else 0) * theta := by cases ha : A (z i) <;> simp [d, sampleDesign, ha] <;> linarith [hcontrast i] have hsum : (∑ i, residualWeight d i * center (A (z i)) (X (z i))) = gram d * theta := by simp_rw [hc, mul_add, ← mul_assoc] rw [Finset.sum_add_distrib, hbase, ← Finset.sum_mul, sum_residualWeight_treatment, zero_add] unfold residualRegression regression simp only [hD, if_pos] change numerator d (fun i => Y (z i)) / gram d - theta = numerator d (fun i => Y (z i) - center (A (z i)) (X (z i))) / gram d simp only [numerator, mul_sub, Finset.sum_sub_distrib, hsum] have hd0 : gram d ≠ 0 := ne_of_gt hD field_simp [hd0]
ClipBounds 1 core · 1 supporting Clipping a real estimate to a symmetric interval contracts its squared error at every target in that interval and bounds the error by four times the squared interval radius. ★ clip_sq_error_le
Squared-error bounds for interval clipping
Clipping a real estimate to a symmetric interval contracts its squared error at every target in that interval and bounds the error by four times the squared interval radius. These deterministic facts require no sampling law.
If the clipping radius R is nonnegative and the target θ lies in the interval from −R to R, then for every real estimate t the squared error of the clipped estimate about θ is at most the squared error of t about θ.
Formal statement
Proof (Lean source)
1 supporting declaration (lemmas, instances)
-
clip_sq_error_le_fourlemma — If the clipping radius R is nonnegative and the target θ lies in the interval from −R to R, then for every real estimate t the squared error of the clipped estimate about θ is at most 4R², irrespective of the un-clipped estimate.hypothesesR t theta :ℝhR :0 ≤ Rhtheta :theta ∈ Icc (-R) Rconclusion(clip R t - theta) ^ 2 ≤ 4 * R ^ 2Proof (Lean source)
lemma clip_sq_error_le_four (R t theta : ℝ) (hR : 0 ≤ R) (htheta : theta ∈ Icc (-R) R) : (clip R t - theta) ^ 2 ≤ 4 * R ^ 2 := by rcases htheta with ⟨hlo, hhi⟩ have hcliplo : -R ≤ clip R t := le_max_left _ _ have hcliphi : clip R t ≤ R := max_le (by linarith) (min_le_left _ _) have hdifflo : -(2 * R) ≤ clip R t - theta := by linarith have hdiffhi : clip R t - theta ≤ 2 * R := by linarith nlinarith [mul_nonneg (sub_nonneg.mpr hdiffhi) (sub_nonneg.mpr hdifflo)]
DesignAlgebra 1 core · 10 supporting The Gram is both a sum of squared residualized treatment weights and the treatment-weighted residualized numerator. ★ gram_update_oscillation
Algebra and coordinate oscillation of the categorical Gram denominator
The Gram is both a sum of squared residualized treatment weights and the treatment-weighted residualized numerator. Adding a single observation to a cell changes its Gram by a number in [0,1]; deleting and adding an observation therefore gives a coordinate-replacement oscillation bound of one.
For a design d of n cell-treatment pairs, replacing the observation at a coordinate i by any cell-treatment pair v changes the Gram denominator by at most one in absolute value.
Formal statement
Proof (Lean source)
10 supporting declarations (lemmas, instances)
-
cellGram_add_treatedlemma — If a cell has total count m and treated count t with the treated count not exceeding the total, then adding one treated observation increases the cell Gram contribution by an amount between zero and one, even when the cell was empty.hypothesesm t :ℕht :t ≤ mProof (Lean source)
lemma cellGram_add_treated (m t : ℕ) (ht : t ≤ m) : 0 ≤ cellGram (m + 1) (t + 1) - cellGram m t ∧ cellGram (m + 1) (t + 1) - cellGram m t ≤ 1 := by by_cases hm : m = 0 · subst m have ht0 : t = 0 := Nat.eq_zero_of_le_zero ht subst t simp [cellGram] have hmpos : (0 : ℝ) < m := by exact_mod_cast Nat.pos_of_ne_zero hm have hmp : (0 : ℝ) < (m : ℝ) + 1 := by positivity have htr : (t : ℝ) ≤ m := by exact_mod_cast ht have htpos : (0 : ℝ) ≤ t := by positivity have heq : cellGram (m + 1) (t + 1) - cellGram m t = ((m : ℝ) - t) ^ 2 / ((m : ℝ) * (m + 1)) := by simp [cellGram, hm, Nat.cast_sub ht, Nat.add_sub_add_right, Nat.cast_add, Nat.cast_one] field_simp ring rw [heq] constructor · positivity · apply (div_le_iff₀ (mul_pos hmpos hmp)).2 nlinarith -
cellGram_add_controllemma — If a cell has total count m and treated count t with the treated count not exceeding the total, then adding one control observation increases the cell Gram contribution by an amount between zero and one, even when the cell was empty.hypothesesm t :ℕht :t ≤ mProof (Lean source)
lemma cellGram_add_control (m t : ℕ) (ht : t ≤ m) : 0 ≤ cellGram (m + 1) t - cellGram m t ∧ cellGram (m + 1) t - cellGram m t ≤ 1 := by by_cases hm : m = 0 · subst m have ht0 : t = 0 := Nat.eq_zero_of_le_zero ht subst t simp [cellGram] have hmpos : (0 : ℝ) < m := by exact_mod_cast Nat.pos_of_ne_zero hm have hmp : (0 : ℝ) < (m : ℝ) + 1 := by positivity have htr : (t : ℝ) ≤ m := by exact_mod_cast ht have htpos : (0 : ℝ) ≤ t := by positivity have heq : cellGram (m + 1) t - cellGram m t = (t : ℝ) ^ 2 / ((m : ℝ) * (m + 1)) := by simp [cellGram, hm, Nat.cast_sub ht, Nat.cast_sub (le_trans ht (le_succ m)), Nat.cast_add, Nat.cast_one] field_simp ring rw [heq] constructor · positivity · apply (div_le_iff₀ (mul_pos hmpos hmp)).2 nlinarith -
sum_cell_constantlemma — In a design d of n cell-treatment pairs, summing a real constant c over the observations in a cell k gives the cell count of k times c.hypothesesconclusion(∑ i, if (d i).1 = k then c else 0) = (cellCount (fun i => (d i).1) k : ℝ) * cProof (Lean source)
lemma sum_cell_constant {n : ℕ} (d : Fin n → κ × Bool) (k : κ) (c : ℝ) : (∑ i, if (d i).1 = k then c else 0) = (cellCount (fun i => (d i).1) k : ℝ) * c := by rw [cellCount, Nat.cast_sum, Finset.sum_mul] apply Finset.sum_congr rfl intro i hi split_ifs <;> simp -
sum_cell_treatmentlemma — In a design d of n cell-treatment pairs, summing the treatment indicators over the observations in a cell k gives the treated count of k.hypothesesconclusion(∑ i, if (d i).1 = k then (if (d i).2 then (1 : ℝ) else 0) else 0)= (treatedCount d k : ℝ)Proof (Lean source)
lemma sum_cell_treatment {n : ℕ} (d : Fin n → κ × Bool) (k : κ) : (∑ i, if (d i).1 = k then (if (d i).2 then (1 : ℝ) else 0) else 0) = (treatedCount d k : ℝ) := by rw [treatedCount, Nat.cast_sum] apply Finset.sum_congr rfl intro i hi by_cases hk : (d i).1 = k <;> cases (d i).2 <;> simp [hk] -
sum_residualWeight_celllemma — In a design d of n cell-treatment pairs, the residualized treatment weights of the observations in any cell k sum to zero, including in empty cells.hypothesesconclusion(∑ i, if (d i).1 = k then residualWeight d i else 0) = 0Proof (Lean source)
lemma sum_residualWeight_cell {n : ℕ} (d : Fin n → κ × Bool) (k : κ) : (∑ i, if (d i).1 = k then residualWeight d i else 0) = 0 := by have heq : (∑ i, if (d i).1 = k then residualWeight d i else 0) = (treatedCount d k : ℝ) - (cellCount (fun i => (d i).1) k : ℝ) * treatmentFraction d k := by rw [← sum_cell_treatment d k, ← sum_cell_constant d k, ← Finset.sum_sub_distrib] apply Finset.sum_congr rfl intro i hi by_cases hk : (d i).1 = k <;> simp [residualWeight, hk] rw [heq] have ht := treatedCount_le d k unfold treatmentFraction split_ifs with hm · have ht0 : treatedCount d k = 0 := Nat.eq_zero_of_le_zero (hm ▸ ht) simp [hm, ht0] · have hm' : (cellCount (fun i => (d i).1) k : ℝ) ≠ 0 := by exact_mod_cast hm field_simp ring -
sum_cell_residual_treatmentlemma — In a design d of n cell-treatment pairs, summing the residualized treatment weight times the treatment indicator over the observations in a cell k gives that cell's Gram contribution.hypothesesconclusion(∑ i, if (d i).1 = k then residualWeight d i * (if (d i).2 then (1 : ℝ) else 0) else 0)= cellGram (cellCount (fun i => (d i).1) k) (treatedCount d k)Proof (Lean source)
lemma sum_cell_residual_treatment {n : ℕ} (d : Fin n → κ × Bool) (k : κ) : (∑ i, if (d i).1 = k then residualWeight d i * (if (d i).2 then (1 : ℝ) else 0) else 0) = cellGram (cellCount (fun i => (d i).1) k) (treatedCount d k) := by have heq : (∑ i, if (d i).1 = k then residualWeight d i * (if (d i).2 then (1 : ℝ) else 0) else 0) = (treatedCount d k : ℝ) * (1 - treatmentFraction d k) := by rw [← sum_cell_treatment d k, Finset.sum_mul] apply Finset.sum_congr rfl intro i hi by_cases hk : (d i).1 = k <;> cases ha : (d i).2 <;> simp [residualWeight, hk, ha] rw [heq] have ht := treatedCount_le d k unfold treatmentFraction cellGram split_ifs with hm · have ht0 : treatedCount d k = 0 := Nat.eq_zero_of_le_zero (hm ▸ ht) simp [ht0] · rw [Nat.cast_sub ht] have hm' : (cellCount (fun i => (d i).1) k : ℝ) ≠ 0 := by exact_mod_cast hm field_simp -
sum_residualWeight_sqlemma — For every design d of n cell-treatment pairs, the sum of the squared residualized treatment weights equals the Gram denominator.hypothesesconclusion(∑ i, residualWeight d i ^ 2) = gram dProof (Lean source)
lemma sum_residualWeight_sq {n : ℕ} (d : Fin n → κ × Bool) : (∑ i, residualWeight d i ^ 2) = gram d := by have hcell (k : κ) : (∑ i, if (d i).1 = k then residualWeight d i ^ 2 else 0) = cellGram (cellCount (fun i => (d i).1) k) (treatedCount d k) := by rw [← sum_cell_residual_treatment d k] have heq : (∑ i, if (d i).1 = k then residualWeight d i ^ 2 else 0) = (∑ i, if (d i).1 = k then residualWeight d i * (if (d i).2 then (1 : ℝ) else 0) else 0) - treatmentFraction d k * (∑ i, if (d i).1 = k then residualWeight d i else 0) := by rw [Finset.mul_sum, ← Finset.sum_sub_distrib] apply Finset.sum_congr rfl intro i hi by_cases hk : (d i).1 = k · cases ha : (d i).2 <;> simp [hk, residualWeight, ha] <;> ring · simp [hk] rw [heq, sum_residualWeight_cell, mul_zero, sub_zero] unfold gram simp_rw [← hcell] rw [Finset.sum_comm] simp -
sum_residualWeight_treatmentlemma — For every design d of n cell-treatment pairs, the sum of the treatment indicators weighted by their residualized treatment weights equals the Gram denominator.hypothesesconclusion(∑ i, residualWeight d i * (if (d i).2 then (1 : ℝ) else 0)) = gram dProof (Lean source)
lemma sum_residualWeight_treatment {n : ℕ} (d : Fin n → κ × Bool) : (∑ i, residualWeight d i * (if (d i).2 then (1 : ℝ) else 0)) = gram d := by unfold gram simp_rw [← sum_cell_residual_treatment d] rw [Finset.sum_comm] simp -
gram_le_sample_quarterlemma — For every design d of n cell-treatment pairs, the Gram denominator is at most n/4.Proof (Lean source)
lemma gram_le_sample_quarter {n : ℕ} (d : Fin n → κ × Bool) : gram d ≤ (n : ℝ) / 4 := by have hcell (k : κ) : cellGram (cellCount (fun i => (d i).1) k) (treatedCount d k) ≤ (cellCount (fun i => (d i).1) k : ℝ) / 4 := by have ht := treatedCount_le d k unfold cellGram split_ifs with hm · simp [hm] · rw [Nat.cast_sub ht] have hmpos : (0 : ℝ) < cellCount (fun i => (d i).1) k := by exact_mod_cast Nat.pos_of_ne_zero hm apply (div_le_iff₀ hmpos).2 nlinarith [sq_nonneg ((cellCount (fun i => (d i).1) k : ℝ) - 2 * (treatedCount d k : ℝ))] calc gram d ≤ ∑ k, (cellCount (fun i => (d i).1) k : ℝ) / 4 := Finset.sum_le_sum fun k _ => hcell k _ = (n : ℝ) / 4 := by rw [← Finset.sum_div, ← Nat.cast_sum, sum_cellCount] -
sum_cellGram_addlemma — If cell total counts m and treated counts t satisfy treated count at most total count in every cell, then adding one observation v increases the summed cell Gram contributions by an amount between zero and one.hypothesesProof (Lean source)
lemma sum_cellGram_add (m t : κ → ℕ) (ht : ∀ k, t k ≤ m k) (v : κ × Bool) : 0 ≤ (∑ k, cellGram (m k + if v.1 = k then 1 else 0) (t k + if v.1 = k ∧ v.2 = true then 1 else 0)) - (∑ k, cellGram (m k) (t k)) ∧ (∑ k, cellGram (m k + if v.1 = k then 1 else 0) (t k + if v.1 = k ∧ v.2 = true then 1 else 0)) - (∑ k, cellGram (m k) (t k)) ≤ 1 := by have heq : (∑ k, cellGram (m k + if v.1 = k then 1 else 0) (t k + if v.1 = k ∧ v.2 = true then 1 else 0)) - (∑ k, cellGram (m k) (t k)) = cellGram (m v.1 + 1) (t v.1 + if v.2 then 1 else 0) - cellGram (m v.1) (t v.1) := by rw [← Finset.sum_sub_distrib] calc _ = ∑ k, if k = v.1 then cellGram (m v.1 + 1) (t v.1 + if v.2 then 1 else 0) - cellGram (m v.1) (t v.1) else 0 := by apply Finset.sum_congr rfl intro k hk by_cases h : k = v.1 · subst k cases v.2 <;> simp · simp [h, Ne.symm h] _ = _ := by simp rw [heq] cases ha : v.2 · simpa [ha] using cellGram_add_control (m v.1) (t v.1) (ht v.1) · simpa [ha] using cellGram_add_treated (m v.1) (t v.1) (ht v.1)
ObservedSupport 1 core · 0 supporting A finite categorical sample almost surely visits only positive-mass cells. ★ ae_sample_cell_positive
Almost-sure support of observed categorical designs
A finite categorical sample almost surely visits only positive-mass cells. This supports occupied-cell assumptions in both conditional residual bounds and risk calculations without imposing assumptions on null cells.
An iid sample law and measurable cell and treatment labels ensure that every observed cell has positive population mass almost surely.
Formal statement
Proof (Lean source)
Occupancy 1 core · 2 supporting Under an arbitrary categorical iid law, the expected repeat count is n minus the expected number of occupied cells. ★ integral_repeatCount_lower
Uniform repeat-count expectation
Under an arbitrary categorical iid law, the expected repeat count is n minus the expected number of occupied cells. Convexity of the empty-cell probability shows that the uniform law minimizes repetitions among laws with at most n available cells. For n at least two this gives n/4; at n=1 repetitions are identically zero. Null cells are included throughout.
In an iid sample of size n from any probability law Q on cells, if there are at least two observations and the number of cells is at most n, then the expected repeat count is at least n/4.
Formal statement
Proof (Lean source)
2 supporting declarations (lemmas, instances)
-
probability_empty_celllemma — In an iid sample of size n from a probability law Q on cells, the probability that a cell k is empty equals (1 − q)ⁿ, where q is the mass Q gives to k.hypothesesProof (Lean source)
lemma probability_empty_cell (n : ℕ) (Q : Measure κ) [IsProbabilityMeasure Q] (k : κ) : ((Measure.pi (fun _ : Fin n => Q)) {x | cellCount x k = 0}).toReal = (1 - (Q {k}).toReal) ^ n := by classical have hevent : {x : Fin n → κ | cellCount x k = 0} = Set.univ.pi (fun _ : Fin n => ({k} : Set κ)ᶜ) := by ext x simp [cellCount, Set.mem_pi] rw [hevent, Measure.pi_pi] simp only [Finset.prod_const, Finset.card_univ, Fintype.card_fin, ENNReal.toReal_pow] have hc := probReal_add_probReal_compl (μ := Q) (measurableSet_singleton k) have hreal : (Q ({k}ᶜ)).toReal = 1 - (Q {k}).toReal := by change (Q.real {k}ᶜ) = 1 - Q.real {k} linarith rw [hreal] -
integral_repeatCounttheorem — In an iid sample of size n from any probability law Q on cells, the expected repeat count equals n minus the sum over cells of the occupancy probabilities 1 − (1 − q)ⁿ, where q is the cell's mass; zero-mass cells are included.hypothesesProof (Lean source)
theorem integral_repeatCount (n : ℕ) (Q : Measure κ) [IsProbabilityMeasure Q] : (∫ x : Fin n → κ, (repeatCount x : ℝ) ∂Measure.pi (fun _ : Fin n => Q)) = (n : ℝ) - ∑ k, (1 - (1 - (Q {k}).toReal) ^ n) := by classical let μ := Measure.pi (fun _ : Fin n => Q) have hidentity (x : Fin n → κ) : (repeatCount x : ℝ) = (n : ℝ) - ∑ k, (1 - if cellCount x k = 0 then (1 : ℝ) else 0) := by have hcell (k : κ) : ((cellCount x k - 1 : ℕ) : ℝ) = (cellCount x k : ℝ) - (1 - if cellCount x k = 0 then (1 : ℝ) else 0) := by by_cases h : cellCount x k = 0 · simp [h] · simp only [h, if_false, sub_zero] rw [Nat.cast_sub (by omega : 1 ≤ cellCount x k), Nat.cast_one] simp only [repeatCount, Nat.cast_sum, hcell, Finset.sum_sub_distrib] rw [← Nat.cast_sum, sum_cellCount] have hempty (k : κ) : (∫ x : Fin n → κ, (if cellCount x k = 0 then (1 : ℝ) else 0) ∂μ) = (1 - (Q {k}).toReal) ^ n := by have hm : MeasurableSet {x : Fin n → κ | cellCount x k = 0} := Set.to_countable _ |>.measurableSet have hf : (fun x : Fin n → κ => if cellCount x k = 0 then (1 : ℝ) else 0) = {x : Fin n → κ | cellCount x k = 0}.indicator (fun _ => (1 : ℝ)) := by funext x simp [indicator] rw [hf, integral_indicator_const 1 hm] simpa [μ, Measure.real] using probability_empty_cell n Q k simp_rw [hidentity] rw [integral_sub (integrable_const _) Integrable.of_finite, integral_finsetSum _ (fun _ _ => Integrable.of_finite)] simp only [integral_const, Measure.real, measure_univ, ENNReal.toReal_one, smul_eq_mul, one_mul] congr 1 apply Finset.sum_congr rfl intro k hk rw [integral_sub (integrable_const _) Integrable.of_finite, hempty] simp
OccupancyScalar 1 core · 1 supporting The empty-cell polynomial for any probability vector with at most n entries has at least n/4 expected repetitions for n at least two. ★ occupancy_polynomial_lower
Scalar occupancy inequalities
The empty-cell polynomial for any probability vector with at most n entries has at least n/4 expected repetitions for n at least two. This module isolates the analytic inequalities from the finite-product probability calculations.
For a sample size n and a vector q of cell masses, if n is at least two, there are at most n cells, and the masses are nonnegative and sum to one, then n minus the sum over cells of 1 − (1 − q)ⁿ — the expected repeat count — is at least n/4.
Formal statement
Proof (Lean source)
1 supporting declaration (lemmas, instances)
-
uniform_empty_probability_lowerlemma — If the sample size n is at least two, then the empty-cell probability under the uniform law on n cells, (1 − 1/n)ⁿ, is at least one quarter.hypothesesn :ℕhn :2 ≤ nconclusion(1 / 4 : ℝ) ≤ (1 - 1 / (n : ℝ)) ^ nProof (Lean source)
lemma uniform_empty_probability_lower (n : ℕ) (hn : 2 ≤ n) : (1 / 4 : ℝ) ≤ (1 - 1 / (n : ℝ)) ^ n := by -- Induct from the exact value at two. The ratio of consecutive bases is -- 1 + 1/(m²-1); Bernoulli bounds its m-th power from below. induction n, hn using Nat.le_induction with | base => norm_num | succ m hm ih => have hmR : (2 : ℝ) ≤ m := by exact_mod_cast hm have hm0 : (m : ℝ) ≠ 0 := by linarith have hmp : (m : ℝ) + 1 ≠ 0 := by linarith have hd : 0 < (m : ℝ) ^ 2 - 1 := by nlinarith have hb : 0 ≤ 1 - 1 / (m : ℝ) := by rw [sub_nonneg, div_le_iff₀ (by linarith : (0 : ℝ) < m)] linarith have hbern := one_add_mul_le_pow (a := 1 / ((m : ℝ) ^ 2 - 1)) (by have hx : (0 : ℝ) ≤ 1 / ((m : ℝ) ^ 2 - 1) := by positivity linarith) m have hfac : 1 ≤ (1 + (m : ℝ) / ((m : ℝ) ^ 2 - 1)) * (1 - 1 / ((m : ℝ) + 1)) := by field_simp nlinarith have hid : 1 - 1 / ((m : ℝ) + 1) = (1 - 1 / (m : ℝ)) * (1 + 1 / ((m : ℝ) ^ 2 - 1)) := by field_simp ring have hstep : (1 - 1 / (m : ℝ)) ^ m ≤ (1 - 1 / ((m : ℝ) + 1)) ^ (m + 1) := by rw [pow_succ, hid, mul_pow] have hnew : 0 ≤ 1 - 1 / ((m : ℝ) + 1) := by rw [sub_nonneg, div_le_iff₀ (by linarith : (0 : ℝ) < m + 1)] linarith have hmul := mul_le_mul_of_nonneg_right hbern hnew have hprod : 1 ≤ (1 + 1 / ((m : ℝ) ^ 2 - 1)) ^ m * ((1 - 1 / (m : ℝ)) * (1 + 1 / ((m : ℝ) ^ 2 - 1))) := by rw [← hid] exact hfac.trans (by simpa only [div_eq_mul_inv, one_mul] using hmul) nlinarith [mul_nonneg (pow_nonneg hb m) (sub_nonneg.mpr hprod)] exact ih.trans (by simpa only [Nat.cast_add, Nat.cast_one] using hstep)
ResidualWeights 2 core · 2 supporting This module isolates the square-integrability and variance calculation for a sum of arm-cell residuals with arbitrary coefficients depending on the entire observed categorical design. ★ integral_designWeightedResidual_sq_le
Arbitrary finite-design residual coefficients
This module isolates the square-integrability and variance calculation for a sum of arm-cell residuals with arbitrary coefficients depending on the entire observed categorical design. It imposes no bounded-outcome condition. The regression application takes each coefficient to be the residualized treatment weight times the guarded inverse Gram. No categorical conditioning or Gram tail theorem is needed in this lower dependency layer.
The design-weighted residual sum of a sample z of n observations with cell label X, treatment A and outcome Y adds, over observations, the residual of the outcome from its arm-and-cell centre times a coefficient that may depend on the whole observed design.
Definition (Lean source)
Under a probability law μ of one observation, suppose the cell label X and the treatment A are measurable, and on each arm-cell event the residual of the outcome Y from its centre is square integrable, integrates to zero, and has integral of its square at most the event's mass times M². Then for any design-dependent coefficients b and any nonnegative weight W on designs of n cell-treatment pairs, the expectation under the iid product law of W at the observed design times the squared design-weighted residual sum is at most M² times the expectation of W times the sum of the squared coefficients.
Formal statement
Proof (Lean source)
2 supporting declarations (lemmas, instances)
-
designWeightedResidual_memLplemma — Under a probability law μ of one observation, if the cell label X and the treatment A are measurable, Y is the outcome, and each arm-cell-supported residual from the centres is square integrable, then for every sample size n and every design-dependent coefficient the design-weighted residual sum is square integrable under the iid product law.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝhX :hA :hmem :∀ a k, MemLp (supportedArmGroupResidual X A Y center a k) 2 μconclusionProof (Lean source)
lemma designWeightedResidual_memLp (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (hX : Measurable X) (hA : Measurable A) (hmem : ∀ a k, MemLp (supportedArmGroupResidual X A Y center a k) 2 μ) (b : (Fin n → κ × Bool) → Fin n → ℝ) : MemLp (designWeightedResidual X A Y center b) 2 (Measure.pi (fun _ : Fin n => μ)) := by classical have hr : MemLp (fun ω => Y ω - center (A ω) (X ω)) 2 μ := by have hs := memLp_finsetSum univ (fun (c : Bool × κ) _ => hmem c.1 c.2) simpa only [← residual_eq_sum_supported X A Y center] using hs unfold designWeightedResidual exact memLp_finsetSum univ fun i _ => memLp_mul_design μ X A hX hA _ (hr.comp_measurePreserving (measurePreserving_eval (fun _ : Fin n => μ) i)) (fun d => b d i) -
residualRegression_eq_designWeightedResiduallemma — For a sample z of n observations with any cell label X, treatment A, outcome Y and arm-and-cell centres, the guarded residual regression coefficient equals the design-weighted residual sum whose coefficients are the guarded inverse Gram times the residualized treatment weights, also at a zero Gram.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕX :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝz :Fin n → ΩconclusionresidualRegression X A Y center zProof (Lean source)
lemma residualRegression_eq_designWeightedResidual {n : ℕ} (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (z : Fin n → Ω) : residualRegression X A Y center z = designWeightedResidual X A Y center (fun d i => inverseGram d * residualWeight d i) z := by classical unfold residualRegression regression designWeightedResidual numerator inverseGram by_cases hp : 0 < gram (sampleDesign X A z) · simp only [hp, if_pos, Finset.mul_sum, div_eq_mul_inv, Finset.sum_mul] apply Finset.sum_congr rfl intro i _ ring · simp [hp]
Risk 1 core · 5 supporting The good-design conditional second moment costs M²/(c n). ★ clipped_regression_mse_le
Finite-sample MSE of clipped within-cell regression
The good-design conditional second moment costs M²/(c n). On the bad event, interval clipping bounds squared error by four times the squared interval radius. Combining the derived Gram tail and residual theorem yields a uniform 1/n risk bound, including D=0, n=1, arbitrary masses, and null cells.
The general theorem allows a clipping radius R distinct from the residual moment envelope M. With R=M and epsilon=1/4 its constant is 33152/9, less than the reference constant 72+4(1296+729/64).
Suppose the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, the sample size n is positive and at least the number of cells, the clipping radius R is nonnegative, the target θ lies between −R and R, the overlap margin ε is positive and at most one half, every cell of positive probability has propensity between ε and 1 − ε, and on every such cell the treated centre minus the control centre equals θ. Then the mean squared error of the clipped within-cell regression estimator about θ is at most (M²/c + 4R²·C)/n, where c is the good-design threshold and C the uniform Gram lower-tail constant at ε.
Formal statement
Proof (Lean source)
5 supporting declarations (lemmas, instances)
-
integrable_clipped_error_sqlemma — Clipped regression squared error is integrable under any finite iid law when the observation maps are measurable, the clipping radius is nonnegative, and the target lies in the clipping interval. The integrability conclusion needs no outcome moment bound.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝR theta :ℝhX :hA :hY :hR :0 ≤ Rhtheta :theta ∈ Icc (-R) RconclusionIntegrable (fun z : Fin n → Ω => (clippedEstimator X A Y R z - theta) ^ 2) (Measure.pi (fun _ : Fin n => μ))Proof (Lean source)
lemma integrable_clipped_error_sq (n : ℕ) (μ : Measure Ω) [IsFiniteMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (R theta : ℝ) (hX : Measurable X) (hA : Measurable A) (hY : Measurable Y) (hR : 0 ≤ R) (htheta : theta ∈ Icc (-R) R) : Integrable (fun z : Fin n → Ω => (clippedEstimator X A Y R z - theta) ^ 2) (Measure.pi (fun _ : Fin n => μ)) := by have hm : Measurable (fun z : Fin n → Ω => (clippedEstimator X A Y R z - theta) ^ 2) := ((measurable_clippedEstimator X A Y R hX hA hY).sub measurable_const).pow_const 2 apply Integrable.of_bound hm.aestronglyMeasurable (4 * R ^ 2) filter_upwards [] with z rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)] exact clip_sq_error_le_four R _ theta hR htheta -
integral_good_error_sq_letheorem — Suppose the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, the clipping radius R is nonnegative, the target θ lies between −R and R, and on every cell of positive probability the treated centre minus the control centre equals θ. Then for every positive Gram threshold t and every sample size n, the integral of the squared error of the clipped estimator about θ over samples whose Gram denominator is at least t is at most M²/t.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝM R theta t :ℝh :ResidualAssumptions μ X A Y center MhR :0 ≤ Rhtheta :theta ∈ Icc (-R) Rht :0 < thcontrast :∀ kif0 < cellProbability (μ.map (fun ω => (X ω, A ω))) kconclusion(∫ z in {z : Fin n → Ω | t ≤ gram (sampleDesign X A z)}, (clippedEstimator X A Y R z - theta) ^ 2 ∂Measure.pi (fun _ : Fin n => μ))≤ M ^ 2 / tProof (Lean source)
theorem integral_good_error_sq_le (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (M R theta t : ℝ) (h : ResidualAssumptions μ X A Y center M) (hR : 0 ≤ R) (htheta : theta ∈ Icc (-R) R) (ht : 0 < t) (hcontrast : ∀ k, 0 < cellProbability (μ.map (fun ω => (X ω, A ω))) k → center true k - center false k = theta) : (∫ z in {z : Fin n → Ω | t ≤ gram (sampleDesign X A z)}, (clippedEstimator X A Y R z - theta) ^ 2 ∂Measure.pi (fun _ : Fin n => μ)) ≤ M ^ 2 / t := by classical let P := Measure.pi (fun _ : Fin n => μ) let S : Set (Fin n → Ω) := {z | t ≤ gram (sampleDesign X A z)} let W : (Fin n → κ × Bool) → ℝ := fun d => if t ≤ gram d then 1 else 0 have hS : MeasurableSet S := measurableSet_le measurable_const ((measurable_designStatistic gram).comp (measurable_sampleDesign X A h.X_measurable h.A_measurable)) have hf := integrable_clipped_error_sq n μ X A Y R theta h.X_measurable h.A_measurable h.Y_measurable hR htheta have hg := (residualRegression_memLp n μ X A Y center M h).integrable_sq have hgood : (∫ z in S, (clippedEstimator X A Y R z - theta) ^ 2 ∂P) ≤ ∫ z in S, residualRegression X A Y center z ^ 2 ∂P := by apply setIntegral_mono_ae_restrict hf.integrableOn hg.integrableOn filter_upwards [ae_restrict_of_ae (ae_sample_cell_positive n μ X A h.X_measurable h.A_measurable), ae_restrict_mem hS] with z hz hzt have hD : 0 < gram (sampleDesign X A z) := lt_of_lt_of_le ht hzt have heq := regression_sub_eq_residualRegression X A Y center theta z hD (fun i => hcontrast _ (hz i)) simpa only [clippedEstimator, heq] using clip_sq_error_le R (regression (sampleDesign X A z) (fun i => Y (z i))) theta hR htheta have hw := integral_weighted_residualRegression_sq_le n μ X A Y center M h W (by intro d; dsimp [W]; split_ifs <;> norm_num) have hleft : (fun z : Fin n → Ω => W (sampleDesign X A z) * residualRegression X A Y center z ^ 2) = S.indicator (fun z => residualRegression X A Y center z ^ 2) := by funext z by_cases hz : t ≤ gram (sampleDesign X A z) <;> simp [W, S, hz] rw [hleft, integral_indicator hS] at hw have hb (z : Fin n → Ω) : 0 ≤ W (sampleDesign X A z) * inverseGram (sampleDesign X A z) ∧ W (sampleDesign X A z) * inverseGram (sampleDesign X A z) ≤ 1 / t := by by_cases hz : t ≤ gram (sampleDesign X A z) · have hD := lt_of_lt_of_le ht hz simpa [W, hz, inverseGram, hD, one_div] using intro (inv_nonneg.mpr hD.le) (one_div_le_one_div_of_le ht hz) · simp [W, hz, le_of_lt ht] have hi : Integrable (fun z : Fin n → Ω => W (sampleDesign X A z) * inverseGram (sampleDesign X A z)) P := by apply Integrable.of_bound (((measurable_designStatistic (fun d => W d * inverseGram d)).comp (measurable_sampleDesign X A h.X_measurable h.A_measurable)).aestronglyMeasurable) (1 / t) filter_upwards [] with z dsimp only [Function.comp_def] rw [Real.norm_eq_abs, abs_of_nonneg (hb z).1] exact (hb z).2 have hbound : (∫ z : Fin n → Ω, W (sampleDesign X A z) * inverseGram (sampleDesign X A z) ∂P) ≤ 1 / t := by calc _ ≤ ∫ _ : Fin n → Ω, (1 / t : ℝ) ∂P := integral_mono_ae hi (integrable_const _) (Filter.Eventually.of_forall (fun z => (hb z).2)) _ = 1 / t := by simp [P] calc _ ≤ ∫ z in S, residualRegression X A Y center z ^ 2 ∂P := hgood _ ≤ M ^ 2 * ∫ z : Fin n → Ω, W (sampleDesign X A z) * inverseGram (sampleDesign X A z) ∂P := hw _ ≤ M ^ 2 * (1 / t) := mul_le_mul_of_nonneg_left hbound (sq_nonneg M) _ = M ^ 2 / t := by ring -
integral_clipped_error_sq_le_splittheorem — Suppose the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, the clipping radius R is nonnegative, the target θ lies between −R and R, and on every cell of positive probability the treated centre minus the control centre equals θ. Then for every positive Gram threshold t and every sample size n, the mean squared error of the clipped estimator about θ is at most M²/t plus 4R² times the probability that the Gram denominator is below t. That bad-design probability is subsequently bounded by the Gram lower tail.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝM R theta t :ℝh :ResidualAssumptions μ X A Y center MhR :0 ≤ Rhtheta :theta ∈ Icc (-R) Rht :0 < thcontrast :∀ kif0 < cellProbability (μ.map (fun ω => (X ω, A ω))) kconclusion≤ M ^ 2 / tProof (Lean source)
theorem integral_clipped_error_sq_le_split (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (M R theta t : ℝ) (h : ResidualAssumptions μ X A Y center M) (hR : 0 ≤ R) (htheta : theta ∈ Icc (-R) R) (ht : 0 < t) (hcontrast : ∀ k, 0 < cellProbability (μ.map (fun ω => (X ω, A ω))) k → center true k - center false k = theta) : (∫ z : Fin n → Ω, (clippedEstimator X A Y R z - theta) ^ 2 ∂Measure.pi (fun _ : Fin n => μ)) ≤ M ^ 2 / t + 4 * R ^ 2 * ((Measure.pi (fun _ : Fin n => μ)) {z | gram (sampleDesign X A z) < t}).toReal := by let P := Measure.pi (fun _ : Fin n => μ) let S : Set (Fin n → Ω) := {z | t ≤ gram (sampleDesign X A z)} have hS : MeasurableSet S := measurableSet_le measurable_const ((measurable_designStatistic gram).comp (measurable_sampleDesign X A h.X_measurable h.A_measurable)) have hf := integrable_clipped_error_sq n μ X A Y R theta h.X_measurable h.A_measurable h.Y_measurable hR htheta have hgood := integral_good_error_sq_le n μ X A Y center M R theta t h hR htheta ht hcontrast have hbad : (∫ z in Sᶜ, (clippedEstimator X A Y R z - theta) ^ 2 ∂P) ≤ 4 * R ^ 2 * (P Sᶜ).toReal := by calc _ ≤ ∫ _ : Fin n → Ω in Sᶜ, (4 * R ^ 2 : ℝ) ∂P := by apply setIntegral_mono_ae hf.integrableOn (integrable_const _) filter_upwards [] with z exact clip_sq_error_le_four R _ theta hR htheta _ = 4 * R ^ 2 * (P Sᶜ).toReal := by rw [setIntegral_const, smul_eq_mul] exact mul_comm _ _ have hcompl : Sᶜ = {z | gram (sampleDesign X A z) < t} := by ext z simp [S, not_le] rw [← integral_add_compl hS hf] calc _ ≤ M ^ 2 / t + 4 * R ^ 2 * (P Sᶜ).toReal := add_le_add hgood hbad _ = _ := by rw [hcompl] -
clipped_regression_mse_le_quartertheorem — Suppose the observation law μ, label X, treatment A, outcome Y, centres and envelope M satisfy the residual assumptions, the sample size n is positive and at least the number of cells, the envelope M is nonnegative, the target θ lies between −M and M, every cell of positive probability has propensity between 1/4 and 3/4, and on every such cell the treated centre minus the control centre equals θ. Then the within-cell regression estimator clipped at radius M has mean squared error about θ at most (33152/9)·M²/n, uniformly over cell masses.hypothesesΩ :sharedType u_1κ :sharedType u_2n :ℕμ :X :Ω → κA :Ω → BoolY :Ω → ℝcenter :Bool → κ → ℝM theta :ℝh :ResidualAssumptions μ X A Y center Mhn :0 < nhcard :card κ ≤ nhM :0 ≤ Mhtheta :theta ∈ Icc (-M) Mhoverlap :∀ kif0 < cellProbability (μ.map (fun ω => (X ω, A ω))) kthen(1 / 4 : ℝ) ≤ propensity (μ.map (fun ω => (X ω, A ω))) k ∧propensity (μ.map (fun ω => (X ω, A ω))) k ≤ 1 - (1 / 4 : ℝ)hcontrast :∀ kif0 < cellProbability (μ.map (fun ω => (X ω, A ω))) kconclusion≤ (33152 / 9 : ℝ) * M ^ 2 / nProof (Lean source)
theorem clipped_regression_mse_le_quarter (n : ℕ) (μ : Measure Ω) [IsProbabilityMeasure μ] (X : Ω → κ) (A : Ω → Bool) (Y : Ω → ℝ) (center : Bool → κ → ℝ) (M theta : ℝ) (h : ResidualAssumptions μ X A Y center M) (hn : 0 < n) (hcard : card κ ≤ n) (hM : 0 ≤ M) (htheta : theta ∈ Icc (-M) M) (hoverlap : ∀ k, 0 < cellProbability (μ.map (fun ω => (X ω, A ω))) k → (1 / 4 : ℝ) ≤ propensity (μ.map (fun ω => (X ω, A ω))) k ∧ propensity (μ.map (fun ω => (X ω, A ω))) k ≤ 1 - (1 / 4 : ℝ)) (hcontrast : ∀ k, 0 < cellProbability (μ.map (fun ω => (X ω, A ω))) k → center true k - center false k = theta) : (∫ z : Fin n → Ω, (clippedEstimator X A Y M z - theta) ^ 2 ∂Measure.pi (fun _ : Fin n => μ)) ≤ (33152 / 9 : ℝ) * M ^ 2 / n := by have hmse := clipped_regression_mse_le n μ X A Y center M M theta (1 / 4) h hn hcard hM htheta (by norm_num) (by norm_num) hoverlap hcontrast have hconstant : M ^ 2 / gramThreshold (1 / 4) + 4 * M ^ 2 * gramTailConstant (1 / 4) = (33152 / 9 : ℝ) * M ^ 2 := by norm_num [gramThreshold, gramTailConstant] ring rwa [hconstant] at hmse -
quarter_constant_le_paperlemma — The quarter-overlap risk constant 33152/9 is at most the reference constant 72 + 4(1296 + 729/64).conclusion(33152 / 9 : ℝ) ≤ 72 + 4 * (1296 + 729 / 64)Proof (Lean source)
lemma quarter_constant_le_paper : (33152 / 9 : ℝ) ≤ 72 + 4 * (1296 + 729 / 64) := by norm_num