Formalization: Random Group Formation and the Equal-group CR2 Variance in Finite Populations
The complete Lean development behind this paper — every definition, lemma, and theorem of its module, including helpers the paper text never cites. Identifiers link within this page, into the Causalean library, or out to the official Mathlib docs.
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Basic 11 declarations This file defines the deterministic composition-indexed potential-outcome schedule, the uniform fixed-cardinality slice, and its group-level moments.
Dense random-group experiments: finite-population slice
This file defines the deterministic composition-indexed potential-outcome schedule, the uniform fixed-cardinality slice, and its group-level moments.
def Population
The labeled finite population.
Definition (Lean source)
The fixed-cardinality slice of candidate groups.
The two treatment arms.
Definition (Lean source)
def
Arm :
Type
given by
def PotentialOutcome
A deterministic potential-outcome schedule, defined only for units belonging to the group.
The uniform design on the M-slice.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
hM :
M ≤ n
def
given by
completeRandomization M (by simpa using hM)
Uniform-slice inner product.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
hM :
M ≤ n
f g :
Omega n M → ℝ
def
sliceInner n M hM f g :
ℝ
given by
(slice n M hM).E (fun A => f A * g A)
Squared uniform-slice norm.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
hM :
M ≤ n
f :
Omega n M → ℝ
def
sliceNormSq n M hM f :
ℝ
given by
sliceInner n M hM f f
Uniform-slice norm.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
hM :
M ≤ n
f :
Omega n M → ℝ
def
sliceNorm n M hM f :
ℝ
given by
sqrt (sliceNormSq n M hM f)
def armTable
The arm-specific mean outcome of a candidate group.
Definition (Lean source)
parameters
def
armTable n M Y z A :
ℝ
given by
(∑ i ∈ A.1.attach, Y A i z) / (M : ℝ)
The centered arm table.
Definition (Lean source)
parameters
def
armTableCentered n M hM Y z A :
ℝ
The uniform-slice arm variance.
Definition (Lean source)
parameters
def
armVar n M hM Y z :
ℝ
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Asymptotics 20 declarations This file bundles each deterministic row of the experiment and defines exactly the five asymptotic/support conditions used by the paper.
Triangular arrays and the dense schedule class
This file bundles each deterministic row of the experiment and defines exactly the five asymptotic/support conditions used by the paper.
A triangular array of feasible dense random-group experiment rows.
Definition (Lean source)
parameters
M :
ℕ
fields
@realizes M(fixed across rows)
groupSize_ge_two :
2 ≤ M
popSize :
ℕ → ℕ
groups :
ℕ → ℕ
treated :
ℕ → ℕ
groups_pos :
∀ r, 0 < groups r
grouped_le :
∀ r, M * groups r ≤ popSize r
@realizes N_n(M times G and at most n)
treated_pos :
∀ r, 0 < treated r
treated_lt :
∀ r, treated r < groups r
treated_le :
∀ r, treated r ≤ groups r
schedule :
∀ r, PotentialOutcome (popSize r) M
def controls
Row control-group count.
Definition (Lean source)
def grouped
Row grouped-unit count.
Definition (Lean source)
Row treatment fraction.
Definition (Lean source)
parameters
def
treatmentFraction A r :
ℝ
given by
pFrac (A.groups r) (A.treated r) (A.treated_pos r) (A.treated_lt r)
def design
The row's joint finite design.
Definition (Lean source)
parameters
def
design A r :
FiniteDesign (PartitionTuple (A.popSize r) M (A.groups r) × TreatmentSpace (A.groups r) (A.treated r))
given by
randomPartitionDesign (A.popSize r) M (A.groups r) (A.treated r) (A.grouped_le r) (A.treated_le r)
def variance
The row's exact PAME variance.
Definition (Lean source)
parameters
def
variance A r :
ℝ
given by
sigmaSq (A.schedule r) (A.grouped_le r) (A.treated_pos r) (A.treated_lt r)
theorem groupSize_le
Each feasible row contains at least one whole group.
Formal statement
Proof (Lean source)
lemma ScheduleArray.groupSize_le {M : ℕ} (A : ScheduleArray M) (r : ℕ) : M ≤ A.popSize r := by
exact le_trans (le_mul_of_pos_right M (A.groups_pos r)) (A.grouped_le r)
def armVariance
Row arm variance.
Definition (Lean source)
parameters
def
armVariance A r z :
ℝ
given by
armVar (A.popSize r) M (A.groupSize_le r) (A.schedule r) z
def armCrossCov
Row ordered-disjoint covariance.
Definition (Lean source)
parameters
def
armCrossCov A r a b :
ℝ
given by
crossCov (A.popSize r) M (A.groupSize_le r) (A.schedule r) a b
def contrastCrossCov
Row ordered-disjoint arm-contrast covariance.
Definition (Lean source)
parameters
def
contrastCrossCov A r :
ℝ
given by
crossCovContrast (A.popSize r) M (A.groupSize_le r) (A.schedule r)
def leadingVariance
Row independent-group leading variance.
Definition (Lean source)
parameters
def
leadingVariance A r :
ℝ
given by
indepGroupVar (A.popSize r) M (A.groups r) (A.treated r) (A.groupSize_le r) (A.treated_pos r) (A.treated_lt r) (A.schedule r)
def degreeOne
Row degree-one arm-difference energy.
Definition (Lean source)
parameters
def
degreeOne A hMpos J r :
ℝ
given by
degreeOneEnergy (A.popSize r) M hMpos (A.groupSize_le r) (J r) (A.schedule r)
def cr2
Row scalar CR2 statistic.
Definition (Lean source)
parameters
def
cr2 A r :
PartitionTuple (A.popSize r) M (A.groups r) × TreatmentSpace (A.groups r) (A.treated r) → ℝ
given by
if h : 2 ≤ A.treated r ∧
2
≤ A.controls r then cr2Var (A.schedule r) h.1 (by simpa [ScheduleArray.controls] using h.2) else fun _ => 0
def GroupCountGrowth
The number of randomized groups tends to infinity.
Definition (Lean source)
parameters
M :
ℕ
A :
def
GroupCountGrowth A :
Prop
theorem popSize_tendsto_atTop
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
Proof (Lean source)
lemma ScheduleArray.popSize_tendsto_atTop {M : ℕ} (A : ScheduleArray M)
(hGrowth : GroupCountGrowth A) : Tendsto A.popSize atTop atTop := by
unfold GroupCountGrowth at hGrowth
rw [tendsto_atTop_atTop] at hGrowth ⊢
intro b
obtain ⟨R, hR⟩ := hGrowth b
refine ⟨R, fun r hr => (hR r hr).trans ?_⟩
have hMpos : 0 < M := lt_of_lt_of_le (by omega) A.groupSize_ge_two
exact le_trans (Nat.le_mul_of_pos_left (A.groups r) hMpos)
(A.grouped_le r)
def StableTreatmentFraction
Treatment fractions converge to an interior limit.
Definition (Lean source)
parameters
def
StableTreatmentFraction A p :
Prop
def SamplingFractionLimit
Grouped-unit fractions converge in the closed unit interval.
Definition (Lean source)
parameters
def
SamplingFractionLimit A rho :
Prop
def BoundedSchedule
The schedule is uniformly bounded over rows, groups, members, and arms.
Definition (Lean source)
parameters
def
BoundedSchedule A B :
Prop
given by
clause 1
0 < B
clause 2
r (S : Omega (A.popSize r) M) (i : {j // j ∈ S.1}) (z : Arm) :
|A.schedule r S i z| ≤ B
def ScaledVarianceNondegenerate
The liminf of the group-scaled exact variance has a positive uniform floor.
Definition (Lean source)
parameters
def
ScaledVarianceNondegenerate A cSigma :
Prop
structure DenseScheduleClass
The paper's dense bounded schedule-array class.
Definition (Lean source)
parameters
assumes
growth :
fraction :
sampling :
SamplingFractionLimit A rho
bounded :
BoundedSchedule A B
nondegenerate :
ScaledVarianceNondegenerate A cSigma
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Cr2Concentration 37 declarations
def tableSchedule
For the stated inputs, table schedule is defined by the formula below.
Definition (Lean source)
parameters
def
given by
fun S _ a => if a = z then f S else 0
theorem armTable_tableSchedule_same
Given the stated population sizes, design objects, functions, and conditions, the arm table table schedule same result holds.
Formal statement
hypotheses
conclusion
armTable n M (tableSchedule z f) z S = f S
Proof (Lean source)
lemma armTable_tableSchedule_same {n M : ℕ} (hM : 0 < M) (z : Arm)
(f : Omega n M → ℝ) (S : Omega n M) :
armTable n M (tableSchedule z f) z S = f S := by
unfold armTable tableSchedule
simp [S.2, Nat.cast_ne_zero.mpr hM.ne']
theorem armTable_tableSchedule_ne
Given the stated population sizes, design objects, functions, and conditions, the arm table table schedule ne result holds.
Formal statement
hypotheses
conclusion
armTable n M (tableSchedule z f) a S = 0
Proof (Lean source)
lemma armTable_tableSchedule_ne {n M : ℕ} (z a : Arm) (hza : a ≠ z)
(f : Omega n M → ℝ) (S : Omega n M) :
armTable n M (tableSchedule z f) a S = 0 := by
unfold armTable tableSchedule
simp [hza]
def selectedMean
For the stated inputs, selected mean is defined by the formula below.
Definition (Lean source)
parameters
def
selectedMean f w z :
ℝ
given by
if z then (∑ g ∈ w.2.1, f (w.1.1 g)) / (G1 : ℝ) else (∑ g ∈ (Finset.univ.filter fun g : Fin G => g ∉ w.2.1), f (w.1.1 g)) / ((G - G1 : ℕ) : ℝ)
theorem pameHat_tableSchedule_true
Given the stated population sizes, design objects, functions, and conditions, the pame hat table schedule true result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hM :
0 < M
hG1pos :
0 < G1
hG1lt :
G1 < G
f :
Omega n M → ℝ
w :
PartitionTuple n M G × TreatmentSpace G G1
conclusion
Proof (Lean source)
lemma pameHat_tableSchedule_true {n M G G1 : ℕ} (hM : 0 < M)
(hG1pos : 0 < G1) (hG1lt : G1 < G) (f : Omega n M → ℝ)
(w : PartitionTuple n M G × TreatmentSpace G G1) :
pameHat (tableSchedule true f) hG1pos hG1lt w = selectedMean f w true := by
simp [pameHat, selectedMean,
armTable_tableSchedule_same hM, armTable_tableSchedule_ne]
theorem pameHat_tableSchedule_false
Given the stated population sizes, design objects, functions, and conditions, the pame hat table schedule false result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hM :
0 < M
hG1pos :
0 < G1
hG1lt :
G1 < G
f :
Omega n M → ℝ
w :
PartitionTuple n M G × TreatmentSpace G G1
conclusion
Proof (Lean source)
lemma pameHat_tableSchedule_false {n M G G1 : ℕ} (hM : 0 < M)
(hG1pos : 0 < G1) (hG1lt : G1 < G) (f : Omega n M → ℝ)
(w : PartitionTuple n M G × TreatmentSpace G G1) :
pameHat (tableSchedule false f) hG1pos hG1lt w = -selectedMean f w false := by
simp [pameHat, selectedMean,
armTable_tableSchedule_same hM, armTable_tableSchedule_ne]
theorem pame_tableSchedule_true
Given the stated population sizes, design objects, functions, and conditions, the pame table schedule true result holds.
Formal statement
hypotheses
n M :
ℕ
hMpos :
0 < M
hMn :
M ≤ n
f :
Omega n M → ℝ
conclusion
Proof (Lean source)
lemma pame_tableSchedule_true {n M : ℕ} (hMpos : 0 < M) (hMn : M ≤ n)
(f : Omega n M → ℝ) :
pame n M hMn (tableSchedule true f) = (slice n M hMn).E f := by
unfold pame
rw [show armTable n M (tableSchedule true f) true = f by
funext S; exact armTable_tableSchedule_same hMpos true f S,
show armTable n M (tableSchedule true f) false = fun _ => 0 by
funext S; exact armTable_tableSchedule_ne true false (by decide) f S]
simp
theorem pame_tableSchedule_false
Given the stated population sizes, design objects, functions, and conditions, the pame table schedule false result holds.
Formal statement
hypotheses
n M :
ℕ
hMpos :
0 < M
hMn :
M ≤ n
f :
Omega n M → ℝ
conclusion
Proof (Lean source)
lemma pame_tableSchedule_false {n M : ℕ} (hMpos : 0 < M) (hMn : M ≤ n)
(f : Omega n M → ℝ) :
pame n M hMn (tableSchedule false f) = -(slice n M hMn).E f := by
unfold pame
rw [show armTable n M (tableSchedule false f) true = fun _ => 0 by
funext S; exact armTable_tableSchedule_ne false true (by decide) f S,
show armTable n M (tableSchedule false f) false = f by
funext S; exact armTable_tableSchedule_same hMpos false f S]
simp
theorem armVar_tableSchedule_same
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma armVar_tableSchedule_same {n M : ℕ} (hMpos : 0 < M) (hMn : M ≤ n)
(z : Arm) (f : Omega n M → ℝ) :
armVar n M hMn (tableSchedule z f) z = (slice n M hMn).Var f := by
unfold armVar
apply (slice n M hMn).Var_congr
exact armTable_tableSchedule_same hMpos z f
theorem armVar_tableSchedule_ne
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
armVar n M hMn (tableSchedule z f) a = 0
Proof (Lean source)
lemma armVar_tableSchedule_ne {n M : ℕ} (hMn : M ≤ n)
(z a : Arm) (hza : a ≠ z) (f : Omega n M → ℝ) :
armVar n M hMn (tableSchedule z f) a = 0 := by
unfold armVar
rw [show armTable n M (tableSchedule z f) a = fun _ => 0 by
funext S; exact armTable_tableSchedule_ne z a hza f S]
simp [FiniteDesign.Var_eq]
theorem crossCov_tableSchedule_same
Given the stated population sizes, design objects, functions, and conditions, the cross cov table schedule same result holds.
Formal statement
Proof (Lean source)
lemma crossCov_tableSchedule_same {n M : ℕ} (hMpos : 0 < M) (hMn : M ≤ n)
(z : Arm) (f : Omega n M → ℝ) :
crossCov n M hMn (tableSchedule z f) z z =
sliceInner n M hMn (fun S => f S - (slice n M hMn).E f)
(kneserOp n M (fun S => f S - (slice n M hMn).E f)) := by
unfold crossCov armTableCentered
rw [show armTable n M (tableSchedule z f) z = f by
funext S; exact armTable_tableSchedule_same hMpos z f S]
theorem crossCov_tableSchedule_left_ne
Given the stated population sizes, design objects, functions, and conditions, the cross cov table schedule left ne result holds.
Formal statement
hypotheses
conclusion
crossCov n M hMn (tableSchedule z f) a b = 0
Proof (Lean source)
lemma crossCov_tableSchedule_left_ne {n M : ℕ} (hMn : M ≤ n)
(z a b : Arm) (haz : a ≠ z) (f : Omega n M → ℝ) :
crossCov n M hMn (tableSchedule z f) a b = 0 := by
unfold crossCov armTableCentered
rw [show armTable n M (tableSchedule z f) a = fun _ => 0 by
funext S; exact armTable_tableSchedule_ne z a haz f S]
simp [sliceInner, FiniteDesign.E]
theorem crossCov_tableSchedule_right_ne
Given the stated population sizes, design objects, functions, and conditions, the cross cov table schedule right ne result holds.
Formal statement
hypotheses
conclusion
crossCov n M hMn (tableSchedule z f) a b = 0
Proof (Lean source)
lemma crossCov_tableSchedule_right_ne {n M : ℕ} (hMn : M ≤ n)
(z a b : Arm) (hbz : b ≠ z) (f : Omega n M → ℝ) :
crossCov n M hMn (tableSchedule z f) a b = 0 := by
unfold crossCov armTableCentered
rw [show armTable n M (tableSchedule z f) b = fun _ => 0 by
funext S; exact armTable_tableSchedule_ne z b hbz f S]
simp [kneserOp, kneserAdjacency, sliceInner, FiniteDesign.E]
theorem selectedMean_variance_true
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hM :
2 ≤ M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
f :
Omega n M → ℝ
conclusion
+ (1 - 1 / (G1 : ℝ)) * sliceInner n M (le_trans (le_mul_of_pos_right M (by omega)) hMG) (fun S => f S - (slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f) (kneserOp n M (fun S => f S - (slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f))
Proof (Lean source)
lemma selectedMean_variance_true {n M G G1 : ℕ} (hMG : M * G ≤ n)
(hM : 2 ≤ M) (hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1)
(f : Omega n M → ℝ) :
(randomPartitionDesign n M G G1 hMG (by omega)).Var
(fun w => selectedMean f w true) =
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).Var f / (G1 : ℝ) +
(1 - 1 / (G1 : ℝ)) *
sliceInner n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)
(fun S => f S -
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)
(kneserOp n M (fun S => f S -
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)) := by
let hG1pos : 0 < G1 := by omega
let hG1lt : G1 < G := by omega
let hMn : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
let C := sliceInner n M hMn (fun S => f S - (slice n M hMn).E f)
(kneserOp n M (fun S => f S - (slice n M hMn).E f))
have hex := (exact_pame_variance n M G G1 hMG hM hG1 hG0
(tableSchedule true f)).2.1
have hv : (randomPartitionDesign n M G G1 hMG (by omega)).Var
(pameHat (tableSchedule true f) hG1pos hG1lt) =
indepGroupVar n M G G1 hMn hG1pos hG1lt (tableSchedule true f) / (G : ℝ) +
crossCovContrast n M hMn (tableSchedule true f) -
crossCov n M hMn (tableSchedule true f) true true / (G1 : ℝ) -
crossCov n M hMn (tableSchedule true f) false false / ((G - G1 : ℕ) : ℝ) := by
simpa [sigmaSq] using hex
rw [show (randomPartitionDesign n M G G1 hMG (by omega)).Var
(fun w => selectedMean f w true) =
(randomPartitionDesign n M G G1 hMG (by omega)).Var
(pameHat (tableSchedule true f) hG1pos hG1lt) by
apply (randomPartitionDesign n M G G1 hMG (by omega)).Var_congr
intro w
exact (pameHat_tableSchedule_true (by omega) hG1pos hG1lt f w).symm,
hv]
unfold indepGroupVar crossCovContrast pFrac
rw [armVar_tableSchedule_same (by omega) hMn,
armVar_tableSchedule_ne hMn true false (by decide),
crossCov_tableSchedule_same (by omega) hMn,
crossCov_tableSchedule_left_ne hMn true false false (by decide),
crossCov_tableSchedule_right_ne hMn true true false (by decide)]
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast (by omega : G ≠ 0)
have hG1r : (G1 : ℝ) ≠ 0 := by exact_mod_cast (by omega : G1 ≠ 0)
field_simp
ring
theorem selectedMean_variance_false
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hM :
2 ≤ M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
f :
Omega n M → ℝ
conclusion
+ (1 - 1 / ((G - G1 : ℕ) : ℝ)) * sliceInner n M (le_trans (le_mul_of_pos_right M (by omega)) hMG) (fun S => f S - (slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f) (kneserOp n M (fun S => f S - (slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f))
Proof (Lean source)
lemma selectedMean_variance_false {n M G G1 : ℕ} (hMG : M * G ≤ n)
(hM : 2 ≤ M) (hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1)
(f : Omega n M → ℝ) :
(randomPartitionDesign n M G G1 hMG (by omega)).Var
(fun w => selectedMean f w false) =
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).Var f /
((G - G1 : ℕ) : ℝ) +
(1 - 1 / ((G - G1 : ℕ) : ℝ)) *
sliceInner n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)
(fun S => f S -
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)
(kneserOp n M (fun S => f S -
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)) := by
let hG1pos : 0 < G1 := by omega
let hG1lt : G1 < G := by omega
let hMn : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
have hex := (exact_pame_variance n M G G1 hMG hM hG1 hG0
(tableSchedule false f)).2.1
have hv : (randomPartitionDesign n M G G1 hMG (by omega)).Var
(pameHat (tableSchedule false f) hG1pos hG1lt) =
indepGroupVar n M G G1 hMn hG1pos hG1lt (tableSchedule false f) / (G : ℝ) +
crossCovContrast n M hMn (tableSchedule false f) -
crossCov n M hMn (tableSchedule false f) true true / (G1 : ℝ) -
crossCov n M hMn (tableSchedule false f) false false / ((G - G1 : ℕ) : ℝ) := by
simpa [sigmaSq] using hex
rw [show (randomPartitionDesign n M G G1 hMG (by omega)).Var
(fun w => selectedMean f w false) =
(randomPartitionDesign n M G G1 hMG (by omega)).Var
(pameHat (tableSchedule false f) hG1pos hG1lt) by
rw [FiniteDesign.Var_eq, FiniteDesign.Var_eq]
have hp : pameHat (tableSchedule false f) hG1pos hG1lt =
fun w => -selectedMean f w false := by
funext w
exact pameHat_tableSchedule_false (by omega) hG1pos hG1lt f w
rw [hp, (randomPartitionDesign n M G G1 hMG (by omega)).E_neg]
congr 1
· apply (randomPartitionDesign n M G G1 hMG (by omega)).E_congr
intro w
ring
· ring,
hv]
unfold indepGroupVar crossCovContrast pFrac
rw [armVar_tableSchedule_same (by omega) hMn,
armVar_tableSchedule_ne hMn false true (by decide),
crossCov_tableSchedule_same (by omega) hMn,
crossCov_tableSchedule_left_ne hMn false true true (by decide),
crossCov_tableSchedule_left_ne hMn false true false (by decide)]
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast (by omega : G ≠ 0)
have hG0r : ((G - G1 : ℕ) : ℝ) ≠ 0 := by exact_mod_cast (by omega : G - G1 ≠ 0)
rw [Nat.cast_sub (by omega : G1 ≤ G)]
field_simp
ring
theorem treated_tendsto_atTop
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
A :
p :
ℝ
hGrowth :
hFraction :
Proof (Lean source)
lemma treated_tendsto_atTop {M : ℕ} (A : ScheduleArray M) (p : ℝ)
(hGrowth : GroupCountGrowth A) (hFraction : StableTreatmentFraction A p) :
Tendsto A.treated atTop atTop := by
rw [← tendsto_natCast_atTop_iff (R := ℝ)]
have hG : Tendsto (fun r => (A.groups r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hGrowth
have hprod := hFraction.2.2.pos_mul_atTop hFraction.1 hG
apply Tendsto.congr' _ hprod
filter_upwards [] with r
unfold ScheduleArray.treatmentFraction pFrac
have hGr : (A.groups r : ℝ) ≠ 0 := by exact_mod_cast (A.groups_pos r).ne'
field_simp
theorem controls_tendsto_atTop
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
A :
p :
ℝ
hGrowth :
hFraction :
Proof (Lean source)
lemma controls_tendsto_atTop {M : ℕ} (A : ScheduleArray M) (p : ℝ)
(hGrowth : GroupCountGrowth A) (hFraction : StableTreatmentFraction A p) :
Tendsto A.controls atTop atTop := by
rw [← tendsto_natCast_atTop_iff (R := ℝ)]
have hG : Tendsto (fun r => (A.groups r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hGrowth
have hone : Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1) := tendsto_const_nhds
have hden : Tendsto (fun r => 1 - A.treatmentFraction r) atTop (nhds (1 - p)) :=
hone.sub hFraction.2.2
have hprod := hden.pos_mul_atTop (by linarith [hFraction.2.1]) hG
apply Tendsto.congr' _ hprod
filter_upwards [] with r
unfold ScheduleArray.controls ScheduleArray.treatmentFraction pFrac
rw [Nat.cast_sub (A.treated_le r)]
have hGr : (A.groups r : ℝ) ≠ 0 := by exact_mod_cast (A.groups_pos r).ne'
field_simp
def withTable
For the stated inputs, with table is defined by the formula below.
Definition (Lean source)
parameters
def
given by
clause 1
groupSize_ge_two := A.groupSize_ge_two
clause 2
popSize := A.popSize
clause 3
groups := A.groups
clause 4
treated := A.treated
clause 5
groups_pos := A.groups_pos
clause 6
grouped_le := A.grouped_le
clause 7
treated_pos := A.treated_pos
clause 8
treated_lt := A.treated_lt
clause 9
treated_le := A.treated_le
clause 10
schedule := fun r => tableSchedule z (f r)
theorem withTable_bounded
Given the stated population sizes, design objects, functions, and conditions, the with table bounded result holds.
Formal statement
hypotheses
M :
ℕ
A :
z :
f :
∀ r
if
Omega (A.popSize r) M
then
ℝ
K :
ℝ
hK :
0 < K
hf :
∀ r S, |f r S| ≤ K
conclusion
BoundedSchedule (A.withTable z f) K
Proof (Lean source)
lemma withTable_bounded {M : ℕ} (A : ScheduleArray M) (z : Arm)
(f : ∀ r, Omega (A.popSize r) M → ℝ) (K : ℝ) (hK : 0 < K)
(hf : ∀ r S, |f r S| ≤ K) : BoundedSchedule (A.withTable z f) K := by
refine ⟨hK, ?_⟩
intro r S i a
change |if a = z then f r S else 0| ≤ K
split_ifs
· exact hf r S
· simpa using hK.le
theorem sliceVar_le_sq_of_abs_le
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma sliceVar_le_sq_of_abs_le {n M : ℕ} (hMn : M ≤ n)
(f : Omega n M → ℝ) (K : ℝ) (hf : ∀ S, |f S| ≤ K) :
(slice n M hMn).Var f ≤ K ^ 2 := by
rw [FiniteDesign.Var_eq]
calc
(slice n M hMn).E (fun S => f S ^ 2) - ((slice n M hMn).E f) ^ 2 ≤
(slice n M hMn).E (fun S => f S ^ 2) := sub_le_self _ (sq_nonneg _)
_ = sliceInner n M hMn f f := by
unfold sliceInner
apply (slice n M hMn).E_congr
intro S
ring
_ ≤ K ^ 2 := sliceInner_self_le_sq_of_abs_le hMn f K hf
theorem selectedMean_expectation_true
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hM :
2 ≤ M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
f :
Omega n M → ℝ
conclusion
Proof (Lean source)
lemma selectedMean_expectation_true {n M G G1 : ℕ} (hMG : M * G ≤ n)
(hM : 2 ≤ M) (hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1)
(f : Omega n M → ℝ) :
(randomPartitionDesign n M G G1 hMG (by omega)).E
(fun w => selectedMean f w true) =
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f := by
let hG1pos : 0 < G1 := by omega
let hG1lt : G1 < G := by omega
have hex := (exact_pame_variance n M G G1 hMG hM hG1 hG0
(tableSchedule true f)).1
rw [show (fun w => selectedMean f w true) =
pameHat (tableSchedule true f) hG1pos hG1lt by
funext w; exact (pameHat_tableSchedule_true (by omega) hG1pos hG1lt f w).symm,
hex]
exact pame_tableSchedule_true (by omega)
(le_trans (le_mul_of_pos_right M (by omega)) hMG) f
theorem selectedMean_expectation_false
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hM :
2 ≤ M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
f :
Omega n M → ℝ
conclusion
Proof (Lean source)
lemma selectedMean_expectation_false {n M G G1 : ℕ} (hMG : M * G ≤ n)
(hM : 2 ≤ M) (hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1)
(f : Omega n M → ℝ) :
(randomPartitionDesign n M G G1 hMG (by omega)).E
(fun w => selectedMean f w false) =
(slice n M (le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f := by
let hG1pos : 0 < G1 := by omega
let hG1lt : G1 < G := by omega
have hex := (exact_pame_variance n M G G1 hMG hM hG1 hG0
(tableSchedule false f)).1
rw [pame_tableSchedule_false (by omega)
(le_trans (le_mul_of_pos_right M (by omega)) hMG) f] at hex
have hp : pameHat (tableSchedule false f) hG1pos hG1lt =
fun w => -selectedMean f w false := by
funext w
exact pameHat_tableSchedule_false (by omega) hG1pos hG1lt f w
rw [hp, (randomPartitionDesign n M G G1 hMG (by omega)).E_neg] at hex
linarith
theorem selectedMean_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p :
ℝ
J :
∀ r, JohnsonProjections (A.popSize r) M
hGrowth :
hFraction :
hJohnson :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
z :
f :
∀ r
if
Omega (A.popSize r) M
then
ℝ
K :
ℝ
hK :
0 < K
hf :
∀ r S, |f r S| ≤ K
conclusion
FiniteDesign.TendstoInProb A.design (fun r w => selectedMean (f r) w z) (fun r => (slice (A.popSize r) M (A.groupSize_le r)).E (f r))
Proof (Lean source)
lemma selectedMean_tendstoInProb {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p : ℝ) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hGrowth : GroupCountGrowth A) (hFraction : StableTreatmentFraction A p)
(hJohnson : ∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : ∀ r, KneserAdjacencySpectrum (A.popSize r) M)
(z : Arm) (f : ∀ r, Omega (A.popSize r) M → ℝ) (K : ℝ) (hK : 0 < K)
(hf : ∀ r S, |f r S| ≤ K) :
FiniteDesign.TendstoInProb A.design
(fun r w => selectedMean (f r) w z)
(fun r => (slice (A.popSize r) M (A.groupSize_le r)).E (f r)) := by
have hpop : Tendsto A.popSize atTop atTop := by
apply tendsto_atTop_mono (fun r => ?_) hGrowth
exact le_trans (Nat.le_mul_of_pos_left (A.groups r) (by omega : 0 < M))
(A.grouped_le r)
have hpopR : Tendsto (fun r => (A.popSize r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hpop
have hinvPop : Tendsto (fun r => ((A.popSize r : ℝ))⁻¹) atTop (nhds 0) :=
hpopR.inv_tendsto_atTop
have h4 : ∀ᶠ r in atTop, 4 * M ≤ A.popSize r :=
(tendsto_atTop.1 hpop) (4 * M)
let A' := A.withTable z f
have hA'bounded : BoundedSchedule A' K := withTable_bounded A z f K hK hf
have hcross : Tendsto (fun r =>
sliceInner (A.popSize r) M (A.groupSize_le r)
(fun S => f r S - (slice (A.popSize r) M (A.groupSize_le r)).E (f r))
(kneserOp (A.popSize r) M
(fun S => f r S - (slice (A.popSize r) M (A.groupSize_le r)).E (f r))))
atTop (nhds 0) := by
rw [tendsto_zero_iff_abs_tendsto_zero]
refine squeeze_zero' (g := fun r =>
(M + 1 : ℝ) * (((2 * M : ℕ) : ℝ) / (A.popSize r : ℝ)) * (2 * K) ^ 2)
(Filter.Eventually.of_forall fun r => abs_nonneg _) ?_ ?_
· filter_upwards [h4] with r hr
have hb := armCrossCov_abs_le_ge_four hM A' J K hA'bounded r hr
(hJohnson r) (hKneser r) z
dsimp [A', ScheduleArray.withTable, ScheduleArray.armCrossCov] at hb
rw [crossCov_tableSchedule_same (by omega) (A.groupSize_le r)] at hb
exact hb
· have hc : Tendsto (fun _ : ℕ =>
(M + 1 : ℝ) * ((2 * M : ℕ) : ℝ) * (2 * K) ^ 2) atTop
(nhds ((M + 1 : ℝ) * ((2 * M : ℕ) : ℝ) * (2 * K) ^ 2)) :=
tendsto_const_nhds
convert hc.mul hinvPop using 1
· funext r
simp only [div_eq_mul_inv]
ring
· norm_num
have htreat := treated_tendsto_atTop A p hGrowth hFraction
have hcontrol := controls_tendsto_atTop A p hGrowth hFraction
have htreatInv : Tendsto (fun r => ((A.treated r : ℝ))⁻¹) atTop (nhds 0) :=
(tendsto_natCast_atTop_atTop.comp htreat).inv_tendsto_atTop
have hcontrolInv : Tendsto (fun r => ((A.controls r : ℝ))⁻¹) atTop (nhds 0) :=
(tendsto_natCast_atTop_atTop.comp hcontrol).inv_tendsto_atTop
have hboth : ∀ᶠ r in atTop, 2 ≤ A.treated r ∧ 2 ≤ A.controls r := by
filter_upwards [(tendsto_atTop.1 htreat) 2, (tendsto_atTop.1 hcontrol) 2] with r ht hc
exact ⟨ht, hc⟩
have hvar : Tendsto (fun r => (A.design r).Var
(fun w => selectedMean (f r) w z)) atTop (nhds 0) := by
have harmInv : Tendsto (fun r => ((armCount (A.groups r) (A.treated r) z : ℕ) : ℝ)⁻¹)
atTop (nhds 0) := by
cases z <;> simp only [armCount]
· simpa [ScheduleArray.controls] using hcontrolInv
· exact htreatInv
have hvarPart : Tendsto (fun r =>
(slice (A.popSize r) M (A.groupSize_le r)).Var (f r) /
(armCount (A.groups r) (A.treated r) z : ℝ)) atTop (nhds 0) := by
rw [tendsto_zero_iff_abs_tendsto_zero]
refine squeeze_zero (g := fun r => K ^ 2 *
((armCount (A.groups r) (A.treated r) z : ℝ))⁻¹)
(fun r => abs_nonneg _) (fun r => ?_) ?_
· have hcount : 0 < armCount (A.groups r) (A.treated r) z := by
cases z
· change 0 < A.groups r - A.treated r
exact Nat.sub_pos_of_lt (A.treated_lt r)
· simp [armCount, A.treated_pos r]
change |(slice (A.popSize r) M (A.groupSize_le r)).Var (f r) /
(armCount (A.groups r) (A.treated r) z : ℝ)| ≤ _
rw [abs_of_nonneg (div_nonneg ((slice (A.popSize r) M
(A.groupSize_le r)).Var_nonneg _) (by positivity)), div_eq_mul_inv]
exact mul_le_mul_of_nonneg_right
(sliceVar_le_sq_of_abs_le (A.groupSize_le r) (f r) K (hf r)) (by positivity)
· have hc : Tendsto (fun _ : ℕ => K ^ 2) atTop (nhds (K ^ 2)) :=
tendsto_const_nhds
simpa using hc.mul harmInv
have hfactor : Tendsto (fun r =>
1 - 1 / (armCount (A.groups r) (A.treated r) z : ℝ)) atTop (nhds 1) := by
simpa [div_eq_mul_inv] using tendsto_const_nhds.sub harmInv
have hformula : ∀ᶠ r in atTop,
(A.design r).Var (fun w => selectedMean (f r) w z) =
(slice (A.popSize r) M (A.groupSize_le r)).Var (f r) /
(armCount (A.groups r) (A.treated r) z : ℝ) +
(1 - 1 / (armCount (A.groups r) (A.treated r) z : ℝ)) *
sliceInner (A.popSize r) M (A.groupSize_le r)
(fun S => f r S - (slice (A.popSize r) M (A.groupSize_le r)).E (f r))
(kneserOp (A.popSize r) M
(fun S => f r S -
(slice (A.popSize r) M (A.groupSize_le r)).E (f r))) := by
filter_upwards [hboth] with r hr
unfold ScheduleArray.design
cases z
· exact selectedMean_variance_false (A.grouped_le r) hM hr.1
(by simpa [ScheduleArray.controls] using hr.2) (f r)
· exact selectedMean_variance_true (A.grouped_le r) hM hr.1
(by simpa [ScheduleArray.controls] using hr.2) (f r)
apply Tendsto.congr' (hformula.mono fun _ hr => hr.symm)
simpa using hvarPart.add (hfactor.mul hcross)
have hbase := FiniteDesign.tendstoInProb_of_var A.design
(fun r w => selectedMean (f r) w z) hvar
intro ε hε
have htail := hbase ε hε
apply Tendsto.congr' _ htail
filter_upwards [hboth] with r hr
apply (A.design r).Pr_congr
intro w
cases z
· simp only [show (A.design r).E (fun w => selectedMean (f r) w false) =
(slice (A.popSize r) M (A.groupSize_le r)).E (f r) by
exact selectedMean_expectation_false (A.grouped_le r) hM hr.1
(by simpa [ScheduleArray.controls] using hr.2) (f r)]
· simp only [show (A.design r).E (fun w => selectedMean (f r) w true) =
(slice (A.popSize r) M (A.groupSize_le r)).E (f r) by
exact selectedMean_expectation_true (A.grouped_le r) hM hr.1
(by simpa [ScheduleArray.controls] using hr.2) (f r)]
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma sampleVariance_finset_eq {α : Type*} [Fintype α] (s : Finset α) (m : ℕ)
(hcard : s.card = m) (hm : 2 ≤ m) (x : α → ℝ) :
(∑ i ∈ s, (x i - (∑ j ∈ s, x j) / (m : ℝ)) ^ 2) / ((m - 1 : ℕ) : ℝ) =
(m : ℝ) / ((m - 1 : ℕ) : ℝ) *
((∑ i ∈ s, x i ^ 2) / (m : ℝ) - ((∑ j ∈ s, x j) / (m : ℝ)) ^ 2) := by
have hmR : (m : ℝ) ≠ 0 := by exact_mod_cast (by omega : m ≠ 0)
have hm1R : ((m - 1 : ℕ) : ℝ) ≠ 0 := by exact_mod_cast (by omega : m - 1 ≠ 0)
simp_rw [sub_sq]
rw [Finset.sum_add_distrib, Finset.sum_sub_distrib]
have hlin : (∑ i ∈ s, 2 * x i * ((∑ j ∈ s, x j) / (m : ℝ))) =
2 * (∑ i ∈ s, x i) * ((∑ j ∈ s, x j) / (m : ℝ)) := by
rw [← Finset.sum_mul, ← Finset.mul_sum]
rw [hlin]
simp only [Finset.sum_const, nsmul_eq_mul, hcard]
field_simp
ring
Given the stated population sizes, design objects, functions, and conditions, the stated cardinality formula holds.
Formal statement
hypotheses
conclusion
(realizedArmSet w z).card = armCount G G1 z
Proof (Lean source)
lemma realizedArmSet_card {n M G G1 : ℕ}
(w : PartitionTuple n M G × TreatmentSpace G G1) (z : Arm) :
(realizedArmSet w z).card = armCount G G1 z := by
cases z
· change (univ \ w.2.1).card = G - G1
rw [Finset.card_sdiff]
simp [w.2.2]
· exact w.2.2
theorem selectedMean_eq_realized
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
conclusion
selectedMean f w z = (∑ i ∈ realizedArmSet w z, f (w.1.1 i)) / (armCount G G1 z : ℝ)
Proof (Lean source)
lemma selectedMean_eq_realized {n M G G1 : ℕ} (f : Omega n M → ℝ)
(w : PartitionTuple n M G × TreatmentSpace G G1) (z : Arm) :
selectedMean f w z =
(∑ i ∈ realizedArmSet w z, f (w.1.1 i)) / (armCount G G1 z : ℝ) := by
cases z
· unfold selectedMean realizedArmSet armCount
simp only [Bool.false_eq_true, ↓reduceIte]
congr 1
apply Finset.sum_congr
· ext i
simp
· intro i hi
rfl
· rfl
theorem selectedMean_abs_le
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
n M G G1 :
ℕ
f :
Omega n M → ℝ
w :
PartitionTuple n M G × TreatmentSpace G G1
z :
K :
ℝ
hf :
∀ S, |f S| ≤ K
hcount :
0 < armCount G G1 z
conclusion
|selectedMean f w z| ≤ K
Proof (Lean source)
lemma selectedMean_abs_le {n M G G1 : ℕ} (f : Omega n M → ℝ)
(w : PartitionTuple n M G × TreatmentSpace G G1) (z : Arm)
(K : ℝ) (hf : ∀ S, |f S| ≤ K) (hcount : 0 < armCount G G1 z) :
|selectedMean f w z| ≤ K := by
rw [selectedMean_eq_realized, abs_div, abs_of_nonneg (by positivity :
0 ≤ (armCount G G1 z : ℝ))]
apply (div_le_iff₀ (by positivity : (0 : ℝ) < armCount G G1 z)).2
calc
|∑ i ∈ realizedArmSet w z, f (w.1.1 i)| ≤
∑ i ∈ realizedArmSet w z, |f (w.1.1 i)| := Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _i ∈ realizedArmSet w z, K := by
gcongr with i hi
exact hf _
_ = (armCount G G1 z : ℝ) * K := by
rw [Finset.sum_const, nsmul_eq_mul, realizedArmSet_card]
_ = K * (armCount G G1 z : ℝ) := by ring
theorem armSampleVar_eq_selectedMoments
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
Y :
PotentialOutcome n M
w :
PartitionTuple n M G × TreatmentSpace G G1
z :
hArm :
2 ≤ armCount G G1 z
conclusion
armSampleVar Y w z hArm
= (armCount G G1 z : ℝ) / ((armCount G G1 z - 1 : ℕ) : ℝ) * (selectedMean (fun S => armTable n M Y z S ^ 2) w z - selectedMean (armTable n M Y z) w z ^ 2)
Proof (Lean source)
lemma armSampleVar_eq_selectedMoments {n M G G1 : ℕ} (Y : PotentialOutcome n M)
(w : PartitionTuple n M G × TreatmentSpace G G1) (z : Arm)
(hArm : 2 ≤ armCount G G1 z) :
armSampleVar Y w z hArm =
(armCount G G1 z : ℝ) / ((armCount G G1 z - 1 : ℕ) : ℝ) *
(selectedMean (fun S => armTable n M Y z S ^ 2) w z -
selectedMean (armTable n M Y z) w z ^ 2) := by
have hselected (g : Omega n M → ℝ) : selectedMean g w z =
(∑ i ∈ realizedArmSet w z, g (w.1.1 i)) / (armCount G G1 z : ℝ) :=
selectedMean_eq_realized g w z
have hobs (g : Fin G) (hg : g ∈ realizedArmSet w z) :
obsGroupMean Y w g = armTable n M Y z (w.1.1 g) := by
cases z
· simp only [realizedArmSet, Bool.false_eq_true, ↓reduceIte,
mem_sdiff, Finset.mem_univ, true_and] at hg
simp [obsGroupMean, hg]
· simp only [realizedArmSet, ↓reduceIte] at hg
simp [obsGroupMean, hg]
have hmean : armObsMean Y w z = selectedMean (armTable n M Y z) w z := by
unfold armObsMean
rw [hselected]
congr 1
apply Finset.sum_congr rfl
intro g hg
exact hobs g hg
unfold armSampleVar
rw [hmean]
have hcard := realizedArmSet_card w z
have halg := sampleVariance_finset_eq (realizedArmSet w z) (armCount G G1 z)
hcard hArm (fun g => armTable n M Y z (w.1.1 g))
have hsum : (∑ g ∈ realizedArmSet w z,
(obsGroupMean Y w g - selectedMean (armTable n M Y z) w z) ^ 2) =
∑ g ∈ realizedArmSet w z,
(armTable n M Y z (w.1.1 g) -
(∑ i ∈ realizedArmSet w z, armTable n M Y z (w.1.1 i)) /
(armCount G G1 z : ℝ)) ^ 2 := by
apply Finset.sum_congr rfl
intro g hg
rw [hobs g hg, hselected]
rw [hsum, halg, hselected, hselected]
Given the stated population sizes, design objects, functions, and conditions, the bounded in prob of pointwise bound result holds.
Formal statement
hypotheses
Ω :
ℕ → Type*
∀ r, Fintype (Ω r)
D :
∀ r, FiniteDesign (Ω r)
X :
∀ r
if
Ω r
then
ℝ
K :
ℝ
hX :
∀ r w, |X r w| ≤ K
conclusion
FiniteDesign.BoundedInProb D X
Proof (Lean source)
lemma boundedInProb_of_pointwise_bound {Ω : ℕ → Type*} [∀ r, Fintype (Ω r)]
(D : ∀ r, FiniteDesign (Ω r)) (X : ∀ r, Ω r → ℝ) (K : ℝ)
(hX : ∀ r w, |X r w| ≤ K) : FiniteDesign.BoundedInProb D X := by
intro η hη
refine ⟨K + 1, ?_⟩
filter_upwards [] with r
have hz : (D r).Pr (fun w => K + 1 ≤ |X r w|) = 0 := by
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
apply Finset.sum_eq_zero
intro w hw
simp [not_le.mpr (lt_of_le_of_lt (hX r w) (lt_add_one K))]
rw [hz]
exact hη.le
Given the stated population sizes, design objects, functions, and conditions, the square of bounded result holds.
Formal statement
hypotheses
Ω :
X :
∀ r
if
Ω r
then
ℝ
a :
ℕ → ℝ
K :
ℝ
hXbound :
∀ r w, |X r w| ≤ K
habound :
∀ r, |a r| ≤ K
hX :
FiniteDesign.TendstoInProb D X a
conclusion
FiniteDesign.TendstoInProb D (fun r w => X r w ^ 2) (fun r => a r ^ 2)
Proof (Lean source)
lemma FiniteDesign.TendstoInProb.square_of_bounded
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)]
{D : ∀ r, FiniteDesign (Ω r)} {X : ∀ r, Ω r → ℝ} {a : ℕ → ℝ}
(K : ℝ) (hXbound : ∀ r w, |X r w| ≤ K) (habound : ∀ r, |a r| ≤ K)
(hX : FiniteDesign.TendstoInProb D X a) :
FiniteDesign.TendstoInProb D (fun r w => X r w ^ 2) (fun r => a r ^ 2) := by
have hcenter : FiniteDesign.TendstoInProb D (fun r w => X r w - a r) (fun _ => 0) := by
intro ε hε
simpa only [sub_zero, sub_zero] using hX ε hε
have hsumBound : ∀ r w, |X r w + a r| ≤ 2 * K := by
intro r w
calc
|X r w + a r| ≤ |X r w| + |a r| := abs_add_le _ _
_ ≤ K + K := add_le_add (hXbound r w) (habound r)
_ = 2 * K := by ring
have hbounded := boundedInProb_of_pointwise_bound D (fun r w => X r w + a r)
(2 * K) hsumBound
have hprod := hcenter.mul_boundedInProb hbounded
intro ε hε
have ht := hprod ε hε
change Tendsto (fun m => (D m).Pr (fun z => ε ≤ |X m z ^ 2 - a m ^ 2|))
atTop (nhds 0)
apply Tendsto.congr' _ ht
filter_upwards [] with m
apply (D m).Pr_congr
intro w
constructor <;> intro hh
· convert hh using 1; ring
· convert hh using 1; ring
theorem scale_tendsto_one_of_bounded Lemma FiniteDesign.TendstoInProb.scale_tendsto_one_of_bounded in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
Ω :
X :
∀ r
if
Ω r
then
ℝ
a c :
ℕ → ℝ
K :
ℝ
hXbound :
∀ r w, |X r w| ≤ K
hX :
FiniteDesign.TendstoInProb D X a
conclusion
FiniteDesign.TendstoInProb D (fun r w => c r * X r w) a
Proof (Lean source)
lemma FiniteDesign.TendstoInProb.scale_tendsto_one_of_bounded
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)]
{D : ∀ r, FiniteDesign (Ω r)} {X : ∀ r, Ω r → ℝ} {a c : ℕ → ℝ}
(K : ℝ) (hXbound : ∀ r w, |X r w| ≤ K)
(hX : FiniteDesign.TendstoInProb D X a) (hc : Tendsto c atTop (nhds 1)) :
FiniteDesign.TendstoInProb D (fun r w => c r * X r w) a := by
have hcenter : FiniteDesign.TendstoInProb D (fun r w => X r w - a r) (fun _ => 0) := by
intro ε hε
simpa only [sub_zero, sub_zero] using hX ε hε
have hcProb := FiniteDesign.deterministic_tendstoInProb D c 1 hc
have hc0 : FiniteDesign.TendstoInProb D (fun r _ => c r - 1) (fun _ => 0) := by
intro ε hε
simpa only [sub_zero, sub_zero] using hcProb ε hε
have hbounded := boundedInProb_of_pointwise_bound D X K hXbound
have hprod := hc0.mul_boundedInProb hbounded
have hadd := hprod.add hcenter
intro ε hε
have ht := hadd ε hε
change Tendsto (fun m => (D m).Pr (fun z => ε ≤ |c m * X m z - a m|))
atTop (nhds 0)
apply Tendsto.congr' _ ht
filter_upwards [] with m
apply (D m).Pr_congr
intro w
constructor <;> intro hh
· convert hh using 1; ring
· convert hh using 1; ring
theorem armSampleVar_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p B :
ℝ
J :
∀ r, JohnsonProjections (A.popSize r) M
hGrowth :
hFraction :
hBounded :
BoundedSchedule A B
hJohnson :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
z :
conclusion
FiniteDesign.TendstoInProb A.design (fun r w => if h : 2 ≤ armCount (A.groups r) (A.treated r) z then armSampleVar (A.schedule r) w z h else 0) (fun r => A.armVariance r z)
Proof (Lean source)
lemma armSampleVar_tendstoInProb {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p B : ℝ) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hGrowth : GroupCountGrowth A) (hFraction : StableTreatmentFraction A p)
(hBounded : BoundedSchedule A B)
(hJohnson : ∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : ∀ r, KneserAdjacencySpectrum (A.popSize r) M) (z : Arm) :
FiniteDesign.TendstoInProb A.design
(fun r w => if h : 2 ≤ armCount (A.groups r) (A.treated r) z then
armSampleVar (A.schedule r) w z h else 0)
(fun r => A.armVariance r z) := by
let f : ∀ r, Omega (A.popSize r) M → ℝ := fun r S =>
armTable (A.popSize r) M (A.schedule r) z S
have hB : 0 < B := hBounded.1
have hf : ∀ r S, |f r S| ≤ B := fun r S =>
armTable_abs_le_of_boundedSchedule A B hBounded r z S
have hfsq : ∀ r S, |(f r S) ^ 2| ≤ B ^ 2 := by
intro r S
rw [abs_pow]
exact pow_le_pow_left₀ (abs_nonneg _) (hf r S) 2
let μ : ℕ → ℝ := fun r => (slice (A.popSize r) M (A.groupSize_le r)).E (f r)
let μ2 : ℕ → ℝ := fun r =>
(slice (A.popSize r) M (A.groupSize_le r)).E (fun S => (f r S) ^ 2)
have hmean := selectedMean_tendstoInProb hM A p J hGrowth hFraction hJohnson hKneser
z f B hB hf
have hsecond := selectedMean_tendstoInProb hM A p J hGrowth hFraction hJohnson hKneser
z (fun r S => (f r S) ^ 2) (B ^ 2) (sq_pos_of_pos hB) hfsq
have hcount : ∀ r, 0 < armCount (A.groups r) (A.treated r) z := by
intro r
cases z
· change 0 < A.groups r - A.treated r
exact Nat.sub_pos_of_lt (A.treated_lt r)
· simpa [armCount] using A.treated_pos r
have hmeanBound : ∀ r w, |selectedMean (f r) w z| ≤ B := fun r w =>
selectedMean_abs_le (f r) w z B (hf r) (hcount r)
have hμBound : ∀ r, |μ r| ≤ B := fun r =>
sliceExpectation_abs_le_of_abs_le (A.groupSize_le r) (f r) B (hf r)
have hmeanSq := FiniteDesign.TendstoInProb.square_of_bounded B hmeanBound hμBound hmean
have hbracket := hsecond.sub hmeanSq
have hbracketBound : ∀ r
(w : PartitionTuple (A.popSize r) M (A.groups r) ×
TreatmentSpace (A.groups r) (A.treated r)),
|selectedMean (fun S => (f r S) ^ 2) w z - selectedMean (f r) w z ^ 2| ≤
2 * B ^ 2 := by
intro r w
have h2 := selectedMean_abs_le (fun S => (f r S) ^ 2) w z (B ^ 2)
(hfsq r) (hcount r)
have h1 := hmeanBound r w
calc
|selectedMean (fun S => (f r S) ^ 2) w z - selectedMean (f r) w z ^ 2| ≤
|selectedMean (fun S => (f r S) ^ 2) w z| +
|selectedMean (f r) w z ^ 2| := abs_sub _ _
_ ≤ B ^ 2 + B ^ 2 := by
gcongr
rw [abs_pow]
exact pow_le_pow_left₀ (abs_nonneg _) h1 2
_ = 2 * B ^ 2 := by ring
have harm : Tendsto (fun r => armCount (A.groups r) (A.treated r) z) atTop atTop := by
cases z
· change Tendsto A.controls atTop atTop
exact controls_tendsto_atTop A p hGrowth hFraction
· simpa [armCount] using treated_tendsto_atTop A p hGrowth hFraction
have harmInv : Tendsto (fun r => ((armCount (A.groups r) (A.treated r) z : ℕ) : ℝ)⁻¹)
atTop (nhds 0) := (tendsto_natCast_atTop_atTop.comp harm).inv_tendsto_atTop
have hfactor : Tendsto (fun r =>
(armCount (A.groups r) (A.treated r) z : ℝ) /
((armCount (A.groups r) (A.treated r) z - 1 : ℕ) : ℝ)) atTop (nhds 1) := by
have hden : Tendsto (fun r => 1 -
((armCount (A.groups r) (A.treated r) z : ℝ))⁻¹) atTop (nhds 1) := by
simpa using tendsto_const_nhds.sub harmInv
have hone : Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1) := tendsto_const_nhds
have hquot := hone.div hden (by norm_num : (1 : ℝ) ≠ 0)
have hlarge : ∀ᶠ r in atTop, 2 ≤ armCount (A.groups r) (A.treated r) z :=
(tendsto_atTop.1 harm) 2
have heq : ∀ᶠ r in atTop,
(armCount (A.groups r) (A.treated r) z : ℝ) /
((armCount (A.groups r) (A.treated r) z - 1 : ℕ) : ℝ) =
1 / (1 - ((armCount (A.groups r) (A.treated r) z : ℝ))⁻¹) := by
filter_upwards [hlarge] with r hr
rw [Nat.cast_sub (by omega : 1 ≤ armCount (A.groups r) (A.treated r) z)]
have hm : (armCount (A.groups r) (A.treated r) z : ℝ) ≠ 0 := by positivity
field_simp
norm_num
simpa using Tendsto.congr' (heq.mono fun _ hh => hh.symm) hquot
have hscaled := FiniteDesign.TendstoInProb.scale_tendsto_one_of_bounded
(2 * B ^ 2) hbracketBound hbracket hfactor
have hlarge : ∀ᶠ r in atTop, 2 ≤ armCount (A.groups r) (A.treated r) z :=
(tendsto_atTop.1 harm) 2
intro ε hε
have ht := hscaled ε hε
apply Tendsto.congr' _ ht
filter_upwards [hlarge] with r hr
apply (A.design r).Pr_congr
intro w
have hsample := armSampleVar_eq_selectedMoments (A.schedule r) w z hr
have htarget : A.armVariance r z = μ2 r - μ r ^ 2 := by
unfold ScheduleArray.armVariance armVar μ2 μ f
rw [FiniteDesign.Var_eq]
simp only [dif_pos hr]
rw [hsample, htarget]
theorem boundedInProb_deterministic_of_tendsto Lemma boundedInProb_deterministic_of_tendsto in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.BoundedInProb D (fun r _ => a r)
Proof (Lean source)
lemma boundedInProb_deterministic_of_tendsto {Ω : ℕ → Type*} [∀ r, Fintype (Ω r)]
(D : ∀ r, FiniteDesign (Ω r)) (a : ℕ → ℝ) (c : ℝ)
(ha : Tendsto a atTop (nhds c)) :
FiniteDesign.BoundedInProb D (fun r _ => a r) := by
intro η hη
refine ⟨|c| + 2, ?_⟩
have hev : ∀ᶠ r in atTop, |a r| < |c| + 1 := by
have hopen : Iio (|c| + 1) ∈ nhds |c| := Iio_mem_nhds (lt_add_one _)
exact (ha.norm.eventually hopen)
filter_upwards [hev] with r hr
have hz : (D r).Pr (fun _ => |c| + 2 ≤ |a r|) = 0 := by
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
apply Finset.sum_eq_zero
intro w hw
rw [if_neg, mul_zero]
exact not_le.mpr (hr.trans (by linarith))
rw [hz]
exact hη.le
theorem cr2_centered_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p B :
ℝ
J :
∀ r, JohnsonProjections (A.popSize r) M
hGrowth :
hFraction :
hBounded :
BoundedSchedule A B
hJohnson :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
conclusion
FiniteDesign.TendstoInProb A.design (fun r w => (A.groups r : ℝ) * A.cr2 r w - A.leadingVariance r) (fun _ => 0)
Proof (Lean source)
lemma cr2_centered_tendstoInProb {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p B : ℝ) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hGrowth : GroupCountGrowth A) (hFraction : StableTreatmentFraction A p)
(hBounded : BoundedSchedule A B)
(hJohnson : ∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : ∀ r, KneserAdjacencySpectrum (A.popSize r) M) :
FiniteDesign.TendstoInProb A.design
(fun r w => (A.groups r : ℝ) * A.cr2 r w - A.leadingVariance r)
(fun _ => 0) := by
let s : Arm → ∀ r,
PartitionTuple (A.popSize r) M (A.groups r) ×
TreatmentSpace (A.groups r) (A.treated r) → ℝ := fun z r w =>
if h : 2 ≤ armCount (A.groups r) (A.treated r) z then
armSampleVar (A.schedule r) w z h else 0
have hs1 := armSampleVar_tendstoInProb hM A p B J hGrowth hFraction hBounded
hJohnson hKneser true
have hs0 := armSampleVar_tendstoInProb hM A p B J hGrowth hFraction hBounded
hJohnson hKneser false
have he1 : FiniteDesign.TendstoInProb A.design
(fun r w => s true r w - A.armVariance r true) (fun _ => 0) := by
intro ε hε
simpa only [s, sub_zero] using hs1 ε hε
have he0 : FiniteDesign.TendstoInProb A.design
(fun r w => s false r w - A.armVariance r false) (fun _ => 0) := by
intro ε hε
simpa only [s, sub_zero] using hs0 ε hε
have hpInv : Tendsto (fun r => (A.treatmentFraction r)⁻¹) atTop (nhds p⁻¹) :=
hFraction.2.2.inv₀ hFraction.1.ne'
have hone : Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1) := tendsto_const_nhds
have hq : Tendsto (fun r => 1 - A.treatmentFraction r) atTop (nhds (1 - p)) :=
hone.sub hFraction.2.2
have hqInv : Tendsto (fun r => (1 - A.treatmentFraction r)⁻¹) atTop
(nhds (1 - p)⁻¹) := hq.inv₀ (by linarith [hFraction.2.1])
have hb1 := boundedInProb_deterministic_of_tendsto A.design
(fun r => (A.treatmentFraction r)⁻¹) p⁻¹ hpInv
have hb0 := boundedInProb_deterministic_of_tendsto A.design
(fun r => (1 - A.treatmentFraction r)⁻¹) (1 - p)⁻¹ hqInv
have hprod1 := he1.mul_boundedInProb hb1
have hprod0 := he0.mul_boundedInProb hb0
have hadd := hprod1.add hprod0
intro ε hε
have ht := hadd ε hε
have htreat := treated_tendsto_atTop A p hGrowth hFraction
have hcontrol := controls_tendsto_atTop A p hGrowth hFraction
have hboth : ∀ᶠ r in atTop, 2 ≤ A.treated r ∧ 2 ≤ A.controls r := by
filter_upwards [(tendsto_atTop.1 htreat) 2, (tendsto_atTop.1 hcontrol) 2] with r h1 h0
exact ⟨h1, h0⟩
apply Tendsto.congr' _ ht
filter_upwards [hboth] with r hr
apply (A.design r).Pr_congr
intro w
have hp : A.treatmentFraction r = (A.treated r : ℝ) / (A.groups r : ℝ) := rfl
have hGr : (A.groups r : ℝ) ≠ 0 := by exact_mod_cast (A.groups_pos r).ne'
have htr : (A.treated r : ℝ) ≠ 0 := by exact_mod_cast (A.treated_pos r).ne'
have hcr : ((A.controls r : ℕ) : ℝ) ≠ 0 := by exact_mod_cast (by omega : A.controls r ≠ 0)
have h0 : 2 ≤ armCount (A.groups r) (A.treated r) false := by
simpa [armCount, ScheduleArray.controls] using hr.2
have h1 : 2 ≤ armCount (A.groups r) (A.treated r) true := by
simpa [armCount] using hr.1
have hcNat : 2 ≤ A.groups r - A.treated r := by
simpa [ScheduleArray.controls] using hr.2
have hs1eq : s true r w = armSampleVar (A.schedule r) w true h1 := by
simp only [s, dif_pos h1]
have hs0eq : s false r w = armSampleVar (A.schedule r) w false
h0 := by
simp only [s, dif_pos h0]
have hcr2eq : A.cr2 r w = cr2Var (A.schedule r) hr.1 hcNat w := by
unfold ScheduleArray.cr2
rw [dif_pos hr]
have halg : (A.groups r : ℝ) * A.cr2 r w - A.leadingVariance r =
(s true r w - A.armVariance r true) * (A.treatmentFraction r)⁻¹ +
(s false r w - A.armVariance r false) *
(1 - A.treatmentFraction r)⁻¹ := by
rw [hcr2eq, hs1eq, hs0eq]
unfold cr2Var ScheduleArray.leadingVariance indepGroupVar
ScheduleArray.armVariance ScheduleArray.treatmentFraction pFrac
rw [Nat.cast_sub (A.treated_le r)]
field_simp [ScheduleArray.controls]
ring
rw [halg]
simp
theorem cr2_variance_gap_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p rho B :
ℝ
J :
∀ r, JohnsonProjections (A.popSize r) M
hGrowth :
hFraction :
hSampling :
SamplingFractionLimit A rho
hBounded :
BoundedSchedule A B
hJohnson :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
conclusion
FiniteDesign.TendstoInProb A.design (fun r w => (A.groups r : ℝ) * (A.cr2 r w - A.variance r) - rho * A.degreeOne (by omega) J r) (fun _ => 0)
Proof (Lean source)
lemma cr2_variance_gap_tendstoInProb {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p rho B : ℝ) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hGrowth : GroupCountGrowth A) (hFraction : StableTreatmentFraction A p)
(hSampling : SamplingFractionLimit A rho) (hBounded : BoundedSchedule A B)
(hJohnson : ∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : ∀ r, KneserAdjacencySpectrum (A.popSize r) M) :
FiniteDesign.TendstoInProb A.design
(fun r w => (A.groups r : ℝ) * (A.cr2 r w - A.variance r) -
rho * A.degreeOne (by omega) J r) (fun _ => 0) := by
have hcr2 := cr2_centered_tendstoInProb hM A p B J hGrowth hFraction hBounded
hJohnson hKneser
obtain ⟨C, hC, hdense⟩ := dense_projection_limit hM p rho B
have hv := (hdense A J hGrowth hFraction hSampling hBounded hJohnson hKneser).1
have hvProb := FiniteDesign.deterministic_tendstoInProb A.design
(fun r => (A.groups r : ℝ) * A.variance r -
(A.leadingVariance r - rho * A.degreeOne (by omega) J r)) 0 hv
have hsub := hcr2.sub hvProb
intro ε hε
have ht := hsub ε hε
apply Tendsto.congr' _ ht
filter_upwards [] with r
apply (A.design r).Pr_congr
intro w
constructor <;> intro hh
· convert hh using 1; ring
· convert hh using 1; ring
theorem scaledVariance_eventually_lower
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
∀ᶠ r in atTop, cSigma / 2 < (A.groups r : ℝ) * A.variance r
Proof (Lean source)
lemma scaledVariance_eventually_lower {M : ℕ} (A : ScheduleArray M) (cSigma : ℝ)
(hNondeg : ScaledVarianceNondegenerate A cSigma) :
∀ᶠ r in atTop, cSigma / 2 < (A.groups r : ℝ) * A.variance r := by
have hlt : ((cSigma / 2 : ℝ) : EReal) < liminf (fun r =>
(((A.groups r : ℝ) * A.variance r : ℝ) : EReal)) atTop := by
apply lt_of_lt_of_le _ hNondeg.2
exact EReal.coe_lt_coe_iff.mpr (by linarith [hNondeg.1] : cSigma / 2 < cSigma)
have hev := eventually_lt_of_lt_liminf hlt (by isBoundedDefault)
filter_upwards [hev] with r hr
exact EReal.coe_lt_coe_iff.mp hr
theorem exists_global_abs_bound_of_eventually Lemma exists_global_abs_bound_of_eventually in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
Proof (Lean source)
lemma exists_global_abs_bound_of_eventually (a : ℕ → ℝ) (C : ℝ)
(hC : ∀ᶠ r in atTop, |a r| ≤ C) : ∃ K : ℝ, ∀ r, |a r| ≤ K := by
rw [eventually_atTop] at hC
obtain ⟨N, hN⟩ := hC
let K := |C| + ∑ r ∈ range N, |a r|
refine ⟨K, ?_⟩
intro r
by_cases hr : N ≤ r
· calc
|a r| ≤ C := hN r hr
_ ≤ |C| := le_abs_self C
_ ≤ K := by
dsimp [K]
exact le_add_of_nonneg_right (sum_nonneg fun _ _ => abs_nonneg _)
· have hrmem : r ∈ range N := Finset.mem_range.mpr (lt_of_not_ge hr)
calc
|a r| ≤ ∑ i ∈ range N, |a i| := Finset.single_le_sum
(fun i _ => abs_nonneg (a i)) hrmem
_ ≤ K := by dsimp [K]; linarith [abs_nonneg C]
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Cr2Ratio 5 declarations CR2 ratio and convergence helpers
CR2 ratio and convergence helpers
theorem cr2_ratio_leading_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p rho B cSigma :
ℝ
J :
∀ r, JohnsonProjections (A.popSize r) M
hClass :
DenseScheduleClass A p rho B cSigma
hJohnson :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
conclusion
FiniteDesign.TendstoInProb A.design (fun r w => A.cr2 r w / A.variance r) (fun r => A.leadingVariance r / (A.leadingVariance r - rho * A.degreeOne (by omega) J r))
Proof (Lean source)
lemma cr2_ratio_leading_tendstoInProb {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p rho B cSigma : ℝ) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hClass : DenseScheduleClass A p rho B cSigma)
(hJohnson : ∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : ∀ r, KneserAdjacencySpectrum (A.popSize r) M) :
FiniteDesign.TendstoInProb A.design
(fun r w => A.cr2 r w / A.variance r)
(fun r => A.leadingVariance r /
(A.leadingVariance r - rho * A.degreeOne (by omega) J r)) := by
let d : ℕ → ℝ := fun r =>
A.leadingVariance r - rho * A.degreeOne (by omega) J r
have hu := cr2_centered_tendstoInProb hM A p B J hClass.growth hClass.fraction
hClass.bounded hJohnson hKneser
obtain ⟨C0, hC0, hdense⟩ := dense_projection_limit hM p rho B
have hv := (hdense A J hClass.growth hClass.fraction hClass.sampling hClass.bounded
hJohnson hKneser).1
have hxlow := scaledVariance_eventually_lower A cSigma hClass.nondegenerate
have herr : ∀ᶠ r in atTop,
|(A.groups r : ℝ) * A.variance r - d r| < cSigma / 4 := by
have heps : 0 < cSigma / 4 := by linarith [hClass.nondegenerate.1]
simpa [Real.dist_eq, d] using (Metric.tendsto_atTop.1 hv (cSigma / 4) heps)
have hdlow : ∀ᶠ r in atTop, cSigma / 4 < d r := by
filter_upwards [hxlow, herr] with r hx he
have hab := (le_abs_self ((A.groups r : ℝ) * A.variance r - d r)).trans_lt he
linarith
have hf (z : Arm) : ∀ r S,
|armTable (A.popSize r) M (A.schedule r) z S| ≤ B := fun r S =>
armTable_abs_le_of_boundedSchedule A B hClass.bounded r z S
have hV (z : Arm) (r : ℕ) : A.armVariance r z ≤ B ^ 2 :=
sliceVar_le_sq_of_abs_le (A.groupSize_le r) _ B (hf z r)
have hpEv : ∀ᶠ r in atTop, p / 2 < A.treatmentFraction r :=
(tendsto_order.1 hClass.fraction.2.2).1 (p / 2) (by linarith [hClass.fraction.1])
have hqEv : ∀ᶠ r in atTop, (1 - p) / 2 < 1 - A.treatmentFraction r := by
have hone : Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1) := tendsto_const_nhds
have hq := hone.sub hClass.fraction.2.2
exact (tendsto_order.1 hq).1 ((1 - p) / 2) (by linarith [hClass.fraction.2.1])
let Rmax := B ^ 2 / (p / 2) + B ^ 2 / ((1 - p) / 2)
have hRbound : ∀ᶠ r in atTop, |A.leadingVariance r| ≤ Rmax := by
filter_upwards [hpEv, hqEv] with r hp hq
have hp0 : 0 < A.treatmentFraction r := lt_trans (by linarith [hClass.fraction.1]) hp
have hq0 : 0 < 1 - A.treatmentFraction r :=
lt_trans (by linarith [hClass.fraction.2.1]) hq
have hR0 : 0 ≤ A.leadingVariance r := by
unfold ScheduleArray.leadingVariance indepGroupVar
exact add_nonneg (div_nonneg ((slice _ _ _).Var_nonneg _) hp0.le)
(div_nonneg ((slice _ _ _).Var_nonneg _) hq0.le)
rw [abs_of_nonneg hR0]
unfold ScheduleArray.leadingVariance indepGroupVar Rmax
apply add_le_add
· exact div_le_div₀ (sq_nonneg B) (hV true r)
(by linarith [hClass.fraction.1]) hp.le
· exact div_le_div₀ (sq_nonneg B) (hV false r)
(by linarith [hClass.fraction.2.1]) hq.le
have hdInvEv : ∀ᶠ r in atTop, |(d r)⁻¹| ≤ 4 / cSigma := by
filter_upwards [hdlow] with r hd
have hd0 : 0 < d r := by linarith [hClass.nondegenerate.1]
rw [abs_inv, abs_of_pos hd0]
rw [inv_eq_one_div, le_div_iff₀ hClass.nondegenerate.1]
rw [div_mul_eq_mul_div, div_le_iff₀ hd0]
nlinarith [hClass.nondegenerate.1]
obtain ⟨Kd, hKd⟩ := exists_global_abs_bound_of_eventually (fun r => (d r)⁻¹)
(4 / cSigma) hdInvEv
have hKd0 : 0 ≤ Kd := le_trans (abs_nonneg ((d 0)⁻¹)) (hKd 0)
have hdinvB := boundedInProb_of_pointwise_bound A.design (fun r _ => (d r)⁻¹) Kd
(fun r _ => hKd r)
have hnumErr := hu.mul_boundedInProb hdinvB
have hnum : FiniteDesign.TendstoInProb A.design
(fun r w => ((A.groups r : ℝ) * A.cr2 r w) / d r)
(fun r => A.leadingVariance r / d r) := by
intro ε hε
have ht := hnumErr ε hε
apply Tendsto.congr' _ ht
filter_upwards [] with r
apply (A.design r).Pr_congr
intro w
have heq : ((A.groups r : ℝ) * A.cr2 r w) / d r -
A.leadingVariance r / d r =
((A.groups r : ℝ) * A.cr2 r w - A.leadingVariance r) * (d r)⁻¹ := by
simp only [div_eq_mul_inv]
ring
rw [heq]
simp
have hyOrd : Tendsto (fun r => ((A.groups r : ℝ) * A.variance r) / d r)
atTop (nhds 1) := by
have hzero : Tendsto (fun r =>
((A.groups r : ℝ) * A.variance r - d r) / d r) atTop (nhds 0) := by
have hvabs : Tendsto (fun r =>
|(A.groups r : ℝ) * A.variance r - d r|) atTop (nhds 0) :=
(tendsto_zero_iff_abs_tendsto_zero (fun r =>
(A.groups r : ℝ) * A.variance r - d r)).1 hv
rw [tendsto_zero_iff_abs_tendsto_zero]
refine squeeze_zero (g := fun r =>
|(A.groups r : ℝ) * A.variance r - d r| * Kd)
(fun r => abs_nonneg _) (fun r => ?_) ?_
· change |((A.groups r : ℝ) * A.variance r - d r) / d r| ≤ _
rw [abs_div, div_eq_mul_inv]
have h := hKd r
rw [abs_inv] at h
exact mul_le_mul_of_nonneg_left h (abs_nonneg _)
· simpa using hvabs.mul_const Kd
have := (tendsto_const_nhds : Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1)).add hzero
have heq : ∀ᶠ r in atTop, 1 +
((A.groups r : ℝ) * A.variance r - d r) / d r =
((A.groups r : ℝ) * A.variance r) / d r := by
filter_upwards [hdlow] with r hd
have hdne : d r ≠ 0 := ne_of_gt (by linarith [hClass.nondegenerate.1])
field_simp
ring
simpa using this.congr' heq
have hy := FiniteDesign.deterministic_tendstoInProb A.design _ 1 hyOrd
have haEv : ∀ᶠ r in atTop, |A.leadingVariance r / d r| ≤ Rmax * (4 / cSigma) := by
filter_upwards [hRbound, hdInvEv] with r hR hd
rw [div_eq_mul_inv, abs_mul]
have hRmax0 : 0 ≤ Rmax := by
dsimp [Rmax]
exact add_nonneg (div_nonneg (sq_nonneg B) (by linarith [hClass.fraction.1]))
(div_nonneg (sq_nonneg B) (by linarith [hClass.fraction.2.1]))
exact mul_le_mul hR hd (abs_nonneg _) hRmax0
obtain ⟨Ka, hKa⟩ := exists_global_abs_bound_of_eventually
(fun r => A.leadingVariance r / d r) (Rmax * (4 / cSigma)) haEv
have hratio := FiniteDesign.tendstoInProb_div_one A.design
(fun r w => ((A.groups r : ℝ) * A.cr2 r w) / d r)
(fun r w => ((A.groups r : ℝ) * A.variance r) / d r)
(fun r => A.leadingVariance r / d r) Ka hKa hnum hy
intro ε hε
have ht := hratio ε hε
apply Tendsto.congr' _ ht
filter_upwards [hdlow, hxlow] with r hd hx
apply (A.design r).Pr_congr
intro w
have hdne : d r ≠ 0 := ne_of_gt (by linarith [hClass.nondegenerate.1])
have hGne : (A.groups r : ℝ) ≠ 0 := by exact_mod_cast (A.groups_pos r).ne'
have hvne : A.variance r ≠ 0 := by
intro hz
rw [hz, mul_zero] at hx
linarith [hClass.nondegenerate.1]
have heq : ((A.groups r : ℝ) * A.cr2 r w / d r) /
((A.groups r : ℝ) * A.variance r / d r) = A.cr2 r w / A.variance r := by
field_simp
rw [heq]
Given the stated population sizes, design objects, functions, and conditions, the retarget result holds.
Formal statement
hypotheses
Ω :
X :
∀ r
if
Ω r
then
ℝ
a :
ℕ → ℝ
c :
ℝ
hX :
FiniteDesign.TendstoInProb D X a
conclusion
FiniteDesign.TendstoInProb D X (fun _ => c)
Proof (Lean source)
lemma FiniteDesign.TendstoInProb.retarget
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{X : ∀ r, Ω r → ℝ} {a : ℕ → ℝ} {c : ℝ}
(hX : FiniteDesign.TendstoInProb D X a) (ha : Tendsto a atTop (nhds c)) :
FiniteDesign.TendstoInProb D X (fun _ => c) := by
have hdet := FiniteDesign.deterministic_tendstoInProb D a c ha
have hsum := hX.add hdet
intro ε hε
have ht := hsum ε hε
apply Tendsto.congr' _ ht
filter_upwards [] with r
apply (D r).Pr_congr
intro w
constructor <;> intro hh
· convert hh using 1; ring
· convert hh using 1; ring
theorem tendsto_of_deterministic_tendstoInProb Lemma tendsto_of_deterministic_tendstoInProb in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
Ω :
a :
ℕ → ℝ
c :
ℝ
h :
FiniteDesign.TendstoInProb D (fun r _ => a r) (fun _ => c)
Proof (Lean source)
lemma tendsto_of_deterministic_tendstoInProb
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{a : ℕ → ℝ} {c : ℝ}
(h : FiniteDesign.TendstoInProb D (fun r _ => a r) (fun _ => c)) :
Tendsto a atTop (nhds c) := by
apply Metric.tendsto_atTop.2
intro ε hε
have ht := h ε hε
have hev : ∀ᶠ r in atTop,
(D r).Pr (fun _ => ε ≤ |a r - c|) < 1 / 2 :=
(tendsto_order.1 ht).2 (1 / 2) (by norm_num)
obtain ⟨N, hN⟩ := eventually_atTop.1 hev
refine ⟨N, fun r hrN => ?_⟩
have hr := hN r hrN
rw [Real.dist_eq]
by_contra hnot
have hge : ε ≤ |a r - c| := le_of_not_gt hnot
have hone : (D r).Pr (fun _ => ε ≤ |a r - c|) = 1 := by
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
simp [hge, (D r).p_sum]
rw [hone] at hr
norm_num at hr
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
Ω :
X :
∀ r
if
Ω r
then
ℝ
a :
ℕ → ℝ
c :
ℝ
ha :
FiniteDesign.TendstoInProb D X a
hc :
FiniteDesign.TendstoInProb D X (fun _ => c)
Proof (Lean source)
lemma tendstoInProb_target_unique
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{X : ∀ r, Ω r → ℝ} {a : ℕ → ℝ} {c : ℝ}
(ha : FiniteDesign.TendstoInProb D X a)
(hc : FiniteDesign.TendstoInProb D X (fun _ => c)) :
Tendsto a atTop (nhds c) := by
have hrefl : FiniteDesign.TendstoInProb D (fun r _ => a r) a := by
intro ε hε
simp [FiniteDesign.Pr, FiniteDesign.E, FiniteDesign.ind, not_le.mpr hε]
have hsum := (hrefl.sub ha).add hc
apply tendsto_of_deterministic_tendstoInProb (D := D)
intro ε hε
have ht := hsum ε hε
apply Tendsto.congr' _ ht
filter_upwards [] with r
apply (D r).Pr_congr
intro w
constructor <;> intro hh
· convert hh using 1; ring
· convert hh using 1; ring
theorem lower_tail_vanishes_of_tendstoInProb Lemma lower_tail_vanishes_of_tendstoInProb in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma lower_tail_vanishes_of_tendstoInProb
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{X : ∀ r, Ω r → ℝ} {a : ℕ → ℝ}
(hX : FiniteDesign.TendstoInProb D X a) (ha : ∀ᶠ r in atTop, 1 ≤ a r) :
∀ ε : ℝ, 0 < ε →
Tendsto (fun r => (D r).Pr (fun w => X r w < 1 - ε)) atTop (nhds 0) := by
intro ε hε
have ht := hX ε hε
apply squeeze_zero' (Eventually.of_forall fun r => (D r).Pr_nonneg _) _ ht
filter_upwards [ha] with r har
apply (D r).Pr_mono
intro w hw
have hdiff : ε < a r - X r w := by linarith
exact le_trans hdiff.le (by simpa using neg_le_abs (X r w - a r))
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.DiagonalSupport 1 declarations This file turns finite-design convergence in probability into finite supports whose probability tends to one and on which the error vanishes uniformly.
High-probability diagonal supports
This file turns finite-design convergence in probability into finite supports whose probability tends to one and on which the error vanishes uniformly.
theorem exists_uniform_support Lemma FiniteDesign.TendstoInProb.exists_uniform_support in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the exists uniform support result holds.
Formal statement
hypotheses
Ω :
X :
∀ r
if
Ω r
then
ℝ
c :
ℝ
hX :
FiniteDesign.TendstoInProb D X (fun _ => c)
Proof (Lean source)
lemma FiniteDesign.TendstoInProb.exists_uniform_support
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] [∀ r, DecidableEq (Ω r)]
{D : ∀ r, FiniteDesign (Ω r)}
{X : ∀ r, Ω r → ℝ} {c : ℝ}
(hX : FiniteDesign.TendstoInProb D X (fun _ => c)) :
∃ (Γ : ∀ r, Finset (Ω r)) (e : ℕ → ℝ),
Tendsto (fun r => (D r).Pr (fun w => w ∈ Γ r)) atTop (nhds 1) ∧
Tendsto e atTop (nhds 0) ∧
(∀ r, 0 < e r) ∧
(∀ r w, w ∈ Γ r → |X r w - c| < e r) ∧
Tendsto (fun r => sSup {x : ℝ | ∃ w ∈ Γ r,
x = |X r w - c|}) atTop (nhds 0) := by
classical
let δ : ℕ → ℝ := fun k => 1 / ((k : ℝ) + 1)
have hδpos : ∀ k, 0 < δ k := by
intro k
dsimp [δ]
positivity
have hcut : ∀ k, ∃ N, ∀ r, N ≤ r →
(D r).Pr (fun w => δ k ≤ |X r w - c|) < δ k := by
intro k
have ht := hX (δ k) (hδpos k)
have hev : ∀ᶠ r in atTop,
(D r).Pr (fun w => δ k ≤ |X r w - c|) < δ k :=
(tendsto_order.1 ht).2 (δ k) (hδpos k)
exact eventually_atTop.1 hev
choose N hN using hcut
let m : ℕ → ℕ := fun k => if k = 0 then 0 else max k (N k)
let level : ℕ → ℕ := fun r => Nat.findGreatest (fun k => m k ≤ r) r
have hlevel : Tendsto level atTop atTop := by
rw [tendsto_atTop_atTop]
intro k
refine ⟨max k (m k), fun r hr => ?_⟩
apply Nat.le_findGreatest
· exact (le_max_left k (m k)).trans hr
· exact (le_max_right k (m k)).trans hr
let e : ℕ → ℝ := fun r => δ (level r)
have he : Tendsto e atTop (nhds 0) :=
tendsto_one_div_add_atTop_nhds_zero_nat.comp hlevel
let Γ : ∀ r, Finset (Ω r) := fun r => Finset.univ.filter
(fun w => |X r w - c| < e r)
have hlevel_spec : ∀ r, m (level r) ≤ r := by
intro r
apply Nat.findGreatest_spec (P := fun k => m k ≤ r) (zero_le r)
simp [m]
have hbad : Tendsto (fun r => (D r).Pr
(fun w => e r ≤ |X r w - c|)) atTop (nhds 0) := by
exact squeeze_zero (fun r => (D r).Pr_nonneg _) (fun r => by
by_cases hk : level r = 0
· simpa [e, δ, hk] using (D r).Pr_le_one
(fun w => e r ≤ |X r w - c|)
· exact (hN (level r) r ((le_max_right _ _).trans (by
simpa [m, hk] using hlevel_spec r))).le) he
have hprob : Tendsto (fun r => (D r).Pr (fun w => w ∈ Γ r))
atTop (nhds 1) := by
apply Tendsto.congr' _ (by simpa using
(tendsto_const_nhds.sub hbad : Tendsto (fun r => 1 -
(D r).Pr (fun w => e r ≤ |X r w - c|)) atTop (nhds (1 - 0))))
filter_upwards [] with r
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
simp only [Γ, mem_filter, Finset.mem_univ, true_and]
symm
calc
∑ z, (D r).p z * (if |X r z - c| < e r then 1 else 0) =
(∑ z, (D r).p z) -
∑ z, (D r).p z * (if e r ≤ |X r z - c| then 1 else 0) := by
rw [← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro z _
by_cases hz : |X r z - c| < e r
· simp [hz, not_le.mpr hz]
· simp [hz, le_of_not_gt hz]
_ = 1 - ∑ z, (D r).p z *
(if e r ≤ |X r z - c| then 1 else 0) := by rw [(D r).p_sum]
have hnonempty : ∀ᶠ r in atTop, (Γ r).Nonempty := by
have hpositive : ∀ᶠ r in atTop,
(1 : ℝ) / 2 < (D r).Pr (fun w => w ∈ Γ r) :=
(tendsto_order.1 hprob).1 ((1 : ℝ) / 2) (by norm_num)
filter_upwards [hpositive] with r hr
by_contra hempty
rw [Finset.not_nonempty_iff_eq_empty] at hempty
have hz : (D r).Pr (fun w => w ∈ Γ r) = 0 := by
rw [hempty]
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
simp
linarith
have hsSup : Tendsto (fun r => sSup {x : ℝ | ∃ w ∈ Γ r,
x = |X r w - c|}) atTop (nhds 0) := by
refine squeeze_zero' ?_ ?_ he
· filter_upwards [hnonempty] with r hr
obtain ⟨w, hw⟩ := hr
let S : Set ℝ := {x : ℝ | ∃ w ∈ Γ r, x = |X r w - c|}
have hmem : |X r w - c| ∈ S := ⟨w, hw, rfl⟩
have hbdd : BddAbove S := by
refine ⟨e r, ?_⟩
rintro x ⟨v, hv, rfl⟩
exact (by simpa [Γ] using hv : |X r v - c| < e r).le
exact (abs_nonneg _).trans (le_csSup hbdd hmem)
· filter_upwards [hnonempty] with r hr
apply csSup_le
· obtain ⟨w, hw⟩ := hr
exact ⟨|X r w - c|, w, hw, rfl⟩
· rintro x ⟨w, hw, rfl⟩
exact (by simpa [Γ] using hw : |X r w - c| < e r).le
exact ⟨Γ, e, hprob, he, fun r => hδpos (level r), (by
intro r w hw
simpa [Γ] using hw), hsSup⟩
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Estimator 15 declarations This file defines the estimand and estimator on the two-stage assignment space, the exact design variance, the independent-group leading term, the degree-one correction, and the equal-group scalar CR2 statistic.
PAME, exact variance, and scalar CR2 statistics
This file defines the estimand and estimator on the two-stage assignment space, the exact design variance, the independent-group leading term, the degree-one correction, and the equal-group scalar CR2 statistic.
def pFrac
Treated-group fraction.
Definition (Lean source)
parameters
G G1 :
ℕ
_hG1pos :
0 < G1
_hG1lt :
G1 < G
def
pFrac G G1 _hG1pos _hG1lt :
ℝ
given by
(G1 : ℝ) / (G : ℝ)
def obsGroupMean
The observed group mean under its assigned arm.
Definition (Lean source)
parameters
def
obsGroupMean Y w g :
ℝ
Finite-population PAME.
Difference in realized treated and control group means.
Definition (Lean source)
parameters
n M G G1 :
ℕ
Y :
PotentialOutcome n M
_hG1pos :
0 < G1
_hG1lt :
G1 < G
w :
PartitionTuple n M G × TreatmentSpace G G1
def
pameHat Y _hG1pos _hG1lt w :
ℝ
The estimand and its design-based estimator, exposed jointly as required by the paper's definition.
Definition (Lean source)
parameters
def
Exact design variance of the PAME estimator.
Definition (Lean source)
parameters
def
sigmaSq Y hMG hG1pos hG1lt :
ℝ
given by
Independent-group leading variance.
def degreeOneEnergy
Degree-one energy of the arm-table contrast.
Definition (Lean source)
parameters
def
degreeOneEnergy n M hMpos hM J Y :
ℝ
structure DenseCorrectionValues
Both deterministic quantities entering the dense variance correction.
Definition (Lean source)
fields
degreeOne :
ℝ
leadingVariance :
ℝ
def denseCorrection
The paper's degree-one energy and independent-group leading variance, constructed together from a genuine Johnson projection family.
Definition (Lean source)
parameters
def
given by
clause 1
degreeOne := degreeOneEnergy n M hMpos hM J Y
clause 2
leadingVariance := indepGroupVar n M G G1 hM hG1pos hG1lt Y
def realizedArmSet
The realized groups in arm z.
Definition (Lean source)
parameters
given by
if z then w.2.1 else univ \ w.2.1
def armCount
The fixed number of groups in arm z.
Definition (Lean source)
def armObsMean
Realized mean within one treatment arm.
Definition (Lean source)
parameters
def
armObsMean Y w z :
ℝ
given by
(∑ g ∈ realizedArmSet w z, obsGroupMean Y w g) / (armCount G G1 z : ℝ)
def armSampleVar
Within-arm sample variance with the paper's G_z - 1 denominator.
Definition (Lean source)
parameters
n M G G1 :
ℕ
Y :
PotentialOutcome n M
w :
PartitionTuple n M G × TreatmentSpace G G1
z :
_hArm :
2 ≤ armCount G G1 z
def
armSampleVar Y w z _hArm :
ℝ
given by
(∑ g ∈ realizedArmSet w z, (obsGroupMean Y w g - armObsMean Y w z) ^ 2) / ((armCount G G1 z - 1 : ℕ) : ℝ)
Equal-group scalar CR2 statistic.
Definition (Lean source)
parameters
n M G G1 :
ℕ
Y :
PotentialOutcome n M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
w :
PartitionTuple n M G × TreatmentSpace G G1
def
cr2Var Y hG1 hG0 w :
ℝ
given by
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.ExactVariance 19 declarations Exchangeable moments for the exact two-stage variance calculation
Exchangeable moments for the exact two-stage variance calculation
Given the stated population sizes, design objects, functions, and conditions, the finite design ext p result holds.
Formal statement
Proof (Lean source)
lemma finiteDesign_ext_p {Ω : Type*} [Fintype Ω] (D D' : FiniteDesign Ω)
(h : D.p = D'.p) : D = D' := by
cases D with
| mk p hp hs =>
cases D' with
| mk p' hp' hs' =>
simp only at h
subst p'
rfl
theorem randomPartitionDesign_eq_compoundCore
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hG1 :
G1 ≤ G
conclusion
randomPartitionDesign n M G G1 hMG hG1
= compoundCore (uniformPartitionTuple n M G hMG) (fun _ => completeRandomization G1 (by simpa using hG1))
Proof (Lean source)
lemma randomPartitionDesign_eq_compoundCore (n M G G1 : ℕ)
(hMG : M * G ≤ n) (hG1 : G1 ≤ G) :
randomPartitionDesign n M G G1 hMG hG1 =
compoundCore (uniformPartitionTuple n M G hMG)
(fun _ => completeRandomization G1 (by simpa using hG1)) := by
classical
let D := randomPartitionDesign n M G G1 hMG hG1
let D' := compoundCore (uniformPartitionTuple n M G hMG)
(fun _ : PartitionTuple n M G =>
completeRandomization (V := Fin G) G1 (by simpa using hG1))
change D = D'
apply finiteDesign_ext_p D D'
funext w
simp only [D, D', randomPartitionDesign, FiniteDesign.map_p, compoundCore]
calc
(∑ x, @ite ℝ ((twoStagePairEquiv n M G G1) x = w)
(propDecidable _)
((prodDesign (twoStageDesigns n M G G1 hMG hG1)).p x) 0) =
∑ y, (@ite ℝ (y = w) (propDecidable _)
((prodDesign (twoStageDesigns n M G G1 hMG hG1)).p
((twoStagePairEquiv n M G G1).symm y)) 0) := by
exact Fintype.sum_equiv (twoStagePairEquiv n M G G1) _ _ (fun _ => rfl)
_ = _ := by
rw [Finset.sum_eq_single w]
· simp [twoStageDesigns, prodDesign_p, twoStagePairEquiv]
exact mul_comm _ _
· intro y _ hy
simp [hy]
· simp
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
(compoundCore D₁ D₂).E F = D₁.E (fun s => (D₂ s).E (fun w => F (s, w)))
Proof (Lean source)
lemma finiteDesign_E_compoundCore_tower {Ω₁ Ω₂ : Type*} [Fintype Ω₁] [Fintype Ω₂]
(D₁ : FiniteDesign Ω₁) (D₂ : Ω₁ → FiniteDesign Ω₂) (F : Ω₁ × Ω₂ → ℝ) :
(compoundCore D₁ D₂).E F = D₁.E (fun s => (D₂ s).E (fun w => F (s, w))) := by
unfold FiniteDesign.E compoundCore
rw [Fintype.sum_prod_type]
apply Finset.sum_congr rfl
intro s _
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro w _
ring
theorem finiteDesign_Var_compoundCore_tower Lemma finiteDesign_Var_compoundCore_tower in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
(compoundCore D₁ D₂).Var X
= D₁.E (fun s => (D₂ s).Var (fun w => X (s, w)))
+ D₁.Var (fun s => (D₂ s).E (fun w => X (s, w)))
Proof (Lean source)
lemma finiteDesign_Var_compoundCore_tower {Ω₁ Ω₂ : Type*} [Fintype Ω₁] [Fintype Ω₂]
(D₁ : FiniteDesign Ω₁) (D₂ : Ω₁ → FiniteDesign Ω₂) (X : Ω₁ × Ω₂ → ℝ) :
(compoundCore D₁ D₂).Var X =
D₁.E (fun s => (D₂ s).Var (fun w => X (s, w))) +
D₁.Var (fun s => (D₂ s).E (fun w => X (s, w))) := by
rw [FiniteDesign.Var_eq, finiteDesign_E_compoundCore_tower,
finiteDesign_E_compoundCore_tower]
have hsq : (fun s => (D₂ s).E (fun w => X (s, w) ^ 2)) =
fun s => (D₂ s).Var (fun w => X (s, w)) +
((D₂ s).E (fun w => X (s, w))) ^ 2 := by
funext s
rw [FiniteDesign.Var_eq]
ring
rw [hsq, FiniteDesign.E_add, FiniteDesign.Var_eq]
ring
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
(D.map f).Var X = D.Var (fun w => X (f w))
Proof (Lean source)
lemma finiteDesign_Var_map {Ω Ω' : Type*} [Fintype Ω] [Fintype Ω']
(D : FiniteDesign Ω) (f : Ω → Ω') (X : Ω' → ℝ) :
(D.map f).Var X = D.Var (fun w => X (f w)) := by
rw [FiniteDesign.Var_eq, FiniteDesign.Var_eq, FiniteDesign.E_map,
FiniteDesign.E_map]
theorem randomPartition_E_partition
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
(randomPartitionDesign n M G G1 hMG hG1).E (fun w => F w.1)
= (uniformPartitionTuple n M G hMG).E F
Proof (Lean source)
lemma randomPartition_E_partition (n M G G1 : ℕ)
(hMG : M * G ≤ n) (hG1 : G1 ≤ G) (F : PartitionTuple n M G → ℝ) :
(randomPartitionDesign n M G G1 hMG hG1).E (fun w => F w.1) =
(uniformPartitionTuple n M G hMG).E F := by
rw [randomPartitionDesign, FiniteDesign.E_map]
exact FiniteDesign.E_prod_apply (twoStageDesigns n M G G1 hMG hG1) false F
theorem expectedSampleVariance_exchangeable Lemma expectedSampleVariance_exchangeable in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma expectedSampleVariance_exchangeable {Ω : Type*} [Fintype Ω]
(D : FiniteDesign Ω) (G : ℕ) (hG : 2 ≤ G) (X : Fin G → Ω → ℝ) (q r : ℝ)
(hsq : ∀ i, D.E (fun w => X i w ^ 2) = q)
(hpair : ∀ i j, i ≠ j → D.E (fun w => X i w * X j w) = r) :
D.E (fun w => S1 (fun i => X i w)) = q - r := by
have hGpos : 0 < G := by omega
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast hGpos.ne'
have hG1r : (G : ℝ) - 1 ≠ 0 := by
have : (1 : ℝ) < G := by exact_mod_cast hG
linarith
unfold S1 popMeanV
simp_rw [sum_sub_mean_sq hGpos]
rw [show (fun w => ((∑ j, X j w ^ 2) - (∑ i, X i w) ^ 2 / (G : ℝ)) /
((G : ℝ) - 1)) = fun w =>
((∑ j, X j w ^ 2) - (1 / (G : ℝ)) * (∑ i, X i w) ^ 2) *
(1 / ((G : ℝ) - 1)) by funext w; ring]
rw [FiniteDesign.E_mul_const, FiniteDesign.E_sub, FiniteDesign.E_sum]
simp_rw [hsq]
have hsqsum : D.E (fun w => (∑ i, X i w) ^ 2) =
∑ i : Fin G, ∑ j : Fin G, if i = j then q else r := by
have hpoint : (fun w => (∑ i, X i w) ^ 2) =
fun w => ∑ i, ∑ j, X i w * X j w := by
funext w
rw [sq, Finset.sum_mul_sum]
rw [hpoint, FiniteDesign.E_sum]
apply Finset.sum_congr rfl
intro i _
rw [FiniteDesign.E_sum]
apply Finset.sum_congr rfl
intro j _
by_cases hij : i = j
· subst j
simpa [pow_two] using hsq i
· simp [hij, hpair i j hij]
rw [D.E_const_mul (1 / (G : ℝ)) (fun w => (∑ i, X i w) ^ 2), hsqsum]
have hcollapse : (∑ i : Fin G, ∑ j : Fin G, if i = j then q else r) =
r * (∑ _i : Fin G, (1 : ℝ)) ^ 2 +
(q - r) * ∑ _i : Fin G, (1 : ℝ) ^ 2 := by
simpa only [one_mul] using
(sum_sum_ite_quadratic univ (fun _ : Fin G => (1 : ℝ)) q r)
rw [hcollapse]
simp only [Finset.sum_const, Finset.card_univ, card_fin, nsmul_eq_mul,
one_pow, mul_one]
field_simp
ring
theorem partition_expected_sample_variance
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hG :
2 ≤ G
hG1 :
G1 ≤ G
f :
Omega n M → ℝ
conclusion
(uniformPartitionTuple n M G hMG).E (fun T => S1 (fun g => f (T.1 g)))
- sliceInner n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG) (fun A => f A - (slice n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f) (kneserOp n M (fun A => f A - (slice n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f))
Proof (Lean source)
lemma partition_expected_sample_variance (n M G G1 : ℕ)
(hMG : M * G ≤ n) (hG : 2 ≤ G) (hG1 : G1 ≤ G)
(f : Omega n M → ℝ) :
(uniformPartitionTuple n M G hMG).E
(fun T => S1 (fun g => f (T.1 g))) =
(slice n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).Var f -
sliceInner n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)
(fun A => f A - (slice n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)
(kneserOp n M (fun A => f A - (slice n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)) := by
let hM : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
have h2MG : 2 * M ≤ M * G := by
simpa [Nat.mul_comm] using Nat.mul_le_mul_left M hG
have h2M : 2 * M ≤ n := le_trans h2MG hMG
let D := uniformPartitionTuple n M G hMG
let q := (slice n M hM).E (fun A => f A ^ 2)
let r := (orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1 * f P.1.2)
have hsq (g : Fin G) : D.E (fun T => f (T.1 g) ^ 2) = q := by
rw [← randomPartition_E_partition n M G G1 hMG hG1]
exact marginal_uniform_coord n M G G1 hMG (by omega) hG1 g (fun A => f A ^ 2)
have hpair (g g' : Fin G) (hgg' : g ≠ g') :
D.E (fun T => f (T.1 g) * f (T.1 g')) = r := by
rw [← randomPartition_E_partition n M G G1 hMG hG1]
simpa only [groupAt] using
(marginal_uniform_ordered_disjoint_pair n M G G1 hMG hG1 h2M g g' hgg'
(fun P => f P.1.1 * f P.1.2))
rw [expectedSampleVariance_exchangeable D G hG (fun g T => f (T.1 g)) q r hsq hpair]
let mu := (slice n M hM).E f
have hcenterPair :
(orderedDisjointPairDesign n M h2M).E
(fun P => (f P.1.1 - mu) * (f P.1.2 - mu)) = r - mu ^ 2 := by
have hpoint : (fun P : OrderedDisjointPair n M =>
(f P.1.1 - mu) * (f P.1.2 - mu)) =
(fun P => f P.1.1 * f P.1.2 - mu * f P.1.1 - mu * f P.1.2 + mu ^ 2) := by
funext P
ring
rw [hpoint]
rw [FiniteDesign.E_add, FiniteDesign.E_sub, FiniteDesign.E_sub,
FiniteDesign.E_const_mul, FiniteDesign.E_const_mul,
orderedDisjointPair_first_E_eq_slice h2M,
orderedDisjointPair_second_E_eq_slice h2M, FiniteDesign.E_const]
change r - mu * mu - mu * mu + mu ^ 2 = r - mu ^ 2
ring
rw [show sliceInner n M hM (fun A => f A - (slice n M hM).E f)
(kneserOp n M (fun A => f A - (slice n M hM).E f)) =
(orderedDisjointPairDesign n M h2M).E
(fun P => (f P.1.1 - mu) * (f P.1.2 - mu)) by
symm
simpa only [mu] using
(orderedDisjointPair_E_eq n M h2M
(fun A => f A - (slice n M hM).E f)
(fun A => f A - (slice n M hM).E f))]
rw [hcenterPair, FiniteDesign.Var_eq]
change q - r = (q - mu ^ 2) - (r - mu ^ 2)
ring
theorem pameHat_eq_diffInMeans
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M G G1 :
ℕ
Y :
PotentialOutcome n M
hG1pos :
0 < G1
hG1lt :
G1 < G
T :
PartitionTuple n M G
S :
TreatmentSpace G G1
Proof (Lean source)
lemma pameHat_eq_diffInMeans {n M G G1 : ℕ} (Y : PotentialOutcome n M)
(hG1pos : 0 < G1) (hG1lt : G1 < G)
(T : PartitionTuple n M G) (S : TreatmentSpace G G1) :
pameHat Y hG1pos hG1lt (T, S) =
diffInMeans G1 (fun g => armTable n M Y true (T.1 g))
(fun g => armTable n M Y false (T.1 g)) S := by
classical
simp [pameHat, diffInMeans, treatedMean, controlMean, Finset.sum_filter]
theorem cr2Var_eq_varHat
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
Y :
PotentialOutcome n M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
T :
PartitionTuple n M G
S :
TreatmentSpace G G1
Proof (Lean source)
lemma cr2Var_eq_varHat {n M G G1 : ℕ} (Y : PotentialOutcome n M)
(hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1)
(T : PartitionTuple n M G) (S : TreatmentSpace G G1) :
cr2Var Y hG1 hG0 (T, S) =
varHat G1 (fun g => armTable n M Y true (T.1 g))
(fun g => armTable n M Y false (T.1 g))
(crdToBoolOn G1 S) := by
classical
let a : Fin G → ℝ := fun g => armTable n M Y true (T.1 g)
let b : Fin G → ℝ := fun g => armTable n M Y false (T.1 g)
let z := crdToBoolOn G1 S
have hsum (f : Fin G → ℝ) :
(∑ x ∈ S.1, f x) = ∑ x, if x ∈ S.1 then f x else 0 := by
rw [← Finset.sum_filter]
simp
have hObsT : armObsMean Y (T, S) true = obsMeanTreated G1 a z := by
simp [armObsMean, obsMeanTreated, realizedArmSet, armCount, obsGroupMean,
a, z, T,
crdToBoolOn, FiniteDesign.ind]
congr 1
apply Finset.sum_congr rfl
intro g hg
simp [hg]
have hObsC : armObsMean Y (T, S) false = obsMeanControl G1 b z := by
simp [armObsMean, obsMeanControl, realizedArmSet, armCount, obsGroupMean,
b, z, T,
crdToBoolOn, FiniteDesign.ind, Nat.cast_sub (by omega : G1 ≤ G)]
congr 1
rw [hsum, ← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro g _
by_cases hg : g ∈ S.1 <;> simp [hg]
have hVarT : armSampleVar Y (T, S) true (by simpa [armCount] using hG1) =
ShatTreated G1 a z := by
simp [armSampleVar, ShatTreated, realizedArmSet, armCount, hObsT,
obsGroupMean, a, z, T,
crdToBoolOn, FiniteDesign.ind,
Nat.cast_sub (by omega : 1 ≤ G1)]
congr 1
apply Finset.sum_congr rfl
intro g hg
simp [hg]
have hVarC : armSampleVar Y (T, S) false (by simpa [armCount] using hG0) =
ShatControl G1 b z := by
simp [armSampleVar, ShatControl, realizedArmSet, armCount, hObsC,
obsGroupMean, b, z, T,
crdToBoolOn, FiniteDesign.ind,
Nat.cast_sub (by omega : 1 ≤ G - G1), Nat.cast_sub (by omega : G1 ≤ G)]
congr 1
rw [hsum, ← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro g _
by_cases hg : g ∈ S.1 <;> simp [hg]
unfold cr2Var varHat
rw [hVarT, hVarC, Nat.cast_sub (by omega : G1 ≤ G)]
theorem conditional_E_pameHat
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
Proof (Lean source)
lemma conditional_E_pameHat {n M G G1 : ℕ} (Y : PotentialOutcome n M)
(hG1pos : 0 < G1) (hG1lt : G1 < G) (T : PartitionTuple n M G) :
(completeRandomization (V := Fin G) G1 (by simpa using hG1lt.le)).E
(fun S => pameHat Y hG1pos hG1lt (T, S)) =
(∑ g, (armTable n M Y true (T.1 g) -
armTable n M Y false (T.1 g))) / (G : ℝ) := by
rw [(completeRandomization (V := Fin G) G1 (by simpa using hG1lt.le)).E_congr
(fun S => pameHat_eq_diffInMeans Y hG1pos hG1lt T S)]
rw [E_diffInMeans_eq_sate G1 hG1pos (by simpa using hG1lt)]
simp [sateEstimand]
theorem conditional_Var_pameHat
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
(completeRandomization (V := Fin G) G1 (by simpa using hG1lt.le)).Var (fun S => pameHat Y hG1pos hG1lt (T, S))
Proof (Lean source)
lemma conditional_Var_pameHat {n M G G1 : ℕ} (Y : PotentialOutcome n M)
(hG1pos : 0 < G1) (hG1lt : G1 < G) (T : PartitionTuple n M G) :
(completeRandomization (V := Fin G) G1 (by simpa using hG1lt.le)).Var
(fun S => pameHat Y hG1pos hG1lt (T, S)) =
S1 (fun g => armTable n M Y true (T.1 g)) / (G1 : ℝ) +
S0 (fun g => armTable n M Y false (T.1 g)) / ((G : ℝ) - G1) -
Stau (fun g => armTable n M Y true (T.1 g))
(fun g => armTable n M Y false (T.1 g)) / (G : ℝ) := by
let a := fun g : Fin G => armTable n M Y true (T.1 g)
let b := fun g : Fin G => armTable n M Y false (T.1 g)
have hp : (fun S => pameHat Y hG1pos hG1lt (T, S)) = diffInMeans G1 a b := by
funext S
exact pameHat_eq_diffInMeans Y hG1pos hG1lt T S
rw [hp]
have hneg : diffInMeans G1 a b = fun S => -tauHat G1 a b (crdToBoolOn G1 S) := by
funext S
have ht : (∑ x, a x * T x
(crdToBoolOn G1 S)) = ∑ x, if x ∈ S.1 then a x else 0 := by
apply Finset.sum_congr rfl
intro x _
by_cases hx : x ∈ S.1 <;> simp [T,
crdToBoolOn, FiniteDesign.ind, hx]
have hc : (∑ x, b x * (1 - T x
(crdToBoolOn G1 S))) = ∑ x, if x ∈ S.1 then 0 else b x := by
apply Finset.sum_congr rfl
intro x _
by_cases hx : x ∈ S.1 <;> simp [T,
crdToBoolOn, FiniteDesign.ind, hx]
rw [diffInMeans, treatedMean, controlMean, tauHat, ht, hc]
rw [card_fin, Nat.cast_sub hG1lt.le]
ring
rw [hneg]
have hminus : (fun S => -tauHat G1 a b (crdToBoolOn G1 S)) =
fun S => (-1 : ℝ) * tauHat G1 a b (crdToBoolOn G1 S) := by
funext S
ring
rw [hminus, FiniteDesign.Var_const_mul]
norm_num
rw [← finiteDesign_Var_map
(completeRandomization (V := Fin G) G1 (by simpa using hG1lt.le))
(crdToBoolOn G1) (tauHat G1 a b)]
change (crd G1 hG1lt.le).Var (tauHat G1 a b) = _
exact Var_tauHat_CRD G1 a b hG1pos hG1lt
theorem conditional_E_cr2Var
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
Proof (Lean source)
lemma conditional_E_cr2Var {n M G G1 : ℕ} (Y : PotentialOutcome n M)
(hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1) (T : PartitionTuple n M G) :
(completeRandomization (V := Fin G) G1
(by simpa using (by omega : G1 ≤ G))).E
(fun S => cr2Var Y hG1 hG0 (T, S)) =
S1 (fun g => armTable n M Y true (T.1 g)) / (G1 : ℝ) +
S0 (fun g => armTable n M Y false (T.1 g)) / ((G - G1 : ℕ) : ℝ) := by
let a := fun g : Fin G => armTable n M Y true (T.1 g)
let b := fun g : Fin G => armTable n M Y false (T.1 g)
rw [(completeRandomization (V := Fin G) G1
(by simpa using (by omega : G1 ≤ G))).E_congr
(fun S => cr2Var_eq_varHat Y hG1 hG0 T S)]
rw [show (completeRandomization (V := Fin G) G1
(by simpa using (by omega : G1 ≤ G))).E
(fun S => varHat G1 a b (crdToBoolOn G1 S)) =
(crd G1 (by omega : G1 ≤ G)).E (varHat G1 a b) by
rw [crd, crdOn, FiniteDesign.E_map]]
unfold varHat
rw [FiniteDesign.E_add]
rw [show (fun w => ShatTreated G1 a w / (G1 : ℝ)) =
fun w => (G1 : ℝ)⁻¹ * ShatTreated G1 a w by funext w; ring,
show (fun w => ShatControl G1 b w / ((G : ℝ) - G1)) =
fun w => ((G : ℝ) - G1)⁻¹ * ShatControl G1 b w by funext w; ring,
FiniteDesign.E_const_mul, FiniteDesign.E_const_mul]
rw [E_ShatTreated G1 a (crd G1 (by omega)) hG1 (by omega)
(fun j => crd_mean G1 (by omega) j)
(fun j k h => crd_pair G1 (by omega) j k h)
(fun w hw => crd_supp G1 (by omega) w hw)]
rw [E_ShatControl G1 b (crd G1 (by omega)) hG1 (by omega)
(fun j => crd_mean G1 (by omega) j)
(fun j k h => crd_pair G1 (by omega) j k h)
(fun w hw => crd_supp G1 (by omega) w hw)]
rw [Nat.cast_sub (by omega : G1 ≤ G)]
ring
theorem partition_E_mean
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hG :
0 < G
hG1 :
G1 ≤ G
f :
Omega n M → ℝ
conclusion
(uniformPartitionTuple n M G hMG).E (fun T => (∑ g, f (T.1 g)) / (G : ℝ))
Proof (Lean source)
lemma partition_E_mean (n M G G1 : ℕ) (hMG : M * G ≤ n) (hG : 0 < G)
(hG1 : G1 ≤ G) (f : Omega n M → ℝ) :
(uniformPartitionTuple n M G hMG).E (fun T => (∑ g, f (T.1 g)) / (G : ℝ)) =
(slice n M (le_trans (le_mul_of_pos_right M hG) hMG)).E f := by
rw [← randomPartition_E_partition n M G G1 hMG hG1]
change (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => (∑ g, f (groupAt n M G G1 w g)) / (G : ℝ)) = _
calc
_ = (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => (G : ℝ)⁻¹ * ∑ g, f (groupAt n M G G1 w g)) := by
apply (randomPartitionDesign n M G G1 hMG hG1).E_congr
intro w
ring
_ = _ := by
rw [FiniteDesign.E_const_mul, FiniteDesign.E_sum]
simp_rw [marginal_uniform_coord n M G G1 hMG hG hG1]
simp only [Finset.sum_const, Finset.card_univ, card_fin, nsmul_eq_mul]
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast hG.ne'
field_simp
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
theorem orderedDisjointPair_E_mul_comm
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M :
ℕ
h2M :
2 * M ≤ n
f g :
Omega n M → ℝ
conclusion
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1 * g P.1.2)
= (orderedDisjointPairDesign n M h2M).E (fun P => g P.1.1 * f P.1.2)
Proof (Lean source)
lemma orderedDisjointPair_E_mul_comm {n M : ℕ} (h2M : 2 * M ≤ n)
(f g : Omega n M → ℝ) :
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1 * g P.1.2) =
(orderedDisjointPairDesign n M h2M).E (fun P => g P.1.1 * f P.1.2) := by
classical
unfold FiniteDesign.E orderedDisjointPairDesign uniformFiniteDesign
calc
(∑ x : OrderedDisjointPair n M, (1 / (card (OrderedDisjointPair n M) : ℝ)) *
(f x.1.1 * g x.1.2)) =
∑ x : OrderedDisjointPair n M, (1 / (card (OrderedDisjointPair n M) : ℝ)) *
(f (orderedDisjointPairSwap x).1.1 * g (orderedDisjointPairSwap x).1.2) := by
exact Fintype.sum_equiv orderedDisjointPairSwap _ _ (fun _ => rfl)
_ = _ := by
apply Finset.sum_congr rfl
intro x _
change _ * (f x.1.2 * g x.1.1) = _
ring
theorem disjointCov_sub_self
Given the stated population sizes, design objects, functions, and conditions, the disjoint cov sub self result holds.
Formal statement
hypotheses
n M :
ℕ
h2M :
2 * M ≤ n
hM :
M ≤ n
f g :
Omega n M → ℝ
conclusion
sliceInner n M hM (fun A => (f A - g A) - (slice n M hM).E (fun A => f A - g A)) (kneserOp n M (fun A => (f A - g A) - (slice n M hM).E (fun A => f A - g A)))
= sliceInner n M hM (fun A => f A - (slice n M hM).E f) (kneserOp n M (fun A => f A - (slice n M hM).E f))
Proof (Lean source)
lemma disjointCov_sub_self (n M : ℕ) (h2M : 2 * M ≤ n) (hM : M ≤ n)
(f g : Omega n M → ℝ) :
sliceInner n M hM
(fun A => (f A - g A) - (slice n M hM).E (fun A => f A - g A))
(kneserOp n M
(fun A => (f A - g A) - (slice n M hM).E (fun A => f A - g A))) =
sliceInner n M hM (fun A => f A - (slice n M hM).E f)
(kneserOp n M (fun A => f A - (slice n M hM).E f)) +
sliceInner n M hM (fun A => g A - (slice n M hM).E g)
(kneserOp n M (fun A => g A - (slice n M hM).E g)) -
2 * sliceInner n M hM (fun A => f A - (slice n M hM).E f)
(kneserOp n M (fun A => g A - (slice n M hM).E g)) := by
rw [FiniteDesign.E_sub]
have hc : (fun A => (f A - g A) - ((slice n M hM).E f - (slice n M hM).E g)) =
fun A => (f A - (slice n M hM).E f) - (g A - (slice n M hM).E g) := by
funext A
ring
rw [hc]
rw [← orderedDisjointPair_E_eq n M h2M,
← orderedDisjointPair_E_eq n M h2M,
← orderedDisjointPair_E_eq n M h2M,
← orderedDisjointPair_E_eq n M h2M]
have hp : (fun P : OrderedDisjointPair n M =>
((f P.1.1 - (slice n M hM).E f) - (g P.1.1 - (slice n M hM).E g)) *
((f P.1.2 - (slice n M hM).E f) - (g P.1.2 - (slice n M hM).E g))) =
fun P =>
(f P.1.1 - (slice n M hM).E f) * (f P.1.2 - (slice n M hM).E f) -
(f P.1.1 - (slice n M hM).E f) * (g P.1.2 - (slice n M hM).E g) -
(g P.1.1 - (slice n M hM).E g) * (f P.1.2 - (slice n M hM).E f) +
(g P.1.1 - (slice n M hM).E g) * (g P.1.2 - (slice n M hM).E g) := by
funext P
ring
rw [hp,
FiniteDesign.E_add, FiniteDesign.E_sub, FiniteDesign.E_sub]
have hsym : (orderedDisjointPairDesign n M h2M).E
(fun P => (g P.1.1 - (slice n M hM).E g) *
(f P.1.2 - (slice n M hM).E f)) =
(orderedDisjointPairDesign n M h2M).E
(fun P => (f P.1.1 - (slice n M hM).E f) *
(g P.1.2 - (slice n M hM).E g)) :=
orderedDisjointPair_E_mul_comm h2M
(fun A => g A - (slice n M hM).E g)
(fun A => f A - (slice n M hM).E f)
rw [hsym]
ring
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma varianceMean_exchangeable {Ω : Type*} [Fintype Ω] (D : FiniteDesign Ω)
(G : ℕ) (hG : 0 < G) (X : Fin G → Ω → ℝ) (v c : ℝ)
(hvar : ∀ i, D.Var (X i) = v)
(hcov : ∀ i j, i ≠ j → D.Cov (X i) (X j) = c) :
D.Var (fun w => (∑ i, X i w) / (G : ℝ)) =
(v + ((G : ℝ) - 1) * c) / (G : ℝ) := by
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast hG.ne'
have hform : (fun w => (∑ i, X i w) / (G : ℝ)) =
fun w => ∑ i, (1 / (G : ℝ)) * X i w := by
funext w
rw [Finset.sum_div]
exact Finset.sum_congr rfl (fun i _ => by ring)
rw [D.Var_congr (congrFun hform), FiniteDesign.Var_linear_comb]
have hterms : (∑ i : Fin G, ∑ j : Fin G,
(1 / (G : ℝ)) * (1 / (G : ℝ)) * D.Cov (X i) (X j)) =
∑ i : Fin G, ∑ j : Fin G,
(1 / (G : ℝ)) * (1 / (G : ℝ)) * if i = j then v else c := by
apply Finset.sum_congr rfl
intro i _
apply Finset.sum_congr rfl
intro j _
by_cases hij : i = j
· subst j
rw [FiniteDesign.Cov_self, hvar i]
simp
· simp [hij, hcov i j hij]
rw [hterms, sum_sum_ite_quadratic univ
(fun _ : Fin G => 1 / (G : ℝ)) v c]
simp only [Finset.sum_const, Finset.card_univ, card_fin, nsmul_eq_mul]
field_simp
ring
theorem partition_mean_variance
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hG :
2 ≤ G
hG1 :
G1 ≤ G
f :
Omega n M → ℝ
conclusion
(uniformPartitionTuple n M G hMG).Var (fun T => (∑ g, f (T.1 g)) / (G : ℝ))
= ((slice n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).Var f + ((G : ℝ) - 1) * sliceInner n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG) (fun A => f A - (slice n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f) (kneserOp n M (fun A => f A - (slice n M (by exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f))) / (G : ℝ)
Proof (Lean source)
lemma partition_mean_variance (n M G G1 : ℕ) (hMG : M * G ≤ n)
(hG : 2 ≤ G) (hG1 : G1 ≤ G) (f : Omega n M → ℝ) :
(uniformPartitionTuple n M G hMG).Var
(fun T => (∑ g, f (T.1 g)) / (G : ℝ)) =
((slice n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).Var f +
((G : ℝ) - 1) *
sliceInner n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)
(fun A => f A - (slice n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f)
(kneserOp n M (fun A => f A - (slice n M (by
exact le_trans (le_mul_of_pos_right M (by omega)) hMG)).E f))) /
(G : ℝ) := by
let hM : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
have h2MG : 2 * M ≤ M * G := by
simpa [Nat.mul_comm] using Nat.mul_le_mul_left M hG
have h2M : 2 * M ≤ n := le_trans h2MG hMG
let D := uniformPartitionTuple n M G hMG
let v := (slice n M hM).Var f
let c := sliceInner n M hM (fun A => f A - (slice n M hM).E f)
(kneserOp n M (fun A => f A - (slice n M hM).E f))
have hvar (g : Fin G) : D.Var (fun T => f (T.1 g)) = v := by
change D.Var (fun T => f (T.1 g)) = (slice n M hM).Var f
rw [FiniteDesign.Var_eq, FiniteDesign.Var_eq]
rw [← randomPartition_E_partition n M G G1 hMG hG1]
have hsq := marginal_uniform_coord n M G G1 hMG (by omega) hG1 g
(fun A => f A ^ 2)
rw [show (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f (w.1.1 g) ^ 2) = (slice n M hM).E (fun A => f A ^ 2) by
simpa only [groupAt] using hsq]
rw [← randomPartition_E_partition n M G G1 hMG hG1]
have hm := marginal_uniform_coord n M G G1 hMG (by omega) hG1 g f
rw [show (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f (w.1.1 g)) = (slice n M hM).E f by
simpa only [groupAt] using hm]
have hcov (g g' : Fin G) (hgg' : g ≠ g') :
D.Cov (fun T => f (T.1 g)) (fun T => f (T.1 g')) = c := by
rw [FiniteDesign.Cov_eq]
rw [← randomPartition_E_partition n M G G1 hMG hG1]
have hp := marginal_uniform_ordered_disjoint_pair n M G G1 hMG hG1 h2M
g g' hgg' (fun P => f P.1.1 * f P.1.2)
rw [show (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f (w.1.1 g) * f (w.1.1 g')) =
(orderedDisjointPairDesign n M h2M).E
(fun P => f P.1.1 * f P.1.2) by simpa only [groupAt] using hp]
rw [← randomPartition_E_partition n M G G1 hMG hG1]
have hmg := marginal_uniform_coord n M G G1 hMG (by omega) hG1 g f
rw [show (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f (w.1.1 g)) = (slice n M hM).E f by
simpa only [groupAt] using hmg]
rw [← randomPartition_E_partition n M G G1 hMG hG1]
have hmg' := marginal_uniform_coord n M G G1 hMG (by omega) hG1 g' f
rw [show (randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f (w.1.1 g')) = (slice n M hM).E f by
simpa only [groupAt] using hmg']
let mu := (slice n M hM).E f
have hcenter : (orderedDisjointPairDesign n M h2M).E
(fun P => (f P.1.1 - mu) * (f P.1.2 - mu)) =
(orderedDisjointPairDesign n M h2M).E
(fun P => f P.1.1 * f P.1.2) - mu ^ 2 := by
have hp : (fun P : OrderedDisjointPair n M =>
(f P.1.1 - mu) * (f P.1.2 - mu)) =
fun P => f P.1.1 * f P.1.2 - mu * f P.1.1 - mu * f P.1.2 + mu ^ 2 := by
funext P
ring
rw [hp, FiniteDesign.E_add, FiniteDesign.E_sub, FiniteDesign.E_sub,
FiniteDesign.E_const_mul, FiniteDesign.E_const_mul,
orderedDisjointPair_first_E_eq_slice h2M,
orderedDisjointPair_second_E_eq_slice h2M, FiniteDesign.E_const]
change _ - mu * mu - mu * mu + mu ^ 2 = _ - mu ^ 2
ring
have hc : (orderedDisjointPairDesign n M h2M).E
(fun P => (f P.1.1 - mu) * (f P.1.2 - mu)) = c := by
change _ = sliceInner n M hM (fun A => f A - (slice n M hM).E f)
(kneserOp n M (fun A => f A - (slice n M hM).E f))
simpa only [mu] using
(orderedDisjointPair_E_eq n M h2M
(fun A => f A - (slice n M hM).E f)
(fun A => f A - (slice n M hM).E f))
rw [hcenter] at hc
simpa [mu, pow_two] using hc
exact varianceMean_exchangeable D G (by omega) (fun g T => f (T.1 g)) v c hvar hcov
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Kneser 24 declarations This file gives the paper's data-only projection family, normalized disjointness operator, spectral multipliers, covariance functionals, and two cited logical gates.
Johnson components and Kneser covariance
This file gives the paper's data-only projection family, normalized disjointness operator, spectral multipliers, covariance functionals, and two cited logical gates.
def johnsonHarmonicSpace
The canonical degree-k Johnson harmonic space: the degree-at-most-k space orthogonal to all lower-degree inclusion monomials.
Definition (Lean source)
Genuine orthogonal Johnson projections onto the canonical harmonic degree spaces, including the centered orthogonal decomposition asserted in the paper.
Definition (Lean source)
parameters
n M :
ℕ
fields
slice_nonempty :
M ≤ n
proj :
Fin (M + 1) → (Omega n M → ℝ) → (Omega n M → ℝ)
@realizes \Pi_{n,k}(orthogonal projection onto canonical degree k)
map_add :
∀ k f g, proj k (fun A => f A + g A) = fun A => proj k f A + proj k g A
map_smul :
∀ k (c : ℝ) f, proj k (fun A => c * f A) = fun A => c * proj k f A
range_eq :
∀ k, range (proj k) = johnsonHarmonicSpace n M slice_nonempty k
fixes_range :
∀ k f
if
f ∈ johnsonHarmonicSpace n M slice_nonempty k
then
proj k f = f
residual_orthogonal :
∀ k f g
if
g ∈ johnsonHarmonicSpace n M slice_nonempty k
then
sliceInner n M slice_nonempty (fun A => f A - proj k f A) g = 0
def canonicalJohnsonProjections
The canonical Johnson projection family supplied by the reusable Johnson--Kneser substrate.
Definition (Lean source)
parameters
n M :
ℕ
hM :
M ≤ n
def
given by
by let p : Fin (M + 1) → (Omega n M → ℝ) → (Omega n M → ℝ) := fun k f A
=> harmonicProjection n M k (toLp 2 f) A refine { slice_nonempty := hM proj := p map_add := ?_ map_smul := ?_ range_eq := ?_ fixes_range := ?_ residual_orthogonal := ?_ } · intro k f g funext A dsimp [p] exact congrArg (fun q => q A) (harmonicProjection_add k (toLp 2 f) (toLp 2 g)) · intro k c f funext A dsimp [p] exact congrArg (fun q => q A) (harmonicProjection_smul k c (toLp 2 f)) · intro k ext f constructor · rintro ⟨g, rfl⟩ exact harmonicProjection_mem k (toLp 2 g) · intro hf refine ⟨f, ?_⟩ funext A dsimp [p] exact congrArg (fun q => q A) ((harmonicProjection_eq_self_iff k (toLp 2 f)).2 hf) · intro k f hf funext A dsimp [p] exact congrArg (fun q => q A) ((harmonicProjection_eq_self_iff k (toLp 2 f)).2 hf) · intro k f g hg rw [sliceInner, slice] simp only [completeRandomization, FiniteDesign.E, one_div] rw [← Finset.mul_sum] exact sliceInner_residual_eq_zero hM k (toLp 2 f) (toLp 2 g) hg
def johnsonComponents
The degree-indexed component family of a slice function.
Definition (Lean source)
parameters
def
given by
fun k => J.proj k f
def kneserAdjacency
The unnormalized Kneser adjacency sum.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
f :
Omega n M → ℝ
A :
Omega n M
def
kneserAdjacency n M f A :
ℝ
given by
∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1), f C
theorem kneserDegree
When two groups fit in the population, every Kneser neighborhood has the stated cardinality.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
h2M :
2 * M ≤ n
A :
Omega n M
conclusion
((Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1).card) = (n - M).choose M
Proof (Lean source)
lemma kneserDegree (h2M : 2 * M ≤ n) (A : Omega n M) :
((Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1).card) =
(n - M).choose M := by
rw [← Fintype.card_coe]
let e : {C : Omega n M // Disjoint C.1 A.1} ≃
↥(powersetCard M A.1ᶜ) := {
toFun := fun C => ⟨C.1.1, by
rw [Finset.mem_powersetCard]
exact ⟨by
rw [Finset.subset_iff]
intro x hx
simp only [Finset.mem_compl]
exact Finset.disjoint_left.mp C.2 hx, C.1.2⟩⟩
invFun := fun C => ⟨⟨C.1, (Finset.mem_powersetCard.mp C.2).2⟩, by
rw [Finset.disjoint_left]
intro x hx
exact (by
have := (Finset.mem_powersetCard.mp C.2).1 hx
simpa using this)⟩
left_inv := by intro C; cases C; rfl
right_inv := by intro C; cases C; rfl }
let e0 : ↥(Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1) ≃
{C : Omega n M // Disjoint C.1 A.1} := {
toFun := fun C => ⟨C.1, (Finset.mem_filter.mp C.2).2⟩
invFun := fun C => ⟨C.1, Finset.mem_filter.mpr ⟨Finset.mem_univ _, C.2⟩⟩
left_inv := by intro C; cases C; rfl
right_inv := by intro C; cases C; rfl }
calc
card ↥(Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1) =
card {C : Omega n M // Disjoint C.1 A.1} := Fintype.card_congr e0
_ = card ↥(powersetCard M A.1ᶜ) := Fintype.card_congr e
_ = (n - M).choose M := by
rw [Fintype.card_coe, Finset.card_powersetCard,
Finset.card_compl, card_fin, A.2]
The normalized Kneser disjointness operator.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
f :
Omega n M → ℝ
A :
Omega n M
def
kneserOp n M f A :
ℝ
given by
kneserAdjacency n M f A / ((n - M).choose M : ℝ)
The normalized degree-k Kneser eigenvalue.
Definition (Lean source)
parameters
def
kneserEigenvalue n M k :
ℝ
given by
(-1 : ℝ) ^ k.1 * (M.descFactorial k.1 : ℝ) / ((n - M).descFactorial k.1 : ℝ)
Ordered-disjoint covariance of two arm tables.
Definition (Lean source)
parameters
def
crossCov n M hM Y a b :
ℝ
given by
Ordered-disjoint covariance of the arm contrast.
def OrderedDisjointPair
Ordered pairs of disjoint slice elements.
Definition (Lean source)
parameters
n M :
ℕ
def
OrderedDisjointPair n M :
Type
given by
{P : Omega n M × Omega n M // Disjoint P.1.1 P.2.1}
def uniformFiniteDesign
A uniform design on any inhabited finite type.
Definition (Lean source)
parameters
α :
def
given by
by classical exact { p := fun _ => 1 / (Fintype.card α : ℝ) p_nonneg := fun _ => one_div_nonneg.mpr (Nat.cast_nonneg _) p_sum := by rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul] have hc : (Fintype.card α : ℝ) ≠ 0 := by exact_mod_cast Fintype.card_ne_zero field_simp [hc] }
theorem orderedDisjointPair_nonempty
If twice the group size does not exceed the population, then the ordered-disjoint-pair type is nonempty.
Formal statement
Proof (Lean source)
lemma orderedDisjointPair_nonempty (h2M : 2 * M ≤ n) :
Nonempty (OrderedDisjointPair n M) := by
classical
have hMn : M ≤ n := by omega
obtain ⟨A, -, hAcard⟩ :=
Finset.exists_subset_card_eq (s := (Finset.univ : Finset (Fin n)))
(n := M) (by simpa using hMn)
have hMcomp : M ≤ Aᶜ.card := by
rw [Finset.card_compl, card_fin, hAcard]
omega
obtain ⟨C, hCsub, hCcard⟩ := Finset.exists_subset_card_eq hMcomp
exact ⟨⟨(⟨A, hAcard⟩, ⟨C, hCcard⟩), by
rw [Finset.disjoint_left]
intro x hxA hxC
have hxcomp := hCsub hxC
simpa [hxA] using hxcomp⟩⟩
The uniform ordered-disjoint-pair design.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
h2M :
2 * M ≤ n
def
given by
by letI : Nonempty (OrderedDisjointPair n M) := orderedDisjointPair_nonempty n M h2M exact uniformFiniteDesign (OrderedDisjointPair n M)
def orderedPairSigmaEquiv
Ordered disjoint pairs are a dependent pair of a slice element and a disjoint second slice element.
Definition (Lean source)
parameters
n M :
ℕ
def
orderedPairSigmaEquiv n M :
OrderedDisjointPair n M ≃ Σ A : Omega n M, {C : Omega n M // Disjoint C.1 A.1}
theorem orderedDisjointFiber_card
When two groups fit in the population, the fiber of groups disjoint from a fixed group has the Kneser degree.
Formal statement
Proof (Lean source)
-- @node: orderedDisjointFiber_card
lemma orderedDisjointFiber_card (h2M : 2 * M ≤ n) (A : Omega n M) :
card {C : Omega n M // Disjoint C.1 A.1} = (n - M).choose M := by
let e : {C : Omega n M // Disjoint C.1 A.1} ≃
↥(Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1) := {
toFun := fun C => ⟨C.1, Finset.mem_filter.mpr ⟨Finset.mem_univ _, C.2⟩⟩
invFun := fun C => ⟨C.1, (Finset.mem_filter.mp C.2).2⟩
left_inv := by intro C; cases C; rfl
right_inv := by intro C; cases C; rfl }
rw [Fintype.card_congr e, Fintype.card_coe]
exact kneserDegree n M h2M A
When two groups fit in the population, the ordered-disjoint-pair space has slice cardinality times Kneser degree.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
h2M :
2 * M ≤ n
conclusion
card (OrderedDisjointPair n M) = n.choose M * (n - M).choose M
Proof (Lean source)
-- @node: orderedDisjointPair_card
lemma orderedDisjointPair_card (h2M : 2 * M ≤ n) :
card (OrderedDisjointPair n M) = n.choose M * (n - M).choose M := by
rw [Fintype.card_congr (orderedPairSigmaEquiv n M), Fintype.card_sigma]
simp_rw [orderedDisjointFiber_card n M h2M]
simp [Fintype.card_finset_len]
theorem orderedDisjointPair_sum
For two slice functions, a sum over ordered disjoint pairs is the corresponding iterated slice-and-neighborhood sum.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
f g :
Omega n M → ℝ
conclusion
(∑ P : OrderedDisjointPair n M, f P.1.1 * g P.1.2)
= ∑ A : Omega n M, ∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1), f A * g C
Proof (Lean source)
-- @node: orderedDisjointPair_sum
lemma orderedDisjointPair_sum (f g : Omega n M → ℝ) :
(∑ P : OrderedDisjointPair n M, f P.1.1 * g P.1.2) =
∑ A : Omega n M,
∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1),
f A * g C := by
calc
(∑ P : OrderedDisjointPair n M, f P.1.1 * g P.1.2) =
∑ Q : Σ A : Omega n M, {C : Omega n M // Disjoint C.1 A.1},
f Q.1 * g Q.2.1 := by
exact Fintype.sum_equiv (orderedPairSigmaEquiv n M) _ _ (fun _ => rfl)
_ = ∑ A : Omega n M, ∑ C : {C : Omega n M // Disjoint C.1 A.1},
f A * g C.1 := by
apply Fintype.sum_sigma
_ = ∑ A : Omega n M,
∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1),
f A * g C := by
apply Finset.sum_congr rfl
intro A _
let e : {C : Omega n M // Disjoint C.1 A.1} ≃
↥(Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1) := {
toFun := fun C => ⟨C.1, Finset.mem_filter.mpr ⟨Finset.mem_univ _, C.2⟩⟩
invFun := fun C => ⟨C.1, (Finset.mem_filter.mp C.2).2⟩
left_inv := by intro C; cases C; rfl
right_inv := by intro C; cases C; rfl }
calc
(∑ C : {C : Omega n M // Disjoint C.1 A.1}, f A * g C.1) =
∑ C : ↥(Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1),
f A * g C.1 := by
exact Fintype.sum_equiv e _ _ (fun _ => rfl)
_ = ∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1),
f A * g C := by
change (∑ C ∈ (Finset.univ.filter fun C : Omega n M =>
Disjoint C.1 A.1).attach, f A * g C.1) = _
exact Finset.sum_attach _ (fun C => f A * g C)
theorem orderedDisjointPair_E_eq
When two groups fit in the population, ordered-pair expectation agrees with the Kneser inner-product formula.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
h2M :
2 * M ≤ n
f g :
Omega n M → ℝ
conclusion
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1 * g P.1.2)
= sliceInner n M (by omega) f (kneserOp n M g)
Proof (Lean source)
-- @node: orderedDisjointPair_E_eq
lemma orderedDisjointPair_E_eq (h2M : 2 * M ≤ n) (f g : Omega n M → ℝ) :
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1 * g P.1.2) =
sliceInner n M (by omega) f (kneserOp n M g) := by
classical
have hnchoose : n.choose M ≠ 0 := Nat.choose_ne_zero (by omega)
have hdegchoose : (n - M).choose M ≠ 0 := Nat.choose_ne_zero (by omega)
simp only [orderedDisjointPairDesign, uniformFiniteDesign, FiniteDesign.E,
sliceInner, slice, completeRandomization, kneserOp, kneserAdjacency]
rw [← Finset.mul_sum]
rw [orderedDisjointPair_sum n M f g, orderedDisjointPair_card n M h2M]
simp only [Fintype.card_finset_len, card_fin]
have hcollapse :
(∑ A : Omega n M,
∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1),
f A * g C) =
∑ A : Omega n M, f A *
∑ C ∈ (Finset.univ.filter fun C : Omega n M => Disjoint C.1 A.1), g C := by
apply Finset.sum_congr rfl
intro A _
rw [Finset.mul_sum]
rw [hcollapse]
push_cast
rw [← Finset.mul_sum]
simp_rw [div_eq_mul_inv, mul_assoc]
simp_rw [← mul_assoc]
rw [← Finset.sum_mul]
field_simp
theorem kneserChooseRatio_eq_descFactorialRatio Lemma kneserChooseRatio_eq_descFactorialRatio in the paper ↗
When two groups fit in the population, the quotient of unnormalized Kneser eigenvalue magnitudes is the ratio of the corresponding falling factorials.
Formal statement
hypotheses
conclusion
(((n - M - k.1).choose (M - k.1) : ℕ) : ℝ) / (((n - M).choose M : ℕ) : ℝ)
= (M.descFactorial k.1 : ℝ) / ((n - M).descFactorial k.1 : ℝ)
Proof (Lean source)
-- @node: kneserChooseRatio_eq_descFactorialRatio
lemma kneserChooseRatio_eq_descFactorialRatio (h2M : 2 * M ≤ n)
(k : Fin (M + 1)) :
(((n - M - k.1).choose (M - k.1) : ℕ) : ℝ) /
(((n - M).choose M : ℕ) : ℝ) =
(M.descFactorial k.1 : ℝ) / ((n - M).descFactorial k.1 : ℝ) := by
have hkM : k.1 ≤ M := by omega
have hMn : M ≤ n - M := by omega
have hkden : 0 < (n - M).descFactorial k.1 :=
Nat.descFactorial_pos.mpr (by omega)
have hchoose : 0 < (n - M).choose M := Nat.choose_pos hMn
have hsplit := Nat.descFactorial_mul_descFactorial (n := n - M) hkM
have hfac := Nat.factorial_mul_descFactorial (n := M) hkM
have hnum := Nat.descFactorial_eq_factorial_mul_choose (n - M - k.1) (M - k.1)
have htotal := Nat.descFactorial_eq_factorial_mul_choose (n - M) M
have hcrossNat :
(n - M - k.1).choose (M - k.1) * (n - M).descFactorial k.1 =
M.descFactorial k.1 * (n - M).choose M := by
apply Nat.eq_of_mul_eq_mul_left (Nat.factorial_pos (M - k.1))
calc
(M - k.1).factorial *
((n - M - k.1).choose (M - k.1) * (n - M).descFactorial k.1) =
(n - M - k.1).descFactorial (M - k.1) *
(n - M).descFactorial k.1 := by rw [hnum]; ac_rfl
_ = (n - M).descFactorial M := hsplit
_ = M.factorial * (n - M).choose M := htotal
_ = ((M - k.1).factorial * M.descFactorial k.1) *
(n - M).choose M := by rw [hfac]
_ = (M - k.1).factorial *
(M.descFactorial k.1 * (n - M).choose M) := by ac_rfl
field_simp
exact_mod_cast hcrossNat.trans (by ac_rfl)
Yuval Filmus (2016), An Orthogonal Basis for Functions over a Slice of the Boolean Hypercube, Theorem 4.1 and Lemma 4.3, arXiv:1406.0142v2, pp. 10 and 12. The cited result supplies the orthogonal direct-sum decomposition of functions on the uniform slice and the Bose--Mesner eigenspace identification.
Definition (Lean source)
parameters
n M :
ℕ
J :
def
JohnsonOrthogonalDecomposition n M J :
Sort 0
Andries E. Brouwer, Sebastian M. Cioaba, Ferdinand Ihringer, and Matt McGinnis (2018), The Smallest Eigenvalues of Hamming Graphs, Johnson Graphs and Other Distance-Regular Graphs with Classical Parameters, Proposition 3.1, arXiv:1709.09011, p. 10. It gives the unnormalized Kneser adjacency eigenvalue (-1)^k * choose (n-M-k) (M-k).
Definition (Lean source)
parameters
n M :
ℕ
def
KneserAdjacencySpectrum n M :
Sort 0
given by
∀ h2M : 2 * M ≤ n,
∀ (k : Fin (M + 1)) (f : Omega n M → ℝ),
f ∈ johnsonHarmonicSpace n M (by omega) k → kneserAdjacency n M f
= fun A => ((-1 : ℝ) ^ k.1 * ((n - M - k.1).choose (M - k.1) : ℝ)) * f A
For a feasible slice size, the canonical projection family has the Johnson orthogonal decomposition required here.
Formal statement
hypotheses
n M :
ℕ
hM :
M ≤ n
conclusion
Proof (Lean source)
-- @node: lem:classical-johnson-decomposition
lemma canonicalJohnsonOrthogonalDecomposition (n M : ℕ) (hM : M ≤ n) :
JohnsonOrthogonalDecomposition n M (canonicalJohnsonProjections n M hM) := by
intro h2M
constructor
· intro f A
have h := sum_positive_harmonicProjection_eq_center (by omega)
(toLp 2 f)
let A' : Omega n M :=
⟨A.1, A.2⟩
have hA := congrFun (congrArg ofLp h) A'
simp only [WithLp.ofLp_sum] at hA
rw [Finset.sum_apply A'
(Finset.univ.filter (fun i : Fin (M + 1) => 0 < i.1))] at hA
simp [center, constFn] at hA
have hE : (slice n M (by omega)).E f = mean (toLp 2 f) := by
rw [slice]
simp only [completeRandomization, FiniteDesign.E, one_div, mean]
rw [completeRandomization_card, card_fin,
card_omega (by omega),
← Finset.mul_sum]
rfl
rw [hE]
simpa [canonicalJohnsonProjections, center, constFn, Finset.sum_apply, A',
Pi.sub_apply] using hA
· intro k l f g hkl
rw [sliceInner, slice]
simp only [completeRandomization, FiniteDesign.E, one_div]
rw [← Finset.mul_sum]
exact johnsonHarmonic_pairwise_orthogonal (by omega) hkl
(harmonicProjection_mem k (toLp 2 f))
(harmonicProjection_mem l (toLp 2 g))
The canonical Kneser adjacency operator has the stated harmonic spectrum.
Formal statement
Proof (Lean source)
-- @node: lem:classical-kneser-spectrum
lemma canonicalKneserAdjacencySpectrum (n M : ℕ) :
KneserAdjacencySpectrum n M := by
intro h2M k f hf
have h := kneserAdjacency_eigen
h2M k (toLp 2 f) hf
funext A
have hA := congrFun (congrArg ofLp h) A
simp only [WithLp.ofLp_smul] at hA
change (∑ x, if Disjoint A.1 x.1 then f x else 0) =
((-1 : ℝ) ^ k.1 * ((n - M - k.1).choose (M - k.1) : ℝ)) * f A at hA
simpa [kneserAdjacency,
kneserAdjacency,
Finset.sum_filter, disjoint_comm,
mul_ite, PiLp.smul_apply, Pi.smul_apply, smul_eq_mul] using hA
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.PartitionDesign 15 declarations Only the uniform law on ordered disjoint group tuples is new.
Uniform ordered partitions and the independent two-stage design
Only the uniform law on ordered disjoint group tuples is new. The independent combination with complete treatment randomization uses the existing dependent product design and its pushforward operation.
def PartitionTuple
Ordered tuples of pairwise-disjoint groups.
def TreatmentSpace
Complete treatment assignments to the realized groups.
theorem partitionTuple_nonempty
If the requested groups fit in the population, the ordered partition space is nonempty.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
G :
ℕ
hMG :
M * G ≤ n
conclusion
Nonempty (PartitionTuple n M G)
Proof (Lean source)
lemma partitionTuple_nonempty (hMG : M * G ≤ n) : Nonempty (PartitionTuple n M G) := by
classical
have hGM : G * M ≤ n := by simpa [Nat.mul_comm] using hMG
let E : Fin G × Fin M ↪ Fin n :=
finProdFinEquiv.toEmbedding.trans (Fin.castLEEmb hGM)
let e (g : Fin G) : Fin M ↪ Fin n :=
(⟨fun i => (g, i), by
intro i j hij
exact congrArg Prod.snd hij⟩ : Fin M ↪ Fin G × Fin M).trans E
let T : Fin G → Omega n M := fun g =>
⟨Finset.univ.map (e g), by simp⟩
exact ⟨⟨T, by
intro g g' hgg'
rw [Finset.disjoint_left]
intro x hx hx'
simp only [T, Finset.mem_map] at hx hx'
obtain ⟨i, -, rfl⟩ := hx
obtain ⟨j, -, hij⟩ := hx'
change E (g', j) = E (g, i) at hij
have hp : (g, i) = (g', j) := E.injective hij.symm
exact hgg' (congrArg fst hp)⟩⟩
def uniformPartitionTuple
The uniform first-stage law.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
G :
ℕ
hMG :
M * G ≤ n
def
given by
by letI : Nonempty (PartitionTuple n M G) := partitionTuple_nonempty n M G hMG exact uniformFiniteDesign (PartitionTuple n M G)
def twoStageSpace
The two dependent coordinate spaces: partition tuple, then treatment set.
Definition (Lean source)
parameters
n M G G1 :
ℕ
def
instance twoStageSpaceFintype
For the population and group-count parameters and a stage indicator, the two-stage sample space has a finite enumeration.
Definition (Lean source)
noncomputable instance twoStageSpaceFintype (i : Bool) :
Fintype (twoStageSpace n M G G1 i) := by
cases i <;> simp only [twoStageSpace] <;> infer_instance
def twoStageDesigns
Coordinate designs for the dependent product.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
hMG :
M * G ≤ n
hG1 :
G1 ≤ G
i :
def
given by
by intro i cases i · simpa only [twoStageSpace] using uniformPartitionTuple n M G hMG · simpa only [twoStageSpace] using completeRandomization G1 (by simpa using hG1)
The two Boolean-indexed stage coordinates, repackaged as an ordinary pair.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
def
twoStagePairEquiv n M G G1 :
Independent uniform partition and complete group-treatment randomization.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
hMG :
M * G ≤ n
hG1 :
G1 ≤ G
def
given by
Relabeling population units by a permutation relabels ordered partition tuples.
Definition (Lean source)
parameters
def
given by
clause 1
toFun T := ⟨fun g
=> ⟨T.1 g |>.1.map σ.toEmbedding, by simp [(T.1 g).2]⟩, by intro g g' hgg' rw [Finset.disjoint_left] intro x hx hx' simp only [Finset.mem_map] at hx hx' obtain ⟨i, hi, rfl⟩ := hx obtain ⟨j, hj, hij⟩ := hx' exact Finset.disjoint_left.mp (T.2 g g' hgg') hi (σ.injective hij.symm ▸ hj)⟩
clause 2
invFun T := ⟨fun g
=> ⟨T.1 g |>.1.map σ.symm.toEmbedding, by simp [(T.1 g).2]⟩, by intro g g' hgg' rw [Finset.disjoint_left] intro x hx hx' simp only [Finset.mem_map] at hx hx' obtain ⟨i, hi, rfl⟩ := hx obtain ⟨j, hj, hij⟩ := hx' exact Finset.disjoint_left.mp (T.2 g g' hgg') hi (σ.symm.injective hij.symm ▸ hj)⟩
Relabeling population units by a permutation relabels ordered disjoint pairs.
Definition (Lean source)
parameters
def
given by
clause 1
toFun P := ⟨(⟨P.1.1.1.map σ.toEmbedding, by simp [P.1.1.2]⟩, ⟨P.1.2.1.map σ.toEmbedding, by simp [P.1.2.2]⟩), by rw [Finset.disjoint_left] intro x hx hx' simp only [Finset.mem_map] at hx hx' obtain ⟨i, hi, rfl⟩ := hx obtain ⟨j, hj, hij⟩ := hx' exact Finset.disjoint_left.mp P.2 hi (σ.injective hij.symm ▸ hj)⟩
clause 2
invFun P := ⟨(⟨P.1.1.1.map σ.symm.toEmbedding, by simp [P.1.1.2]⟩, ⟨P.1.2.1.map σ.symm.toEmbedding, by simp [P.1.2.2]⟩), by rw [Finset.disjoint_left] intro x hx hx' simp only [Finset.mem_map] at hx hx' obtain ⟨i, hi, rfl⟩ := hx obtain ⟨j, hj, hij⟩ := hx' exact Finset.disjoint_left.mp P.2 hi (σ.symm.injective hij.symm ▸ hj)⟩
For two ordered disjoint pairs, one population relabeling maps the first pair to the second.
Formal statement
hypotheses
Proof (Lean source)
lemma orderedDisjointPair_perm_exists (P Q : OrderedDisjointPair n M) :
∃ σ : Perm (Fin n),
P.1.1.1.map σ.toEmbedding = Q.1.1.1 ∧
P.1.2.1.map σ.toEmbedding = Q.1.2.1 := by
classical
let e1 : ↥P.1.1.1 ≃ ↥Q.1.1.1 := P.1.1.1.equivOfCardEq (by rw [P.1.1.2, Q.1.1.2])
let e2 : ↥P.1.2.1 ≃ ↥Q.1.2.1 := P.1.2.1.equivOfCardEq (by rw [P.1.2.2, Q.1.2.2])
let f : ↥P.1.1.1 ⊕ ↥P.1.2.1 → Fin n := Sum.elim Subtype.val Subtype.val
let q : ↥P.1.1.1 ⊕ ↥P.1.2.1 → Fin n :=
Sum.elim (fun i => (e1 i).1) (fun i => (e2 i).1)
have hf : Injective f := by
intro x y hxy
cases x with
| inl x =>
cases y with
| inl y => exact congrArg inl (Subtype.ext hxy)
| inr y =>
change x.1 = y.1 at hxy
have hy : x.1 ∈ P.1.2.1 := hxy.symm ▸ y.2
exact elim (Finset.disjoint_left.mp P.2 x.2 hy)
| inr x =>
cases y with
| inl y =>
change x.1 = y.1 at hxy
have hx : y.1 ∈ P.1.2.1 := hxy ▸ x.2
exact elim (Finset.disjoint_left.mp P.2 y.2 hx)
| inr y => exact congrArg inr (Subtype.ext hxy)
have hq : Injective q := by
intro x y hxy
cases x with
| inl x =>
cases y with
| inl y => exact congrArg inl (e1.injective (Subtype.ext hxy))
| inr y =>
change (e1 x).1 = (e2 y).1 at hxy
have hy : (e1 x).1 ∈ Q.1.2.1 := hxy.symm ▸ (e2 y).2
exact elim (Finset.disjoint_left.mp Q.2 (e1 x).2 hy)
| inr x =>
cases y with
| inl y =>
change (e2 x).1 = (e1 y).1 at hxy
have hx : (e1 y).1 ∈ Q.1.2.1 := hxy ▸ (e2 x).2
exact elim (Finset.disjoint_left.mp Q.2 (e1 y).2 hx)
| inr y => exact congrArg inr (e2.injective (Subtype.ext hxy))
obtain ⟨σ, hσ⟩ := Equiv.Perm.exists_extending_pair f q hf hq
refine ⟨σ, ?_, ?_⟩
· apply Finset.Subset.antisymm
· intro x hx
obtain ⟨i, hi, rfl⟩ := Finset.mem_map.mp hx
have hiσ := hσ (inl ⟨i, hi⟩)
change σ i = (e1 ⟨i, hi⟩).1 at hiσ
change σ i ∈ Q.1.1.1
rw [hiσ]
exact (e1 ⟨i, hi⟩).2
· intro x hx
let j : ↥Q.1.1.1 := ⟨x, hx⟩
let i := e1.symm j
exact Finset.mem_map.mpr ⟨i.1, i.2, by
have hiσ := hσ (inl i)
change σ i.1 = (e1 i).1 at hiσ
simpa [i, j] using hiσ⟩
· apply Finset.Subset.antisymm
· intro x hx
obtain ⟨i, hi, rfl⟩ := Finset.mem_map.mp hx
have hiσ := hσ (inr ⟨i, hi⟩)
change σ i = (e2 ⟨i, hi⟩).1 at hiσ
change σ i ∈ Q.1.2.1
rw [hiσ]
exact (e2 ⟨i, hi⟩).2
· intro x hx
let j : ↥Q.1.2.1 := ⟨x, hx⟩
let i := e2.symm j
exact Finset.mem_map.mpr ⟨i.1, i.2, by
have hiσ := hσ (inr i)
change σ i.1 = (e2 i).1 at hiσ
simpa [i, j] using hiσ⟩
def groupAt
The realized group in coordinate g.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
w :
PartitionTuple n M G × TreatmentSpace G G1
g :
Fin G
def
groupAt n M G G1 w g :
Omega n M
given by
w.1.1 g
theorem marginal_uniform_coord
When the groups fit in the population, there is at least one group, and the treatment count is feasible, each group coordinate is marginally uniform on the slice.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
hMG :
M * G ≤ n
hG :
0 < G
hG1 :
G1 ≤ G
g :
Fin G
f :
Omega n M → ℝ
conclusion
(randomPartitionDesign n M G G1 hMG hG1).E (fun w => f (groupAt n M G G1 w g))
Proof (Lean source)
lemma marginal_uniform_coord (hMG : M * G ≤ n) (hG : 0 < G) (hG1 : G1 ≤ G)
(g : Fin G) (f : Omega n M → ℝ) :
(randomPartitionDesign n M G G1 hMG hG1).E (fun w => f (groupAt n M G G1 w g)) =
(slice n M (le_trans (le_mul_of_pos_right M hG) hMG)).E f := by
classical
let hM : M ≤ n := le_trans (le_mul_of_pos_right M hG) hMG
let D := uniformPartitionTuple n M G hMG
have hprod :
(randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f (groupAt n M G G1 w g)) =
D.E (fun T => f (T.1 g)) := by
rw [randomPartitionDesign, FiniteDesign.E_map]
exact FiniteDesign.E_prod_apply (twoStageDesigns n M G G1 hMG hG1) false
(fun T => f (T.1 g))
rw [hprod]
let A0 : Omega n M := (choice (partitionTuple_nonempty n M G hMG)).1 g
let c : ℕ := card {T : PartitionTuple n M G // T.1 g = A0}
have hfiber : ∀ A : Omega n M,
card {T : PartitionTuple n M G // T.1 g = A} = c := by
intro A
obtain ⟨σ, hσ⟩ := Equiv.Perm.exists_map_finset_eq A.1 A0.1 (by rw [A.2, A0.2])
let e : {T : PartitionTuple n M G // T.1 g = A} ≃
{T : PartitionTuple n M G // T.1 g = A0} :=
(partitionTuplePermEquiv n M G σ).subtypeEquiv
(by
intro T
constructor
· intro hT
apply Subtype.ext
simpa [partitionTuplePermEquiv, hT] using hσ
· intro hm
apply Subtype.ext
apply Finset.map_injective σ.toEmbedding
have hm' := congrArg (fun S : Omega n M => S.1) hm
simpa [partitionTuplePermEquiv, hσ] using hm'
)
exact Fintype.card_congr e
have hcpos : 0 < c := by
let T0 : PartitionTuple n M G := choice (partitionTuple_nonempty n M G hMG)
have hnon : Nonempty {T : PartitionTuple n M G // T.1 g = T0.1 g} :=
⟨⟨T0, rfl⟩⟩
rw [← hfiber (T0.1 g)]
exact Fintype.card_pos
have hsum : (∑ T : PartitionTuple n M G, f (T.1 g)) = c * ∑ A : Omega n M, f A := by
rw [← Fintype.sum_fiberwise (fun T : PartitionTuple n M G => T.1 g)
(fun T => f (T.1 g))]
simp_rw [show ∀ A : Omega n M,
(∑ T : {T : PartitionTuple n M G // T.1 g = A}, f (T.1.1 g)) =
c * f A by
intro A
calc
(∑ T : {T : PartitionTuple n M G // T.1 g = A}, f (T.1.1 g)) =
∑ _T : {T : PartitionTuple n M G // T.1 g = A}, f A := by
apply Finset.sum_congr rfl
intro T _
rw [T.2]
_ = c * f A := by simp [nsmul_eq_mul, hfiber A]
]
rw [Finset.mul_sum]
have hcard : card (PartitionTuple n M G) = c * card (Omega n M) := by
calc
card (PartitionTuple n M G) =
∑ A : Omega n M,
card {T : PartitionTuple n M G // T.1 g = A} := by
simpa using Fintype.card_congr
(Equiv.sigmaFiberEquiv (fun T : PartitionTuple n M G => T.1 g)).symm
_ = c * card (Omega n M) := by simp [hfiber, Nat.mul_comm]
simp only [D, uniformPartitionTuple, uniformFiniteDesign, FiniteDesign.E, slice,
completeRandomization]
rw [← Finset.mul_sum, ← Finset.mul_sum]
rw [hsum, hcard]
simp only [Fintype.card_finset_len]
push_cast
field_simp [show (c : ℝ) ≠ 0 by exact_mod_cast hcpos.ne']
theorem marginal_uniform_ordered_disjoint_pair
When the groups and treatment count are feasible, two groups fit in the population, and the selected coordinates differ, the two coordinates are uniform over ordered disjoint pairs.
Formal statement
hypotheses
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
hMG :
M * G ≤ n
hG1 :
G1 ≤ G
h2M :
2 * M ≤ n
g g' :
Fin G
hgg' :
g ≠ g'
f :
OrderedDisjointPair n M → ℝ
conclusion
(randomPartitionDesign n M G G1 hMG hG1).E (fun w => f ⟨(groupAt n M G G1 w g, groupAt n M G G1 w g'), w.1.2 g g' hgg'⟩)
= (orderedDisjointPairDesign n M h2M).E f
Proof (Lean source)
lemma marginal_uniform_ordered_disjoint_pair (hMG : M * G ≤ n) (hG1 : G1 ≤ G)
(h2M : 2 * M ≤ n) (g g' : Fin G) (hgg' : g ≠ g')
(f : OrderedDisjointPair n M → ℝ) :
(randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f ⟨(groupAt n M G G1 w g, groupAt n M G G1 w g'),
w.1.2 g g' hgg'⟩) =
(orderedDisjointPairDesign n M h2M).E f := by
classical
let D := uniformPartitionTuple n M G hMG
let pairAt (T : PartitionTuple n M G) : OrderedDisjointPair n M :=
⟨(T.1 g, T.1 g'), T.2 g g' hgg'⟩
have hprod :
(randomPartitionDesign n M G G1 hMG hG1).E
(fun w => f ⟨(groupAt n M G G1 w g, groupAt n M G G1 w g'),
w.1.2 g g' hgg'⟩) = D.E (fun T => f (pairAt T)) := by
rw [randomPartitionDesign, FiniteDesign.E_map]
exact FiniteDesign.E_prod_apply (twoStageDesigns n M G G1 hMG hG1) false
(fun T => f (pairAt T))
rw [hprod]
let P0 : OrderedDisjointPair n M :=
pairAt (choice (partitionTuple_nonempty n M G hMG))
let c : ℕ := card {T : PartitionTuple n M G // pairAt T = P0}
have hfiber : ∀ P : OrderedDisjointPair n M,
card {T : PartitionTuple n M G // pairAt T = P} = c := by
intro P
obtain ⟨σ, hσ1, hσ2⟩ := orderedDisjointPair_perm_exists n M P P0
have hσP : orderedDisjointPairPermEquiv n M σ P = P0 := by
apply Subtype.ext
apply Prod.ext <;> apply Subtype.ext
· exact hσ1
· exact hσ2
let e : {T : PartitionTuple n M G // pairAt T = P} ≃
{T : PartitionTuple n M G // pairAt T = P0} :=
(partitionTuplePermEquiv n M G σ).subtypeEquiv (by
intro T
have hcomm : pairAt (partitionTuplePermEquiv n M G σ T) =
orderedDisjointPairPermEquiv n M σ (pairAt T) := rfl
rw [hcomm, ← hσP]
exact (orderedDisjointPairPermEquiv n M σ).injective.eq_iff.symm)
exact Fintype.card_congr e
have hcpos : 0 < c := by
let T0 : PartitionTuple n M G := choice (partitionTuple_nonempty n M G hMG)
have hnon : Nonempty {T : PartitionTuple n M G // pairAt T = pairAt T0} :=
⟨⟨T0, rfl⟩⟩
rw [← hfiber (pairAt T0)]
exact Fintype.card_pos
have hsum : (∑ T : PartitionTuple n M G, f (pairAt T)) =
c * ∑ P : OrderedDisjointPair n M, f P := by
rw [← Fintype.sum_fiberwise pairAt (fun T => f (pairAt T))]
simp_rw [show ∀ P : OrderedDisjointPair n M,
(∑ T : {T : PartitionTuple n M G // pairAt T = P}, f (pairAt T.1)) =
c * f P by
intro P
calc
(∑ T : {T : PartitionTuple n M G // pairAt T = P}, f (pairAt T.1)) =
∑ _T : {T : PartitionTuple n M G // pairAt T = P}, f P := by
apply Finset.sum_congr rfl
intro T _
rw [T.2]
_ = c * f P := by simp [nsmul_eq_mul, hfiber P]]
rw [Finset.mul_sum]
have hcard : card (PartitionTuple n M G) =
c * card (OrderedDisjointPair n M) := by
calc
card (PartitionTuple n M G) =
∑ P : OrderedDisjointPair n M,
card {T : PartitionTuple n M G // pairAt T = P} := by
simpa using Fintype.card_congr (Equiv.sigmaFiberEquiv pairAt).symm
_ = c * card (OrderedDisjointPair n M) := by
simp [hfiber, Nat.mul_comm]
simp only [D, uniformPartitionTuple, uniformFiniteDesign, FiniteDesign.E,
orderedDisjointPairDesign]
rw [← Finset.mul_sum, ← Finset.mul_sum, hsum, hcard]
push_cast
field_simp [show (c : ℝ) ≠ 0 by exact_mod_cast hcpos.ne']
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.QVDiagonalAssembly 4 declarations Finite-design conditioning and deterministic-array membership facts used by the one-realization impossibility argument.
Diagonal lower-bound assembly helpers
Finite-design conditioning and deterministic-array membership facts used by the one-realization impossibility argument.
def withSchedule
Replace only the deterministic schedule in a fixed design skeleton.
Definition (Lean source)
parameters
def
given by
clause 1
groupSize_ge_two := A.groupSize_ge_two
clause 2
popSize := A.popSize
clause 3
groups := A.groups
clause 4
treated := A.treated
clause 5
groups_pos := A.groups_pos
clause 6
grouped_le := A.grouped_le
clause 7
treated_pos := A.treated_pos
clause 8
treated_lt := A.treated_lt
clause 9
treated_le := A.treated_le
clause 10
schedule := Y
theorem abs_conditionedExpectation_sub_le_two_compl Lemma abs_conditionedExpectation_sub_le_two_compl in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
|D.E (fun w => if w ∈ Γ then f w else 0) / D.Pr (fun w => w ∈ Γ) - D.E f|
≤ 2 * (1 - D.Pr (fun w => w ∈ Γ))
Proof (Lean source)
lemma abs_conditionedExpectation_sub_le_two_compl
{Ω : Type*} [Fintype Ω] [DecidableEq Ω]
(D : FiniteDesign Ω) (Γ : Finset Ω) (f : Ω → ℝ)
(hf : ∀ w, 0 ≤ f w ∧ f w ≤ 1)
(hq : 0 < D.Pr (fun w => w ∈ Γ)) :
|D.E (fun w => if w ∈ Γ then f w else 0) /
D.Pr (fun w => w ∈ Γ) - D.E f| ≤
2 * (1 - D.Pr (fun w => w ∈ Γ)) := by
let q := D.Pr (fun w => w ∈ Γ)
let a := D.E (fun w => if w ∈ Γ then f w else 0)
let b := D.E (fun w => if w ∈ Γ then 0 else f w)
have hqpos : 0 < q := by simpa [q] using hq
have ha0 : 0 ≤ a := by
dsimp [a]
apply D.E_nonneg
intro w
by_cases hw : w ∈ Γ <;> simp [hw, (hf w).1]
have hb0 : 0 ≤ b := by
dsimp [b]
apply D.E_nonneg
intro w
by_cases hw : w ∈ Γ <;> simp [hw, (hf w).1]
have haq : a ≤ q := by
unfold a q FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
apply Finset.sum_le_sum
intro w _
by_cases hw : w ∈ Γ
· simpa [hw] using mul_le_mul_of_nonneg_left (hf w).2 (D.p_nonneg w)
· simp [hw]
have hbq : b ≤ 1 - q := by
have hcomp : D.Pr (fun w => w ∉ Γ) = 1 - q := by
unfold q FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
rw [← D.p_sum, ← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro w _
by_cases hw : w ∈ Γ <;> simp [hw, D.p_sum]
rw [← hcomp]
unfold b FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
apply Finset.sum_le_sum
intro w _
by_cases hw : w ∈ Γ
· simp [hw]
· simpa [hw] using mul_le_mul_of_nonneg_left (hf w).2 (D.p_nonneg w)
have hef : D.E f = a + b := by
unfold a b FiniteDesign.E
rw [← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro w _
by_cases hw : w ∈ Γ <;> simp [hw]
have hq1 : q ≤ 1 := D.Pr_le_one _
have hx0 : 0 ≤ a * (1 - q) / q :=
div_nonneg (mul_nonneg ha0 (sub_nonneg.mpr hq1)) hqpos.le
have hxq : a * (1 - q) / q ≤ 1 - q := by
rw [div_le_iff₀ hqpos]
nlinarith
rw [hef]
have hid : a / q - (a + b) = a * (1 - q) / q - b := by
field_simp [hqpos.ne']
ring
rw [hid, abs_le]
constructor <;> nlinarith
theorem denseClass_withSchedule_of_scaled_tendsto
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
A :
p rho B c d :
ℝ
hg :
hp :
hr :
SamplingFractionLimit A rho
hB :
0 < B
hc :
0 < c
hcd :
c < d
Y :
∀ r, PotentialOutcome (A.popSize r) M
hb :
∀ r S i z, |Y r S i z| ≤ B
conclusion
DenseScheduleClass (A.withSchedule Y) p rho B c
Proof (Lean source)
lemma denseClass_withSchedule_of_scaled_tendsto {M : ℕ} (A : ScheduleArray M)
(p rho B c d : ℝ) (hg : GroupCountGrowth A)
(hp : StableTreatmentFraction A p) (hr : SamplingFractionLimit A rho)
(hB : 0 < B) (hc : 0 < c) (hcd : c < d)
(Y : ∀ r, PotentialOutcome (A.popSize r) M)
(hb : ∀ r S i z, |Y r S i z| ≤ B)
(ht : Tendsto (fun r => (A.groups r : ℝ) * sigmaSq (Y r)
(A.grouped_le r) (A.treated_pos r) (A.treated_lt r)) atTop (nhds d)) :
DenseScheduleClass (A.withSchedule Y) p rho B c := by
refine ⟨hg, hp, hr, ⟨hB, hb⟩, hc, ?_⟩
have ht' : Tendsto (fun r => (((A.groups r : ℝ) * sigmaSq (Y r)
(A.grouped_le r) (A.treated_pos r) (A.treated_lt r) : ℝ) : EReal))
atTop (nhds (d : EReal)) := EReal.tendsto_coe.2 ht
change (c : EReal) ≤ liminf (fun r => (((A.groups r : ℝ) *
sigmaSq (Y r) (A.grouped_le r) (A.treated_pos r) (A.treated_lt r) : ℝ) : EReal)) atTop
rw [ht'.liminf_eq]
exact_mod_cast hcd.le
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma tendsto_of_uniform_support_error {Ω : ℕ → Type*}
(X : ∀ r, Ω r → ℝ) (c : ℝ) (Γ : ∀ r, Finset (Ω r)) (e : ℕ → ℝ)
(he : Tendsto e atTop (nhds 0)) (_he0 : ∀ r, 0 < e r)
(hbound : ∀ r w, w ∈ Γ r → |X r w - c| < e r)
(u : ∀ r, Ω r) (hu : ∀ r, u r ∈ Γ r) :
Tendsto (fun r => X r (u r)) atTop (nhds c) := by
rw [Metric.tendsto_atTop]
intro ε hε
have hev : ∀ᶠ r in atTop, e r < ε := (tendsto_order.1 he).2 ε hε
apply eventually_atTop.1
filter_upwards [hev] with r hr
simpa [Real.dist_eq] using (hbound r (u r) (hu r)).trans hr
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.RademacherDegreeOne 18 declarations This file connects centered inclusion-linear functions in the reusable Johnson space to the paper-local uniform-slice expectation.
Degree-one slice bridges for additive schedules
This file connects centered inclusion-linear functions in the reusable Johnson space to the paper-local uniform-slice expectation.
theorem mem_johnsonHarmonic_one_of_degreeOne_mean_zero Lemma mem_johnsonHarmonic_one_of_degreeOne_mean_zero in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated membership property holds.
Formal statement
hypotheses
conclusion
f ∈ johnsonHarmonic n M 1
Proof (Lean source)
lemma mem_johnsonHarmonic_one_of_degreeOne_mean_zero {n M : ℕ}
(hM : M ≤ n) (f : SliceFn n M) (hf : f ∈ degreeAtMost n M 1)
(hmean : mean f = 0) : f ∈ johnsonHarmonic n M 1 := by
unfold johnsonHarmonic
constructor
· exact hf
· intro g hg
obtain ⟨c, rfl⟩ := (mem_degreeAtMost_zero_iff hM _).1 hg
rw [PiLp.inner_apply]
simp only [Real.inner_apply, constFn]
unfold mean at hmean
have hcard : (card
(Omega n M) : ℝ) ≠ 0 := by
rw [card_omega hM]
exact_mod_cast (Nat.choose_pos hM).ne'
have hsum : ∑ A, f A = 0 := by
apply (mul_eq_zero.mp hmean).resolve_left
exact inv_ne_zero hcard
rw [← Finset.mul_sum, hsum, mul_zero]
theorem johnsonMean_sampleMean_eq_sliceExpectation
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
Proof (Lean source)
lemma johnsonMean_sampleMean_eq_sliceExpectation {n M : ℕ} (hM : M ≤ n)
(x : Fin n → ℝ) :
mean (toLp 2 (fun A :
Omega n M =>
(∑ i ∈ A.1, x i) / (M : ℝ))) =
(slice n M hM).E (sampleMean M x) := by
classical
let e : Omega n M ≃
Omega n M :=
{ toFun := fun A => ⟨A.1, A.2⟩
invFun := fun A => ⟨A.1, A.2⟩
left_inv := fun A => by cases A; rfl
right_inv := fun A => by cases A; rfl }
unfold mean slice FiniteDesign.E completeRandomization sampleMean
simp only [one_div]
rw [← Finset.mul_sum]
have hcard : card
(Omega n M) =
card (Omega n M) := by
exact Fintype.card_congr e.symm
rw [hcard]
congr 1
symm
exact Fintype.sum_equiv e _ _ (fun A => by
change (∑ i, if i ∈ A.1 then x i else 0) / (M : ℝ) =
(∑ i ∈ (e A).1, x i) / (M : ℝ)
rw [← Finset.sum_filter]
congr 2
have heA : (e A).1 = A.1 := by
change A.1 = A.1
rfl
ext i
simp [heA])
Given the stated population sizes, design objects, functions, and conditions, the stated membership property holds.
Formal statement
Proof (Lean source)
lemma sampleMean_mem_degreeAtMost_one {n M : ℕ} (x : Fin n → ℝ) :
toLp 2 (fun A :
Omega n M =>
(∑ i ∈ A.1, x i) / (M : ℝ)) ∈ degreeAtMost n M 1 := by
have hgen (i : Fin n) : inclusionMonomial (M := M) {i} ∈ degreeAtMost n M 1 :=
Submodule.subset_span ⟨{i}, by simp, rfl⟩
have hsum : (∑ i : Fin n,
(x i / (M : ℝ)) • inclusionMonomial (M := M) {i}) ∈
degreeAtMost n M 1 :=
Submodule.sum_mem _ (fun i _ => Submodule.smul_mem _ _ (hgen i))
convert hsum using 1
ext A
simp [inclusionMonomial, Finset.sum_div]
theorem centeredSampleMean_mem_johnsonHarmonic_one
Given the stated population sizes, design objects, functions, and conditions, the stated membership property holds.
Formal statement
hypotheses
conclusion
toLp 2 (fun A : Omega n M => sampleMean M x A - (slice n M hMn).E (sampleMean M x)) ∈ johnsonHarmonic n M 1
Proof (Lean source)
lemma centeredSampleMean_mem_johnsonHarmonic_one {n M : ℕ}
(hMn : M ≤ n) (x : Fin n → ℝ) :
toLp 2 (fun A : Omega n M =>
sampleMean M x A - (slice n M hMn).E (sampleMean M x)) ∈
johnsonHarmonic n M 1 := by
unfold Causalean.Mathlib.Combinatorics.JohnsonKneser.Omega
let f : WithLp 2 ({A : Finset (Fin n) // A.card = M} → ℝ) :=
toLp 2 (fun A => sampleMean M x A)
have hf : f ∈ degreeAtMost n M 1 := by
simpa [f, sampleMean,
Causalean.Mathlib.Combinatorics.JohnsonKneser.Omega] using
sampleMean_mem_degreeAtMost_one (M := M) x
have hm : mean f = (slice n M hMn).E (sampleMean M x) := by
simpa [f, sampleMean,
Causalean.Mathlib.Combinatorics.JohnsonKneser.Omega] using
johnsonMean_sampleMean_eq_sliceExpectation hMn x
have heq : toLp 2 (fun A : {A : Finset (Fin n) // A.card = M} =>
sampleMean M x A - (slice n M hMn).E (sampleMean M x)) =
f - toLp 2 (fun _ : {A : Finset (Fin n) // A.card = M} => mean f) := by
ext A
simp [f, hm]
rw [heq]
apply mem_johnsonHarmonic_one_of_degreeOne_mean_zero hMn
· apply Submodule.sub_mem
· exact hf
· apply degreeAtMost_mono (n := n) (M := M) (zero_le 1)
have hc := (mem_degreeAtMost_zero_iff hMn
(constFn (n := n) (M := M) (mean f))).2 ⟨mean f, rfl⟩
simpa [constFn,
Causalean.Mathlib.Combinatorics.JohnsonKneser.Omega] using hc
· unfold mean
change (card ({A : Finset (Fin n) // A.card = M}) : ℝ)⁻¹ *
∑ A : {A : Finset (Fin n) // A.card = M}, (f A -
(card ({A : Finset (Fin n) // A.card = M}) : ℝ)⁻¹ * ∑ S, f S) = 0
rw [Finset.sum_sub_distrib, Finset.sum_const, Finset.card_univ,
nsmul_eq_mul]
have hcard : (card ({A : Finset (Fin n) // A.card = M}) : ℝ) ≠ 0 := by
rw [Fintype.card_finset_len, card_fin]
exact_mod_cast (Nat.choose_pos hMn).ne'
field_simp [hcard]
ring
theorem johnsonProj_sampleMean_eq_centered
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
Proof (Lean source)
lemma johnsonProj_sampleMean_eq_centered {n M : ℕ} (hMpos : 0 < M)
(h2M : 2 * M ≤ n) (J : JohnsonProjections n M)
(hJohnson : JohnsonOrthogonalDecomposition n M J) (x : Fin n → ℝ) :
J.proj ⟨1, by omega⟩ (sampleMean M x) =
fun A => sampleMean M x A - (slice n M (by omega)).E (sampleMean M x) := by
let k : Fin (M + 1) := ⟨1, by omega⟩
let f : Omega n M → ℝ := sampleMean M x
let fc : Omega n M → ℝ :=
fun A => f A - (slice n M (by omega)).E f
have hmem : fc ∈ johnsonHarmonicSpace n M J.slice_nonempty k := by
unfold johnsonHarmonicSpace
have hh := centeredSampleMean_mem_johnsonHarmonic_one
(by omega : M ≤ n) x
simpa [k, fc, f] using hh
have hfix : J.proj k fc = fc := J.fixes_range _ _ hmem
have hk : 0 < k.1 := by simp [k]
have hcenter := johnson_proj_centered_eq h2M J hJohnson k hk f
rw [← hcenter, hfix]
theorem degreeOneEnergy_independentPrior_eq_popVar
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M :
ℕ
hMpos :
0 < M
h2M :
2 * M ≤ n
hMn :
M ≤ n
J :
hJohnson :
conclusion
degreeOneEnergy n M hMpos hMn J (independentPriorSchedule n M u)
= (1 / (M : ℝ) - 1 / (n : ℝ)) * popVar (fun i => rademacherSign (u (i, true)) - rademacherSign (u (i, false)))
Proof (Lean source)
lemma degreeOneEnergy_independentPrior_eq_popVar {n M : ℕ}
(hMpos : 0 < M) (h2M : 2 * M ≤ n) (hMn : M ≤ n)
(J : JohnsonProjections n M) (hJohnson : JohnsonOrthogonalDecomposition n M J)
(u : Fin n × Bool → Bool) :
degreeOneEnergy n M hMpos hMn J (independentPriorSchedule n M u) =
(1 / (M : ℝ) - 1 / (n : ℝ)) *
popVar (fun i => rademacherSign (u (i, true)) - rademacherSign (u (i, false))) := by
let x : Fin n → ℝ := fun i =>
rademacherSign (u (i, true)) - rademacherSign (u (i, false))
have hf : (fun A => armTable n M (independentPriorSchedule n M u) true A -
armTable n M (independentPriorSchedule n M u) false A) = sampleMean M x := by
rw [armTable_independentPrior_eq_sampleMean,
armTable_independentPrior_eq_sampleMean]
funext A
unfold sampleMean
rw [← sub_div, ← Finset.sum_sub_distrib]
apply congrArg (fun q : ℝ => q / (M : ℝ))
apply Finset.sum_congr rfl
intro i _
by_cases hi : i ∈ A.1 <;> simp [hi, x]
unfold degreeOneEnergy
rw [hf, johnsonProj_sampleMean_eq_centered hMpos h2M J hJohnson x]
rw [show sliceNorm n M hMn
(fun A => sampleMean M x A - (slice n M hMn).E (sampleMean M x)) ^ 2 =
(slice n M hMn).Var (sampleMean M x) by
unfold sliceNorm sliceNormSq sliceInner
rw [Real.sq_sqrt]
· rw [FiniteDesign.Var_eq]
let m := (slice n M hMn).E (sampleMean M x)
have hp : (fun A =>
(sampleMean M x A - m) * (sampleMean M x A - m)) =
fun A => sampleMean M x A ^ 2 - 2 * m * sampleMean M x A + m ^ 2 := by
funext A
ring
rw [hp, (slice n M hMn).E_add, (slice n M hMn).E_sub,
(slice n M hMn).E_const_mul, (slice n M hMn).E_const]
dsimp [m]
ring
· exact (slice n M hMn).E_nonneg (fun A => mul_self_nonneg _)]
unfold slice
have hn2 : 2 ≤ n := by omega
simpa using (Var_sampleMean (U := Fin n) M (by simpa using hMn) hMpos
(by simpa using hn2) x)
For the stated inputs, rad cross mean is defined by the formula below.
Definition (Lean source)
parameters
def
radCrossMean n u :
ℝ
given by
Given the stated population sizes, design objects, functions, and conditions, the rad pair e result holds.
Formal statement
Proof (Lean source)
lemma radPair_E (n : ℕ) (i : Fin n) :
(priorIndependent n).E (fun u =>
rademacherSign (u (i, true)) * rademacherSign (u (i, false))) = 0 := by
unfold priorIndependent
rw [FiniteDesign.E_prod_apply₂ _ (by simp : (i, true) ≠ (i, false))]
norm_num [coinDesign_E, rademacherSign]
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma radPair_var (n : ℕ) (i : Fin n) :
(priorIndependent n).Var (fun u =>
rademacherSign (u (i, true)) * rademacherSign (u (i, false))) = 1 := by
rw [FiniteDesign.Var_eq, radPair_E]
have hsquare : (fun u : Fin n × Bool → Bool =>
(rademacherSign (u (i, true)) * rademacherSign (u (i, false))) ^ 2) =
fun _ => (1 : ℝ) := by
funext u
cases u (i, true) <;> cases u (i, false) <;> norm_num [rademacherSign]
rw [hsquare, FiniteDesign.E_const]
norm_num
Given the stated population sizes, design objects, functions, and conditions, the rad pair cov ne result holds.
Formal statement
hypotheses
n :
ℕ
Fin n
hij :
i ≠ j
conclusion
(priorIndependent n).Cov (fun u => rademacherSign (u (i, true)) * rademacherSign (u (i, false))) (fun u => rademacherSign (u (j, true)) * rademacherSign (u (j, false)))
= 0
Proof (Lean source)
lemma radPair_cov_ne {n : ℕ} {i j : Fin n} (hij : i ≠ j) :
(priorIndependent n).Cov
(fun u => rademacherSign (u (i, true)) * rademacherSign (u (i, false)))
(fun u => rademacherSign (u (j, true)) * rademacherSign (u (j, false))) = 0 := by
unfold priorIndependent
apply FiniteDesign.Cov_prod_disjoint_zero _
({(i, true), (i, false)} : Finset (Fin n × Bool))
({(j, true), (j, false)} : Finset (Fin n × Bool))
· simp_all [eq_comm]
· intro w w' hw
rw [hw (i, true) (by simp), hw (i, false) (by simp)]
· intro w w' hw
rw [hw (j, true) (by simp), hw (j, false) (by simp)]
theorem radCrossMean_E
Given the stated population sizes, design objects, functions, and conditions, the rad cross mean e result holds.
Formal statement
Proof (Lean source)
lemma radCrossMean_E (n : ℕ) :
(priorIndependent n).E (radCrossMean n) = 0 := by
unfold radCrossMean
rw [show (fun u : Fin n × Bool → Bool =>
(∑ i, rademacherSign (u (i, true)) * rademacherSign (u (i, false))) / (n : ℝ)) =
(fun u => ∑ i, (1 / (n : ℝ)) *
(rademacherSign (u (i, true)) * rademacherSign (u (i, false)))) by
funext u
rw [← Finset.mul_sum]
ring]
rw [FiniteDesign.E_sum]
apply Finset.sum_eq_zero
intro i _
rw [FiniteDesign.E_const_mul]
rw [radPair_E]
ring
theorem radCrossMean_var
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma radCrossMean_var (n : ℕ) :
(priorIndependent n).Var (radCrossMean n) =
(n : ℝ) * (1 / (n : ℝ)) ^ 2 := by
unfold radCrossMean
rw [show (fun u : Fin n × Bool → Bool =>
(∑ i, rademacherSign (u (i, true)) * rademacherSign (u (i, false))) / (n : ℝ)) =
(fun u => ∑ i, (1 / (n : ℝ)) *
(rademacherSign (u (i, true)) * rademacherSign (u (i, false)))) by
funext u
rw [← Finset.mul_sum]
ring]
rw [(priorIndependent n).Var_linear_comb]
calc
(∑ i, ∑ j,
1 / (n : ℝ) * (1 / (n : ℝ)) *
(priorIndependent n).Cov
(fun u => rademacherSign (u (i, true)) * rademacherSign (u (i, false)))
(fun u => rademacherSign (u (j, true)) * rademacherSign (u (j, false)))) =
∑ i : Fin n, (1 / (n : ℝ)) ^ 2 := by
apply Finset.sum_congr rfl
intro i _
rw [Finset.sum_eq_single i]
· rw [FiniteDesign.Cov_self, radPair_var]
ring
· intro j _ hji
rw [radPair_cov_ne (Ne.symm hji)]
ring
· simp
_ = (n : ℝ) * (1 / (n : ℝ)) ^ 2 := by simp
theorem radCrossMean_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r)) (fun r => radCrossMean (N r)) (fun _ => 0)
Proof (Lean source)
lemma radCrossMean_tendstoInProb (N : ℕ → ℕ) (hN : Tendsto N atTop atTop) :
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r))
(fun r => radCrossMean (N r)) (fun _ => 0) := by
have hv : Tendsto (fun r =>
(priorIndependent (N r)).Var (radCrossMean (N r))) atTop (nhds 0) := by
simp_rw [radCrossMean_var]
have hNR : Tendsto (fun r => (N r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hN
have hinv := hNR.inv_tendsto_atTop
convert hinv using 1
funext r
by_cases hz : N r = 0
· simp [hz]
· change (N r : ℝ) * (1 / (N r : ℝ)) ^ 2 = (N r : ℝ)⁻¹
field_simp
have h := FiniteDesign.tendstoInProb_of_var
(D := fun r => priorIndependent (N r)) (X := fun r => radCrossMean (N r)) hv
simpa [radCrossMean_E] using h
theorem popVar_independentDifference_eq
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
= (n : ℝ) / ((n : ℝ) - 1) * (2 - 2 * radCrossMean n u - (radArmMean n true u - radArmMean n false u) ^ 2)
Proof (Lean source)
lemma popVar_independentDifference_eq {n : ℕ} (hn : 2 ≤ n)
(u : Fin n × Bool → Bool) :
popVar (fun i => rademacherSign (u (i, true)) - rademacherSign (u (i, false))) =
(n : ℝ) / ((n : ℝ) - 1) *
(2 - 2 * radCrossMean n u -
(radArmMean n true u - radArmMean n false u) ^ 2) := by
unfold popVar popMean radCrossMean radArmMean
have hnR : (n : ℝ) ≠ 0 := by exact_mod_cast (by omega : n ≠ 0)
have hn1R : (n : ℝ) - 1 ≠ 0 := by
have : (1 : ℝ) < n := by exact_mod_cast hn
linarith
have hsignsq (i : Fin n) (z : Bool) : rademacherSign (u (i, z)) ^ 2 = 1 := by
cases u (i, z) <;> norm_num [rademacherSign]
simp_rw [sub_sq]
rw [Finset.sum_sub_distrib, Finset.sum_add_distrib]
simp_rw [hsignsq]
rw [Finset.sum_sub_distrib, Finset.sum_const, Finset.card_univ,
card_fin, nsmul_eq_mul]
rw [Finset.sum_add_distrib, Finset.sum_sub_distrib, Finset.sum_const,
Finset.card_univ, card_fin, nsmul_eq_mul]
rw [← Finset.sum_mul, ← Finset.mul_sum]
simp_rw [Finset.sum_sub_distrib]
rw [show (∑ x : Fin n,
2 * rademacherSign (u (x, true)) * rademacherSign (u (x, false))) =
2 * ∑ x : Fin n,
rademacherSign (u (x, true)) * rademacherSign (u (x, false)) by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
ring]
field_simp [hnR, hn1R]
ring
Given the stated population sizes, design objects, functions, and conditions, the deterministic mul result holds.
Formal statement
hypotheses
Ω :
X :
∀ r
if
Ω r
then
ℝ
x c :
ℝ
a :
ℕ → ℝ
hX :
FiniteDesign.TendstoInProb D X (fun _ => x)
conclusion
FiniteDesign.TendstoInProb D (fun r w => a r * X r w) (fun _ => c * x)
Proof (Lean source)
lemma FiniteDesign.TendstoInProb.deterministic_mul
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{X : ∀ r, Ω r → ℝ} {x c : ℝ} {a : ℕ → ℝ}
(hX : FiniteDesign.TendstoInProb D X (fun _ => x))
(ha : Tendsto a atTop (nhds c)) :
FiniteDesign.TendstoInProb D (fun r w => a r * X r w) (fun _ => c * x) := by
intro ε hε
let K := |c| + 1
have hK : 0 < K := by dsimp [K]; positivity
have hδX : 0 < ε / (2 * K) := by positivity
have hδa : 0 < ε / (2 * (|x| + 1)) := by positivity
have haOne : ∀ᶠ r in atTop, |a r - c| < 1 := by
simpa [Real.dist_eq] using (Metric.tendsto_atTop.1 ha 1 zero_lt_one)
have haClose : ∀ᶠ r in atTop, |a r - c| < ε / (2 * (|x| + 1)) := by
simpa [Real.dist_eq] using (Metric.tendsto_atTop.1 ha _ hδa)
apply squeeze_zero' (Eventually.of_forall fun r => (D r).Pr_nonneg _) _
(hX (ε / (2 * K)) hδX)
filter_upwards [haOne, haClose] with r har hac
apply (D r).Pr_mono
intro w hw
by_contra hnear
push Not at hnear
have haBound : |a r| ≤ K := by
have htri : |a r| ≤ |a r - c| + |c| := by
calc
|a r| = |(a r - c) + c| := by ring_nf
_ ≤ |a r - c| + |c| := abs_add_le _ _
dsimp [K]
linarith
have hprod : |a r * X r w - c * x| ≤
|a r| * |X r w - x| + |a r - c| * |x| := by
calc
|a r * X r w - c * x| =
|a r * (X r w - x) + (a r - c) * x| := by ring_nf
_ ≤ |a r * (X r w - x)| + |(a r - c) * x| := abs_add_le _ _
_ = |a r| * |X r w - x| + |a r - c| * |x| := by rw [abs_mul, abs_mul]
have hfirst : |a r| * |X r w - x| < ε / 2 := by
calc
|a r| * |X r w - x| ≤ K * |X r w - x| :=
mul_le_mul_of_nonneg_right haBound (abs_nonneg _)
_ < K * (ε / (2 * K)) := mul_lt_mul_of_pos_left hnear hK
_ = ε / 2 := by field_simp
have hsecond : |a r - c| * |x| < ε / 2 := by
have hxlt : |x| < |x| + 1 := by linarith
calc
|a r - c| * |x| ≤ |a r - c| * (|x| + 1) :=
mul_le_mul_of_nonneg_left hxlt.le (abs_nonneg _)
_ < (ε / (2 * (|x| + 1))) * (|x| + 1) :=
mul_lt_mul_of_pos_right hac (by positivity)
_ = ε / 2 := by field_simp
linarith
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma natCast_ratio_sub_one_tendsto (N : ℕ → ℕ) (hN : Tendsto N atTop atTop) :
Tendsto (fun r => (N r : ℝ) / ((N r : ℝ) - 1)) atTop (nhds 1) := by
have hNR : Tendsto (fun r => (N r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hN
have hInv : Tendsto (fun r => ((N r : ℝ))⁻¹) atTop (nhds 0) :=
hNR.inv_tendsto_atTop
have hden : Tendsto (fun r => 1 - ((N r : ℝ))⁻¹) atTop (nhds 1) := by
simpa using tendsto_const_nhds.sub hInv
have hdenInv : Tendsto (fun r => (1 - ((N r : ℝ))⁻¹)⁻¹) atTop (nhds 1) := by
simpa using hden.inv₀ (by norm_num : (1 : ℝ) ≠ 0)
apply hdenInv.congr'
filter_upwards [hN.eventually (eventually_ge_atTop 1)] with r hr
have hne : (N r : ℝ) ≠ 0 := by exact_mod_cast (by omega : N r ≠ 0)
field_simp
theorem independentDifferencePopVar_tendstoInProb Lemma independentDifferencePopVar_tendstoInProb in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r)) (fun _r u => popVar (fun i => rademacherSign (u (i, true)) - rademacherSign (u (i, false)))) (fun _ => 2)
Proof (Lean source)
lemma independentDifferencePopVar_tendstoInProb (N : ℕ → ℕ)
(hNtwo : ∀ r, 2 ≤ N r) (hN : Tendsto N atTop atTop) :
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r))
(fun _r u => popVar (fun i =>
rademacherSign (u (i, true)) - rademacherSign (u (i, false))))
(fun _ => 2) := by
let D := fun r => priorIndependent (N r)
have hcross := radCrossMean_tendstoInProb N hN
have hmean := (radArmMean_tendstoInProb N hN true).sub
(radArmMean_tendstoInProb N hN false)
have hmeanSq := FiniteDesign.tendstoInProb_continuousMap D
(fun r u => radArmMean (N r) true u - radArmMean (N r) false u)
0 (fun x => x ^ 2) (by simpa [D] using hmean) (continuousAt_id.pow 2)
have hconst : FiniteDesign.TendstoInProb D (fun _ _ => (2 : ℝ)) (fun _ => 2) :=
FiniteDesign.deterministic_tendstoInProb D (fun _ => 2) 2 tendsto_const_nhds
have hinner := (hconst.add (hcross.const_mul (-2))).sub hmeanSq
have hinner' : FiniteDesign.TendstoInProb D
(fun r u => 2 - 2 * radCrossMean (N r) u -
(radArmMean (N r) true u - radArmMean (N r) false u) ^ 2)
(fun _ => 2) := by
simpa [D, sub_eq_add_neg] using hinner
have hratio := natCast_ratio_sub_one_tendsto N hN
have hscaled := FiniteDesign.TendstoInProb.deterministic_mul hinner' hratio
convert hscaled using 1
· funext r u
rw [popVar_independentDifference_eq (hNtwo r)]
· funext r
ring
theorem scheduleArray_independent_degreeOne_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
A :
hGrowth :
J :
∀ r, JohnsonProjections (A.popSize r) M
hJohnson :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
conclusion
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r)) (fun r u => degreeOneEnergy (A.popSize r) M (A.groupSize_ge_two.trans' (by omega)) (A.groupSize_le r) (J r) (independentPriorSchedule (A.popSize r) M u)) (fun _ => 2 / (M : ℝ))
Proof (Lean source)
lemma scheduleArray_independent_degreeOne_tendstoInProb {M : ℕ} (A : ScheduleArray M)
(hGrowth : GroupCountGrowth A) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hJohnson : ∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)) :
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r))
(fun r u => degreeOneEnergy (A.popSize r) M
(A.groupSize_ge_two.trans' (by omega)) (A.groupSize_le r) (J r)
(independentPriorSchedule (A.popSize r) M u))
(fun _ => 2 / (M : ℝ)) := by
have hN := A.popSize_tendsto_atTop hGrowth
have hNtwo : ∀ r, 2 ≤ A.popSize r := fun r =>
le_trans A.groupSize_ge_two (A.groupSize_le r)
have hpop := independentDifferencePopVar_tendstoInProb A.popSize hNtwo hN
have hNR : Tendsto (fun r => (A.popSize r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hN
have hInv : Tendsto (fun r => ((A.popSize r : ℝ))⁻¹) atTop (nhds 0) :=
hNR.inv_tendsto_atTop
have hcoef : Tendsto (fun r => 1 / (M : ℝ) - 1 / (A.popSize r : ℝ))
atTop (nhds (1 / (M : ℝ))) := by
have hc : Tendsto (fun _ : ℕ => (1 / (M : ℝ))) atTop
(nhds (1 / (M : ℝ))) := tendsto_const_nhds
simpa only [one_div, sub_zero] using hc.sub hInv
have hscaled := FiniteDesign.TendstoInProb.deterministic_mul hpop hcoef
convert hscaled using 1
· funext r u
have htpos := A.treated_pos r
have htlt := A.treated_lt r
have hGtwo : 2 ≤ A.groups r := by omega
have h2M : 2 * M ≤ A.popSize r :=
le_trans (by simpa [Nat.mul_comm] using Nat.mul_le_mul_left M hGtwo)
(A.grouped_le r)
rw [degreeOneEnergy_independentPrior_eq_popVar
(A.groupSize_ge_two.trans' (by omega)) h2M (A.groupSize_le r) (J r)
(hJohnson r)]
· funext r
ring
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.RademacherMoments 20 declarations Exact slice-variance formulas and the basic product-design law of large numbers used by the Rademacher mixture separation argument.
Finite-product Rademacher moments
Exact slice-variance formulas and the basic product-design law of large numbers used by the Rademacher mixture separation argument.
For the stated inputs, rad mean is defined by the formula below.
Definition (Lean source)
theorem radMean_E
Given the stated population sizes, design objects, functions, and conditions, the rad mean e result holds.
Proof (Lean source)
lemma radMean_E (n : ℕ) : (priorSame n).E (radMean n) = 0 := by
unfold priorSame radMean
rw [show (fun u : Fin n → Bool => (∑ i, rademacherSign (u i)) / (n : ℝ)) =
(fun u => ∑ i, (1 / (n : ℝ)) * rademacherSign (u i)) by
funext u
rw [← Finset.mul_sum]
ring]
rw [FiniteDesign.E_sum]
apply Finset.sum_eq_zero
intro i _
rw [show (fun u : Fin n → Bool => 1 / (n : ℝ) * rademacherSign (u i)) =
fun u => 1 / (n : ℝ) * (fun b => rademacherSign b) (u i) by rfl,
FiniteDesign.E_const_mul, FiniteDesign.E_prod_apply, coinDesign_E]
norm_num [rademacherSign]
theorem radMean_var
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
Proof (Lean source)
lemma radMean_var (n : ℕ) :
(priorSame n).Var (radMean n) = (n : ℝ) * (1 / (n : ℝ)) ^ 2 := by
unfold priorSame radMean
rw [show (fun u : Fin n → Bool => (∑ i, rademacherSign (u i)) / (n : ℝ)) =
(fun u => ∑ i, (1 / (n : ℝ)) * rademacherSign (u i)) by
funext u
rw [← Finset.mul_sum]
ring]
rw [FiniteDesign.Var_prod_linear_comb]
simp only [FiniteDesign.Var_eq, coinDesign_E, rademacherSign]
norm_num
theorem popVar_rademacher
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma popVar_rademacher {n : ℕ} (hn : 0 < n) (u : Fin n → Bool) :
popVar (fun i => rademacherSign (u i)) =
(n : ℝ) / ((n : ℝ) - 1) * (1 - radMean n u ^ 2) := by
unfold popVar popMean radMean
have hncast : (n : ℝ) ≠ 0 := by exact_mod_cast hn.ne'
have hsq (i : Fin n) : rademacherSign (u i) ^ 2 = 1 := by
cases u i <;> simp [rademacherSign]
simp_rw [sub_sq, hsq]
rw [Finset.sum_add_distrib, Finset.sum_sub_distrib, Finset.sum_const,
Finset.card_univ, card_fin, nsmul_eq_mul]
rw [show (∑ x, 2 * rademacherSign (u x) *
((∑ x, rademacherSign (u x)) / (n : ℝ))) =
2 * (∑ x, rademacherSign (u x)) *
((∑ x, rademacherSign (u x)) / (n : ℝ)) by
rw [Finset.mul_sum, Finset.sum_mul]]
rw [show (∑ _x : Fin n, ((∑ x, rademacherSign (u x)) / (n : ℝ)) ^ 2) =
(n : ℝ) * ((∑ x, rademacherSign (u x)) / (n : ℝ)) ^ 2 by simp]
field_simp
ring
theorem radMean_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma radMean_tendstoInProb (N : ℕ → ℕ) (hN : Tendsto N atTop atTop) :
FiniteDesign.TendstoInProb (fun r => priorSame (N r))
(fun r => radMean (N r)) (fun _ => 0) := by
have hv : Tendsto (fun r => (priorSame (N r)).Var (radMean (N r)))
atTop (nhds 0) := by
simp_rw [radMean_var]
have hNR : Tendsto (fun r => (N r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hN
have hinv := hNR.inv_tendsto_atTop
convert hinv using 1
funext r
by_cases hz : N r = 0
· simp [hz]
· change (N r : ℝ) * (1 / (N r : ℝ)) ^ 2 = (N r : ℝ)⁻¹
field_simp
have h := FiniteDesign.tendstoInProb_of_var
(D := fun r => priorSame (N r)) (X := fun r => radMean (N r)) hv
simpa [radMean_E] using h
For the stated inputs, rad arm mean is defined by the formula below.
Definition (Lean source)
theorem radArmMean_E
Given the stated population sizes, design objects, functions, and conditions, the rad arm mean e result holds.
Formal statement
Proof (Lean source)
lemma radArmMean_E (n : ℕ) (z : Bool) :
(priorIndependent n).E (radArmMean n z) = 0 := by
unfold priorIndependent radArmMean
rw [show (fun u : Fin n × Bool → Bool =>
(∑ i, rademacherSign (u (i, z))) / (n : ℝ)) =
(fun u => ∑ i, (1 / (n : ℝ)) * rademacherSign (u (i, z))) by
funext u
rw [← Finset.mul_sum]
ring]
rw [FiniteDesign.E_sum]
apply Finset.sum_eq_zero
intro i _
rw [show (fun u : Fin n × Bool → Bool =>
1 / (n : ℝ) * rademacherSign (u (i, z))) =
fun u => 1 / (n : ℝ) * (fun b => rademacherSign b) (u (i, z)) by rfl,
FiniteDesign.E_const_mul, FiniteDesign.E_prod_apply, coinDesign_E]
norm_num [rademacherSign]
theorem radArmMean_var
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n :
ℕ
z :
conclusion
Proof (Lean source)
lemma radArmMean_var (n : ℕ) (z : Bool) :
(priorIndependent n).Var (radArmMean n z) =
(n : ℝ) * (1 / (n : ℝ)) ^ 2 := by
unfold priorIndependent radArmMean
rw [show (fun u : Fin n × Bool → Bool =>
(∑ i, rademacherSign (u (i, z))) / (n : ℝ)) =
(fun u => ∑ q : Fin n × Bool,
(if q.2 = z then 1 / (n : ℝ) else 0) * rademacherSign (u q)) by
funext u
rw [Fintype.sum_prod_type]
cases z <;> simp [← Finset.mul_sum] <;> ring]
rw [FiniteDesign.Var_prod_linear_comb]
simp only [FiniteDesign.Var_eq, coinDesign_E, rademacherSign]
norm_num
rw [Fintype.sum_prod_type]
cases z <;> simp
theorem radArmMean_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r)) (fun r => radArmMean (N r) z) (fun _ => 0)
Proof (Lean source)
lemma radArmMean_tendstoInProb (N : ℕ → ℕ) (hN : Tendsto N atTop atTop)
(z : Bool) :
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r))
(fun r => radArmMean (N r) z) (fun _ => 0) := by
have hv : Tendsto (fun r =>
(priorIndependent (N r)).Var (radArmMean (N r) z)) atTop (nhds 0) := by
simp_rw [radArmMean_var]
have hNR : Tendsto (fun r => (N r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hN
have hinv := hNR.inv_tendsto_atTop
convert hinv using 1
funext r
by_cases hz : N r = 0
· simp [hz]
· change (N r : ℝ) * (1 / (N r : ℝ)) ^ 2 = (N r : ℝ)⁻¹
field_simp
have h := FiniteDesign.tendstoInProb_of_var
(D := fun r => priorIndependent (N r))
(X := fun r => radArmMean (N r) z) hv
simpa [radArmMean_E] using h
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma rademacherSliceCoefficient_tendsto {M : ℕ} (hM : 0 < M)
(N : ℕ → ℕ) (hN : Tendsto N atTop atTop) :
Tendsto (fun r =>
(1 / (M : ℝ) - 1 / (N r : ℝ)) * (N r : ℝ) / ((N r : ℝ) - 1))
atTop (nhds (1 / (M : ℝ))) := by
have hNR : Tendsto (fun r => (N r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hN
have hInv : Tendsto (fun r => ((N r : ℝ))⁻¹) atTop (nhds 0) :=
hNR.inv_tendsto_atTop
have hRatio : Tendsto (fun r => (N r : ℝ) / ((N r : ℝ) - 1))
atTop (nhds 1) := by
have hden : Tendsto (fun r => 1 - ((N r : ℝ))⁻¹) atTop (nhds 1) := by
simpa using tendsto_const_nhds.sub hInv
have hdenInv : Tendsto (fun r => (1 - ((N r : ℝ))⁻¹)⁻¹) atTop (nhds 1) := by
simpa using hden.inv₀ (by norm_num : (1 : ℝ) ≠ 0)
have hrewrite : ∀ᶠ r in atTop,
(N r : ℝ) / ((N r : ℝ) - 1) = (1 - ((N r : ℝ))⁻¹)⁻¹ := by
filter_upwards [hN.eventually (eventually_ge_atTop 1)] with r hr
have hne : (N r : ℝ) ≠ 0 := by exact_mod_cast (by omega : N r ≠ 0)
field_simp
have hrewrite' : (fun r => (1 - ((N r : ℝ))⁻¹)⁻¹) =ᶠ[atTop]
(fun r => (N r : ℝ) / ((N r : ℝ) - 1)) :=
hrewrite.mono fun _ hr => hr.symm
exact hdenInv.congr' hrewrite'
have hleft : Tendsto (fun r => 1 / (M : ℝ) - ((N r : ℝ))⁻¹)
atTop (nhds (1 / (M : ℝ))) := by
simpa using (tendsto_const_nhds.sub hInv)
convert hleft.mul hRatio using 1 <;> ring
theorem armTable_samePrior_eq_sampleMean
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma armTable_samePrior_eq_sampleMean {n M : ℕ} (u : Fin n → Bool) (z : Bool) :
armTable n M (samePriorSchedule n M u) z =
sampleMean M (fun i => rademacherSign (u i)) := by
funext A
simp only [armTable, samePriorSchedule, sampleMean]
congr 1
calc
_ = ∑ x ∈ A.1, rademacherSign (u x) := Finset.sum_attach _ _
_ = _ := by simp
theorem armTable_independentPrior_eq_sampleMean
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
conclusion
armTable n M (independentPriorSchedule n M u) z
= sampleMean M (fun i => rademacherSign (u (i, z)))
Proof (Lean source)
lemma armTable_independentPrior_eq_sampleMean {n M : ℕ}
(u : Fin n × Bool → Bool) (z : Bool) :
armTable n M (independentPriorSchedule n M u) z =
sampleMean M (fun i => rademacherSign (u (i, z))) := by
funext A
simp only [armTable, independentPriorSchedule, sampleMean]
congr 1
calc
_ = ∑ x ∈ A.1, rademacherSign (u (x, z)) := Finset.sum_attach _ _
_ = _ := by simp
theorem armVar_samePrior_eq_popVar
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
armVar n M hM (samePriorSchedule n M u) z
= (1 / (M : ℝ) - 1 / (n : ℝ)) * popVar (fun i => rademacherSign (u i))
Proof (Lean source)
lemma armVar_samePrior_eq_popVar {n M : ℕ} (hM : M ≤ n) (hMpos : 0 < M)
(hn : 2 ≤ n) (u : Fin n → Bool) (z : Bool) :
armVar n M hM (samePriorSchedule n M u) z =
(1 / (M : ℝ) - 1 / (n : ℝ)) * popVar (fun i => rademacherSign (u i)) := by
unfold armVar
rw [armTable_samePrior_eq_sampleMean]
unfold slice
simpa using (Var_sampleMean (U := Fin n) M (by simpa using hM) hMpos
(by simpa using hn) (fun i => rademacherSign (u i)))
theorem armVar_independentPrior_eq_popVar
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
conclusion
armVar n M hM (independentPriorSchedule n M u) z
= (1 / (M : ℝ) - 1 / (n : ℝ)) * popVar (fun i => rademacherSign (u (i, z)))
Proof (Lean source)
lemma armVar_independentPrior_eq_popVar {n M : ℕ} (hM : M ≤ n) (hMpos : 0 < M)
(hn : 2 ≤ n) (u : Fin n × Bool → Bool) (z : Bool) :
armVar n M hM (independentPriorSchedule n M u) z =
(1 / (M : ℝ) - 1 / (n : ℝ)) *
popVar (fun i => rademacherSign (u (i, z))) := by
unfold armVar
rw [armTable_independentPrior_eq_sampleMean]
unfold slice
simpa using (Var_sampleMean (U := Fin n) M (by simpa using hM) hMpos
(by simpa using hn) (fun i => rademacherSign (u (i, z))))
theorem armVar_samePrior_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma armVar_samePrior_tendstoInProb {M : ℕ} (hM : 0 < M)
(N : ℕ → ℕ) (hMN : ∀ r, M ≤ N r) (hNtwo : ∀ r, 2 ≤ N r)
(hN : Tendsto N atTop atTop) (z : Bool) :
FiniteDesign.TendstoInProb (fun r => priorSame (N r))
(fun r u => armVar (N r) M (hMN r) (samePriorSchedule (N r) M u) z)
(fun _ => 1 / (M : ℝ)) := by
let c : ℕ → ℝ := fun r =>
(1 / (M : ℝ) - 1 / (N r : ℝ)) * (N r : ℝ) / ((N r : ℝ) - 1)
have hc : Tendsto c atTop (nhds (1 / (M : ℝ))) :=
rademacherSliceCoefficient_tendsto hM N hN
have hm := radMean_tendstoInProb N hN
have hm2 := FiniteDesign.tendstoInProb_continuousMap
(fun r => priorSame (N r)) (fun r => radMean (N r)) 0 (fun x => x ^ 2) hm
(continuousAt_id.pow 2)
intro ε hε
have ht := hm2 (ε / 2) (by linarith)
have hec : ∀ᶠ r in atTop, |c r - 1 / (M : ℝ)| < ε / 2 := by
simpa [Real.dist_eq] using (Metric.tendsto_atTop.1 hc (ε / 2) (by linarith))
apply squeeze_zero' (Eventually.of_forall fun r => (priorSame (N r)).Pr_nonneg _) _ ht
filter_upwards [hec] with r hcr
apply (priorSame (N r)).Pr_mono
intro u hu
rw [armVar_samePrior_eq_popVar (hMN r) hM (hNtwo r),
popVar_rademacher (lt_of_lt_of_le hM (hMN r))] at hu
have hform :
(1 / (M : ℝ) - 1 / (N r : ℝ)) *
((N r : ℝ) / ((N r : ℝ) - 1) * (1 - radMean (N r) u ^ 2)) =
c r * (1 - radMean (N r) u ^ 2) := by
dsimp [c]
ring
rw [hform] at hu
have hMr : (0 : ℝ) < M := by exact_mod_cast hM
have hMone : (1 : ℝ) ≤ M := by exact_mod_cast hM
have hNtwoR : (2 : ℝ) ≤ N r := by exact_mod_cast hNtwo r
have hNr : (0 : ℝ) < N r := by linarith
have hNr1 : (0 : ℝ) < (N r : ℝ) - 1 := by linarith
have hMNreal : (M : ℝ) ≤ N r := by exact_mod_cast hMN r
have hc_formula : c r =
((N r : ℝ) - M) / ((M : ℝ) * ((N r : ℝ) - 1)) := by
dsimp [c]
field_simp
have hc_nonneg : 0 ≤ c r := by
rw [hc_formula]
exact div_nonneg (sub_nonneg.mpr hMNreal) (mul_nonneg hMr.le hNr1.le)
have hc_le_one : c r ≤ 1 := by
rw [hc_formula, div_le_one (mul_pos hMr hNr1)]
nlinarith [mul_nonneg hNr.le (sub_nonneg.mpr hMone)]
have hbound :
|c r * (1 - radMean (N r) u ^ 2) - 1 / (M : ℝ)| ≤
|c r - 1 / (M : ℝ)| + radMean (N r) u ^ 2 := by
have hid : c r * (1 - radMean (N r) u ^ 2) - 1 / (M : ℝ) =
(c r - 1 / (M : ℝ)) - c r * radMean (N r) u ^ 2 := by ring
calc
_ = |(c r - 1 / (M : ℝ)) - c r * radMean (N r) u ^ 2| := by rw [hid]
_ ≤ |c r - 1 / (M : ℝ)| + |c r * radMean (N r) u ^ 2| := abs_sub _ _
_ ≤ |c r - 1 / (M : ℝ)| + radMean (N r) u ^ 2 := by
rw [abs_of_nonneg (mul_nonneg hc_nonneg (sq_nonneg _))]
nlinarith [mul_le_mul_of_nonneg_right hc_le_one
(sq_nonneg (radMean (N r) u))]
have hsquare : ε / 2 ≤ radMean (N r) u ^ 2 := by linarith
rw [zero_pow (by omega : 2 ≠ 0), sub_zero, abs_of_nonneg (sq_nonneg _)]
exact hsquare
theorem armVar_independentPrior_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
M :
ℕ
hM :
0 < M
N :
ℕ → ℕ
hMN :
∀ r, M ≤ N r
hNtwo :
∀ r, 2 ≤ N r
z :
conclusion
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r)) (fun r u => armVar (N r) M (hMN r) (independentPriorSchedule (N r) M u) z) (fun _ => 1 / (M : ℝ))
Proof (Lean source)
lemma armVar_independentPrior_tendstoInProb {M : ℕ} (hM : 0 < M)
(N : ℕ → ℕ) (hMN : ∀ r, M ≤ N r) (hNtwo : ∀ r, 2 ≤ N r)
(hN : Tendsto N atTop atTop) (z : Bool) :
FiniteDesign.TendstoInProb (fun r => priorIndependent (N r))
(fun r u => armVar (N r) M (hMN r) (independentPriorSchedule (N r) M u) z)
(fun _ => 1 / (M : ℝ)) := by
let c : ℕ → ℝ := fun r =>
(1 / (M : ℝ) - 1 / (N r : ℝ)) * (N r : ℝ) / ((N r : ℝ) - 1)
have hc : Tendsto c atTop (nhds (1 / (M : ℝ))) :=
rademacherSliceCoefficient_tendsto hM N hN
have hm := radArmMean_tendstoInProb N hN z
have hm2 := FiniteDesign.tendstoInProb_continuousMap
(fun r => priorIndependent (N r)) (fun r => radArmMean (N r) z) 0
(fun x => x ^ 2) hm (continuousAt_id.pow 2)
intro ε hε
have ht := hm2 (ε / 2) (by linarith)
have hec : ∀ᶠ r in atTop, |c r - 1 / (M : ℝ)| < ε / 2 := by
simpa [Real.dist_eq] using (Metric.tendsto_atTop.1 hc (ε / 2) (by linarith))
apply squeeze_zero'
(Eventually.of_forall fun r => (priorIndependent (N r)).Pr_nonneg _) _ ht
filter_upwards [hec] with r hcr
apply (priorIndependent (N r)).Pr_mono
intro u hu
rw [armVar_independentPrior_eq_popVar (hMN r) hM (hNtwo r),
popVar_rademacher (lt_of_lt_of_le hM (hMN r))] at hu
change ε ≤ |(1 / (M : ℝ) - 1 / (N r : ℝ)) *
((N r : ℝ) / ((N r : ℝ) - 1) * (1 - radArmMean (N r) z u ^ 2)) -
1 / (M : ℝ)| at hu
have hform :
(1 / (M : ℝ) - 1 / (N r : ℝ)) *
((N r : ℝ) / ((N r : ℝ) - 1) * (1 - radArmMean (N r) z u ^ 2)) =
c r * (1 - radArmMean (N r) z u ^ 2) := by
dsimp [c]
ring
rw [hform] at hu
have hMr : (0 : ℝ) < M := by exact_mod_cast hM
have hMone : (1 : ℝ) ≤ M := by exact_mod_cast hM
have hNtwoR : (2 : ℝ) ≤ N r := by exact_mod_cast hNtwo r
have hNr : (0 : ℝ) < N r := by linarith
have hNr1 : (0 : ℝ) < (N r : ℝ) - 1 := by linarith
have hMNreal : (M : ℝ) ≤ N r := by exact_mod_cast hMN r
have hc_formula : c r =
((N r : ℝ) - M) / ((M : ℝ) * ((N r : ℝ) - 1)) := by
dsimp [c]
field_simp
have hc_nonneg : 0 ≤ c r := by
rw [hc_formula]
exact div_nonneg (sub_nonneg.mpr hMNreal) (mul_nonneg hMr.le hNr1.le)
have hc_le_one : c r ≤ 1 := by
rw [hc_formula, div_le_one (mul_pos hMr hNr1)]
nlinarith [mul_nonneg hNr.le (sub_nonneg.mpr hMone)]
have hbound :
|c r * (1 - radArmMean (N r) z u ^ 2) - 1 / (M : ℝ)| ≤
|c r - 1 / (M : ℝ)| + radArmMean (N r) z u ^ 2 := by
have hid : c r * (1 - radArmMean (N r) z u ^ 2) - 1 / (M : ℝ) =
(c r - 1 / (M : ℝ)) - c r * radArmMean (N r) z u ^ 2 := by ring
calc
_ = |(c r - 1 / (M : ℝ)) - c r * radArmMean (N r) z u ^ 2| := by rw [hid]
_ ≤ |c r - 1 / (M : ℝ)| + |c r * radArmMean (N r) z u ^ 2| := abs_sub _ _
_ ≤ |c r - 1 / (M : ℝ)| + radArmMean (N r) z u ^ 2 := by
rw [abs_of_nonneg (mul_nonneg hc_nonneg (sq_nonneg _))]
nlinarith [mul_le_mul_of_nonneg_right hc_le_one
(sq_nonneg (radArmMean (N r) z u))]
have hsquare : ε / 2 ≤ radArmMean (N r) z u ^ 2 := by linarith
rw [zero_pow (by omega : 2 ≠ 0), sub_zero, abs_of_nonneg (sq_nonneg _)]
exact hsquare
theorem scheduleArray_same_armVar_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) (fun r u => armVar (A.popSize r) M (A.groupSize_le r) (samePriorSchedule (A.popSize r) M u) z) (fun _ => 1 / (M : ℝ))
Proof (Lean source)
lemma scheduleArray_same_armVar_tendstoInProb {M : ℕ} (A : ScheduleArray M)
(hGrowth : GroupCountGrowth A) (z : Bool) :
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(fun r u => armVar (A.popSize r) M (A.groupSize_le r)
(samePriorSchedule (A.popSize r) M u) z)
(fun _ => 1 / (M : ℝ)) := by
exact armVar_samePrior_tendstoInProb
(A.groupSize_ge_two.trans' (by omega)) A.popSize A.groupSize_le
(fun r => le_trans A.groupSize_ge_two (A.groupSize_le r))
(A.popSize_tendsto_atTop hGrowth) z
theorem scheduleArray_independent_armVar_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r)) (fun r u => armVar (A.popSize r) M (A.groupSize_le r) (independentPriorSchedule (A.popSize r) M u) z) (fun _ => 1 / (M : ℝ))
Proof (Lean source)
lemma scheduleArray_independent_armVar_tendstoInProb {M : ℕ} (A : ScheduleArray M)
(hGrowth : GroupCountGrowth A) (z : Bool) :
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r))
(fun r u => armVar (A.popSize r) M (A.groupSize_le r)
(independentPriorSchedule (A.popSize r) M u) z)
(fun _ => 1 / (M : ℝ)) := by
exact armVar_independentPrior_tendstoInProb
(A.groupSize_ge_two.trans' (by omega)) A.popSize A.groupSize_le
(fun r => le_trans A.groupSize_ge_two (A.groupSize_le r))
(A.popSize_tendsto_atTop hGrowth) z
theorem degreeOneEnergy_samePrior
Given the stated population sizes, design objects, functions, and conditions, the degree one energy same prior result holds.
Formal statement
hypotheses
conclusion
degreeOneEnergy n M hMpos hM J (samePriorSchedule n M u) = 0
Proof (Lean source)
lemma degreeOneEnergy_samePrior {n M : ℕ} (hMpos : 0 < M) (hM : M ≤ n)
(J : JohnsonProjections n M) (u : Fin n → Bool) :
degreeOneEnergy n M hMpos hM J (samePriorSchedule n M u) = 0 := by
unfold degreeOneEnergy
have hz : (fun A => armTable n M (samePriorSchedule n M u) true A -
armTable n M (samePriorSchedule n M u) false A) = 0 := by
funext A
simp [armTable, samePriorSchedule]
rw [hz]
have hpzero : J.proj ⟨1, by omega⟩ 0 = 0 := by
have hzero := J.map_smul ⟨1, by omega⟩ 0 (fun _ : Omega n M => (0 : ℝ))
funext A
change J.proj ⟨1, by omega⟩ (fun _ => 0) A = 0
simpa only [zero_mul] using congrFun hzero A
rw [hpzero]
simp [sliceNorm, sliceNormSq, sliceInner]
theorem scheduleArray_same_degreeOne_tendstoInProb
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) (fun r u => degreeOneEnergy (A.popSize r) M (A.groupSize_ge_two.trans' (by omega)) (A.groupSize_le r) (J r) (samePriorSchedule (A.popSize r) M u)) (fun _ => 0)
Proof (Lean source)
lemma scheduleArray_same_degreeOne_tendstoInProb {M : ℕ} (A : ScheduleArray M)
(J : ∀ r, JohnsonProjections (A.popSize r) M) :
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(fun r u => degreeOneEnergy (A.popSize r) M
(A.groupSize_ge_two.trans' (by omega)) (A.groupSize_le r)
(J r) (samePriorSchedule (A.popSize r) M u)) (fun _ => 0) := by
intro ε hε
have hzero : (fun r => (priorSame (A.popSize r)).Pr (fun u =>
ε ≤ |degreeOneEnergy (A.popSize r) M
(A.groupSize_ge_two.trans' (by omega)) (A.groupSize_le r)
(J r) (samePriorSchedule (A.popSize r) M u) - 0|)) = fun _ => 0 := by
funext r
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
simp [degreeOneEnergy_samePrior, not_le.mpr hε]
rw [hzero]
exact tendsto_const_nhds
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.RademacherPriors 15 declarations This file realizes the common-sign and independent-arm product priors as finite product designs, embeds their signs into additive schedules, and records the one-realization observation channel.
Product Rademacher schedule priors and observation channel
This file realizes the common-sign and independent-arm product priors as finite product designs, embeds their signs into additive schedules, and records the one-realization observation channel.
def rademacherSign
Convert a fair coin to a Rademacher sign.
Definition (Lean source)
Common-arm product Rademacher prior.
Definition (Lean source)
parameters
n :
ℕ
def
given by
Independent-arm product Rademacher prior.
Definition (Lean source)
parameters
n :
ℕ
def
given by
def samePriorSchedule
Additive schedule induced by common signs.
Definition (Lean source)
def independentPriorSchedule
Additive schedule induced by independent arm-specific signs.
Definition (Lean source)
One observed realization: partition, treatment allocation, and observed outcomes for every member of every realized group.
Definition (Lean source)
def observe
The one-realization observation channel.
Definition (Lean source)
parameters
n :
ℕ
M :
ℕ
G :
ℕ
G1 :
ℕ
Y :
PotentialOutcome n M
w :
PartitionTuple n M G × TreatmentSpace G G1
def
given by
clause 1
partition := w.1
clause 2
treatment := w.2
clause 3
outcomes := fun g i => Y (w.1.1 g) i (g ∈ w.2.1)
def VarianceStatistic
A generic statistic of one realization.
Definition (Lean source)
For the stated inputs, select swap equiv is defined by the formula below.
Definition (Lean source)
parameters
ι :
Type*
z :
ι → Bool
given by
clause 1
toFun u := (fun i => u (i, z i), fun i => u (i, !z i))
clause 2
invFun w := fun q => if q.2 = z q.1 then w.1 q.1 else w.2 q.1
clause 3
left_inv u := by funext q rcases q with ⟨i, b⟩ cases b <;> cases hz : z i <;> simp [hz]
clause 4
right_inv w := by rcases w with ⟨v, t⟩ apply Prod.ext <;> funext i · simp · have hne : !z i
≠ z i := by cases z i <;> simp simp
Given the stated population sizes, design objects, functions, and conditions, the fair coin weight result holds.
Formal statement
Proof (Lean source)
lemma fairCoinWeight {ι : Type*} [Fintype ι] (u : ι → Bool) :
(∏ i, if u i = true then (1 : ℝ) / 2 else 1 - 1 / 2) =
((1 : ℝ) / 2) ^ card ι := by
calc
(∏ i, if u i = true then (1 : ℝ) / 2 else 1 - 1 / 2) =
∏ _i : ι, ((1 : ℝ) / 2) := by
apply Finset.prod_congr rfl
intro i _
cases u i <;> norm_num
_ = ((1 : ℝ) / 2) ^ card ι := by simp
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
Proof (Lean source)
lemma priorIndependent_E_select {n : ℕ} (z : Fin n → Bool)
(f : (Fin n → Bool) → ℝ) :
(priorIndependent n).E (fun u => f (fun i => u (i, z i))) =
(priorSame n).E f := by
classical
unfold priorIndependent priorSame FiniteDesign.E prodDesign
simp only [coinDesign, cond_eq_ite]
simp_rw [fairCoinWeight]
let e := selectSwapEquiv z
calc
(∑ x : (Fin n × Bool → Bool), (1 / 2 : ℝ) ^ card (Fin n × Bool) *
f (fun i => x (i, z i))) =
∑ y : (Fin n → Bool) × (Fin n → Bool),
(1 / 2 : ℝ) ^ card (Fin n × Bool) * f y.1 := by
apply Fintype.sum_equiv e
(fun x : Fin n × Bool → Bool =>
(1 / 2 : ℝ) ^ card (Fin n × Bool) * f (fun i => x (i, z i)))
(fun y : (Fin n → Bool) × (Fin n → Bool) =>
(1 / 2 : ℝ) ^ card (Fin n × Bool) * f y.1)
intro x
simp [e, selectSwapEquiv]
_ = ∑ x, (1 / 2 : ℝ) ^ card (Fin n) * f x := by
rw [Fintype.sum_prod_type]
simp only [Fintype.card_prod, card_fin, Fintype.card_bool,
Fintype.card_fun, Finset.sum_const, Finset.card_univ, nsmul_eq_mul]
apply Finset.sum_congr rfl
intro x _
rw [Nat.cast_pow, Nat.cast_ofNat, pow_mul, pow_two]
have hpow : (2 : ℝ) ^ n * (1 / 2 : ℝ) ^ n = 1 := by
rw [← mul_pow]
norm_num
calc
(2 : ℝ) ^ n * ((1 / 2 : ℝ) ^ n * (1 / 2 : ℝ) ^ n * f x) =
((2 : ℝ) ^ n * (1 / 2 : ℝ) ^ n) * ((1 / 2 : ℝ) ^ n * f x) := by ring
_ = (1 / 2 : ℝ) ^ n * f x := by rw [hpow, one_mul]
For the stated inputs, observed arm selector is defined by the formula below.
Definition (Lean source)
parameters
def
theorem observedArmSelector_of_mem
Given the stated population sizes, design objects, functions, and conditions, the observed arm selector of mem result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
w :
PartitionTuple n M G × TreatmentSpace G G1
g :
Fin G
i :
Fin n
hi :
i ∈ (w.1.1 g).1
conclusion
observedArmSelector w i = decide (g ∈ w.2.1)
Proof (Lean source)
lemma observedArmSelector_of_mem {n M G G1 : ℕ}
(w : PartitionTuple n M G × TreatmentSpace G G1) (g : Fin G) (i : Fin n)
(hi : i ∈ (w.1.1 g).1) : observedArmSelector w i = decide (g ∈ w.2.1) := by
classical
apply Bool.eq_iff_iff.mpr
simp only [observedArmSelector, decide_eq_true_eq]
constructor
· rintro ⟨g', hi', hg'⟩
by_cases hgg : g' = g
· exact hgg ▸ hg'
· exact elim (Finset.disjoint_left.mp (w.1.2 g' g hgg) hi' hi)
· intro hg
exact ⟨g, hi, hg⟩
theorem observe_independent_eq_same_selected
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
conclusion
observe n M G G1 (independentPriorSchedule n M u) w
Proof (Lean source)
lemma observe_independent_eq_same_selected {n M G G1 : ℕ}
(w : PartitionTuple n M G × TreatmentSpace G G1)
(u : Fin n × Bool → Bool) :
observe n M G G1 (independentPriorSchedule n M u) w =
observe n M G G1
(samePriorSchedule n M (fun i => u (i, observedArmSelector w i))) w := by
rcases w with ⟨T, Z⟩
unfold observe
congr
funext g i
change rademacherSign (u (i.1, decide (g ∈ Z.1))) =
rademacherSign (u (i.1, observedArmSelector (T, Z) i.1))
rw [observedArmSelector_of_mem (T, Z) g i.1 i.2]
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
D.E (fun a => E.E (f a)) = E.E (fun b => D.E (fun a => f a b))
Proof (Lean source)
lemma finiteDesign_E_swap {α β : Type*} [Fintype α] [Fintype β]
(D : FiniteDesign α) (E : FiniteDesign β) (f : α → β → ℝ) :
D.E (fun a => E.E (f a)) = E.E (fun b => D.E (fun a => f a b)) := by
unfold FiniteDesign.E
calc
(∑ a, D.p a * ∑ b, E.p b * f a b) =
∑ a, ∑ b, D.p a * (E.p b * f a b) := by
apply Finset.sum_congr rfl
intro a _
rw [Finset.mul_sum]
_ = ∑ b, ∑ a, E.p b * (D.p a * f a b) := by
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro b _
apply Finset.sum_congr rfl
intro a _
ring
_ = ∑ b, E.p b * ∑ a, D.p a * f a b := by
apply Finset.sum_congr rfl
intro b _
rw [Finset.mul_sum]
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.RademacherScaledVariance 6 declarations This file supplies the rowwise spectral identities used to assemble the two Rademacher-prior variance limits.
Exact scaled variance for additive Rademacher schedules
This file supplies the rowwise spectral identities used to assemble the two Rademacher-prior variance limits.
Given the stated population sizes, design objects, functions, and conditions, the congr eventually result holds.
Formal statement
hypotheses
Ω :
X Y :
∀ r
if
Ω r
then
ℝ
a :
ℕ → ℝ
hXY :
∀ᶠ r in atTop, ∀ w, X r w = Y r w
hY :
FiniteDesign.TendstoInProb D Y a
conclusion
FiniteDesign.TendstoInProb D X a
Proof (Lean source)
lemma FiniteDesign.TendstoInProb.congr_eventually
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{X Y : ∀ r, Ω r → ℝ} {a : ℕ → ℝ}
(hXY : ∀ᶠ r in atTop, ∀ w, X r w = Y r w)
(hY : FiniteDesign.TendstoInProb D Y a) :
FiniteDesign.TendstoInProb D X a := by
intro ε hε
apply Tendsto.congr' _ (hY ε hε)
filter_upwards [hXY] with r hr
apply (D r).Pr_congr
intro w
rw [hr w]
theorem probability_ge_tendsto_one_of_tendstoInProb Lemma probability_ge_tendsto_one_of_tendstoInProb in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma probability_ge_tendsto_one_of_tendstoInProb
{Ω : ℕ → Type*} [∀ r, Fintype (Ω r)] {D : ∀ r, FiniteDesign (Ω r)}
{X : ∀ r, Ω r → ℝ} {c t : ℝ}
(hX : FiniteDesign.TendstoInProb D X (fun _ => c)) (ht : t < c) :
Tendsto (fun r => (D r).Pr (fun w => t ≤ X r w)) atTop (nhds 1) := by
have hbad : Tendsto (fun r => (D r).Pr (fun w => X r w < t)) atTop (nhds 0) := by
apply squeeze_zero' (Eventually.of_forall fun r => (D r).Pr_nonneg _) _
(hX (c - t) (sub_pos.mpr ht))
exact Eventually.of_forall fun r => by
apply (D r).Pr_mono
intro w hw
have : c - t < c - X r w := by linarith
exact this.le.trans (by simpa using neg_le_abs (X r w - c))
apply Tendsto.congr' _ (by simpa using
(tendsto_const_nhds.sub hbad : Tendsto (fun r => 1 -
(D r).Pr (fun w => X r w < t)) atTop (nhds (1 - 0))))
filter_upwards [] with r
unfold FiniteDesign.Pr FiniteDesign.E FiniteDesign.ind
calc
1 - ∑ z, (D r).p z * (if X r z < t then 1 else 0) =
(∑ z, (D r).p z) -
∑ z, (D r).p z * (if X r z < t then 1 else 0) := by
rw [(D r).p_sum]
_ = ∑ z, ((D r).p z -
(D r).p z * (if X r z < t then 1 else 0)) := by
rw [Finset.sum_sub_distrib]
_ = ∑ z, (D r).p z * (if t ≤ X r z then 1 else 0) := by
apply Finset.sum_congr rfl
intro z _
by_cases hz : t ≤ X r z
· simp [hz, not_lt_of_ge hz]
· simp [hz, lt_of_not_ge hz]
theorem kneserEigenvalue_one
Given the stated population sizes, design objects, functions, and conditions, the kneser eigenvalue one result holds.
Formal statement
hypotheses
n M :
ℕ
hMtwo :
2 ≤ M
h2M :
2 * M ≤ n
conclusion
kneserEigenvalue n M ⟨1, by omega⟩ = -(M : ℝ) / ((n : ℝ) - M)
Proof (Lean source)
lemma kneserEigenvalue_one {n M : ℕ} (hMtwo : 2 ≤ M) (h2M : 2 * M ≤ n) :
kneserEigenvalue n M ⟨1, by omega⟩ =
-(M : ℝ) / ((n : ℝ) - M) := by
unfold kneserEigenvalue
simp only [descFactorial, Nat.sub_zero, pow_one, neg_mul, mul_one]
rw [Nat.cast_sub (by omega : M ≤ n)]
ring
theorem crossCov_eq_kneserEigenvalue_mul_armVar_of_sampleMean
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M :
ℕ
hMtwo :
2 ≤ M
h2M :
2 * M ≤ n
hMn :
M ≤ n
J :
hJohnson :
hKneser :
Y :
PotentialOutcome n M
z :
x :
Fin n → ℝ
htable :
armTable n M Y z = sampleMean M x
conclusion
Proof (Lean source)
lemma crossCov_eq_kneserEigenvalue_mul_armVar_of_sampleMean
{n M : ℕ} (hMtwo : 2 ≤ M) (h2M : 2 * M ≤ n) (hMn : M ≤ n)
(J : JohnsonProjections n M)
(hJohnson : JohnsonOrthogonalDecomposition n M J)
(hKneser : KneserAdjacencySpectrum n M) (Y : PotentialOutcome n M)
(z : Arm) (x : Fin n → ℝ)
(htable : armTable n M Y z = sampleMean M x) :
crossCov n M hMn Y z z =
kneserEigenvalue n M ⟨1, by omega⟩ * armVar n M hMn Y z := by
let f : Omega n M → ℝ := sampleMean M x
let fc : Omega n M → ℝ :=
fun S => f S - (slice n M hMn).E f
have hproj : J.proj ⟨1, by omega⟩ f = fc := by
simpa [f, fc] using johnsonProj_sampleMean_eq_centered (by omega) h2M J hJohnson x
have heigen := (exact_kneser_identity n M hMtwo h2M J Y hJohnson hKneser).1
⟨1, by omega⟩ f
rw [hproj] at heigen
unfold crossCov armTableCentered armVar
rw [htable]
change sliceInner n M hMn fc (kneserOp n M fc) =
kneserEigenvalue n M ⟨1, by omega⟩ * (slice n M hMn).Var f
rw [heigen, sliceInner_const_mul_right]
rw [(slice n M hMn).Var_eq]
unfold sliceInner fc
rw [show (fun A => (f A - (slice n M hMn).E f) *
(f A - (slice n M hMn).E f)) =
fun A => f A ^ 2 - 2 * (slice n M hMn).E f * f A +
((slice n M hMn).E f) ^ 2 by funext A; ring,
(slice n M hMn).E_add, (slice n M hMn).E_sub,
(slice n M hMn).E_const_mul, (slice n M hMn).E_const]
ring
theorem crossCovContrast_eq_kneserEigenvalue_mul_degreeOne_of_sampleMean
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M :
ℕ
hMtwo :
2 ≤ M
h2M :
2 * M ≤ n
hMn :
M ≤ n
J :
hJohnson :
hKneser :
Y :
PotentialOutcome n M
x :
Fin n → ℝ
conclusion
crossCovContrast n M hMn Y
= kneserEigenvalue n M ⟨1, by omega⟩ * degreeOneEnergy n M (by omega) hMn J Y
Proof (Lean source)
lemma crossCovContrast_eq_kneserEigenvalue_mul_degreeOne_of_sampleMean
{n M : ℕ} (hMtwo : 2 ≤ M) (h2M : 2 * M ≤ n) (hMn : M ≤ n)
(J : JohnsonProjections n M)
(hJohnson : JohnsonOrthogonalDecomposition n M J)
(hKneser : KneserAdjacencySpectrum n M) (Y : PotentialOutcome n M)
(x : Fin n → ℝ)
(htable : (fun S => armTable n M Y true S - armTable n M Y false S) =
sampleMean M x) :
crossCovContrast n M hMn Y =
kneserEigenvalue n M ⟨1, by omega⟩ * degreeOneEnergy n M (by omega) hMn J Y := by
let f : Omega n M → ℝ := sampleMean M x
let fc : Omega n M → ℝ :=
fun S => f S - (slice n M hMn).E f
have hproj : J.proj ⟨1, by omega⟩ f = fc := by
simpa [f, fc] using johnsonProj_sampleMean_eq_centered (by omega) h2M J hJohnson x
have heigen := (exact_kneser_identity n M hMtwo h2M J Y hJohnson hKneser).1
⟨1, by omega⟩ f
rw [hproj] at heigen
unfold degreeOneEnergy
rw [htable, hproj]
unfold crossCovContrast crossCov armTableCentered
rw [← disjointCov_sub_self n M h2M hMn]
rw [htable]
have hcentered : (fun S => armTable n M Y true S - armTable n M Y false S -
(slice n M hMn).E (sampleMean M x)) = fc := by
funext S
rw [show armTable n M Y true S - armTable n M Y false S =
sampleMean M x S from congrFun htable S]
rw [hcentered]
change sliceInner n M hMn fc (kneserOp n M fc) =
kneserEigenvalue n M ⟨1, by omega⟩ * sliceNorm n M hMn fc ^ 2
rw [heigen, sliceInner_const_mul_right]
unfold sliceNorm sliceNormSq
rw [Real.sq_sqrt (sliceInner_self_nonneg hMn fc)]
theorem scaledSigmaSq_eq_of_additive_armTables
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMtwo :
2 ≤ M
hMG :
M * G ≤ n
hG1two :
2 ≤ G1
hG0two :
2 ≤ G - G1
hG1pos :
0 < G1
hG1lt :
G1 < G
hMn :
M ≤ n
h2M :
2 * M ≤ n
J :
hJohnson :
hKneser :
Y :
PotentialOutcome n M
x1 x0 xd :
Fin n → ℝ
conclusion
(G : ℝ) * sigmaSq Y hMG hG1pos hG1lt
= (1 - kneserEigenvalue n M ⟨1, by omega⟩) * indepGroupVar n M G G1 hMn hG1pos hG1lt Y
+ (G : ℝ) * kneserEigenvalue n M ⟨1, by omega⟩ * degreeOneEnergy n M (by omega) hMn J Y
Proof (Lean source)
lemma scaledSigmaSq_eq_of_additive_armTables
{n M G G1 : ℕ} (hMtwo : 2 ≤ M) (hMG : M * G ≤ n)
(hG1two : 2 ≤ G1) (hG0two : 2 ≤ G - G1)
(hG1pos : 0 < G1) (hG1lt : G1 < G) (hMn : M ≤ n) (h2M : 2 * M ≤ n)
(J : JohnsonProjections n M)
(hJohnson : JohnsonOrthogonalDecomposition n M J)
(hKneser : KneserAdjacencySpectrum n M) (Y : PotentialOutcome n M)
(x1 x0 xd : Fin n → ℝ)
(htable1 : armTable n M Y true = sampleMean M x1)
(htable0 : armTable n M Y false = sampleMean M x0)
(htabled : (fun S => armTable n M Y true S - armTable n M Y false S) =
sampleMean M xd) :
(G : ℝ) * sigmaSq Y hMG hG1pos hG1lt =
(1 - kneserEigenvalue n M ⟨1, by omega⟩) *
indepGroupVar n M G G1 hMn hG1pos hG1lt Y +
(G : ℝ) * kneserEigenvalue n M ⟨1, by omega⟩ *
degreeOneEnergy n M (by omega) hMn J Y := by
have hv := (exact_pame_variance n M G G1 hMG hMtwo hG1two hG0two Y).2.1
have hc1 := crossCov_eq_kneserEigenvalue_mul_armVar_of_sampleMean
hMtwo h2M hMn J hJohnson hKneser Y true x1 htable1
have hc0 := crossCov_eq_kneserEigenvalue_mul_armVar_of_sampleMean
hMtwo h2M hMn J hJohnson hKneser Y false x0 htable0
have hct := crossCovContrast_eq_kneserEigenvalue_mul_degreeOne_of_sampleMean
hMtwo h2M hMn J hJohnson hKneser Y xd htabled
rw [hv, hc1, hc0, hct]
unfold indepGroupVar pFrac
have hGne : (G : ℝ) ≠ 0 := by exact_mod_cast (by omega : G ≠ 0)
have hG1ne : (G1 : ℝ) ≠ 0 := by exact_mod_cast (by omega : G1 ≠ 0)
have hG0ne : ((G - G1 : ℕ) : ℝ) ≠ 0 := by exact_mod_cast (by omega : G - G1 ≠ 0)
rw [Nat.cast_sub hG1lt.le]
field_simp
ring
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Software 8 declarations The declarations in this file are cited logical gates.
Versioned software contracts
The declarations in this file are cited logical gates. They expose only the
mathematical claims attributed to the pinned clubSandwich and sandwich
sources; no Lean proof of package semantics is asserted.
def UnweightedFullRankLmFit
Algebraic semantics of an unweighted full-column-rank OLS fit with possibly unequal cluster sizes, including fitted-observation residuals and the normal equations.
Definition (Lean source)
parameters
def
UnweightedFullRankLmFit m X y e :
Prop
given by
∃ beta : Fin p → ℝ,
clause 1
g i :
e g i = y g i - ∑ j, X g i j * beta j
clause 2
j :
∑ g, ∑ i, X g i j * e g i = 0
clause 3
gram :
∑ g, (X g)ᵀ * X g
gram * gram⁻¹ = 1
gram⁻¹ * gram = 1
structure ClubSandwichCall
The observable configuration and return value of a clubSandwich::vcovCR call.
def pinnedClubSandwichCall
The pinned version-0.7.0 default CR2 call used in the paper.
Definition (Lean source)
parameters
def
structure SandwichBreadCall
The observable configuration and return value of a sandwich::bread call.
def pinnedSandwichBreadCall
The pinned version-3.1-3 unweighted bread call used in the paper.
Definition (Lean source)
parameters
def
given by
clause 1
version := "3.1-3"
clause 2
fittedObservations := Nobs
clause 3
unweighted := true
clause 4
result := bread
def cr2Meat
The CR2 cluster meat corresponding to arbitrarily sized block matrices and residual vectors.
James E. Pustejovsky (2026), clubSandwich version 0.7.0 source package, CRAN, R/lm.R lines 47--52 and 70--72; R/S3-methods.R lines 21--33 and 63--65; R/clubSandwich.R lines 167--175, 216--225, 237--287; and R/CR-adjustments.R lines 5--7 and 22--42. The source tarball SHA-256 is f3cd9cd5840022b8d1354dce3e7e4a820edcd47411b9f95bdd200c4e040032ed. For an unweighted full-rank fit with an arbitrary fitted-observation count and arbitrarily sized cluster-row blocks, identity working target, and positive-definite cluster leverage complements, the returned CR2 matrix is the stated sandwich.
Definition (Lean source)
parameters
def
ClubSandwichCR2Contract m Nobs call X y e H A bread breadTilde :
Sort 0
given by
call.version = "0.7.0" ∧
call.vcovType = "CR2" ∧
call.targetOmitted = true ∧
call.inverseVarOmitted = true ∧
call.formOmitted = true ∧
call.identityTarget = true ∧
call.clustersAreGroups = true ∧
Nobs = ∑ g, m g ∧
(UnweightedFullRankLmFit m X y e → (∀ g, H g = X g * ((∑ g, (X g)ᵀ * X g)⁻¹) * (X g)ᵀ) → (∀ g, PosDef (1 - H g)) → (∀ g, PosDef (A g) ∧ A g = (A g)ᵀ ∧ A g * A g * (1 - H g) = 1) → breadTilde = ((Nobs : ℝ)⁻¹) • bread → call.result = breadTilde * cr2Meat m X e A * breadTilde)
Achim Zeileis and Thomas Lumley (2026), sandwich version 3.1-3 source package, CRAN, R/bread.R lines 10--15. The source tarball SHA-256 is 960006cf4fcbada936b43acd04ddd8c0d1255570f41dda046ce778873f546134. For a full-column-rank unweighted lm fit with arbitrarily sized cluster-row blocks, bread is the fitted-observation count times the inverse Gram matrix.
Definition (Lean source)
parameters
def
SandwichLmBreadContract m Nobs call X Xblock y e :
Sort 0
given by
call.version = "3.1-3" ∧
call.fittedObservations = Nobs ∧
call.unweighted = true ∧
Nobs = ∑ g, m g ∧
(UnweightedFullRankLmFit m Xblock y e → Xᵀ * X = ∑ g, (Xblock g)ᵀ * Xblock g → call.result = (Nobs : ℝ) • ((Xᵀ * X)⁻¹))
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.Helpers.Witness 4 declarations This file defines the balanced four-plus/four-minus vector and its composition schedule for the paper's finite exact-moment witness.
Balanced eight-unit witness
This file defines the balanced four-plus/four-minus vector and its composition schedule for the paper's finite exact-moment witness.
The balanced eight-unit sign vector.
Definition (Lean source)
structure EightUnitWitness
A bundled witness whose type fixes the paper's population, group size, group count, treated count, and grouped-unit count.
Definition (Lean source)
fields
schedule :
PotentialOutcome 8 2
groups :
ℕ := 4
treated :
ℕ := 2
groupedUnits :
ℕ := 8
groups_eq :
groups = 4
treated_eq :
treated = 2
grouped_eq :
groupedUnits = 2 * groups
The bundled balanced n = N = 8, M = 2, G = 4, G1 = 2 witness.
def witness8
The schedule component of the bundled eight-unit witness.
Definition (Lean source)
def
given by
witness8Bundle.schedule
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TClubsandwichConsumer 4 declarations The deterministic equal-group identity is conditional on the two versioned software contracts and on the explicit intercept-plus-binary-treatment fit data.
clubSandwich CR2 consumer identity
The deterministic equal-group identity is conditional on the two versioned software contracts and on the explicit intercept-plus-binary-treatment fit data.
def EqualGroupBinaryFitSpec
The algebraic data fixed by an unweighted equal-group intercept-plus-binary- treatment OLS fit.
Definition (Lean source)
parameters
def
EqualGroupBinaryFitSpec Y w rowUnit X Xstack y e H A bread breadTilde :
Prop
given by
clause 1
g i j :
X g i j = if j = 0 then 1 else if g ∈ w.2.1 then 1 else 0
clause 2
g i :
y g i = Y (w.1.1 g) (rowUnit g i) (g ∈ w.2.1)
clause 4
UnweightedFullRankLmFit (fun _ => M) X y e
clause 6
Xstackᵀ * Xstack = ∑ g, (X g)ᵀ * X g
clause 7
g :
H g = X g * ((Xstackᵀ * Xstack)⁻¹) * (X g)ᵀ
clause 10
breadTilde = ((G * M : ℕ) : ℝ)⁻¹ • bread
Given the stated population sizes, design objects, functions, and conditions, the matrix sandwich rank one result holds.
Formal statement
Proof (Lean source)
lemma matrix_sandwich_rank_one {a b : Type} [Fintype a] [Fintype b]
(B : Matrix b b ℝ) (X : Matrix a b ℝ) (A : Matrix a a ℝ) (e : a → ℝ)
(hB : B = Bᵀ) :
B * Xᵀ * A * vecMulVec e e * Aᵀ * X * B =
vecMulVec (B *ᵥ (Xᵀ *ᵥ (A *ᵥ e)))
(B *ᵥ (Xᵀ *ᵥ (A *ᵥ e))) := by
let P := B * Xᵀ * A
have hfactor : B * Xᵀ * A * vecMulVec e e * Aᵀ * X * B =
P * vecMulVec e e * Pᵀ := by
dsimp [P]
simp only [Matrix.transpose_mul, Matrix.transpose_transpose]
rw [← hB]
simp [Matrix.mul_assoc]
rw [hfactor, Matrix.mul_vecMulVec, Matrix.vecMulVec_mul,
Matrix.vecMul_transpose]
simp [P, Matrix.mulVec_mulVec, Matrix.mul_assoc]
Given the stated population sizes, design objects, functions, and conditions, the pos def rank one inverse sqrt result holds.
Formal statement
Proof (Lean source)
lemma posDef_rankOne_inverse_sqrt {M q : ℕ} (hM : 0 < M) (hq : 2 ≤ q)
(H A : Matrix (Fin M) (Fin M) ℝ)
(hH : ∀ i j, H i j = 1 / ((M : ℝ) * (q : ℝ)))
(hA : A.PosDef) (hAsymm : A = Aᵀ)
(hA2 : A * A * (1 - H) = 1) :
∀ i j, A i j = (if i = j then 1 else 0) +
(sqrt ((q : ℝ) / ((q - 1 : ℕ) : ℝ)) - 1) / (M : ℝ) := by
classical
let u : Fin M → ℝ := fun _ => 1
let v : Fin M → Fin M → ℝ := fun j k =>
(if k = j then 1 else 0) - 1 / (M : ℝ)
let a : ℝ := sqrt ((q : ℝ) / ((q - 1 : ℕ) : ℝ))
let Q : Matrix (Fin M) (Fin M) ℝ := 1 - H
have hMr : (M : ℝ) ≠ 0 := by exact_mod_cast hM.ne'
have hqr : (q : ℝ) ≠ 0 := by positivity
have hq1r : ((q - 1 : ℕ) : ℝ) ≠ 0 := by
exact_mod_cast (by omega : q - 1 ≠ 0)
have ha_pos : 0 < a := by
dsimp [a]
positivity
have hEqVec (x : Fin M → ℝ) :
A *ᵥ (A *ᵥ (Q *ᵥ x)) = x := by
simpa only [Q, Matrix.mulVec_mulVec, Matrix.one_mulVec, Matrix.mul_assoc] using
congrArg (fun K => K *ᵥ x) hA2
have hQv (j : Fin M) : Q *ᵥ v j = v j := by
ext i
simp [Q, mulVec, dotProduct, Matrix.one_apply, hH, v, hMr,
sub_mul, mul_sub]
field_simp
ring
have hA2v (j : Fin M) : A *ᵥ (A *ᵥ v j) = v j := by
simpa [hQv] using hEqVec (v j)
have hplusPos : (A + 1).PosDef := hA.add .one
have hplusInj : Injective (A + 1).mulVec :=
Matrix.mulVec_injective_iff_isUnit.mpr hplusPos.isUnit
have hAv (j : Fin M) : A *ᵥ v j = v j := by
apply hplusInj
simp only [Matrix.add_mulVec, Matrix.one_mulVec]
rw [hA2v]
abel
have hQu : Q *ᵥ u = (((q - 1 : ℕ) : ℝ) / (q : ℝ)) • u := by
ext i
simp [Q, mulVec, dotProduct, Matrix.one_apply, hH, u, hMr,
Nat.cast_sub (by omega : 1 ≤ q)]
field_simp
have hA2u : A *ᵥ (A *ᵥ u) = a ^ 2 • u := by
have he := hEqVec u
rw [hQu] at he
simp only [Matrix.mulVec_smul] at he
ext i
have hei := congrFun he i
simp only [Pi.smul_apply, u, smul_eq_mul] at hei ⊢
rw [Real.sq_sqrt (by positivity : 0 ≤ (q : ℝ) / ((q - 1 : ℕ) : ℝ))]
field_simp at hei ⊢
linarith
have hasmulPos : (a • (1 : Matrix (Fin M) (Fin M) ℝ)).PosDef :=
Matrix.PosDef.smul .one ha_pos
have haplusPos : (A + a • 1).PosDef := hA.add hasmulPos
have haplusInj : Injective (A + a • 1).mulVec :=
Matrix.mulVec_injective_iff_isUnit.mpr haplusPos.isUnit
have hAu : A *ᵥ u = a • u := by
apply haplusInj
simp only [Matrix.add_mulVec, Matrix.smul_mulVec, Matrix.one_mulVec]
rw [hA2u, Matrix.mulVec_smul]
module
intro i j
have hv := congrFun (hAv j) i
have hu := congrFun hAu i
simp only [mulVec, dotProduct, v] at hv
simp only [mulVec, dotProduct, u, Pi.smul_apply] at hu
simp_rw [mul_sub] at hv
rw [Finset.sum_sub_distrib] at hv
simp at hu
simp only [mul_ite, mul_one, mul_zero, Finset.sum_ite_eq', Finset.mem_univ,
if_true] at hv
rw [← Finset.sum_mul] at hv
rw [hu] at hv
dsimp [a] at hv hu ⊢
field_simp at hv ⊢
linarith
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hM :
2 ≤ M
hG1 :
2 ≤ G1
hG0 :
2 ≤ G - G1
Y :
PotentialOutcome n M
w :
PartitionTuple n M G × TreatmentSpace G G1
hFit :
EqualGroupBinaryFitSpec Y w rowUnit X Xstack y e H A bread breadTilde
hClubSandwichCR2Contract_of_gate :
ClubSandwichCR2Contract (fun _ => M) (G * M) (pinnedClubSandwichCall Sigma) X y e H A bread breadTilde
hSandwichLmBreadContract_of_gate :
SandwichLmBreadContract (fun _ => M) (G * M) (pinnedSandwichBreadCall (G * M) bread) Xstack X y e
conclusion
Sigma 1 1 = cr2Var Y hG1 hG0 w
Proof (Lean source)
theorem clubsandwich_cr2_consumer_identity {n M G G1 : ℕ}
(hM : 2 ≤ M) (hG1 : 2 ≤ G1) (hG0 : 2 ≤ G - G1)
(Y : PotentialOutcome n M)
(w : PartitionTuple n M G × TreatmentSpace G G1)
(rowUnit : ∀ g, Fin M ≃ {i : Fin n // i ∈ (w.1.1 g).1})
(X : Fin G → Matrix (Fin M) (Fin 2) ℝ)
(Xstack : Matrix (Fin (G * M)) (Fin 2) ℝ)
(y : Fin G → Fin M → ℝ)
(e : Fin G → Fin M → ℝ)
(H A : Fin G → Matrix (Fin M) (Fin M) ℝ)
(bread breadTilde Sigma : Matrix (Fin 2) (Fin 2) ℝ)
(hFit : EqualGroupBinaryFitSpec Y w rowUnit X Xstack y e H A bread breadTilde)
(hClubSandwichCR2Contract_of_gate :
ClubSandwichCR2Contract (fun _ => M) (G * M)
(pinnedClubSandwichCall Sigma) X y e H A bread breadTilde)
(hSandwichLmBreadContract_of_gate :
SandwichLmBreadContract (fun _ => M) (G * M)
(pinnedSandwichBreadCall (G * M) bread) Xstack X y e) :
Sigma 1 1 = cr2Var Y hG1 hG0 w := by
classical
rcases hFit with
⟨hX, hy, hGroupMean, hLm, hResidualMean, hStack, hH, hHpos, hA,
hBreadScale⟩
rcases hClubSandwichCR2Contract_of_gate with
⟨_, _, _, _, _, _, _, _, hClub⟩
rcases hSandwichLmBreadContract_of_gate with
⟨_, _, _, _, hSandwich⟩
have hG1le : G1 ≤ G := by omega
have hGpos : 0 < G := by omega
have hMpos : 0 < M := by omega
have hG0pos : 0 < G - G1 := by omega
have hG1r : (G1 : ℝ) ≠ 0 := by positivity
have hG0r : (G : ℝ) - (G1 : ℝ) ≠ 0 := by
rw [← Nat.cast_sub hG1le]
positivity
have hGram : ∀ i j, (Xstackᵀ * Xstack) i j =
if i = 0 then
if j = 0 then (M : ℝ) * G else (M : ℝ) * G1
else if j = 0 then (M : ℝ) * G1 else (M : ℝ) * G1 := by
intro i j
rw [hStack, Matrix.sum_apply]
simp only [Matrix.mul_apply, Matrix.transpose_apply]
fin_cases i <;> fin_cases j
· simp [hX]; ring
· simp [hX, Finset.sum_ite_irrel, w.2.2]; ring
· simp [hX, Finset.sum_ite_irrel, w.2.2]; ring
· simp [hX, Finset.sum_ite_irrel, w.2.2]; ring
have hBread : bread = ((G * M : ℕ) : ℝ) • (Xstackᵀ * Xstack)⁻¹ := by
simpa [pinnedSandwichBreadCall] using hSandwich hLm hStack
have hBreadTilde : breadTilde = (Xstackᵀ * Xstack)⁻¹ := by
rw [hBreadScale, hBread]
ext i j
simp [Nat.cast_mul]
field_simp
have hBreadRow : ∀ i, breadTilde 1 i =
if i = 0 then -1 / ((M : ℝ) * (G - G1 : ℝ))
else 1 / ((M : ℝ) * (G - G1 : ℝ)) + 1 / ((M : ℝ) * (G1 : ℝ)) := by
intro i
rw [hBreadTilde, Matrix.inv_def]
fin_cases i <;>
simp [Matrix.smul_apply, Matrix.det_fin_two, Matrix.adjugate_fin_two, hGram,
Nat.cast_sub hG1le] <;>
field_simp [hMpos.ne', hG1r, hG0r]; ring
have hBreadCol : ∀ i, breadTilde i 1 =
if i = 0 then -1 / ((M : ℝ) * (G - G1 : ℝ))
else 1 / ((M : ℝ) * (G - G1 : ℝ)) + 1 / ((M : ℝ) * (G1 : ℝ)) := by
intro i
rw [hBreadTilde, Matrix.inv_def]
fin_cases i <;>
simp [Matrix.smul_apply, Matrix.det_fin_two, Matrix.adjugate_fin_two, hGram,
Nat.cast_sub hG1le] <;>
field_simp [hMpos.ne', hG1r, hG0r]; ring
have hBread00 : breadTilde 0 0 = 1 / ((M : ℝ) * (G - G1 : ℝ)) := by
rw [hBreadTilde, Matrix.inv_def]
simp [Matrix.smul_apply, Matrix.det_fin_two, Matrix.adjugate_fin_two, hGram]
field_simp [hMpos.ne', hG1r, hG0r]
have hHEntry : ∀ g i j, H g i j =
1 / ((M : ℝ) * (armCount G G1 (g ∈ w.2.1) : ℝ)) := by
intro g i j
rw [hH g, ← hBreadTilde]
simp only [Matrix.mul_apply, Matrix.transpose_apply, Fin.sum_univ_two]
by_cases hg : g ∈ w.2.1
· simp [hX, hBreadRow, hBreadCol, hBread00, hg, armCount]
field_simp [hMpos.ne', hG1r]
ring
· simp [hX, hBreadRow, hBreadCol, hBread00, hg, armCount, Nat.cast_sub hG1le]
have hAEntry : ∀ g i j, A g i j =
(if i = j then 1 else 0) +
(sqrt ((armCount G G1 (g ∈ w.2.1) : ℝ) /
((armCount G G1 (g ∈ w.2.1) - 1 : ℕ) : ℝ)) - 1) / (M : ℝ) := by
intro g
have hq : 2 ≤ armCount G G1 (g ∈ w.2.1) := by
by_cases hg : g ∈ w.2.1 <;> simp [armCount, hg, hG1, hG0]
exact posDef_rankOne_inverse_sqrt hMpos hq (H g) (A g) (hHEntry g)
(hA g).1 (hA g).2.1 (hA g).2.2
have hSigma : Sigma = breadTilde * cr2Meat (fun _ => M) X e A * breadTilde := by
simpa [pinnedClubSandwichCall] using
hClub hLm (by simpa [hStack] using hH) hHpos hA hBreadScale
have hMne : (M : ℝ) ≠ 0 := by exact_mod_cast (by omega : M ≠ 0)
have hAColSum (g : Fin G) (j : Fin M) :
(∑ i, A g i j) =
sqrt ((armCount G G1 (g ∈ w.2.1) : ℝ) /
((armCount G G1 (g ∈ w.2.1) - 1 : ℕ) : ℝ)) := by
simp_rw [hAEntry]
rw [Finset.sum_add_distrib]
simp only [Finset.sum_ite_eq', Finset.mem_univ, if_true, Finset.sum_const,
Finset.card_univ, card_fin, nsmul_eq_mul]
field_simp
ring
have hASumE (g : Fin G) :
(∑ i, (A g *ᵥ e g) i) =
sqrt ((armCount G G1 (g ∈ w.2.1) : ℝ) /
((armCount G G1 (g ∈ w.2.1) - 1 : ℕ) : ℝ)) * ∑ j, e g j := by
simp only [mulVec, dotProduct]
rw [Finset.sum_comm]
simp_rw [← Finset.sum_mul, hAColSum]
rw [Finset.mul_sum]
have hBreadSymm : breadTilde = breadTildeᵀ := by
ext i j
fin_cases i <;> fin_cases j <;>
simp [Matrix.transpose_apply, hBreadRow, hBreadCol]
let score : Fin G → Fin 2 → ℝ := fun g =>
breadTilde *ᵥ ((X g)ᵀ *ᵥ (A g *ᵥ e g))
have hScore (g : Fin G) : score g 1 =
if hg : g ∈ w.2.1 then
sqrt ((G1 : ℝ) / ((G1 - 1 : ℕ) : ℝ)) *
(obsGroupMean Y w g - armObsMean Y w true) / (G1 : ℝ)
else
-(sqrt (((G - G1 : ℕ) : ℝ) / ((G - G1 - 1 : ℕ) : ℝ)) *
(obsGroupMean Y w g - armObsMean Y w false) / ((G - G1 : ℕ) : ℝ)) := by
unfold score
simp only [mulVec, dotProduct, Matrix.transpose_apply, Fin.sum_univ_two]
by_cases hg : g ∈ w.2.1
· simp [hg, hX, hBreadRow]
rw [show (∑ i, ∑ j, A g i j * e g j) =
sqrt ((G1 : ℝ) / ((G1 - 1 : ℕ) : ℝ)) * ∑ j, e g j by
simpa [mulVec, dotProduct, hg, armCount] using hASumE g]
have hr := hResidualMean g
simp [hg] at hr
have hrsum : (∑ j, e g j) = (M : ℝ) *
(obsGroupMean Y w g - armObsMean Y w true) := by
calc
(∑ j, e g j) =
(obsGroupMean Y w g - armObsMean Y w true) * (M : ℝ) :=
(div_eq_iff hMne).mp hr
_ = _ := by ring
rw [hrsum]
field_simp
rw [Real.sqrt_div (by positivity)]
ring
· simp [hg, hX, hBreadRow]
rw [show (∑ i, ∑ j, A g i j * e g j) =
sqrt (((G - G1 : ℕ) : ℝ) / ((G - G1 - 1 : ℕ) : ℝ)) * ∑ j, e g j by
simpa [mulVec, dotProduct, hg, armCount] using hASumE g]
have hr := hResidualMean g
simp [hg] at hr
have hrsum : (∑ j, e g j) = (M : ℝ) *
(obsGroupMean Y w g - armObsMean Y w false) := by
calc
(∑ j, e g j) =
(obsGroupMean Y w g - armObsMean Y w false) * (M : ℝ) :=
(div_eq_iff hMne).mp hr
_ = _ := by ring
rw [hrsum]
rw [Nat.cast_sub (by omega : G1 ≤ G)]
have hcast : (G1 : ℝ) < (G : ℝ) := by
exact_mod_cast (by omega : G1 < G)
have hG0pos : 0 < (G : ℝ) - (G1 : ℝ) := by
linarith
have hG0r : (G : ℝ) - (G1 : ℝ) ≠ 0 := ne_of_gt hG0pos
have hsqrt0 : sqrt ((G - G1 - 1 : ℕ) : ℝ) ≠ 0 := by
exact ne_of_gt (Real.sqrt_pos.2 (by exact_mod_cast (by omega : 0 < G - G1 - 1)))
field_simp
rw [Real.sqrt_div hG0pos.le]
field_simp [hsqrt0]
have hSandwich : breadTilde * cr2Meat (fun _ => M) X e A * breadTilde =
∑ g, vecMulVec (score g) (score g) := by
unfold cr2Meat
rw [Matrix.mul_sum, Matrix.sum_mul]
apply Finset.sum_congr rfl
intro g _
simpa [score, Matrix.mul_assoc] using
matrix_sandwich_rank_one breadTilde (X g) (A g) (e g) hBreadSymm
rw [hSigma, hSandwich]
simp only [Matrix.sum_apply, Matrix.vecMulVec_apply]
simp_rw [hScore]
have hG1nz : (G1 : ℝ) ≠ 0 := by positivity
have hG1mnz : ((G1 - 1 : ℕ) : ℝ) ≠ 0 := by
exact_mod_cast (by omega : G1 - 1 ≠ 0)
have hG0nz : ((G - G1 : ℕ) : ℝ) ≠ 0 := by positivity
have hG0mnz : ((G - G1 - 1 : ℕ) : ℝ) ≠ 0 := by
exact_mod_cast (by omega : G - G1 - 1 ≠ 0)
have htreated (d : ℝ) :
(sqrt ((G1 : ℝ) / ((G1 - 1 : ℕ) : ℝ)) * d / (G1 : ℝ)) ^ 2 =
d ^ 2 / ((G1 - 1 : ℕ) : ℝ) / (G1 : ℝ) := by
rw [show (sqrt ((G1 : ℝ) / ((G1 - 1 : ℕ) : ℝ)) * d /
(G1 : ℝ)) ^ 2 =
sqrt ((G1 : ℝ) / ((G1 - 1 : ℕ) : ℝ)) ^ 2 * d ^ 2 /
(G1 : ℝ) ^ 2 by ring]
rw [Real.sq_sqrt (by positivity)]
field_simp
have hcontrol (d : ℝ) :
(sqrt (((G - G1 : ℕ) : ℝ) / ((G - G1 - 1 : ℕ) : ℝ)) * d /
((G - G1 : ℕ) : ℝ)) ^ 2 =
d ^ 2 / ((G - G1 - 1 : ℕ) : ℝ) / ((G - G1 : ℕ) : ℝ) := by
rw [show (sqrt (((G - G1 : ℕ) : ℝ) /
((G - G1 - 1 : ℕ) : ℝ)) * d / ((G - G1 : ℕ) : ℝ)) ^ 2 =
sqrt (((G - G1 : ℕ) : ℝ) /
((G - G1 - 1 : ℕ) : ℝ)) ^ 2 * d ^ 2 /
((G - G1 : ℕ) : ℝ) ^ 2 by ring]
rw [Real.sq_sqrt (by positivity)]
field_simp
simp only [← pow_two]
rw [show (∑ g,
(if hg : g ∈ w.2.1 then
sqrt ((G1 : ℝ) / ((G1 - 1 : ℕ) : ℝ)) *
(obsGroupMean Y w g - armObsMean Y w true) / (G1 : ℝ)
else
-(sqrt (((G - G1 : ℕ) : ℝ) / ((G - G1 - 1 : ℕ) : ℝ)) *
(obsGroupMean Y w g - armObsMean Y w false) /
((G - G1 : ℕ) : ℝ))) ^ 2) =
∑ g, if g ∈ w.2.1 then
(obsGroupMean Y w g - armObsMean Y w true) ^ 2 /
((G1 - 1 : ℕ) : ℝ) / (G1 : ℝ)
else
(obsGroupMean Y w g - armObsMean Y w false) ^ 2 /
((G - G1 - 1 : ℕ) : ℝ) / ((G - G1 : ℕ) : ℝ) by
apply Finset.sum_congr rfl
intro g _
by_cases hg : g ∈ w.2.1
· rw [dif_pos hg, if_pos hg]
exact htreated _
· rw [dif_neg hg, if_neg hg, neg_sq]
exact hcontrol _]
rw [Finset.sum_ite]
simp only [cr2Var, armSampleVar, realizedArmSet, armCount,
Bool.false_eq_true, ↓reduceIte]
have hfilterPos : Finset.univ.filter (fun g : Fin G => g ∈ w.2.1) = w.2.1 := by
ext g
simp
have hfilterNeg : Finset.univ.filter (fun g : Fin G => g ∉ w.2.1) =
univ \ w.2.1 := by
ext g
simp
rw [hfilterPos, hfilterNeg]
simp only [Finset.sum_div]
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TCr2PhaseFrontier 1 declarations CR2 dense-regime phase frontier
CR2 dense-regime phase frontier
Given the stated population sizes, design objects, functions, and conditions, the cr2 phase frontier result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p rho B cSigma :
ℝ
J :
∀ r, JohnsonProjections (A.popSize r) M
hClass :
DenseScheduleClass A p rho B cSigma
hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
conclusion
conclusion 1
FiniteDesign.TendstoInProb A.design (fun r w => (A.groups r : ℝ) * A.cr2 r w - A.leadingVariance r) (fun _ => 0)
conclusion 2
FiniteDesign.TendstoInProb A.design (fun r w => (A.groups r : ℝ) * (A.cr2 r w - A.variance r) - rho * A.degreeOne (by omega) J r) (fun _ => 0)
conclusion 3
ε :
ℝ
0 < ε
Proof (Lean source)
theorem cr2_phase_frontier {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p rho B cSigma : ℝ) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(hClass : DenseScheduleClass A p rho B cSigma)
(hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M) :
FiniteDesign.TendstoInProb A.design
(fun r w => (A.groups r : ℝ) * A.cr2 r w - A.leadingVariance r)
(fun _ => 0) ∧
FiniteDesign.TendstoInProb A.design
(fun r w => (A.groups r : ℝ) * (A.cr2 r w - A.variance r) -
rho * A.degreeOne (by omega) J r)
(fun _ => 0) ∧
(∀ ε : ℝ, 0 < ε → Tendsto (fun r =>
(A.design r).Pr (fun w => A.cr2 r w / A.variance r < 1 - ε))
atTop (nhds 0)) ∧
(FiniteDesign.TendstoInProb A.design
(fun r w => A.cr2 r w / A.variance r) (fun _ => 1) ↔
Tendsto (fun r =>
rho * A.degreeOne (by omega) J r /
(A.leadingVariance r - rho * A.degreeOne (by omega) J r))
atTop (nhds 0)) := by
let d : ℕ → ℝ := fun r =>
A.leadingVariance r - rho * A.degreeOne (by omega) J r
let q : ℕ → ℝ := fun r => rho * A.degreeOne (by omega) J r / d r
let a : ℕ → ℝ := fun r => A.leadingVariance r / d r
have hfirst := cr2_centered_tendstoInProb hM A p B J hClass.growth hClass.fraction
hClass.bounded hJohnsonOrthogonalDecomposition_of_gate hKneserAdjacencySpectrum_of_gate
have hsecond := cr2_variance_gap_tendstoInProb hM A p rho B J hClass.growth
hClass.fraction hClass.sampling hClass.bounded
hJohnsonOrthogonalDecomposition_of_gate hKneserAdjacencySpectrum_of_gate
have hratio : FiniteDesign.TendstoInProb A.design
(fun r w => A.cr2 r w / A.variance r) a := by
simpa [a, d] using cr2_ratio_leading_tendstoInProb hM A p rho B cSigma J hClass
hJohnsonOrthogonalDecomposition_of_gate hKneserAdjacencySpectrum_of_gate
obtain ⟨C0, hC0, hdense⟩ := dense_projection_limit hM p rho B
have hv := (hdense A J hClass.growth hClass.fraction hClass.sampling hClass.bounded
hJohnsonOrthogonalDecomposition_of_gate hKneserAdjacencySpectrum_of_gate).1
have hxlow := scaledVariance_eventually_lower A cSigma hClass.nondegenerate
have herr : ∀ᶠ r in atTop,
|(A.groups r : ℝ) * A.variance r - d r| < cSigma / 4 := by
have heps : 0 < cSigma / 4 := by linarith [hClass.nondegenerate.1]
simpa [Real.dist_eq, d] using (Metric.tendsto_atTop.1 hv (cSigma / 4) heps)
have hdlow : ∀ᶠ r in atTop, cSigma / 4 < d r := by
filter_upwards [hxlow, herr] with r hx he
have hab := (le_abs_self ((A.groups r : ℝ) * A.variance r - d r)).trans_lt he
linarith
have haq : ∀ᶠ r in atTop, a r = 1 + q r := by
filter_upwards [hdlow] with r hd
have hdne : d r ≠ 0 := ne_of_gt (by linarith [hClass.nondegenerate.1])
have hdenne : A.leadingVariance r -
rho * A.degreeOne (by omega) J r ≠ 0 := by
simpa [d] using hdne
dsimp [a, q, d]
field_simp [hdenne]
ring
have haOne : ∀ᶠ r in atTop, 1 ≤ a r := by
filter_upwards [haq, hdlow] with r heq hd
rw [heq]
have hrho : 0 ≤ rho := hClass.sampling.1
have henergy : 0 ≤ A.degreeOne (by omega) J r := by
unfold ScheduleArray.degreeOne degreeOneEnergy
positivity
have hd0 : 0 ≤ d r := by linarith [hClass.nondegenerate.1]
exact le_add_of_nonneg_right (div_nonneg (mul_nonneg hrho henergy) hd0)
refine ⟨hfirst, hsecond, lower_tail_vanishes_of_tendstoInProb hratio haOne, ?_⟩
constructor
· intro hratioOne
have ha := tendstoInProb_target_unique hratio hratioOne
have hsub := ha.sub_const 1
have heq : ∀ᶠ r in atTop, a r - 1 = q r := by
filter_upwards [haq] with r hr
linarith
simpa [q, d] using hsub.congr' heq
· intro hq
have ha : Tendsto a atTop (nhds 1) := by
have hs := (tendsto_const_nhds :
Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1)).add hq
have heq : ∀ᶠ r in atTop, 1 + q r = a r := by
filter_upwards [haq] with r hr
exact hr.symm
simpa using hs.congr' heq
exact FiniteDesign.TendstoInProb.retarget hratio ha
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TDenseProjectionLimit 12 declarations The deterministic group-scaled variance expansion is stated for bounded schedule arrays without a scaled-variance nondegeneracy premise.
Dense projection expansion
The deterministic group-scaled variance expansion is stated for bounded schedule arrays without a scaled-variance nondegeneracy premise.
The pooled Johnson degrees at least two, at the group-count scale.
Definition (Lean source)
parameters
def
higherDegreeContribution A J r :
ℝ
theorem armTable_abs_le_of_boundedSchedule
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
conclusion
|armTable (A.popSize r) M (A.schedule r) z S| ≤ B
Proof (Lean source)
lemma armTable_abs_le_of_boundedSchedule {M : ℕ} (A : ScheduleArray M) (B : ℝ)
(hBounded : BoundedSchedule A B) (r : ℕ) (z : Arm)
(S : Omega (A.popSize r) M) :
|armTable (A.popSize r) M (A.schedule r) z S| ≤ B := by
have hMpos : (0 : ℝ) < M := by exact_mod_cast A.groupSize_ge_two.trans' (by omega)
rw [armTable, abs_div, abs_of_pos hMpos]
apply (div_le_iff₀ hMpos).2
calc
|∑ i ∈ S.1.attach, A.schedule r S i z| ≤
∑ i ∈ S.1.attach, |A.schedule r S i z| := Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _i ∈ S.1.attach, B := by
gcongr with i hi
exact hBounded.2 r S i z
_ = B * M := by simp [S.2]; ring
theorem sliceInner_self_le_sq_of_abs_le
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
n M :
ℕ
hMn :
M ≤ n
f :
Omega n M → ℝ
K :
ℝ
hf :
∀ S, |f S| ≤ K
conclusion
sliceInner n M hMn f f ≤ K ^ 2
Proof (Lean source)
lemma sliceInner_self_le_sq_of_abs_le {n M : ℕ} (hMn : M ≤ n)
(f : Omega n M → ℝ) (K : ℝ) (hf : ∀ S, |f S| ≤ K) :
sliceInner n M hMn f f ≤ K ^ 2 := by
unfold sliceInner FiniteDesign.E
calc
(∑ S, (slice n M hMn).p S * (f S * f S)) ≤
∑ S, (slice n M hMn).p S * K ^ 2 := by
apply Finset.sum_le_sum
intro S _
have hK : 0 ≤ K := (abs_nonneg (f S)).trans (hf S)
apply mul_le_mul_of_nonneg_left _ ((slice n M hMn).p_nonneg S)
simpa [pow_two] using (sq_le_sq₀ (abs_nonneg (f S)) hK).2 (hf S)
_ = K ^ 2 := by
rw [← Finset.sum_mul]
simp [(slice n M hMn).p_sum]
theorem johnsonProjection_energy_le_of_abs_le
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
Proof (Lean source)
lemma johnsonProjection_energy_le_of_abs_le {n M : ℕ} (h2M : 2 * M ≤ n)
(J : JohnsonProjections n M)
(k : Fin (M + 1)) (f : Omega n M → ℝ) (K : ℝ)
(hf : ∀ S, |f S| ≤ K) :
sliceNorm n M (by omega) (J.proj k f) ^ 2 ≤ K ^ 2 := by
let P := J.proj k f
let R : Omega n M → ℝ := fun S => f S - P S
have hP : P ∈ johnsonHarmonicSpace n M J.slice_nonempty k := by
rw [← J.range_eq k]
exact ⟨f, rfl⟩
have hOrth : sliceInner n M J.slice_nonempty R P = 0 :=
J.residual_orthogonal k f P hP
have hdecomp : sliceInner n M J.slice_nonempty f f =
sliceInner n M J.slice_nonempty R R + sliceInner n M J.slice_nonempty P P := by
have hpoint : (fun S => f S * f S) = fun S =>
R S * R S + P S * P S + 2 * (R S * P S) := by
funext S
dsimp [R, P]
ring
unfold sliceInner at hOrth ⊢
rw [hpoint, (slice n M J.slice_nonempty).E_add,
(slice n M J.slice_nonempty).E_add,
(slice n M J.slice_nonempty).E_const_mul, hOrth]
ring
have hproj : sliceInner n M J.slice_nonempty P P ≤
sliceInner n M J.slice_nonempty f f := by
rw [hdecomp]
exact le_add_of_nonneg_left (sliceInner_self_nonneg J.slice_nonempty R)
rw [show sliceNorm n M (by omega) P ^ 2 = sliceInner n M (by omega) P P by
unfold sliceNorm sliceNormSq
exact Real.sq_sqrt (sliceInner_self_nonneg (by omega) P)]
simpa only [P, Subsingleton.elim J.slice_nonempty (by omega : M ≤ n)] using
hproj.trans (sliceInner_self_le_sq_of_abs_le J.slice_nonempty f K hf)
theorem kneserEigenvalue_abs_le_one
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
Proof (Lean source)
lemma kneserEigenvalue_abs_le_one {n M : ℕ} (h2M : 2 * M ≤ n)
(k : Fin (M + 1)) : |kneserEigenvalue n M k| ≤ 1 := by
have hkM : k.1 ≤ M := by omega
have hbase : M ≤ n - M := by omega
have hden : 0 < (n - M).descFactorial k.1 :=
Nat.descFactorial_pos.mpr (hkM.trans hbase)
have hnumden : M.descFactorial k.1 ≤ (n - M).descFactorial k.1 :=
Nat.descFactorial_le k.1 hbase
simp only [kneserEigenvalue, abs_div, abs_mul, abs_pow, abs_neg, abs_one, one_pow]
rw [abs_of_nonneg (by positivity : 0 ≤ (M.descFactorial k.1 : ℝ)),
abs_of_nonneg (by positivity : 0 ≤ ((n - M).descFactorial k.1 : ℝ))]
simp only [one_mul]
exact (div_le_one (by exact_mod_cast hden)).2 (by exact_mod_cast hnumden)
theorem kneserEigenvalue_abs_le_ge_four
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
conclusion
|kneserEigenvalue n M k| ≤ ((2 * M : ℕ) : ℝ) ^ k.1 / (n : ℝ) ^ k.1
Proof (Lean source)
lemma kneserEigenvalue_abs_le_ge_four {n M : ℕ} (hMpos : 0 < M)
(h4M : 4 * M ≤ n)
(k : Fin (M + 1)) :
|kneserEigenvalue n M k| ≤ ((2 * M : ℕ) : ℝ) ^ k.1 / (n : ℝ) ^ k.1 := by
have hkM : k.1 ≤ M := by omega
have hMn : M ≤ n - M := by omega
have hkden : k.1 ≤ n - M := hkM.trans hMn
have hnpos : (0 : ℝ) < n := by exact_mod_cast (by omega : 0 < n)
have hdenpos : (0 : ℝ) < (n - M).descFactorial k.1 := by
exact_mod_cast Nat.descFactorial_pos.mpr hkden
have hnum : ((M.descFactorial k.1 : ℕ) : ℝ) ≤ (M : ℝ) ^ k.1 := by
exact_mod_cast Nat.descFactorial_le_pow M k.1
have hbaseNat : n ≤ 2 * (n - M + 1 - k.1) := by omega
have hbase : (n : ℝ) / 2 ≤ (n - M + 1 - k.1 : ℕ) := by
have hbaseR : (n : ℝ) ≤ 2 * ((n - M + 1 - k.1 : ℕ) : ℝ) := by
exact_mod_cast hbaseNat
linarith
have hden : ((n : ℝ) / 2) ^ k.1 ≤
((n - M).descFactorial k.1 : ℕ) := by
calc
((n : ℝ) / 2) ^ k.1 ≤ ((n - M + 1 - k.1 : ℕ) : ℝ) ^ k.1 :=
pow_le_pow_left₀ (by positivity) hbase _
_ ≤ ((n - M).descFactorial k.1 : ℕ) := by
exact_mod_cast Nat.pow_sub_le_descFactorial (n - M) k.1
simp only [kneserEigenvalue, abs_div, abs_mul, abs_pow, abs_neg, abs_one, one_pow]
rw [abs_of_nonneg (by positivity : 0 ≤ (M.descFactorial k.1 : ℝ)),
abs_of_nonneg (by positivity : 0 ≤ ((n - M).descFactorial k.1 : ℝ))]
simp only [one_mul]
calc
(M.descFactorial k.1 : ℝ) / ((n - M).descFactorial k.1 : ℝ) ≤
(M : ℝ) ^ k.1 / ((n : ℝ) / 2) ^ k.1 := by
gcongr
_ = ((2 * M : ℕ) : ℝ) ^ k.1 / (n : ℝ) ^ k.1 := by
push_cast
rw [mul_pow, div_pow]
field_simp
theorem johnsonSpectralSum_abs_le
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
n M d :
ℕ
hMpos :
0 < M
h4M :
4 * M ≤ n
J :
hs :
∀ k ∈ s, d ≤ k.1
f :
Omega n M → ℝ
K :
ℝ
hf :
∀ S, |f S| ≤ K
conclusion
|∑ k ∈ s, kneserEigenvalue n M k * sliceNorm n M (by omega) (J.proj k f) ^ 2|
≤ (s.card : ℝ) * (((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ d * K ^ 2
Proof (Lean source)
lemma johnsonSpectralSum_abs_le {n M d : ℕ} (hMpos : 0 < M) (h4M : 4 * M ≤ n)
(J : JohnsonProjections n M) (s : Finset (Fin (M + 1)))
(hs : ∀ k ∈ s, d ≤ k.1) (f : Omega n M → ℝ) (K : ℝ)
(hf : ∀ S, |f S| ≤ K) :
|∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) (J.proj k f) ^ 2| ≤
(s.card : ℝ) * (((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ d * K ^ 2 := by
have hnpos : (0 : ℝ) < n := by exact_mod_cast (by omega : 0 < n)
have hq0 : 0 ≤ (((2 * M : ℕ) : ℝ) / (n : ℝ)) := by positivity
have hq1 : (((2 * M : ℕ) : ℝ) / (n : ℝ)) ≤ 1 := by
apply (div_le_one hnpos).2
exact_mod_cast (by omega : 2 * M ≤ n)
calc
|∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) (J.proj k f) ^ 2| ≤
∑ k ∈ s, |kneserEigenvalue n M k *
sliceNorm n M (by omega) (J.proj k f) ^ 2| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _k ∈ s, ((((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ d * K ^ 2) := by
apply Finset.sum_le_sum
intro k hk
rw [abs_mul]
have hsqabs : |sliceNorm n M (by omega) (J.proj k f) ^ 2| =
sliceNorm n M (by omega) (J.proj k f) ^ 2 := abs_of_nonneg (sq_nonneg _)
rw [hsqabs]
have heig : |kneserEigenvalue n M k| ≤
(((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ d := by
calc
|kneserEigenvalue n M k| ≤
((2 * M : ℕ) : ℝ) ^ k.1 / (n : ℝ) ^ k.1 :=
kneserEigenvalue_abs_le_ge_four hMpos h4M k
_ = (((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ k.1 := by rw [div_pow]
_ ≤ (((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ d :=
pow_le_pow_of_le_one hq0 hq1 (hs k hk)
exact mul_le_mul heig
(johnsonProjection_energy_le_of_abs_le (by omega) J k f K hf)
(sq_nonneg _) (pow_nonneg hq0 _)
_ = (s.card : ℝ) * (((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ d * K ^ 2 := by
simp
ring
theorem higherDegreeContribution_uniform_bound
Given the stated population sizes, design objects, functions, and conditions, the higher degree contribution uniform bound result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
J :
∀ r, JohnsonProjections (A.popSize r) M
B :
ℝ
hBounded :
BoundedSchedule A B
r :
ℕ
conclusion
|higherDegreeContribution A J r|
≤ (64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2) / (A.popSize r : ℝ)
Proof (Lean source)
lemma higherDegreeContribution_uniform_bound {M : ℕ} (hM : 2 ≤ M)
(A : ScheduleArray M) (J : ∀ r, JohnsonProjections (A.popSize r) M)
(B : ℝ) (hBounded : BoundedSchedule A B) (r : ℕ) :
|higherDegreeContribution A J r| ≤
(64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2) / (A.popSize r : ℝ) := by
let n := A.popSize r
let s := Finset.univ.filter fun k : Fin (M + 1) => 2 ≤ k.1
let f : Omega n M → ℝ := fun S =>
armTable n M (A.schedule r) true S - armTable n M (A.schedule r) false S
have hnpos : (0 : ℝ) < n := by
exact_mod_cast lt_of_lt_of_le (Nat.mul_pos (by omega) (A.groups_pos r)) (A.grouped_le r)
have hGle : (A.groups r : ℝ) ≤ (n : ℝ) := by
exact_mod_cast le_trans (Nat.le_mul_of_pos_left (A.groups r) (by omega)) (A.grouped_le r)
have hcard : (s.card : ℝ) ≤ (M + 1 : ℝ) := by
exact_mod_cast (calc
s.card ≤ (Finset.univ : Finset (Fin (M + 1))).card :=
Finset.card_le_card (filter_subset _ _)
_ = M + 1 := by simp)
have hf : ∀ S, |f S| ≤ 2 * B := by
intro S
dsimp [f, n]
calc
|armTable (A.popSize r) M (A.schedule r) true S -
armTable (A.popSize r) M (A.schedule r) false S| ≤
|armTable (A.popSize r) M (A.schedule r) true S| +
|armTable (A.popSize r) M (A.schedule r) false S| := abs_sub _ _
_ ≤ B + B := add_le_add
(armTable_abs_le_of_boundedSchedule A B hBounded r true S)
(armTable_abs_le_of_boundedSchedule A B hBounded r false S)
_ = 2 * B := by ring
have h2M : 2 * M ≤ n := by
have htpos := A.treated_pos r
have htlt := A.treated_lt r
exact le_trans (by
have : 2 ≤ A.groups r := by omega
simpa [Nat.mul_comm] using Nat.mul_le_mul_left M this) (by simpa [n] using A.grouped_le r)
have hraw : |higherDegreeContribution A J r| =
(A.groups r : ℝ) * |∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2| := by
unfold higherDegreeContribution
rw [abs_mul, abs_of_nonneg (by positivity : 0 ≤ (A.groups r : ℝ))]
rw [hraw]
by_cases hlarge : 4 * M ≤ n
· have hspectral := johnsonSpectralSum_abs_le (by omega : 0 < M) hlarge
(J r) s (fun k hk => (Finset.mem_filter.mp hk).2) f (2 * B) hf
calc
(A.groups r : ℝ) * |∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2| ≤
(n : ℝ) * ((M + 1 : ℝ) * (((2 * M : ℕ) : ℝ) / (n : ℝ)) ^ 2 *
(2 * B) ^ 2) := by
apply mul_le_mul hGle
· exact hspectral.trans (mul_le_mul_of_nonneg_right
(mul_le_mul_of_nonneg_right hcard (sq_nonneg _)) (sq_nonneg _))
· positivity
· positivity
_ ≤ (64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2) / (n : ℝ) := by
push_cast
field_simp
nlinarith [sq_nonneg B]
· have hspectral : |∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2| ≤
(s.card : ℝ) * (2 * B) ^ 2 := by
calc
|∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2| ≤
∑ k ∈ s, |kneserEigenvalue n M k *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _k ∈ s, (2 * B) ^ 2 := by
apply Finset.sum_le_sum
intro k hk
rw [abs_mul]
have hsqabs : |sliceNorm n M (by omega) ((J r).proj k f) ^ 2| =
sliceNorm n M (by omega) ((J r).proj k f) ^ 2 :=
abs_of_nonneg (sq_nonneg _)
rw [hsqabs]
calc
|kneserEigenvalue n M k| *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2 ≤
1 * sliceNorm n M (by omega) ((J r).proj k f) ^ 2 :=
mul_le_mul_of_nonneg_right (kneserEigenvalue_abs_le_one h2M k)
(sq_nonneg _)
_ ≤ 1 * (2 * B) ^ 2 := mul_le_mul_of_nonneg_left
(johnsonProjection_energy_le_of_abs_le h2M (J r) k f (2 * B) hf)
zero_le_one
_ = (2 * B) ^ 2 := one_mul _
_ = (s.card : ℝ) * (2 * B) ^ 2 := by simp
have hnupper : (n : ℝ) ≤ 4 * M := by exact_mod_cast (by omega : n ≤ 4 * M)
calc
(A.groups r : ℝ) * |∑ k ∈ s, kneserEigenvalue n M k *
sliceNorm n M (by omega) ((J r).proj k f) ^ 2| ≤
(n : ℝ) * ((M + 1 : ℝ) * (2 * B) ^ 2) := by
apply mul_le_mul hGle
· exact hspectral.trans (mul_le_mul_of_nonneg_right hcard (sq_nonneg _))
· positivity
· positivity
_ ≤ (64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2) / (n : ℝ) := by
apply (le_div_iff₀ hnpos).2
have hsqn : (n : ℝ) ^ 2 ≤ (4 * (M : ℝ)) ^ 2 :=
(sq_le_sq₀ (by positivity) (by positivity)).2 hnupper
have hcore : 4 * (n : ℝ) ^ 2 ≤ 64 * (M : ℝ) ^ 2 := by nlinarith
calc
(n : ℝ) * ((M + 1 : ℝ) * (2 * B) ^ 2) * (n : ℝ) =
(4 * (n : ℝ) ^ 2) * ((M + 1 : ℝ) * B ^ 2) := by ring
_ ≤ (64 * (M : ℝ) ^ 2) * ((M + 1 : ℝ) * B ^ 2) :=
mul_le_mul_of_nonneg_right hcore (mul_nonneg (by positivity) (sq_nonneg B))
_ = 64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2 := by ring
theorem scaledContrastCrossCov_decomposition
Given the stated population sizes, design objects, functions, and conditions, the scaled contrast cross cov decomposition result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
J :
∀ r, JohnsonProjections (A.popSize r) M
r :
ℕ
hJohnson :
JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
KneserAdjacencySpectrum (A.popSize r) M
conclusion
(A.groups r : ℝ) * A.contrastCrossCov r
= -((M : ℝ) * (A.groups r : ℝ) / ((A.popSize r : ℝ) - M)) * A.degreeOne (by omega) J r
+ higherDegreeContribution A J r
Proof (Lean source)
lemma scaledContrastCrossCov_decomposition {M : ℕ} (hM : 2 ≤ M)
(A : ScheduleArray M) (J : ∀ r, JohnsonProjections (A.popSize r) M) (r : ℕ)
(hJohnson : JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : KneserAdjacencySpectrum (A.popSize r) M) :
(A.groups r : ℝ) * A.contrastCrossCov r =
-((M : ℝ) * (A.groups r : ℝ) / ((A.popSize r : ℝ) - M)) *
A.degreeOne (by omega) J r + higherDegreeContribution A J r := by
let one : Fin (M + 1) := ⟨1, by omega⟩
let high := Finset.univ.filter fun k : Fin (M + 1) => 2 ≤ k.1
let positive := Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1
have hgroups : 2 ≤ A.groups r := by
have htpos := A.treated_pos r
have htlt := A.treated_lt r
omega
have h2M : 2 * M ≤ A.popSize r := by
exact le_trans (by simpa [Nat.mul_comm] using Nat.mul_le_mul_left M hgroups)
(A.grouped_le r)
have hsplit : positive = insert one high := by
ext k
simp only [positive, high, mem_filter, Finset.mem_univ, true_and,
Finset.mem_insert]
constructor
· intro hk
by_cases hk1 : k.1 = 1
· left
exact Fin.ext hk1
· right
omega
· rintro (rfl | hk)
· simp [one]
· omega
have honeNot : one ∉ high := by simp [one, high]
have hexact := (exact_kneser_identity (A.popSize r) M hM h2M (J r)
(A.schedule r) hJohnson hKneser).2.2
change A.contrastCrossCov r = ∑ k ∈ positive,
kneserEigenvalue (A.popSize r) M k *
sliceNorm (A.popSize r) M (A.groupSize_le r)
((J r).proj k (fun S => armTable (A.popSize r) M (A.schedule r) true S -
armTable (A.popSize r) M (A.schedule r) false S)) ^ 2 at hexact
rw [hsplit, Finset.sum_insert honeNot] at hexact
rw [hexact]
unfold higherDegreeContribution ScheduleArray.degreeOne degreeOneEnergy
simp only [high, one, kneserEigenvalue, descFactorial, Nat.sub_zero, neg_mul]
simp only [pow_one, mul_one]
rw [Nat.cast_sub (by omega : M ≤ A.popSize r)]
have hden : (A.popSize r : ℝ) - M ≠ 0 := by
have : (M : ℝ) < A.popSize r := by exact_mod_cast (by omega : M < A.popSize r)
linarith
field_simp
theorem sliceExpectation_abs_le_of_abs_le
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M :
ℕ
hMn :
M ≤ n
f :
Omega n M → ℝ
K :
ℝ
hf :
∀ S, |f S| ≤ K
conclusion
|(slice n M hMn).E f| ≤ K
Proof (Lean source)
lemma sliceExpectation_abs_le_of_abs_le {n M : ℕ} (hMn : M ≤ n)
(f : Omega n M → ℝ) (K : ℝ) (hf : ∀ S, |f S| ≤ K) :
|(slice n M hMn).E f| ≤ K := by
calc
|(slice n M hMn).E f| ≤ (slice n M hMn).E (fun S => |f S|) := by
unfold FiniteDesign.E
calc
|∑ S, (slice n M hMn).p S * f S| ≤
∑ S, |(slice n M hMn).p S * f S| := Finset.abs_sum_le_sum_abs _ _
_ = ∑ S, (slice n M hMn).p S * |f S| := by
apply Finset.sum_congr rfl
intro S _
rw [abs_mul, abs_of_nonneg ((slice n M hMn).p_nonneg S)]
_ ≤ (slice n M hMn).E (fun _ => K) := by
unfold FiniteDesign.E
apply Finset.sum_le_sum
intro S _
exact mul_le_mul_of_nonneg_left (hf S) ((slice n M hMn).p_nonneg S)
_ = K := (slice n M hMn).E_const K
theorem armCrossCov_abs_le_ge_four
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
J :
∀ r, JohnsonProjections (A.popSize r) M
B :
ℝ
hBounded :
BoundedSchedule A B
r :
ℕ
h4M :
4 * M ≤ A.popSize r
hJohnson :
JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneser :
KneserAdjacencySpectrum (A.popSize r) M
z :
conclusion
|A.armCrossCov r z z|
≤ (M + 1 : ℝ) * (((2 * M : ℕ) : ℝ) / (A.popSize r : ℝ)) * (2 * B) ^ 2
Proof (Lean source)
lemma armCrossCov_abs_le_ge_four {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(J : ∀ r, JohnsonProjections (A.popSize r) M) (B : ℝ)
(hBounded : BoundedSchedule A B) (r : ℕ) (h4M : 4 * M ≤ A.popSize r)
(hJohnson : JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneser : KneserAdjacencySpectrum (A.popSize r) M) (z : Arm) :
|A.armCrossCov r z z| ≤
(M + 1 : ℝ) * (((2 * M : ℕ) : ℝ) / (A.popSize r : ℝ)) * (2 * B) ^ 2 := by
let n := A.popSize r
let f := armTable n M (A.schedule r) z
let fc := armTableCentered n M (A.groupSize_le r) (A.schedule r) z
let positive := Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1
have h2M : 2 * M ≤ n := by omega
have hf : ∀ S, |f S| ≤ B := by
intro S
exact armTable_abs_le_of_boundedSchedule A B hBounded r z S
have hfc : ∀ S, |fc S| ≤ 2 * B := by
intro S
dsimp [fc, armTableCentered]
calc
|f S - (slice n M (A.groupSize_le r)).E f| ≤
|f S| + |(slice n M (A.groupSize_le r)).E f| := abs_sub _ _
_ ≤ B + B := add_le_add (hf S)
(sliceExpectation_abs_le_of_abs_le (A.groupSize_le r) f B hf)
_ = 2 * B := by ring
have hcenter : (slice n M (A.groupSize_le r)).E fc = 0 := by
change (slice n M (A.groupSize_le r)).E
(fun S => f S - (slice n M (A.groupSize_le r)).E f) = 0
rw [(slice n M (A.groupSize_le r)).E_sub]
simp
have hcov := (exact_kneser_identity n M hM h2M (J r) (A.schedule r)
hJohnson hKneser).2.1 fc fc hcenter hcenter
have hcross : A.armCrossCov r z z =
(orderedDisjointPairDesign n M h2M).Cov
(fun P => fc P.1.1) (fun P => fc P.1.2) := by
change sliceInner n M (A.groupSize_le r) fc (kneserOp n M fc) = _
rw [(orderedDisjointPairDesign n M h2M).Cov_eq,
orderedDisjointPair_first_E_eq_slice h2M,
orderedDisjointPair_second_E_eq_slice h2M]
have hc : (slice n M (by omega)).E fc = 0 := by simpa only using hcenter
rw [hc]
simp only [zero_mul, sub_zero]
symm
simpa only using orderedDisjointPair_E_eq n M h2M fc fc
rw [hcross, hcov]
have hnorm (k : Fin (M + 1)) :
sliceInner n M (by omega) ((J r).proj k fc) ((J r).proj k fc) =
sliceNorm n M (by omega) ((J r).proj k fc) ^ 2 := by
unfold sliceNorm sliceNormSq
symm
exact Real.sq_sqrt (sliceInner_self_nonneg (by omega) _)
simp_rw [hnorm]
have hspectral := johnsonSpectralSum_abs_le (by omega : 0 < M) h4M (J r) positive
(fun k hk => (Finset.mem_filter.mp hk).2) fc (2 * B) hfc
rw [pow_one] at hspectral
exact hspectral.trans (by
have hcard : (positive.card : ℝ) ≤ (M + 1 : ℝ) := by
exact_mod_cast (calc
positive.card ≤ (Finset.univ : Finset (Fin (M + 1))).card :=
Finset.card_le_card (filter_subset _ _)
_ = M + 1 := by simp)
have hq : 0 ≤ (((2 * M : ℕ) : ℝ) / (A.popSize r : ℝ)) := by positivity
exact mul_le_mul_of_nonneg_right
(mul_le_mul_of_nonneg_right hcard hq) (sq_nonneg _))
Given the stated population sizes, design objects, functions, and conditions, the dense projection limit result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
p rho B :
ℝ
conclusion
∃ C : ℝ,
0 < C
conclusion 1
A :
J :
∀ r, JohnsonProjections (A.popSize r) M
SamplingFractionLimit A rho
BoundedSchedule A B
hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
Proof (Lean source)
theorem dense_projection_limit {M : ℕ} (hM : 2 ≤ M) (p rho B : ℝ) :
∃ C : ℝ, 0 < C ∧ ∀ (A : ScheduleArray M)
(J : ∀ r, JohnsonProjections (A.popSize r) M),
GroupCountGrowth A →
StableTreatmentFraction A p →
SamplingFractionLimit A rho →
BoundedSchedule A B →
∀ (hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M),
Tendsto (fun r =>
(A.groups r : ℝ) * A.variance r -
(A.leadingVariance r - rho * A.degreeOne (by omega) J r))
atTop (nhds 0) ∧
Tendsto (fun r =>
(A.groups r : ℝ) * A.contrastCrossCov r + rho * A.degreeOne (by omega) J r)
atTop (nhds 0) ∧
Tendsto (fun r =>
A.armCrossCov r true true / A.treatmentFraction r +
A.armCrossCov r false false / (1 - A.treatmentFraction r))
atTop (nhds 0) ∧
∀ r, |higherDegreeContribution A J r| ≤ C / (A.popSize r : ℝ) := by
let C := 64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2 + 1
refine ⟨C, by dsimp [C]; positivity, ?_⟩
intro A J hGrowth hFraction hSampling hBounded hJohnson hKneser
have hMpos : 0 < M := by omega
have hpop : Tendsto A.popSize atTop atTop := by
apply tendsto_atTop_mono (fun r => ?_) hGrowth
exact le_trans (Nat.le_mul_of_pos_left (A.groups r) hMpos) (A.grouped_le r)
have hpopR : Tendsto (fun r => (A.popSize r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hpop
have hinv : Tendsto (fun r => ((A.popSize r : ℝ))⁻¹) atTop (nhds 0) :=
hpopR.inv_tendsto_atTop
have hMratio : Tendsto (fun r => (M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 0) := by
simpa [div_eq_mul_inv] using (tendsto_const_nhds.mul hinv :
Tendsto (fun r => (M : ℝ) * ((A.popSize r : ℝ))⁻¹) atTop (nhds (M * 0)))
have hden : Tendsto (fun r => 1 - (M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 1) := by
simpa using (tendsto_const_nhds.sub hMratio :
Tendsto (fun r => 1 - (M : ℝ) / (A.popSize r : ℝ)) atTop (nhds (1 - 0)))
let coeff : ℕ → ℝ := fun r =>
(M : ℝ) * (A.groups r : ℝ) / ((A.popSize r : ℝ) - M)
have hcoeff : Tendsto coeff atTop (nhds rho) := by
have hquot := hSampling.2.2.div hden (by norm_num : (1 : ℝ) ≠ 0)
convert hquot using 1
· funext r
dsimp [coeff, ScheduleArray.grouped]
have hnpos : (A.popSize r : ℝ) ≠ 0 := by
exact_mod_cast (lt_of_lt_of_le (Nat.mul_pos hMpos (A.groups_pos r))
(A.grouped_le r)).ne'
field_simp
push_cast
rfl
· norm_num
have henergyBound (r : ℕ) :
A.degreeOne (by omega) J r ≤ (2 * B) ^ 2 := by
apply johnsonProjection_energy_le_of_abs_le
(le_trans (by
have htpos := A.treated_pos r
have htlt := A.treated_lt r
have hg : 2 ≤ A.groups r := by omega
simpa [Nat.mul_comm] using Nat.mul_le_mul_left M hg) (A.grouped_le r))
intro S
calc
|armTable (A.popSize r) M (A.schedule r) true S -
armTable (A.popSize r) M (A.schedule r) false S| ≤
|armTable (A.popSize r) M (A.schedule r) true S| +
|armTable (A.popSize r) M (A.schedule r) false S| := abs_sub _ _
_ ≤ B + B := add_le_add
(armTable_abs_le_of_boundedSchedule A B hBounded r true S)
(armTable_abs_le_of_boundedSchedule A B hBounded r false S)
_ = 2 * B := by ring
have henergyNonneg (r : ℕ) : 0 ≤ A.degreeOne (by omega) J r := by
unfold ScheduleArray.degreeOne degreeOneEnergy
exact sq_nonneg _
have hhigher : Tendsto (higherDegreeContribution A J) atTop (nhds 0) := by
rw [tendsto_zero_iff_abs_tendsto_zero]
refine squeeze_zero (fun r => abs_nonneg _)
(fun r => higherDegreeContribution_uniform_bound hM A J B hBounded r) ?_
· have hc : Tendsto (fun _ : ℕ =>
64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2) atTop
(nhds (64 * (M : ℝ) ^ 2 * (M + 1 : ℝ) * B ^ 2)) := tendsto_const_nhds
convert hc.mul hinv using 1 <;> simp [div_eq_mul_inv]
have hcoeffEnergy : Tendsto (fun r => (rho - coeff r) *
A.degreeOne (by omega) J r) atTop (nhds 0) := by
rw [tendsto_zero_iff_abs_tendsto_zero]
refine squeeze_zero (fun r => abs_nonneg _)
(fun r => show |(rho - coeff r) * A.degreeOne (by omega) J r| ≤
|rho - coeff r| * (2 * B) ^ 2 by
rw [abs_mul, abs_of_nonneg (henergyNonneg r)]
exact mul_le_mul_of_nonneg_left (henergyBound r) (abs_nonneg _)) ?_
· have hdif : Tendsto (fun r => |rho - coeff r|) atTop (nhds 0) := by
have hrho : Tendsto (fun _ : ℕ => rho) atTop (nhds rho) := tendsto_const_nhds
have hzero : Tendsto (fun r => rho - coeff r) atTop (nhds 0) := by
simpa using hrho.sub hcoeff
exact (tendsto_zero_iff_abs_tendsto_zero _).1 hzero
simpa using hdif.mul_const ((2 * B) ^ 2)
have hcontrast : Tendsto (fun r =>
(A.groups r : ℝ) * A.contrastCrossCov r +
rho * A.degreeOne (by omega) J r) atTop (nhds 0) := by
have heq : (fun r => (A.groups r : ℝ) * A.contrastCrossCov r +
rho * A.degreeOne (by omega) J r) =
fun r => (rho - coeff r) * A.degreeOne (by omega) J r +
higherDegreeContribution A J r := by
funext r
rw [scaledContrastCrossCov_decomposition hM A J r (hJohnson r) (hKneser r)]
dsimp [coeff]
ring
rw [heq]
simpa using hcoeffEnergy.add hhigher
have hcross (z : Arm) : Tendsto (fun r => A.armCrossCov r z z)
atTop (nhds 0) := by
rw [tendsto_zero_iff_abs_tendsto_zero]
have h4 : ∀ᶠ r in atTop, 4 * M ≤ A.popSize r :=
(tendsto_atTop.1 hpop) (4 * M)
refine squeeze_zero' (Filter.Eventually.of_forall fun r => abs_nonneg _)
(h4.mono fun r hr => armCrossCov_abs_le_ge_four hM A J B hBounded r hr
(hJohnson r) (hKneser r) z) ?_
· have hconst : Tendsto (fun r =>
(M + 1 : ℝ) * (((2 * M : ℕ) : ℝ) / (A.popSize r : ℝ)) *
(2 * B) ^ 2) atTop (nhds 0) := by
have hc : Tendsto (fun _ : ℕ =>
(M + 1 : ℝ) * ((2 * M : ℕ) : ℝ)) atTop
(nhds ((M + 1 : ℝ) * ((2 * M : ℕ) : ℝ))) := tendsto_const_nhds
convert (hc.mul hinv).mul_const ((2 * B) ^ 2) using 1
· funext r
push_cast
simp only [div_eq_mul_inv]
ring
· norm_num
exact hconst
have harmSum : Tendsto (fun r =>
A.armCrossCov r true true / A.treatmentFraction r +
A.armCrossCov r false false / (1 - A.treatmentFraction r))
atTop (nhds 0) := by
have hpne : p ≠ 0 := ne_of_gt hFraction.1
have hqne : 1 - p ≠ 0 := by linarith [hFraction.2.1]
have htrue := (hcross true).div hFraction.2.2 hpne
have hone : Tendsto (fun _ : ℕ => (1 : ℝ)) atTop (nhds 1) := tendsto_const_nhds
have hcontrolDen : Tendsto (fun r => 1 - A.treatmentFraction r)
atTop (nhds (1 - p)) := hone.sub hFraction.2.2
have hfalse := (hcross false).div hcontrolDen hqne
simpa using htrue.add hfalse
have hvariance : Tendsto (fun r =>
(A.groups r : ℝ) * A.variance r -
(A.leadingVariance r - rho * A.degreeOne (by omega) J r))
atTop (nhds 0) := by
have htreated : ∀ᶠ r in atTop, 2 ≤ A.treated r := by
have hlower := (tendsto_order.1 hFraction.2.2).1 (p / 2) (by linarith [hFraction.1])
have hGreal : Tendsto (fun r => (A.groups r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hGrowth
have hGlarge := hGreal.eventually_gt_atTop (4 / p)
filter_upwards [hlower, hGlarge] with r hfr hGr
by_contra ht
have htpos := A.treated_pos r
have htone : A.treated r = 1 := by omega
have hGposR : (0 : ℝ) < A.groups r := by exact_mod_cast A.groups_pos r
unfold ScheduleArray.treatmentFraction pFrac at hfr
rw [htone] at hfr
norm_num at hfr
have hpG : 4 < p * (A.groups r : ℝ) := by
simpa [mul_comm] using (div_lt_iff₀ hFraction.1).1 hGr
have : p * (A.groups r : ℝ) < 2 := by
have hh : (p / 2) * (A.groups r : ℝ) < 1 := by
apply (lt_div_iff₀ hGposR).1
simpa [one_div] using hfr
linarith
linarith
have hcontrols : ∀ᶠ r in atTop, 2 ≤ A.controls r := by
have hupper := (tendsto_order.1 hFraction.2.2).2 ((1 + p) / 2)
(by linarith [hFraction.2.1])
have hGreal : Tendsto (fun r => (A.groups r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hGrowth
have hGlarge := hGreal.eventually_gt_atTop (4 / (1 - p))
filter_upwards [hupper, hGlarge] with r hfr hGr
by_contra hc
have hcpos : 0 < A.controls r := by
unfold ScheduleArray.controls
have htlt := A.treated_lt r
omega
have hcone : A.controls r = 1 := by
omega
have hGposR : (0 : ℝ) < A.groups r := by exact_mod_cast A.groups_pos r
have hcast : (A.treated r : ℝ) = (A.groups r : ℝ) - 1 := by
have htNat : A.treated r = A.groups r - 1 := by
unfold ScheduleArray.controls at hcone
omega
rw [htNat, Nat.cast_sub (A.groups_pos r)]
norm_num
unfold ScheduleArray.treatmentFraction pFrac at hfr
rw [hcast] at hfr
have hqG : 4 < (1 - p) * (A.groups r : ℝ) := by
simpa [mul_comm] using
(div_lt_iff₀ (by linarith [hFraction.2.1] : 0 < 1 - p)).1 hGr
have : (1 - p) * (A.groups r : ℝ) < 2 := by
have hh := (div_lt_iff₀ hGposR).1 hfr
field_simp at hh
linarith
linarith
have htarget : Tendsto (fun r =>
((A.groups r : ℝ) * A.contrastCrossCov r +
rho * A.degreeOne (by omega) J r) -
(A.armCrossCov r true true / A.treatmentFraction r +
A.armCrossCov r false false / (1 - A.treatmentFraction r)))
atTop (nhds 0) := by
simpa using hcontrast.sub harmSum
apply Tendsto.congr' _ htarget
filter_upwards [htreated, hcontrols] with r ht hc
have hex := (exact_pame_variance (A.popSize r) M (A.groups r) (A.treated r)
(A.grouped_le r) hM ht (by simpa [ScheduleArray.controls] using hc)
(A.schedule r)).2.1
change A.variance r = _ at hex
rw [hex]
unfold ScheduleArray.leadingVariance ScheduleArray.treatmentFraction
ScheduleArray.armCrossCov ScheduleArray.contrastCrossCov
have hGr : (A.groups r : ℝ) ≠ 0 := by exact_mod_cast (A.groups_pos r).ne'
have htR : (A.treated r : ℝ) ≠ 0 := by exact_mod_cast (A.treated_pos r).ne'
have hcR : ((A.groups r - A.treated r : ℕ) : ℝ) ≠ 0 := by
exact_mod_cast (by
simpa [ScheduleArray.controls] using (show A.controls r ≠ 0 by omega))
rw [Nat.cast_sub (A.treated_le r)]
dsimp [pFrac]
field_simp
ring
refine ⟨hvariance, hcontrast, harmSum, ?_⟩
intro r
exact (higherDegreeContribution_uniform_bound hM A J B hBounded r).trans (by
have hnpos : (0 : ℝ) < A.popSize r := by
exact_mod_cast lt_of_lt_of_le (Nat.mul_pos hMpos (A.groups_pos r)) (A.grouped_le r)
apply (div_le_div_iff_of_pos_right hnpos).2
dsimp [C]
linarith)
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TEightUnitWitness 14 declarations Exact moments of the eight-unit witness
Exact moments of the eight-unit witness
theorem witness8_armTable_eq_sampleMean
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
Proof (Lean source)
lemma witness8_armTable_eq_sampleMean (A : Omega 8 2) :
armTable 8 2 witness8 true A = FinitePopulationMoments.sampleMean 2 a8 A := by
unfold armTable witness8 witness8Bundle
rw [Finset.sum_const, Finset.card_attach, A.2]
simp [FinitePopulationMoments.sampleMean]
theorem witness8_sampleMean_eq_signSum
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
A :
Omega 8 2
conclusion
FinitePopulationMoments.sampleMean 2 a8 A
= (∑ i ∈ A.1, if i.1 < 4 then (1 : ℝ) else -1) / 2
Proof (Lean source)
lemma witness8_sampleMean_eq_signSum (A : Omega 8 2) :
FinitePopulationMoments.sampleMean 2 a8 A =
(∑ i ∈ A.1, if i.1 < 4 then (1 : ℝ) else -1) / 2 := by
unfold FinitePopulationMoments.sampleMean a8
rw [Finset.sum_ite]
simp
theorem witness8_armTable_mean
the witness8 arm table mean result holds.
Proof (Lean source)
lemma witness8_armTable_mean :
(slice 8 2 (by omega)).E (armTable 8 2 witness8 true) = 0 := by
rw [show armTable 8 2 witness8 true =
FinitePopulationMoments.sampleMean 2 a8 by
funext A
exact witness8_armTable_eq_sampleMean A]
unfold slice
rw [FinitePopulationMoments.E_sampleMean 2 (by norm_num) (by norm_num) a8]
norm_num [FinitePopulationMoments.popMean, a8, Fin.sum_univ_succ]
theorem witness8_armVar_treated
the stated variance result holds.
Proof (Lean source)
lemma witness8_armVar_treated :
armVar 8 2 (by omega) witness8 true = (3 : ℝ) / 7 := by
rw [armVar, show armTable 8 2 witness8 true =
FinitePopulationMoments.sampleMean 2 a8 by
funext A
exact witness8_armTable_eq_sampleMean A]
unfold slice
rw [FinitePopulationMoments.Var_sampleMean 2 (by norm_num) (by norm_num)
(by norm_num) a8]
norm_num [FinitePopulationMoments.popVar, FinitePopulationMoments.popMean, a8,
Fin.sum_univ_succ]
theorem witness8_armTable_control
the witness8 arm table control result holds.
theorem witness8_armVar_control
the stated variance result holds.
Proof (Lean source)
theorem witness8_crossCov_control_right
Given the stated population sizes, design objects, functions, and conditions, the witness8 cross cov control right result holds.
Proof (Lean source)
lemma witness8_crossCov_control_right (a : Arm) :
crossCov 8 2 (by omega) witness8 a false = 0 := by
unfold crossCov armTableCentered
rw [witness8_armTable_control]
have hz : (slice 8 2 (by omega)).E (0 : Omega 8 2 → ℝ) = 0 := by
change (slice 8 2 (by omega)).E (fun _ => 0) = 0
apply FiniteDesign.E_const
rw [hz]
simp [sliceInner, kneserOp, kneserAdjacency]
theorem witness8_crossCov_control_left
Given the stated population sizes, design objects, functions, and conditions, the witness8 cross cov control left result holds.
Proof (Lean source)
lemma witness8_crossCov_control_left (b : Arm) :
crossCov 8 2 (by omega) witness8 false b = 0 := by
unfold crossCov armTableCentered
rw [witness8_armTable_control]
have hz : (slice 8 2 (by omega)).E (0 : Omega 8 2 → ℝ) = 0 := by
change (slice 8 2 (by omega)).E (fun _ => 0) = 0
apply FiniteDesign.E_const
rw [hz]
simp [sliceInner, kneserOp, kneserAdjacency]
the witness8 ordered disjoint sign sum result holds.
Formal statement
conclusion
(∑ P : OrderedDisjointPair 8 2, (∑ i ∈ P.1.1.1, if i.1 < 4 then (1 : ℤ) else -1) * (∑ i ∈ P.1.2.1, if i.1 < 4 then (1 : ℤ) else -1))
= -240
Proof (Lean source)
lemma witness8_orderedDisjoint_signSum :
(∑ P : OrderedDisjointPair 8 2,
(∑ i ∈ P.1.1.1, if i.1 < 4 then (1 : ℤ) else -1) *
(∑ i ∈ P.1.2.1, if i.1 < 4 then (1 : ℤ) else -1)) = -240 := by
decide +kernel
theorem witness8_orderedDisjoint_signSum_real Lemma witness8_orderedDisjoint_signSum_real in the paper ↗
the witness8 ordered disjoint sign sum real result holds.
Formal statement
conclusion
(∑ P : OrderedDisjointPair 8 2, (∑ i ∈ P.1.1.1, if i.1 < 4 then (1 : ℝ) else -1) * (∑ i ∈ P.1.2.1, if i.1 < 4 then (1 : ℝ) else -1))
= -240
Proof (Lean source)
lemma witness8_orderedDisjoint_signSum_real :
(∑ P : OrderedDisjointPair 8 2,
(∑ i ∈ P.1.1.1, if i.1 < 4 then (1 : ℝ) else -1) *
(∑ i ∈ P.1.2.1, if i.1 < 4 then (1 : ℝ) else -1)) = -240 := by
exact_mod_cast witness8_orderedDisjoint_signSum
theorem witness8_crossCov_treated
the witness8 cross cov treated result holds.
Proof (Lean source)
lemma witness8_crossCov_treated :
crossCov 8 2 (by omega) witness8 true true = -(1 : ℝ) / 7 := by
unfold crossCov
rw [← orderedDisjointPair_E_eq 8 2 (by omega)]
simp only [armTableCentered, witness8_armTable_mean, sub_zero]
simp_rw [witness8_armTable_eq_sampleMean]
change (∑ P : OrderedDisjointPair 8 2,
(1 / (card (OrderedDisjointPair 8 2) : ℝ)) *
(FinitePopulationMoments.sampleMean 2 a8 P.1.1 *
FinitePopulationMoments.sampleMean 2 a8 P.1.2)) = -(1 : ℝ) / 7
rw [orderedDisjointPair_card 8 2 (by omega)]
norm_num only [choose]
simp_rw [witness8_sampleMean_eq_signSum]
calc
(∑ P : OrderedDisjointPair 8 2,
(1 / (420 : ℝ)) *
((∑ i ∈ P.1.1.1, if i.1 < 4 then (1 : ℝ) else -1) / 2 *
((∑ i ∈ P.1.2.1, if i.1 < 4 then (1 : ℝ) else -1) / 2))) =
(1 / 1680 : ℝ) * ∑ P : OrderedDisjointPair 8 2,
(∑ i ∈ P.1.1.1, if i.1 < 4 then (1 : ℝ) else -1) *
(∑ i ∈ P.1.2.1, if i.1 < 4 then (1 : ℝ) else -1) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro P _
ring
_ = -(1 / 7 : ℝ) := by
rw [witness8_orderedDisjoint_signSum_real]
norm_num
Given the stated population sizes, design objects, functions, and conditions, the stated nonnegativity result holds.
Formal statement
Proof (Lean source)
lemma sum_positive_fin_three (F : Fin 3 → ℝ) :
(∑ k ∈ (Finset.univ.filter fun k : Fin 3 => 0 < k.1), F k) = F 1 + F 2 := by
rw [Finset.sum_filter]
simp [Fin.sum_univ_succ]
theorem witness8_degreeOne_from_spectrum
Given the stated population sizes, design objects, functions, and conditions, the witness8 degree one from spectrum result holds.
Formal statement
hypotheses
J :
hJohnson :
f :
Omega 8 2 → ℝ
hfmean :
(slice 8 2 (by omega)).E f = 0
hcov :
(∑ k ∈ (Finset.univ.filter fun k : Fin 3 => 0 < k.1), kneserEigenvalue 8 2 k * sliceNorm 8 2 (by omega) (J.proj k f) ^ 2)
= -(1 : ℝ) / 7
Proof (Lean source)
lemma witness8_degreeOne_from_spectrum (J : JohnsonProjections 8 2)
(hJohnson : JohnsonOrthogonalDecomposition 8 2 J)
(f : Omega 8 2 → ℝ) (hfmean : (slice 8 2 (by omega)).E f = 0)
(hvar : (slice 8 2 (by omega)).Var f = (3 : ℝ) / 7)
(hcov : (∑ k ∈ (Finset.univ.filter fun k : Fin 3 => 0 < k.1),
kneserEigenvalue 8 2 k * sliceNorm 8 2 (by omega) (J.proj k f) ^ 2) =
-(1 : ℝ) / 7) :
sliceNorm 8 2 (by omega) (J.proj (1 : Fin 3) f) ^ 2 = (3 : ℝ) / 7 := by
let p1 := J.proj (1 : Fin 3) f
let p2 := J.proj (2 : Fin 3) f
have hs : (Finset.univ.filter fun k : Fin 3 => 0 < k.1) = {1, 2} := by decide
have hf (A : Omega 8 2) : p1 A + p2 A = f A := by
have h := (hJohnson (by omega)).1 f A
rw [hs] at h
simpa [hfmean, p1, p2, add_comm] using h
have horth : sliceInner 8 2 (by omega) p1 p2 = 0 :=
(hJohnson (by omega)).2 (1 : Fin 3) (2 : Fin 3) f f (by decide)
have hsum : sliceNorm 8 2 (by omega) p1 ^ 2 +
sliceNorm 8 2 (by omega) p2 ^ 2 = (3 : ℝ) / 7 := by
have hvinner : sliceInner 8 2 (by omega) f f = (3 : ℝ) / 7 := by
rw [FiniteDesign.Var_eq] at hvar
simpa [sliceInner, pow_two, hfmean] using hvar
have hsq (g : Omega 8 2 → ℝ) :
sliceNorm 8 2 (by omega) g ^ 2 = sliceInner 8 2 (by omega) g g := by
unfold sliceNorm sliceNormSq
exact Real.sq_sqrt (sliceInner_self_nonneg (by omega) g)
rw [hsq, hsq, ← hvinner]
unfold sliceInner
rw [show (fun A => f A * f A) =
(fun A => p1 A * p1 A + p2 A * p2 A + 2 * (p1 A * p2 A)) by
funext A
rw [← hf A]
ring]
rw [(slice 8 2 (by omega)).E_add, (slice 8 2 (by omega)).E_add,
(slice 8 2 (by omega)).E_const_mul]
change _ = _ + _ + 2 * sliceInner 8 2 (by omega) p1 p2
rw [horth]
ring
have hcov' : -(1 : ℝ) / 3 * sliceNorm 8 2 (by omega) p1 ^ 2 +
(1 : ℝ) / 15 * sliceNorm 8 2 (by omega) p2 ^ 2 = -(1 : ℝ) / 7 := by
rw [sum_positive_fin_three] at hcov
norm_num [p1, p2, kneserEigenvalue, descFactorial] at hcov ⊢
exact hcov
change sliceNorm 8 2 (by omega) p1 ^ 2 = _
linarith
Given the stated population sizes, design objects, functions, and conditions, the eight unit witness moments result holds.
Formal statement
hypotheses
J :
hJohnsonOrthogonalDecomposition_of_gate :
hKneserAdjacencySpectrum_of_gate :
conclusion
conclusion 2
degreeOneEnergy 8 2 (by omega) (by omega) J witness8 = (3 : ℝ) / 7
conclusion 4
conclusion 5
= (2 : ℝ) / 7
Proof (Lean source)
theorem eight_unit_witness_moments (J : JohnsonProjections 8 2)
(hJohnsonOrthogonalDecomposition_of_gate : JohnsonOrthogonalDecomposition 8 2 J)
(hKneserAdjacencySpectrum_of_gate : KneserAdjacencySpectrum 8 2) :
armVar 8 2 (by omega) witness8 true = (3 : ℝ) / 7 ∧
degreeOneEnergy 8 2 (by omega) (by omega) J witness8 = (3 : ℝ) / 7 ∧
crossCov 8 2 (by omega) witness8 true true = -(1 : ℝ) / 7 ∧
sigmaSq witness8 (by omega : 2 * 4 ≤ 8) (by omega : 0 < 2) (by omega : 2 < 4) =
(1 : ℝ) / 7 ∧
(randomPartitionDesign 8 2 4 2 (by omega) (by omega)).E
(cr2Var witness8 (by omega) (by omega)) =
(2 : ℝ) / 7 := by
have hspectral :=
(exact_kneser_identity 8 2 (by omega) (by omega) J witness8
hJohnsonOrthogonalDecomposition_of_gate hKneserAdjacencySpectrum_of_gate).2.2
have hcontrast : crossCovContrast 8 2 (by omega) witness8 = -(1 : ℝ) / 7 := by
rw [crossCovContrast, witness8_crossCov_treated,
witness8_crossCov_control_left, witness8_crossCov_control_right]
ring
rw [hcontrast] at hspectral
let f : Omega 8 2 → ℝ := fun A =>
armTable 8 2 witness8 true A - armTable 8 2 witness8 false A
have hf : f = armTable 8 2 witness8 true := by
funext A
simp [f, witness8_armTable_control]
have hfmean : (slice 8 2 (by omega)).E f = 0 := by
rw [hf, witness8_armTable_mean]
have hfvar : (slice 8 2 (by omega)).Var f = (3 : ℝ) / 7 := by
rw [hf]
exact witness8_armVar_treated
have hdegree : degreeOneEnergy 8 2 (by omega) (by omega) J witness8 =
(3 : ℝ) / 7 := by
unfold degreeOneEnergy
exact witness8_degreeOne_from_spectrum J hJohnsonOrthogonalDecomposition_of_gate
f hfmean hfvar (by simpa [f] using hspectral.symm)
have hexact := exact_pame_variance 8 2 4 2 (by omega) (by omega) (by omega)
(by omega) witness8
dsimp only at hexact
refine ⟨witness8_armVar_treated, hdegree, witness8_crossCov_treated, ?_, ?_⟩
· rw [hexact.2.1]
unfold indepGroupVar
rw [witness8_armVar_treated, witness8_crossCov_treated,
witness8_armVar_control, witness8_crossCov_control_left, hcontrast]
norm_num [indepGroupVar, pFrac, crossCovContrast]
· rw [hexact.2.2.1, witness8_armVar_treated, witness8_crossCov_treated,
witness8_armVar_control, witness8_crossCov_control_left]
norm_num
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TExactKneserIdentity 13 declarations The paper's exact identity is stated conditionally on the two cited Johnson/Kneser logical gates.
Exact Kneser spectral identity
The paper's exact identity is stated conditionally on the two cited Johnson/Kneser logical gates. Stage 3 supplies the Lean proof of the conditional result.
theorem sliceInner_sum_left
Given the stated population sizes, design objects, functions, and conditions, the slice inner sum left result holds.
Formal statement
hypotheses
conclusion
sliceInner n M hM (fun A => ∑ i ∈ s, f i A) g = ∑ i ∈ s, sliceInner n M hM (f i) g
Proof (Lean source)
lemma sliceInner_sum_left {n M : ℕ} (hM : M ≤ n) {ι : Type*}
(s : Finset ι) (f : ι → Omega n M → ℝ) (g : Omega n M → ℝ) :
sliceInner n M hM (fun A => ∑ i ∈ s, f i A) g =
∑ i ∈ s, sliceInner n M hM (f i) g := by
unfold sliceInner
rw [← (slice n M hM).E_sum s (fun i A => f i A * g A)]
apply (slice n M hM).E_congr
intro A
rw [Finset.sum_mul]
theorem sliceInner_sum_right
Given the stated population sizes, design objects, functions, and conditions, the slice inner sum right result holds.
Formal statement
hypotheses
conclusion
sliceInner n M hM f (fun A => ∑ i ∈ s, g i A) = ∑ i ∈ s, sliceInner n M hM f (g i)
Proof (Lean source)
lemma sliceInner_sum_right {n M : ℕ} (hM : M ≤ n) {ι : Type*}
(s : Finset ι) (f : Omega n M → ℝ) (g : ι → Omega n M → ℝ) :
sliceInner n M hM f (fun A => ∑ i ∈ s, g i A) =
∑ i ∈ s, sliceInner n M hM f (g i) := by
unfold sliceInner
rw [← (slice n M hM).E_sum s (fun i A => f A * g i A)]
apply (slice n M hM).E_congr
intro A
rw [Finset.mul_sum]
theorem sliceInner_const_mul_right
Given the stated population sizes, design objects, functions, and conditions, the slice inner const mul right result holds.
Formal statement
hypotheses
n M :
ℕ
hM :
M ≤ n
c :
ℝ
f g :
Omega n M → ℝ
conclusion
sliceInner n M hM f (fun A => c * g A) = c * sliceInner n M hM f g
Proof (Lean source)
lemma sliceInner_const_mul_right {n M : ℕ} (hM : M ≤ n) (c : ℝ)
(f g : Omega n M → ℝ) :
sliceInner n M hM f (fun A => c * g A) = c * sliceInner n M hM f g := by
unfold sliceInner
rw [← (slice n M hM).E_const_mul c (fun A => f A * g A)]
apply (slice n M hM).E_congr
intro A
ring
theorem sliceInner_self_eq_zero
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M :
ℕ
hM :
M ≤ n
f :
Omega n M → ℝ
hzero :
sliceInner n M hM f f = 0
conclusion
f = 0
Proof (Lean source)
lemma sliceInner_self_eq_zero {n M : ℕ} (hM : M ≤ n) (f : Omega n M → ℝ)
(hzero : sliceInner n M hM f f = 0) : f = 0 := by
classical
funext A
have hc : 0 < (n.choose M : ℝ) := by
exact_mod_cast Nat.choose_pos hM
have hterm : 0 ≤ (1 / (n.choose M : ℝ)) * (f A * f A) :=
mul_nonneg (one_div_nonneg.mpr hc.le) (mul_self_nonneg _)
have hle : (1 / (n.choose M : ℝ)) * (f A * f A) ≤
∑ B : Omega n M, (1 / (n.choose M : ℝ)) * (f B * f B) := by
apply Finset.single_le_sum
(fun B _ => mul_nonneg (one_div_nonneg.mpr hc.le) (mul_self_nonneg _))
(Finset.mem_univ A)
have hsum : (∑ B : Omega n M,
(1 / (n.choose M : ℝ)) * (f B * f B)) = 0 := by
simpa [sliceInner, slice, completeRandomization, FiniteDesign.E] using hzero
rw [hsum] at hle
have htermzero : (1 / (n.choose M : ℝ)) * (f A * f A) = 0 :=
le_antisymm hle hterm
rcases mul_eq_zero.mp htermzero with hweight | hsq
· exact elim ((one_div_ne_zero (ne_of_gt hc)) hweight)
· simpa using mul_self_eq_zero.mp hsq
theorem sliceInner_self_nonneg
Given the stated population sizes, design objects, functions, and conditions, the stated nonnegativity result holds.
Formal statement
hypotheses
n M :
ℕ
hM :
M ≤ n
f :
Omega n M → ℝ
conclusion
0 ≤ sliceInner n M hM f f
Proof (Lean source)
lemma sliceInner_self_nonneg {n M : ℕ} (hM : M ≤ n) (f : Omega n M → ℝ) :
0 ≤ sliceInner n M hM f f := by
unfold sliceInner FiniteDesign.E
exact sum_nonneg fun A _ =>
mul_nonneg ((slice n M hM).p_nonneg A) (mul_self_nonneg _)
theorem sliceInner_sub_self
Given the stated population sizes, design objects, functions, and conditions, the slice inner sub self result holds.
Formal statement
hypotheses
n M :
ℕ
hM :
M ≤ n
f g :
Omega n M → ℝ
conclusion
sliceInner n M hM (fun A => f A - g A) (fun A => f A - g A)
= sliceInner n M hM f f + sliceInner n M hM g g - 2 * sliceInner n M hM f g
Proof (Lean source)
lemma sliceInner_sub_self {n M : ℕ} (hM : M ≤ n) (f g : Omega n M → ℝ) :
sliceInner n M hM (fun A => f A - g A) (fun A => f A - g A) =
sliceInner n M hM f f + sliceInner n M hM g g -
2 * sliceInner n M hM f g := by
unfold sliceInner
rw [show (fun A => (f A - g A) * (f A - g A)) =
(fun A => f A * f A + g A * g A - 2 * (f A * g A)) by
funext A; ring,
(slice n M hM).E_sub, (slice n M hM).E_add,
(slice n M hM).E_const_mul]
theorem kneserOp_sum
Given the stated population sizes, design objects, functions, and conditions, the kneser op sum result holds.
Formal statement
Proof (Lean source)
theorem orderedDisjointPair_first_E_eq_slice
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M :
ℕ
h2M :
2 * M ≤ n
f :
Omega n M → ℝ
conclusion
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1) = (slice n M (by omega)).E f
Proof (Lean source)
lemma orderedDisjointPair_first_E_eq_slice {n M : ℕ} (h2M : 2 * M ≤ n)
(f : Omega n M → ℝ) :
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1) =
(slice n M (by omega)).E f := by
calc
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1) =
(orderedDisjointPairDesign n M h2M).E
(fun P => f P.1.1 * (fun _ : Omega n M => (1 : ℝ)) P.1.2) := by
apply (orderedDisjointPairDesign n M h2M).E_congr
simp
_ = sliceInner n M (by omega) f (kneserOp n M (fun _ => 1)) :=
orderedDisjointPair_E_eq n M h2M f (fun _ => 1)
_ = (slice n M (by omega)).E f := by
unfold sliceInner
apply (slice n M (by omega)).E_congr
intro A
have hd : ((n - M).choose M : ℝ) ≠ 0 := by
exact_mod_cast Nat.choose_ne_zero (by omega : M ≤ n - M)
simp [kneserOp, kneserAdjacency, kneserDegree n M h2M A, hd]
For the stated inputs, ordered disjoint pair swap is defined by the formula below.
Definition (Lean source)
parameters
n M :
ℕ
def
theorem orderedDisjointPair_second_E_eq_slice
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
n M :
ℕ
h2M :
2 * M ≤ n
f :
Omega n M → ℝ
conclusion
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.2) = (slice n M (by omega)).E f
Proof (Lean source)
lemma orderedDisjointPair_second_E_eq_slice {n M : ℕ} (h2M : 2 * M ≤ n)
(f : Omega n M → ℝ) :
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.2) =
(slice n M (by omega)).E f := by
classical
calc
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.2) =
(orderedDisjointPairDesign n M h2M).E (fun P => f P.1.1) := by
unfold FiniteDesign.E orderedDisjointPairDesign uniformFiniteDesign
exact Fintype.sum_equiv orderedDisjointPairSwap _ _ (fun _ => rfl)
_ = (slice n M (by omega)).E f :=
orderedDisjointPair_first_E_eq_slice h2M f
theorem johnson_proj_const_eq_zero
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
conclusion
J.proj k (fun _ => c) = 0
Proof (Lean source)
lemma johnson_proj_const_eq_zero {n M : ℕ} (h2M : 2 * M ≤ n)
(J : JohnsonProjections n M) (hJohnson : JohnsonOrthogonalDecomposition n M J)
(k : Fin (M + 1)) (hk : 0 < k.1) (c : ℝ) :
J.proj k (fun _ => c) = 0 := by
classical
let s := Finset.univ.filter fun l : Fin (M + 1) => 0 < l.1
have hkMem : k ∈ s := Finset.mem_filter.mpr ⟨Finset.mem_univ _, hk⟩
have hsum : (fun A => ∑ l ∈ s, J.proj l (fun _ => c) A) = 0 := by
funext A
change (∑ l ∈ s, J.proj l (fun _ => c) A) = (0 : ℝ)
simpa only [s, FiniteDesign.E_const, sub_self] using (hJohnson h2M).1 (fun _ => c) A
have hinner : sliceInner n M (by omega) (J.proj k (fun _ => c))
(J.proj k (fun _ => c)) = 0 := by
have hz : sliceInner n M (by omega) (fun A => ∑ l ∈ s, J.proj l (fun _ => c) A)
(J.proj k (fun _ => c)) = 0 := by rw [hsum]; simp [sliceInner]
rw [sliceInner_sum_left] at hz
rw [Finset.sum_eq_single k] at hz
· exact hz
· intro l hl hlk
rw [(hJohnson h2M).2 l k (fun _ => c) (fun _ => c) hlk]
· exact fun hk' => (hk' hkMem).elim
exact sliceInner_self_eq_zero (by omega) _ hinner
theorem johnson_proj_centered_eq
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
Proof (Lean source)
lemma johnson_proj_centered_eq {n M : ℕ} (h2M : 2 * M ≤ n)
(J : JohnsonProjections n M) (hJohnson : JohnsonOrthogonalDecomposition n M J)
(k : Fin (M + 1)) (hk : 0 < k.1) (f : Omega n M → ℝ) :
J.proj k (fun A => f A - (slice n M (by omega)).E f) = J.proj k f := by
let m := (slice n M (by omega)).E f
calc
J.proj k (fun A => f A - (slice n M (by omega)).E f) =
J.proj k (fun A => f A + (-m) * (fun _ => (1 : ℝ)) A) := by
congr 1
funext A
dsimp [m]
ring
_ = fun A => J.proj k f A + J.proj k (fun A => (-m) * (fun _ => (1 : ℝ)) A) A :=
J.map_add k f _
_ = fun A => J.proj k f A + (-m) * J.proj k (fun _ => (1 : ℝ)) A := by
rw [J.map_smul]
_ = J.proj k f := by
rw [johnson_proj_const_eq_zero h2M J hJohnson k hk 1]
simp
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
hypotheses
n M :
ℕ
hM :
2 ≤ M
h2M :
2 * M ≤ n
J :
Y :
PotentialOutcome n M
hJohnsonOrthogonalDecomposition_of_gate :
hKneserAdjacencySpectrum_of_gate :
conclusion
conclusion 1
k :
Fin (M + 1)
f :
Omega n M → ℝ
kneserOp n M (J.proj k f) = fun A => kneserEigenvalue n M k * J.proj k f A
conclusion 2
(orderedDisjointPairDesign n M h2M).Cov (fun P => f P.1.1) (fun P => g P.1.2)
= ∑ k ∈ (Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1), kneserEigenvalue n M k * sliceInner n M (by omega) (J.proj k f) (J.proj k g)
conclusion 3
crossCovContrast n M (by omega) Y
Proof (Lean source)
theorem exact_kneser_identity (n M : ℕ) (hM : 2 ≤ M) (h2M : 2 * M ≤ n)
(J : JohnsonProjections n M) (Y : PotentialOutcome n M)
(hJohnsonOrthogonalDecomposition_of_gate : JohnsonOrthogonalDecomposition n M J)
(hKneserAdjacencySpectrum_of_gate : KneserAdjacencySpectrum n M) :
(∀ (k : Fin (M + 1)) (f : Omega n M → ℝ),
kneserOp n M (J.proj k f) =
fun A => kneserEigenvalue n M k * J.proj k f A) ∧
(∀ (f g : Omega n M → ℝ),
(slice n M (by omega)).E f = 0 →
(slice n M (by omega)).E g = 0 →
(orderedDisjointPairDesign n M h2M).Cov
(fun P => f P.1.1) (fun P => g P.1.2) =
∑ k ∈ (Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1),
kneserEigenvalue n M k *
sliceInner n M (by omega) (J.proj k f) (J.proj k g)) ∧
crossCovContrast n M (by omega) Y =
∑ k ∈ (Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1),
kneserEigenvalue n M k *
sliceNorm n M (by omega)
(J.proj k (fun A => armTable n M Y true A - armTable n M Y false A)) ^ 2 := by
classical
have hEigen : ∀ (k : Fin (M + 1)) (f : Omega n M → ℝ),
kneserOp n M (J.proj k f) =
fun A => kneserEigenvalue n M k * J.proj k f A := by
intro k f
unfold kneserOp
rw [hKneserAdjacencySpectrum_of_gate h2M k (J.proj k f)]
· funext A
simp only [kneserEigenvalue]
calc
(-1 : ℝ) ^ k.1 * ((n - M - k.1).choose (M - k.1) : ℝ) *
J.proj k f A / ((n - M).choose M : ℝ) =
(-1 : ℝ) ^ k.1 *
(((n - M - k.1).choose (M - k.1) : ℝ) /
((n - M).choose M : ℝ)) * J.proj k f A := by ring
_ = (-1 : ℝ) ^ k.1 *
((M.descFactorial k.1 : ℝ) / ((n - M).descFactorial k.1 : ℝ)) *
J.proj k f A := by
rw [kneserChooseRatio_eq_descFactorialRatio n M h2M k]
_ = ((-1 : ℝ) ^ k.1 * (M.descFactorial k.1 : ℝ) /
((n - M).descFactorial k.1 : ℝ)) * J.proj k f A := by ring
· rw [← J.range_eq k]
exact ⟨f, rfl⟩
have hCov : ∀ (f g : Omega n M → ℝ),
(slice n M (by omega)).E f = 0 →
(slice n M (by omega)).E g = 0 →
(orderedDisjointPairDesign n M h2M).Cov
(fun P => f P.1.1) (fun P => g P.1.2) =
∑ k ∈ (Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1),
kneserEigenvalue n M k *
sliceInner n M (by omega) (J.proj k f) (J.proj k g) := by
intro f g hf hg
let s := Finset.univ.filter fun k : Fin (M + 1) => 0 < k.1
have hfSum : (fun A => ∑ k ∈ s, J.proj k f A) = f := by
funext A
simpa only [s, hf, sub_zero] using
(hJohnsonOrthogonalDecomposition_of_gate h2M).1 f A
have hgSum : (fun A => ∑ k ∈ s, J.proj k g A) = g := by
funext A
simpa only [s, hg, sub_zero] using
(hJohnsonOrthogonalDecomposition_of_gate h2M).1 g A
rw [(orderedDisjointPairDesign n M h2M).Cov_eq,
orderedDisjointPair_first_E_eq_slice h2M,
orderedDisjointPair_second_E_eq_slice h2M, hf, hg]
simp only [zero_mul, sub_zero]
rw [orderedDisjointPair_E_eq n M h2M]
calc
sliceInner n M (by omega) f (kneserOp n M g) =
sliceInner n M (by omega) (fun A => ∑ k ∈ s, J.proj k f A)
(kneserOp n M (fun A => ∑ k ∈ s, J.proj k g A)) := by
rw [hfSum, hgSum]
_ = ∑ k ∈ s, kneserEigenvalue n M k *
sliceInner n M (by omega) (J.proj k f) (J.proj k g) := by
rw [kneserOp_sum, sliceInner_sum_left]
simp_rw [hEigen, sliceInner_sum_right, sliceInner_const_mul_right]
apply Finset.sum_congr rfl
intro k hk
rw [Finset.sum_eq_single k]
· intro l hl hkl
rw [(hJohnsonOrthogonalDecomposition_of_gate h2M).2 k l f g hkl.symm]
simp
· exact fun hk' => (hk' hk).elim
refine ⟨hEigen, hCov, ?_⟩
let h₁ := armTable n M Y true
let h₀ := armTable n M Y false
let h₁c := armTableCentered n M (by omega) Y true
let h₀c := armTableCentered n M (by omega) Y false
have hCentered (z : Arm) :
(slice n M (by omega)).E (armTableCentered n M (by omega) Y z) = 0 := by
unfold armTableCentered
rw [(slice n M (by omega)).E_sub]
simp
have hCross (a b : Arm) :
crossCov n M (by omega) Y a b =
(orderedDisjointPairDesign n M h2M).Cov
(fun P => armTableCentered n M (by omega) Y a P.1.1)
(fun P => armTableCentered n M (by omega) Y b P.1.2) := by
rw [(orderedDisjointPairDesign n M h2M).Cov_eq,
orderedDisjointPair_first_E_eq_slice h2M,
orderedDisjointPair_second_E_eq_slice h2M, hCentered, hCentered]
simp only [zero_mul, sub_zero]
rw [orderedDisjointPair_E_eq n M h2M]
rfl
have hProjSub (k : Fin (M + 1)) :
J.proj k (fun A => h₁ A - h₀ A) =
fun A => J.proj k h₁ A - J.proj k h₀ A := by
calc
J.proj k (fun A => h₁ A - h₀ A) =
J.proj k (fun A => h₁ A + (-1 : ℝ) * h₀ A) := by
congr 1
funext A
ring
_ = fun A => J.proj k h₁ A + J.proj k (fun A => (-1 : ℝ) * h₀ A) A :=
J.map_add k h₁ _
_ = fun A => J.proj k h₁ A + (-1 : ℝ) * J.proj k h₀ A := by
rw [J.map_smul]
_ = fun A => J.proj k h₁ A - J.proj k h₀ A := by
funext A
ring
rw [crossCovContrast, hCross, hCross, hCross,
hCov h₁c h₁c (hCentered true) (hCentered true),
hCov h₀c h₀c (hCentered false) (hCentered false),
hCov h₁c h₀c (hCentered true) (hCentered false)]
simp only [Finset.mul_sum]
rw [← Finset.sum_add_distrib, ← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro k hk
have hkpos : 0 < k.1 := (Finset.mem_filter.mp hk).2
rw [show h₁c = (fun A => h₁ A - (slice n M (by omega)).E h₁) by rfl,
show h₀c = (fun A => h₀ A - (slice n M (by omega)).E h₀) by rfl,
johnson_proj_centered_eq h2M J hJohnsonOrthogonalDecomposition_of_gate k hkpos h₁,
johnson_proj_centered_eq h2M J hJohnsonOrthogonalDecomposition_of_gate k hkpos h₀,
hProjSub]
rw [show sliceNorm n M (by omega)
(fun A => J.proj k h₁ A - J.proj k h₀ A) ^ 2 =
sliceInner n M (by omega)
(fun A => J.proj k h₁ A - J.proj k h₀ A)
(fun A => J.proj k h₁ A - J.proj k h₀ A) by
unfold sliceNorm sliceNormSq
exact Real.sq_sqrt (sliceInner_self_nonneg (by omega) _),
sliceInner_sub_self]
ring
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TExactPameVariance 1 declarations Exact PAME variance and CR2 expectation
Exact PAME variance and CR2 expectation
Given the stated population sizes, design objects, functions, and conditions, the stated variance result holds.
Formal statement
hypotheses
n M G G1 :
ℕ
hMG :
M * G ≤ n
hMtwo :
2 ≤ M
hG1two :
2 ≤ G1
hG0two :
2 ≤ G - G1
Y :
PotentialOutcome n M
conclusion
hG1pos :
by omega (: 0 < G1)
hG1lt :
by omega (: G1 < G)
hG1le :
le_of_lt hG1lt (: G1 ≤ G)
hM :
le_trans (le_mul_of_pos_right M (by omega)) hMG (: M ≤ n)
sigmaSq Y hMG hG1pos hG1lt
= indepGroupVar n M G G1 hM hG1pos hG1lt Y / (G : ℝ)
+ crossCovContrast n M hM Y
(randomPartitionDesign n M G G1 hMG hG1le).E (cr2Var Y hG1two hG0two)
- sigmaSq Y hMG hG1pos hG1lt
= -crossCovContrast n M hM Y
Proof (Lean source)
theorem exact_pame_variance (n M G G1 : ℕ) (hMG : M * G ≤ n)
(hMtwo : 2 ≤ M) (hG1two : 2 ≤ G1) (hG0two : 2 ≤ G - G1)
(Y : PotentialOutcome n M) :
let hG1pos : 0 < G1 := by omega
let hG1lt : G1 < G := by omega
let hG1le : G1 ≤ G := Nat.le_of_lt hG1lt
let hM : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
(randomPartitionDesign n M G G1 hMG hG1le).E (pameHat Y hG1pos hG1lt) = pame n M hM Y ∧
sigmaSq Y hMG hG1pos hG1lt =
indepGroupVar n M G G1 hM hG1pos hG1lt Y / (G : ℝ) +
crossCovContrast n M hM Y -
crossCov n M hM Y true true / (G1 : ℝ) -
crossCov n M hM Y false false / ((G - G1 : ℕ) : ℝ) ∧
(randomPartitionDesign n M G G1 hMG hG1le).E (cr2Var Y hG1two hG0two) =
(armVar n M hM Y true - crossCov n M hM Y true true) / (G1 : ℝ) +
(armVar n M hM Y false - crossCov n M hM Y false false) /
((G - G1 : ℕ) : ℝ) ∧
(randomPartitionDesign n M G G1 hMG hG1le).E (cr2Var Y hG1two hG0two) -
sigmaSq Y hMG hG1pos hG1lt =
-crossCovContrast n M hM Y := by
dsimp only
let hG1pos : 0 < G1 := by omega
let hG1lt : G1 < G := by omega
let hG1le : G1 ≤ G := hG1lt.le
let hG : 2 ≤ G := by omega
let hM : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
let h2M : 2 * M ≤ n := by
exact le_trans (by nlinarith : 2 * M ≤ M * G) hMG
let Dp := uniformPartitionTuple n M G hMG
let Dt := completeRandomization (V := Fin G) G1 (by simpa using hG1le)
let f1 := armTable n M Y true
let f0 := armTable n M Y false
let ft : Omega n M → ℝ := fun A => f1 A - f0 A
have hcrossT : sliceInner n M hM
(fun A => ft A - (slice n M hM).E ft)
(kneserOp n M (fun A => ft A - (slice n M hM).E ft)) =
crossCovContrast n M hM Y := by
rw [show ft = fun A => f1 A - f0 A by rfl,
disjointCov_sub_self n M h2M hM f1 f0]
rfl
have hES1 : Dp.E (fun T => S1 (fun g => f1 (T.1 g))) =
armVar n M hM Y true - crossCov n M hM Y true true := by
let hMc : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
have hs := partition_expected_sample_variance n M G G1 hMG hG hG1le f1
change Dp.E (fun T => S1 (fun g => f1 (T.1 g))) =
(slice n M hMc).Var f1 - sliceInner n M hMc
(fun A => f1 A - (slice n M hMc).E f1)
(kneserOp n M (fun A => f1 A - (slice n M hMc).E f1)) at hs
have hh : hMc = hM := Subsingleton.elim _ _
subst hMc
change Dp.E (fun T => S1 (fun g => f1 (T.1 g))) =
(slice n M hM).Var f1 - sliceInner n M hM
(fun A => f1 A - (slice n M hM).E f1)
(kneserOp n M (fun A => f1 A - (slice n M hM).E f1))
exact hs
have hES0 : Dp.E (fun T => S0 (fun g => f0 (T.1 g))) =
armVar n M hM Y false - crossCov n M hM Y false false := by
change Dp.E (fun T => S1 (fun g => f0 (T.1 g))) = _
let hMc : M ≤ n := le_trans (le_mul_of_pos_right M (by omega)) hMG
have hs := partition_expected_sample_variance n M G G1 hMG hG hG1le f0
change Dp.E (fun T => S1 (fun g => f0 (T.1 g))) =
(slice n M hMc).Var f0 - sliceInner n M hMc
(fun A => f0 A - (slice n M hMc).E f0)
(kneserOp n M (fun A => f0 A - (slice n M hMc).E f0)) at hs
have hh : hMc = hM := Subsingleton.elim _ _
subst hMc
change Dp.E (fun T => S1 (fun g => f0 (T.1 g))) =
(slice n M hM).Var f0 - sliceInner n M hM
(fun A => f0 A - (slice n M hM).E f0)
(kneserOp n M (fun A => f0 A - (slice n M hM).E f0))
exact hs
have hESt : Dp.E (fun T => Stau (fun g => f1 (T.1 g))
(fun g => f0 (T.1 g))) = (slice n M hM).Var ft - crossCovContrast n M hM Y := by
simp_rw [Stau_eq_S1_sub]
rw [show (fun T : PartitionTuple n M G => S1 (fun g => f1 (T.1 g) - f0 (T.1 g))) =
(fun T => S1 (fun g => ft (T.1 g))) by rfl]
rw [partition_expected_sample_variance n M G G1 hMG hG hG1le ft, hcrossT]
have hVarMean : Dp.Var (fun T => (∑ g, ft (T.1 g)) / (G : ℝ)) =
((slice n M hM).Var ft + ((G : ℝ) - 1) * crossCovContrast n M hM Y) /
(G : ℝ) := by
rw [partition_mean_variance n M G G1 hMG hG hG1le ft, hcrossT]
have hEmean : Dp.E (fun T => (∑ g, ft (T.1 g)) / (G : ℝ)) =
(slice n M hM).E ft := partition_E_mean n M G G1 hMG (by omega) hG1le ft
have hUnbiased : (randomPartitionDesign n M G G1 hMG hG1le).E
(pameHat Y hG1pos hG1lt) = pame n M hM Y := by
rw [randomPartitionDesign_eq_compoundCore n M G G1 hMG hG1le,
finiteDesign_E_compoundCore_tower]
simp_rw [conditional_E_pameHat Y hG1pos hG1lt]
rw [hEmean]
simp [ft, f1, f0, pame, FiniteDesign.E_sub]
have hVar : sigmaSq Y hMG hG1pos hG1lt =
(armVar n M hM Y true - crossCov n M hM Y true true) / (G1 : ℝ) +
(armVar n M hM Y false - crossCov n M hM Y false false) /
((G - G1 : ℕ) : ℝ) + crossCovContrast n M hM Y := by
rw [sigmaSq, randomPartitionDesign_eq_compoundCore n M G G1 hMG hG1le,
finiteDesign_Var_compoundCore_tower]
simp_rw [conditional_Var_pameHat Y hG1pos hG1lt]
have hcondForm : (fun T : PartitionTuple n M G =>
S1 (fun g => f1 (T.1 g)) / (G1 : ℝ) +
S0 (fun g => f0 (T.1 g)) / ((G : ℝ) - G1) -
Stau (fun g => f1 (T.1 g)) (fun g => f0 (T.1 g)) / (G : ℝ)) =
(fun T => (G1 : ℝ)⁻¹ * S1 (fun g => f1 (T.1 g)) +
((G : ℝ) - G1)⁻¹ * S0 (fun g => f0 (T.1 g)) -
(G : ℝ)⁻¹ * Stau (fun g => f1 (T.1 g)) (fun g => f0 (T.1 g))) := by
funext T
ring
rw [hcondForm]
rw [FiniteDesign.E_sub, FiniteDesign.E_add, FiniteDesign.E_const_mul,
FiniteDesign.E_const_mul, FiniteDesign.E_const_mul, hES1, hES0, hESt]
rw [show Dp.Var (fun T => Dt.E (fun S => pameHat Y hG1pos hG1lt (T, S))) =
Dp.Var (fun T => (∑ g, ft (T.1 g)) / (G : ℝ)) by
apply Dp.Var_congr
intro T
exact conditional_E_pameHat Y hG1pos hG1lt T]
rw [hVarMean, Nat.cast_sub hG1le]
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast (by omega : G ≠ 0)
ring_nf
field_simp
ring
have hCR2 : (randomPartitionDesign n M G G1 hMG hG1le).E
(cr2Var Y hG1two hG0two) =
(armVar n M hM Y true - crossCov n M hM Y true true) / (G1 : ℝ) +
(armVar n M hM Y false - crossCov n M hM Y false false) /
((G - G1 : ℕ) : ℝ) := by
rw [randomPartitionDesign_eq_compoundCore n M G G1 hMG hG1le,
finiteDesign_E_compoundCore_tower]
simp_rw [conditional_E_cr2Var Y hG1two hG0two]
have hcrForm : (fun T : PartitionTuple n M G =>
S1 (fun g => f1 (T.1 g)) / (G1 : ℝ) +
S0 (fun g => f0 (T.1 g)) / ((G - G1 : ℕ) : ℝ)) =
(fun T => (G1 : ℝ)⁻¹ * S1 (fun g => f1 (T.1 g)) +
((G - G1 : ℕ) : ℝ)⁻¹ * S0 (fun g => f0 (T.1 g))) := by
funext T
ring
rw [hcrForm]
rw [FiniteDesign.E_add, FiniteDesign.E_const_mul,
FiniteDesign.E_const_mul, hES1, hES0]
ring
refine ⟨hUnbiased, ?_, hCR2, ?_⟩
· rw [hVar]
unfold indepGroupVar pFrac
rw [Nat.cast_sub hG1le]
have hG1r : (G1 : ℝ) ≠ 0 := by exact_mod_cast (by omega : G1 ≠ 0)
have hG0r : (G : ℝ) - G1 ≠ 0 := by
have : (G1 : ℝ) < G := by exact_mod_cast hG1lt
linarith
have hGr : (G : ℝ) ≠ 0 := by exact_mod_cast (by omega : G ≠ 0)
field_simp
ring
· rw [hCR2, hVar]
ring
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TQvDiagonalImpossibility 13 declarations The lower bound is encoded in witness form: explicit high-prior-probability finite supports, conditioned-mixture separation, uniform support-wise variance limits, and a worst-case risk bound for every one-realization sta
One-realization diagonal impossibility
The lower bound is encoded in witness form: explicit high-prior-probability finite supports, conditioned-mixture separation, uniform support-wise variance limits, and a worst-case risk bound for every one-realization statistic.
def conditionedSameExpectation
Conditioned common-sign expectation, written as a finite weighted ratio.
Definition (Lean source)
parameters
def
conditionedSameExpectation A r Γ φ :
ℝ
given by
(priorSame (A.popSize r)).E (fun u => if u ∈ Γ then (A.design r).E (fun w => φ (observe (A.popSize r) M (A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w)) else 0) / (priorSame (A.popSize r)).Pr (fun u => u ∈ Γ)
def conditionedIndependentExpectation
Conditioned independent-arm expectation, written as a finite weighted ratio.
Definition (Lean source)
parameters
def
conditionedIndependentExpectation A r Γ φ :
ℝ
given by
(priorIndependent (A.popSize r)).E (fun u => if u ∈ Γ then (A.design r).E (fun w => φ (observe (A.popSize r) M (A.groups r) (A.treated r) (independentPriorSchedule (A.popSize r) M u) w)) else 0) / (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γ)
def conditionedMixtureTV
Total variation of the two conditioned finite mixtures, in its bounded-test form.
Definition (Lean source)
parameters
M :
ℕ
A :
r :
ℕ
def
conditionedMixtureTV A r Γsame Γind :
ℝ
given by
sSup {d : ℝ | ∃ φ : ObservedData (A.popSize r) M (A.groups r) (A.treated r) → ℝ, (∀ o, 0 ≤ φ o ∧ φ o ≤ 1) ∧ d = |conditionedSameExpectation A r Γsame φ - conditionedIndependentExpectation A r Γind φ|}
The support-wise diagonal certificate used by the fuzzy-hypothesis argument.
Definition (Lean source)
parameters
def
QVDiagonalCertificate A p rho B cSigma dSame dInd :
Prop
given by
clause 1
u :
∀ r
Fin (A.popSize r)
Bool, (∀ r, u r ∈ Γsame r) → DenseScheduleClass (A.withSchedule fun r => samePriorSchedule (A.popSize r) M (u r)) p rho B cSigma
clause 2
Bool, (∀ r, u r ∈ Γind r) → DenseScheduleClass (A.withSchedule fun r => independentPriorSchedule (A.popSize r) M (u r)) p rho B cSigma
clause 3
clause 4
Tendsto (fun r => sSup {x : ℝ | ∃ u ∈ Γsame r, x = |sameScaledVariance A r u - dSame|}) atTop (nhds 0)
clause 5
Tendsto (fun r => sSup {x : ℝ | ∃ u ∈ Γind r, x = |independentScaledVariance A r u - dInd|}) atTop (nhds 0)
Worst-case rowwise relative-error probability over the dense class with the fixed design skeleton A.
Definition (Lean source)
parameters
M :
ℕ
A :
p rho B cSigma :
ℝ
S :
∀ r, VarianceStatistic (A.popSize r) M (A.groups r) (A.treated r)
ε :
ℝ
r :
ℕ
def
worstCaseRatioError A p rho B cSigma S ε r :
ℝ
given by
sSup {q : ℝ | ∃ Y : ∀ r, PotentialOutcome (A.popSize r) M, DenseScheduleClass (A.withSchedule Y) p rho B cSigma ∧ q = (A.design r).Pr (fun w => |S r (observe (A.popSize r) M (A.groups r) (A.treated r) (Y r) w) / sigmaSq (Y r) (A.grouped_le r) (A.treated_pos r) (A.treated_lt r) - 1| > ε)}
theorem conditionedFiniteExpectation_nonneg Lemma conditionedFiniteExpectation_nonneg in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
0 ≤ D.E (fun w => if w ∈ Γ then f w else 0) / D.Pr (fun w => w ∈ Γ)
Proof (Lean source)
lemma conditionedFiniteExpectation_nonneg {Ω : Type*} [Fintype Ω] [DecidableEq Ω]
(D : FiniteDesign Ω) (Γ : Finset Ω) (f : Ω → ℝ)
(hf : ∀ w, 0 ≤ f w) :
0 ≤ D.E (fun w => if w ∈ Γ then f w else 0) /
D.Pr (fun w => w ∈ Γ) := by
exact div_nonneg (D.E_nonneg fun w => by split <;> simp_all) (D.Pr_nonneg _)
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
D.E (fun w => if w ∈ Γ then f w else 0) / D.Pr (fun w => w ∈ Γ) ≤ c
Proof (Lean source)
lemma conditionedFiniteExpectation_le {Ω : Type*} [Fintype Ω] [DecidableEq Ω]
(D : FiniteDesign Ω) (Γ : Finset Ω) (f : Ω → ℝ) (c : ℝ)
(hq : 0 < D.Pr (fun w => w ∈ Γ)) (hf : ∀ w, w ∈ Γ → f w ≤ c) :
D.E (fun w => if w ∈ Γ then f w else 0) /
D.Pr (fun w => w ∈ Γ) ≤ c := by
rw [div_le_iff₀ hq]
unfold FiniteDesign.E FiniteDesign.Pr FiniteDesign.ind
calc
(∑ w, D.p w * if w ∈ Γ then f w else 0) ≤
∑ w, D.p w * if w ∈ Γ then c else 0 := by
apply Finset.sum_le_sum
intro w _
by_cases hw : w ∈ Γ
· simpa [hw] using mul_le_mul_of_nonneg_left (hf w hw) (D.p_nonneg w)
· simp [hw]
_ = c * ∑ w, D.p w * if w ∈ Γ then 1 else 0 := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro w _
by_cases hw : w ∈ Γ <;> simp [hw]; ring
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
D.E (fun w => if w ∈ Γ then 1 - f w else 0) / D.Pr (fun w => w ∈ Γ)
= 1 - D.E (fun w => if w ∈ Γ then f w else 0) / D.Pr (fun w => w ∈ Γ)
Proof (Lean source)
lemma conditionedFiniteExpectation_compl {Ω : Type*} [Fintype Ω] [DecidableEq Ω]
(D : FiniteDesign Ω) (Γ : Finset Ω) (f : Ω → ℝ)
(hq : 0 < D.Pr (fun w => w ∈ Γ)) :
D.E (fun w => if w ∈ Γ then 1 - f w else 0) /
D.Pr (fun w => w ∈ Γ) =
1 - D.E (fun w => if w ∈ Γ then f w else 0) /
D.Pr (fun w => w ∈ Γ) := by
have hnum : D.E (fun w => if w ∈ Γ then 1 - f w else 0) =
D.Pr (fun w => w ∈ Γ) - D.E (fun w => if w ∈ Γ then f w else 0) := by
change (∑ w, D.p w * (if w ∈ Γ then 1 - f w else 0)) =
(∑ w, D.p w * (if w ∈ Γ then 1 else 0)) -
∑ w, D.p w * (if w ∈ Γ then f w else 0)
rw [← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro w _
by_cases hw : w ∈ Γ <;> simp [hw]; ring
rw [hnum]
field_simp [hq.ne']
theorem tendsto_of_eventually_uniform_support_error Lemma tendsto_of_eventually_uniform_support_error in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the indicated sequence converges to its stated limit.
Formal statement
Proof (Lean source)
lemma tendsto_of_eventually_uniform_support_error {Ω : ℕ → Type*}
(X : ∀ r, Ω r → ℝ) (c : ℝ) (Γ : ∀ r, Finset (Ω r)) (e : ℕ → ℝ)
(he : Tendsto e atTop (nhds 0))
(hbound : ∀ r w, w ∈ Γ r → |X r w - c| < e r)
(u : ∀ r, Ω r) (hu : ∀ᶠ r in atTop, u r ∈ Γ r) :
Tendsto (fun r => X r (u r)) atTop (nhds c) := by
rw [Metric.tendsto_atTop]
intro ε hε
have hev : ∀ᶠ r in atTop, e r < ε := (tendsto_order.1 he).2 ε hε
apply eventually_atTop.1
filter_upwards [hu, hev] with r hur hr
simpa [Real.dist_eq] using (hbound r (u r) hur).trans hr
Given the stated population sizes, design objects, functions, and conditions, the ratio good implies same decision result holds.
Formal statement
hypotheses
a b η ε g σ s :
ℝ
hb :
0 < b
hab :
b < a
hη :
η = (a - b) / 8
hε :
ε = (a - b) / (8 * a)
hg :
0 < g
hσ0 :
0 ≤ σ
hv :
|g * σ - a| < η
hr :
|s / σ - 1| ≤ ε
conclusion
(a + b) / 2 < g * s
Proof (Lean source)
lemma ratio_good_implies_same_decision {a b η ε g σ s : ℝ}
(hb : 0 < b) (hab : b < a) (hη : η = (a - b) / 8)
(hε : ε = (a - b) / (8 * a)) (hg : 0 < g) (hσ0 : 0 ≤ σ)
(hv : |g * σ - a| < η) (hr : |s / σ - 1| ≤ ε) :
(a + b) / 2 < g * s := by
have ha : 0 < a := hb.trans hab
have hgap : 0 < a - b := sub_pos.mpr hab
have hηpos : 0 < η := by rw [hη]; positivity
have hηgap : η < a - b := by rw [hη]; nlinarith
have hvlow : a - η < g * σ := by linarith [(abs_lt.mp hv).1]
have hvpos : 0 < g * σ := by nlinarith
have hσ : 0 < σ := by nlinarith
have hεpos : 0 < ε := by rw [hε]; positivity
have hεone : ε < 1 := by
rw [hε, div_lt_one (by positivity : 0 < 8 * a)]
nlinarith
have hrlow : 1 - ε ≤ s / σ := by
have := (abs_le.mp hr).1
linarith
have hbasepos : 0 < a - η := by rw [hη]; nlinarith
have hmul : (1 - ε) * (a - η) < (s / σ) * (g * σ) :=
(mul_lt_mul_of_pos_left hvlow (by linarith)).trans_le
(mul_le_mul_of_nonneg_right hrlow hvpos.le)
have hid : (s / σ) * (g * σ) = g * s := by field_simp
rw [hid] at hmul
have hmargin : (a + b) / 2 < (1 - ε) * (a - η) := by
rw [hη, hε]
field_simp [ha.ne']
nlinarith
exact hmargin.trans hmul
theorem ratio_good_implies_independent_decision Lemma ratio_good_implies_independent_decision in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the ratio good implies independent decision result holds.
Formal statement
hypotheses
a b η ε g σ s :
ℝ
hb :
0 < b
hab :
b < a
hη :
η = (a - b) / 8
hε :
ε = (a - b) / (8 * a)
hg :
0 < g
hσ0 :
0 ≤ σ
hv :
|g * σ - b| < η
hr :
|s / σ - 1| ≤ ε
conclusion
g * s < (a + b) / 2
Proof (Lean source)
lemma ratio_good_implies_independent_decision {a b η ε g σ s : ℝ}
(hb : 0 < b) (hab : b < a) (hη : η = (a - b) / 8)
(hε : ε = (a - b) / (8 * a)) (hg : 0 < g) (hσ0 : 0 ≤ σ)
(hv : |g * σ - b| < η) (hr : |s / σ - 1| ≤ ε) :
g * s < (a + b) / 2 := by
have ha : 0 < a := hb.trans hab
have hgap : 0 < a - b := sub_pos.mpr hab
have hηpos : 0 < η := by rw [hη]; positivity
have hvup : g * σ < b + η := by linarith [(abs_lt.mp hv).2]
have hεpos : 0 < ε := by rw [hε]; positivity
have hεone : ε < 1 := by
rw [hε, div_lt_one (by positivity : 0 < 8 * a)]
nlinarith
have hσ : 0 < σ := by
rcases hσ0.eq_or_lt with hσeq | hσlt
· subst σ
simp at hr
linarith
· exact hσlt
have hvpos : 0 < g * σ := mul_pos hg hσ
have hrup : s / σ ≤ 1 + ε := by
have := (abs_le.mp hr).2
linarith
have hupperpos : 0 < b + η := hvpos.trans hvup
have hmul : (s / σ) * (g * σ) < (1 + ε) * (b + η) :=
(mul_le_mul_of_nonneg_right hrup hvpos.le).trans_lt
(mul_lt_mul_of_pos_left hvup (by positivity))
have hid : (s / σ) * (g * σ) = g * s := by field_simp
rw [hid] at hmul
have hmargin : (1 + ε) * (b + η) < (a + b) / 2 := by
rw [hη, hε]
field_simp [ha.ne']
nlinarith
exact hmul.trans hmargin
theorem conditionedMixtureTV_le_complements
Given the stated population sizes, design objects, functions, and conditions, the stated bound holds.
Formal statement
hypotheses
conclusion
conditionedMixtureTV A r Γsame Γind
+ 2 * (1 - (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind))
Proof (Lean source)
lemma conditionedMixtureTV_le_complements {M : ℕ} (A : ScheduleArray M) (r : ℕ)
(Γsame : Finset (Fin (A.popSize r) → Bool))
(Γind : Finset (Fin (A.popSize r) × Bool → Bool))
(hs : 0 < (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame))
(hi : 0 < (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind)) :
conditionedMixtureTV A r Γsame Γind ≤
2 * (1 - (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame)) +
2 * (1 - (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind)) := by
let C := 2 * (1 - (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame)) +
2 * (1 - (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind))
have hbound : ∀ d ∈ {d : ℝ | ∃ φ : ObservedData (A.popSize r) M
(A.groups r) (A.treated r) → ℝ,
(∀ o, 0 ≤ φ o ∧ φ o ≤ 1) ∧
d = |conditionedSameExpectation A r Γsame φ -
conditionedIndependentExpectation A r Γind φ|}, d ≤ C := by
rintro d ⟨φ, hφ, rfl⟩
have hsu := abs_conditionedExpectation_sub_le_two_compl
(priorSame (A.popSize r)) Γsame
(fun u => (A.design r).E (fun w => φ (observe (A.popSize r) M
(A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w)))
(fun u => ⟨(A.design r).E_nonneg (fun w => (hφ (observe
(A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w)).1),
(A.design r).E_le_one (fun w => (hφ (observe
(A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w)).2)⟩) hs
have hiu := abs_conditionedExpectation_sub_le_two_compl
(priorIndependent (A.popSize r)) Γind
(fun u => (A.design r).E (fun w => φ (observe (A.popSize r) M
(A.groups r) (A.treated r) (independentPriorSchedule (A.popSize r) M u) w)))
(fun u => ⟨(A.design r).E_nonneg (fun w => (hφ (observe
(A.popSize r) M (A.groups r) (A.treated r)
(independentPriorSchedule (A.popSize r) M u) w)).1),
(A.design r).E_le_one (fun w => (hφ (observe
(A.popSize r) M (A.groups r) (A.treated r)
(independentPriorSchedule (A.popSize r) M u) w)).2)⟩) hi
have hsu' : |conditionedSameExpectation A r Γsame φ -
sameObservedExpectation A r φ| ≤
2 * (1 - (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame)) := by
simpa [conditionedSameExpectation, sameObservedExpectation] using hsu
have hiu' : |independentObservedExpectation A r φ -
conditionedIndependentExpectation A r Γind φ| ≤
2 * (1 - (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind)) := by
simpa [conditionedIndependentExpectation, independentObservedExpectation,
abs_sub_comm] using hiu
have htri := abs_sub_le (conditionedSameExpectation A r Γsame φ)
(sameObservedExpectation A r φ) (conditionedIndependentExpectation A r Γind φ)
have hiu'' : |sameObservedExpectation A r φ -
conditionedIndependentExpectation A r Γind φ| ≤
2 * (1 - (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind)) := by
rw [sameObservedExpectation_eq_independent A r φ]
exact hiu'
exact htri.trans (by dsimp [C]; exact add_le_add hsu' hiu'')
unfold conditionedMixtureTV
apply csSup_le
· refine ⟨0, ?_⟩
refine ⟨fun _ => 0, ?_, ?_⟩
· intro o
norm_num
· simp [conditionedSameExpectation, conditionedIndependentExpectation]
· intro d hd
exact hbound d hd
Given the stated population sizes, design objects, functions, and conditions, the qv diagonal impossibility result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p rho B cSigma :
ℝ
hrho :
0 < rho
hB :
1 ≤ B
hGroupCountGrowth :
hStableTreatmentFraction :
hSamplingFractionLimit :
SamplingFractionLimit A rho
J :
∀ r, JohnsonProjections (A.popSize r) M
hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
hcSigma :
0 < cSigma ∧ cSigma < 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ)
conclusion
dSame :
1 / ((M : ℝ) * p * (1 - p))
dInd :
dSame - 2 * rho / (M : ℝ)
QVDiagonalCertificate A p rho B cSigma dSame dInd
∃ ε : ℝ,
0 < ε
S :
Proof (Lean source)
theorem qv_diagonal_impossibility {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p rho B cSigma : ℝ) (hrho : 0 < rho) (hB : 1 ≤ B)
(hGroupCountGrowth : GroupCountGrowth A)
(hStableTreatmentFraction : StableTreatmentFraction A p)
(hSamplingFractionLimit : SamplingFractionLimit A rho)
(J : ∀ r, JohnsonProjections (A.popSize r) M)
(hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M)
(hcSigma : 0 < cSigma ∧
cSigma < 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ)) :
let dSame := 1 / ((M : ℝ) * p * (1 - p))
let dInd := dSame - 2 * rho / (M : ℝ)
QVDiagonalCertificate A p rho B cSigma dSame dInd ∧
∃ ε : ℝ, 0 < ε ∧ ∀ S,
(1 : ℝ) / 2 ≤ liminf
(fun r => worstCaseRatioError A p rho B cSigma S ε r) atTop := by
dsimp only
let dSame : ℝ := 1 / ((M : ℝ) * p * (1 - p))
let dInd : ℝ := dSame - 2 * rho / (M : ℝ)
obtain ⟨heq, _, _, _, _, hsame, hind, hgap, _, _⟩ :=
rademacher_mixture_separation hM A p rho hrho J hGroupCountGrowth
hStableTreatmentFraction hSamplingFractionLimit
hJohnsonOrthogonalDecomposition_of_gate hKneserAdjacencySpectrum_of_gate
obtain ⟨Γsame, es, hΓsame, hes, hes0, hsameBound, hsameSup⟩ :=
FiniteDesign.TendstoInProb.exists_uniform_support hsame
obtain ⟨Γind, ei, hΓind, hei, hei0, hindBound, hindSup⟩ :=
FiniteDesign.TendstoInProb.exists_uniform_support hind
have hB0 : 0 < B := lt_of_lt_of_le (by norm_num) hB
have hdInd : cSigma < dInd := by simpa [dInd, dSame] using hcSigma.2
have hdSame : cSigma < dSame := by
have := hgap
dsimp [dSame, dInd] at hdInd ⊢
linarith
have hclassSame : ∀ u : ∀ r, Fin (A.popSize r) → Bool,
(∀ r, u r ∈ Γsame r) →
DenseScheduleClass
(A.withSchedule fun r => samePriorSchedule (A.popSize r) M (u r))
p rho B cSigma := by
intro u hu
apply denseClass_withSchedule_of_scaled_tendsto A p rho B cSigma dSame
hGroupCountGrowth hStableTreatmentFraction hSamplingFractionLimit
hB0 hcSigma.1 hdSame
· intro r S i z
cases h : u r i.1 <;> simp [samePriorSchedule, rademacherSign, h, hB]
· exact tendsto_of_uniform_support_error (sameScaledVariance A) dSame
Γsame es hes hes0 hsameBound u hu
have hclassInd : ∀ u : ∀ r, Fin (A.popSize r) × Bool → Bool,
(∀ r, u r ∈ Γind r) →
DenseScheduleClass
(A.withSchedule fun r => independentPriorSchedule (A.popSize r) M (u r))
p rho B cSigma := by
intro u hu
apply denseClass_withSchedule_of_scaled_tendsto A p rho B cSigma dInd
hGroupCountGrowth hStableTreatmentFraction hSamplingFractionLimit
hB0 hcSigma.1 hdInd
· intro r S i z
cases h : u r (i.1, z) <;>
simp [independentPriorSchedule, rademacherSign, h, hB]
· exact tendsto_of_uniform_support_error (independentScaledVariance A) dInd
Γind ei hei hei0 hindBound u hu
have hTV : Tendsto (fun r => conditionedMixtureTV A r (Γsame r) (Γind r))
atTop (nhds 0) := by
have hspos : ∀ᶠ r in atTop,
0 < (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame r) :=
(tendsto_order.1 hΓsame).1 0 (by norm_num)
have hipos : ∀ᶠ r in atTop,
0 < (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind r) :=
(tendsto_order.1 hΓind).1 0 (by norm_num)
have hupp : Tendsto (fun r =>
2 * (1 - (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame r)) +
2 * (1 - (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind r)))
atTop (nhds 0) := by
convert (tendsto_const_nhds.mul (tendsto_const_nhds.sub hΓsame)).add
(tendsto_const_nhds.mul (tendsto_const_nhds.sub hΓind)) using 1; norm_num
apply squeeze_zero' _ _ hupp
· filter_upwards [hspos, hipos] with r hsr hir
unfold conditionedMixtureTV
have hmem : (0 : ℝ) ∈ {d : ℝ | ∃ φ : ObservedData (A.popSize r) M
(A.groups r) (A.treated r) → ℝ,
(∀ o, 0 ≤ φ o ∧ φ o ≤ 1) ∧
d = |conditionedSameExpectation A r (Γsame r) φ -
conditionedIndependentExpectation A r (Γind r) φ|} := by
refine ⟨fun _ => 0, (by norm_num), ?_⟩
simp [conditionedSameExpectation, conditionedIndependentExpectation]
by_cases hb : BddAbove {d : ℝ | ∃ φ : ObservedData (A.popSize r) M
(A.groups r) (A.treated r) → ℝ,
(∀ o, 0 ≤ φ o ∧ φ o ≤ 1) ∧
d = |conditionedSameExpectation A r (Γsame r) φ -
conditionedIndependentExpectation A r (Γind r) φ|}
· exact le_csSup hb hmem
· rw [csSup_of_not_bddAbove hb]
simp
· filter_upwards [hspos, hipos] with r hsr hir
exact conditionedMixtureTV_le_complements A r (Γsame r) (Γind r) hsr hir
refine ⟨⟨Γsame, Γind, hΓsame, hΓind, hclassSame, hclassInd,
hTV, hsameSup, hindSup⟩, ?_⟩
classical
have hdIndPos : 0 < dInd := hcSigma.1.trans hdInd
have hdOrder : dInd < dSame := by simpa [dInd] using hgap
let ε : ℝ := (dSame - dInd) / (8 * dSame)
let η : ℝ := (dSame - dInd) / 8
have hεpos : 0 < ε := by
exact div_pos (sub_pos.mpr hdOrder) (mul_pos (by norm_num) (hdIndPos.trans hdOrder))
have hηpos : 0 < η := by exact div_pos (sub_pos.mpr hdOrder) (by norm_num)
refine ⟨ε, hεpos, fun S => ?_⟩
have hspos : ∀ᶠ r in atTop,
0 < (priorSame (A.popSize r)).Pr (fun u => u ∈ Γsame r) :=
(tendsto_order.1 hΓsame).1 0 (by norm_num)
have hipos : ∀ᶠ r in atTop,
0 < (priorIndependent (A.popSize r)).Pr (fun u => u ∈ Γind r) :=
(tendsto_order.1 hΓind).1 0 (by norm_num)
have hsnonempty : ∀ᶠ r in atTop, (Γsame r).Nonempty := by
filter_upwards [hspos] with r hr
by_contra hn
rw [Finset.not_nonempty_iff_eq_empty] at hn
simp [hn, FiniteDesign.Pr, FiniteDesign.E, FiniteDesign.ind] at hr
have hinonempty : ∀ᶠ r in atTop, (Γind r).Nonempty := by
filter_upwards [hipos] with r hr
by_contra hn
rw [Finset.not_nonempty_iff_eq_empty] at hn
simp [hn, FiniteDesign.Pr, FiniteDesign.E, FiniteDesign.ind] at hr
have hessmall : ∀ᶠ r in atTop, es r < η :=
(tendsto_order.1 hes).2 η hηpos
have heismall : ∀ᶠ r in atTop, ei r < η :=
(tendsto_order.1 hei).2 η hηpos
have hrow : ∀ᶠ r in atTop,
(1 - conditionedMixtureTV A r (Γsame r) (Γind r)) / 2 ≤
worstCaseRatioError A p rho B cSigma S ε r := by
filter_upwards [hspos, hipos, hsnonempty, hinonempty, hessmall, heismall]
with r hsp hip hsne hine hse hie
let threshold : ℝ := (dSame + dInd) / 2
let sameWrong : ObservedData (A.popSize r) M (A.groups r) (A.treated r) → ℝ :=
fun o => if (A.groups r : ℝ) * S r o > threshold then 0 else 1
let indWrong : ObservedData (A.popSize r) M (A.groups r) (A.treated r) → ℝ :=
fun o => if (A.groups r : ℝ) * S r o > threshold then 1 else 0
have hsameWrong_le : conditionedSameExpectation A r (Γsame r) sameWrong ≤
worstCaseRatioError A p rho B cSigma S ε r := by
apply conditionedFiniteExpectation_le _ _ _ _ hsp
intro u hu
have hsubset : ∀ w, sameWrong (observe (A.popSize r) M (A.groups r)
(A.treated r) (samePriorSchedule (A.popSize r) M u) w) = 1 →
|S r (observe (A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w) /
sigmaSq (samePriorSchedule (A.popSize r) M u) (A.grouped_le r)
(A.treated_pos r) (A.treated_lt r) - 1| > ε := by
intro w hw
by_contra hgood
have hgood' : |S r (observe (A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w) /
sigmaSq (samePriorSchedule (A.popSize r) M u) (A.grouped_le r)
(A.treated_pos r) (A.treated_lt r) - 1| ≤ ε := le_of_not_gt hgood
have hvar := ratio_good_implies_same_decision hdIndPos hdOrder rfl rfl
(by exact_mod_cast A.groups_pos r)
((A.design r).Var_nonneg (pameHat (samePriorSchedule (A.popSize r) M u)
(A.treated_pos r) (A.treated_lt r)))
(lt_trans (hsameBound r u hu) hse) hgood'
simp [sameWrong, threshold, hvar] at hw
have hpr : (A.design r).Pr (fun w => sameWrong (observe (A.popSize r) M
(A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w) = 1) ≤
(A.design r).Pr (fun w =>
|S r (observe (A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w) /
sigmaSq (samePriorSchedule (A.popSize r) M u) (A.grouped_le r)
(A.treated_pos r) (A.treated_lt r) - 1| > ε) :=
(A.design r).Pr_mono _ _ hsubset
have hclass : ∃ Y : ∀ k, PotentialOutcome (A.popSize k) M,
DenseScheduleClass (A.withSchedule Y) p rho B cSigma ∧
Y r = samePriorSchedule (A.popSize r) M u := by
let base : ∀ k, Fin (A.popSize k) → Bool := fun k =>
if hk : (Γsame k).Nonempty then hk.choose else fun _ => false
let us : ∀ k, Fin (A.popSize k) → Bool := fun k =>
if hkr : k = r then hkr ▸ u else base k
have humem : ∀ᶠ k in atTop, us k ∈ Γsame k := by
filter_upwards [hsnonempty] with k hk
by_cases hkr : k = r
· subst k
simpa [us] using hu
· simp [us, hkr, base, hk, hk.choose_spec]
refine ⟨fun k => samePriorSchedule (A.popSize k) M (us k), ?_, by simp [us]⟩
apply denseClass_withSchedule_of_scaled_tendsto A p rho B cSigma dSame
hGroupCountGrowth hStableTreatmentFraction hSamplingFractionLimit
hB0 hcSigma.1 hdSame
· intro k T i z
cases h : us k i.1 <;> simp [samePriorSchedule, rademacherSign, h, hB]
· exact tendsto_of_eventually_uniform_support_error
(sameScaledVariance A) dSame Γsame es hes hsameBound us humem
obtain ⟨Y, hYclass, hYr⟩ := hclass
calc
(A.design r).E (fun w => sameWrong (observe (A.popSize r) M
(A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w)) =
(A.design r).Pr (fun w => sameWrong (observe (A.popSize r) M
(A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w) = 1) := by
apply (A.design r).E_congr
intro w
simp only [FiniteDesign.ind]
by_cases hw : (A.groups r : ℝ) * S r (observe (A.popSize r) M
(A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w) > threshold <;>
simp [sameWrong, hw]
_ ≤ (A.design r).Pr (fun w =>
|S r (observe (A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w) /
sigmaSq (samePriorSchedule (A.popSize r) M u) (A.grouped_le r)
(A.treated_pos r) (A.treated_lt r) - 1| > ε) := hpr
_ ≤ worstCaseRatioError A p rho B cSigma S ε r := by
unfold worstCaseRatioError
apply le_csSup
· refine ⟨1, ?_⟩
rintro q ⟨Y', _, rfl⟩
exact (A.design r).Pr_le_one _
· refine ⟨Y, hYclass, ?_⟩
rw [hYr]
have hindWrong_le : conditionedIndependentExpectation A r (Γind r) indWrong ≤
worstCaseRatioError A p rho B cSigma S ε r := by
apply conditionedFiniteExpectation_le _ _ _ _ hip
intro u hu
have hsubset : ∀ w, indWrong (observe (A.popSize r) M (A.groups r)
(A.treated r) (independentPriorSchedule (A.popSize r) M u) w) = 1 →
|S r (observe (A.popSize r) M (A.groups r) (A.treated r)
(independentPriorSchedule (A.popSize r) M u) w) /
sigmaSq (independentPriorSchedule (A.popSize r) M u) (A.grouped_le r)
(A.treated_pos r) (A.treated_lt r) - 1| > ε := by
intro w hw
by_contra hgood
have hgood' := le_of_not_gt hgood
have hvar := ratio_good_implies_independent_decision hdIndPos hdOrder rfl rfl
(by exact_mod_cast A.groups_pos r)
((A.design r).Var_nonneg (pameHat (independentPriorSchedule (A.popSize r) M u)
(A.treated_pos r) (A.treated_lt r)))
(lt_trans (hindBound r u hu) hie) hgood'
simp [indWrong, threshold, not_lt_of_ge hvar.le] at hw
have hpr := (A.design r).Pr_mono _ _ hsubset
let base : ∀ k, Fin (A.popSize k) × Bool → Bool := fun k =>
if hk : (Γind k).Nonempty then hk.choose else fun _ => false
let ui : ∀ k, Fin (A.popSize k) × Bool → Bool := fun k =>
if hkr : k = r then hkr ▸ u else base k
have humem : ∀ᶠ k in atTop, ui k ∈ Γind k := by
filter_upwards [hinonempty] with k hk
by_cases hkr : k = r
· subst k
simpa [ui] using hu
· simp [ui, hkr, base, hk, hk.choose_spec]
let Y : ∀ k, PotentialOutcome (A.popSize k) M := fun k =>
independentPriorSchedule (A.popSize k) M (ui k)
have hYclass : DenseScheduleClass (A.withSchedule Y) p rho B cSigma := by
apply denseClass_withSchedule_of_scaled_tendsto A p rho B cSigma dInd
hGroupCountGrowth hStableTreatmentFraction hSamplingFractionLimit
hB0 hcSigma.1 hdInd
-- … truncated; follow the source link for the rest …
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TRademacherMixtureSeparation 6 declarations The two product priors induce the same one-realization observation law while their group-scaled exact variances separate at positive sampling density.
Rademacher mixture separation
The two product priors induce the same one-realization observation law while their group-scaled exact variances separate at positive sampling density.
def sameObservedExpectation
Expected value of a statistic of observed data under the common-sign mixture.
Definition (Lean source)
parameters
def
sameObservedExpectation A r φ :
ℝ
given by
(priorSame (A.popSize r)).E (fun u => (A.design r).E (fun w => φ (observe (A.popSize r) M (A.groups r) (A.treated r) (samePriorSchedule (A.popSize r) M u) w)))
def independentObservedExpectation
Expected value of a statistic of observed data under the independent-arm mixture.
Definition (Lean source)
parameters
def
independentObservedExpectation A r φ :
ℝ
given by
(priorIndependent (A.popSize r)).E (fun u => (A.design r).E (fun w => φ (observe (A.popSize r) M (A.groups r) (A.treated r) (independentPriorSchedule (A.popSize r) M u) w)))
def sameScaledVariance
Group-scaled exact variance under a common-sign schedule draw.
Definition (Lean source)
parameters
def
sameScaledVariance A r u :
ℝ
given by
(A.groups r : ℝ) * sigmaSq (samePriorSchedule (A.popSize r) M u) (A.grouped_le r) (A.treated_pos r) (A.treated_lt r)
def independentScaledVariance
Group-scaled exact variance under an independent-arm schedule draw.
Definition (Lean source)
parameters
def
independentScaledVariance A r u :
ℝ
given by
(A.groups r : ℝ) * sigmaSq (independentPriorSchedule (A.popSize r) M u) (A.grouped_le r) (A.treated_pos r) (A.treated_lt r)
theorem sameObservedExpectation_eq_independent
Given the stated population sizes, design objects, functions, and conditions, the stated expectation identity holds.
Formal statement
hypotheses
conclusion
sameObservedExpectation A r φ = independentObservedExpectation A r φ
Proof (Lean source)
lemma sameObservedExpectation_eq_independent {M : ℕ} (A : ScheduleArray M) (r : ℕ)
(φ : ObservedData (A.popSize r) M (A.groups r) (A.treated r) → ℝ) :
sameObservedExpectation A r φ = independentObservedExpectation A r φ := by
unfold sameObservedExpectation independentObservedExpectation
calc
(priorSame (A.popSize r)).E (fun u => (A.design r).E (fun w =>
φ (observe (A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w))) =
(A.design r).E (fun w => (priorSame (A.popSize r)).E (fun u =>
φ (observe (A.popSize r) M (A.groups r) (A.treated r)
(samePriorSchedule (A.popSize r) M u) w))) :=
finiteDesign_E_swap _ _ _
_ = (A.design r).E (fun w => (priorIndependent (A.popSize r)).E (fun u =>
φ (observe (A.popSize r) M (A.groups r) (A.treated r)
(independentPriorSchedule (A.popSize r) M u) w))) := by
apply (A.design r).E_congr
intro w
rw [← priorIndependent_E_select (observedArmSelector w)]
apply (priorIndependent (A.popSize r)).E_congr
intro u
rw [observe_independent_eq_same_selected]
_ = (priorIndependent (A.popSize r)).E (fun u => (A.design r).E (fun w =>
φ (observe (A.popSize r) M (A.groups r) (A.treated r)
(independentPriorSchedule (A.popSize r) M u) w))) :=
(finiteDesign_E_swap _ _ _).symm
Given the stated population sizes, design objects, functions, and conditions, the rademacher mixture separation result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
A :
p rho :
ℝ
hrho :
0 < rho
J :
∀ r, JohnsonProjections (A.popSize r) M
hGroupCountGrowth :
hStableTreatmentFraction :
hSamplingFractionLimit :
SamplingFractionLimit A rho
hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r)
hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M
conclusion
conclusion 2
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) (fun r u => armVar (A.popSize r) M (A.groupSize_le r) (samePriorSchedule (A.popSize r) M u) true) (fun _ => 1 / (M : ℝ))
conclusion 3
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) (fun r u => armVar (A.popSize r) M (A.groupSize_le r) (samePriorSchedule (A.popSize r) M u) false) (fun _ => 1 / (M : ℝ))
conclusion 4
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) (fun r u => degreeOneEnergy (A.popSize r) M (by omega) (A.groupSize_le r) (J r) (samePriorSchedule (A.popSize r) M u)) (fun _ => 0)
conclusion 5
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r)) (fun r u => degreeOneEnergy (A.popSize r) M (by omega) (A.groupSize_le r) (J r) (independentPriorSchedule (A.popSize r) M u)) (fun _ => 2 / (M : ℝ))
conclusion 6
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) (sameScaledVariance A) (fun _ => 1 / ((M : ℝ) * p * (1 - p)))
conclusion 7
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r)) (independentScaledVariance A) (fun _ => 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ))
conclusion 8
0 < 2 * rho / (M : ℝ)
conclusion 9
2 / (M : ℝ) ≤ 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ)
conclusion 10
cSigma :
ℝ
0 < cSigma
cSigma < 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ)
Tendsto (fun r => (priorIndependent (A.popSize r)).Pr (fun u => cSigma ≤ independentScaledVariance A r u)) atTop (nhds 1)
Proof (Lean source)
theorem rademacher_mixture_separation {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p rho : ℝ) (hrho : 0 < rho)
(J : ∀ r, JohnsonProjections (A.popSize r) M)
(hGroupCountGrowth : GroupCountGrowth A)
(hStableTreatmentFraction : StableTreatmentFraction A p)
(hSamplingFractionLimit : SamplingFractionLimit A rho)
(hJohnsonOrthogonalDecomposition_of_gate :
∀ r, JohnsonOrthogonalDecomposition (A.popSize r) M (J r))
(hKneserAdjacencySpectrum_of_gate :
∀ r, KneserAdjacencySpectrum (A.popSize r) M) :
(∀ r φ, sameObservedExpectation A r φ = independentObservedExpectation A r φ) ∧
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(fun r u => armVar (A.popSize r) M (A.groupSize_le r)
(samePriorSchedule (A.popSize r) M u) true) (fun _ => 1 / (M : ℝ)) ∧
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(fun r u => armVar (A.popSize r) M (A.groupSize_le r)
(samePriorSchedule (A.popSize r) M u) false) (fun _ => 1 / (M : ℝ)) ∧
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(fun r u => degreeOneEnergy (A.popSize r) M (by omega) (A.groupSize_le r)
(J r) (samePriorSchedule (A.popSize r) M u)) (fun _ => 0) ∧
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r))
(fun r u => degreeOneEnergy (A.popSize r) M (by omega) (A.groupSize_le r)
(J r) (independentPriorSchedule (A.popSize r) M u)) (fun _ => 2 / (M : ℝ)) ∧
FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(sameScaledVariance A) (fun _ => 1 / ((M : ℝ) * p * (1 - p))) ∧
FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r))
(independentScaledVariance A)
(fun _ => 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ)) ∧
0 < 2 * rho / (M : ℝ) ∧
2 / (M : ℝ) ≤ 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ) ∧
∀ cSigma : ℝ, 0 < cSigma →
cSigma < 1 / ((M : ℝ) * p * (1 - p)) - 2 * rho / (M : ℝ) →
Tendsto (fun r => (priorSame (A.popSize r)).Pr
(fun u => cSigma ≤ sameScaledVariance A r u)) atTop (nhds 1) ∧
Tendsto (fun r => (priorIndependent (A.popSize r)).Pr
(fun u => cSigma ≤ independentScaledVariance A r u)) atTop (nhds 1) := by
have hp0 := hStableTreatmentFraction.1
have hp1 := hStableTreatmentFraction.2.1
have hp := hStableTreatmentFraction.2.2
have hrho1 := hSamplingFractionLimit.2.1
have hpop := A.popSize_tendsto_atTop hGroupCountGrowth
have hpopR : Tendsto (fun r => (A.popSize r : ℝ)) atTop atTop :=
tendsto_natCast_atTop_atTop.comp hpop
have hinvN : Tendsto (fun r => ((A.popSize r : ℝ))⁻¹) atTop (nhds 0) :=
hpopR.inv_tendsto_atTop
have hpInv : Tendsto (fun r => (A.treatmentFraction r)⁻¹) atTop (nhds p⁻¹) :=
hp.inv₀ hp0.ne'
have hq : Tendsto (fun r => 1 - A.treatmentFraction r) atTop (nhds (1 - p)) :=
tendsto_const_nhds.sub hp
have hqInv : Tendsto (fun r => (1 - A.treatmentFraction r)⁻¹)
atTop (nhds (1 - p)⁻¹) := hq.inv₀ (by linarith)
have hsameV1 := scheduleArray_same_armVar_tendstoInProb A hGroupCountGrowth true
have hsameV0 := scheduleArray_same_armVar_tendstoInProb A hGroupCountGrowth false
have hindV1 := scheduleArray_independent_armVar_tendstoInProb A hGroupCountGrowth true
have hindV0 := scheduleArray_independent_armVar_tendstoInProb A hGroupCountGrowth false
have hsameE := scheduleArray_same_degreeOne_tendstoInProb A J
have hindE := scheduleArray_independent_degreeOne_tendstoInProb A hGroupCountGrowth J
hJohnsonOrthogonalDecomposition_of_gate
let rsame : (r : ℕ) → (Fin (A.popSize r) → Bool) → ℝ := fun r u =>
armVar (A.popSize r) M (A.groupSize_le r)
(samePriorSchedule (A.popSize r) M u) true / A.treatmentFraction r +
armVar (A.popSize r) M (A.groupSize_le r)
(samePriorSchedule (A.popSize r) M u) false / (1 - A.treatmentFraction r)
let rind : (r : ℕ) → (Fin (A.popSize r) × Bool → Bool) → ℝ := fun r u =>
armVar (A.popSize r) M (A.groupSize_le r)
(independentPriorSchedule (A.popSize r) M u) true / A.treatmentFraction r +
armVar (A.popSize r) M (A.groupSize_le r)
(independentPriorSchedule (A.popSize r) M u) false / (1 - A.treatmentFraction r)
have hsameR : FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r)) rsame
(fun _ => 1 / ((M : ℝ) * p * (1 - p))) := by
have h1 := FiniteDesign.TendstoInProb.deterministic_mul hsameV1 hpInv
have h0 := FiniteDesign.TendstoInProb.deterministic_mul hsameV0 hqInv
have hs := h1.add h0
convert hs using 1
· funext r u
dsimp [rsame]
simp only [div_eq_mul_inv]
ring
· funext r
have hMr : (M : ℝ) ≠ 0 := by exact_mod_cast (by omega : M ≠ 0)
field_simp [hp0.ne', sub_ne_zero.mpr (ne_of_gt hp1)]
ring
have hindR : FiniteDesign.TendstoInProb (fun r => priorIndependent (A.popSize r)) rind
(fun _ => 1 / ((M : ℝ) * p * (1 - p))) := by
have h1 := FiniteDesign.TendstoInProb.deterministic_mul hindV1 hpInv
have h0 := FiniteDesign.TendstoInProb.deterministic_mul hindV0 hqInv
have hs := h1.add h0
convert hs using 1
· funext r u
dsimp [rind]
simp only [div_eq_mul_inv]
ring
· funext r
have hMr : (M : ℝ) ≠ 0 := by exact_mod_cast (by omega : M ≠ 0)
field_simp [hp0.ne', sub_ne_zero.mpr (ne_of_gt hp1)]
ring
let lam : ℕ → ℝ := fun r => kneserEigenvalue (A.popSize r) M ⟨1, by omega⟩
have hlam : Tendsto lam atTop (nhds 0) := by
have hden : Tendsto (fun r => 1 - (M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 1) := by
have hmzero : Tendsto (fun r => (M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 0) := by
simpa [div_eq_mul_inv] using
(tendsto_const_nhds.mul hinvN : Tendsto (fun r => (M : ℝ) *
((A.popSize r : ℝ))⁻¹) atTop (nhds ((M : ℝ) * 0)))
simpa using tendsto_const_nhds.sub hmzero
have hfrac : Tendsto (fun r => -(M : ℝ) / (A.popSize r : ℝ) /
(1 - (M : ℝ) / (A.popSize r : ℝ))) atTop (nhds 0) := by
have hnum : Tendsto (fun r => -(M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 0) := by
simpa [div_eq_mul_inv] using
(tendsto_const_nhds.mul hinvN : Tendsto (fun r => (-(M : ℝ)) *
((A.popSize r : ℝ))⁻¹) atTop (nhds ((-(M : ℝ)) * 0)))
change Tendsto ((fun r => -(M : ℝ) / (A.popSize r : ℝ)) /
(fun r => 1 - (M : ℝ) / (A.popSize r : ℝ))) atTop (nhds 0)
simpa only [zero_div] using hnum.div hden (by norm_num : (1 : ℝ) ≠ 0)
apply Tendsto.congr' _ hfrac
filter_upwards [] with r
have hGtwo : 2 ≤ A.groups r := by
have := A.treated_pos r
have := A.treated_lt r
omega
have h2Mr : 2 * M ≤ A.popSize r := by
calc
2 * M = M * 2 := Nat.mul_comm 2 M
_ ≤ M * A.groups r := Nat.mul_le_mul_left M hGtwo
_ ≤ A.popSize r := A.grouped_le r
dsimp [lam]
rw [kneserEigenvalue_one hM h2Mr]
have hn : (A.popSize r : ℝ) ≠ 0 := by
exact_mod_cast (lt_of_lt_of_le (Nat.mul_pos (by omega) (A.groups_pos r))
(A.grouped_le r)).ne'
field_simp
have hGlambda : Tendsto (fun r => (A.groups r : ℝ) * lam r)
atTop (nhds (-rho)) := by
let coeff : ℕ → ℝ := fun r =>
(M : ℝ) * (A.groups r : ℝ) / ((A.popSize r : ℝ) - M)
have hden : Tendsto (fun r => 1 - (M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 1) := by
have hz : Tendsto (fun r => (M : ℝ) / (A.popSize r : ℝ))
atTop (nhds 0) := by
simpa [div_eq_mul_inv] using
(tendsto_const_nhds.mul hinvN : Tendsto (fun r => (M : ℝ) *
((A.popSize r : ℝ))⁻¹) atTop (nhds ((M : ℝ) * 0)))
simpa using tendsto_const_nhds.sub hz
have hc : Tendsto coeff atTop (nhds rho) := by
have hquot := hSamplingFractionLimit.2.2.div hden (by norm_num : (1 : ℝ) ≠ 0)
convert hquot using 1
· funext r
dsimp [coeff, ScheduleArray.grouped]
have hn : (A.popSize r : ℝ) ≠ 0 := by
exact_mod_cast (lt_of_lt_of_le (Nat.mul_pos (by omega) (A.groups_pos r))
(A.grouped_le r)).ne'
field_simp
push_cast
ring
· norm_num
have hneg : Tendsto (fun r => -coeff r) atTop (nhds (-rho)) := hc.neg
apply Tendsto.congr' _ hneg
filter_upwards [] with r
have hGtwo : 2 ≤ A.groups r := by
have := A.treated_pos r
have := A.treated_lt r
omega
have h2Mr : 2 * M ≤ A.popSize r := by
calc
2 * M = M * 2 := Nat.mul_comm 2 M
_ ≤ M * A.groups r := Nat.mul_le_mul_left M hGtwo
_ ≤ A.popSize r := A.grouped_le r
dsimp [lam, coeff]
rw [kneserEigenvalue_one hM h2Mr]
ring
have hOneMinusLam : Tendsto (fun r => 1 - lam r) atTop (nhds 1) := by
simpa using tendsto_const_nhds.sub hlam
have htreatTwo : ∀ᶠ r in atTop, 2 ≤ A.treated r := by
have hlower := (tendsto_order.1 hp).1 (p / 2) (by linarith)
have hGlarge := (tendsto_natCast_atTop_atTop.comp hGroupCountGrowth).eventually_gt_atTop
(4 / p)
filter_upwards [hlower, hGlarge] with r hfr hGr
by_contra ht
have htone : A.treated r = 1 := by
have := A.treated_pos r
omega
have hGpos : (0 : ℝ) < A.groups r := by exact_mod_cast A.groups_pos r
unfold ScheduleArray.treatmentFraction pFrac at hfr
rw [htone] at hfr
norm_num at hfr
have hpG : 4 < p * (A.groups r : ℝ) := by
simpa [mul_comm] using (div_lt_iff₀ hp0).1 hGr
have : p * (A.groups r : ℝ) < 2 := by
rw [← one_div] at hfr
have hh := mul_lt_mul_of_pos_right hfr hGpos
calc
p * (A.groups r : ℝ) =
(p / 2 * (A.groups r : ℝ)) * 2 := by ring
_ < (1 / (A.groups r : ℝ) * (A.groups r : ℝ)) * 2 := by
gcongr
_ = 2 := by field_simp
linarith
have hcontrolTwo : ∀ᶠ r in atTop, 2 ≤ A.controls r := by
have hupper := (tendsto_order.1 hp).2 ((1 + p) / 2) (by linarith)
have hGlarge := (tendsto_natCast_atTop_atTop.comp hGroupCountGrowth).eventually_gt_atTop
(4 / (1 - p))
filter_upwards [hupper, hGlarge] with r hfr hGr
by_contra hc
have hcone : A.controls r = 1 := by
have : 0 < A.controls r := by
unfold ScheduleArray.controls
have := A.treated_lt r
omega
omega
have hGpos : (0 : ℝ) < A.groups r := by exact_mod_cast A.groups_pos r
have hcast : (A.treated r : ℝ) = (A.groups r : ℝ) - 1 := by
have hn : A.treated r = A.groups r - 1 := by
unfold ScheduleArray.controls at hcone
omega
rw [hn, Nat.cast_sub (A.groups_pos r)]
norm_num
unfold ScheduleArray.treatmentFraction pFrac at hfr
rw [hcast] at hfr
have hqG : 4 < (1 - p) * (A.groups r : ℝ) := by
simpa [mul_comm] using (div_lt_iff₀ (by linarith)).1 hGr
have : (1 - p) * (A.groups r : ℝ) < 2 := by
have hh := (div_lt_iff₀ hGpos).1 hfr
field_simp at hh
linarith
linarith
have hsameScaled : FiniteDesign.TendstoInProb (fun r => priorSame (A.popSize r))
(sameScaledVariance A) (fun _ => 1 / ((M : ℝ) * p * (1 - p))) := by
have hs := (FiniteDesign.TendstoInProb.deterministic_mul hsameR hOneMinusLam).add
(FiniteDesign.TendstoInProb.deterministic_mul hsameE hGlambda)
apply FiniteDesign.TendstoInProb.congr_eventually _ (by
convert hs using 1; funext r; ring)
filter_upwards [htreatTwo, hcontrolTwo] with r ht hc
intro u
let x : Fin (A.popSize r) → ℝ := fun i => rademacherSign (u i)
have htable (z : Bool) : armTable (A.popSize r) M
(samePriorSchedule (A.popSize r) M u) z = sampleMean M x := by
simpa [x] using armTable_samePrior_eq_sampleMean (n := A.popSize r) (M := M) u z
have htabled : (fun S => armTable (A.popSize r) M
(samePriorSchedule (A.popSize r) M u) true S - armTable (A.popSize r) M
(samePriorSchedule (A.popSize r) M u) false S) = sampleMean M (fun _ => 0) := by
funext S
rw [htable true, htable false]
simp [sampleMean]
simpa [sameScaledVariance, rsame, ScheduleArray.treatmentFraction,
indepGroupVar, lam] using scaledSigmaSq_eq_of_additive_armTables hM
(A.grouped_le r) ht (by simpa [ScheduleArray.controls] using hc)
(A.treated_pos r) (A.treated_lt r) (A.groupSize_le r) (by
-- … truncated; follow the source link for the rest …
CausalSmith.Experimentation.EXP_DenseGroupPartitionProjectionPhase_Research.TSparseBeyondBirthday 11 declarations Sparse consistency beyond the birthday scale
Sparse consistency beyond the birthday scale
def birthdayGroups
The benchmark group count floor (n^(3/4) / M).
Definition (Lean source)
parameters
M n :
ℕ
def
birthdayGroups M n :
ℕ
given by
⌊Real.rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ)⌋₊
The corresponding grouped-unit count.
Definition (Lean source)
The three unconditional arithmetic facts of the birthday-scale benchmark.
Definition (Lean source)
An array's grouped-unit counts agree row-by-row with the birthday benchmark.
Definition (Lean source)
parameters
def
CountAlignment A M :
Prop
given by
A.grouped r = birthdayGroupedUnits M (A.popSize r)
theorem birthdayGroupedUnits_ratio_identity
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
Proof (Lean source)
lemma birthdayGroupedUnits_ratio_identity (M n : ℕ) (hM : 0 < M) (hn : 0 < n) :
(birthdayGroupedUnits M n : ℝ) / (n : ℝ) =
((⌊Real.rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ)⌋₊ : ℝ) /
(rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ))) *
rpow (n : ℝ) (- (1 / 4 : ℝ)) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hMR : (M : ℝ) ≠ 0 := by exact_mod_cast (Nat.ne_of_gt hM)
have hpow : rpow (n : ℝ) (- (1 / 4 : ℝ)) =
rpow (n : ℝ) (3 / 4 : ℝ) / (n : ℝ) := by
calc
rpow (n : ℝ) (- (1 / 4 : ℝ)) =
rpow (n : ℝ) ((3 / 4 : ℝ) - 1) := by norm_num
_ = rpow (n : ℝ) (3 / 4 : ℝ) / rpow (n : ℝ) 1 :=
Real.rpow_sub hnR _ _
_ = rpow (n : ℝ) (3 / 4 : ℝ) / (n : ℝ) := by
congr 1
exact Real.rpow_one _
rw [birthdayGroupedUnits, birthdayGroups, Nat.cast_mul, hpow]
have hp : rpow (n : ℝ) (3 / 4 : ℝ) ≠ 0 :=
ne_of_gt (Real.rpow_pos_of_pos hnR _)
field_simp
theorem birthdayGroupedUnits_feasible
Given the stated population sizes, design objects, functions, and conditions, the birthday grouped units feasible result holds.
Formal statement
Proof (Lean source)
lemma birthdayGroupedUnits_feasible (M n : ℕ) (hM : 0 < M) (hn : 1 ≤ n) :
birthdayGroupedUnits M n ≤ n := by
have hreal : (birthdayGroupedUnits M n : ℝ) ≤ (n : ℝ) := by
calc
(birthdayGroupedUnits M n : ℝ) = (M : ℝ) *
(⌊Real.rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ)⌋₊ : ℝ) := by
rw [birthdayGroupedUnits, birthdayGroups, Nat.cast_mul]
_ ≤ (M : ℝ) * (rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ)) :=
mul_le_mul_of_nonneg_left
(Nat.floor_le (div_nonneg (Real.rpow_nonneg (by positivity) _) (Nat.cast_nonneg _)))
(Nat.cast_nonneg _)
_ = rpow (n : ℝ) (3 / 4 : ℝ) := by
field_simp [show (M : ℝ) ≠ 0 by exact_mod_cast (Nat.ne_of_gt hM)]
_ ≤ (n : ℝ) := Real.rpow_le_self_of_one_le (by exact_mod_cast hn) (by norm_num)
exact_mod_cast hreal
theorem birthdayGroupedUnits_lower_bound
Given the stated population sizes, design objects, functions, and conditions, the birthday grouped units lower bound result holds.
Formal statement
hypotheses
M n :
ℕ
hM :
0 < M
conclusion
rpow (n : ℝ) (3 / 4 : ℝ) - (M : ℝ) ≤ birthdayGroupedUnits M n
Proof (Lean source)
lemma birthdayGroupedUnits_lower_bound (M n : ℕ) (hM : 0 < M) :
rpow (n : ℝ) (3 / 4 : ℝ) - (M : ℝ) ≤ birthdayGroupedUnits M n := by
have hMR : 0 < (M : ℝ) := by exact_mod_cast hM
have hf := Nat.lt_floor_add_one
(rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ))
have hmul := mul_lt_mul_of_pos_left hf hMR
have hMne : (M : ℝ) ≠ 0 := ne_of_gt hMR
simp only [mul_div_cancel₀ _ hMne, mul_add] at hmul
rw [birthdayGroupedUnits, birthdayGroups, Nat.cast_mul]
linarith
theorem birthdayGroupedUnits_power_identity Lemma birthdayGroupedUnits_power_identity in the paper ↗
Given the stated population sizes, design objects, functions, and conditions, the stated equality holds.
Formal statement
Proof (Lean source)
lemma birthdayGroupedUnits_power_identity (n : ℕ) (hn : 0 < n) :
(rpow (n : ℝ) (3 / 4 : ℝ) / 2) ^ 2 / (n : ℝ) =
(1 / 4 : ℝ) * rpow (n : ℝ) (1 / 2 : ℝ) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hsq : (rpow (n : ℝ) (3 / 4 : ℝ)) ^ 2 =
rpow (n : ℝ) (3 / 2 : ℝ) := by
calc
_ = rpow (rpow (n : ℝ) (3 / 4 : ℝ)) (2 : ℝ) :=
(Real.rpow_natCast _ 2).symm
_ = rpow (n : ℝ) ((3 / 4 : ℝ) * 2) :=
(Real.rpow_mul (le_of_lt hnR) _ _).symm
_ = _ := by norm_num
rw [div_pow, hsq]
have hdiv : rpow (n : ℝ) (3 / 2 : ℝ) / (n : ℝ) =
rpow (n : ℝ) (1 / 2 : ℝ) := by
calc
_ = rpow (n : ℝ) (3 / 2 : ℝ) / rpow (n : ℝ) 1 := by
congr 1
exact (Real.rpow_one _).symm
_ = rpow (n : ℝ) ((3 / 2 : ℝ) - 1) := (Real.rpow_sub hnR _ _).symm
_ = _ := by norm_num
rw [show (2 : ℝ) ^ 2 = 4 by norm_num]
calc
rpow (n : ℝ) (3 / 2 : ℝ) / 4 / (n : ℝ) =
(1 / 4 : ℝ) * (rpow (n : ℝ) (3 / 2 : ℝ) / (n : ℝ)) := by ring
_ = _ := by rw [hdiv]
theorem birthdayBenchmark_proof
Given the stated population sizes, design objects, functions, and conditions, the birthday benchmark proof result holds.
Formal statement
Proof (Lean source)
lemma birthdayBenchmark_proof (M : ℕ) (hM : 0 < M) : BirthdayBenchmark M := by
have hcast : Tendsto (fun n : ℕ => (n : ℝ)) atTop atTop := tendsto_natCast_atTop_atTop
have hpow : Tendsto (fun n : ℕ => rpow (n : ℝ) (3 / 4 : ℝ)) atTop atTop :=
(tendsto_rpow_atTop (by norm_num : (0 : ℝ) < 3 / 4)).comp hcast
have hx : Tendsto (fun n : ℕ => rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ))
atTop atTop := hpow.atTop_div_const (by exact_mod_cast hM)
have hfloorRatio : Tendsto (fun n : ℕ =>
(⌊Real.rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ)⌋₊ : ℝ) /
(rpow (n : ℝ) (3 / 4 : ℝ) / (M : ℝ))) atTop (nhds 1) :=
tendsto_nat_floor_div_atTop.comp hx
have hneg : Tendsto (fun n : ℕ => rpow (n : ℝ) (- (1 / 4 : ℝ)))
atTop (nhds 0) :=
(tendsto_rpow_neg_atTop (by norm_num : (0 : ℝ) < 1 / 4)).comp hcast
have hratio : Tendsto (fun n => (birthdayGroupedUnits M n : ℝ) / (n : ℝ))
atTop (nhds 0) := by
convert hfloorRatio.mul hneg using 1
· funext n
by_cases hn : n = 0
· subst n
simp [birthdayGroupedUnits, birthdayGroups]
· exact birthdayGroupedUnits_ratio_identity M n hM (Nat.pos_of_ne_zero hn)
· norm_num
refine ⟨?_, hratio, ?_⟩
· filter_upwards [eventually_atTop.2 ⟨1, fun _ hn => hn⟩] with n hn
exact birthdayGroupedUnits_feasible M n hM hn
· have hsqrt : Tendsto (fun n : ℕ =>
(1 / 4 : ℝ) * rpow (n : ℝ) (1 / 2 : ℝ)) atTop atTop :=
((tendsto_rpow_atTop (by norm_num : (0 : ℝ) < 1 / 2)).comp hcast).const_mul_atTop
(by norm_num)
apply tendsto_atTop_mono' atTop _ hsqrt
filter_upwards [hpow.eventually (eventually_ge_atTop (2 * (M : ℝ))),
eventually_atTop.2 ⟨1, fun _ hn => hn⟩] with n hlarge hn
have hnpos : 0 < n := by omega
have hlower := birthdayGroupedUnits_lower_bound M n hM
have hhalf : rpow (n : ℝ) (3 / 4 : ℝ) / 2 ≤
(birthdayGroupedUnits M n : ℝ) := by nlinarith
have hpow_nonneg : 0 ≤ rpow (n : ℝ) (3 / 4 : ℝ) / 2 :=
div_nonneg (Real.rpow_nonneg (Nat.cast_nonneg _) _) (by norm_num)
have hgroup_nonneg : 0 ≤ (birthdayGroupedUnits M n : ℝ) := Nat.cast_nonneg _
calc
(1 / 4 : ℝ) * rpow (n : ℝ) (1 / 2 : ℝ) =
(rpow (n : ℝ) (3 / 4 : ℝ) / 2) ^ 2 / (n : ℝ) :=
(birthdayGroupedUnits_power_identity n hnpos).symm
_ ≤ ((birthdayGroupedUnits M n : ℝ) ^ 2) / (n : ℝ) := by gcongr
theorem sparse_dense_ratio_consistency
Given the stated population sizes, design objects, functions, and conditions, the sparse dense ratio consistency result holds.
Formal statement
hypotheses
conclusion
FiniteDesign.TendstoInProb A.design (fun r w => A.cr2 r w / A.variance r) (fun _ => 1)
Proof (Lean source)
lemma sparse_dense_ratio_consistency {M : ℕ} (hM : 2 ≤ M) (A : ScheduleArray M)
(p B cSigma : ℝ) (hClass : DenseScheduleClass A p 0 B cSigma) :
FiniteDesign.TendstoInProb A.design
(fun r w => A.cr2 r w / A.variance r) (fun _ => 1) := by
let J : ∀ r, JohnsonProjections (A.popSize r) M :=
fun r => canonicalJohnsonProjections (A.popSize r) M (A.groupSize_le r)
have hfront := cr2_phase_frontier hM A p 0 B cSigma J hClass
(fun r => canonicalJohnsonOrthogonalDecomposition (A.popSize r) M (A.groupSize_le r))
(fun r => canonicalKneserAdjacencySpectrum (A.popSize r) M)
apply hfront.2.2.2.mpr
simpa using (tendsto_const_nhds : Tendsto (fun _ : ℕ => (0 : ℝ)) atTop (nhds 0))
Given the stated population sizes, design objects, functions, and conditions, the sparse beyond birthday result holds.
Formal statement
hypotheses
M :
ℕ
hM :
2 ≤ M
conclusion
conclusion 1
FiniteDesign.TendstoInProb A.design (fun r w => A.cr2 r w / A.variance r) (fun _ => 1)
conclusion 2
conclusion 3
FiniteDesign.TendstoInProb A.design (fun r w => A.cr2 r w / A.variance r) (fun _ => 1)
Proof (Lean source)
theorem sparse_beyond_birthday {M : ℕ} (hM : 2 ≤ M) :
(∀ (A : ScheduleArray M) (p B cSigma : ℝ),
DenseScheduleClass A p 0 B cSigma →
FiniteDesign.TendstoInProb A.design
(fun r w => A.cr2 r w / A.variance r) (fun _ => 1)) ∧
BirthdayBenchmark M ∧
(∀ (A : ScheduleArray M) (p B cSigma : ℝ),
CountAlignment A M →
GroupCountGrowth A →
StableTreatmentFraction A p →
BoundedSchedule A B →
ScaledVarianceNondegenerate A cSigma →
FiniteDesign.TendstoInProb A.design
(fun r w => A.cr2 r w / A.variance r) (fun _ => 1)) := by
have hMpos : 0 < M := by omega
have hbench := birthdayBenchmark_proof M hMpos
refine ⟨?_, hbench, ?_⟩
· intro A p B cSigma hClass
exact sparse_dense_ratio_consistency hM A p B cSigma hClass
· intro A p B cSigma hAlign hGrowth hFraction hBounded hNondegenerate
have hpop : Tendsto A.popSize atTop atTop := by
apply tendsto_atTop_mono (fun r => ?_) hGrowth
exact le_trans (Nat.le_mul_of_pos_left (A.groups r) hMpos) (A.grouped_le r)
have hsampLimit : Tendsto (fun r => (A.grouped r : ℝ) / (A.popSize r : ℝ))
atTop (nhds 0) := by
convert hbench.2.1.comp hpop using 1
funext r
simp only [Function.comp_apply]
rw [hAlign r]
have hClass : DenseScheduleClass A p 0 B cSigma :=
⟨hGrowth, hFraction, ⟨le_rfl, zero_le_one, hsampLimit⟩, hBounded, hNondegenerate⟩
exact sparse_dense_ratio_consistency hM A p B cSigma hClass