Stat.UStatistic.Order­M

Fixed- and mixed-order U-statistics: ordered injective tuples, partial-matching product and covariance expansions, normalization bounds, Hájek projections, variance identities, degeneracy conditions, remainder negligibility, CLTs, and the order-2 bridge.

Basic 15 core · 13 supporting This module defines fixed-order U-statistics over ordered injective tuples. ★ uStatisticOrder★ hoeffding_decomp_order

Fixed-order U-statistics

This module defines fixed-order U-statistics over ordered injective tuples. The core objects are injectiveTuples, injectiveTupleCount, uStatisticOrder, uMeanOrder, the coordinatewise first projections uProjOrderAt and uProjOrder, and the residual kernel uDegenOrder.

The main structural results are the falling-factorial tuple counts injectiveTuples_card_eq_descFactorial and injectiveTupleCount_eq_descFactorial, the pointwise Hoeffding decomposition hoeffding_decomp_order, centering and integrability lemmas for the first-order influence function, and uDegenOrder_integral_tail_eq_zero, which proves coordinatewise first-order degeneracy of the residual under the stated finite-product/Fubini hypotheses. The order-2 compatibility layer is supplied by pairKernel, sum_injectiveTuples_two_eq_offDiag, and uStatisticOrder_two_eq_uStatistic.

def injectiveTuples reviewed
Causalean.Stat

For a tuple length and a number of available sample positions, the collection of ordered injective tuples consists of all ordered selections of the specified length whose entries are distinct positions among the first specified number of observations.

Definition (Lean source)
m n :
injectiveTuples m n :
Finset (Fin m → Fin n)
by classical exact Finset.univ.filter Injective
Causalean.Stat.injectiveTuples · Causalean/Stat/UStatistic/OrderM/Basic.lean:49
def injectiveTupleCount reviewed
Causalean.Stat

For a tuple length and a number of available sample positions, the ordered injective-tuple count is the real-valued number of ordered selections of that length with distinct entries among the first specified number of observations.

Definition (Lean source)
m n :
injectiveTupleCount m n :
((injectiveTuples m n).card : ℝ)
Causalean.Stat.injectiveTupleCount · Causalean/Stat/UStatistic/OrderM/Basic.lean:54
def injectiveSubtypeEquivEmbedding reviewed
Causalean.Stat

For a domain size and a codomain size, the equivalence between injective maps and embeddings identifies every injective map from an mm-element index set to an nn-element index set with the corresponding embedding, and conversely.

Definition (Lean source)
m n :
injectiveSubtypeEquivEmbedding m n :
{t : Fin m → Fin n // Injective t} ≃ (Fin m ↪ Fin n)
clause 1
toFun t := ⟨t.1, t.2⟩
clause 2
invFun f := ⟨f, f.2⟩
clause 3
left_inv t := by cases t; rfl
clause 4
right_inv f := by cases f; rfl
Causalean.Stat.injectiveSubtypeEquivEmbedding · Causalean/Stat/UStatistic/OrderM/Basic.lean:58
def uStatisticOrder reviewed
Causalean.Stat

Given an independent and identically distributed sample on a measurable sample space, with a specified sample-space measure and observation-space measure, an order-mm real-valued kernel, and a sample size, the fixed-order U-statistic maps each sample outcome to the average of the kernel over all ordered mm-tuples of distinct observations among its first nn observations.

Definition (Lean source)
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
n :
uStatisticOrder S h n :
Ω → ℝ
fun ω
=> (injectiveTupleCount m n)⁻¹ * ∑ t ∈ injectiveTuples m n, h (fun j => S.Z (t j : ℕ) ω)
Causalean.Stat.uStatisticOrder · Causalean/Stat/UStatistic/OrderM/Basic.lean:85 · uses IIDSample
def uMeanOrder reviewed
Causalean.Stat

Given a measurable observation space and an order-mm real-valued kernel on it and a measure on that space, the population mean of the kernel is its integral under the mm-fold product of that measure.

Definition (Lean source)
X :
Type u_2
shared
m :
h :
(Fin m → X) → ℝ
P :
uMeanOrder h P :
∫ z, h z ∂(Measure.pi fun _ : Fin m => P)
def insertCoord reviewed
Causalean.Stat

Given an order mm, a distinguished coordinate, a value in the observation space, and values for every remaining coordinate, the completed coordinate tuple assigns the given value to the distinguished coordinate and the supplied remaining values to all other coordinates.

Definition (Lean source)
X :
Type u_2
shared
m :
j :
Fin m
x :
X
tail :
({k : Fin m // k ≠ j}) → X
insertCoord j x tail :
Fin m → X
fun k => if hkj : k = j then x else tail ⟨k, hkj⟩
def uProjOrderAt reviewed
Causalean.Stat

Given an order-mm real-valued kernel, a distinguished coordinate, and a measure on the observation space, the coordinate-specific first Hoeffding projection maps a proposed value at that coordinate to the kernel integrated over the product measure for all other coordinates, minus the kernel's population mean.

Definition (Lean source)
X :
Type u_2
shared
m :
j :
Fin m
h :
(Fin m → X) → ℝ
P :
uProjOrderAt j h P :
X → ℝ
fun x
=> (∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P))
def uProjOrder reviewed
Causalean.Stat

Given a positive order mm, an order-mm real-valued kernel, and a measure on the observation space, the first Hoeffding projection is the coordinate-specific first projection obtained by treating the first coordinate as distinguished.

Definition (Lean source)
X :
Type u_2
shared
m :
h :
(Fin m → X) → ℝ
P :
uProjOrder h P :
X → ℝ
uProjOrderAt (⟨0, Nat.pos_of_ne_zero (NeZero.ne m)⟩ : Fin m) h P
def uDegenOrder reviewed
Causalean.Stat

Given a positive order mm, an order-mm real-valued kernel, and a measure on the observation space, the higher-order residual kernel maps each mm-tuple to the original kernel value minus its population mean and minus the sum of all coordinate-specific first Hoeffding projections at that tuple.

Definition (Lean source)
X :
Type u_2
shared
m :
h :
(Fin m → X) → ℝ
P :
uDegenOrder h P :
(Fin m → X) → ℝ
fun z => h z - uMeanOrder h P - ∑ j : Fin m, uProjOrderAt j h P (z j)
theorem hoeffding_decomp_order reviewed
Causalean.Stat

For an order-m kernel h, population law P, and an m-tuple of points z, the kernel value decomposes as the population mean plus the sum of the m coordinatewise first Hoeffding projections plus the degenerate higher-order residual kernel evaluated at z.

Formal statement
X :
Type u_2
shared
m :
h :
(Fin m → X) → ℝ
P :
z :
Fin m → X
h z = uMeanOrder h P + (∑ j : Fin m, uProjOrderAt j h P (z j)) + uDegenOrder h P z
Proof (Lean source)
theorem hoeffding_decomp_order {m : ℕ} [NeZero m] (h : (Fin m → X) → ℝ) (P : Measure X) (z : Fin m → X) : h z = uMeanOrder h P + (∑ j : Fin m, uProjOrderAt j h P (z j)) + uDegenOrder h P z := by simp only [uDegenOrder] ring
def pairKernel reviewed
Causalean.Stat

Given an observation space and a real-valued binary kernel, the corresponding two-coordinate kernel maps an ordered pair to the binary kernel evaluated at its first and second entries.

Definition (Lean source)
X :
Type u_2
shared
h :
X → X → ℝ
pairKernel h :
(Fin 2 → X) → ℝ
fun z => h (z 0) (z 1)
def finiteInjectiveTuples reviewed
Causalean.Stat

For a finite coordinate family and a sample size, the collection of injective sample assignments consists of all assignments sending distinct coordinates to distinct positions among the first nn observations.

Definition (Lean source)
ι :
Type*
n :
finiteInjectiveTuples ι n :
Finset (ι → Fin n)
by classical exact Finset.univ.filter Injective
Causalean.Stat.finiteInjectiveTuples · Causalean/Stat/UStatistic/OrderM/Basic.lean:425
def normalizedFiniteKernelStatistic reviewed
Causalean.Stat

Given an independent and identically distributed sample on a measurable sample space, with a specified sample-space measure and observation-space measure, a finite coordinate family, a real-valued kernel indexed by that family, and a sample size, the normalized finite-kernel statistic maps each sample outcome to the average kernel value over every injective assignment of the coordinate family to the first nn sample positions. This is an average when the family has at most nn members; otherwise there are no such assignments, the normalizing count is zero, and the value is zero by the inverse-of-zero convention.

Definition (Lean source)
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
ι :
Type*
k :
(ι → X) → ℝ
n :
normalizedFiniteKernelStatistic S k n :
Ω → ℝ
fun ω
=> ((n.descFactorial (card ι) : ℝ)⁻¹) * ∑ t ∈ finiteInjectiveTuples ι n, k (fun i => S.Z (t i : ℕ) ω)
Causalean.Stat.normalizedFiniteKernelStatistic · Causalean/Stat/UStatistic/OrderM/Basic.lean:449 · uses IIDSample
def orderedProductKernel reviewed
Causalean.Stat

Given an observation space, an order, and one real-valued function of an observation for each coordinate, the ordered-product kernel maps an ordered rr-tuple to the product of its coordinate-specific function values.

Definition (Lean source)
X :
Type u_2
shared
r :
f :
Fin r → X → ℝ
orderedProductKernel f :
(Fin r → X) → ℝ
fun z => ∏ i, f i (z i)
Causalean.Stat.orderedProductKernel · Causalean/Stat/UStatistic/OrderM/Basic.lean:455
def normalizedOrderedProductStatistic reviewed
Causalean.Stat

Given an independent and identically distributed sample on a measurable sample space, with a specified sample-space measure and observation-space measure, an order, one real-valued function of an observation for each coordinate, and a sample size, the normalized ordered-product statistic maps each sample outcome to the average, over all injective ordered rr-tuples from its first nn observations, of the product of the corresponding coordinate-specific function values. For r>nr > n there are no such tuples and the value is zero by convention.

Definition (Lean source)
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
r :
f :
Fin r → X → ℝ
n :
normalizedOrderedProductStatistic S f n :
Ω → ℝ
Causalean.Stat.normalizedOrderedProductStatistic · Causalean/Stat/UStatistic/OrderM/Basic.lean:459 · uses IIDSample
13 supporting declarations (lemmas, instances)
Hajek 6 core · 5 supporting Develops the fixed-order Hájek decomposition for U-statistics indexed by injective ordered m-tuples. ★ uStatisticOrder_sub_uMean_eq★ uStatisticOrder_isAsymLinear

Develops the fixed-order Hájek decomposition for U-statistics indexed by injective ordered m-tuples.

The main objects are uInfluenceOrder, the sum of the coordinatewise first Hoeffding projections; uRemainderOrder, the U-statistic formed from the higher-order residual kernel; and OrderDegenerateNegligible, the √n-scale negligibility hypothesis consumed by the fixed-order CLT. The theorem uStatisticOrder_sub_uMean_eq proves the exact finite-sample decomposition, and uStatisticOrder_isAsymLinear packages it as asymptotic linearity once the residual term is negligible.

def uInfluenceOrder reviewed
Causalean.Stat

For a measurable observation space, an order mm, a real-valued kernel of that order, and an observation distribution, the first-order influence function maps each observation to the sum of the kernel's first Hoeffding projections over all mm coordinate positions.

Definition (Lean source)
X :
Type u_2
shared
m :
h :
(Fin m → X) → ℝ
P :
uInfluenceOrder h P :
X → ℝ
fun x => ∑ j : Fin m, uProjOrderAt j h P x
Causalean.Stat.uInfluenceOrder · Causalean/Stat/UStatistic/OrderM/Hajek.lean:41
def uRemainderOrder reviewed
Causalean.Stat

For an i.i.d. sample on a measurable sample space, with observations in a measurable observation space and their probability distribution, a positive integer order, a real-valued kernel of that order, and a sample size, the higher-order Hájek remainder statistic is the order-mm U-statistic of the kernel remaining after all of its first-order Hoeffding projections have been removed.

Definition (Lean source)
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
n :
uRemainderOrder S h n :
Ω → ℝ
Causalean.Stat.uRemainderOrder · Causalean/Stat/UStatistic/OrderM/Hajek.lean:60 · uses IIDSample
structure OrderDegenKernel reviewed
Causalean.Stat

Fully degenerate order-m kernel. A kernel g on m-tuples over X is completely degenerate under the product measure PmP^{\otimes m} when g is measurable, g is invariant under permuting its m coordinates, integrating g over any one coordinate against P gives zero, whichever coordinate and values are held fixed for the rest, and g is square-integrable under PmP^{\otimes m}.

Definition (Lean source)
P :
m :
g :
(Fin m → X) → ℝ
meas :
symm :
∀ σ : Perm (Fin m), ∀ z, g (z ∘ σ) = g z
deg :
∀ j (tail : ({k : Fin m // k ≠ j}) → X), ∫ x, g (insertCoord j x tail) ∂P = 0
sq :
Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
Causalean.Stat.OrderDegenKernel · Causalean/Stat/UStatistic/OrderM/Hajek.lean:69
def OrderDegenerateNegligible reviewed
Causalean.Stat

For an i.i.d. sample on a measurable sample space, with observations in a measurable observation space and their probability distribution, a positive integer order, and a real-valued kernel of that order, the order-mm degeneracy-negligibility condition asserts that the kernel's higher-order Hájek remainder, multiplied by n\sqrt n, converges in probability to zero as the sample size tends to infinity.

Definition (Lean source)
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
OrderDegenerateNegligible S h :
Prop
IsLittleOp (fun n ω => sqrt (n : ℝ) * uRemainderOrder S h n ω) (fun _ => (1 : ℝ)) μ
Causalean.Stat.OrderDegenerateNegligible · Causalean/Stat/UStatistic/OrderM/Hajek.lean:95 · uses IIDSample
theorem uStatisticOrder_sub_uMean_eq reviewed
Causalean.Stat

Hájek decomposition for a fixed-order U-statistic. For an i.i.d. sample S, order-m kernel h, and sample outcome ω, if the sample size is at least m, then the order-m U-statistic centered at its population mean decomposes exactly as the average of the first-order influence function over the first n sample points plus the higher-order Hájek remainder statistic.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
n :
hmn :
m ≤ n
ω :
Ω
= (n : ℝ)⁻¹ * (∑ i ∈ range n, uInfluenceOrder h P (S.Z i ω))
+ uRemainderOrder S h n ω
Proof (Lean source)
theorem uStatisticOrder_sub_uMean_eq (S : IIDSample Ω X μ P) {m : ℕ} [NeZero m] (h : (Fin m → X) → ℝ) {n : ℕ} (hmn : m ≤ n) (ω : Ω) : uStatisticOrder S h n ω - uMeanOrder h P = (n : ℝ)⁻¹ * (∑ i ∈ range n, uInfluenceOrder h P (S.Z i ω)) + uRemainderOrder S h n ω := by classical have hmpos : 0 < m := Nat.pos_of_ne_zero (NeZero.ne m) have hnpos_nat : 0 < n := lt_of_lt_of_le hmpos hmn have hnne : (n : ℝ) ≠ 0 := by exact_mod_cast (Nat.ne_of_gt hnpos_nat) have hcount_ne : injectiveTupleCount m n ≠ 0 := injectiveTupleCount_ne_zero hmn have hterm : ∀ t ∈ injectiveTuples m n, h (fun j => S.Z (t j : ℕ) ω) = uMeanOrder h P + (∑ j : Fin m, uProjOrderAt j h P (S.Z (t j : ℕ) ω)) + uDegenOrder h P (fun j => S.Z (t j : ℕ) ω) := fun t _ => hoeffding_decomp_order h P _ have hproj : (∑ t ∈ injectiveTuples m n, ∑ j : Fin m, uProjOrderAt j h P (S.Z (t j : ℕ) ω)) = (injectiveTupleCount m n / (n : ℝ)) * (∑ i ∈ range n, uInfluenceOrder h P (S.Z i ω)) := by rw [Finset.sum_comm] rw [show (∑ j : Fin m, ∑ t ∈ injectiveTuples m n, uProjOrderAt j h P (S.Z (t j : ℕ) ω)) = ∑ j : Fin m, (injectiveTupleCount m n / (n : ℝ)) * ∑ i ∈ range n, uProjOrderAt j h P (S.Z i ω) from by apply Finset.sum_congr rfl intro j _ exact sum_injectiveTuples_apply_eq_range hmn j (fun i => uProjOrderAt j h P (S.Z i ω))] rw [← Finset.mul_sum] congr 1 rw [Finset.sum_comm] simp [uInfluenceOrder] have hsum : (∑ t ∈ injectiveTuples m n, h (fun j => S.Z (t j : ℕ) ω)) = injectiveTupleCount m n * uMeanOrder h P + (injectiveTupleCount m n / (n : ℝ)) * (∑ i ∈ range n, uInfluenceOrder h P (S.Z i ω)) + ∑ t ∈ injectiveTuples m n, uDegenOrder h P (fun j => S.Z (t j : ℕ) ω) := by rw [Finset.sum_congr rfl hterm] rw [Finset.sum_add_distrib, Finset.sum_add_distrib] rw [Finset.sum_const, nsmul_eq_mul] rw [hproj] simp only [injectiveTupleCount] simp only [uStatisticOrder, uRemainderOrder, hsum] field_simp [hcount_ne, hnne] ring
theorem uStatisticOrder_isAsymLinear reviewed
Causalean.Stat

Fixed-order U-statistic asymptotic linearity. For an i.i.d. sample S and an order-m kernel h, write ψ for the summed coordinatewise first Hoeffding projection of h. If ψ has population mean zero and is square-integrable, and if the higher-order Hájek remainder of the order-m U-statistic is negligible at the √n scale, then the order-m U-statistic is asymptotically linear toward its population mean uMeanOrder h P, with influence function ψ.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
hψ_mean :
∫ x, uInfluenceOrder h P x ∂P = 0
hψ_sq :
Integrable (fun x => (uInfluenceOrder h P x) ^ 2) P
Proof (Lean source)
theorem uStatisticOrder_isAsymLinear (S : IIDSample Ω X μ P) {m : ℕ} [NeZero m] (h : (Fin m → X) → ℝ) (hψ_mean : ∫ x, uInfluenceOrder h P x ∂P = 0) (hψ_sq : Integrable (fun x => (uInfluenceOrder h P x) ^ 2) P) (hneg : OrderDegenerateNegligible S h) : IsAsymLinear (uStatisticOrder S h) (uMeanOrder h P) (uInfluenceOrder h P) S (fun r => range r) := by refine ⟨hψ_mean, hψ_sq, ?_⟩ refine isLittleOp_of_eventuallyEq hneg ?_ filter_upwards [eventually_ge_atTop m] with n hn exact uStatisticOrder_remainder_eq S h hn
5 supporting declarations (lemmas, instances)
First­Degen­Kernel 2 core · 1 supporting This module introduces OrderFirstDegenKernel, a measurable square-integrable order-m kernel whose conditional mean is zero after integrating out all coordinates except any chosen one. ★ integral_eq_zero

First-order degenerate fixed-order kernels

This module introduces OrderFirstDegenKernel, a measurable square-integrable order-m kernel whose conditional mean is zero after integrating out all coordinates except any chosen one. This is the degeneracy notion satisfied by the first-order Hoeffding residual in the fixed-order U-statistic CLT.

The namespace results show that such kernels are integrable and have product-law mean zero (OrderFirstDegenKernel.integrable and OrderFirstDegenKernel.integral_eq_zero). Generic IIDSample transport lemmas from OrderM.Variance supply the downstream bounds.

structure OrderFirstDegenKernel reviewed
Causalean.Stat

First-order degenerate order-m kernel. A kernel g on m-tuples over X, together with the population measure P, is first-order degenerate when g is measurable, its first Hoeffding projection vanishes in every coordinate — integrating g over the other m1m-1 coordinates against the product measure leaves zero, whichever coordinate and value are held fixed, and g is square-integrable under the product measure PmP^{\otimes m}.

Definition (Lean source)
P :
m :
g :
(Fin m → X) → ℝ
meas :
firstDeg :
∀ (j : Fin m) (x : X)
if
∫ tail : ({k : Fin m // k ≠ j})
then
X, g (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P) = 0

The first Hoeffding projection in every coordinate vanishes: integrating out the m − 1 tail coordinates leaves 0.

sq :
Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
Causalean.Stat.OrderFirstDegenKernel · Causalean/Stat/UStatistic/OrderM/FirstDegenKernel.lean:44
theorem integral_eq_zero reviewed
Causalean.Stat.OrderFirstDegenKernel

Population mean of a first-order degenerate kernel is zero. If the order-m kernel g is first-order degenerate: measurable, square-integrable under the m-fold product law, and with zero mean after integrating out all but any single coordinate, then the population mean of g under the m-fold product law is zero.

Formal statement
X :
Type u_2
shared
m :
(Fin m → X) → ℝ
shared
uMeanOrder g P = 0
Proof (Lean source)
theorem integral_eq_zero [IsFiniteMeasure P] (hg : OrderFirstDegenKernel P g) : uMeanOrder g P = 0 := by classical let j : Fin m := ⟨0, Nat.pos_of_ne_zero (NeZero.ne m)⟩ let p : Fin m → Prop := fun k => k = j let π : Measure (Fin m → X) := Measure.pi fun _ : Fin m => P let πhead : Measure ({k : Fin m // p k} → X) := @Measure.pi {k : Fin m // p k} (fun _ => X) (fintype p) (fun _ => inferInstance) (fun _ => P) let πtail : Measure ({k : Fin m // ¬ p k} → X) := @Measure.pi {k : Fin m // ¬ p k} (fun _ => X) (fintype fun k => ¬ p k) (fun _ => inferInstance) (fun _ => P) let e := MeasurableEquiv.piEquivPiSubtypeProd (fun _ : Fin m => X) p let F : (({k : Fin m // p k} → X) × ({k : Fin m // ¬ p k} → X)) → ℝ := fun q => g (e.symm q) have hmp : MeasurePreserving e π (πhead.prod πtail) := by simpa [π, πhead, πtail, e] using (measurePreserving_piEquivPiSubtypeProd (μ := fun _ : Fin m => P) (α := fun _ : Fin m => X) p) have hπhead_eval : πhead = @Measure.pi {k : Fin m // p k} (fun _ => X) (Fintype.subtypeEq j) (fun _ => inferInstance) (fun _ => P) := by dsimp [πhead] letI : Fintype {k : Fin m // p k} := fintype p refine Measure.pi_eq (μ := fun _ : {k : Fin m // p k} => P) (μ' := @Measure.pi {k : Fin m // p k} (fun _ => X) (Fintype.subtypeEq j) (fun _ => inferInstance) (fun _ => P)) ?_ intro s hs letI : Fintype {k : Fin m // p k} := Fintype.subtypeEq j rw [Measure.pi_pi] simp have hFsm : AEStronglyMeasurable F (πhead.prod πtail) := by exact (hg.meas.comp e.symm.measurable).aestronglyMeasurable have hFint : Integrable F (πhead.prod πtail) := by have hcomp : Integrable (fun z : Fin m → X => F (e z)) π := by simpa [F, e, π] using hg.integrable exact (hmp.integrable_comp hFsm).mp hcomp have hsplit : ∫ z, g z ∂π = ∫ q, F q ∂(πhead.prod πtail) := by have h := hmp.integral_comp' F simpa [F, e, π] using h rw [uMeanOrder] change ∫ z, g z ∂π = 0 rw [hsplit, integral_prod F hFint] have hinner : ∀ head : {k : Fin m // p k} → X, (∫ tail : {k : Fin m // ¬ p k} → X, F (head, tail) ∂πtail) = 0 := by intro head let a0 : {k : Fin m // p k} := ⟨j, rfl⟩ have hhead : (fun tail : {k : Fin m // ¬ p k} → X => F (head, tail)) = fun tail => F ((fun _ : {k : Fin m // p k} => head a0), tail) := by funext tail congr 2 ext a have ha : a = a0 := by cases a with | mk val property => simp only [p] at property subst val rfl rw [ha] rw [hhead] have hfun : (fun tail : {k : Fin m // ¬ p k} → X => F ((fun _ : {k : Fin m // p k} => head a0), tail)) = fun tail : ({k : Fin m // k ≠ j}) → X => g (insertCoord j (head a0) tail) := by funext tail change g (fun k : Fin m => if h : k = j then head a0 else tail ⟨k, h⟩) = g (insertCoord j (head a0) tail) rfl rw [hfun] exact hg.firstDeg j (head a0) rw [show (fun head : {k : Fin m // p k} → X => ∫ tail : {k : Fin m // ¬ p k} → X, F (head, tail) ∂πtail) = fun _ => 0 by funext head exact hinner head] simp
Causalean.Stat.OrderFirstDegenKernel.integral_eq_zero · Causalean/Stat/UStatistic/OrderM/FirstDegenKernel.lean:79 · uses OrderFirstDegenKernel , uMeanOrder
1 supporting declaration (lemmas, instances)
Partial­Matching 11 core · 8 supporting 2 to review This module packages a partial matching between two finite coordinate sets as an equivalence between selected subsets. ★ card_partialMatchingsOfSize

Finite partial matchings

This module packages a partial matching between two finite coordinate sets as an equivalence between selected subsets. It provides the fixed-cardinality families used to classify collisions between two ordered injective tuples.

structure PartialMatching reviewed
Causalean.Stat

A partial matching between ordered coordinate sets of sizes r and s selects a subset from each side and pairs the selected coordinates bijectively.

Definition (Lean source)
r s :
The selected coordinates on the left.
left :
The selected coordinates on the right.
right :
The one-to-one pairing between the selected coordinate sets.
equiv :
(left : Set (Fin r)) ≃ (right : Set (Fin s))
def size reviewed
Causalean.Stat.PartialMatching

For a partial matching, its matching size is the number of selected left coordinates, equivalently the number of paired coordinates.

Definition (Lean source)
r :
shared
s :
shared
M :
size M :
M.left.card
def empty reviewed
Causalean.Stat.PartialMatching

For two nonnegative coordinate-set sizes, the empty partial matching selects no coordinate on either side and therefore contains no pairs.

Definition (Lean source)
r s :
empty r s :
clause 1
left := ∅
clause 2
right := ∅
clause 3
equiv := { toFun := fun x => elim (by simpa using x.property) invFun := fun x => elim (by simpa using x.property) left_inv := fun x => elim (by simpa using x.property) right_inv := fun x => elim (by simpa using x.property) }
abbrev MergedIndex reviewed
Causalean.Stat.PartialMatching

For a partial matching between coordinate sets of sizes rr and ss, the merged coordinate set contains every left coordinate together with precisely those right coordinates that are not selected by the matching.

Definition (Lean source)
r :
shared
s :
shared
M :
MergedIndex M :
Type
Fin r ⊕ {j : Fin s // j ∉ M.right}
Causalean.Stat.PartialMatching.MergedIndex · Causalean/Stat/UStatistic/OrderM/PartialMatching.lean:80 · uses PartialMatching
def leftInjection reviewed
Causalean.Stat.PartialMatching

For a partial matching and a left coordinate, the left-coordinate injection assigns that coordinate its own position in the merged coordinate set.

Definition (Lean source)
r :
shared
s :
shared
M :
i :
Fin r
leftInjection M i :
M.MergedIndex
inl i
def rightInjection reviewed
Causalean.Stat.PartialMatching

For a partial matching and a right coordinate, the right-coordinate injection assigns the coordinate to its matched left-coordinate position when it is matched, and otherwise to its own separate position in the merged coordinate set.

Definition (Lean source)
r :
shared
s :
shared
M :
j :
Fin s
rightInjection M j :
M.MergedIndex
by classical by_cases hj : j ∈ M.right · exact inl (M.equiv.symm ⟨j, hj⟩).1 · exact inr ⟨j, hj⟩
def partialMatchingEquivSigma reviewed
Causalean.Stat

For two nonnegative coordinate-set sizes, the partial-matching representation equivalence bijects partial matchings with a selected subset of each coordinate set and a bijection between the two selected subsets.

Definition (Lean source)
r s :
partialMatchingEquivSigma r s :
PartialMatching r s ≃ Σ left : Finset (Fin r), Σ right : Finset (Fin s), (left : Set (Fin r)) ≃ (right : Set (Fin s))
clause 1
toFun M := ⟨M.left, M.right, M.equiv⟩
clause 2
invFun M := ⟨M.1, M.2.1, M.2.2⟩
clause 3
left_inv M := by cases M; rfl
clause 4
right_inv M := by cases M; rfl
Causalean.Stat.partialMatchingEquivSigma · Causalean/Stat/UStatistic/OrderM/PartialMatching.lean:114 · uses PartialMatching
instance equivFintype unreviewed
Causalean.Stat

For two finite sets, the collection of bijections between them is itself a finite collection.

Definition (Lean source)
α β :
Type*
equivFintype :
Fintype (α ≃ β)
by classical let e : (α ≃ β) ≃ {f : α → β // Bijective f} := { toFun := fun f => ⟨f, f.bijective⟩ invFun := fun f => Equiv.ofBijective f.1 f.2 left_inv := fun f => Equiv.ext (fun x => rfl) right_inv := fun f => Subtype.ext (funext (fun x => rfl)) } exact Fintype.ofEquiv {f : α → β // Bijective f} e.symm
instance finsetFintype unreviewed
Causalean.Stat

For a finite set, the collection of all its finite subsets is itself a finite collection.

Definition (Lean source)
α :
Type*
finsetFintype :
clause 1
elems := Finset.univ.powerset
clause 2
complete := by simp
def partialMatchingsOfSize reviewed
Causalean.Stat

For two nonnegative coordinate-set sizes and a prescribed number of pairs, the fixed-size partial-matching family is the finite set of all partial matchings having exactly that prescribed number of paired coordinates.

Definition (Lean source)
r s h :
partialMatchingsOfSize r s h :
by classical exact Finset.univ.filter (fun M => M.size = h)
theorem card_partialMatchingsOfSize reviewed
Causalean.Stat

For coordinate-set sizes r and s and matching size h, the number of partial matchings is the product of the two subset counts and the number of permutations of h objects.

Formal statement
r s h :
(partialMatchingsOfSize r s h).card = choose r h * choose s h * h.factorial
Proof (Lean source)
theorem card_partialMatchingsOfSize (r s h : ℕ) : (partialMatchingsOfSize r s h).card = choose r h * choose s h * h.factorial := by /- Count the left and right `h`-subsets, then identify equivalences between two `h`-element subtypes with permutations of `Fin h`. -/ classical let e : {M : PartialMatching r s // M.size = h} ≃ Σ left : {L : Finset (Fin r) // L.card = h}, Σ right : {R : Finset (Fin s) // R.card = h}, (left.1 : Set (Fin r)) ≃ (right.1 : Set (Fin s)) := { toFun := fun M => ⟨⟨M.1.left, M.2⟩, ⟨⟨M.1.right, by simpa [M.2] using M.1.right_card⟩, M.1.equiv⟩⟩ invFun := fun M => ⟨⟨M.1.1, M.2.1.1, M.2.2⟩, M.1.2⟩ left_inv := by rintro ⟨⟨left, right, equiv⟩, hsize⟩ rfl right_inv := by rintro ⟨⟨left, hleft⟩, ⟨⟨right, hright⟩, equiv⟩⟩ rfl } rw [← Fintype.card_of_subtype (partialMatchingsOfSize r s h) (fun M => mem_partialMatchingsOfSize M)] rw [Fintype.card_congr e, Fintype.card_sigma] simp_rw [Fintype.card_sigma] have hequiv : ∀ (left : {L : Finset (Fin r) // L.card = h}) (right : {R : Finset (Fin s) // R.card = h}), card ((left.1 : Set (Fin r)) ≃ (right.1 : Set (Fin s))) = h.factorial := by intro left right have hcard : card (left.1 : Set (Fin r)) = card (right.1 : Set (Fin s)) := by simp [left.2, right.2] let f := choice (Fintype.card_eq.mp hcard) exact (@Fintype.card_congr ((left.1 : Set (Fin r)) ≃ (right.1 : Set (Fin s))) ((left.1 : Set (Fin r)) ≃ (right.1 : Set (Fin s))) equivFintype Equiv.instFintype (Equiv.refl _)).trans (by simpa [left.2] using Fintype.card_equiv f) simp_rw [hequiv] simp [Fintype.card_finset_len, Nat.mul_assoc]
8 supporting declarations (lemmas, instances)
Variance 4 core · 24 supporting Provides the product-law and L² infrastructure for fixed-order U-statistics. ★ map_tuple_eq★ orderTerm_diag★ memLp_rescaled_order

Provides the product-law and infrastructure for fixed-order U-statistics.

For injectively indexed sample tuples, map_fintype_tuple_eq and map_tuple_eq identify the joint law with the product measure. The remaining public lemmas transfer integrability, unbiasedness, mean-zero, and diagonal second-moment facts from an order-m kernel under P^m to the corresponding sample terms and U-statistics. The declaration zetaOrder names the kernel second moment used by the exact and upper-bound variance arguments.

theorem map_tuple_eq reviewed
Causalean.Stat.IIDSample

For an i.i.d. sample S, sample size n, and order m, if the index map t : Fin m → Fin n is injective, then the joint law of the sample coordinates selected by t is the m-fold product measure P^m.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m n :
Fin m → Fin n
ht :
μ.map (fun ω : Ω => fun j : Fin m => S.Z (t j : ℕ) ω) = Measure.pi (fun _ : Fin m => P)
Proof (Lean source)
theorem map_tuple_eq (S : IIDSample Ω X μ P) {m n : ℕ} {t : Fin m → Fin n} (ht : Injective t) : μ.map (fun ω : Ω => fun j : Fin m => S.Z (t j : ℕ) ω) = Measure.pi (fun _ : Fin m => P) := by exact S.map_fintype_tuple_eq ht
Causalean.Stat.IIDSample.map_tuple_eq · Causalean/Stat/UStatistic/OrderM/Variance.lean:62 · uses IIDSample
def zetaOrder reviewed
Causalean.Stat.IIDSample

For a measurable observation space, a measure on that space, a nonnegative integer kernel order mm, and a real-valued kernel of mm observations, the order-mm kernel second moment is g(z)2dPm(z)\int g(z)^2\,dP^m(z), where PmP^m is the product measure of mm independent draws from the given measure.

Definition (Lean source)
X :
Type u_2
shared
P :
m :
g :
(Fin m → X) → ℝ
zetaOrder P g :
∫ z, (g z) ^ 2 ∂(Measure.pi fun _ : Fin m => P)
Causalean.Stat.IIDSample.zetaOrder · Causalean/Stat/UStatistic/OrderM/Variance.lean:361
theorem orderTerm_diag reviewed
Causalean.Stat.IIDSample

For an i.i.d. sample S and order-m kernel g that is measurable, if the index map t is injective, then the second moment of the kernel term evaluated along the sample coordinates selected by t equals the kernel's second moment ζ_m under the m-fold product law.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
shared
n :
shared
g :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
hmeas :
t :
Fin m → Fin n
ht :
∫ ω, (g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ = zetaOrder P g
Proof (Lean source)
theorem orderTerm_diag (hmeas : Measurable g) {t : Fin m → Fin n} (ht : Injective t) : ∫ ω, (g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ = zetaOrder P g := by rw [zetaOrder] rw [← S.map_tuple_eq ht] rw [integral_map (measurable_pi_lambda _ (fun j : Fin m => S.meas (t j : ℕ))).aemeasurable (hmeas.pow_const 2).aestronglyMeasurable]
Causalean.Stat.IIDSample.orderTerm_diag · Causalean/Stat/UStatistic/OrderM/Variance.lean:393 · uses IIDSample , zetaOrder
theorem memLp_rescaled_order reviewed
Causalean.Stat.IIDSample

For an i.i.d. sample S, order-m kernel g that is measurable and square-integrable under the m-fold product law, and sample size n, the √n-rescaled order-m U-statistic of g is square-integrable.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
hmeas :
hsq :
Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
n :
MemLp (fun ω => sqrt (n : ℝ) * uStatisticOrder S g n ω) 2 μ
Proof (Lean source)
theorem memLp_rescaled_order (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) (n : ℕ) : MemLp (fun ω => sqrt (n : ℝ) * uStatisticOrder S g n ω) 2 μ := by have hsum : MemLp (fun ω => ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) 2 μ := S.memLp_injectiveTuples_sum hmeas hsq n have : (fun ω => sqrt (n : ℝ) * uStatisticOrder S g n ω) = (fun ω => (sqrt (n : ℝ) * (injectiveTupleCount m n)⁻¹) * ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) := by funext ω simp only [uStatisticOrder] ring rw [this] exact hsum.const_mul _
Causalean.Stat.IIDSample.memLp_rescaled_order · Causalean/Stat/UStatistic/OrderM/Variance.lean:452 · uses IIDSample , uStatisticOrder
24 supporting declarations (lemmas, instances)
  • map_fintype_tuple_eq theorem — The joint law of any finite collection of distinct sample coordinates is the corresponding product law.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    S :
    IIDSample Ω X μ P
    ι :
    Type*
    n :
    ι → Fin n
    hr :
    μ.map (fun ω : Ω => fun i : ι => S.Z (r i : ℕ) ω) = Measure.pi (fun _ : ι => P)
    Proof (Lean source)
    theorem map_fintype_tuple_eq (S : IIDSample Ω X μ P) {ι : Type*} [Fintype ι] {n : ℕ} {r : ι → Fin n} (hr : Injective r) : μ.map (fun ω : Ω => fun i : ι => S.Z (r i : ℕ) ω) = Measure.pi (fun _ : ι => P) := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure have hrNat : Injective (fun i : ι => (r i : ℕ)) := by intro a b hab exact hr (Fin.ext hab) have hindep : iIndepFun (fun i : ι => S.Z (r i : ℕ)) μ := S.indep.precomp hrNat have hmap := (ProbabilityTheory.iIndepFun_iff_map_fun_eq_pi_map (fun i : ι => (S.meas (r i : ℕ)).aemeasurable)).mp hindep calc μ.map (fun ω : Ω => fun i : ι => S.Z (r i : ℕ) ω) = Measure.pi (fun i : ι => μ.map (S.Z (r i : ℕ))) := hmap _ = Measure.pi (fun _ : ι => P) := by congr with i rw [S.map_eq (r i : ℕ)]
    Causalean.Stat.IIDSample.map_fintype_tuple_eq · Causalean/Stat/UStatistic/OrderM/Variance.lean:36
  • integrable theorem — A square-integrable order-m degenerate kernel is integrable under the product law.
    X :
    Type u_2
    shared
    m :
    (Fin m → X) → ℝ
    shared
    hg :
    Integrable g (Measure.pi fun _ : Fin m => P)
    Proof (Lean source)
    @[fun_prop] theorem integrable [IsFiniteMeasure P] (hg : OrderDegenKernel P g) : Integrable g (Measure.pi fun _ : Fin m => P) := ((memLp_two_iff_integrable_sq hg.meas.aestronglyMeasurable).mpr hg.sq).integrable (by norm_num)
    Causalean.Stat.OrderDegenKernel.integrable · Causalean/Stat/UStatistic/OrderM/Variance.lean:82
  • integral_eq_zero theorem — A fully degenerate order-m kernel has zero product-law mean.
    X :
    Type u_2
    shared
    m :
    (Fin m → X) → ℝ
    shared
    hg :
    uMeanOrder g P = 0
    Proof (Lean source)
    theorem integral_eq_zero (hg : OrderDegenKernel P g) : uMeanOrder g P = 0 := by classical let j : Fin m := ⟨0, Nat.pos_of_ne_zero (NeZero.ne m)⟩ let p : Fin m → Prop := fun k => k = j let π : Measure (Fin m → X) := Measure.pi fun _ : Fin m => P let πhead : Measure ({k : Fin m // p k} → X) := @Measure.pi {k : Fin m // p k} (fun _ => X) (fintype p) (fun _ => inferInstance) (fun _ => P) let πtail : Measure ({k : Fin m // ¬ p k} → X) := @Measure.pi {k : Fin m // ¬ p k} (fun _ => X) (fintype fun k => ¬ p k) (fun _ => inferInstance) (fun _ => P) let e := MeasurableEquiv.piEquivPiSubtypeProd (fun _ : Fin m => X) p let F : (({k : Fin m // p k} → X) × ({k : Fin m // ¬ p k} → X)) → ℝ := fun q => g (e.symm q) have hmp : MeasurePreserving e π (πhead.prod πtail) := by simpa [π, πhead, πtail, e] using (measurePreserving_piEquivPiSubtypeProd (μ := fun _ : Fin m => P) (α := fun _ : Fin m => X) p) have hπhead_eval : πhead = @Measure.pi {k : Fin m // p k} (fun _ => X) (Fintype.subtypeEq j) (fun _ => inferInstance) (fun _ => P) := by dsimp [πhead] letI : Fintype {k : Fin m // p k} := fintype p refine Measure.pi_eq (μ := fun _ : {k : Fin m // p k} => P) (μ' := @Measure.pi {k : Fin m // p k} (fun _ => X) (Fintype.subtypeEq j) (fun _ => inferInstance) (fun _ => P)) ?_ intro s hs letI : Fintype {k : Fin m // p k} := Fintype.subtypeEq j rw [Measure.pi_pi] simp have hFsm : AEStronglyMeasurable F (πhead.prod πtail) := by exact (hg.meas.comp e.symm.measurable).aestronglyMeasurable have hFint : Integrable F (πhead.prod πtail) := by have hcomp : Integrable (fun z : Fin m → X => F (e z)) π := by simpa [F, e, π] using hg.integrable exact (hmp.integrable_comp hFsm).mp hcomp have hsplit : ∫ z, g z ∂π = ∫ q, F q ∂(πhead.prod πtail) := by have h := hmp.integral_comp' F simpa [F, e, π] using h rw [uMeanOrder] change ∫ z, g z ∂π = 0 rw [hsplit, integral_prod_symm F hFint] have hinner : ∀ tail : {k : Fin m // ¬ p k} → X, (∫ head : {k : Fin m // p k} → X, F (head, tail) ∂πhead) = 0 := by intro tail let a0 : {k : Fin m // p k} := ⟨j, rfl⟩ have hmpu : MeasurePreserving (Function.eval a0) πhead P := by rw [hπhead_eval] simpa [p, a0] using (measurePreserving_eval (fun _ : {k : Fin m // p k} => P) a0) have hpoint : (fun head : {k : Fin m // p k} → X => F (head, tail)) = fun head => F ((fun _ : {k : Fin m // p k} => head a0), tail) := by funext head congr 2 ext a have ha : a = a0 := by cases a with | mk val property => simp only [p] at property subst val rfl rw [ha] have hchange : (∫ head : {k : Fin m // p k} → X, F (head, tail) ∂πhead) = ∫ x : X, F ((fun _ : {k : Fin m // p k} => x), tail) ∂P := by rw [hpoint] have hsm : AEStronglyMeasurable (fun x : X => F ((fun _ : {k : Fin m // p k} => x), tail)) (Measure.map (Function.eval a0) πhead) := by rw [hmpu.map_eq] exact (hg.meas.comp (by change Measurable (fun x : X => e.symm ((fun _ : {k : Fin m // p k} => x), tail)) measurability)).aestronglyMeasurable have hmap := integral_map hmpu.measurable.aemeasurable hsm rw [hmpu.map_eq] at hmap exact hmap.symm rw [hchange] have hfun : (fun x : X => F ((fun _ : {k : Fin m // p k} => x), tail)) = fun x : X => g (insertCoord j x tail) := by funext x change g (fun k : Fin m => if h : k = j then x else tail ⟨k, h⟩) = g (insertCoord j x tail) rfl rw [hfun] exact hg.deg j tail rw [show (fun tail : {k : Fin m // ¬ p k} → X => ∫ head : {k : Fin m // p k} → X, F (head, tail) ∂πhead) = fun _ => 0 by funext tail exact hinner tail] simp
    Causalean.Stat.OrderDegenKernel.integral_eq_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:90
  • integrable_orderKernelTerm theorem — An order-m kernel term along an injective tuple is integrable whenever the kernel is integrable under the product law.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    S :
    IIDSample Ω X μ P
    shared
    h :
    (Fin m → X) → ℝ
    hmeas :
    hint :
    Integrable h (Measure.pi fun _ : Fin m => P)
    t :
    Fin m → Fin n
    ht :
    Integrable (fun ω => h (fun j => S.Z (t j : ℕ) ω)) μ
    Proof (Lean source)
    theorem integrable_orderKernelTerm {h : (Fin m → X) → ℝ} (hmeas : Measurable h) (hint : Integrable h (Measure.pi fun _ : Fin m => P)) {t : Fin m → Fin n} (ht : Injective t) : Integrable (fun ω => h (fun j => S.Z (t j : ℕ) ω)) μ := by have hmap : Integrable h (μ.map (fun ω : Ω => fun j : Fin m => S.Z (t j : ℕ) ω)) := by rw [S.map_tuple_eq ht] exact hint exact (integrable_map_measure hmeas.aestronglyMeasurable (measurable_pi_lambda _ (fun j : Fin m => S.meas (t j : ℕ))).aemeasurable).mp hmap
    Causalean.Stat.IIDSample.integrable_orderKernelTerm · Causalean/Stat/UStatistic/OrderM/Variance.lean:194
  • integral_orderKernelTerm_eq theorem — The expectation of an injectively indexed order-m kernel term equals the kernel's product-law mean.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    S :
    IIDSample Ω X μ P
    shared
    h :
    (Fin m → X) → ℝ
    hmeas :
    t :
    Fin m → Fin n
    ht :
    ∫ ω, h (fun j => S.Z (t j : ℕ) ω) ∂μ = ∫ z, h z ∂(Measure.pi fun _ : Fin m => P)
    Proof (Lean source)
    theorem integral_orderKernelTerm_eq {h : (Fin m → X) → ℝ} (hmeas : Measurable h) {t : Fin m → Fin n} (ht : Injective t) : ∫ ω, h (fun j => S.Z (t j : ℕ) ω) ∂μ = ∫ z, h z ∂(Measure.pi fun _ : Fin m => P) := by rw [← S.map_tuple_eq ht] rw [integral_map (measurable_pi_lambda _ (fun j : Fin m => S.meas (t j : ℕ))).aemeasurable hmeas.aestronglyMeasurable]
    Causalean.Stat.IIDSample.integral_orderKernelTerm_eq · Causalean/Stat/UStatistic/OrderM/Variance.lean:209
  • integral_orderKernelTerm_eq_zero_of_uMean_zero theorem — An injectively indexed order-m kernel term has mean zero whenever the kernel has zero product-law mean.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    S :
    IIDSample Ω X μ P
    shared
    h :
    (Fin m → X) → ℝ
    hmeas :
    t :
    Fin m → Fin n
    ht :
    hmean_zero :
    uMeanOrder h P = 0
    ∫ ω, h (fun j => S.Z (t j : ℕ) ω) ∂μ = 0
    Proof (Lean source)
    theorem integral_orderKernelTerm_eq_zero_of_uMean_zero {h : (Fin m → X) → ℝ} (hmeas : Measurable h) {t : Fin m → Fin n} (ht : Injective t) (hmean_zero : uMeanOrder h P = 0) : ∫ ω, h (fun j => S.Z (t j : ℕ) ω) ∂μ = 0 := by rw [S.integral_orderKernelTerm_eq hmeas ht, ← uMeanOrder, hmean_zero]
    Causalean.Stat.IIDSample.integral_orderKernelTerm_eq_zero_of_uMean_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:221
  • integral_uStatisticOrder_eq_uMean theorem — The fixed-order U-statistic is unbiased: its expectation is the product-law kernel mean.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    S :
    IIDSample Ω X μ P
    shared
    m n :
    (Fin m → X) → ℝ
    hmeas :
    hint :
    Integrable h (Measure.pi fun _ : Fin m => P)
    hmn :
    m ≤ n
    ∫ ω, uStatisticOrder S h n ω ∂μ = uMeanOrder h P
    Proof (Lean source)
    theorem integral_uStatisticOrder_eq_uMean {m n : ℕ} {h : (Fin m → X) → ℝ} (hmeas : Measurable h) (hint : Integrable h (Measure.pi fun _ : Fin m => P)) (hmn : m ≤ n) : ∫ ω, uStatisticOrder S h n ω ∂μ = uMeanOrder h P := by classical have hcount_ne : injectiveTupleCount m n ≠ 0 := injectiveTupleCount_ne_zero hmn have hcard_ne : ((injectiveTuples m n).card : ℝ) ≠ 0 := by simpa [injectiveTupleCount] using hcount_ne have hterm_int : ∀ t ∈ injectiveTuples m n, Integrable (fun ω => h (fun j => S.Z (t j : ℕ) ω)) μ := by intro t ht exact S.integrable_orderKernelTerm hmeas hint ((Finset.mem_filter.mp ht).2) simp only [uStatisticOrder] integral_linearity have hsum_eval : (∑ t ∈ injectiveTuples m n, ∫ ω, h (fun j => S.Z (t j : ℕ) ω) ∂μ) = ∑ _t ∈ injectiveTuples m n, ∫ z, h z ∂(Measure.pi fun _ : Fin m => P) := by apply Finset.sum_congr rfl intro t ht exact S.integral_orderKernelTerm_eq hmeas ((Finset.mem_filter.mp ht).2) rw [hsum_eval] rw [Finset.sum_const, nsmul_eq_mul, uMeanOrder] rw [injectiveTupleCount] field_simp [hcard_ne]
    Causalean.Stat.IIDSample.integral_uStatisticOrder_eq_uMean · Causalean/Stat/UStatistic/OrderM/Variance.lean:230
  • integral_uStatisticOrder_eq_zero_of_uMean_zero theorem — A fixed-order U-statistic with product-law mean zero has expectation zero.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    S :
    IIDSample Ω X μ P
    shared
    m n :
    (Fin m → X) → ℝ
    hmeas :
    hint :
    Integrable h (Measure.pi fun _ : Fin m => P)
    hmn :
    m ≤ n
    hmean_zero :
    uMeanOrder h P = 0
    ∫ ω, uStatisticOrder S h n ω ∂μ = 0
    Proof (Lean source)
    theorem integral_uStatisticOrder_eq_zero_of_uMean_zero {m n : ℕ} {h : (Fin m → X) → ℝ} (hmeas : Measurable h) (hint : Integrable h (Measure.pi fun _ : Fin m => P)) (hmn : m ≤ n) (hmean_zero : uMeanOrder h P = 0) : ∫ ω, uStatisticOrder S h n ω ∂μ = 0 := by rw [S.integral_uStatisticOrder_eq_uMean hmeas hint hmn, hmean_zero]
    Causalean.Stat.IIDSample.integral_uStatisticOrder_eq_zero_of_uMean_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:262
  • integral_rescaled_uStatisticOrder_eq_sqrt_mul_uMean theorem — The rescaled fixed-order U-statistic has mean equal to the same rescaling of the product-law kernel mean.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    S :
    IIDSample Ω X μ P
    shared
    m n :
    (Fin m → X) → ℝ
    hmeas :
    hint :
    Integrable h (Measure.pi fun _ : Fin m => P)
    hmn :
    m ≤ n
    ∫ ω, sqrt (n : ℝ) * uStatisticOrder S h n ω ∂μ = sqrt (n : ℝ) * uMeanOrder h P
    Proof (Lean source)
    theorem integral_rescaled_uStatisticOrder_eq_sqrt_mul_uMean {m n : ℕ} {h : (Fin m → X) → ℝ} (hmeas : Measurable h) (hint : Integrable h (Measure.pi fun _ : Fin m => P)) (hmn : m ≤ n) : ∫ ω, sqrt (n : ℝ) * uStatisticOrder S h n ω ∂μ = sqrt (n : ℝ) * uMeanOrder h P := by integral_linearity rw [S.integral_uStatisticOrder_eq_uMean hmeas hint hmn]
    Causalean.Stat.IIDSample.integral_rescaled_uStatisticOrder_eq_sqrt_mul_uMean · Causalean/Stat/UStatistic/OrderM/Variance.lean:273
  • integral_rescaled_uStatisticOrder_eq_zero_of_uMean_zero theorem — If an order-m kernel has product-law mean zero, then the rescaled fixed-order U-statistic has mean zero.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    S :
    IIDSample Ω X μ P
    shared
    m n :
    (Fin m → X) → ℝ
    hmeas :
    hint :
    Integrable h (Measure.pi fun _ : Fin m => P)
    hmn :
    m ≤ n
    hmean_zero :
    uMeanOrder h P = 0
    ∫ ω, sqrt (n : ℝ) * uStatisticOrder S h n ω ∂μ = 0
    Proof (Lean source)
    theorem integral_rescaled_uStatisticOrder_eq_zero_of_uMean_zero {m n : ℕ} {h : (Fin m → X) → ℝ} (hmeas : Measurable h) (hint : Integrable h (Measure.pi fun _ : Fin m => P)) (hmn : m ≤ n) (hmean_zero : uMeanOrder h P = 0) : ∫ ω, sqrt (n : ℝ) * uStatisticOrder S h n ω ∂μ = 0 := by rw [S.integral_rescaled_uStatisticOrder_eq_sqrt_mul_uMean hmeas hint hmn, hmean_zero, mul_zero]
    Causalean.Stat.IIDSample.integral_rescaled_uStatisticOrder_eq_zero_of_uMean_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:286
  • integral_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero theorem — A fully degenerate order-m kernel whose product-law mean is zero gives a mean-zero fixed-order U-statistic. The product-law mean-zero assumption is kept explicit here rather than inferred from coordinatewise degeneracy.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hg :
    hmn :
    m ≤ n
    hmean_zero :
    uMeanOrder g P = 0
    ∫ ω, uStatisticOrder S g n ω ∂μ = 0
    Proof (Lean source)
    theorem integral_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero [IsFiniteMeasure P] [NeZero m] (hg : OrderDegenKernel P g) (hmn : m ≤ n) (hmean_zero : uMeanOrder g P = 0) : ∫ ω, uStatisticOrder S g n ω ∂μ = 0 := S.integral_uStatisticOrder_eq_zero_of_uMean_zero hg.meas hg.integrable hmn hmean_zero
    Causalean.Stat.IIDSample.integral_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:299
  • integral_uStatisticOrder_eq_zero_of_degenKernel theorem — A fully degenerate order-m kernel gives a mean-zero fixed-order U-statistic in the nonempty sampling regime m ≤ n.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hg :
    hmn :
    m ≤ n
    ∫ ω, uStatisticOrder S g n ω ∂μ = 0
    Proof (Lean source)
    theorem integral_uStatisticOrder_eq_zero_of_degenKernel [NeZero m] (hg : OrderDegenKernel P g) (hmn : m ≤ n) : ∫ ω, uStatisticOrder S g n ω ∂μ = 0 := by letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable exact S.integral_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero hg hmn hg.integral_eq_zero
    Causalean.Stat.IIDSample.integral_uStatisticOrder_eq_zero_of_degenKernel · Causalean/Stat/UStatistic/OrderM/Variance.lean:309
  • integral_orderTerm_eq_zero_of_degenKernel_uMean_zero theorem — An injectively indexed fully degenerate order-m kernel term has mean zero when its product-law mean is zero.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hg :
    t :
    Fin m → Fin n
    ht :
    hmean_zero :
    uMeanOrder g P = 0
    ∫ ω, g (fun j => S.Z (t j : ℕ) ω) ∂μ = 0
    Proof (Lean source)
    theorem integral_orderTerm_eq_zero_of_degenKernel_uMean_zero [NeZero m] (hg : OrderDegenKernel P g) {t : Fin m → Fin n} (ht : Injective t) (hmean_zero : uMeanOrder g P = 0) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) ∂μ = 0 := S.integral_orderKernelTerm_eq_zero_of_uMean_zero hg.meas ht hmean_zero
    Causalean.Stat.IIDSample.integral_orderTerm_eq_zero_of_degenKernel_uMean_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:320
  • integral_orderTerm_eq_zero theorem — An injectively indexed fully degenerate order-m kernel term has mean zero.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hg :
    t :
    Fin m → Fin n
    ht :
    ∫ ω, g (fun j => S.Z (t j : ℕ) ω) ∂μ = 0
    Proof (Lean source)
    theorem integral_orderTerm_eq_zero [NeZero m] (hg : OrderDegenKernel P g) {t : Fin m → Fin n} (ht : Injective t) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) ∂μ = 0 := S.integral_orderTerm_eq_zero_of_degenKernel_uMean_zero hg ht hg.integral_eq_zero
    Causalean.Stat.IIDSample.integral_orderTerm_eq_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:329
  • integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero theorem — A fully degenerate order-m kernel whose product-law mean is zero gives a mean-zero rescaled fixed-order U-statistic. This is a mean statement only; it does not assert the variance bound or negligibility.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hg :
    hmn :
    m ≤ n
    hmean_zero :
    uMeanOrder g P = 0
    ∫ ω, sqrt (n : ℝ) * uStatisticOrder S g n ω ∂μ = 0
    Proof (Lean source)
    theorem integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero [IsFiniteMeasure P] [NeZero m] (hg : OrderDegenKernel P g) (hmn : m ≤ n) (hmean_zero : uMeanOrder g P = 0) : ∫ ω, sqrt (n : ℝ) * uStatisticOrder S g n ω ∂μ = 0 := S.integral_rescaled_uStatisticOrder_eq_zero_of_uMean_zero hg.meas hg.integrable hmn hmean_zero
    Causalean.Stat.IIDSample.integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero · Causalean/Stat/UStatistic/OrderM/Variance.lean:337
  • integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel theorem — A fully degenerate order-m kernel gives a mean-zero rescaled fixed-order U-statistic in the nonempty sampling regime m ≤ n. This is a mean statement only; it does not assert the variance bound or negligibility.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hg :
    hmn :
    m ≤ n
    ∫ ω, sqrt (n : ℝ) * uStatisticOrder S g n ω ∂μ = 0
    Proof (Lean source)
    theorem integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel [NeZero m] (hg : OrderDegenKernel P g) (hmn : m ≤ n) : ∫ ω, sqrt (n : ℝ) * uStatisticOrder S g n ω ∂μ = 0 := by letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable exact S.integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel_uMean_zero hg hmn hg.integral_eq_zero
    Causalean.Stat.IIDSample.integral_rescaled_uStatisticOrder_eq_zero_of_degenKernel · Causalean/Stat/UStatistic/OrderM/Variance.lean:349
  • integrable_orderTerm theorem — An order-m kernel term along an injective tuple is integrable.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hint :
    Integrable g (Measure.pi fun _ : Fin m => P)
    t :
    Fin m → Fin n
    ht :
    Integrable (fun ω => g (fun j => S.Z (t j : ℕ) ω)) μ
    Proof (Lean source)
    theorem integrable_orderTerm (hmeas : Measurable g) (hint : Integrable g (Measure.pi fun _ : Fin m => P)) {t : Fin m → Fin n} (ht : Injective t) : Integrable (fun ω => g (fun j => S.Z (t j : ℕ) ω)) μ := S.integrable_orderKernelTerm hmeas hint ht
    Causalean.Stat.IIDSample.integrable_orderTerm · Causalean/Stat/UStatistic/OrderM/Variance.lean:371
  • integrable_orderTerm_sq theorem — The square of an order-m kernel term along an injective tuple is integrable.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsq :
    Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
    t :
    Fin m → Fin n
    ht :
    Integrable (fun ω => (g (fun j => S.Z (t j : ℕ) ω)) ^ 2) μ
    Proof (Lean source)
    theorem integrable_orderTerm_sq (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) {t : Fin m → Fin n} (ht : Injective t) : Integrable (fun ω => (g (fun j => S.Z (t j : ℕ) ω)) ^ 2) μ := by have hmap : Integrable (fun z => (g z) ^ 2) (μ.map (fun ω : Ω => fun j : Fin m => S.Z (t j : ℕ) ω)) := by rw [S.map_tuple_eq ht] exact hsq exact (integrable_map_measure (hmeas.pow_const 2).aestronglyMeasurable (measurable_pi_lambda _ (fun j : Fin m => S.meas (t j : ℕ))).aemeasurable).mp hmap
    Causalean.Stat.IIDSample.integrable_orderTerm_sq · Causalean/Stat/UStatistic/OrderM/Variance.lean:379
  • memLp_orderTerm theorem — Each injective order-m kernel term is in L².
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsq :
    Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
    t :
    Fin m → Fin n
    ht :
    MemLp (fun ω => g (fun j => S.Z (t j : ℕ) ω)) 2 μ
    Proof (Lean source)
    theorem memLp_orderTerm (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) {t : Fin m → Fin n} (ht : Injective t) : MemLp (fun ω => g (fun j => S.Z (t j : ℕ) ω)) 2 μ := by have hm : AEStronglyMeasurable (fun ω => g (fun j => S.Z (t j : ℕ) ω)) μ := (hmeas.comp (measurable_pi_lambda _ (fun j : Fin m => S.meas (t j : ℕ)))).aestronglyMeasurable exact (memLp_two_iff_integrable_sq hm).mpr (S.integrable_orderTerm_sq hmeas hsq ht)
    Causalean.Stat.IIDSample.memLp_orderTerm · Causalean/Stat/UStatistic/OrderM/Variance.lean:407
  • integrable_orderTerm_mul theorem — The product of two injective order-m kernel terms is integrable.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    shared
    n :
    shared
    g :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsq :
    Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
    t q :
    Fin m → Fin n
    ht :
    hq :
    Integrable (fun ω => g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) μ
    Proof (Lean source)
    theorem integrable_orderTerm_mul (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) : Integrable (fun ω => g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) μ := (S.memLp_orderTerm hmeas hsq ht).integrable_mul (S.memLp_orderTerm hmeas hsq hq)
    Causalean.Stat.IIDSample.integrable_orderTerm_mul · Causalean/Stat/UStatistic/OrderM/Variance.lean:418
  • integrable_injectiveTuples_sum theorem — The injective-tuple sum of an order-m kernel is integrable.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hint :
    Integrable g (Measure.pi fun _ : Fin m => P)
    n :
    Integrable (fun ω => ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) μ
    Proof (Lean source)
    @[fun_prop] theorem integrable_injectiveTuples_sum (hmeas : Measurable g) (hint : Integrable g (Measure.pi fun _ : Fin m => P)) (n : ℕ) : Integrable (fun ω => ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) μ := by apply integrable_finset_sum intro t ht exact S.integrable_orderTerm hmeas hint ((Finset.mem_filter.mp ht).2)
    Causalean.Stat.IIDSample.integrable_injectiveTuples_sum · Causalean/Stat/UStatistic/OrderM/Variance.lean:428
  • memLp_injectiveTuples_sum theorem — The injective-tuple sum of an order-m kernel is in L².
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsq :
    Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
    n :
    MemLp (fun ω => ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) 2 μ
    Proof (Lean source)
    theorem memLp_injectiveTuples_sum (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) (n : ℕ) : MemLp (fun ω => ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) 2 μ := by have hsum := memLp_finset_sum (μ := μ) (p := 2) (injectiveTuples m n) (f := fun t ω => g (fun j => S.Z (t j : ℕ) ω)) (fun t ht => S.memLp_orderTerm hmeas hsq ((Finset.mem_filter.mp ht).2)) simpa using hsum
    Causalean.Stat.IIDSample.memLp_injectiveTuples_sum · Causalean/Stat/UStatistic/OrderM/Variance.lean:440
  • zetaOrder_nonneg theorem — ζ_m is nonnegative.
    X :
    Type u_2
    shared
    m :
    (Fin m → X) → ℝ
    0 ≤ zetaOrder P g
    Proof (Lean source)
    theorem zetaOrder_nonneg {m : ℕ} {g : (Fin m → X) → ℝ} : 0 ≤ zetaOrder P g := integral_nonneg (fun _ => sq_nonneg _)
    Causalean.Stat.IIDSample.zetaOrder_nonneg · Causalean/Stat/UStatistic/OrderM/Variance.lean:471
  • integral_normalizedFiniteKernelStatistic theorem — For an i.i.d. sample, a finite coordinate family, a kernel, and sample size n, if the number of coordinates does not exceed the sample size, the kernel is measurable, and the kernel is integrable under the product law, the expected normalized statistic equals the kernel's product-law mean.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    S :
    IIDSample Ω X μ P
    ι :
    Type*
    (ι → X) → ℝ
    n :
    hcard :
    card ι ≤ n
    hkmeas :
    hkint :
    Integrable k (Measure.pi fun _ : ι => P)
    ∫ ω, normalizedFiniteKernelStatistic S k n ω ∂μ = ∫ z, k z ∂(Measure.pi fun _ : ι => P)
    Proof (Lean source)
    theorem integral_normalizedFiniteKernelStatistic (S : IIDSample Ω X μ P) {ι : Type*} [Fintype ι] {k : (ι → X) → ℝ} {n : ℕ} (hcard : card ι ≤ n) (hkmeas : Measurable k) (hkint : Integrable k (Measure.pi fun _ : ι => P)) : ∫ ω, normalizedFiniteKernelStatistic S k n ω ∂μ = ∫ z, k z ∂(Measure.pi fun _ : ι => P) := by /- Expand the finite sum, transport each injective assignment with `IIDSample.map_fintype_tuple_eq`, and cancel the positive tuple count. -/ classical have hdesc_ne : (n.descFactorial (card ι) : ℝ) ≠ 0 := by exact_mod_cast (Nat.descFactorial_pos.mpr hcard).ne' have hterm_int : ∀ t ∈ finiteInjectiveTuples ι n, Integrable (fun ω => k (fun i => S.Z (t i : ℕ) ω)) μ := by intro t ht have htinj : Injective t := (Finset.mem_filter.mp ht).2 have hmap : Integrable k (μ.map (fun ω : Ω => fun i : ι => S.Z (t i : ℕ) ω)) := by rw [S.map_fintype_tuple_eq htinj] exact hkint exact (integrable_map_measure hkmeas.aestronglyMeasurable (measurable_pi_lambda _ (fun i : ι => S.meas (t i : ℕ))).aemeasurable).mp hmap simp only [normalizedFiniteKernelStatistic] integral_linearity have hsum_eval : (∑ t ∈ finiteInjectiveTuples ι n, ∫ ω, k (fun i => S.Z (t i : ℕ) ω) ∂μ) = ∑ _t ∈ finiteInjectiveTuples ι n, ∫ z, k z ∂(Measure.pi fun _ : ι => P) := by apply Finset.sum_congr rfl intro t ht have htinj : Injective t := (Finset.mem_filter.mp ht).2 rw [← S.map_fintype_tuple_eq htinj] rw [integral_map (measurable_pi_lambda _ (fun i : ι => S.meas (t i : ℕ))).aemeasurable hkmeas.aestronglyMeasurable] rw [hsum_eval, Finset.sum_const, nsmul_eq_mul] rw [finiteInjectiveTuples_card] field_simp [hdesc_ne]
    Causalean.Stat.integral_normalizedFiniteKernelStatistic · Causalean/Stat/UStatistic/OrderM/Variance.lean:489
Mixed­Order­Covariance 6 core · 5 supporting This module classifies pairs of injective tuples by their cross-tuple collision matching. ★ centeredCrossMoment_normalizedOrderedProductStatistic

Mixed-order partial-matching expansion

This module classifies pairs of injective tuples by their cross-tuple collision matching. It gives pointwise product and product-law expectation expansions, then exposes the size-zero normalization correction in the centered identity.

def mergedProductKernel reviewed
Causalean.Stat

For two nonnegative orders, a family of real-valued left factors indexed by the first order, a family of real-valued right factors indexed by the second order, and a partial matching between their indices, the merged product kernel assigns to each collection of observations indexed by the merged coordinates the product of all left and right factors, evaluating matched factors at their common observation and unmatched factors at separate observations.

Definition (Lean source)
X :
Type u_2
shared
r s :
f :
Fin r → X → ℝ
g :
Fin s → X → ℝ
M :
mergedProductKernel f g M :
(M.MergedIndex → X) → ℝ
fun z
=> (∏ i : Fin r, f i (z (M.leftInjection i))) * ∏ j : Fin s, g j (z (M.rightInjection j))
def mergedProductMoment reviewed
Causalean.Stat

For two nonnegative orders, a measure on a measurable observation space, families of real-valued left and right factors, and a partial matching between their indices, the merged product moment is the integral of the associated merged product kernel under independent draws from that measure, one draw for each merged coordinate.

Definition (Lean source)
X :
Type u_2
shared
r s :
P :
f :
Fin r → X → ℝ
g :
Fin s → X → ℝ
M :
mergedProductMoment P f g M :
∫ z, mergedProductKernel f g M z ∂(Measure.pi fun _ : M.MergedIndex => P)
def matchingNormalization reviewed
Causalean.Stat

For a nonnegative sample size, two nonnegative factor orders, and a partial matching between their indices, the matching normalization is the falling factorial of the sample size at the number of distinct observations induced by the matching, divided by the product of the two marginal falling factorials.

Definition (Lean source)
n :
r s :
M :
matchingNormalization n M :
(n.descFactorial (r + s - M.size) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ))
def orderedProductMean reviewed
Causalean.Stat

For a nonnegative order, a measure on a measurable observation space, and a family of real-valued coordinate factors, the ordered-product mean is the integral of their coordinatewise product under independent draws from that measure, one draw for every coordinate.

Definition (Lean source)
X :
Type u_2
shared
r :
P :
f :
Fin r → X → ℝ
orderedProductMean P f :
∫ z, orderedProductKernel f z ∂(Measure.pi fun _ : Fin r => P)
def centeredCrossMoment reviewed
Causalean.Stat

For a measure on a measurable sample space and two real-valued random variables on that sample space, the centered cross moment is the integral of their product minus the product of their separate integrals.

Definition (Lean source)
Ω :
Type u_1
shared
μ :
A B :
Ω → ℝ
centeredCrossMoment μ A B :
(∫ ω, A ω * B ω ∂μ) - (∫ ω, A ω ∂μ) * (∫ ω, B ω ∂μ)
theorem centeredCrossMoment_normalizedOrderedProductStatistic reviewed
Causalean.Stat

Under a probability population law, for an i.i.d. sample, statistic orders and a sample size, if the first order does not exceed the sample size and the second order does not exceed the sample size, then for two coordinate-function families, when their product kernels are measurable, their product kernels are integrable, every merged kernel is measurable, and every merged kernel is integrable, their centered cross moment equals an explicit disjoint normalization correction plus the merged moments from every positive-size partial matching.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
S :
IIDSample Ω X μ P
r s n :
hrn :
r ≤ n
hsn :
s ≤ n
f :
Fin r → X → ℝ
g :
Fin s → X → ℝ
hintF :
Integrable (orderedProductKernel f) (Measure.pi fun _ : Fin r => P)
hintG :
Integrable (orderedProductKernel g) (Measure.pi fun _ : Fin s => P)
hmeas :
hint :
∀ M : PartialMatching r s,
Integrable (mergedProductKernel f g M) (Measure.pi fun _ : M.MergedIndex => P)
= (matchingNormalization n (PartialMatching.empty r s) - 1) * orderedProductMean P f * orderedProductMean P g
+ ∑ h ∈ (range (min r s + 1)).filter (fun h => 0 < h), ∑ M ∈ partialMatchingsOfSize r s h, matchingNormalization n M * mergedProductMoment P f g M
Proof (Lean source)
theorem centeredCrossMoment_normalizedOrderedProductStatistic [IsProbabilityMeasure P] (S : IIDSample Ω X μ P) {r s n : ℕ} (hrn : r ≤ n) (hsn : s ≤ n) (f : Fin r → X → ℝ) (g : Fin s → X → ℝ) (hmeasF : Measurable (orderedProductKernel f)) (hmeasG : Measurable (orderedProductKernel g)) (hintF : Integrable (orderedProductKernel f) (Measure.pi fun _ : Fin r => P)) (hintG : Integrable (orderedProductKernel g) (Measure.pi fun _ : Fin s => P)) (hmeas : ∀ M : PartialMatching r s, Measurable (mergedProductKernel f g M)) (hint : ∀ M : PartialMatching r s, Integrable (mergedProductKernel f g M) (Measure.pi fun _ : M.MergedIndex => P)) : centeredCrossMoment μ (normalizedOrderedProductStatistic S f n) (normalizedOrderedProductStatistic S g n) = (matchingNormalization n (PartialMatching.empty r s) - 1) * orderedProductMean P f * orderedProductMean P g + ∑ h ∈ (range (min r s + 1)).filter (fun h => 0 < h), ∑ M ∈ partialMatchingsOfSize r s h, matchingNormalization n M * mergedProductMoment P f g M := by /- Combine the product-moment expansion with unbiasedness of each marginal. Use `eq_empty_of_size_eq_zero` and `mergedProductMoment_empty` to rewrite the unique size-zero term, then partition the remaining sizes by `0 < h`. -/ classical have hfmean : ∫ ω, normalizedOrderedProductStatistic S f n ω ∂μ = orderedProductMean P f := by unfold normalizedOrderedProductStatistic orderedProductMean exact integral_normalizedFiniteKernelStatistic S (by simpa using hrn) hmeasF hintF have hgmean : ∫ ω, normalizedOrderedProductStatistic S g n ω ∂μ = orderedProductMean P g := by unfold normalizedOrderedProductStatistic orderedProductMean exact integral_normalizedFiniteKernelStatistic S (by simpa using hsn) hmeasG hintG let F : ℕ → ℝ := fun h => ∑ M ∈ partialMatchingsOfSize r s h, matchingNormalization n M * mergedProductMoment P f g M have hzero : F 0 = matchingNormalization n (PartialMatching.empty r s) * (orderedProductMean P f * orderedProductMean P g) := by dsimp only [F] rw [Finset.sum_eq_single (PartialMatching.empty r s)] · rw [mergedProductMoment_empty f g hmeasF hmeasG hintF hintG] · intro M hM hne exact elim (hne (PartialMatching.eq_empty_of_size_eq_zero M ((mem_partialMatchingsOfSize M).mp hM))) · intro hnot exact elim (hnot (by simp)) let H := range (min r s + 1) have hnonpos : H.filter (fun h => ¬0 < h) = {0} := by ext h simp [H] have hsplit : (∑ h ∈ H, F h) = F 0 + ∑ h ∈ H.filter (fun h => 0 < h), F h := by have hpart := Finset.sum_filter_add_sum_filter_not H (fun h => 0 < h) F rw [hnonpos] at hpart simpa [add_comm] using hpart.symm unfold centeredCrossMoment rw [integral_normalizedOrderedProductStatistic_mul S hrn hsn f g hmeas hint, hfmean, hgmean] change (∑ h ∈ H, F h) - orderedProductMean P f * orderedProductMean P g = _ rw [hsplit, hzero] ring
5 supporting declarations (lemmas, instances)
CLT 2 core · 0 supporting This module states the asymptotic-normality interface for fixed-order U-statistics. ★ uStatisticOrder_clt★ uStatisticOrder_clt_of_regular

Fixed-order U-statistic CLTs

This module states the asymptotic-normality interface for fixed-order U-statistics. uStatisticOrder_clt converts a centered, square-integrable summed first projection and an explicit higher-order remainder-negligibility hypothesis into the Gaussian limit of the √n-rescaled statistic.

The end-to-end theorem uStatisticOrder_clt_of_regular discharges the negligibility hypothesis from regularity of the residual kernel via orderDegenerateNegligible_of_residual. Together these declarations are the public CLT endpoint for the OrderM U-statistic development.

theorem uStatisticOrder_clt reviewed
Causalean.Stat

Fixed-order U-statistic CLT. For an i.i.d. sample S and an order-m kernel h, write ψ for the summed coordinatewise first Hoeffding projection of h. If ψ is measurable, has population mean zero, and is square-integrable, if the higher-order Hájek remainder of the order-m U-statistic is negligible at the √n scale, and if the √n-rescaled U-statistic is almost-everywhere measurable at every sample size, then the √n-rescaled U-statistic converges in distribution to the centered Gaussian law with variance ∫ψ²dP.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
hψ_meas :
hψ_mean :
∫ x, uInfluenceOrder h P x ∂P = 0
hψ_sq :
Integrable (fun x => (uInfluenceOrder h P x) ^ 2) P
hθn_meas :
∀ n : ℕ,
AEMeasurable (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r) n) μ
Tendsto_dist (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r)) (gaussianMeasure 0 (∫ x, (uInfluenceOrder h P x) ^ 2 ∂P)) μ hθn_meas
Proof (Lean source)
theorem uStatisticOrder_clt (S : IIDSample Ω X μ P) {m : ℕ} [NeZero m] (h : (Fin m → X) → ℝ) (hψ_meas : Measurable (uInfluenceOrder h P)) (hψ_mean : ∫ x, uInfluenceOrder h P x ∂P = 0) (hψ_sq : Integrable (fun x => (uInfluenceOrder h P x) ^ 2) P) (hneg : OrderDegenerateNegligible S h) (hθn_meas : ∀ n : ℕ, AEMeasurable (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r) n) μ) : Tendsto_dist (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r)) (gaussianMeasure 0 (∫ x, (uInfluenceOrder h P x) ^ 2 ∂P)) μ hθn_meas := by have hAL : IsAsymLinear (uStatisticOrder S h) (uMeanOrder h P) (uInfluenceOrder h P) S (fun r => range r) := uStatisticOrder_isAsymLinear S h hψ_mean hψ_sq hneg exact hAL.tendsto_normal hψ_meas hθn_meas
theorem uStatisticOrder_clt_of_regular reviewed
Causalean.Stat

Fixed-order U-statistic CLT (end-to-end). For an i.i.d. sample S and an order-m kernel h, write g for the higher-order Hájek residual of h and ψ for the summed coordinatewise first Hoeffding projection of h. If the residual g is measurable and square-integrable under the m-fold product law, if for every coordinate integrating h over the remaining m − 1 coordinates yields an integrable function of that coordinate with the same population mean uMeanOrder h P in every coordinate and h remains integrable in the remaining coordinates for every fixed value of that coordinate, and if ψ is measurable, mean zero, square-integrable, and the √n-rescaled U-statistic is almost-everywhere measurable at every sample size, then the √n-rescaled order-m U-statistic converges in distribution to the centered Gaussian law with variance ∫ψ²dP.

Formal statement
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
hmeas :
hL2 :
Integrable (fun z => (uDegenOrder h P z) ^ 2) (Measure.pi fun _ : Fin m => P)
hslice_int :
∀ j : Fin m,
Integrable (fun x => ∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) P
hmean :
∀ j : Fin m,
∫ x, (∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) ∂P
hrow :
∀ (j : Fin m) (x : X),
Integrable (fun tail : ({k : Fin m // k ≠ j}) → X => h (insertCoord j x tail)) (Measure.pi fun _ : {k : Fin m // k ≠ j} => P)
hψ_meas :
hψ_mean :
∫ x, uInfluenceOrder h P x ∂P = 0
hψ_sq :
Integrable (fun x => (uInfluenceOrder h P x) ^ 2) P
hθn_meas :
∀ n : ℕ,
AEMeasurable (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r) n) μ
Tendsto_dist (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r)) (gaussianMeasure 0 (∫ x, (uInfluenceOrder h P x) ^ 2 ∂P)) μ hθn_meas
Proof (Lean source)
theorem uStatisticOrder_clt_of_regular {Ω X : Type*} [MeasurableSpace Ω] [MeasurableSpace X] {μ : Measure Ω} {P : Measure X} [IsProbabilityMeasure μ] (S : IIDSample Ω X μ P) {m : ℕ} [NeZero m] (h : (Fin m → X) → ℝ) (hmeas : Measurable (uDegenOrder h P)) (hL2 : Integrable (fun z => (uDegenOrder h P z) ^ 2) (Measure.pi fun _ : Fin m => P)) (hslice_int : ∀ j : Fin m, Integrable (fun x => ∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) P) (hmean : ∀ j : Fin m, ∫ x, (∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) ∂P = uMeanOrder h P) (hrow : ∀ (j : Fin m) (x : X), Integrable (fun tail : ({k : Fin m // k ≠ j}) → X => h (insertCoord j x tail)) (Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) (hψ_meas : Measurable (uInfluenceOrder h P)) (hψ_mean : ∫ x, uInfluenceOrder h P x ∂P = 0) (hψ_sq : Integrable (fun x => (uInfluenceOrder h P x) ^ 2) P) (hθn_meas : ∀ n : ℕ, AEMeasurable (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r) n) μ) : Tendsto_dist (IsAsymLinear.rescaledEstimator (uStatisticOrder S h) (uMeanOrder h P) (fun r => range r)) (gaussianMeasure 0 (∫ x, (uInfluenceOrder h P x) ^ 2 ∂P)) μ hθn_meas := by letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable have hneg : OrderDegenerateNegligible S h := orderDegenerateNegligible_of_residual S h hmeas hL2 hslice_int hmean hrow exact uStatisticOrder_clt S h hψ_meas hψ_mean hψ_sq hneg hθn_meas
Exact­Variance 3 core · 3 supporting This module proves the exact second-moment calculation for a completely degenerate fixed-order kernel. ★ integral_injectiveTuples_sum_sq_degen★ integral_rescaled_order_sq_degen

Exact variance for completely degenerate fixed-order U-statistics

This module proves the exact second-moment calculation for a completely degenerate fixed-order kernel. The cross-term lemmas IIDSample.crossterm_eq_zeta_of_image_eq and IIDSample.crossterm_eq_zero_of_image_ne classify pairs of injective tuples by whether their images agree; card_injectiveTuples_image_eq counts the same-image reorderings.

The headline variance identities are IIDSample.integral_injectiveTuples_sum_sq_degen for the raw injective-tuple sum and IIDSample.integral_rescaled_order_sq_degen for the √n-rescaled degenerate U-statistic. These sharpen the general rate bound to an exact formula in the completely degenerate case.

def permOfImageEq reviewed
Causalean.Stat.IIDSample

Given a nonnegative integer mm specifying the size of a source index set, a nonnegative integer nn specifying the size of a target index set, two maps from the source index set into the target index set, the assumption that both maps are injective, and the assumption that their images coincide, the selected permutation of the source index set reorders the first map into the second. It first selects, for each target value of the second map, a source index having the same first-map value, and then selects, for each target value of the first map, a source index having the same second-map value.

Definition (Lean source)
m :
shared
n :
Fin m → Fin n
ht :
hq :
himg :
Finset.univ.image t = Finset.univ.image q
permOfImageEq ht hq himg :
Perm (Fin m)
by classical let f : Fin m → Fin m := fun j
=> choose ((Finset.mem_image.mp (by have hqmem : q j ∈ Finset.univ.image q := Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩ simp [himg])) : ∃ i, i ∈ univ ∧ t i = q j) let r : Fin m → Fin m := fun i => choose ((Finset.mem_image.mp (by have htmem : t i ∈ Finset.univ.image t := Finset.mem_image.mpr ⟨i, Finset.mem_univ i, rfl⟩ simp [← himg])) : ∃ j, j ∈ univ ∧ q j = t i) have hf : ∀ j, t (f j) = q j := by intro j exact (Classical.choose_spec ((Finset.mem_image.mp (by have hqmem : q j ∈ Finset.univ.image q := Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩ simp [himg])) : ∃ i, i ∈ univ ∧ t i = q j)).2 have hr : ∀ i, q (r i) = t i := by intro i exact (Classical.choose_spec ((Finset.mem_image.mp (by have htmem : t i ∈ Finset.univ.image t := Finset.mem_image.mpr ⟨i, Finset.mem_univ i, rfl⟩ simp [← himg])) : ∃ j, j ∈ univ ∧ q j = t i)).2 exact { toFun := f invFun := r left_inv := by intro j apply hq rw [hr (f j), hf j] right_inv := by intro i apply ht rw [hf (r i), hr i] }
Causalean.Stat.IIDSample.permOfImageEq · Causalean/Stat/UStatistic/OrderM/ExactVariance.lean:69
theorem integral_injectiveTuples_sum_sq_degen reviewed
Causalean.Stat.IIDSample

Exact second moment of the injective-tuple sum. For an i.i.d. sample S and sample size n, if the order-m kernel g is completely degenerate: symmetric, square-integrable, and with zero conditional mean after integrating out any single coordinate, then the second moment of the sum of g over all ordered injective m-tuples drawn from the first n sample indices equals m! · n^{(m)} · ζ_m, where n^{(m)} is the number of such tuples and ζ_m is the kernel's second moment under the m-fold product law.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
hg :
n :
∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ
= (m.factorial : ℝ) * injectiveTupleCount m n * zetaOrder P g
Proof (Lean source)
theorem integral_injectiveTuples_sum_sq_degen (hg : OrderDegenKernel P g) (n : ℕ) : ∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ = (m.factorial : ℝ) * injectiveTupleCount m n * zetaOrder P g := by letI : IsProbabilityMeasure μ := ⟨by calc μ univ = μ (S.Z 0 ⁻¹' univ) := by simp _ = (μ.map (S.Z 0)) univ := (Measure.map_apply (S.meas 0) MeasurableSet.univ).symm _ = P univ := by rw [S.law] _ = 1 := measure_univ⟩ letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable classical let T : Finset (Fin m → Fin n) := injectiveTuples m n let F : (Fin m → Fin n) → (Fin m → Fin n) → ℝ := fun t q => ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ have hinj_of_mem_T : ∀ t ∈ T, Injective t := by intro t ht have ht' : t ∈ injectiveTuples m n := by simpa [T] using ht exact (Finset.mem_filter.mp ht').2 have hexpand : (fun ω => (∑ t ∈ T, g (fun j => S.Z (t j : ℕ) ω)) ^ 2) = (fun ω => ∑ t ∈ T, ∑ q ∈ T, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) := by funext ω rw [sq, Finset.sum_mul_sum] rw [show (∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ) = ∫ ω, (∑ t ∈ T, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ by rfl] rw [hexpand] have hterm_int : ∀ t ∈ T, Integrable (fun ω => ∑ q ∈ T, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) μ := fun t ht => by apply integrable_finset_sum intro q hq exact S.integrable_orderTerm_mul hg.meas hg.sq (hinj_of_mem_T t ht) (hinj_of_mem_T q hq) integral_linearity have hpush : ∀ t ∈ T, ∫ ω, ∑ q ∈ T, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = ∑ q ∈ T, F t q := by intro t ht exact integral_finset_sum _ (fun q hq => S.integrable_orderTerm_mul hg.meas hg.sq (hinj_of_mem_T t ht) (hinj_of_mem_T q hq)) rw [Finset.sum_congr rfl hpush] have hinner : ∀ t ∈ T, ∑ q ∈ T, F t q = (m.factorial : ℝ) * zetaOrder P g := by intro t ht let SAME : Finset (Fin m → Fin n) := T.filter (fun q => Finset.univ.image q = Finset.univ.image t) have hsub : SAME ⊆ T := Finset.filter_subset _ _ have hzero : ∀ q ∈ T, q ∉ SAME → F t q = 0 := by intro q hq hqnot have htinj := hinj_of_mem_T t ht have hqinj := hinj_of_mem_T q hq have hne : Finset.univ.image t ≠ Finset.univ.image q := by intro h exact hqnot (Finset.mem_filter.mpr ⟨hq, h.symm⟩) simpa [F] using S.crossterm_eq_zero_of_image_ne hg.meas hg.sq hg.deg htinj hqinj hne rw [← Finset.sum_subset hsub hzero] have hsame : ∀ q ∈ SAME, F t q = zetaOrder P g := by intro q hq have htinj := hinj_of_mem_T t ht have hqT : q ∈ T := (Finset.mem_filter.mp hq).1 have hqinj := hinj_of_mem_T q hqT have himg : Finset.univ.image t = Finset.univ.image q := (Finset.mem_filter.mp hq).2.symm simpa [F] using S.crossterm_eq_zeta_of_image_eq hg.meas hg.symm htinj hqinj himg rw [Finset.sum_congr rfl hsame] rw [Finset.sum_const, nsmul_eq_mul] have htinj := hinj_of_mem_T t ht have hcard : SAME.card = m.factorial := by simpa [SAME, T] using card_injectiveTuples_image_eq (m := m) (n := n) htinj rw [hcard] rw [Finset.sum_congr rfl hinner] rw [Finset.sum_const, nsmul_eq_mul] simp only [T, injectiveTupleCount] ring
theorem integral_rescaled_order_sq_degen reviewed
Causalean.Stat.IIDSample

Exact variance of the rescaled degenerate fixed-order U-statistic. For an i.i.d. sample S, if the order-m kernel g is completely degenerate and the sample size is at least m, then the second moment of the √n-rescaled order-m U-statistic of g equals n · m! · ζ_m / n^{(m)}, which specializes to 2ζ/(n−1) when m = 2.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
hg :
n :
hmn :
m ≤ n
∫ ω, (sqrt (n : ℝ) * uStatisticOrder S g n ω) ^ 2 ∂μ
= (n : ℝ) * (m.factorial : ℝ) * zetaOrder P g / injectiveTupleCount m n
Proof (Lean source)
theorem integral_rescaled_order_sq_degen (hg : OrderDegenKernel P g) {n : ℕ} (hmn : m ≤ n) : ∫ ω, (sqrt (n : ℝ) * uStatisticOrder S g n ω) ^ 2 ∂μ = (n : ℝ) * (m.factorial : ℝ) * zetaOrder P g / injectiveTupleCount m n := by letI : IsProbabilityMeasure μ := ⟨by calc μ univ = μ (S.Z 0 ⁻¹' univ) := by simp _ = (μ.map (S.Z 0)) univ := (Measure.map_apply (S.meas 0) MeasurableSet.univ).symm _ = P univ := by rw [S.law] _ = 1 := measure_univ⟩ letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable classical let K : Ω → ℝ := fun ω => ∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω) have hcount_ne : injectiveTupleCount m n ≠ 0 := injectiveTupleCount_ne_zero hmn have hnnonneg : 0 ≤ (n : ℝ) := by positivity have hpoint : (fun ω => (sqrt (n : ℝ) * uStatisticOrder S g n ω) ^ 2) = (fun ω => ((n : ℝ) * (injectiveTupleCount m n)⁻¹ ^ 2) * (K ω) ^ 2) := by funext ω simp only [uStatisticOrder, K] rw [mul_pow, Real.sq_sqrt hnnonneg] ring rw [hpoint] integral_linearity rw [show (∫ ω, (K ω) ^ 2 ∂μ) = ∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ by rfl] rw [S.integral_injectiveTuples_sum_sq_degen hg n] field_simp [hcount_ne]
3 supporting declarations (lemmas, instances)
  • crossterm_eq_zeta_of_image_eq theorem — Equal-image cross term. If two ordered injective m-tuples have the same image, then q is a reordering of t, so by symmetry g(Z_q) = g(Z_t) and the cross expectation is ζ_m = zetaOrder P g.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsymm :
    ∀ σ : Perm (Fin m), ∀ z, g (z ∘ σ) = g z
    n :
    Fin m → Fin n
    ht :
    hq :
    himg :
    Finset.univ.image t = Finset.univ.image q
    ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = zetaOrder P g
    Proof (Lean source)
    theorem crossterm_eq_zeta_of_image_eq (hmeas : Measurable g) (hsymm : ∀ σ : Perm (Fin m), ∀ z, g (z ∘ σ) = g z) {n : ℕ} {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) (himg : Finset.univ.image t = Finset.univ.image q) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = zetaOrder P g := by classical let σ : Perm (Fin m) := permOfImageEq (m := m) ht hq himg have hq_rewrite : ∀ ω, (fun j => S.Z (q j : ℕ) ω) = (fun j => S.Z (t j : ℕ) ω) ∘ σ := by intro ω funext j rw [Function.comp_apply] have hσ := permOfImageEq_apply (m := m) ht hq himg j rw [← hσ] have hcongr : (fun ω => g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) = (fun ω => (g (fun j => S.Z (t j : ℕ) ω)) ^ 2) := by funext ω rw [hq_rewrite ω, hsymm σ] ring rw [hcongr] rw [zetaOrder, ← S.map_tuple_eq ht] rw [integral_map (measurable_pi_lambda _ (fun j : Fin m => S.meas (t j : ℕ))).aemeasurable (hmeas.pow_const 2).aestronglyMeasurable]
    Causalean.Stat.IIDSample.crossterm_eq_zeta_of_image_eq · Causalean/Stat/UStatistic/OrderM/ExactVariance.lean:128
  • crossterm_eq_zero_of_image_ne theorem — Distinct-image cross term. If two ordered injective m-tuples have different images, complete degeneracy kills the cross expectation: some index of t is absent from q, and integrating that coordinate out gives 0.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsq :
    Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
    hdeg :
    ∀ (j : Fin m) (tail : ({k : Fin m // k ≠ j}) → X), ∫ x, g (insertCoord j x tail) ∂P = 0
    n :
    Fin m → Fin n
    ht :
    hq :
    himg :
    Finset.univ.image t ≠ Finset.univ.image q
    ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0
    Proof (Lean source)
    theorem crossterm_eq_zero_of_image_ne (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) (hdeg : ∀ (j : Fin m) (tail : ({k : Fin m // k ≠ j}) → X), ∫ x, g (insertCoord j x tail) ∂P = 0) {n : ℕ} {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) (himg : Finset.univ.image t ≠ Finset.univ.image q) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0 := by letI : IsProbabilityMeasure μ := ⟨by calc μ univ = μ (S.Z 0 ⁻¹' univ) := by simp _ = (μ.map (S.Z 0)) univ := (Measure.map_apply (S.meas 0) MeasurableSet.univ).symm _ = P univ := by rw [S.law] _ = 1 := measure_univ⟩ classical let A : Finset (Fin n) := Finset.univ.image t let B : Finset (Fin n) := Finset.univ.image q have hAcard : A.card = m := by simpa [A] using (Finset.card_image_of_injective (s := (Finset.univ : Finset (Fin m))) ht) have hBcard : B.card = m := by simpa [B] using (Finset.card_image_of_injective (s := (Finset.univ : Finset (Fin m))) hq) have hnot_subset : ¬ A ⊆ B := by intro hsub apply himg have hcard_le : B.card ≤ A.card := by rw [hAcard, hBcard] simpa [A, B] using (Finset.eq_of_subset_of_card_le hsub hcard_le) rcases Finset.not_subset.mp hnot_subset with ⟨a, haA, haB⟩ rcases Finset.mem_image.mp haA with ⟨p, _hp, htp⟩ subst a let R : Finset (Fin n) := (A ∪ B).erase (t p) let XR : Ω → (R → X) := fun ω i => S.Z (i.1 : ℕ) ω let πR : Measure (R → X) := Measure.pi fun _ : R => P let tailOf : (R → X) → ({k : Fin m // k ≠ p} → X) := fun xr k => xr ⟨t k.1, by have hmemA : t k.1 ∈ A := Finset.mem_image.mpr ⟨k.1, Finset.mem_univ k.1, rfl⟩ have hne : t k.1 ≠ t p := fun h => k.2 (ht h) simp [R, hmemA, hne]⟩ let Ψ : (R → X) → ℝ := fun xr => g (fun j => xr ⟨q j, by have hmemB : q j ∈ B := Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩ have hne : q j ≠ t p := by intro h exact haB (by simpa [B, h] using hmemB) simp [R, hmemB, hne]⟩) let F : X × (R → X) → ℝ := fun z => g (insertCoord p z.1 (tailOf z.2)) * Ψ z.2 have hXRmeas : Measurable XR := by fun_prop have hX0meas : Measurable (fun ω : Ω => S.Z (t p : ℕ) ω) := S.meas (t p : ℕ) have hPairMeas : Measurable (fun ω : Ω => (S.Z (t p : ℕ) ω, XR ω)) := hX0meas.prodMk hXRmeas have hXRmap : μ.map XR = πR := by have hr : Injective (fun i : R => i.1) := by intro i j hij exact Subtype.ext hij simpa [XR, πR, R] using (S.map_fintype_tuple_eq (ι := R) (r := fun i : R => i.1) hr) have hdisj : Disjoint ({t p} : Finset (Fin n)) R := by rw [Finset.disjoint_iff_inter_eq_empty] ext x constructor · intro hx rcases Finset.mem_inter.mp hx with ⟨hx0, hxR⟩ have hxeq : x = t p := by simpa using hx0 subst x simp [R] at hxR · intro hx simp at hx have hindFin : iIndepFun (fun i : Fin n => S.Z (i : ℕ)) μ := S.indep.precomp (fun _ _ h => Fin.ext h) have hindBlocks : IndepFun (fun ω : Ω => fun i : ({t p} : Finset (Fin n)) => S.Z (i.1 : ℕ) ω) XR μ := by simpa [XR] using (ProbabilityTheory.iIndepFun.indepFun_finset ({t p} : Finset (Fin n)) R hdisj hindFin (fun i : Fin n => S.meas (i : ℕ))) have hEval : Measurable (fun x : ({t p} : Finset (Fin n)) → X => x ⟨t p, by simp⟩) := measurable_pi_apply _ have hind : IndepFun (fun ω : Ω => S.Z (t p : ℕ) ω) XR μ := by have hcomp := hindBlocks.comp hEval measurable_id simpa [Function.comp_def] using hcomp have hΦ : Measurable (fun z : X × (R → X) => g (insertCoord p z.1 (tailOf z.2))) := by exact hmeas.comp (measurable_pi_lambda _ (fun j : Fin m => by by_cases hj : j = p · subst j simpa [insertCoord] using measurable_fst · simpa [tailOf, insertCoord, hj, Function.comp_def] using (measurable_pi_apply (⟨t j, by have hmemA : t j ∈ A := Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩ have hne : t j ≠ t p := fun h => hj (ht h) simp [R, hmemA, hne]⟩ : R)).comp measurable_snd)) have hΨ : Measurable Ψ := by exact hmeas.comp (measurable_pi_lambda _ (fun j : Fin m => measurable_pi_apply _)) have hFmeas : Measurable F := hΦ.mul (hΨ.comp measurable_snd) have ht_rewrite : ∀ ω, g (fun j => S.Z (t j : ℕ) ω) = g (insertCoord p (S.Z (t p : ℕ) ω) (tailOf (XR ω))) := by intro ω congr 1 funext j by_cases hj : j = p · subst j simp [insertCoord] · simp [tailOf, XR, insertCoord, hj] have hq_rewrite : ∀ ω, g (fun j => S.Z (q j : ℕ) ω) = Ψ (XR ω) := by intro ω rfl have hcomp_int : Integrable (fun ω => F (S.Z (t p : ℕ) ω, XR ω)) μ := by have hterm_mem : ∀ {r : Fin m → Fin n}, Injective r → MemLp (fun ω => g (fun j => S.Z (r j : ℕ) ω)) 2 μ := by intro r hr have hm : AEStronglyMeasurable (fun ω => g (fun j => S.Z (r j : ℕ) ω)) μ := (hmeas.comp (measurable_pi_lambda _ (fun j : Fin m => S.meas (r j : ℕ)))).aestronglyMeasurable apply (memLp_two_iff_integrable_sq hm).mpr have hmap : Integrable (fun z => (g z) ^ 2) (μ.map (fun ω : Ω => fun j : Fin m => S.Z (r j : ℕ) ω)) := by rw [S.map_tuple_eq hr] exact hsq exact (integrable_map_measure (hmeas.pow_const 2).aestronglyMeasurable (measurable_pi_lambda _ (fun j : Fin m => S.meas (r j : ℕ))).aemeasurable).mp hmap have horig := (hterm_mem ht).integrable_mul (hterm_mem hq) refine horig.congr ?_ filter_upwards with ω simp [F, ht_rewrite ω, hq_rewrite ω] have hmap_pair : μ.map (fun ω : Ω => (S.Z (t p : ℕ) ω, XR ω)) = P.prod (μ.map XR) := by have h := (indepFun_iff_map_prod_eq_prod_map_map hX0meas.aemeasurable hXRmeas.aemeasurable).mp hind simpa [S.map_eq (t p : ℕ)] using h have hFint_map : Integrable F (P.prod (μ.map XR)) := by have hmap_int : Integrable F (μ.map fun ω : Ω => (S.Z (t p : ℕ) ω, XR ω)) := (integrable_map_measure hFmeas.aestronglyMeasurable hPairMeas.aemeasurable).mpr hcomp_int simpa [hmap_pair] using hmap_int have hinner : ∀ xr : R → X, (∫ x : X, F (x, xr) ∂P) = 0 := by intro xr change (∫ x : X, g (insertCoord p x (tailOf xr)) * Ψ xr ∂P) = 0 integral_linearity rw [hdeg p (tailOf xr), zero_mul] calc ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = ∫ ω, F (S.Z (t p : ℕ) ω, XR ω) ∂μ := by apply integral_congr_ae filter_upwards with ω simp [F, ht_rewrite ω, hq_rewrite ω] _ = ∫ z, F z ∂(μ.map fun ω : Ω => (S.Z (t p : ℕ) ω, XR ω)) := by rw [integral_map hPairMeas.aemeasurable hFmeas.aestronglyMeasurable] _ = ∫ z, F z ∂(P.prod (μ.map XR)) := by rw [hmap_pair] _ = ∫ xr, ∫ x, F (x, xr) ∂P ∂(μ.map XR) := by rw [integral_prod_symm F hFint_map] _ = 0 := by rw [show (fun xr : R → X => ∫ x, F (x, xr) ∂P) = fun _ => 0 by funext xr exact hinner xr] simp
    Causalean.Stat.IIDSample.crossterm_eq_zero_of_image_ne · Causalean/Stat/UStatistic/OrderM/ExactVariance.lean:161
  • card_injectiveTuples_image_eq theorem — For an ordered injective m-tuple t, the ordered injective tuples with the same image are exactly its m! reorderings.
    m :
    shared
    n :
    Fin m → Fin n
    ht :
    ((injectiveTuples m n).filter (fun q => Finset.univ.image q = Finset.univ.image t)).card
    = m.factorial
    Proof (Lean source)
    theorem card_injectiveTuples_image_eq {n : ℕ} {t : Fin m → Fin n} (ht : Injective t) : ((injectiveTuples m n).filter (fun q => Finset.univ.image q = Finset.univ.image t)).card = m.factorial := by classical have hcard_perm : (Finset.univ : Finset (Perm (Fin m))).card = m.factorial := by rw [Finset.card_univ, Fintype.card_perm, Fintype.card_fin] rw [← hcard_perm] symm refine Finset.card_bij (fun σ _ => t ∘ σ) ?hmem ?hinj ?hsurj · intro σ _hσ rw [mem_filter] refine ⟨?_, ?_⟩ · simp [injectiveTuples, ht.comp σ.injective] · ext x constructor · intro hx rcases Finset.mem_image.mp hx with ⟨j, _hj, hjx⟩ exact Finset.mem_image.mpr ⟨σ j, Finset.mem_univ _, hjx⟩ · intro hx rcases Finset.mem_image.mp hx with ⟨j, _hj, hjx⟩ exact Finset.mem_image.mpr ⟨σ.symm j, Finset.mem_univ _, by simp [comp, hjx]⟩ · intro σ₁ _ σ₂ _ hσ ext j exact congrArg val (ht (congrFun hσ j)) · intro q hq rw [mem_filter] at hq have hqinj : Injective q := by simpa [injectiveTuples] using hq.1 let σ : Perm (Fin m) := permOfImageEq (m := m) ht hqinj hq.2.symm refine ⟨σ, Finset.mem_univ σ, ?_⟩ funext j exact permOfImageEq_apply (m := m) ht hqinj hq.2.symm j
    Causalean.Stat.IIDSample.card_injectiveTuples_image_eq · Causalean/Stat/UStatistic/OrderM/ExactVariance.lean:335
Mixed­Order­Bounds 1 core · 3 supporting This module supplies uniform bounds for overlap normalizations and for the size-zero disjoint correction. ★ factorialMatchingRatio_le

Mixed-order falling-factorial normalization bounds

This module supplies uniform bounds for overlap normalizations and for the size-zero disjoint correction. The statements are symmetric in the two orders and depend only on a common upper bound R.

theorem factorialMatchingRatio_le reviewed
Causalean.Stat

For sample size n, orders r and s, common order bound R, and overlap size h, if the first order is at most R, the second order is at most R, the sample size is at least four times R squared, and the overlap fits inside both orders, the mixed falling-factorial ratio is at most exp(1) divided by n to the overlap size.

Formal statement
n r s R h :
hr :
r ≤ R
hs :
s ≤ R
hn :
4 * R ^ 2 ≤ n
hh :
h ≤ min r s
(n.descFactorial (r + s - h) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ))
exp 1 / (n : ℝ) ^ h
Proof (Lean source)
theorem factorialMatchingRatio_le {n r s R h : ℕ} (hr : r ≤ R) (hs : s ≤ R) (hn : 4 * R ^ 2 ≤ n) (hh : h ≤ min r s) : (n.descFactorial (r + s - h) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ)) ≤ exp 1 / (n : ℝ) ^ h := by rcases le_total r s with hrs | hsr · exact orderedFactorialMatchingRatio_le hrs (by nlinarith) (hh.trans (min_le_right _ _)) · simpa [Nat.add_comm, mul_comm] using (orderedFactorialMatchingRatio_le hsr (by nlinarith) (hh.trans (min_le_left _ _)))
Causalean.Stat.factorialMatchingRatio_le · Causalean/Stat/UStatistic/OrderM/MixedOrderBounds.lean:217
3 supporting declarations (lemmas, instances)
  • matchingNormalization_le theorem — For sample size n, orders r and s, common order bound R, and overlap size h, a partial matching has normalization at most exp(1) divided by n to the overlap size when the first order is at most R, the second order is at most R, the sample size is at least four times R squared, and the matching has size h as claimed.
    n r s R h :
    hr :
    r ≤ R
    hs :
    s ≤ R
    hn :
    4 * R ^ 2 ≤ n
    hM :
    matchingNormalization n M ≤ exp 1 / (n : ℝ) ^ h
    Proof (Lean source)
    theorem matchingNormalization_le {n r s R h : ℕ} {M : PartialMatching r s} (hr : r ≤ R) (hs : s ≤ R) (hn : 4 * R ^ 2 ≤ n) (hM : M ∈ partialMatchingsOfSize r s h) : matchingNormalization n M ≤ exp 1 / (n : ℝ) ^ h := by have hh : h ≤ min r s := by simpa [(mem_partialMatchingsOfSize M).mp hM] using M.size_le_min rw [matchingNormalization_of_mem hM] exact factorialMatchingRatio_le hr hs hn hh
    Causalean.Stat.matchingNormalization_le · Causalean/Stat/UStatistic/OrderM/MixedOrderBounds.lean:234
  • factorialDisjointCorrection_le theorem — For sample size n, orders r and s, and common order bound R, if the first order is at most R, the second order is at most R, and the sample size is at least four times R squared, the absolute disjoint normalization correction is at most twice R squared divided by n.
    n r s R :
    hr :
    r ≤ R
    hs :
    s ≤ R
    hn :
    4 * R ^ 2 ≤ n
    |(n.descFactorial (r + s) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ)) - 1|
    ≤ 2 * (R : ℝ) ^ 2 / n
    Proof (Lean source)
    theorem factorialDisjointCorrection_le {n r s R : ℕ} (hr : r ≤ R) (hs : s ≤ R) (hn : 4 * R ^ 2 ≤ n) : |(n.descFactorial (r + s) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ)) - 1| ≤ 2 * (R : ℝ) ^ 2 / n := by rcases le_total r s with hrs | hsr · calc |(n.descFactorial (r + s) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ)) - 1| ≤ 2 * (s : ℝ) ^ 2 / n := orderedFactorialDisjointCorrection_le hrs (by nlinarith) _ ≤ 2 * (R : ℝ) ^ 2 / n := by have hnum : 2 * s ^ 2 ≤ 2 * R ^ 2 := Nat.mul_le_mul_left 2 (Nat.pow_le_pow_left hs 2) apply div_le_div_of_nonneg_right (by exact_mod_cast hnum) (by positivity) · calc |(n.descFactorial (r + s) : ℝ) / ((n.descFactorial r : ℝ) * (n.descFactorial s : ℝ)) - 1| = |(n.descFactorial (s + r) : ℝ) / ((n.descFactorial s : ℝ) * (n.descFactorial r : ℝ)) - 1| := by simp [Nat.add_comm, mul_comm] _ ≤ 2 * (r : ℝ) ^ 2 / n := orderedFactorialDisjointCorrection_le hsr (by nlinarith) _ ≤ 2 * (R : ℝ) ^ 2 / n := by have hnum : 2 * r ^ 2 ≤ 2 * R ^ 2 := Nat.mul_le_mul_left 2 (Nat.pow_le_pow_left hr 2) apply div_le_div_of_nonneg_right (by exact_mod_cast hnum) (by positivity)
    Causalean.Stat.factorialDisjointCorrection_le · Causalean/Stat/UStatistic/OrderM/MixedOrderBounds.lean:249
  • emptyMatchingNormalization_sub_one_le theorem — For sample size n, orders r and s, and common order bound R, if the first order is at most R, the second order is at most R, and the sample size is at least four times R squared, the empty partial matching's normalization differs from one by at most twice R squared divided by n.
    n r s R :
    hr :
    r ≤ R
    hs :
    s ≤ R
    hn :
    4 * R ^ 2 ≤ n
    |matchingNormalization n (PartialMatching.empty r s) - 1| ≤ 2 * (R : ℝ) ^ 2 / n
    Proof (Lean source)
    theorem emptyMatchingNormalization_sub_one_le {n r s R : ℕ} (hr : r ≤ R) (hs : s ≤ R) (hn : 4 * R ^ 2 ≤ n) : |matchingNormalization n (PartialMatching.empty r s) - 1| ≤ 2 * (R : ℝ) ^ 2 / n := by rw [matchingNormalization] simp only [PartialMatching.empty_size, Nat.sub_zero] exact factorialDisjointCorrection_le hr hs hn
    Causalean.Stat.emptyMatchingNormalization_sub_one_le · Causalean/Stat/UStatistic/OrderM/MixedOrderBounds.lean:280
Order­Two 1 core · 4 supporting Specializes the fixed-order U-statistic CLT to the order-2 theory. ★ uStatistic_clt_of_symmetric_via_orderM

Specializes the fixed-order U-statistic CLT to the order-2 theory.

The bridge sends a two-argument kernel h : X → X → ℝ to `pairKernel h : (Fin 2 → X) → ℝ and proves that the order-m` mean, projections, influence function, degenerate residual, and statistic agree with their order-2 counterparts. The final theorem, uStatistic_clt_of_symmetric_via_orderM, derives the classical symmetric-kernel order-2 CLT from uStatisticOrder_clt_of_regular.

theorem uStatistic_clt_of_symmetric_via_orderM reviewed
Causalean.Stat

Order-2 U-statistic CLT, derived from the fixed-order-m CLT. For an i.i.d. sample S and a two-argument kernel h : X → X → ℝ that is symmetric and jointly measurable, suppose the order-2 Hájek residual of h is square-integrable under the product law P × P, the row integral x ↦ ∫h(x,y)dP(y) is integrable and each row y ↦ h(x,y) is itself integrable for every x, the first Hoeffding projection of h is square-integrable, and the √n-rescaled U-statistic is almost-everywhere measurable at every sample size. Then the √n-rescaled order-2 U-statistic converges in distribution to the centered Gaussian law with variance 4ζ₁, where ζ₁ is the variance of the first Hoeffding projection.

Formal statement
S :
IIDSample Ω X μ P
h :
X → X → ℝ
hsymm :
∀ x y, h x y = h y x
hmeas :
Measurable fun p : X × X => h p.1 p.2
hL2 :
Integrable (fun p : X × X => (uDegen h P p.1 p.2) ^ 2) (P.prod P)
hint :
Integrable (fun x => ∫ y, h x y ∂P) P
hrow :
∀ x, Integrable (fun y => h x y) P
hproj_sq :
Integrable (fun x => (uProj h P x) ^ 2) P
hθn_meas :
∀ n : ℕ,
AEMeasurable (IsAsymLinear.rescaledEstimator (uStatistic S h) (uMean h P) (fun m => range m) n) μ
Proof (Lean source)
theorem uStatistic_clt_of_symmetric_via_orderM {Ω X : Type*} [MeasurableSpace Ω] [MeasurableSpace X] {μ : Measure Ω} {P : Measure X} (S : IIDSample Ω X μ P) (h : X → X → ℝ) (hsymm : ∀ x y, h x y = h y x) (hmeas : Measurable fun p : X × X => h p.1 p.2) (hL2 : Integrable (fun p : X × X => (uDegen h P p.1 p.2) ^ 2) (P.prod P)) (hint : Integrable (fun x => ∫ y, h x y ∂P) P) (hrow : ∀ x, Integrable (fun y => h x y) P) (hproj_sq : Integrable (fun x => (uProj h P x) ^ 2) P) (hθn_meas : ∀ n : ℕ, AEMeasurable (IsAsymLinear.rescaledEstimator (uStatistic S h) (uMean h P) (fun m => range m) n) μ) : letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure Tendsto_dist (IsAsymLinear.rescaledEstimator (uStatistic S h) (uMean h P) (fun m => range m)) (gaussianMeasure 0 (∫ x, ((fun x => 2 * uProj h P x) x) ^ 2 ∂P)) μ hθn_meas := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable let e := MeasurableEquiv.piFinTwo (fun _ : Fin 2 => X) have hmp : MeasurePreserving e (Measure.pi fun _ : Fin 2 => P) (P.prod P) := by simpa [e] using (measurePreserving_piFinTwo (fun _ : Fin 2 => P)) have hproj_meas : Measurable (uProj h P) := by have hsm : StronglyMeasurable fun x => ∫ y, h x y ∂P := hmeas.stronglyMeasurable.integral_prod_right' exact hsm.measurable.sub measurable_const have hdeg_meas_prod : Measurable (fun p : X × X => uDegen h P p.1 p.2) := by have h1 : Measurable fun p : X × X => uProj h P p.1 := hproj_meas.comp measurable_fst have h2 : Measurable fun p : X × X => uProj h P p.2 := hproj_meas.comp measurable_snd simp only [uDegen] exact ((hmeas.sub measurable_const).sub h1).sub h2 have hproj_int : Integrable (uProj h P) P := uProj_integrable hint have hdegen_int : Integrable (fun p : X × X => uDegen h P p.1 p.2) (P.prod P) := ((memLp_two_iff_integrable_sq hdeg_meas_prod.aestronglyMeasurable).mpr hL2).integrable (by norm_num) have hh_int : Integrable (fun p : X × X => h p.1 p.2) (P.prod P) := by have hsum_int : Integrable (fun p : X × X => uMean h P + uProj h P p.1 + uProj h P p.2 + uDegen h P p.1 p.2) (P.prod P) := by have hfst : Integrable (fun p : X × X => uProj h P p.1) (P.prod P) := hproj_int.comp_fst P have hsnd : Integrable (fun p : X × X => uProj h P p.2) (P.prod P) := hproj_int.comp_snd P exact (((integrable_const (uMean h P)).add hfst).add hsnd).add hdegen_int have hfun : (fun p : X × X => uMean h P + uProj h P p.1 + uProj h P p.2 + uDegen h P p.1 p.2) = fun p => h p.1 p.2 := by funext p exact (hoeffding_decomp h P p.1 p.2).symm simpa [hfun] using hsum_int have hmean_bridge : uMeanOrder (pairKernel h) P = uMean h P := uMeanOrder_pairKernel (P := P) (h := h) hh_int have hproj_bridge : ∀ j : Fin 2, uProjOrderAt j (pairKernel h) P = uProj h P := fun j => uProjOrderAt_pairKernel_of_symm (P := P) (h := h) hsymm hh_int hrow j have hψ_bridge : uInfluenceOrder (pairKernel h) P = fun x => 2 * uProj h P x := uInfluenceOrder_pairKernel_of_symm (P := P) (h := h) hsymm hh_int hrow have hstat_bridge : uStatisticOrder S (pairKernel h) = uStatistic S h := by funext n exact uStatisticOrder_two_eq_uStatistic S h n have hdeg_bridge : uDegenOrder (pairKernel h) P = fun z => uDegen h P (z 0) (z 1) := uDegenOrder_pairKernel_of_symm (P := P) (h := h) hsymm hh_int hrow have hmeas' : Measurable (uDegenOrder (pairKernel h) P) := by have hraw : Measurable (fun z : Fin 2 → X => uDegen h P (z 0) (z 1)) := by exact hdeg_meas_prod.comp e.measurable simpa [hdeg_bridge] using hraw have hL2' : Integrable (fun z => (uDegenOrder (pairKernel h) P z) ^ 2) (Measure.pi fun _ : Fin 2 => P) := by have hraw : Integrable (fun z : Fin 2 → X => (uDegen h P (z 0) (z 1)) ^ 2) (Measure.pi fun _ : Fin 2 => P) := by exact hmp.integrable_comp_of_integrable hL2 simpa [hdeg_bridge] using hraw have hslice_int' : ∀ j : Fin 2, Integrable (fun x => ∫ tail : ({k : Fin 2 // k ≠ j}) → X, pairKernel h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin 2 // k ≠ j} => P)) P := by intro j fin_cases j · change Integrable (fun x => ∫ tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X, pairKernel h (insertCoord (0 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (0 : Fin 2)} => P)) P have hfun : (fun x => ∫ tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X, pairKernel h (insertCoord (0 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (0 : Fin 2)} => P)) = fun x => ∫ y, h x y ∂P := by funext x let a : {k : Fin 2 // ¬ k = (0 : Fin 2)} := ⟨1, by norm_num⟩ have h_eval := integral_pi_eval_eq (P := P) (i := a) (f := fun y => h x y) (hrow x) have htail : (fun tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X => pairKernel h (insertCoord (0 : Fin 2) x tail)) = fun tail => h x (tail a) := by funext tail simp [pairKernel, insertCoord, a] rw [htail, h_eval] simpa [hfun] using hint · change Integrable (fun x => ∫ tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X, pairKernel h (insertCoord (1 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (1 : Fin 2)} => P)) P have hfun : (fun x => ∫ tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X, pairKernel h (insertCoord (1 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (1 : Fin 2)} => P)) = fun x => ∫ y, h x y ∂P := by funext x let a : {k : Fin 2 // ¬ k = (1 : Fin 2)} := ⟨0, by norm_num⟩ have hcol : Integrable (fun y => h y x) P := by have hfun_col : (fun y => h y x) = fun y => h x y := by funext y exact hsymm y x rw [hfun_col] exact hrow x have h_eval := integral_pi_eval_eq (P := P) (i := a) (f := fun y => h y x) hcol have hswap : (∫ y, h y x ∂P) = ∫ y, h x y ∂P := by congr 1 funext y exact hsymm y x have htail : (fun tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X => pairKernel h (insertCoord (1 : Fin 2) x tail)) = fun tail => h (tail a) x := by funext tail simp [pairKernel, insertCoord, a] rw [htail, h_eval, hswap] simpa [hfun] using hint have hmean' : ∀ j : Fin 2, ∫ x, (∫ tail : ({k : Fin 2 // k ≠ j}) → X, pairKernel h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin 2 // k ≠ j} => P)) ∂P = uMeanOrder (pairKernel h) P := by intro j fin_cases j · change ∫ x, (∫ tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X, pairKernel h (insertCoord (0 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (0 : Fin 2)} => P)) ∂P = uMeanOrder (pairKernel h) P have hfun : (fun x => ∫ tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X, pairKernel h (insertCoord (0 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (0 : Fin 2)} => P)) = fun x => ∫ y, h x y ∂P := by funext x let a : {k : Fin 2 // ¬ k = (0 : Fin 2)} := ⟨1, by norm_num⟩ have h_eval := integral_pi_eval_eq (P := P) (i := a) (f := fun y => h x y) (hrow x) have htail : (fun tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X => pairKernel h (insertCoord (0 : Fin 2) x tail)) = fun tail => h x (tail a) := by funext tail simp [pairKernel, insertCoord, a] rw [htail, h_eval] rw [hfun, hmean_bridge] rfl · change ∫ x, (∫ tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X, pairKernel h (insertCoord (1 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (1 : Fin 2)} => P)) ∂P = uMeanOrder (pairKernel h) P have hfun : (fun x => ∫ tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X, pairKernel h (insertCoord (1 : Fin 2) x tail) ∂(Measure.pi fun _ : {k : Fin 2 // ¬ k = (1 : Fin 2)} => P)) = fun x => ∫ y, h x y ∂P := by funext x let a : {k : Fin 2 // ¬ k = (1 : Fin 2)} := ⟨0, by norm_num⟩ have hcol : Integrable (fun y => h y x) P := by have hfun_col : (fun y => h y x) = fun y => h x y := by funext y exact hsymm y x rw [hfun_col] exact hrow x have h_eval := integral_pi_eval_eq (P := P) (i := a) (f := fun y => h y x) hcol have hswap : (∫ y, h y x ∂P) = ∫ y, h x y ∂P := by congr 1 funext y exact hsymm y x have htail : (fun tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X => pairKernel h (insertCoord (1 : Fin 2) x tail)) = fun tail => h (tail a) x := by funext tail simp [pairKernel, insertCoord, a] rw [htail, h_eval, hswap] rw [hfun, hmean_bridge] rfl have hrow' : ∀ (j : Fin 2) (x : X), Integrable (fun tail : ({k : Fin 2 // k ≠ j}) → X => pairKernel h (insertCoord j x tail)) (Measure.pi fun _ : {k : Fin 2 // k ≠ j} => P) := by intro j x fin_cases j · change Integrable (fun tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X => pairKernel h (insertCoord (0 : Fin 2) x tail)) (Measure.pi fun _ : {k : Fin 2 // ¬ k = (0 : Fin 2)} => P) let a : {k : Fin 2 // ¬ k = (0 : Fin 2)} := ⟨1, by norm_num⟩ have htail : (fun tail : ({k : Fin 2 // ¬ k = (0 : Fin 2)}) → X => pairKernel h (insertCoord (0 : Fin 2) x tail)) = fun tail => h x (tail a) := by funext tail simp [pairKernel, insertCoord, a] rw [htail] have hmp_eval := measurePreserving_eval (fun _ : {k : Fin 2 // ¬ k = (0 : Fin 2)} => P) a simpa [Function.comp_def] using hmp_eval.integrable_comp_of_integrable (hrow x) · change Integrable (fun tail : ({k : Fin 2 // ¬ k = (1 : Fin 2)}) → X => pairKernel h (insertCoord (1 : Fin 2) x tail)) (Measure.pi fun _ : {k : Fin 2 // ¬ k = (1 : Fin 2)} => P) let a : {k : Fin 2 // ¬ k = (1 : Fin 2)} := ⟨0, by norm_num⟩ have hcol : Integrable (fun y => h y x) P := by have hfun_col : (fun y => h y x) = fun y => h x y := by funext y exact hsymm y x rw [hfun_col] exact hrow x -- … truncated; follow the source link for the rest …
4 supporting declarations (lemmas, instances)
Remainder­Negligible 2 core · 1 supporting Discharges the fixed-order Hájek remainder negligibility hypothesis. ★ orderDegenerateNegligible_of_firstDegen★ orderDegenerateNegligible_of_residual

Discharges the fixed-order Hájek remainder negligibility hypothesis.

The theorem orderDegenerateNegligible_of_firstDegen proves that a first-order degenerate kernel has √n-rescaled U-statistic o_p(1), using the second-moment estimate from OrderM.RemainderSecondMoment and Chebyshev's inequality. The public wrapper orderDegenerateNegligible_of_residual applies this result to the residual kernel uDegenOrder h P, producing the OrderDegenerateNegligible S h hypothesis required by the fixed-order asymptotic-linearity and CLT statements.

theorem orderDegenerateNegligible_of_firstDegen reviewed
Causalean.Stat.IIDSample

Negligibility of the higher-order remainder. For an i.i.d. sample S, if the order-m kernel g is first-order degenerate: measurable, square-integrable, and mean zero after integrating out any single coordinate, then the √n-rescaled order-m U-statistic of g converges to zero in probability, i.e. it is o_p(1).

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
IsLittleOp (fun n ω => sqrt (n : ℝ) * uStatisticOrder S g n ω) (fun _ => (1 : ℝ)) μ
Proof (Lean source)
theorem orderDegenerateNegligible_of_firstDegen [IsFiniteMeasure P] (hg : OrderFirstDegenKernel P g) : IsLittleOp (fun n ω => sqrt (n : ℝ) * uStatisticOrder S g n ω) (fun _ => (1 : ℝ)) μ := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure intro ε hε rcases S.integral_rescaled_order_sq_le hg with ⟨C, hCnn, hCbound⟩ have hb_tendsto : Tendsto (fun n : ℕ => ofReal ((C / ε ^ 2) / (n : ℝ))) atTop (𝓝 0) := by rw [← ENNReal.ofReal_zero] apply ENNReal.tendsto_ofReal have : Tendsto (fun n : ℕ => (C / ε ^ 2) * ((n : ℝ))⁻¹) atTop (𝓝 ((C / ε ^ 2) * 0)) := by apply Filter.Tendsto.const_mul exact tendsto_natCast_atTop_atTop.inv_tendsto_atTop simpa [div_eq_mul_inv, mul_zero] using this have hbound : ∀ᶠ n : ℕ in atTop, μ {ω | ε * (fun _ => (1 : ℝ)) n < |sqrt (n : ℝ) * uStatisticOrder S g n ω|} ≤ ofReal ((C / ε ^ 2) / (n : ℝ)) := by filter_upwards [eventually_ge_atTop m] with n hmn set X : Ω → ℝ := fun ω => sqrt (n : ℝ) * uStatisticOrder S g n ω with hXdef have hmem : MemLp X 2 μ := by simpa [X, hXdef] using S.memLp_rescaled_order hg.meas hg.sq n have hmean : ∫ ω, X ω ∂μ = 0 := by simpa [X, hXdef] using S.integral_rescaled_order_eq_zero hg hmn have hvar_le : variance X μ ≤ C / (n : ℝ) := by rw [ProbabilityTheory.variance_eq_sub hmem, hmean] simp only [Pi.pow_apply] simpa [X, hXdef] using hCbound hmn have hcheb := ProbabilityTheory.meas_ge_le_variance_div_sq hmem (c := ε) hε simp only [hmean, sub_zero] at hcheb have hsub : {ω | ε * (fun _ => (1 : ℝ)) n < |sqrt (n : ℝ) * uStatisticOrder S g n ω|} ⊆ {ω | ε ≤ |X ω|} := by intro ω hω simp only [Set.mem_setOf_eq, mul_one, X] at hω ⊢ exact le_of_lt hω refine le_trans (measure_mono hsub) ?_ calc μ {ω | ε ≤ |X ω|} ≤ ofReal (variance X μ / ε ^ 2) := hcheb _ ≤ ofReal ((C / (n : ℝ)) / ε ^ 2) := by apply ENNReal.ofReal_le_ofReal gcongr _ = ofReal ((C / ε ^ 2) / (n : ℝ)) := by exact congrArg ofReal (by ring) refine tendsto_of_tendsto_of_tendsto_of_le_of_le' tendsto_const_nhds hb_tendsto (Eventually.of_forall (fun n => zero_le)) hbound
theorem orderDegenerateNegligible_of_residual reviewed
Causalean.Stat

The higher-order remainder of a fixed-order U-statistic is negligible. For an i.i.d. sample S and order-m kernel h, write g for the higher-order Hájek residual of h. If g is measurable and square-integrable under the m-fold product law, and if for every coordinate integrating h over the remaining m − 1 coordinates yields an integrable function of that coordinate with the same population mean uMeanOrder h P in every coordinate and h remains integrable in the remaining coordinates for every fixed value of that coordinate, then the √n-rescaled higher-order residual U-statistic √n · Gₙ converges to zero in probability, i.e. it is o_p(1). This discharges the OrderDegenerateNegligible hypothesis consumed by the order-m CLT uStatisticOrder_clt (Causalean.Stat.UStatistic.OrderM.CLT).

Formal statement
S :
IIDSample Ω X μ P
m :
h :
(Fin m → X) → ℝ
hmeas :
hL2 :
Integrable (fun z => (uDegenOrder h P z) ^ 2) (Measure.pi fun _ : Fin m => P)
hslice_int :
∀ j : Fin m,
Integrable (fun x => ∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) P
hmean :
∀ j : Fin m,
∫ x, (∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) ∂P
hrow :
∀ (j : Fin m) (x : X),
Integrable (fun tail : ({k : Fin m // k ≠ j}) → X => h (insertCoord j x tail)) (Measure.pi fun _ : {k : Fin m // k ≠ j} => P)
Proof (Lean source)
theorem orderDegenerateNegligible_of_residual {Ω X : Type*} [MeasurableSpace Ω] [MeasurableSpace X] {μ : Measure Ω} {P : Measure X} (S : IIDSample Ω X μ P) {m : ℕ} [NeZero m] (h : (Fin m → X) → ℝ) (hmeas : Measurable (uDegenOrder h P)) (hL2 : Integrable (fun z => (uDegenOrder h P z) ^ 2) (Measure.pi fun _ : Fin m => P)) (hslice_int : ∀ j : Fin m, Integrable (fun x => ∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) P) (hmean : ∀ j : Fin m, ∫ x, (∫ tail : ({k : Fin m // k ≠ j}) → X, h (insertCoord j x tail) ∂(Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) ∂P = uMeanOrder h P) (hrow : ∀ (j : Fin m) (x : X), Integrable (fun tail : ({k : Fin m // k ≠ j}) → X => h (insertCoord j x tail)) (Measure.pi fun _ : {k : Fin m // k ≠ j} => P)) : OrderDegenerateNegligible S h := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable have hg : OrderFirstDegenKernel P (uDegenOrder h P) := { meas := hmeas firstDeg := fun j x => uDegenOrder_integral_tail_eq_zero hslice_int hmean hrow j x sq := hL2 } unfold OrderDegenerateNegligible uRemainderOrder exact S.orderDegenerateNegligible_of_firstDegen hg
1 supporting declaration (lemmas, instances)
Remainder­Second­Moment 2 core · 8 supporting Proves the second-moment bound for first-order degenerate fixed-order U-statistic remainders. ★ crossterm_eq_zero_of_shared_le_one★ integral_rescaled_order_sq_le

Proves the second-moment bound for first-order degenerate fixed-order U-statistic remainders.

The central public estimate is IIDSample.integral_rescaled_order_sq_le: for a kernel satisfying OrderFirstDegenKernel, there is a finite constant C depending only on the order and ζ_m = E[g²] such that E[(√n * Uₙ)²] ≤ C / n whenever n ≥ m. The proof first shows crossterm_eq_zero_of_shared_le_one, bounding the only nonzero terms by Cauchy-Schwarz, then counts the tuple pairs sharing at least two sample indices and normalizes by the falling factorial denominator.

theorem crossterm_eq_zero_of_shared_le_one reviewed
Causalean.Stat.IIDSample

Cross-term vanishing. For an i.i.d. sample S, sample size n, and order-m kernel g that is first-order degenerate, let t and q be ordered m-tuples of sample indices that are each injective, and suppose the images of t and q share at most one sample index. Then the expected product of the kernel evaluated along t and along q is zero: zero shared indices give independence with mean zero on each factor, while one shared index reduces, after conditioning on it, to first-order degeneracy of each factor.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
n :
Fin m → Fin n
ht :
hq :
hshare :
(Finset.univ.image t ∩ Finset.univ.image q).card ≤ 1
∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0
Proof (Lean source)
theorem crossterm_eq_zero_of_shared_le_one (hg : OrderFirstDegenKernel P g) {n : ℕ} {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) (hshare : (Finset.univ.image t ∩ Finset.univ.image q).card ≤ 1) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0 := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable exact S.crossterm_eq_zero_of_shared_le_one_product_disintegration hg ht hq hshare
Causalean.Stat.IIDSample.crossterm_eq_zero_of_shared_le_one · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:279 · uses IIDSample , OrderFirstDegenKernel
theorem integral_rescaled_order_sq_le reviewed
Causalean.Stat.IIDSample

bound on the rescaled higher-order remainder. For an i.i.d. sample S, if the order-m kernel g is first-order degenerate, then there is a nonnegative constant C, depending only on the order m and the kernel's second moment ζ_m = E[g²], such that the second moment of the √n-rescaled order-m U-statistic of g is at most C/n for every sample size n ≥ m. This is the keystone estimate; it packages cross-term vanishing, the Cauchy–Schwarz bound |E[g_t g_q]| ≤ ζ_m, the O(n^{2m-2}) count of surviving tuple pairs, and the n · (n^{(m)})⁻² normalization.

Formal statement
Ω :
Type u_1
shared
X :
Type u_2
shared
μ :
shared
P :
shared
m :
(Fin m → X) → ℝ
shared
S :
IIDSample Ω X μ P
shared
∃ C : ℝ,
0 ≤ C
conclusion 1
n :
m ≤ n
∫ ω, (sqrt (n : ℝ) * uStatisticOrder S g n ω) ^ 2 ∂μ ≤ C / (n : ℝ)
Proof (Lean source)
theorem integral_rescaled_order_sq_le (hg : OrderFirstDegenKernel P g) : ∃ C : ℝ, 0 ≤ C ∧ ∀ {n : ℕ}, m ≤ n → ∫ ω, (sqrt (n : ℝ) * uStatisticOrder S g n ω) ^ 2 ∂μ ≤ C / (n : ℝ) := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure letI : IsProbabilityMeasure P := by rw [← S.law] exact Measure.isProbabilityMeasure_map (S.meas 0).aemeasurable exact S.integral_rescaled_order_sq_le_counting_normalization hg
8 supporting declarations (lemmas, instances)
  • crossterm_zero_of_disjoint theorem — For a first-order-degenerate kernel, the expected product of terms evaluated on two injective sample-index tuples with disjoint index sets is zero.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    n :
    Fin m → Fin n
    ht :
    hq :
    hdisj :
    Disjoint (Finset.univ.image t) (Finset.univ.image q)
    ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0
    Proof (Lean source)
    theorem crossterm_zero_of_disjoint [IsFiniteMeasure P] (hg : OrderFirstDegenKernel P g) {n : ℕ} {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) (hdisj : Disjoint (Finset.univ.image t) (Finset.univ.image q)) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0 := by classical let A : Finset (Fin n) := Finset.univ.image t let B : Finset (Fin n) := Finset.univ.image q let Xt : Ω → (A → X) := fun ω i => S.Z (i.1 : ℕ) ω let Xq : Ω → (B → X) := fun ω i => S.Z (i.1 : ℕ) ω let φ : (A → X) → ℝ := fun x => g (fun j => x ⟨t j, Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩⟩) let ψ : (B → X) → ℝ := fun x => g (fun j => x ⟨q j, Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩⟩) have hindFin : iIndepFun (fun i : Fin n => S.Z (i : ℕ)) μ := S.indep.precomp (fun _ _ h => Fin.ext h) have hindBlocks : IndepFun Xt Xq μ := by simpa [A, B, Xt, Xq] using (ProbabilityTheory.iIndepFun.indepFun_finset A B hdisj hindFin (fun i : Fin n => S.meas (i : ℕ))) have hφ : Measurable φ := by exact hg.meas.comp (measurable_pi_lambda _ (fun j : Fin m => measurable_pi_apply _)) have hψ : Measurable ψ := by exact hg.meas.comp (measurable_pi_lambda _ (fun j : Fin m => measurable_pi_apply _)) have hind : IndepFun (φ ∘ Xt) (ψ ∘ Xq) μ := hindBlocks.comp hφ hψ have hφsm : AEStronglyMeasurable (φ ∘ Xt) μ := (hφ.comp (measurable_pi_lambda _ (fun i : A => S.meas (i.1 : ℕ)))).aestronglyMeasurable have hψsm : AEStronglyMeasurable (ψ ∘ Xq) μ := (hψ.comp (measurable_pi_lambda _ (fun i : B => S.meas (i.1 : ℕ)))).aestronglyMeasurable have hfactor : ∫ ω, (φ ∘ Xt) ω * (ψ ∘ Xq) ω ∂μ = (∫ ω, (φ ∘ Xt) ω ∂μ) * (∫ ω, (ψ ∘ Xq) ω ∂μ) := hind.integral_fun_mul_eq_mul_integral hφsm hψsm have hmean_t : ∫ ω, (φ ∘ Xt) ω ∂μ = 0 := by simpa [φ, Xt, uMeanOrder] using (S.integral_orderKernelTerm_eq_zero_of_uMean_zero hg.meas ht hg.integral_eq_zero) have hmean_q : ∫ ω, (ψ ∘ Xq) ω ∂μ = 0 := by simpa [ψ, Xq, uMeanOrder] using (S.integral_orderKernelTerm_eq_zero_of_uMean_zero hg.meas hq hg.integral_eq_zero) change ∫ ω, (φ ∘ Xt) ω * (ψ ∘ Xq) ω ∂μ = 0 rw [hfactor, hmean_t, hmean_q, zero_mul]
    Causalean.Stat.IIDSample.crossterm_zero_of_disjoint · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:68
  • crossterm_zero_of_shared_one theorem — For a first-order-degenerate kernel, the expected product of terms evaluated on two injective sample-index tuples sharing exactly one index is zero.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    n :
    Fin m → Fin n
    ht :
    hq :
    a :
    Fin n
    hshare :
    Finset.univ.image t ∩ Finset.univ.image q = {a}
    ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0
    Proof (Lean source)
    theorem crossterm_zero_of_shared_one [IsFiniteMeasure P] (hg : OrderFirstDegenKernel P g) {n : ℕ} {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) {a : Fin n} (hshare : Finset.univ.image t ∩ Finset.univ.image q = {a}) : ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = 0 := by classical have ha_inter : a ∈ Finset.univ.image t ∩ Finset.univ.image q := by rw [hshare] simp have ha_t : a ∈ Finset.univ.image t := (Finset.mem_inter.mp ha_inter).1 have ha_q : a ∈ Finset.univ.image q := (Finset.mem_inter.mp ha_inter).2 rcases Finset.mem_image.mp ha_t with ⟨p, _hp, htp⟩ let Tail : Type := {k : Fin m // k ≠ p} let B : Finset (Fin n) := Finset.univ.image q let XA : Ω → (Tail → X) := fun ω k => S.Z (t k.1 : ℕ) ω let Xq : Ω → (B → X) := fun ω i => S.Z (i.1 : ℕ) ω let πTail : Measure (Tail → X) := Measure.pi fun _ : Tail => P let Φ : (Tail → X) → (B → X) → ℝ := fun tail xq => g (insertCoord p (xq ⟨a, ha_q⟩) tail) let Ψ : (B → X) → ℝ := fun xq => g (fun j => xq ⟨q j, Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩⟩) let F : (Tail → X) × (B → X) → ℝ := fun z => Φ z.1 z.2 * Ψ z.2 have hXAmeas : Measurable XA := by exact measurable_pi_lambda _ (fun k : Tail => S.meas (t k.1 : ℕ)) have hXqmeas : Measurable Xq := by exact measurable_pi_lambda _ (fun i : B => S.meas (i.1 : ℕ)) have hPairMeas : Measurable (fun ω => (XA ω, Xq ω)) := hXAmeas.prodMk hXqmeas have hXAmap : μ.map XA = πTail := by have hr : Injective (fun k : Tail => t k.1) := by intro k l hkl exact Subtype.ext (ht hkl) simpa [XA, πTail, Tail] using (S.map_fintype_tuple_eq (ι := Tail) (r := fun k : Tail => t k.1) hr) have hdisj : Disjoint (Finset.univ.image (fun k : Tail => t k.1)) B := by rw [Finset.disjoint_iff_inter_eq_empty] ext x constructor · intro hx rcases Finset.mem_inter.mp hx with ⟨hxA, hxB⟩ rcases Finset.mem_image.mp hxA with ⟨k, _hk, hkx⟩ have hx_inter : x ∈ Finset.univ.image t ∩ Finset.univ.image q := by exact Finset.mem_inter.mpr ⟨Finset.mem_image.mpr ⟨k.1, Finset.mem_univ k.1, hkx⟩, by simpa [B] using hxB⟩ have hxa : x = a := by have : x ∈ ({a} : Finset (Fin n)) := by simpa [hshare] using hx_inter simpa using this have hkp : k.1 = p := ht (by calc t k.1 = x := hkx _ = a := hxa _ = t p := htp.symm) exact (k.2 hkp).elim · intro hx simp at hx have hindFin : iIndepFun (fun i : Fin n => S.Z (i : ℕ)) μ := S.indep.precomp (fun _ _ h => Fin.ext h) have hindBlocks : IndepFun (fun ω : Ω => fun i : (Finset.univ.image (fun k : Tail => t k.1)) => S.Z (i.1 : ℕ) ω) Xq μ := by simpa [B, Xq] using (ProbabilityTheory.iIndepFun.indepFun_finset (Finset.univ.image (fun k : Tail => t k.1)) B hdisj hindFin (fun i : Fin n => S.meas (i : ℕ))) have htoTail : Measurable (fun w : (Finset.univ.image (fun k : Tail => t k.1)) → X => fun k : Tail => w ⟨t k.1, Finset.mem_image.mpr ⟨k, Finset.mem_univ k, rfl⟩⟩) := by exact measurable_pi_lambda _ (fun k : Tail => measurable_pi_apply _) have hind : IndepFun XA Xq μ := by have hcomp := hindBlocks.comp htoTail measurable_id simpa [XA, Function.comp_def] using hcomp have hΦpair : Measurable (fun z : (Tail → X) × (B → X) => Φ z.1 z.2) := by exact hg.meas.comp (measurable_pi_lambda _ (fun j : Fin m => by by_cases hj : j = p · subst j simpa [Φ, insertCoord, Function.comp_def] using (measurable_pi_apply (⟨a, ha_q⟩ : B)).comp measurable_snd · simpa [Φ, insertCoord, hj, Function.comp_def] using (measurable_pi_apply (⟨j, hj⟩ : Tail)).comp measurable_fst)) have hΨ : Measurable Ψ := by exact hg.meas.comp (measurable_pi_lambda _ (fun j : Fin m => measurable_pi_apply _)) have hΨpair : Measurable (fun z : (Tail → X) × (B → X) => Ψ z.2) := hΨ.comp measurable_snd have hFmeas : Measurable F := hΦpair.mul hΨpair have ht_rewrite : ∀ ω, g (fun j => S.Z (t j : ℕ) ω) = Φ (XA ω) (Xq ω) := by intro ω congr 1 funext j by_cases hj : j = p · subst j simp [Xq, insertCoord, htp] · simp [XA, insertCoord, hj] have hq_rewrite : ∀ ω, g (fun j => S.Z (q j : ℕ) ω) = Ψ (Xq ω) := by intro ω rfl have hcomp_int : Integrable (fun ω => F (XA ω, Xq ω)) μ := by have horig := S.integrable_orderTerm_mul hg.meas hg.sq ht hq refine horig.congr ?_ filter_upwards with ω simp [F, ht_rewrite ω, hq_rewrite ω] have hmap_pair : μ.map (fun ω => (XA ω, Xq ω)) = πTail.prod (μ.map Xq) := by have h := (indepFun_iff_map_prod_eq_prod_map_map hXAmeas.aemeasurable hXqmeas.aemeasurable).mp hind simpa [hXAmap] using h have hFint_map : Integrable F (πTail.prod (μ.map Xq)) := by have hmap_int : Integrable F (μ.map fun ω => (XA ω, Xq ω)) := (integrable_map_measure hFmeas.aestronglyMeasurable hPairMeas.aemeasurable).mpr hcomp_int simpa [hmap_pair] using hmap_int have hinner : ∀ xq : B → X, (∫ tail : Tail → X, F (tail, xq) ∂πTail) = 0 := by intro xq change (∫ tail : Tail → X, g (insertCoord p (xq ⟨a, ha_q⟩) tail) * Ψ xq ∂πTail) = 0 integral_linearity rw [hg.firstDeg p (xq ⟨a, ha_q⟩)] simp calc ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = ∫ ω, F (XA ω, Xq ω) ∂μ := by apply integral_congr_ae filter_upwards with ω simp [F, ht_rewrite ω, hq_rewrite ω] _ = ∫ z, F z ∂(μ.map fun ω => (XA ω, Xq ω)) := by rw [integral_map hPairMeas.aemeasurable hFmeas.aestronglyMeasurable] _ = ∫ z, F z ∂(πTail.prod (μ.map Xq)) := by rw [hmap_pair] _ = ∫ xq, ∫ tail, F (tail, xq) ∂πTail ∂(μ.map Xq) := by rw [integral_prod_symm F hFint_map] _ = 0 := by rw [show (fun xq : B → X => ∫ tail : Tail → X, F (tail, xq) ∂πTail) = fun _ => 0 by funext xq exact hinner xq] simp
    Causalean.Stat.IIDSample.crossterm_zero_of_shared_one · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:114
  • crossterm_abs_le_zeta theorem — The absolute expected product of two kernel evaluations on distinct sample tuples is no larger than the kernel's second moment under the product distribution.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    hmeas :
    hsq :
    Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)
    n :
    Fin m → Fin n
    ht :
    hq :
    |∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ| ≤ zetaOrder P g
    Proof (Lean source)
    theorem crossterm_abs_le_zeta (hmeas : Measurable g) (hsq : Integrable (fun z => (g z) ^ 2) (Measure.pi fun _ : Fin m => P)) {n : ℕ} {t q : Fin m → Fin n} (ht : Injective t) (hq : Injective q) : |∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ| ≤ zetaOrder P g := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure let ft : Ω → ℝ := fun ω => g (fun j => S.Z (t j : ℕ) ω) let fq : Ω → ℝ := fun ω => g (fun j => S.Z (q j : ℕ) ω) have hft0 : MemLp ft 2 μ := S.memLp_orderTerm hmeas hsq ht have hfq0 : MemLp fq 2 μ := S.memLp_orderTerm hmeas hsq hq have hft : MemLp ft (ofReal (2 : ℝ)) μ := by simpa using hft0 have hfq : MemLp fq (ofReal (2 : ℝ)) μ := by simpa using hfq0 have hcs := MeasureTheory.integral_mul_norm_le_Lp_mul_Lq (μ := μ) (f := ft) (g := fq) (p := (2 : ℝ)) (q := (2 : ℝ)) (Real.holderConjugate_iff.mpr (by norm_num)) hft hfq have habs : |∫ ω, ft ω * fq ω ∂μ| ≤ ∫ ω, ‖ft ω‖ * ‖fq ω‖ ∂μ := by calc |∫ ω, ft ω * fq ω ∂μ| ≤ ∫ ω, |ft ω * fq ω| ∂μ := abs_integral_le_integral_abs _ = ∫ ω, ‖ft ω‖ * ‖fq ω‖ ∂μ := by simp [Real.norm_eq_abs, abs_mul] have ht2 : ∫ ω, ‖ft ω‖ ^ (2 : ℝ) ∂μ = zetaOrder P g := by simpa [ft, sq_abs] using S.orderTerm_diag hmeas ht have hq2 : ∫ ω, ‖fq ω‖ ^ (2 : ℝ) ∂μ = zetaOrder P g := by simpa [fq, sq_abs] using S.orderTerm_diag hmeas hq have hznonneg : 0 ≤ zetaOrder P g := zetaOrder_nonneg rw [ht2, hq2] at hcs have hroot : (zetaOrder P g) ^ (1 / (2 : ℝ)) * (zetaOrder P g) ^ (1 / (2 : ℝ)) = zetaOrder P g := by rw [← Real.rpow_add' hznonneg (by norm_num : (1 / (2 : ℝ) + 1 / (2 : ℝ)) ≠ 0)] rw [show (1 / (2 : ℝ) + 1 / (2 : ℝ)) = (1 : ℝ) by norm_num] exact Real.rpow_one (zetaOrder P g) rw [hroot] at hcs exact le_trans habs hcs
    Causalean.Stat.IIDSample.crossterm_abs_le_zeta · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:299
  • sharedPositions_card_eq theorem — For an injective tuple map, the number of positions whose values occur in a second tuple equals the number of values shared by the two tuple images.
    α β :
    Type*
    α → β
    hq :
    (Finset.univ.filter (fun j : α => q j ∈ Finset.univ.image t)).card
    = (Finset.univ.image t ∩ Finset.univ.image q).card
    Proof (Lean source)
    theorem sharedPositions_card_eq {α β : Type*} [Fintype α] [DecidableEq β] {t q : α → β} (hq : Injective q) : (Finset.univ.filter (fun j : α => q j ∈ Finset.univ.image t)).card = (Finset.univ.image t ∩ Finset.univ.image q).card := by have himage : (Finset.univ.filter (fun j : α => q j ∈ Finset.univ.image t)).image q = Finset.univ.image t ∩ Finset.univ.image q := by ext a constructor · intro ha rcases Finset.mem_image.mp ha with ⟨j, hj, rfl⟩ rw [mem_filter] at hj exact Finset.mem_inter.mpr ⟨hj.2, Finset.mem_image.mpr ⟨j, Finset.mem_univ j, rfl⟩⟩ · intro ha rcases Finset.mem_inter.mp ha with ⟨hat, haq⟩ rcases Finset.mem_image.mp haq with ⟨j, _hj, rfl⟩ exact Finset.mem_image.mpr ⟨j, by simpa using hat, rfl⟩ rw [← himage] exact (Finset.card_image_of_injOn (s := Finset.univ.filter (fun j : α => q j ∈ Finset.univ.image t)) (f := q) (fun a _ b _ h => hq h)).symm
    Causalean.Stat.IIDSample.sharedPositions_card_eq · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:387
  • card_share_ge_two_le theorem — The number of ordered pairs of injective tuples of length m drawn from n observations that share at least two sample indices is bounded by a polynomial in n determined by the tuple length.
    m :
    shared
    n :
    ((injectiveTuples m n ×ˢ injectiveTuples m n).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card)).card
    ≤ m.choose 2 * m ^ 2 * n ^ (2 * m - 2)
    Proof (Lean source)
    theorem card_share_ge_two_le {n : ℕ} : ((injectiveTuples m n ×ˢ injectiveTuples m n).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card)).card ≤ m.choose 2 * m ^ 2 * n ^ (2 * m - 2) := by classical by_cases hmzero : m = 0 · subst m simp [injectiveTuples] let share : Finset ((Fin m → Fin n) × (Fin m → Fin n)) := (injectiveTuples m n ×ˢ injectiveTuples m n).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card) let D := {tq : (Fin m → Fin n) × (Fin m → Fin n) // tq ∈ share} let encode : D → shareCountEncoding m n := fun d => let t := d.1.1 let q := d.1.2 have hmem : d.1 ∈ share := d.2 have hpair : d.1 ∈ injectiveTuples m n ×ˢ injectiveTuples m n := (Finset.mem_filter.mp hmem).1 have hq : Injective q := (Finset.mem_filter.mp (Finset.mem_product.mp hpair).2).2 let sp : Finset (Fin m) := Finset.univ.filter (fun j : Fin m => q j ∈ Finset.univ.image t) have hspcard : 2 ≤ sp.card := by have hshare : 2 ≤ (Finset.univ.image t ∩ Finset.univ.image q).card := (Finset.mem_filter.mp hmem).2 rwa [sharedPositions_card_eq (t := t) (q := q) hq] let s : Finset (Fin m) := choose (Finset.powersetCard_nonempty.mpr hspcard) have hs_sp : s ∈ sp.powersetCard 2 := Classical.choose_spec (Finset.powersetCard_nonempty.mpr hspcard) have hs_univ : s ∈ (Finset.univ : Finset (Fin m)).powersetCard 2 := by exact Finset.mem_powersetCard.mpr ⟨fun j _ => Finset.mem_univ j, (Finset.mem_powersetCard.mp hs_sp).2⟩ let k : {j : Fin m // j ∈ s} → Fin m := fun j => choose (Finset.mem_image.mp ((Finset.mem_filter.mp ((Finset.mem_powersetCard.mp hs_sp).1 j.2)).2)) let r : {j : Fin m // j ∉ s} → Fin n := fun j => q j.1 ⟨t, ⟨⟨s, hs_univ⟩, (k, r)⟩⟩ have hdecode : ∀ d : D, shareCountDecodeQ (encode d) = d.1.2 := by intro d funext j dsimp [encode, shareCountDecodeQ] split_ifs with hj · exact (Classical.choose_spec (Finset.mem_image.mp ((Finset.mem_filter.mp ((Classical.choose_spec (Finset.powersetCard_nonempty.mpr (by have hmem : d.1 ∈ share := d.2 have hpair : d.1 ∈ injectiveTuples m n ×ˢ injectiveTuples m n := (Finset.mem_filter.mp hmem).1 have hq : Injective d.1.2 := (Finset.mem_filter.mp (Finset.mem_product.mp hpair).2).2 have hshare : 2 ≤ (Finset.univ.image d.1.1 ∩ Finset.univ.image d.1.2).card := (Finset.mem_filter.mp hmem).2 rwa [sharedPositions_card_eq (t := d.1.1) (q := d.1.2) hq])) |> Finset.mem_powersetCard.mp).1 hj)).2))).2 · rfl have hinj : Injective encode := by intro d₁ d₂ h apply Subtype.ext apply Prod.ext · exact congrArg fst h · have hq := congrFun (congrArg shareCountDecodeQ h) funext j rw [hdecode d₁, hdecode d₂] at hq exact hq j have hcardD : share.card = card D := by simp [D] rw [show ((injectiveTuples m n ×ˢ injectiveTuples m n).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card)).card = share.card by rfl, hcardD] exact le_trans (Fintype.card_le_of_injective encode hinj) (le_trans (shareCountEncoding_card_le m n) (by have hmpos : 0 < m := Nat.pos_of_ne_zero hmzero by_cases h2 : 2 ≤ m · have hexp : m + (m - 2) = 2 * m - 2 := by omega calc n ^ m * m.choose 2 * m ^ 2 * n ^ (m - 2) = m.choose 2 * m ^ 2 * (n ^ m * n ^ (m - 2)) := by ring _ ≤ m.choose 2 * m ^ 2 * n ^ (2 * m - 2) := by rw [← Nat.pow_add, hexp] · have hm1 : m = 1 := by omega subst m simp))
    Causalean.Stat.IIDSample.card_share_ge_two_le · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:414
  • descFactorial_ge theorem — When the sample size is at least the order, its falling factorial is at least the sample-size power divided by the order power.
    m n :
    hmn :
    m ≤ n
    (n : ℝ) ^ m / (m : ℝ) ^ m ≤ (n.descFactorial m : ℝ)
    Proof (Lean source)
    theorem descFactorial_ge {m n : ℕ} (hmn : m ≤ n) : (n : ℝ) ^ m / (m : ℝ) ^ m ≤ (n.descFactorial m : ℝ) := by by_cases hmzero : m = 0 · subst m simp have hmposNat : 0 < m := Nat.pos_of_ne_zero hmzero have hmpos : 0 < (m : ℝ) := by exact_mod_cast hmposNat rw [Nat.descFactorial_eq_prod_range] norm_num [Nat.cast_prod] have hfactor : ∀ i ∈ range m, (n : ℝ) / (m : ℝ) ≤ (n - i : ℕ) := by intro i hi have him : i < m := Finset.mem_range.mp hi have hin : i ≤ n := le_trans (le_of_lt him) hmn rw [Nat.cast_sub hin] have hmnR : (m : ℝ) ≤ (n : ℝ) := by exact_mod_cast hmn have hiR : (i : ℝ) + 1 ≤ (m : ℝ) := by exact_mod_cast (Nat.succ_le_of_lt him) have hmul : (n : ℝ) ≤ ((n : ℝ) - (i : ℝ)) * (m : ℝ) := by nlinarith [mul_nonneg (sub_nonneg.mpr hmnR) (sub_nonneg.mpr (by linarith : (1 : ℝ) ≤ (m : ℝ)))] exact (div_le_iff₀ hmpos).mpr hmul have hprod := Finset.prod_le_prod (s := range m) (f := fun _i : ℕ => (n : ℝ) / (m : ℝ)) (g := fun i : ℕ => (n - i : ℕ)) (fun _i _hi => div_nonneg (Nat.cast_nonneg _) (le_of_lt hmpos)) hfactor simpa [Finset.prod_const, div_pow] using hprod
    Causalean.Stat.IIDSample.descFactorial_ge · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:502
  • integral_injectiveTuples_sum_sq_le_shared_count theorem — For a first-order-degenerate kernel, the second moment of the unnormalised sum over ordered injective sample tuples is bounded by the number of tuple pairs sharing at least two observations times the kernel's second moment.
    Ω :
    Type u_1
    shared
    X :
    Type u_2
    shared
    μ :
    shared
    P :
    shared
    m :
    (Fin m → X) → ℝ
    shared
    S :
    IIDSample Ω X μ P
    shared
    n :
    ∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ
    ≤ (((injectiveTuples m n ×ˢ injectiveTuples m n).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card)).card : ℝ) * zetaOrder P g
    Proof (Lean source)
    theorem integral_injectiveTuples_sum_sq_le_shared_count [IsFiniteMeasure P] (hg : OrderFirstDegenKernel P g) {n : ℕ} : ∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ ≤ (((injectiveTuples m n ×ˢ injectiveTuples m n).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card)).card : ℝ) * zetaOrder P g := by letI : IsProbabilityMeasure μ := S.indep.isProbabilityMeasure classical let T : Finset (Fin m → Fin n) := injectiveTuples m n let SHARE : Finset ((Fin m → Fin n) × (Fin m → Fin n)) := (T ×ˢ T).filter (fun tq => 2 ≤ (Finset.univ.image tq.1 ∩ Finset.univ.image tq.2).card) let F : ((Fin m → Fin n) × (Fin m → Fin n)) → ℝ := fun tq => ∫ ω, g (fun j => S.Z (tq.1 j : ℕ) ω) * g (fun j => S.Z (tq.2 j : ℕ) ω) ∂μ have hinj_of_mem_T : ∀ t ∈ T, Injective t := by intro t ht have ht' : t ∈ injectiveTuples m n := by simpa [T] using ht exact (Finset.mem_filter.mp ht').2 have hexpand : (fun ω => (∑ t ∈ T, g (fun j => S.Z (t j : ℕ) ω)) ^ 2) = (fun ω => ∑ t ∈ T, ∑ q ∈ T, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) := by funext ω rw [sq, Finset.sum_mul_sum] rw [show (∫ ω, (∑ t ∈ injectiveTuples m n, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ) = ∫ ω, (∑ t ∈ T, g (fun j => S.Z (t j : ℕ) ω)) ^ 2 ∂μ by rfl] rw [hexpand] have hterm_int : ∀ t ∈ T, Integrable (fun ω => ∑ q ∈ T, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω)) μ := fun t ht => by apply integrable_finset_sum intro q hq exact S.integrable_orderTerm_mul hg.meas hg.sq (hinj_of_mem_T t ht) (hinj_of_mem_T q hq) integral_linearity have hpush : ∀ t ∈ T, ∫ ω, ∑ q ∈ T, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ = ∑ q ∈ T, ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ := by intro t ht exact integral_finset_sum _ (fun q hq => S.integrable_orderTerm_mul hg.meas hg.sq (hinj_of_mem_T t ht) (hinj_of_mem_T q hq)) rw [Finset.sum_congr rfl hpush] rw [show (∑ t ∈ T, ∑ q ∈ T, ∫ ω, g (fun j => S.Z (t j : ℕ) ω) * g (fun j => S.Z (q j : ℕ) ω) ∂μ) = ∑ tq ∈ T ×ˢ T, F tq by rw [Finset.sum_product]] have hsub : SHARE ⊆ T ×ˢ T := Finset.filter_subset _ _ have hzero : ∀ x ∈ T ×ˢ T, x ∉ SHARE → F x = 0 := by intro x hx hxnot have ht : Injective x.1 := hinj_of_mem_T x.1 (Finset.mem_product.mp hx).1 have hq : Injective x.2 := hinj_of_mem_T x.2 (Finset.mem_product.mp hx).2 have hcard : (Finset.univ.image x.1 ∩ Finset.univ.image x.2).card ≤ 1 := by have : ¬ 2 ≤ (Finset.univ.image x.1 ∩ Finset.univ.image x.2).card := by intro h2 exact hxnot (Finset.mem_filter.mpr ⟨hx, h2⟩) omega simpa [F] using S.crossterm_eq_zero_of_shared_le_one hg ht hq hcard rw [← Finset.sum_subset hsub hzero] have hterm_le : ∀ x ∈ SHARE, F x ≤ zetaOrder P g := by intro x hx have hxT : x ∈ T ×ˢ T := (Finset.mem_filter.mp hx).1 have ht : Injective x.1 := hinj_of_mem_T x.1 (Finset.mem_product.mp hxT).1 have hq : Injective x.2 := hinj_of_mem_T x.2 (Finset.mem_product.mp hxT).2 exact le_trans (le_abs_self (F x)) (by simpa [F] using S.crossterm_abs_le_zeta hg.meas hg.sq ht hq) have hsum := Finset.sum_le_card_nsmul SHARE F (zetaOrder P g) hterm_le simpa [SHARE, T, nsmul_eq_mul] using hsum
    Causalean.Stat.IIDSample.integral_injectiveTuples_sum_sq_le_shared_count · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:532
  • rescaled_order_normalization_le theorem — For a nonnegative second-moment bound and a sample size at least the kernel order, the falling-factorial normalization term is bounded by a constant divided by the sample size.
    m :
    shared
    n :
    hmn :
    m ≤ n
    ζ :
    :
    0 ≤ ζ
    (n : ℝ) * (injectiveTupleCount m n)⁻¹ ^ 2 * ((m.choose 2 : ℝ) * (m : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2) * ζ)
    ≤ ((m.choose 2 : ℝ) * (m : ℝ) ^ (2 * m + 2) * ζ) / (n : ℝ)
    Proof (Lean source)
    theorem rescaled_order_normalization_le {n : ℕ} (hmn : m ≤ n) {ζ : ℝ} (hζ : 0 ≤ ζ) : (n : ℝ) * (injectiveTupleCount m n)⁻¹ ^ 2 * ((m.choose 2 : ℝ) * (m : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2) * ζ) ≤ ((m.choose 2 : ℝ) * (m : ℝ) ^ (2 * m + 2) * ζ) / (n : ℝ) := by have hmposNat : 0 < m := Nat.pos_of_ne_zero (NeZero.ne m) have hnposNat : 0 < n := lt_of_lt_of_le hmposNat hmn have hmpos : 0 < (m : ℝ) := by exact_mod_cast hmposNat have hnpos : 0 < (n : ℝ) := by exact_mod_cast hnposNat have hDpos : 0 < (n.descFactorial m : ℝ) := by exact_mod_cast Nat.descFactorial_pos.mpr hmn have hDlower : (n : ℝ) ^ m / (m : ℝ) ^ m ≤ (n.descFactorial m : ℝ) := descFactorial_ge (m := m) hmn have hlower_pos : 0 < (n : ℝ) ^ m / (m : ℝ) ^ m := by positivity have hinv : ((n.descFactorial m : ℝ)⁻¹) ≤ ((m : ℝ) ^ m / (n : ℝ) ^ m) := by have h := inv_anti₀ hlower_pos hDlower have hrewrite : ((n : ℝ) ^ m / (m : ℝ) ^ m)⁻¹ = (m : ℝ) ^ m / (n : ℝ) ^ m := by field_simp [pow_ne_zero _ hmpos.ne', pow_ne_zero _ hnpos.ne'] simpa [hrewrite] using h have hinv_sq : ((n.descFactorial m : ℝ)⁻¹) ^ 2 ≤ ((m : ℝ) ^ m / (n : ℝ) ^ m) ^ 2 := by exact pow_le_pow_left₀ (inv_nonneg.mpr hDpos.le) hinv 2 rw [injectiveTupleCount_eq_descFactorial] calc (n : ℝ) * ((n.descFactorial m : ℝ)⁻¹) ^ 2 * ((m.choose 2 : ℝ) * (m : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2) * ζ) ≤ (n : ℝ) * ((m : ℝ) ^ m / (n : ℝ) ^ m) ^ 2 * ((m.choose 2 : ℝ) * (m : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2) * ζ) := by gcongr _ = ((m.choose 2 : ℝ) * (m : ℝ) ^ (2 * m + 2) * ζ) / (n : ℝ) := by have hpow_n : (n : ℝ) ^ 2 * (n : ℝ) ^ (m * 2 - 2) = (n : ℝ) ^ (m * 2) := by rw [← pow_add] congr 1 omega have hpow_n' : (n : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2) = (n : ℝ) ^ (2 * m) := by rw [← pow_add] congr 1 omega have hpow_m : ((m : ℝ) ^ m) ^ 2 * (m : ℝ) ^ 2 = (m : ℝ) ^ (2 * m + 2) := by rw [← pow_mul, ← pow_add] congr 1 omega have hpow_n_rhs : ((n : ℝ) ^ m) ^ 2 = (n : ℝ) ^ (2 * m) := by rw [← pow_mul] congr 1 omega field_simp [pow_ne_zero _ hnpos.ne', pow_ne_zero _ hmpos.ne'] calc (n : ℝ) ^ 2 * ((m : ℝ) ^ m) ^ 2 * (m.choose 2 : ℝ) * (m : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2) * ζ = (m.choose 2 : ℝ) * ζ * (((m : ℝ) ^ m) ^ 2 * (m : ℝ) ^ 2) * ((n : ℝ) ^ 2 * (n : ℝ) ^ (2 * m - 2)) := by ring _ = (m.choose 2 : ℝ) * ζ * (m : ℝ) ^ (2 * m + 2) * (n : ℝ) ^ (2 * m) := by rw [hpow_m, hpow_n'] _ = ((n : ℝ) ^ m) ^ 2 * (m.choose 2 : ℝ) * ζ * (m : ℝ) ^ (2 * m + 2) := by rw [hpow_n_rhs] ring
    Causalean.Stat.IIDSample.rescaled_order_normalization_le · Causalean/Stat/UStatistic/OrderM/RemainderSecondMoment.lean:608