Stat.Minimax
Minimax lower-bound machinery: risk definitions, maximal couplings, coordinatewise overlap, Le Cam two-point methods, Pinsker/KL bridges, and rate statements for estimation problems.
FiniteSideInformation 37 core · 43 supporting · 11 submodules This umbrella module exports finite-simplex coordinates, finite squared-risk regularity, exact-versus-empirical comparison, concentration, compact-fiber approximation, the convergence theorem, and its closed-simplex spec FiniteSquaredLoss 24 core · 35 supporting · 4 submodules Finite bounded decision problems: randomized designs, dependent finite observations, conditional mixing under squared loss, and an attaining minimax procedure with a least-favorable finite prior. MomentMatchedMixture 10 to review
41 core · 61 supporting · 5 submodules This umbrella module exports a model-agnostic path from exponential likelihood inner products and matched bounded-prior moments, through one-coordinate and finite-product total variation, to explicit squared-error lower
TotalVariation 2 core · 10 supporting This file defines the statistical total variation distance between two probability measures on a common measurable space. ★ one_sub_tvDist_le_test
Total Variation Distance
This file defines the statistical total variation distance between two probability measures on a common measurable space. It develops elementary bounds and the testing inequality that underlies Le Cam's two-point minimax method.
def tvDist reviewed
⚑ flagged
Causalean.Stat
For a sample space equipped with a σ-algebra and two measures on that space, the statistical total variation distance is the supremum, over all measurable events , of the absolute difference between the two measures' real-valued masses of .
Definition (Lean source)
parameters
μ ν :
Measure Ω
def
tvDist μ ν :
ℝ
given by
⨆ A : {A : Set Ω // MeasurableSet A}, |μ.real A.1 - ν.real A.1|
theorem one_sub_tvDist_le_test reviewed
⚑ flagged
Causalean.Stat
Le Cam testing bound. For probability measures μ and ν on the same space, any measurable rejection region A yields a total testing error — the probability of A under μ plus the probability of the complement of A under ν — that is at least 1 − tvDist(μ,ν).
Formal statement
Proof (Lean source)
theorem one_sub_tvDist_le_test {A : Set Ω} (hA : MeasurableSet A) :
1 - tvDist μ ν ≤ μ.real A + ν.real Aᶜ := by
have hcompl : ν.real Aᶜ = 1 - ν.real A := by
rw [measureReal_compl hA, probReal_univ]
rw [hcompl]
have h := measureReal_sub_le_tvDist (μ := μ) (ν := ν) hA
linarith
10 supporting declarations (lemmas, instances)
-
abs_measureReal_sub_le_onetheorem — Every term of the supremum defining tvDist is bounded by 1.Proof (Lean source)
theorem abs_measureReal_sub_le_one (A : Set Ω) : |μ.real A - ν.real A| ≤ 1 := by have hμ : μ.real A ∈ Icc (0 : ℝ) 1 := ⟨measureReal_nonneg, by have := measureReal_mono (μ := μ) (Set.subset_univ A) (measure_ne_top μ _) simpa [probReal_univ] using this⟩ have hν : ν.real A ∈ Icc (0 : ℝ) 1 := ⟨measureReal_nonneg, by have := measureReal_mono (μ := ν) (Set.subset_univ A) (measure_ne_top ν _) simpa [probReal_univ] using this⟩ rw [abs_le] constructor <;> [nlinarith [hμ.1, hμ.2, hν.1, hν.2]; nlinarith [hμ.1, hμ.2, hν.1, hν.2]] -
bddAbove_tvDist_rangetheorem — The family defining tvDist is bounded above (by 1).hypothesesconclusionProof (Lean source)
theorem bddAbove_tvDist_range : BddAbove (Set.range fun A : {A : Set Ω // MeasurableSet A} => |μ.real A.1 - ν.real A.1|) := by refine ⟨1, ?_⟩ rintro _ ⟨A, rfl⟩ exact abs_measureReal_sub_le_one A.1 -
abs_measureReal_sub_le_tvDisttheorem — For a measurable set A, the gap |μ.real A − ν.real A| is at most tvDist μ ν.Proof (Lean source)
theorem abs_measureReal_sub_le_tvDist {A : Set Ω} (hA : MeasurableSet A) : |μ.real A - ν.real A| ≤ tvDist μ ν := le_ciSup bddAbove_tvDist_range (⟨A, hA⟩ : {A : Set Ω // MeasurableSet A}) -
measureReal_sub_le_tvDisttheorem — The signed gap ν.real A − μ.real A is at most tvDist μ ν.Proof (Lean source)
theorem measureReal_sub_le_tvDist {A : Set Ω} (hA : MeasurableSet A) : ν.real A - μ.real A ≤ tvDist μ ν := (le_abs_self _).trans <| by rw [abs_sub_comm]; exact abs_measureReal_sub_le_tvDist hA -
tvDist_nonnegtheorem — Total variation distance between probability measures is nonnegative.Proof (Lean source)
theorem tvDist_nonneg : 0 ≤ tvDist μ ν := by have := abs_measureReal_sub_le_tvDist (μ := μ) (ν := ν) MeasurableSet.empty simpa using (abs_nonneg _).trans this -
tvDist_le_onetheorem — Total variation distance between probability measures is at most one.Proof (Lean source)
theorem tvDist_le_one : tvDist μ ν ≤ 1 := ciSup_le fun A => abs_measureReal_sub_le_one A.1 -
tvDist_symmtheorem — tvDist is symmetric. -
tvDist_integral_rangetheorem — The expectation gap of a measurable statistic confined to an interval of width c is at most that width times the total-variation distance between the two probability laws.hypothesesconclusion|(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ tvDist μ ν * cProof (Lean source)
theorem tvDist_integral_range (μ ν : Measure Ω) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (f : Ω → ℝ) (hf : Measurable f) (a c : ℝ) (hc : 0 ≤ c) (hrange : ∀ x, f x ∈ Icc a (a + c)) : |(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ tvDist μ ν * c := by let g : Ω → ℝ := fun x => f x - a have hg : Measurable g := hf.sub_const a have hg0 : ∀ x, 0 ≤ g x := fun x => by dsimp [g] linarith [(hrange x).1] have hgc : ∀ x, g x ≤ c := fun x => by dsimp [g] linarith [(hrange x).2] have hgint (ρ : Measure Ω) [IsProbabilityMeasure ρ] : Integrable g ρ := Integrable.of_bound hg.aestronglyMeasurable c (Filter.Eventually.of_forall fun x => by rw [Real.norm_eq_abs, abs_of_nonneg (hg0 x)] exact hgc x) have htail_meas (ρ : Measure Ω) : Measurable (fun t : ℝ => ρ.real {x | t ≤ g x}) := by change Measurable fun t : ℝ => (ρ {x | t ≤ g x}).toReal exact Measurable.ennreal_toReal (Antitone.measurable (fun _ _ hst => measure_mono (fun _ hx => hst.trans hx))) have htail_int (ρ : Measure Ω) [IsProbabilityMeasure ρ] : IntegrableOn (fun t : ℝ => ρ.real {x | t ≤ g x}) (Ioc 0 c) := by exact Integrable.of_bound ((htail_meas ρ).aestronglyMeasurable.mono_measure Measure.restrict_le_self) 1 (Filter.Eventually.of_forall fun t => by rw [Real.norm_eq_abs, abs_of_nonneg measureReal_nonneg] exact measureReal_le_one) have hlayer (ρ : Measure Ω) [IsProbabilityMeasure ρ] : ∫ x, g x ∂ρ = ∫ t in Ioc 0 c, ρ.real {x | t ≤ g x} := by exact (hgint ρ).integral_eq_integral_Ioc_meas_le (Filter.Eventually.of_forall hg0) (Filter.Eventually.of_forall hgc) have hfint (ρ : Measure Ω) [IsProbabilityMeasure ρ] : Integrable f ρ := Integrable.of_bound hf.aestronglyMeasurable (|a| + c) (Filter.Eventually.of_forall fun x => by rw [Real.norm_eq_abs] calc |f x| = |a + g x| := by simp [g] _ ≤ |a| + |g x| := abs_add_le _ _ _ ≤ |a| + c := by gcongr rw [abs_of_nonneg (hg0 x)] exact hgc x) have hshift (ρ : Measure Ω) [IsProbabilityMeasure ρ] : ∫ x, g x ∂ρ = (∫ x, f x ∂ρ) - a := by rw [show g = fun x => f x - a from rfl, integral_sub (hfint ρ) (integrable_const a)] simp have heq : (∫ x, f x ∂μ) - ∫ x, f x ∂ν = (∫ x, g x ∂μ) - ∫ x, g x ∂ν := by rw [hshift μ, hshift ν] ring rw [heq, hlayer μ, hlayer ν, ← integral_sub (htail_int μ) (htail_int ν)] have hbound : ∀ᵐ t ∂volume.restrict (Ioc 0 c), ‖μ.real {x | t ≤ g x} - ν.real {x | t ≤ g x}‖ ≤ tvDist μ ν := by exact Filter.Eventually.of_forall fun t => by rw [Real.norm_eq_abs] exact abs_measureReal_sub_le_tvDist (hg measurableSet_Ici) calc |∫ t in Ioc 0 c, (μ.real {x | t ≤ g x} - ν.real {x | t ≤ g x})| = ‖∫ t in Ioc 0 c, (μ.real {x | t ≤ g x} - ν.real {x | t ≤ g x})‖ := by rw [Real.norm_eq_abs] _ ≤ tvDist μ ν * volume.real (Ioc 0 c) := norm_setIntegral_le_of_norm_le_const_ae (by simp) hbound _ = tvDist μ ν * c := by rw [measureReal_def, Real.volume_Ioc, ENNReal.toReal_ofReal (by linarith : 0 ≤ c - 0)] ring -
tvDist_integral_le_of_range_aetheorem — A measurable statistic that lies almost surely in an interval of width c under both laws has an expectation gap no larger than c times their total-variation distance.hypothesesμ ν :f :Ω → ℝhf :a c :ℝhc :0 ≤ chμ :∀ᵐ x ∂μ, f x ∈ Icc a (a + c)hν :∀ᵐ x ∂ν, f x ∈ Icc a (a + c)conclusion|(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ tvDist μ ν * cProof (Lean source)
theorem tvDist_integral_le_of_range_ae (μ ν : Measure Ω) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (f : Ω → ℝ) (hf : Measurable f) (a c : ℝ) (hc : 0 ≤ c) (hμ : ∀ᵐ x ∂μ, f x ∈ Icc a (a + c)) (hν : ∀ᵐ x ∂ν, f x ∈ Icc a (a + c)) : |(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ tvDist μ ν * c := by let f' : Ω → ℝ := fun x => max a (min (a + c) (f x)) have hf' : Measurable f' := measurable_const.max (measurable_const.min hf) have hac : a ≤ a + c := by linarith have hrange : ∀ x, f' x ∈ Icc a (a + c) := fun x => by constructor · exact le_max_left _ _ · exact max_le hac (min_le_left _ _) have hμ_eq : f' =ᵐ[μ] f := by filter_upwards [hμ] with x hx simp only [f', min_eq_right hx.2, max_eq_right hx.1] have hν_eq : f' =ᵐ[ν] f := by filter_upwards [hν] with x hx simp only [f', min_eq_right hx.2, max_eq_right hx.1] rw [← integral_congr_ae hμ_eq, ← integral_congr_ae hν_eq] exact tvDist_integral_range μ ν f' hf' a c hc hrange -
tvDist_integral_le_of_abs_le_aetheorem — A measurable statistic bounded in absolute value by M almost surely under both laws has an expectation gap no larger than 2M times their total-variation distance.hypothesesμ ν :f :Ω → ℝhf :M :ℝhM :0 ≤ Mhμ :∀ᵐ x ∂μ, |f x| ≤ Mhν :∀ᵐ x ∂ν, |f x| ≤ Mconclusion|(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ 2 * M * tvDist μ νProof (Lean source)
theorem tvDist_integral_le_of_abs_le_ae (μ ν : Measure Ω) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (f : Ω → ℝ) (hf : Measurable f) (M : ℝ) (hM : 0 ≤ M) (hμ : ∀ᵐ x ∂μ, |f x| ≤ M) (hν : ∀ᵐ x ∂ν, |f x| ≤ M) : |(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ 2 * M * tvDist μ ν := by have hμ_range : ∀ᵐ x ∂μ, f x ∈ Icc (-M) (-M + 2 * M) := by filter_upwards [hμ] with x hx have hx' := abs_le.mp hx constructor <;> linarith have hν_range : ∀ᵐ x ∂ν, f x ∈ Icc (-M) (-M + 2 * M) := by filter_upwards [hν] with x hx have hx' := abs_le.mp hx constructor <;> linarith calc |(∫ x, f x ∂μ) - ∫ x, f x ∂ν| ≤ tvDist μ ν * (2 * M) := tvDist_integral_le_of_range_ae μ ν f hf (-M) (2 * M) (by positivity) hμ_range hν_range _ = 2 * M * tvDist μ ν := by ring
ChiSquared 6 core · 10 supporting This module defines the real chi-squared divergence chiSqDiv from the squared deviation of the Radon-Nikodym density and proves its main minimax testing interfaces. ★ tvDist_le_half_sqrt_chiSqDiv★ one_add_chiSqDiv_pi_iid★ one_add_chiSqDiv_pi_iid_general★ testing_error_lower_of_chi★ le_cam_two_point_chisq
Chi-squared divergence
This module defines the real chi-squared divergence chiSqDiv from the squared
deviation of the Radon-Nikodym density and proves its main minimax testing
interfaces. The base results include chiSqDiv_nonneg,
tvDist_le_half_sqrt_chiSqDiv, and chiSqDiv_eq; the tensorization layer gives
chiSqDiv_prod, one_add_chiSqDiv_pi_iid,
one_add_chiSqDiv_pi_iid_general, and chiSqDiv_prod_ancillary; and the
testing layer culminates in testing_error_lower_of_chi and
le_cam_two_point_chisq.
def chiSqDiv reviewed
⚑ flagged
Causalean.Stat
For two measures on the same measurable space, the finite real chi-squared divergence of the first relative to the second is the integral, under the second measure, of the square of one less the Radon--Nikodym density of the first measure with respect to the second, with infinite density values represented by zero in the real-valued integrand.
theorem tvDist_le_half_sqrt_chiSqDiv reviewed
⚑ flagged
Causalean.Stat
Cauchy–Schwarz on Scheffé. For probability measures μ, ν with μ absolutely continuous with respect to ν and whose squared Radon–Nikodym density deviation (dμ/dν − 1)² is ν-integrable, the total variation distance tvDist μ ν is bounded by half the square root of the χ²-divergence chiSqDiv μ ν.
Formal statement
hypotheses
Proof (Lean source)
theorem tvDist_le_half_sqrt_chiSqDiv (μ ν : Measure Ω)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν)
(hint : Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) ν) :
tvDist μ ν ≤ (1/2) * sqrt (chiSqDiv μ ν) := by
-- Density deviation `f = p − 1`.
set f : Ω → ℝ := fun x => (μ.rnDeriv ν x).toReal - 1 with hf_def
have hmeas : AEStronglyMeasurable f ν := by
fun_prop
-- `f ∈ L²(ν)` since `f²` is integrable.
have hfL2 : MemLp f 2 ν := (memLp_two_iff_integrable_sq hmeas).2 hint
-- `|f| ∈ L²(ν)` (same norm) and `1 ∈ L²(ν)` (`ν` probability).
have habsL2 : MemLp (fun x => |f x|) (ofReal 2) ν := by
have h := hfL2.norm
simp only [Real.norm_eq_abs] at h
simpa using h
have honeL2 : MemLp (fun _ : Ω => (1:ℝ)) (ofReal 2) ν := by
simpa using (memLp_const (1:ℝ) : MemLp (fun _ : Ω => (1:ℝ)) 2 ν)
-- Hölder with `p = q = 2`: `∫ |f|·1 ≤ (∫ |f|²)^½ · (∫ 1²)^½`.
have hholder :
∫ x, |f x| * (1:ℝ) ∂ν
≤ (∫ x, |f x| ^ (2:ℝ) ∂ν) ^ (1 / (2:ℝ))
* (∫ x, (1:ℝ) ^ (2:ℝ) ∂ν) ^ (1 / (2:ℝ)) :=
integral_mul_le_Lp_mul_Lq_of_nonneg Real.HolderConjugate.two_two
(Filter.Eventually.of_forall fun x => abs_nonneg _)
(Filter.Eventually.of_forall fun _ => zero_le_one) habsL2 honeL2
-- Simplify the right-hand factors.
have hone : (∫ x, (1:ℝ) ^ (2:ℝ) ∂ν) ^ (1 / (2:ℝ)) = 1 := by
simp
have hLHS : ∫ x, |f x| * (1:ℝ) ∂ν = ∫ x, |f x| ∂ν := by simp
-- `∫ |f|² = chiSqDiv μ ν` (rewrite real power 2 to `^2`).
have hsq : ∫ x, |f x| ^ (2:ℝ) ∂ν = chiSqDiv μ ν := by
rw [chiSqDiv]
apply integral_congr_ae
refine Filter.Eventually.of_forall fun x => ?_
change |f x| ^ (2:ℝ) = (f x) ^ 2
rw [Real.rpow_two, sq_abs]
-- `∫ |f| ≤ √(chiSqDiv μ ν)`.
have hint_abs : ∫ x, |f x| ∂ν ≤ sqrt (chiSqDiv μ ν) := by
rw [hLHS, hsq, hone, mul_one] at hholder
have hrw : (chiSqDiv μ ν) ^ (1 / (2:ℝ)) = sqrt (chiSqDiv μ ν) :=
(Real.sqrt_eq_rpow (chiSqDiv μ ν)).symm
rwa [hrw] at hholder
-- Combine with Scheffé.
have hscheffe := tvDist_le_half_integral_abs_rnDeriv μ ν hac
calc tvDist μ ν ≤ (1/2) * ∫ x, |f x| ∂ν := hscheffe
_ ≤ (1/2) * sqrt (chiSqDiv μ ν) := by
apply mul_le_mul_of_nonneg_left hint_abs (by norm_num)
theorem one_add_chiSqDiv_pi_iid reviewed
⚑ flagged
Causalean.Stat
n-fold i.i.d. tensorization of the χ²-divergence on a finite sample space. For probability measures μ, ν on a finite space Ω with μ absolutely continuous with respect to ν, the χ²-divergence of the n-fold i.i.d. product laws satisfies 1 + χ²(μ^⊗n ‖ ν^⊗n) = (1 + χ²(μ‖ν))^n.
Formal statement
hypotheses
Ω :
μ ν :
hac :
μ ≪ ν
n :
ℕ
Proof (Lean source)
theorem one_add_chiSqDiv_pi_iid {Ω : Type*} [MeasurableSpace Ω] [Fintype Ω]
[MeasurableSingletonClass Ω] (μ ν : Measure Ω)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν) (n : ℕ) :
1 + chiSqDiv (Measure.pi (fun _ : Fin n => μ)) (Measure.pi (fun _ : Fin n => ν))
= (1 + chiSqDiv μ ν) ^ n := by
induction n with
| zero =>
rw [Measure.pi_of_empty (fun _ : Fin 0 => μ),
Measure.pi_of_empty (fun _ : Fin 0 => ν), chiSqDiv_self]
simp
| succ n ih =>
have hμ := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => μ) 0).map_eq
have hν := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => ν) 0).map_eq
rw [← chiSqDiv_map_measurableEquiv
(MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => Ω) 0), hμ, hν]
have hac_pi : Measure.pi (fun _ : Fin n => μ) ≪ Measure.pi (fun _ : Fin n => ν) :=
pi_iid_absolutelyContinuous μ ν hac n
rw [chiSqDiv_prod μ ν (Measure.pi (fun _ : Fin n => μ)) (Measure.pi (fun _ : Fin n => ν))
hac hac_pi
(Integrable.of_finite) (Integrable.of_finite), ih, pow_succ]
ring
theorem one_add_chiSqDiv_pi_iid_general reviewed
⚑ flagged
Causalean.Stat
n-fold i.i.d. tensorization of the χ²-divergence on a GENERAL measurable space. For probability measures μ, ν on any measurable space S such that μ is absolutely continuous with respect to ν and the single-sample squared density deviation (dμ/dν − 1)² is ν-integrable, the χ²-divergence of the n-fold i.i.d. product tensorizes multiplicatively: 1 + χ²(μ^⊗n ‖ ν^⊗n) = (1 + χ²(μ‖ν))^n.
Formal statement
hypotheses
Proof (Lean source)
theorem one_add_chiSqDiv_pi_iid_general {S : Type*} [MeasurableSpace S] (μ ν : Measure S)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν)
(hint : Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) ν) (n : ℕ) :
1 + chiSqDiv (Measure.pi (fun _ : Fin n => μ)) (Measure.pi (fun _ : Fin n => ν))
= (1 + chiSqDiv μ ν) ^ n := by
induction n with
| zero =>
rw [Measure.pi_of_empty (fun _ : Fin 0 => μ),
Measure.pi_of_empty (fun _ : Fin 0 => ν), chiSqDiv_self]
simp
| succ n ih =>
have hμ := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => μ) 0).map_eq
have hν := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => ν) 0).map_eq
rw [← chiSqDiv_map_measurableEquiv
(MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => S) 0), hμ, hν]
have hac_pi : Measure.pi (fun _ : Fin n => μ) ≪ Measure.pi (fun _ : Fin n => ν) :=
pi_iid_absolutelyContinuous μ ν hac n
rw [chiSqDiv_prod μ ν (Measure.pi (fun _ : Fin n => μ)) (Measure.pi (fun _ : Fin n => ν))
hac hac_pi hint (pi_iid_integrable_sq_dev μ ν hac hint n), ih, pow_succ]
ring
lemma testing_error_lower_of_chi reviewed
⚑ flagged
Causalean.Stat
Two-point testing-error floor from a finite χ²-budget. For probability measures P, Q such that P is absolutely continuous with respect to Q and the squared density deviation (dP/dQ − 1)² is Q-integrable, if C is nonnegative and the χ²-divergence chiSqDiv P Q is at most C, then for every measurable test region A, the combined testing error is at least 1/(4(C + 1)): P(Aᶜ) + Q(A) ≥ 1/(4(C + 1)). This is the positive two-point testing floor that powers χ²-budget minimax lower bounds — no test can separate P from Q better than this when their χ²-divergence is bounded.
Formal statement
hypotheses
Ω :
Type*
P Q :
hac :
P ≪ Q
hint :
Integrable (fun x => ((P.rnDeriv Q x).toReal - 1) ^ 2) Q
C :
ℝ
hC :
0 ≤ C
hchi :
chiSqDiv P Q ≤ C
A :
Set Ω
hA :
conclusion
P.real Aᶜ + Q.real A ≥ 1 / (4 * (C + 1))
Proof (Lean source)
lemma testing_error_lower_of_chi {Ω : Type*} [MeasurableSpace Ω]
(P Q : Measure Ω) [IsProbabilityMeasure P] [IsProbabilityMeasure Q]
(hac : P ≪ Q)
(hint : Integrable (fun x => ((P.rnDeriv Q x).toReal - 1) ^ 2) Q)
{C : ℝ} (hC : 0 ≤ C) (hchi : chiSqDiv P Q ≤ C)
{A : Set Ω} (hA : MeasurableSet A) :
P.real Aᶜ + Q.real A ≥ 1 / (4 * (C + 1)) := by
let e : ℝ := P.real Aᶜ + Q.real A
let K : ℝ := C + 1
have hKpos : 0 < K := by
dsimp [K]
linarith
have hKge1 : 1 ≤ K := by
dsimp [K]
linarith
have hQ_nonneg : 0 ≤ Q.real A := measureReal_nonneg
have hPcomp_nonneg : 0 ≤ P.real Aᶜ := measureReal_nonneg
have hcs : P.real A ≤ sqrt (K * Q.real A) := by
simpa [K] using rnDeriv_setIntegral_le_sqrt_chi P Q hac hint hchi hA
have hcomp : P.real Aᶜ = 1 - P.real A := by
rw [measureReal_compl hA, probReal_univ]
have hgap : 1 - e ≤ sqrt (K * Q.real A) := by
dsimp [e]
rw [hcomp]
linarith [hcs, hQ_nonneg]
by_contra hnot
have hlt : e < 1 / (4 * K) := by
exact lt_of_not_ge (by simpa [e, K] using hnot)
have hQ_le_e : Q.real A ≤ e := by
dsimp [e]
linarith
have hQ_lt : Q.real A < 1 / (4 * K) := lt_of_le_of_lt hQ_le_e hlt
have hKQ_lt : K * Q.real A < 1 / 4 := by
calc
K * Q.real A < K * (1 / (4 * K)) := mul_lt_mul_of_pos_left hQ_lt hKpos
_ = 1 / 4 := by field_simp [hKpos.ne']
have hsqrt_lt : sqrt (K * Q.real A) < 1 / 2 := by
rw [Real.sqrt_lt' (by norm_num : (0 : ℝ) < 1 / 2)]
norm_num
exact hKQ_lt
have hc_le_quarter : 1 / (4 * K) ≤ 1 / 4 := by
have hden : 0 < 4 * K := mul_pos (by norm_num) hKpos
rw [one_div_le_one_div hden (by norm_num : (0 : ℝ) < 4)]
nlinarith
have he_lt_quarter : e < 1 / 4 := lt_of_lt_of_le hlt hc_le_quarter
have hleft_gt : 1 / 2 < 1 - e := by
linarith
have hleft_lt : 1 - e < 1 / 2 := lt_of_le_of_lt hgap hsqrt_lt
linarith
lemma le_cam_two_point_chisq reviewed
⚑ flagged
Causalean.Stat
Le Cam two-point testing floor under a χ²-budget (uniform-constant form). Two statements share this structure: for every χ²-budget C ≥ 0 there is a single floor constant c > 0, fixed before the laws are chosen, such that every pair of probability laws with Q-integrable squared density deviation and χ²-divergence at most C has combined testing error P(Aᶜ) + Q(A) at least c on every measurable test region A; and for every per-observation χ²-budget c0 ≥ 0 there is a single floor c' > 0 such that whenever the single-observation χ²-divergence is at most c0/m, the m-fold i.i.d. product experiment retains that same constant testing floor c', by the product χ²-identity 1 + χ²(P^{⊗m}‖Q^{⊗m}) = (1 + χ²)^m ≤ exp(c0).
Formal statement
conclusion
conclusion 1
Cchi :
ℝ
0 ≤ Cchi
∃ c : ℝ,
0 < c
Ω :
Type
Pp Pm :
Measure Ω
Pp ≪ Pm
Integrable (fun x => ((Pp.rnDeriv Pm x).toReal - 1) ^ 2) Pm
chiSqDiv Pp Pm ≤ Cchi
A :
Set Ω
Pp.real Aᶜ + Pm.real A ≥ c
conclusion 2
c0 :
ℝ
0 ≤ c0
∃ c' : ℝ,
0 < c'
Ω :
Type
Pp Pm :
Measure Ω
Pp ≪ Pm
Integrable (fun x => ((Pp.rnDeriv Pm x).toReal - 1) ^ 2) Pm
m :
ℕ
0 < m
chiSqDiv Pp Pm ≤ c0 / (m : ℝ)
Proof (Lean source)
lemma le_cam_two_point_chisq :
(∀ Cchi : ℝ, 0 ≤ Cchi →
∃ c : ℝ, 0 < c ∧
∀ {Ω : Type} [MeasurableSpace Ω]
(Pp Pm : Measure Ω) [IsProbabilityMeasure Pp] [IsProbabilityMeasure Pm],
Pp ≪ Pm →
Integrable (fun x => ((Pp.rnDeriv Pm x).toReal - 1) ^ 2) Pm →
chiSqDiv Pp Pm ≤ Cchi →
∀ A : Set Ω, MeasurableSet A → Pp.real Aᶜ + Pm.real A ≥ c) ∧
(∀ c0 : ℝ, 0 ≤ c0 →
∃ c' : ℝ, 0 < c' ∧
∀ {Ω : Type} [MeasurableSpace Ω]
(Pp Pm : Measure Ω) [IsProbabilityMeasure Pp] [IsProbabilityMeasure Pm],
Pp ≪ Pm →
Integrable (fun x => ((Pp.rnDeriv Pm x).toReal - 1) ^ 2) Pm →
∀ (m : ℕ), 0 < m →
chiSqDiv Pp Pm ≤ c0 / (m : ℝ) →
∀ A : Set (Fin m → Ω), MeasurableSet A →
(Measure.pi (fun _ : Fin m => Pp)).real Aᶜ
+ (Measure.pi (fun _ : Fin m => Pm)).real A ≥ c')
:= by
constructor
· intro Cchi hCchi
refine ⟨1 / (4 * (Cchi + 1)), by positivity, ?_⟩
intro Ω _ Pp Pm _ _ hac hint hchi A hA
exact testing_error_lower_of_chi Pp Pm hac hint hCchi hchi hA
· intro c0 hc0
refine ⟨1 / (4 * exp c0), by positivity, ?_⟩
intro Ω _ Pp Pm _ _ hac hint m hm hchi A hA
let Pprod : Measure (Fin m → Ω) := Measure.pi (fun _ : Fin m => Pp)
let Qprod : Measure (Fin m → Ω) := Measure.pi (fun _ : Fin m => Pm)
have hac_prod : Pprod ≪ Qprod := by
dsimp [Pprod, Qprod]
exact pi_iid_absolutelyContinuous Pp Pm hac m
have hint_prod :
Integrable (fun x => ((Pprod.rnDeriv Qprod x).toReal - 1) ^ 2) Qprod := by
dsimp [Pprod, Qprod]
exact pi_iid_integrable_sq_dev Pp Pm hac hint m
have hchi_non : 0 ≤ chiSqDiv Pp Pm :=
chiSqDiv_nonneg
have hmR : 0 < (m : ℝ) := by exact_mod_cast hm
have hmul : (m : ℝ) * chiSqDiv Pp Pm ≤ c0 := by
have := mul_le_mul_of_nonneg_left hchi (le_of_lt hmR)
field_simp [hmR.ne'] at this
exact this
have hpow :
(1 + chiSqDiv Pp Pm) ^ m ≤
exp ((m : ℝ) * chiSqDiv Pp Pm) := by
calc
(1 + chiSqDiv Pp Pm) ^ m
≤ (exp (chiSqDiv Pp Pm)) ^ m := by
exact pow_le_pow_left₀ (by linarith)
(by
linarith [Real.add_one_le_exp (chiSqDiv Pp Pm)])
m
_ = exp ((m : ℝ) * chiSqDiv Pp Pm) := by
rw [← Real.exp_nat_mul]
have hexp_le :
exp ((m : ℝ) * chiSqDiv Pp Pm) ≤ exp c0 := by
exact Real.exp_le_exp.mpr hmul
have hprod_eq :
1 + chiSqDiv Pprod Qprod =
(1 + chiSqDiv Pp Pm) ^ m := by
dsimp [Pprod, Qprod]
exact one_add_chiSqDiv_pi_iid_general Pp Pm hac hint m
have hchi_prod : chiSqDiv Pprod Qprod ≤ exp c0 - 1 := by
have hone : 1 + chiSqDiv Pprod Qprod ≤ exp c0 := by
rw [hprod_eq]
exact hpow.trans hexp_le
linarith
have hCprod_nonneg : 0 ≤ exp c0 - 1 := by
have h1 : 1 ≤ exp c0 := by
have := Real.add_one_le_exp c0
linarith
linarith
have htest :=
testing_error_lower_of_chi Pprod Qprod hac_prod hint_prod hCprod_nonneg hchi_prod hA
have hconst : 1 / (4 * ((exp c0 - 1) + 1)) = 1 / (4 * exp c0) := by
ring_nf
simpa [Pprod, Qprod, hconst] using htest
10 supporting declarations (lemmas, instances)
-
chiSqDiv_eqtheorem — Expansion of the χ²-divergence: χ²(μ‖ν) = ∫ (dμ/dν)² ∂ν − 1 when μ ≪ ν.hypotheseshac :μ ≪ νhint :Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) νProof (Lean source)
theorem chiSqDiv_eq [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν) (hint : Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) ν) : chiSqDiv μ ν = (∫ x, ((μ.rnDeriv ν x).toReal) ^ 2 ∂ν) - 1 := by set p : Ω → ℝ := fun x => (μ.rnDeriv ν x).toReal with hp_def have hmeas : AEStronglyMeasurable p ν := by fun_prop have hp_int : Integrable p ν := Measure.integrable_toReal_rnDeriv -- `p²` integrable, since `(p − 1)²` is and `p² = (p−1)² + 2p − 1`. have hp_sq : Integrable (fun x => p x ^ 2) ν := by have hexp : (fun x => p x ^ 2) = fun x => (p x - 1) ^ 2 + (2 * p x - 1) := by funext x; ring rw [hexp] exact hint.add ((hp_int.const_mul 2).sub (integrable_const 1)) -- Expand `(p − 1)² = p² − 2p + 1` under the integral. have h2p : Integrable (fun x => 2 * p x) ν := hp_int.const_mul 2 have hsub : ∫ x, (p x ^ 2 - 2 * p x) ∂ν = (∫ x, p x ^ 2 ∂ν) - 2 * (∫ x, p x ∂ν) := by integral_linearity have hadd : ∫ x, ((p x ^ 2 - 2 * p x) + 1) ∂ν = (∫ x, (p x ^ 2 - 2 * p x) ∂ν) + (∫ _ : Ω, (1:ℝ) ∂ν) := by integral_linearity have hexp : chiSqDiv μ ν = (∫ x, p x ^ 2 ∂ν) - 2 * (∫ x, p x ∂ν) + (∫ _ : Ω, (1:ℝ) ∂ν) := by have hcongr : ∀ x, ((μ.rnDeriv ν x).toReal - 1) ^ 2 = (p x ^ 2 - 2 * p x) + 1 := by intro x; rw [hp_def]; ring rw [chiSqDiv, integral_congr_ae (Filter.Eventually.of_forall hcongr), hadd, hsub] -- `∫ p ∂ν = 1` and `∫ 1 ∂ν = 1`. have hp1 : ∫ x, p x ∂ν = 1 := by rw [hp_def, Measure.integral_toReal_rnDeriv hac] rw [measureReal_def, measure_univ]; simp have hone : (∫ _ : Ω, (1:ℝ) ∂ν) = 1 := by simp rw [hexp, hp1, hone]; ring -
rnDeriv_prod_eqtheorem — Product density factorization. For μ₁ ≪ ν₁ and μ₂ ≪ ν₂, the Radon–Nikodym density of the product is (a.e.) the product of the marginal densities.hypothesesμ₁ ν₁ :Measure αμ₂ ν₂ :Measure βSigmaFinite ν₁SigmaFinite ν₂SFinite ν₂μ₁.HaveLebesgueDecomposition ν₁μ₂.HaveLebesgueDecomposition ν₂h₁ :μ₁ ≪ ν₁h₂ :μ₂ ≪ ν₂conclusion(μ₁.prod μ₂).rnDeriv (ν₁.prod ν₂)=ᵐ[ν₁.prod ν₂] fun z => μ₁.rnDeriv ν₁ z.1 * μ₂.rnDeriv ν₂ z.2Proof (Lean source)
theorem rnDeriv_prod_eq (μ₁ ν₁ : Measure α) (μ₂ ν₂ : Measure β) [SigmaFinite ν₁] [SigmaFinite ν₂] [SFinite ν₂] [μ₁.HaveLebesgueDecomposition ν₁] [μ₂.HaveLebesgueDecomposition ν₂] (h₁ : μ₁ ≪ ν₁) (h₂ : μ₂ ≪ ν₂) : (μ₁.prod μ₂).rnDeriv (ν₁.prod ν₂) =ᵐ[ν₁.prod ν₂] fun z => μ₁.rnDeriv ν₁ z.1 * μ₂.rnDeriv ν₂ z.2 := by -- Rewrite `μᵢ = νᵢ.withDensity (rnDeriv μᵢ νᵢ)` and use `prod_withDensity₀`. have hfac : μ₁.prod μ₂ = (ν₁.prod ν₂).withDensity (fun z => μ₁.rnDeriv ν₁ z.1 * μ₂.rnDeriv ν₂ z.2) := by conv_lhs => rw [← Measure.withDensity_rnDeriv_eq _ _ h₁, ← Measure.withDensity_rnDeriv_eq _ _ h₂] exact prod_withDensity₀ (Measure.measurable_rnDeriv _ _).aemeasurable (Measure.measurable_rnDeriv _ _).aemeasurable calc (μ₁.prod μ₂).rnDeriv (ν₁.prod ν₂) =ᵐ[ν₁.prod ν₂] ((ν₁.prod ν₂).withDensity (fun z => μ₁.rnDeriv ν₁ z.1 * μ₂.rnDeriv ν₂ z.2)).rnDeriv (ν₁.prod ν₂) := by rw [hfac] _ =ᵐ[ν₁.prod ν₂] fun z => μ₁.rnDeriv ν₁ z.1 * μ₂.rnDeriv ν₂ z.2 := Measure.rnDeriv_withDensity₀ _ (((Measure.measurable_rnDeriv _ _).comp measurable_fst).mul ((Measure.measurable_rnDeriv _ _).comp measurable_snd)).aemeasurable -
chiSqDiv_prodtheorem — Binary tensorization of the χ²-divergence. For probability measures with μ₁ ≪ ν₁ and μ₂ ≪ ν₂ and integrable squared density deviations, the χ²-divergence tensorizes multiplicatively: 1 + χ²(μ₁⊗μ₂ ‖ ν₁⊗ν₂) = (1 + χ²(μ₁‖ν₁))·(1 + χ²(μ₂‖ν₂)).hypothesesμ₁ ν₁ :Measure αμ₂ ν₂ :Measure βh₁ :μ₁ ≪ ν₁h₂ :μ₂ ≪ ν₂hint₁ :Integrable (fun x => ((μ₁.rnDeriv ν₁ x).toReal - 1) ^ 2) ν₁hint₂ :Integrable (fun y => ((μ₂.rnDeriv ν₂ y).toReal - 1) ^ 2) ν₂Proof (Lean source)
theorem chiSqDiv_prod (μ₁ ν₁ : Measure α) (μ₂ ν₂ : Measure β) [IsProbabilityMeasure μ₁] [IsProbabilityMeasure ν₁] [IsProbabilityMeasure μ₂] [IsProbabilityMeasure ν₂] (h₁ : μ₁ ≪ ν₁) (h₂ : μ₂ ≪ ν₂) (hint₁ : Integrable (fun x => ((μ₁.rnDeriv ν₁ x).toReal - 1) ^ 2) ν₁) (hint₂ : Integrable (fun y => ((μ₂.rnDeriv ν₂ y).toReal - 1) ^ 2) ν₂) : 1 + chiSqDiv (μ₁.prod μ₂) (ν₁.prod ν₂) = (1 + chiSqDiv μ₁ ν₁) * (1 + chiSqDiv μ₂ ν₂) := by -- Marginal densities (as ℝ) and their squares. set p₁ : α → ℝ := fun x => (μ₁.rnDeriv ν₁ x).toReal with hp₁_def set p₂ : β → ℝ := fun y => (μ₂.rnDeriv ν₂ y).toReal with hp₂_def -- `pᵢ` integrable; `pᵢ²` integrable (from `(pᵢ−1)²` integrable). have hp₁_int : Integrable p₁ ν₁ := Measure.integrable_toReal_rnDeriv have hp₂_int : Integrable p₂ ν₂ := Measure.integrable_toReal_rnDeriv have hp₁_sq : Integrable (fun x => p₁ x ^ 2) ν₁ := by have hexp : (fun x => p₁ x ^ 2) = fun x => (p₁ x - 1) ^ 2 + (2 * p₁ x - 1) := by funext x; ring rw [hexp]; exact hint₁.add ((hp₁_int.const_mul 2).sub (integrable_const 1)) have hp₂_sq : Integrable (fun y => p₂ y ^ 2) ν₂ := by have hexp : (fun y => p₂ y ^ 2) = fun y => (p₂ y - 1) ^ 2 + (2 * p₂ y - 1) := by funext y; ring rw [hexp]; exact hint₂.add ((hp₂_int.const_mul 2).sub (integrable_const 1)) -- Product squared density (as ℝ), a.e. equal to `p₁(z.1)²·p₂(z.2)²`. set P : α × β → ℝ := fun z => ((μ₁.prod μ₂).rnDeriv (ν₁.prod ν₂) z).toReal with hP_def have hdens : (μ₁.prod μ₂).rnDeriv (ν₁.prod ν₂) =ᵐ[ν₁.prod ν₂] fun z => μ₁.rnDeriv ν₁ z.1 * μ₂.rnDeriv ν₂ z.2 := rnDeriv_prod_eq μ₁ ν₁ μ₂ ν₂ h₁ h₂ have hPeq : (fun z => P z ^ 2) =ᵐ[ν₁.prod ν₂] fun z => (p₁ z.1 ^ 2) * (p₂ z.2 ^ 2) := by filter_upwards [hdens] with z hz rw [hP_def] simp only [hz, ENNReal.toReal_mul, hp₁_def, hp₂_def] ring -- Product of squared densities is integrable over the product measure. have hPint : Integrable (fun z => (p₁ z.1 ^ 2) * (p₂ z.2 ^ 2)) (ν₁.prod ν₂) := Integrable.mul_prod hp₁_sq hp₂_sq have hPint' : Integrable (fun z => P z ^ 2) (ν₁.prod ν₂) := hPint.congr hPeq.symm -- `(P−1)²` integrable over the product (needed for `chiSqDiv_eq`). have hP_int : Integrable P (ν₁.prod ν₂) := Measure.integrable_toReal_rnDeriv have hPdev : Integrable (fun z => (P z - 1) ^ 2) (ν₁.prod ν₂) := by have hexp : (fun z => (P z - 1) ^ 2) = fun z => P z ^ 2 + (-(2 * P z) + 1) := by funext z; ring rw [hexp]; exact hPint'.add (((hP_int.const_mul 2).neg).add (integrable_const 1)) -- Apply the expansion `χ² = ∫ p² − 1` to all three. have hprod_ac : μ₁.prod μ₂ ≪ ν₁.prod ν₂ := h₁.prod h₂ have e0 : chiSqDiv (μ₁.prod μ₂) (ν₁.prod ν₂) = (∫ z, P z ^ 2 ∂(ν₁.prod ν₂)) - 1 := chiSqDiv_eq hprod_ac hPdev have e1 : chiSqDiv μ₁ ν₁ = (∫ x, p₁ x ^ 2 ∂ν₁) - 1 := chiSqDiv_eq h₁ hint₁ have e2 : chiSqDiv μ₂ ν₂ = (∫ y, p₂ y ^ 2 ∂ν₂) - 1 := chiSqDiv_eq h₂ hint₂ -- Fubini: `∫ P² = (∫ p₁²)·(∫ p₂²)`. have hfubini : (∫ z, P z ^ 2 ∂(ν₁.prod ν₂)) = (∫ x, p₁ x ^ 2 ∂ν₁) * (∫ y, p₂ y ^ 2 ∂ν₂) := by rw [integral_congr_ae hPeq] exact integral_prod_mul (fun x => p₁ x ^ 2) (fun y => p₂ y ^ 2) rw [e0, e1, e2, hfubini]; ring -
chiSqDiv_map_measurableEquivtheorem — χ²-divergence is invariant under a measurable equivalence. Pushing both measures forward through e : Ω ≃ᵐ Ω' leaves the χ²-divergence unchanged.hypothesesΩ Ω' :e :Ω ≃ᵐ Ω'μ ν :Proof (Lean source)
theorem chiSqDiv_map_measurableEquiv {Ω Ω' : Type*} [MeasurableSpace Ω] [MeasurableSpace Ω'] (e : Ω ≃ᵐ Ω') (μ ν : Measure Ω) [SigmaFinite μ] [SigmaFinite ν] : chiSqDiv (μ.map e) (ν.map e) = chiSqDiv μ ν := by rw [chiSqDiv, chiSqDiv] rw [integral_map_equiv e (fun y => (((μ.map e).rnDeriv (ν.map e) y).toReal - 1) ^ 2)] apply integral_congr_ae have hrn := e.measurableEmbedding.rnDeriv_map μ ν filter_upwards [hrn] with x hx simp only [hx] -
chiSqDiv_selftheorem — The χ²-divergence of a (sigma-finite) measure against itself is zero.Proof (Lean source)
theorem chiSqDiv_self {Ω : Type*} [MeasurableSpace Ω] (ρ : Measure Ω) [SigmaFinite ρ] : chiSqDiv ρ ρ = 0 := by rw [chiSqDiv] rw [integral_eq_zero_of_ae] filter_upwards [ρ.rnDeriv_self] with x hx simp [hx] -
pi_iid_absolutelyContinuoustheorem — The n-fold product of μ is absolutely continuous w.r.t. that of ν whenever μ ≪ ν (for sigma-finite factors). Proved by induction via the piFinSuccAbove equivalence and the binary AbsolutelyContinuous.prod.hypothesesProof (Lean source)
theorem pi_iid_absolutelyContinuous {Ω : Type*} [MeasurableSpace Ω] (μ ν : Measure Ω) [SigmaFinite μ] [SigmaFinite ν] (hac : μ ≪ ν) (n : ℕ) : Measure.pi (fun _ : Fin n => μ) ≪ Measure.pi (fun _ : Fin n => ν) := by induction n with | zero => rw [Measure.pi_of_empty (fun _ : Fin 0 => μ), Measure.pi_of_empty (fun _ : Fin 0 => ν)] | succ n ih => set e := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => Ω) 0 with he have hμ := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => μ) 0).map_eq have hν := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => ν) 0).map_eq have hprod : μ.prod (Measure.pi (fun _ : Fin n => μ)) ≪ ν.prod (Measure.pi (fun _ : Fin n => ν)) := hac.prod ih have hmap : (Measure.pi (fun _ : Fin (n + 1) => μ)).map e ≪ (Measure.pi (fun _ : Fin (n + 1) => ν)).map e := by rw [hμ, hν]; exact hprod have hmapped := hmap.map (f := e.symm) e.symm.measurable rwa [Measure.map_map e.symm.measurable e.measurable, MeasurableEquiv.symm_comp_self, Measure.map_id, Measure.map_map e.symm.measurable e.measurable, MeasurableEquiv.symm_comp_self, Measure.map_id] at hmapped -
pi_iid_integrable_sq_devtheorem — Integrability of the squared density deviation (dμ/dν − 1)² propagates from one sample to the n-fold i.i.d. product (d(μ^⊗n)/d(ν^⊗n) − 1)². On a general (possibly continuous) measurable space this is the side-condition that makes chiSqDiv_prod applicable inside the tensorization induction — it is free on a finite space (Integrable.of_finite) but must be derived here, by L²(ν)-tensorization of the single-sample density dμ/dν.hypothesesconclusionIntegrable (fun x => (((Measure.pi (fun _ : Fin n => μ)).rnDeriv (Measure.pi (fun _ : Fin n => ν)) x).toReal - 1) ^ 2) (Measure.pi (fun _ : Fin n => ν))Proof (Lean source)
theorem pi_iid_integrable_sq_dev {S : Type*} [MeasurableSpace S] (μ ν : Measure S) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν) (hint : Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) ν) (n : ℕ) : Integrable (fun x => (((Measure.pi (fun _ : Fin n => μ)).rnDeriv (Measure.pi (fun _ : Fin n => ν)) x).toReal - 1) ^ 2) (Measure.pi (fun _ : Fin n => ν)) := by induction n with | zero => rw [Measure.pi_of_empty (fun _ : Fin 0 => μ), Measure.pi_of_empty (fun _ : Fin 0 => ν)] exact Integrable.of_finite | succ n ih => set μπ : Measure (Fin n → S) := Measure.pi (fun _ : Fin n => μ) with hμπ_def set νπ : Measure (Fin n → S) := Measure.pi (fun _ : Fin n => ν) with hνπ_def set e := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => S) 0 with he set p : S → ℝ := fun x => (μ.rnDeriv ν x).toReal with hp_def set pn : (Fin n → S) → ℝ := fun y => (μπ.rnDeriv νπ y).toReal with hpn_def have hp_int : Integrable p ν := by simpa [hp_def] using (Measure.integrable_toReal_rnDeriv : Integrable (fun x => (μ.rnDeriv ν x).toReal) ν) have hpn_int : Integrable pn νπ := by simpa [hpn_def] using (Measure.integrable_toReal_rnDeriv : Integrable (fun y => (μπ.rnDeriv νπ y).toReal) νπ) have hp_sq : Integrable (fun x => p x ^ 2) ν := by have hdev : Integrable (fun x => (p x - 1) ^ 2) ν := by simpa [hp_def] using hint have hexp : (fun x => p x ^ 2) = fun x => (p x - 1) ^ 2 + (2 * p x - 1) := by funext x; ring rw [hexp]; exact hdev.add ((hp_int.const_mul 2).sub (integrable_const 1)) have hpn_sq : Integrable (fun y => pn y ^ 2) νπ := by have hdev : Integrable (fun y => (pn y - 1) ^ 2) νπ := by simpa [hμπ_def, hνπ_def, hpn_def] using ih have hexp : (fun y => pn y ^ 2) = fun y => (pn y - 1) ^ 2 + (2 * pn y - 1) := by funext y; ring rw [hexp]; exact hdev.add ((hpn_int.const_mul 2).sub (integrable_const 1)) have hac_pi : μπ ≪ νπ := by rw [hμπ_def, hνπ_def] exact pi_iid_absolutelyContinuous μ ν hac n set P : S × (Fin n → S) → ℝ := fun z => ((μ.prod μπ).rnDeriv (ν.prod νπ) z).toReal with hP_def have hdens : (μ.prod μπ).rnDeriv (ν.prod νπ) =ᵐ[ν.prod νπ] fun z => μ.rnDeriv ν z.1 * μπ.rnDeriv νπ z.2 := rnDeriv_prod_eq μ ν μπ νπ hac hac_pi have hPeq : (fun z => P z ^ 2) =ᵐ[ν.prod νπ] fun z => (p z.1 ^ 2) * (pn z.2 ^ 2) := by filter_upwards [hdens] with z hz rw [hP_def] simp only [hz, ENNReal.toReal_mul, hp_def, hpn_def] ring have hPint : Integrable (fun z => (p z.1 ^ 2) * (pn z.2 ^ 2)) (ν.prod νπ) := Integrable.mul_prod hp_sq hpn_sq have hPint' : Integrable (fun z => P z ^ 2) (ν.prod νπ) := hPint.congr hPeq.symm have hP_int : Integrable P (ν.prod νπ) := by simpa [hP_def] using (Measure.integrable_toReal_rnDeriv : Integrable (fun z => ((μ.prod μπ).rnDeriv (ν.prod νπ) z).toReal) (ν.prod νπ)) have hprod_int : Integrable (fun z => (P z - 1) ^ 2) (ν.prod νπ) := by have hexp : (fun z => (P z - 1) ^ 2) = fun z => P z ^ 2 + (-(2 * P z) + 1) := by funext z; ring rw [hexp]; exact hPint'.add (((hP_int.const_mul 2).neg).add (integrable_const 1)) have hμ_map := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => μ) 0).map_eq have hν_map := (measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => ν) 0).map_eq have hprod_map : Integrable (fun z => (P z - 1) ^ 2) (Measure.map e (Measure.pi (fun _ : Fin (n + 1) => ν))) := by rw [hν_map] simpa [hνπ_def] using hprod_int have hcomp : Integrable ((fun z => (P z - 1) ^ 2) ∘ e) (Measure.pi (fun _ : Fin (n + 1) => ν)) := (integrable_map_equiv e (fun z => (P z - 1) ^ 2)).1 hprod_map refine hcomp.congr ?_ have hrn := e.measurableEmbedding.rnDeriv_map (Measure.pi (fun _ : Fin (n + 1) => μ)) (Measure.pi (fun _ : Fin (n + 1) => ν)) filter_upwards [hrn] with x hx change (P (e x) - 1) ^ 2 = ((((Measure.pi (fun _ : Fin (n + 1) => μ)).rnDeriv (Measure.pi (fun _ : Fin (n + 1) => ν)) x).toReal - 1) ^ 2) rw [hP_def, ← hμ_map, ← hν_map, ← he] simp only [hx] -
chiSqDiv_prod_ancillarytheorem — Ancillary product factor leaves the χ²-divergence unchanged. Tensoring both measures with a *common* probability measure ρ (an ancillary coordinate, whose law is the same under μ and ν) does not change the χ²-divergence: χ²(μ⊗ρ ‖ ν⊗ρ) = χ²(μ‖ν). This is the formal content of "an ancillary observation carries no information": its contribution is 1 + χ²(ρ‖ρ) = 1. It is the bridge from the finite-cell lower bound to the continuous-covariate one (the within-cell position is ancillary).hypothesesα β :μ ν :Measure αρ :Measure βhac :μ ≪ νhint :Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) νProof (Lean source)
theorem chiSqDiv_prod_ancillary {α β : Type*} [MeasurableSpace α] [MeasurableSpace β] (μ ν : Measure α) (ρ : Measure β) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] [IsProbabilityMeasure ρ] (hac : μ ≪ ν) (hint : Integrable (fun x => ((μ.rnDeriv ν x).toReal - 1) ^ 2) ν) : chiSqDiv (μ.prod ρ) (ν.prod ρ) = chiSqDiv μ ν := by have hint₂ : Integrable (fun y => ((ρ.rnDeriv ρ y).toReal - 1) ^ 2) ρ := by refine (integrable_zero β ℝ ρ).congr ?_ filter_upwards [ρ.rnDeriv_self] with y hy simp [hy] have h := chiSqDiv_prod μ ν ρ ρ hac (Measure.AbsolutelyContinuous.refl ρ) hint hint₂ rw [chiSqDiv_self] at h simp only [add_zero, mul_one] at h linarith -
rnDeriv_setIntegral_le_sqrt_chilemma — Cauchy–Schwarz mass transfer under a χ²-budget. For probability measures P ≪ Q whose squared density deviation (dP/dQ − 1)² is Q-integrable, and whose χ²-divergence is at most C, the P-mass of any measurable set A is controlled by its Q-mass through P(A) ≤ √((C + 1)·Q(A)). This is the Cauchy–Schwarz step underlying the two-point testing floor: a set that is small under the reference measure Q cannot be large under P when the χ²-budget is finite.hypothesesΩ :Type*P Q :hac :P ≪ Qhint :Integrable (fun x => ((P.rnDeriv Q x).toReal - 1) ^ 2) QC :ℝhchi :chiSqDiv P Q ≤ CA :Set ΩhA :conclusionP.real A ≤ sqrt ((C + 1) * Q.real A)Proof (Lean source)
lemma rnDeriv_setIntegral_le_sqrt_chi {Ω : Type*} [MeasurableSpace Ω] (P Q : Measure Ω) [IsProbabilityMeasure P] [IsProbabilityMeasure Q] (hac : P ≪ Q) (hint : Integrable (fun x => ((P.rnDeriv Q x).toReal - 1) ^ 2) Q) {C : ℝ} (hchi : chiSqDiv P Q ≤ C) {A : Set Ω} (hA : MeasurableSet A) : P.real A ≤ sqrt ((C + 1) * Q.real A) := by set p : Ω → ℝ := fun x => (P.rnDeriv Q x).toReal with hp_def have hp_meas : AEStronglyMeasurable p Q := (Measure.measurable_rnDeriv P Q).ennreal_toReal.aestronglyMeasurable have hp_int : Integrable p Q := by simpa [hp_def] using (Measure.integrable_toReal_rnDeriv : Integrable (fun x => (P.rnDeriv Q x).toReal) Q) have hdev : Integrable (fun x => (p x - 1) ^ 2) Q := by simpa [hp_def] using hint have hp_sq_int : Integrable (fun x => p x ^ 2) Q := by have hexp : (fun x => p x ^ 2) = fun x => (p x - 1) ^ 2 + (2 * p x - 1) := by funext x ring rw [hexp] exact hdev.add ((hp_int.const_mul 2).sub (integrable_const 1)) have hp_sq_eq : ∫ x, p x ^ 2 ∂Q = chiSqDiv P Q + 1 := by have h := chiSqDiv_eq (μ := P) (ν := Q) hac hint linarith have hp_sq_le : ∫ x, p x ^ 2 ∂Q ≤ C + 1 := by linarith let ind : Ω → ℝ := A.indicator (fun _ => (1 : ℝ)) have hind_meas : AEStronglyMeasurable ind Q := (measurable_const.indicator hA).aestronglyMeasurable have hind_sq_meas : AEStronglyMeasurable (fun x => ind x ^ 2) Q := hind_meas.pow 2 have hind_int_sq : Integrable (fun x => ind x ^ 2) Q := by refine Integrable.of_bound hind_sq_meas 1 ?_ filter_upwards with x by_cases hx : x ∈ A <;> simp [ind, hx] have hind_L2 : MemLp ind (ofReal 2) Q := by simpa using (memLp_two_iff_integrable_sq hind_meas).2 hind_int_sq have hp_L2 : MemLp p (ofReal 2) Q := by simpa using (memLp_two_iff_integrable_sq hp_meas).2 hp_sq_int have hind_nonneg : ∀ᵐ x ∂Q, 0 ≤ ind x := by filter_upwards with x by_cases hx : x ∈ A <;> simp [ind, hx] have hp_nonneg : ∀ᵐ x ∂Q, 0 ≤ p x := by filter_upwards with x rw [hp_def] exact ENNReal.toReal_nonneg have hholder : ∫ x, ind x * p x ∂Q ≤ (∫ x, ind x ^ (2 : ℝ) ∂Q) ^ (1 / (2 : ℝ)) * (∫ x, p x ^ (2 : ℝ) ∂Q) ^ (1 / (2 : ℝ)) := by exact integral_mul_le_Lp_mul_Lq_of_nonneg Real.HolderConjugate.two_two hind_nonneg hp_nonneg hind_L2 hp_L2 have hind_sq_eq : ∫ x, ind x ^ (2 : ℝ) ∂Q = Q.real A := by calc ∫ x, ind x ^ (2 : ℝ) ∂Q = ∫ x, ind x ∂Q := by apply integral_congr_ae filter_upwards with x by_cases hx : x ∈ A <;> simp [ind, hx] _ = Q.real A := by exact integral_indicator_one (μ := Q) hA have hp_sq_eq_rpow : ∫ x, p x ^ (2 : ℝ) ∂Q = ∫ x, p x ^ 2 ∂Q := by apply integral_congr_ae filter_upwards with x rw [Real.rpow_two] have hPA : P.real A = ∫ x, ind x * p x ∂Q := by rw [← Measure.setIntegral_toReal_rnDeriv hac A] rw [← integral_indicator hA] apply integral_congr_ae filter_upwards with x by_cases hx : x ∈ A <;> simp [ind, hx, hp_def] have hnonK : 0 ≤ C + 1 := by have hchi_non : 0 ≤ chiSqDiv P Q := chiSqDiv_nonneg linarith rw [hPA] calc ∫ x, ind x * p x ∂Q ≤ (∫ x, ind x ^ (2 : ℝ) ∂Q) ^ (1 / (2 : ℝ)) * (∫ x, p x ^ (2 : ℝ) ∂Q) ^ (1 / (2 : ℝ)) := hholder _ = sqrt (Q.real A) * sqrt (∫ x, p x ^ 2 ∂Q) := by rw [hind_sq_eq, hp_sq_eq_rpow] rw [Real.sqrt_eq_rpow, Real.sqrt_eq_rpow] _ ≤ sqrt (Q.real A) * sqrt (C + 1) := by exact mul_le_mul_of_nonneg_left (Real.sqrt_le_sqrt hp_sq_le) (Real.sqrt_nonneg _) _ = sqrt ((C + 1) * Q.real A) := by rw [Real.sqrt_mul hnonK] ring
MinimaxValue 4 core · 20 supporting This module packages the object inf over estimators of sup over the model class of the risk and its elementary order API. ★ le_minimaxValue_of_two_point★ minimaxValue_mono_class_of_nonneg
The minimax value of a statistical decision problem
This module packages the object inf over estimators of sup over the model class of the risk
and its elementary order API. The estimator family and the model class enter as bare index
types, so a run instantiates them at whatever subtype carries its admissibility and
membership conditions.
Provided here: worstCaseRisk (the supremum of one estimator's risk over the class),
minimaxValue (the infimum of that over the estimators), the four facts every minimax
argument needs about them — nonnegativity, "any admissible estimator is an upper bound",
"a bound uniform over admissible estimators is a lower bound", "shrinking the class cannot
raise the value" — a two-point reduction stated directly against the minimax value, and
bridges to the equivalent set-comprehension spelling sInf {r | ∃ e, ...}.
Conventions for degenerate problems. Real suprema and infima are only conditionally
complete, so an empty index or an unbounded range evaluates to the junk value zero.
This module keeps that convention rather than hiding it, because zero is the conservative
value here: it makes an empty estimator family or an empty model class yield minimax value
zero, which can never certify a positive lower bound. (Contrast coverageInfOrOne in
Causalean/Stat/Minimax/HonestConfidenceSet.lean, where the guarantee runs the other way
and the safe vacuous value is one.) Accordingly the lemmas below carry explicit
Nonempty / BddAbove / BddBelow hypotheses, and each comes with a _of_nonneg
companion that discharges those hypotheses from nonnegativity of the risk, which is what
squared-error, absolute-error and regret losses supply for free.
Everything is stated over ℝ. A problem whose loss lives in ℝ≥0∞ needs none of this
packaging: that order is a complete lattice, so iInf_le, le_iInf, le_iSup and
iSup_le apply with no side conditions at all.
def worstCaseRisk reviewed
⚑ flagged
Causalean.Stat
Given an estimator class, a model class, a real-valued risk for each estimator-model pair, and an estimator, the worst-case risk is the supremum, over all models in the class, of that estimator's risk.
Definition (Lean source)
parameters
E :
Sort u_1
Θ :
Sort u_3
risk :
E → Θ → ℝ
e :
E
def
worstCaseRisk risk e :
ℝ
given by
⨆ θ : Θ, risk e θ
def minimaxValue reviewed
⚑ flagged
Causalean.Stat
Given an estimator class, a model class, and a real-valued risk for each estimator-model pair, the minimax value is the infimum, over all estimators in the class, of their worst-case risks over the model class.
Definition (Lean source)
parameters
E :
Sort u_1
Θ :
Sort u_3
risk :
E → Θ → ℝ
def
minimaxValue risk :
ℝ
given by
⨅ e : E, worstCaseRisk risk e
theorem le_minimaxValue_of_two_point reviewed
⚑ flagged
Causalean.Stat
Two-point reduction to the minimax value. Fix two models θ₀ and θ₁ from the parameter class. If every estimator's risk is bounded above across the parameter class and for each estimator, the larger of its risks at the two fixed models is at least c, then the minimax value of the problem is at least c.
Formal statement
hypotheses
E :
Sort u_1
Θ :
Sort u_3
Nonempty E
risk :
E → Θ → ℝ
c :
ℝ
θ₀ θ₁ :
Θ
htwo :
∀ e, c ≤ max (risk e θ₀) (risk e θ₁)
conclusion
c ≤ minimaxValue risk
Proof (Lean source)
theorem le_minimaxValue_of_two_point [Nonempty E] {risk : E → Θ → ℝ} {c : ℝ} (θ₀ θ₁ : Θ)
(hbdd : ∀ e, BddAbove (range (risk e)))
(htwo : ∀ e, c ≤ max (risk e θ₀) (risk e θ₁)) :
c ≤ minimaxValue risk :=
le_minimaxValue fun e =>
(htwo e).trans (max_le (le_worstCaseRisk (hbdd e) θ₀) (le_worstCaseRisk (hbdd e) θ₁))
theorem minimaxValue_mono_class_of_nonneg reviewed
⚑ flagged
Causalean.Stat
For a map φ embedding the parameter class of the first problem into that of the second, if the first problem's risk is nonnegative, the second problem's risk is nonnegative, each estimator's risk is bounded above across the second parameter class, and the first risk at any model is dominated by the second risk at that model's image under φ, then the minimax value of the first problem is at most that of the second.
Formal statement
hypotheses
E :
Sort u_1
Θ :
Sort u_3
Θ' :
Sort u_4
Nonempty E
risk :
E → Θ → ℝ
risk' :
E → Θ' → ℝ
φ :
Θ → Θ'
hr :
∀ e θ, 0 ≤ risk e θ
hr' :
∀ e θ', 0 ≤ risk' e θ'
hle :
∀ e θ, risk e θ ≤ risk' e (φ θ)
conclusion
minimaxValue risk ≤ minimaxValue risk'
Proof (Lean source)
theorem minimaxValue_mono_class_of_nonneg [Nonempty E] {risk : E → Θ → ℝ}
{risk' : E → Θ' → ℝ} (φ : Θ → Θ')
(hr : ∀ e θ, 0 ≤ risk e θ) (hr' : ∀ e θ', 0 ≤ risk' e θ')
(hbdd : ∀ e, BddAbove (range (risk' e)))
(hle : ∀ e θ, risk e θ ≤ risk' e (φ θ)) :
minimaxValue risk ≤ minimaxValue risk' :=
minimaxValue_le_minimaxValue (bddBelow_range_worstCaseRisk hr)
fun e => ⟨e, worstCaseRisk_mono_class_of_nonneg φ (hbdd e) (hr' e) (hle e)⟩
20 supporting declarations (lemmas, instances)
-
worstCaseRisk_of_isEmpty_classtheorem — Over an empty model class the worst-case risk of every estimator is zero.hypothesesconclusionworstCaseRisk risk e = 0Proof (Lean source)
@[simp] theorem worstCaseRisk_of_isEmpty_class [IsEmpty Θ] (risk : E → Θ → ℝ) (e : E) : worstCaseRisk risk e = 0 := Real.iSup_of_isEmpty _ -
minimaxValue_of_isEmpty_estimatorstheorem — When no estimator is admissible the minimax value is zero, so no positive lower bound on the minimax value can hold vacuously.hypothesesconclusionminimaxValue risk = 0Proof (Lean source)
@[simp] theorem minimaxValue_of_isEmpty_estimators [IsEmpty E] (risk : E → Θ → ℝ) : minimaxValue risk = 0 := Real.iInf_of_isEmpty _ -
worstCaseRisk_of_uniquetheorem — When the model class consists of a single law, the worst-case risk is just the risk at that law. This covers decision problems that minimise a functional of the estimator alone, with no adversarial choice of model.hypothesesconclusionworstCaseRisk risk e = risk e defaultProof (Lean source)
theorem worstCaseRisk_of_unique [Unique Θ] (risk : E → Θ → ℝ) (e : E) : worstCaseRisk risk e = risk e default := ciSup_unique -
worstCaseRisk_nonnegtheorem — A worst-case risk is nonnegative as soon as the risk is, with no boundedness or nonemptiness caveat: an empty class or an unbounded risk both give the value zero.hypothesesconclusion0 ≤ worstCaseRisk risk eProof (Lean source)
theorem worstCaseRisk_nonneg {risk : E → Θ → ℝ} {e : E} (hr : ∀ θ, 0 ≤ risk e θ) : 0 ≤ worstCaseRisk risk e := Real.iSup_nonneg hr -
minimaxValue_nonnegtheorem — The minimax value of a nonnegative-risk problem is nonnegative, with no boundedness or nonemptiness caveat. Squared-error, absolute-error and regret losses all qualify.hypothesesconclusion0 ≤ minimaxValue riskProof (Lean source)
theorem minimaxValue_nonneg {risk : E → Θ → ℝ} (hr : ∀ e θ, 0 ≤ risk e θ) : 0 ≤ minimaxValue risk := Real.iInf_nonneg fun e => worstCaseRisk_nonneg (hr e) -
bddBelow_range_worstCaseRisktheorem — For a nonnegative risk the worst-case risks are bounded below by zero, which is the side condition the infimum over estimators needs.hypothesesconclusionProof (Lean source)
theorem bddBelow_range_worstCaseRisk {risk : E → Θ → ℝ} (hr : ∀ e θ, 0 ≤ risk e θ) : BddBelow (range (worstCaseRisk risk)) := by refine ⟨0, ?_⟩ rintro _ ⟨e, rfl⟩ exact worstCaseRisk_nonneg (hr e) -
le_worstCaseRisktheorem — The risk at any single model in the class is at most the estimator's worst-case risk, provided that estimator's risk is bounded over the class.hypothesesconclusionProof (Lean source)
theorem le_worstCaseRisk {risk : E → Θ → ℝ} {e : E} (hbdd : BddAbove (range (risk e))) (θ : Θ) : risk e θ ≤ worstCaseRisk risk e := le_ciSup hbdd θ -
worstCaseRisk_letheorem — A bound holding at every model in a nonempty class bounds the worst-case risk.hypothesesconclusionworstCaseRisk risk e ≤ cProof (Lean source)
theorem worstCaseRisk_le [Nonempty Θ] {risk : E → Θ → ℝ} {e : E} {c : ℝ} (h : ∀ θ, risk e θ ≤ c) : worstCaseRisk risk e ≤ c := ciSup_le h -
minimaxValue_le_worstCaseRisktheorem — Any admissible estimator is an upper bound. Exhibiting one estimator and bounding its worst-case risk bounds the minimax value, which is how achievability half of a minimax rate is certified. The hypothesis rules out a worst-case risk that decreases without bound across the estimator family.hypothesesconclusionProof (Lean source)
theorem minimaxValue_le_worstCaseRisk {risk : E → Θ → ℝ} (hbdd : BddBelow (range (worstCaseRisk risk))) (e : E) : minimaxValue risk ≤ worstCaseRisk risk e := ciInf_le hbdd e -
minimaxValue_le_worstCaseRisk_of_nonnegtheorem — For a nonnegative risk, any admissible estimator is an upper bound on the minimax value with no further side condition, since zero already bounds the worst-case risks below.hypothesesconclusionProof (Lean source)
theorem minimaxValue_le_worstCaseRisk_of_nonneg {risk : E → Θ → ℝ} (hr : ∀ e θ, 0 ≤ risk e θ) (e : E) : minimaxValue risk ≤ worstCaseRisk risk e := minimaxValue_le_worstCaseRisk (bddBelow_range_worstCaseRisk hr) e -
le_minimaxValuetheorem — A bound uniform over admissible estimators is a lower bound. If every admissible estimator has worst-case risk at least c, the minimax value is at least c. This is the converse half of a minimax rate; it needs at least one admissible estimator to exist, since otherwise the minimax value is zero by convention.hypothesesE :sharedSort u_1Θ :sharedSort u_3Nonempty Erisk :E → Θ → ℝc :ℝh :∀ e, c ≤ worstCaseRisk risk econclusionc ≤ minimaxValue riskProof (Lean source)
theorem le_minimaxValue [Nonempty E] {risk : E → Θ → ℝ} {c : ℝ} (h : ∀ e, c ≤ worstCaseRisk risk e) : c ≤ minimaxValue risk := le_ciInf h -
worstCaseRisk_mono_classtheorem — Comparing two model classes at a fixed estimator: if every model of the first class is matched, through the map, by a model of the second class whose risk is at least as large, the worst-case risk over the first class is at most that over the second. The second class must bound the estimator's risk, and the first must be nonempty; see worstCaseRisk_mono_class_of_nonneg for the version that drops the nonemptiness.hypothesesE :sharedSort u_1Θ :sharedSort u_3Θ' :sharedSort u_4Nonempty Θrisk :E → Θ → ℝrisk' :E → Θ' → ℝe :Eφ :Θ → Θ'hle :∀ θ, risk e θ ≤ risk' e (φ θ)conclusionworstCaseRisk risk e ≤ worstCaseRisk risk' eProof (Lean source)
theorem worstCaseRisk_mono_class [Nonempty Θ] {risk : E → Θ → ℝ} {risk' : E → Θ' → ℝ} {e : E} (φ : Θ → Θ') (hbdd : BddAbove (range (risk' e))) (hle : ∀ θ, risk e θ ≤ risk' e (φ θ)) : worstCaseRisk risk e ≤ worstCaseRisk risk' e := worstCaseRisk_le fun θ => (hle θ).trans (le_worstCaseRisk hbdd (φ θ)) -
worstCaseRisk_mono_class_of_nonnegtheorem — Version of worstCaseRisk_mono_class for a nonnegative larger risk: no nonemptiness assumption on the smaller class is needed, because an empty class contributes worst-case risk zero, which the larger nonnegative worst-case risk already dominates.hypothesesE :sharedSort u_1Θ :sharedSort u_3Θ' :sharedSort u_4risk :E → Θ → ℝrisk' :E → Θ' → ℝe :Eφ :Θ → Θ'hr' :∀ θ', 0 ≤ risk' e θ'hle :∀ θ, risk e θ ≤ risk' e (φ θ)conclusionworstCaseRisk risk e ≤ worstCaseRisk risk' eProof (Lean source)
theorem worstCaseRisk_mono_class_of_nonneg {risk : E → Θ → ℝ} {risk' : E → Θ' → ℝ} {e : E} (φ : Θ → Θ') (hbdd : BddAbove (range (risk' e))) (hr' : ∀ θ', 0 ≤ risk' e θ') (hle : ∀ θ, risk e θ ≤ risk' e (φ θ)) : worstCaseRisk risk e ≤ worstCaseRisk risk' e := by cases isEmpty_or_nonempty Θ with | inl _ => simpa using worstCaseRisk_nonneg hr' | inr _ => exact worstCaseRisk_mono_class φ hbdd hle -
minimaxValue_le_minimaxValuetheorem — Comparison of two minimax problems. If every admissible estimator of the second problem is matched by an estimator of the first whose worst-case risk is no larger, the first minimax value is at most the second. Both restricting the model class and enlarging the estimator family are instances; the boundedness hypothesis is on the first problem's worst-case risks, and the second problem must have at least one estimator.hypothesesE :sharedSort u_1E' :sharedSort u_2Θ :sharedSort u_3Θ' :sharedSort u_4Nonempty E'risk :E → Θ → ℝrisk' :E' → Θ' → ℝhbdd :h :∀ e' : E', ∃ e : E, worstCaseRisk risk e ≤ worstCaseRisk risk' e'conclusionminimaxValue risk ≤ minimaxValue risk'Proof (Lean source)
theorem minimaxValue_le_minimaxValue [Nonempty E'] {risk : E → Θ → ℝ} {risk' : E' → Θ' → ℝ} (hbdd : BddBelow (range (worstCaseRisk risk))) (h : ∀ e' : E', ∃ e : E, worstCaseRisk risk e ≤ worstCaseRisk risk' e') : minimaxValue risk ≤ minimaxValue risk' := by refine le_minimaxValue fun e' => ?_ obtain ⟨e, he⟩ := h e' exact (minimaxValue_le_worstCaseRisk hbdd e).trans he -
minimaxValue_mono_classtheorem — Restricting the model class cannot raise the minimax value. If each model of the smaller class sits, through the map, inside the larger class with no larger risk, then the minimax value over the smaller class is at most the minimax value over the larger one — the same estimators face a weaker adversary. This is the standard step that transfers a published converse proved on a convenient subclass to the full class.hypothesesE :sharedSort u_1Θ :sharedSort u_3Θ' :sharedSort u_4Nonempty ENonempty Θrisk :E → Θ → ℝrisk' :E → Θ' → ℝφ :Θ → Θ'hbddBelow :hle :∀ e θ, risk e θ ≤ risk' e (φ θ)conclusionminimaxValue risk ≤ minimaxValue risk'Proof (Lean source)
theorem minimaxValue_mono_class [Nonempty E] [Nonempty Θ] {risk : E → Θ → ℝ} {risk' : E → Θ' → ℝ} (φ : Θ → Θ') (hbddBelow : BddBelow (range (worstCaseRisk risk))) (hbdd : ∀ e, BddAbove (range (risk' e))) (hle : ∀ e θ, risk e θ ≤ risk' e (φ θ)) : minimaxValue risk ≤ minimaxValue risk' := minimaxValue_le_minimaxValue hbddBelow fun e => ⟨e, worstCaseRisk_mono_class φ (hbdd e) (hle e)⟩ -
worstCaseRisk_eq_sSup_rangetheorem — The worst-case risk is the supremum of the set of risks the estimator attains across the class. Rewriting handle for arguments phrased on the set of attained risks.hypothesesE :sharedSort u_1Θ :sharedSort u_3risk :E → Θ → ℝe :EconclusionProof (Lean source)
theorem worstCaseRisk_eq_sSup_range (risk : E → Θ → ℝ) (e : E) : worstCaseRisk risk e = sSup (range (risk e)) := rfl -
minimaxValue_eq_sInf_rangetheorem — The minimax value is the infimum of the set of worst-case risks the admissible estimators attain. Rewriting handle for arguments phrased on that set.hypothesesE :sharedSort u_1Θ :sharedSort u_3risk :E → Θ → ℝconclusionProof (Lean source)
theorem minimaxValue_eq_sInf_range (risk : E → Θ → ℝ) : minimaxValue risk = sInf (range (worstCaseRisk risk)) := rfl -
minimaxValue_subtype_eq_sInf_worstCaseRisktheorem — The minimax value over the estimators satisfying an admissibility condition, written as the infimum of the set of worst-case risks the admissible estimators attain. This is the bridge for developments that state admissibility by a predicate rather than by a subtype while leaving the inner supremum as it stands.hypothesesΘ :sharedSort u_3A :sharedType u_5A → Proprisk :A → Θ → ℝconclusionminimaxValue (fun (e : Subtype Adm) => risk e.1)= sInf {r : ℝ | ∃ e, Adm e ∧ r = worstCaseRisk risk e}Proof (Lean source)
theorem minimaxValue_subtype_eq_sInf_worstCaseRisk {Adm : A → Prop} (risk : A → Θ → ℝ) : minimaxValue (fun (e : Subtype Adm) => risk e.1) = sInf {r : ℝ | ∃ e, Adm e ∧ r = worstCaseRisk risk e} := by rw [minimaxValue_eq_sInf_range] congr 1 ext r simp only [Set.mem_range, Set.mem_setOf_eq, Subtype.exists] constructor · rintro ⟨e, he, rfl⟩ exact ⟨e, he, rfl⟩ · rintro ⟨e, he, rfl⟩ exact ⟨e, he, rfl⟩ -
worstCaseRisk_subtype_eq_sSup_setOftheorem — The worst-case risk over a class carved out by a membership condition, written as the supremum of the set of risks attained on the class. This is the bridge to the spelling that states the class by a predicate rather than by a subtype.hypothesesA :sharedType u_5B :sharedType u_6B → Proprisk :A → B → ℝe :AconclusionworstCaseRisk (fun (a : A) (θ : Subtype Cls) => risk a θ.1) e= sSup {q : ℝ | ∃ θ, Cls θ ∧ q = risk e θ}Proof (Lean source)
theorem worstCaseRisk_subtype_eq_sSup_setOf {Cls : B → Prop} (risk : A → B → ℝ) (e : A) : worstCaseRisk (fun (a : A) (θ : Subtype Cls) => risk a θ.1) e = sSup {q : ℝ | ∃ θ, Cls θ ∧ q = risk e θ} := by rw [worstCaseRisk, iSup] congr 1 ext q simp only [Set.mem_range, Set.mem_setOf_eq, Subtype.exists, exists_prop] constructor · rintro ⟨θ, hθ, rfl⟩ exact ⟨θ, hθ, rfl⟩ · rintro ⟨θ, hθ, rfl⟩ exact ⟨θ, hθ, rfl⟩ -
minimaxValue_subtype_eq_sInf_setOftheorem — The minimax value over an admissible-estimator condition and a model-class condition, written with set comprehensions instead of subtypes: the infimum of the set of worst-case risks attained by admissible estimators, each of which is the supremum of the set of risks attained on the class. Runs that spell their minimax risk this way rewrite with this lemma and then use the rest of this file.hypothesesA :sharedType u_5B :sharedType u_6Adm :A → PropCls :B → Proprisk :A → B → ℝconclusionProof (Lean source)
theorem minimaxValue_subtype_eq_sInf_setOf (Adm : A → Prop) (Cls : B → Prop) (risk : A → B → ℝ) : minimaxValue (fun (e : Subtype Adm) (θ : Subtype Cls) => risk e.1 θ.1) = sInf {r : ℝ | ∃ e, Adm e ∧ r = sSup {q : ℝ | ∃ θ, Cls θ ∧ q = risk e θ}} := by rw [minimaxValue_subtype_eq_sInf_worstCaseRisk (risk := fun (a : A) (θ : Subtype Cls) => risk a θ.1)] congr 1 ext r simp only [Set.mem_setOf_eq, worstCaseRisk_subtype_eq_sSup_setOf]
Mixture 3 core · 7 supporting This file defines finite mixtures of measures with nonnegative extended-real weights. ★ mixtureReal_le
Finite Mixtures of Measures
This file defines finite mixtures of measures with nonnegative extended-real weights. It proves evaluation, probability-measure, and domination facts used to transfer componentwise bounds to mixtures in minimax arguments.
def mixture reviewed
⚑ flagged
Causalean.Stat
Given a sample space equipped with a σ-algebra, a finite index set, nonnegative extended-real weights indexed by that set, and a measure for each index, the finite mixture measure is the sum of the component measures, each scaled by its corresponding weight.
Definition (Lean source)
parameters
def
given by
∑ i, w i • P i
theorem mixtureReal_le reviewed
⚑ flagged
Causalean.Stat
Domination. If the mixture weights w sum to 1, each component measure P i is a probability measure, and every component assigns .real-mass at most B to the set A, then the mixture measure also assigns .real-mass at most B to A.
Formal statement
hypotheses
ι :
Type u_2
w :
ι → ℝ≥0∞
hw :
∑ i, w i = 1
P :
ι → Measure Ω
∀ i, IsProbabilityMeasure (P i)
A :
Set Ω
B :
ℝ
hB :
∀ i, (P i).real A ≤ B
Proof (Lean source)
theorem mixtureReal_le (w : ι → ℝ≥0∞) (hw : ∑ i, w i = 1)
(P : ι → Measure Ω) [∀ i, IsProbabilityMeasure (P i)]
(A : Set Ω) (B : ℝ) (hB : ∀ i, (P i).real A ≤ B) :
(mixture w P).real A ≤ B := by
have hwfin : ∀ i, w i ≠ ⊤ := by
intro i
have hle : w i ≤ 1 := le_of_le_of_eq (Finset.single_le_sum
(f := w) (fun j _ => zero_le) (Finset.mem_univ i)) hw
exact ne_top_of_le_ne_top ENNReal.one_ne_top hle
have hterm : ∀ i, w i * P i A ≠ ⊤ := by
intro i
exact ENNReal.mul_ne_top (hwfin i) (measure_ne_top _ _)
rw [Measure.real, mixture_apply, ENNReal.toReal_sum (fun i _ => hterm i)]
have hsum : (∑ i, (w i * P i A).toReal) ≤ ∑ i, (w i).toReal * B := by
refine Finset.sum_le_sum (fun i _ => ?_)
rw [ENNReal.toReal_mul]
refine mul_le_mul_of_nonneg_left ?_ ENNReal.toReal_nonneg
have : (P i).real A = (P i A).toReal := rfl
rw [← this]
exact hB i
refine hsum.trans ?_
rw [← Finset.sum_mul]
have hwsum : (∑ i, (w i).toReal) = 1 := by
rw [← ENNReal.toReal_sum (fun i _ => hwfin i), hw, ENNReal.toReal_one]
rw [hwsum, one_mul]
def priorPredictive reviewed
⚑ flagged
Causalean.Stat.Minimax.MomentMatchedMixture
Given a prior measure and a measurable experiment kernel, the prior-predictive law first draws a parameter from the prior and then draws an observation from the experiment at that parameter.
7 supporting declarations (lemmas, instances)
-
mixture_applytheorem — Evaluation: the mixture's mass on a set is the weighted sum of the parts' masses.hypothesesconclusionmixture w P A = ∑ i, w i * P i AProof (Lean source)
theorem mixture_apply (w : ι → ℝ≥0∞) (P : ι → Measure Ω) (A : Set Ω) : mixture w P A = ∑ i, w i * P i A := by simp only [mixture, Measure.coe_finsetSum, Finset.sum_apply, Measure.smul_apply, smul_eq_mul] -
mixture_isProbabilityMeasuretheorem — If the weights sum to 1 and each part is a probability measure, the mixture is one.hypothesesconclusionIsProbabilityMeasure (mixture w P)Proof (Lean source)
theorem mixture_isProbabilityMeasure (w : ι → ℝ≥0∞) (hw : ∑ i, w i = 1) (P : ι → Measure Ω) [∀ i, IsProbabilityMeasure (P i)] : IsProbabilityMeasure (mixture w P) := by refine ⟨?_⟩ rw [mixture_apply] simp only [measure_univ, mul_one] exact hw -
exists_real_ge_mixturetheorem — Witness extraction. Some component carries at least the mixture's .real-mass: since the mixture is a weighted average (weights summing to 1), its mass on A is at most the maximal component mass, attained over the finite index.hypothesesι :sharedType u_2Nonempty ιw :ι → ℝ≥0∞hw :∑ i, w i = 1P :ι → Measure Ω∀ i, IsProbabilityMeasure (P i)A :Set ΩProof (Lean source)
theorem exists_real_ge_mixture [Nonempty ι] (w : ι → ℝ≥0∞) (hw : ∑ i, w i = 1) (P : ι → Measure Ω) [∀ i, IsProbabilityMeasure (P i)] (A : Set Ω) : ∃ i, (mixture w P).real A ≤ (P i).real A := by obtain ⟨i, _, hmax⟩ := Finset.exists_max_image (Finset.univ : Finset ι) (fun i => (P i).real A) Finset.univ_nonempty exact ⟨i, mixtureReal_le w hw P A ((P i).real A) (fun j => hmax j (Finset.mem_univ j))⟩ -
priorPredictive_isProbabilitytheorem — Mixing probability experiment laws from a measurable kernel against a probability prior produces a probability law on observations.hypothesesΘ :sharedType u_1X :sharedType u_2π :Measure ΘK :Kernel Θ XhK :∀ θ, IsProbabilityMeasure (K θ)conclusionProof (Lean source)
theorem priorPredictive_isProbability (π : Measure Θ) (K : Kernel Θ X) [IsProbabilityMeasure π] (hK : ∀ θ, IsProbabilityMeasure (K θ)) : IsProbabilityMeasure (priorPredictive π K) := by rw [priorPredictive] apply isProbabilityMeasure_iff.mpr rw [Measure.bind_apply MeasurableSet.univ K.aemeasurable] simp_rw [isProbabilityMeasure_iff.mp (hK _)] simp -
priorPredictive_applytheorem — The mass that a prior and experiment kernel assign to a measurable observation event equals the prior average of its conditional event probabilities.hypothesesconclusionpriorPredictive π K A = ∫⁻ θ, K θ A ∂πProof (Lean source)
theorem priorPredictive_apply (π : Measure Θ) (K : Kernel Θ X) {A : Set X} (hA : MeasurableSet A) : priorPredictive π K A = ∫⁻ θ, K θ A ∂π := by exact Measure.bind_apply hA K.aemeasurable -
lintegral_priorPredictivetheorem — The lower integral of a nonnegative measurable statistic under the mixture from a prior and experiment kernel equals the iterated prior-then-experiment lower integral.hypothesesconclusion∫⁻ x, f x ∂priorPredictive π K = ∫⁻ θ, ∫⁻ x, f x ∂K θ ∂πProof (Lean source)
theorem lintegral_priorPredictive (π : Measure Θ) (K : Kernel Θ X) (f : X → ℝ≥0∞) (hf : Measurable f) : ∫⁻ x, f x ∂priorPredictive π K = ∫⁻ θ, ∫⁻ x, f x ∂K θ ∂π := by exact Measure.lintegral_bind K.aemeasurable hf.aemeasurable -
integral_priorPredictivetheorem — The expectation of an integrable real-valued statistic under the mixture from a prior and experiment kernel equals the prior average of its conditional expectations.hypothesesΘ :sharedType u_1X :sharedType u_2π :Measure ΘK :Kernel Θ Xf :X → ℝhf :Integrable f (priorPredictive π K)conclusion∫ x, f x ∂priorPredictive π K = ∫ θ, ∫ x, f x ∂K θ ∂πProof (Lean source)
theorem integral_priorPredictive (π : Measure Θ) (K : Kernel Θ X) (f : X → ℝ) (hf : Integrable f (priorPredictive π K)) : ∫ x, f x ∂priorPredictive π K = ∫ θ, ∫ x, f x ∂K θ ∂π := by exact integral_bind K.measurable hf
Assouad 5 core · 7 supporting This file develops Assouad's lower bound for statistical experiments indexed by a Boolean hypercube. ★ assouad_average★ assouad_exists
Assouad hypercube minimax bound
This file develops Assouad's lower bound for statistical experiments indexed by a Boolean hypercube. It introduces coordinate flips and Hamming risk, then uses coordinatewise total-variation bounds between neighboring experiments to derive average and worst-case minimax lower bounds.
def flipBit reviewed
⚑ flagged
Causalean.Stat
Given a hypercube dimension, a coordinate of that hypercube, and a Boolean hypercube vertex, the coordinate-flipped vertex agrees with the given vertex at every coordinate except the specified one, where it takes the opposite Boolean value.
def flipPerm reviewed
⚑ flagged
Causalean.Stat
Given a hypercube dimension and a coordinate of that hypercube, the coordinate-flip permutation is the bijection of Boolean hypercube vertices that flips precisely the specified coordinate.
Definition (Lean source)
def hammingRisk reviewed
⚑ flagged
Causalean.Stat
Given a measurable sample space, a hypercube dimension, a measure for every Boolean hypercube vertex, an estimator returning a Boolean vertex from each sample outcome, and a true vertex, the Hamming risk is the sum, over coordinates, of the measure of the outcomes at which the estimator differs from that true vertex under its associated measure.
theorem assouad_average reviewed
⚑ flagged
Causalean.Stat
Assouad's lemma (average form). Assume each coordinate's decoded-bit event is measurable. If every hypercube vertex's law is within total variation β of each of its d neighbouring vertices' laws, then the average Hamming risk over the cube is at least (d / 2)(1 − β). Choosing the dimension d large and the per-coordinate divergence β small forces a large number of mis-decoded coordinates.
Formal statement
Proof (Lean source)
theorem assouad_average (hmeas : ∀ j (b : Bool), MeasurableSet {ω | est ω j = b})
{β : ℝ} (hβ : ∀ j τ, tvDist (P τ) (P (flipBit j τ)) ≤ β) :
(d / 2 : ℝ) * (1 - β)
≤ (∑ τ, hammingRisk P est τ) / (card (Fin d → Bool)) := by
set C : ℝ := (card (Fin d → Bool) : ℝ) with hC
have hCnat : 0 < card (Fin d → Bool) :=
Fintype.card_pos_iff.mpr ⟨fun _ => false⟩
have hCpos : (0 : ℝ) < C := by rw [hC]; exact_mod_cast hCnat
-- Per coordinate: `C·(1−β)/2 ≤ ∑_τ (coordinate-j error mass)`.
have hG : ∀ j, C * (1 - β) / 2 ≤ ∑ τ, (P τ).real {ω | est ω j ≠ τ j} := by
intro j
have hcoord := sum_decode_ge P est hmeas j
have hβsum : C * (1 - β) ≤ ∑ τ, (1 - tvDist (P τ) (P (flipBit j τ))) := by
have h1 : ∑ _τ : Fin d → Bool, (1 - β)
≤ ∑ τ, (1 - tvDist (P τ) (P (flipBit j τ))) :=
Finset.sum_le_sum (fun τ _ => by linarith [hβ j τ])
have h2 : ∑ _τ : Fin d → Bool, (1 - β) = C * (1 - β) := by
rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, hC]
linarith [h1, h2]
linarith [hβsum, hcoord]
-- Sum the coordinate bounds, then divide by the cube size.
have hswap : ∑ τ, hammingRisk P est τ
= ∑ j, ∑ τ, (P τ).real {ω | est ω j ≠ τ j} := by
simp_rw [hammingRisk]; rw [Finset.sum_comm]
have hfinal : (d : ℝ) * (C * (1 - β) / 2) ≤ ∑ τ, hammingRisk P est τ := by
rw [hswap]
have h2 : (d : ℝ) * (C * (1 - β) / 2) = ∑ _j : Fin d, (C * (1 - β) / 2) := by
rw [Finset.sum_const, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul]
rw [h2]
exact Finset.sum_le_sum (fun j _ => hG j)
rw [le_div_iff₀ hCpos]
have hrw : (d / 2 : ℝ) * (1 - β) * C = (d : ℝ) * (C * (1 - β) / 2) := by ring
rw [hrw]; exact hfinal
theorem assouad_exists reviewed
⚑ flagged
Causalean.Stat
Assouad's lemma (existence form). Under the same hypotheses — each coordinate's decoded-bit event is measurable and every vertex's law is within total variation β of each of its d neighbours — some vertex τ forces Hamming risk at least (d / 2)(1 − β): no cube estimator can decode every vertex's coordinates reliably when neighbouring laws are statistically close.
Formal statement
hypotheses
conclusion
∃ τ, (d / 2 : ℝ) * (1 - β) ≤ hammingRisk P est τ
Proof (Lean source)
theorem assouad_exists (hmeas : ∀ j (b : Bool), MeasurableSet {ω | est ω j = b})
{β : ℝ} (hβ : ∀ j τ, tvDist (P τ) (P (flipBit j τ)) ≤ β) :
∃ τ, (d / 2 : ℝ) * (1 - β) ≤ hammingRisk P est τ := by
set C : ℝ := (card (Fin d → Bool) : ℝ) with hC
have hCnat : 0 < card (Fin d → Bool) :=
Fintype.card_pos_iff.mpr ⟨fun _ => false⟩
have hCpos : (0 : ℝ) < C := by rw [hC]; exact_mod_cast hCnat
have havg := assouad_average P est hmeas hβ
rw [← hC] at havg
by_contra hcon
push_neg at hcon
have hstrict : ∑ τ, hammingRisk P est τ < C * ((d / 2 : ℝ) * (1 - β)) := by
calc ∑ τ, hammingRisk P est τ
< ∑ _τ : Fin d → Bool, ((d / 2 : ℝ) * (1 - β)) :=
Finset.sum_lt_sum_of_nonempty ⟨fun _ => false, Finset.mem_univ _⟩
(fun τ _ => hcon τ)
_ = C * ((d / 2 : ℝ) * (1 - β)) := by
rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, hC]
rw [le_div_iff₀ hCpos] at havg
nlinarith [havg, hstrict]
7 supporting declarations (lemmas, instances)
-
flipBit_selftheorem — Flipping a coordinate changes that coordinate to the opposite Boolean value.Proof (Lean source)
@[simp] theorem flipBit_self (j : Fin d) (τ : Fin d → Bool) : flipBit j τ j = !τ j := Function.update_self _ _ _ -
flipBit_involutivetheorem — Flipping the same coordinate twice is the identity.Proof (Lean source)
theorem flipBit_involutive (j : Fin d) : Involutive (flipBit j) := by intro τ funext k by_cases h : k = j · subst h; simp [flipBit] · simp [flipBit, Function.update_of_ne h] -
measurableSet_decode_netheorem — Coordinate-decoding error sets are measurable, being complements of the measurable level sets {ω | est ω j = b}.hypothesesconclusionMeasurableSet {ω | est ω j ≠ b}Proof (Lean source)
theorem measurableSet_decode_ne (hmeas : ∀ j (b : Bool), MeasurableSet {ω | est ω j = b}) (j : Fin d) (b : Bool) : MeasurableSet {ω | est ω j ≠ b} := by have h : {ω | est ω j ≠ b} = {ω | est ω j = b}ᶜ := by ext ω; simp rw [h]; exact (hmeas j b).compl -
decode_ne_flip_compltheorem — The j-th decoding-error set at the flipped vertex is the complement of the one at τ: the estimator's j-th bit either matches τ j or its flip, never both.hypothesesconclusion{ω | est ω j ≠ (flipBit j τ) j} = {ω | est ω j ≠ τ j}ᶜProof (Lean source)
theorem decode_ne_flip_compl (j : Fin d) (τ : Fin d → Bool) : {ω | est ω j ≠ (flipBit j τ) j} = {ω | est ω j ≠ τ j}ᶜ := by have boollem : ∀ a b : Bool, (a ≠ !b) ↔ (a = b) := by decide ext ω simp only [Set.mem_compl_iff, Set.mem_setOf_eq, flipBit_self, not_not] exact boollem (est ω j) (τ j) -
hammingRisk_pair_getheorem — Per-coordinate pair bound. For each vertex τ, the coordinate-j error mass at τ plus the one at its j-flip is at least 1 − tvDist (P τ) (P (flip j τ)). This is the two-point testing bound applied to the j-th decoded bit.hypothesesProof (Lean source)
theorem hammingRisk_pair_ge (hmeas : ∀ j (b : Bool), MeasurableSet {ω | est ω j = b}) (j : Fin d) (τ : Fin d → Bool) : 1 - tvDist (P τ) (P (flipBit j τ)) ≤ (P τ).real {ω | est ω j ≠ τ j} + (P (flipBit j τ)).real {ω | est ω j ≠ (flipBit j τ) j} := by have hS : MeasurableSet {ω | est ω j ≠ τ j} := measurableSet_decode_ne est hmeas j (τ j) rw [decode_ne_flip_compl est j τ] exact one_sub_tvDist_le_test (μ := P τ) (ν := P (flipBit j τ)) hS -
sum_decode_getheorem — Per-coordinate lower bound. Summed over the cube, the coordinate-j error mass is at least half the cube-sum of 1 − tvDist (P τ) (P (flip j τ)), via pairing each vertex with its j-flip (an involution that preserves the sum).hypothesesProof (Lean source)
theorem sum_decode_ge (hmeas : ∀ j (b : Bool), MeasurableSet {ω | est ω j = b}) (j : Fin d) : ∑ τ, (1 - tvDist (P τ) (P (flipBit j τ))) ≤ 2 * ∑ τ, (P τ).real {ω | est ω j ≠ τ j} := by -- Reindexing by the `j`-flip permutation leaves the cube-sum invariant. have hreindex : ∑ τ, (P (flipBit j τ)).real {ω | est ω j ≠ (flipBit j τ) j} = ∑ τ, (P τ).real {ω | est ω j ≠ τ j} := Equiv.sum_comp (flipPerm j) (fun σ => (P σ).real {ω | est ω j ≠ σ j}) calc ∑ τ, (1 - tvDist (P τ) (P (flipBit j τ))) ≤ ∑ τ, ((P τ).real {ω | est ω j ≠ τ j} + (P (flipBit j τ)).real {ω | est ω j ≠ (flipBit j τ) j}) := Finset.sum_le_sum (fun τ _ => hammingRisk_pair_ge P est hmeas j τ) _ = (∑ τ, (P τ).real {ω | est ω j ≠ τ j}) + ∑ τ, (P (flipBit j τ)).real {ω | est ω j ≠ (flipBit j τ) j} := Finset.sum_add_distrib _ = 2 * ∑ τ, (P τ).real {ω | est ω j ≠ τ j} := by rw [hreindex]; ring
FiniteKernelBayes 7 core · 25 supporting This module supplies a real-valued Bayes-to-minimax bridge for a finite state space. ★ realBayesRisk_le_minimaxValue
Finite-state Bayes risks through probability kernels
This module supplies a real-valued Bayes-to-minimax bridge for a finite state space. A possibly
continuous prior is mixed through a Markov kernel, and the resulting average loss is bounded by
the finite worst-case loss. It also gives deterministic-map and FiniteDesign specializations,
including the finite-sum and measure-integral representations of the mixed risk.
def kernelAverageLoss reviewed
⚑ flagged
Causalean.Stat
Given a measurable parameter space, a finite measurable state space, an action space, a probability kernel from parameters to states, a real-valued loss for each action and state, an action, and a parameter value, the kernel-average loss is the expected loss of that action under the state distribution selected by the kernel at that parameter value.
Definition (Lean source)
parameters
Θ :
Type u_1
S :
Type u_2
A :
Type u_3
K :
Kernel Θ S
loss :
A → S → ℝ
a :
A
θ :
Θ
def
kernelAverageLoss K loss a θ :
ℝ
given by
∫ s, loss a s ∂K θ
def mixedKernelLoss reviewed
⚑ flagged
Causalean.Stat
Given a measurable parameter space, a finite measurable state space, an action space, a prior measure on the parameter space, a probability kernel from parameters to states, a real-valued loss for each action and state, and an action, the mixed kernel loss is that action's kernel-average loss integrated with respect to the prior.
Definition (Lean source)
parameters
Θ :
Type u_1
S :
Type u_2
A :
Type u_3
π :
Measure Θ
K :
Kernel Θ S
loss :
A → S → ℝ
a :
A
def
mixedKernelLoss π K loss a :
ℝ
given by
∫ θ, kernelAverageLoss K loss a θ ∂π
def realBayesRisk reviewed
⚑ flagged
Causalean.Stat
Given a measurable parameter space, a finite measurable state space, an action space, a prior measure on the parameter space, a probability kernel from parameters to states, and a real-valued loss for each action and state, the real Bayes risk is the infimum, over all actions, of their mixed kernel losses.
Definition (Lean source)
parameters
Θ :
Type u_1
S :
Type u_2
A :
Type u_3
π :
Measure Θ
K :
Kernel Θ S
loss :
A → S → ℝ
def
realBayesRisk π K loss :
ℝ
given by
⨅ a, mixedKernelLoss π K loss a
theorem realBayesRisk_le_minimaxValue reviewed
⚑ flagged
Causalean.Stat
With at least one available action and nonnegative statewise loss, the real Bayes risk under a continuous prior and finite-state Markov kernel is at most the finite-state minimax value.
Formal statement
hypotheses
Θ :
Type u_1
S :
Type u_2
A :
Type u_3
Nonempty A
π :
K :
Kernel Θ S
loss :
A → S → ℝ
hloss :
∀ a s, 0 ≤ loss a s
conclusion
realBayesRisk π K loss ≤ minimaxValue loss
Proof (Lean source)
theorem realBayesRisk_le_minimaxValue [Nonempty A]
(π : Measure Θ) [IsProbabilityMeasure π]
(K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ)
(hloss : ∀ a s, 0 ≤ loss a s) :
realBayesRisk π K loss ≤ minimaxValue loss := by
apply le_minimaxValue
intro a
refine (ciInf_le ?_ a).trans (mixedKernelLoss_le_worstCaseRisk π K loss hloss a)
refine ⟨0, ?_⟩
rintro _ ⟨a', rfl⟩
exact mixedKernelLoss_nonneg π K loss hloss a'
def deterministicBayesRisk reviewed
⚑ flagged
Causalean.Stat
Given a measurable parameter space, a finite measurable state space, an action space, a prior measure on the parameter space, a map assigning each parameter a state, and a real-valued loss for each action and state, the deterministic Bayes risk is the infimum, over all actions, of the prior-integrated loss evaluated at the state assigned to each parameter.
Definition (Lean source)
parameters
Θ :
Type u_1
S :
Type u_2
A :
Type u_3
π :
Measure Θ
f :
Θ → S
loss :
A → S → ℝ
def
deterministicBayesRisk π f loss :
ℝ
given by
⨅ a, ∫ θ, loss a (f θ) ∂π
def finiteDesignBayesRisk reviewed
⚑ flagged
Causalean.Stat
Given a finite state space, an action space, a finite randomization design on the state space, and a real-valued loss for each action and state, the finite-design Bayes risk is the infimum, over all actions, of their expected losses under that design.
Definition (Lean source)
parameters
def
finiteDesignBayesRisk D loss :
ℝ
given by
⨅ a, D.E (loss a)
def inducedFiniteDesign reviewed
⚑ flagged
Causalean.Stat
Given a measurable parameter space, a nonempty finite measurable state space with measurable singletons, a probability prior on the parameter space, and a Markov kernel from parameters to states, the induced finite design is the design whose probability assigned to each state is the prior average of the kernel's probability of that state, these probabilities are nonnegative, and their sum over all states is one.
Definition (Lean source)
parameters
def
given by
clause 1
p s := ∫ θ, (K θ).real {s} ∂π
clause 2
p_nonneg := by intro s exact integral_nonneg fun θ ↦ measureReal_nonneg
clause 3
p_sum := by rw [← integral_finsetSum] · simp_rw [sum_measureReal_singleton] simp · intro s _ exact integrable_kernel_singletonReal π K s
25 supporting declarations (lemmas, instances)
-
finite_range_bddAbovetheorem — A real-valued function on a finite state space has a bounded range.Proof (Lean source)
-
finite_range_bddBelowtheorem — A real-valued function on a finite state space has a range bounded below.Proof (Lean source)
-
finite_integrabletheorem — A real-valued statistic on a finite state space is integrable under every finite measure.Proof (Lean source)
theorem finite_integrable (μ : Measure S) [IsFiniteMeasure μ] (f : S → ℝ) : Integrable f μ := by refine Integrable.of_bound (measurable_of_finite f).aestronglyMeasurable (∑ s, ‖f s‖) (ae_of_all μ fun s ↦ ?_) exact Finset.single_le_sum (fun t _ ↦ norm_nonneg (f t)) (Finset.mem_univ s) -
loss_integrable_kerneltheorem — The loss from one action is integrable under every probability distribution emitted by a Markov kernel into a finite state space.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3K :Kernel Θ Sloss :A → S → ℝa :Aθ :ΘconclusionIntegrable (loss a) (K θ)Proof (Lean source)
theorem loss_integrable_kernel (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (a : A) (θ : Θ) : Integrable (loss a) (K θ) := by exact finite_integrable (K θ) (loss a) -
stronglyMeasurable_kernelAverageLosstheorem — The kernel-averaged loss of a finite-state action is measurable as a function of the parameter.hypothesesconclusionStronglyMeasurable (kernelAverageLoss K loss a)Proof (Lean source)
theorem stronglyMeasurable_kernelAverageLoss (K : Kernel Θ S) (loss : A → S → ℝ) (a : A) : StronglyMeasurable (kernelAverageLoss K loss a) := by exact (measurable_of_finite (loss a)).stronglyMeasurable.integral_kernel -
integrable_kernelAverageLosstheorem — Under a probability prior, the kernel-averaged loss of a finite-state action is integrable.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :K :Kernel Θ Sloss :A → S → ℝa :AconclusionIntegrable (kernelAverageLoss K loss a) πProof (Lean source)
theorem integrable_kernelAverageLoss (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (a : A) : Integrable (kernelAverageLoss K loss a) π := by refine Integrable.of_bound (stronglyMeasurable_kernelAverageLoss K loss a).aestronglyMeasurable (∑ s, ‖loss a s‖) (ae_of_all π fun θ ↦ ?_) calc ‖kernelAverageLoss K loss a θ‖ ≤ (∑ s, ‖loss a s‖) * (K θ).real univ := by apply norm_integral_le_of_norm_le_const exact ae_of_all (K θ) fun s ↦ Finset.single_le_sum (fun t _ ↦ norm_nonneg (loss a t)) (Finset.mem_univ s) _ = ∑ s, ‖loss a s‖ := by simp -
kernelAverageLoss_eq_sumtheorem — Kernel averaging on a finite state space equals the sum of statewise losses weighted by the kernel's singleton probabilities.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3K :Kernel Θ Sloss :A → S → ℝa :Aθ :ΘconclusionkernelAverageLoss K loss a θ = ∑ s, (K θ).real {s} * loss a sProof (Lean source)
theorem kernelAverageLoss_eq_sum (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (a : A) (θ : Θ) : kernelAverageLoss K loss a θ = ∑ s, (K θ).real {s} * loss a s := by simpa [kernelAverageLoss] using integral_fintype (loss_integrable_kernel K loss a θ) -
measurable_kernel_singletonRealtheorem — A finite-state Markov kernel's probability of a fixed state is measurable in its source parameter.hypothesesconclusionMeasurable (fun θ ↦ (K θ).real {s})Proof (Lean source)
theorem measurable_kernel_singletonReal (K : Kernel Θ S) [IsMarkovKernel K] (s : S) : Measurable (fun θ ↦ (K θ).real {s}) := by exact (K.measurable_coe (MeasurableSet.singleton s)).ennreal_toReal -
integrable_kernel_singletonRealtheorem — A finite-state Markov kernel's probability of a fixed state is integrable under every probability prior.hypothesesconclusionIntegrable (fun θ ↦ (K θ).real {s}) πProof (Lean source)
theorem integrable_kernel_singletonReal (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (s : S) : Integrable (fun θ ↦ (K θ).real {s}) π := by refine Integrable.of_bound (measurable_kernel_singletonReal K s).aestronglyMeasurable 1 (ae_of_all π fun θ ↦ ?_) rw [Real.norm_eq_abs, abs_of_nonneg measureReal_nonneg, Measure.real_def] have htop : ((K θ) univ) ≠ ⊤ := by simp simpa using ENNReal.toReal_mono htop (measure_mono (Set.subset_univ {s})) -
mixedKernelLoss_eq_sum_integraltheorem — A continuous-prior mixture through a finite-state kernel equals the sum of statewise losses weighted by the prior-averaged singleton probabilities.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :K :Kernel Θ Sloss :A → S → ℝa :AconclusionmixedKernelLoss π K loss a = ∑ s, (∫ θ, (K θ).real {s} ∂π) * loss a sProof (Lean source)
theorem mixedKernelLoss_eq_sum_integral (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (a : A) : mixedKernelLoss π K loss a = ∑ s, (∫ θ, (K θ).real {s} ∂π) * loss a s := by simp_rw [mixedKernelLoss, kernelAverageLoss_eq_sum] rw [integral_finsetSum] · simp_rw [integral_mul_const] · intro s _ exact (integrable_kernel_singletonReal π K s).mul_const (loss a s) -
mixedKernelLoss_nonnegtheorem — If statewise loss is nonnegative, then its prior-and-kernel mixture is nonnegative.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :K :Kernel Θ Sloss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sa :Aconclusion0 ≤ mixedKernelLoss π K loss aProof (Lean source)
theorem mixedKernelLoss_nonneg (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) (a : A) : 0 ≤ mixedKernelLoss π K loss a := by exact integral_nonneg fun θ ↦ integral_nonneg (hloss a) -
mixedKernelLoss_le_worstCaseRisktheorem — If loss is nonnegative in every state, then the continuous-prior average of each action's finite-state kernel loss is no greater than that action's finite worst-case risk.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :K :Kernel Θ Sloss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sa :Aconclusion(∫ θ, ∫ s, loss a s ∂K θ ∂π) ≤ worstCaseRisk loss aProof (Lean source)
theorem mixedKernelLoss_le_worstCaseRisk (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) (a : A) : (∫ θ, ∫ s, loss a s ∂K θ ∂π) ≤ worstCaseRisk loss a := by calc (∫ θ, ∫ s, loss a s ∂K θ ∂π) ≤ ∫ _θ, worstCaseRisk loss a ∂π := by apply integral_mono (integrable_kernelAverageLoss π K loss a) (integrable_const (worstCaseRisk loss a)) intro θ change kernelAverageLoss K loss a θ ≤ worstCaseRisk loss a unfold kernelAverageLoss calc (∫ s, loss a s ∂K θ) ≤ ∫ _s, worstCaseRisk loss a ∂K θ := by apply integral_mono (loss_integrable_kernel K loss a θ) (integrable_const (worstCaseRisk loss a)) intro s exact le_worstCaseRisk (finite_range_bddAbove (loss a)) s _ = worstCaseRisk loss a := by simp _ = worstCaseRisk loss a := by simp -
realBayesRisk_nonnegtheorem — If statewise loss is nonnegative, then the real Bayes risk is nonnegative.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :K :Kernel Θ Sloss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sconclusion0 ≤ realBayesRisk π K lossProof (Lean source)
theorem realBayesRisk_nonneg (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) : 0 ≤ realBayesRisk π K loss := by exact Real.iInf_nonneg fun a ↦ mixedKernelLoss_nonneg π K loss hloss a -
integrable_loss_comptheorem — A finite-state loss composed with a measurable deterministic state map is integrable under every probability prior.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :f :Θ → Shf :loss :A → S → ℝa :AconclusionIntegrable (fun θ ↦ loss a (f θ)) πProof (Lean source)
theorem integrable_loss_comp (π : Measure Θ) [IsProbabilityMeasure π] (f : Θ → S) (hf : Measurable f) (loss : A → S → ℝ) (a : A) : Integrable (fun θ ↦ loss a (f θ)) π := by refine Integrable.of_bound ((measurable_of_finite (loss a)).comp hf).aestronglyMeasurable (∑ s, ‖loss a s‖) (ae_of_all π fun θ ↦ ?_) exact Finset.single_le_sum (fun s _ ↦ norm_nonneg (loss a s)) (Finset.mem_univ (f θ)) -
mixedKernelLoss_deterministictheorem — A measurable deterministic state map has the same mixed kernel loss as the ordinary prior integral of its composed loss.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :f :Θ → Shf :loss :A → S → ℝa :AconclusionmixedKernelLoss π (Kernel.deterministic f hf) loss a = ∫ θ, loss a (f θ) ∂πProof (Lean source)
theorem mixedKernelLoss_deterministic (π : Measure Θ) [IsProbabilityMeasure π] (f : Θ → S) (hf : Measurable f) (loss : A → S → ℝ) (a : A) : mixedKernelLoss π (Kernel.deterministic f hf) loss a = ∫ θ, loss a (f θ) ∂π := by unfold mixedKernelLoss kernelAverageLoss apply integral_congr_ae refine ae_of_all π fun θ ↦ ?_ change (∫ s, loss a s ∂(Kernel.deterministic f hf) θ) = loss a (f θ) rw [Kernel.deterministic_apply, integral_dirac] -
integral_loss_comp_le_worstCaseRisktheorem — If the state map is measurable and the loss is nonnegative in every state, then the prior average of the composed loss is no greater than its finite worst-case risk.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :f :Θ → Shf :loss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sa :Aconclusion(∫ θ, loss a (f θ) ∂π) ≤ worstCaseRisk loss aProof (Lean source)
theorem integral_loss_comp_le_worstCaseRisk (π : Measure Θ) [IsProbabilityMeasure π] (f : Θ → S) (hf : Measurable f) (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) (a : A) : (∫ θ, loss a (f θ) ∂π) ≤ worstCaseRisk loss a := by rw [← mixedKernelLoss_deterministic π f hf loss a] exact mixedKernelLoss_le_worstCaseRisk π (Kernel.deterministic f hf) loss hloss a -
deterministicBayesRisk_eq_realBayesRisktheorem — A measurable deterministic state map gives the same Bayes risk whether viewed directly or as its deterministic Markov kernel.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :f :Θ → Shf :loss :A → S → ℝconclusiondeterministicBayesRisk π f loss = realBayesRisk π (Kernel.deterministic f hf) lossProof (Lean source)
theorem deterministicBayesRisk_eq_realBayesRisk (π : Measure Θ) [IsProbabilityMeasure π] (f : Θ → S) (hf : Measurable f) (loss : A → S → ℝ) : deterministicBayesRisk π f loss = realBayesRisk π (Kernel.deterministic f hf) loss := by simp [deterministicBayesRisk, realBayesRisk, mixedKernelLoss_deterministic] -
deterministicBayesRisk_le_minimaxValuetheorem — With at least one available action, a measurable deterministic state map, and nonnegative statewise loss, the deterministic Bayes risk is at most the finite-state minimax value.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3Nonempty Aπ :f :Θ → Shf :loss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sconclusiondeterministicBayesRisk π f loss ≤ minimaxValue lossProof (Lean source)
theorem deterministicBayesRisk_le_minimaxValue [Nonempty A] (π : Measure Θ) [IsProbabilityMeasure π] (f : Θ → S) (hf : Measurable f) (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) : deterministicBayesRisk π f loss ≤ minimaxValue loss := by rw [deterministicBayesRisk_eq_realBayesRisk π f hf loss] exact realBayesRisk_le_minimaxValue π (Kernel.deterministic f hf) loss hloss -
finiteDesign_integrabletheorem — A real-valued statistic on a finite design space is integrable under the design's induced probability measure.Proof (Lean source)
theorem finiteDesign_integrable (D : FiniteDesign S) (g : S → ℝ) : Integrable g D.toMeasure := by exact finite_integrable D.toMeasure g -
finiteDesign_expectedLoss_eq_sum_eq_integraltheorem — A finite design's expected action loss equals both its finite weighted sum and its induced probability-measure integral.hypothesesconclusionconclusion 1D.E (loss a) = ∑ s, D.p s * loss a sconclusion 2D.E (loss a) = ∫ s, loss a s ∂D.toMeasureProof (Lean source)
theorem finiteDesign_expectedLoss_eq_sum_eq_integral (D : FiniteDesign S) (loss : A → S → ℝ) (a : A) : D.E (loss a) = ∑ s, D.p s * loss a s ∧ D.E (loss a) = ∫ s, loss a s ∂D.toMeasure := by exact ⟨rfl, (D.integral_toMeasure (loss a)).symm⟩ -
finiteDesign_expectedLoss_le_worstCaseRisktheorem — If the statewise loss is nonnegative, then a finite design's expected loss for each action is no greater than that action's finite worst-case risk.hypothesesS :sharedType u_2A :sharedType u_3D :loss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sa :AconclusionD.E (loss a) ≤ worstCaseRisk loss aProof (Lean source)
theorem finiteDesign_expectedLoss_le_worstCaseRisk (D : FiniteDesign S) (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) (a : A) : D.E (loss a) ≤ worstCaseRisk loss a := by rw [← D.integral_toMeasure] calc (∫ s, loss a s ∂D.toMeasure) ≤ ∫ _s, worstCaseRisk loss a ∂D.toMeasure := by apply integral_mono (finiteDesign_integrable D (loss a)) (integrable_const (worstCaseRisk loss a)) intro s exact le_worstCaseRisk (finite_range_bddAbove (loss a)) s _ = worstCaseRisk loss a := by simp -
finiteDesignBayesRisk_le_minimaxValuetheorem — With at least one available action and nonnegative statewise loss, a finite design's Bayes risk is at most the finite-state minimax value.hypothesesS :sharedType u_2A :sharedType u_3Nonempty AD :loss :A → S → ℝhloss :∀ a s, 0 ≤ loss a sconclusionfiniteDesignBayesRisk D loss ≤ minimaxValue lossProof (Lean source)
theorem finiteDesignBayesRisk_le_minimaxValue [Nonempty A] (D : FiniteDesign S) (loss : A → S → ℝ) (hloss : ∀ a s, 0 ≤ loss a s) : finiteDesignBayesRisk D loss ≤ minimaxValue loss := by apply le_minimaxValue intro a refine (ciInf_le ?_ a).trans (finiteDesign_expectedLoss_le_worstCaseRisk D loss hloss a) refine ⟨0, ?_⟩ rintro _ ⟨a', rfl⟩ exact sum_nonneg fun s _ ↦ mul_nonneg (D.p_nonneg s) (hloss a' s) -
inducedFiniteDesign_ptheorem — The induced finite design's mass at each state is the prior integral of the kernel's singleton probability.hypothesesconclusion(inducedFiniteDesign π K).p s = ∫ θ, (K θ).real {s} ∂πProof (Lean source)
theorem inducedFiniteDesign_p (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (s : S) : (inducedFiniteDesign π K).p s = ∫ θ, (K θ).real {s} ∂π := rfl -
inducedFiniteDesign_expectedLoss_eq_mixedKernelLosstheorem — Expected loss under the induced finite design equals the continuous-prior, kernel-averaged loss integral.hypothesesΘ :sharedType u_1S :sharedType u_2A :sharedType u_3π :K :Kernel Θ Sloss :A → S → ℝa :Aconclusion(inducedFiniteDesign π K).E (loss a) = mixedKernelLoss π K loss aProof (Lean source)
theorem inducedFiniteDesign_expectedLoss_eq_mixedKernelLoss (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) (a : A) : (inducedFiniteDesign π K).E (loss a) = mixedKernelLoss π K loss a := by exact (mixedKernelLoss_eq_sum_integral π K loss a).symm -
realBayesRisk_eq_inducedFiniteDesignBayesRisktheorem — A continuous prior mixed through a finite-state Markov kernel has the same real Bayes risk as its induced finite design.hypothesesconclusionProof (Lean source)
theorem realBayesRisk_eq_inducedFiniteDesignBayesRisk (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (loss : A → S → ℝ) : realBayesRisk π K loss = finiteDesignBayesRisk (inducedFiniteDesign π K) loss := by unfold realBayesRisk finiteDesignBayesRisk congr 1 funext a exact (inducedFiniteDesign_expectedLoss_eq_mixedKernelLoss π K loss a).symm
MinimaxRisk 6 core · 7 supporting This file specializes Le Cam's two-point method to real-valued statistical functionals and to estimators based on independent repeated samples. ★ real_two_point_lower_bound★ two_point_lower_bound_of_tvDist_le★ iid_two_point_lower_bound★ two_point_lower_bound_of_chiSqDiv_le
Minimax Risk Lower Bounds
This file specializes Le Cam's two-point method to real-valued statistical functionals and to estimators based on independent repeated samples. It records the total-variation, chi-squared-divergence, and bounded-estimator integrability forms used to certify concrete minimax rates.
theorem real_two_point_lower_bound reviewed
⚑ flagged
Causalean.Stat
Le Cam two-point bound for a real-valued parameter. If two candidate values θ₀, θ₁ : ℝ are 2s-separated, then for any measurable estimator est : Ω → ℝ, the worst-case probability of missing the truth by ≥ s is at least ½(1 − tvDist P₀ P₁). Specialization of half_one_sub_tvDist_le_max_error to Θ = ℝ with dist a b = |a − b|.
Formal statement
hypotheses
Proof (Lean source)
theorem real_two_point_lower_bound {est : Ω → ℝ} (hest : Measurable est)
{θ₀ θ₁ s : ℝ} (hsep : 2 * s ≤ |θ₀ - θ₁|) :
(1 - tvDist P₀ P₁) / 2
≤ max (P₀.real {ω | s ≤ |est ω - θ₀|}) (P₁.real {ω | s ≤ |est ω - θ₁|}) := by
have hsep' : 2 * s ≤ dist θ₀ θ₁ := by rwa [Real.dist_eq]
have h := half_one_sub_tvDist_le_max_error (P₀ := P₀) (P₁ := P₁) (Θ := ℝ) hest hsep'
simpa only [Real.dist_eq] using h
theorem two_point_lower_bound_of_tvDist_le reviewed
⚑ flagged
Causalean.Stat
Le Cam two-point bound with an explicit total-variation bound. If two candidate values θ₀, θ₁ : ℝ are 2s-separated, est : Ω → ℝ is a measurable estimator, and the total variation distance tvDist P₀ P₁ is at most c, then the worst-case probability of missing the truth by ≥ s is at least (1 − c)/2. Variant of real_two_point_lower_bound with an explicit total-variation upper bound.
Formal statement
hypotheses
conclusion
(1 - c) / 2 ≤ max (P₀.real {ω | s ≤ |est ω - θ₀|}) (P₁.real {ω | s ≤ |est ω - θ₁|})
Proof (Lean source)
theorem two_point_lower_bound_of_tvDist_le {est : Ω → ℝ} (hest : Measurable est)
{θ₀ θ₁ s c : ℝ} (hsep : 2 * s ≤ |θ₀ - θ₁|) (hc : tvDist P₀ P₁ ≤ c) :
(1 - c) / 2
≤ max (P₀.real {ω | s ≤ |est ω - θ₀|}) (P₁.real {ω | s ≤ |est ω - θ₁|}) := by
have h := real_two_point_lower_bound (P₀ := P₀) (P₁ := P₁) hest hsep
have : (1 - c) / 2 ≤ (1 - tvDist P₀ P₁) / 2 := by linarith
exact this.trans h
theorem iid_two_point_lower_bound reviewed
⚑ flagged
Causalean.Stat
n-sample structure-agnostic two-point bound. Given two single-observation laws P₀, P₁ and a real functional τ such that the functional values τ P₀, τ P₁ are 2s-separated, then for any measurable estimator est built from n i.i.d. samples (data law Measure.pi (fun _ ↦ Pⱼ)), the worst-case miss probability, using τ P₀ and τ P₁ as the two parameters, is at least ½(1 − tvDist) between the two n-fold product laws.
Formal statement
hypotheses
S :
Type*
P₀ P₁ :
τ :
Measure S → ℝ
n :
ℕ
s :
ℝ
hsep :
2 * s ≤ |τ P₀ - τ P₁|
est :
(Fin n → S) → ℝ
hest :
Measurable est
Proof (Lean source)
theorem iid_two_point_lower_bound {S : Type*} [MeasurableSpace S]
(P₀ P₁ : Measure S) [IsProbabilityMeasure P₀] [IsProbabilityMeasure P₁]
(τ : Measure S → ℝ) (n : ℕ) {s : ℝ} (hsep : 2 * s ≤ |τ P₀ - τ P₁|)
{est : (Fin n → S) → ℝ} (hest : Measurable est) :
(1 - tvDist (Measure.pi fun _ : Fin n => P₀) (Measure.pi fun _ : Fin n => P₁)) / 2
≤ max ((Measure.pi fun _ : Fin n => P₀).real {x | s ≤ |est x - τ P₀|})
((Measure.pi fun _ : Fin n => P₁).real {x | s ≤ |est x - τ P₁|}) :=
real_two_point_lower_bound (P₀ := Measure.pi fun _ : Fin n => P₀)
(P₁ := Measure.pi fun _ : Fin n => P₁) hest hsep
theorem two_point_lower_bound_of_chiSqDiv_le reviewed
⚑ flagged
Causalean.Stat
χ²-form two-point lower bound. For a measurable estimator est, if two candidate values θ₀, θ₁ are 2s-separated, P₀ is absolutely continuous with respect to P₁, the squared density deviation (dP₀/dP₁ − 1)² is P₁-integrable, and the χ²-divergence chiSqDiv P₀ P₁ is at most c, then the worst-case miss probability is at least (1 − ½√c)/2, via tvDist ≤ ½√χ². Since chiSqDiv tensorizes over i.i.d. samples (chiSqDiv_prod) and is computable for explicit families, this is the form used to certify minimax rates.
Formal statement
hypotheses
hest :
Measurable est
θ₀ θ₁ s :
ℝ
hsep :
2 * s ≤ |θ₀ - θ₁|
hac :
P₀ ≪ P₁
hint :
Integrable (fun x => ((P₀.rnDeriv P₁ x).toReal - 1) ^ 2) P₁
c :
ℝ
hc :
chiSqDiv P₀ P₁ ≤ c
Proof (Lean source)
theorem two_point_lower_bound_of_chiSqDiv_le {est : Ω → ℝ} (hest : Measurable est)
{θ₀ θ₁ s : ℝ} (hsep : 2 * s ≤ |θ₀ - θ₁|) (hac : P₀ ≪ P₁)
(hint : Integrable (fun x => ((P₀.rnDeriv P₁ x).toReal - 1) ^ 2) P₁)
{c : ℝ} (hc : chiSqDiv P₀ P₁ ≤ c) :
(1 - (1 / 2) * sqrt c) / 2
≤ max (P₀.real {ω | s ≤ |est ω - θ₀|}) (P₁.real {ω | s ≤ |est ω - θ₁|}) := by
have htv : tvDist P₀ P₁ ≤ (1 / 2) * sqrt c := by
refine (tvDist_le_half_sqrt_chiSqDiv P₀ P₁ hac hint).trans ?_
gcongr
exact two_point_lower_bound_of_tvDist_le hest hsep htv
def sqRisk reviewed
⚑ flagged
Causalean.Stat
Given a measurable observation space, a measure on that space, a real-valued estimator, and a real target value, the squared risk is the expected value, under that measure, of the estimator's squared error from the target.
Definition (Lean source)
parameters
def
sqRisk law est theta :
ℝ
given by
∫ z, (est z - theta) ^ 2 ∂law
def affinePullbackEstimator reviewed
⚑ flagged
Causalean.Stat
Given a source observation space, a target observation space, a deterministic observation rule from the source to the target, real affine slope and offset parameters, and a real-valued target-space estimator, the affine pullback estimator maps each source observation to the target estimator evaluated at its observed target value, minus the offset and divided by the slope.
Definition (Lean source)
parameters
X :
Type uX
Y :
Type uY
phi :
X → Y
a b :
ℝ
targetEst :
Y → ℝ
def
affinePullbackEstimator phi a b targetEst :
X → ℝ
given by
fun z => (targetEst (phi z) - b) / a
7 supporting declarations (lemmas, instances)
-
mse_integrable_of_estimator_boundlemma — Squared-loss integrability for a truncated estimator. If a measurable estimator T takes values in the bounded interval [-M, M] (with M ≥ 0), then under any finite measure Q its squared loss (T − θ)² against an arbitrary target θ is integrable, because it is bounded by the constant (M + |θ|)². This is the routine integrability bookkeeping needed before the worst-case squared risk of a truncated estimator can be compared in a two-point lower bound.hypothesesconclusionIntegrable (fun s => (T s - theta) ^ 2) QProof (Lean source)
lemma mse_integrable_of_estimator_bound {S : Type*} [MeasurableSpace S] (Q : Measure S) [IsFiniteMeasure Q] (T : S → ℝ) (hT : Measurable T) {M theta : ℝ} (hM : 0 ≤ M) (hbound : ∀ s, T s ∈ Icc (-M) M) : Integrable (fun s => (T s - theta) ^ 2) Q := by refine Integrable.of_bound ((hT.sub measurable_const).pow_const (2 : ℕ)).aestronglyMeasurable ((M + |theta|) ^ 2) ?_ filter_upwards with s have hTabs : |T s| ≤ M := abs_le.mpr (hbound s) have hsub : |T s - theta| ≤ M + |theta| := (abs_sub (T s) theta).trans (add_le_add hTabs le_rfl) have hC : 0 ≤ M + |theta| := add_nonneg hM (abs_nonneg theta) have hsq : (T s - theta) ^ 2 ≤ (M + |theta|) ^ 2 := by nlinarith [hsub, abs_nonneg (T s - theta), hC, sq_abs (T s - theta)] simpa [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg (T s - theta))] using hsq -
integral_le_sSup_range_of_isProbabilityMeasurelemma — Bayes risk never exceeds worst-case risk. Averaging an integrable risk function over the parameter space against any prior *probability* distribution gives at most the supremum of that risk over the parameter space (assuming the risk is bounded above). This is the step that lets a minimax lower bound be certified by exhibiting a single prior and bounding its average risk from below.hypothesesProof (Lean source)
lemma integral_le_sSup_range_of_isProbabilityMeasure {Θ : Type*} [MeasurableSpace Θ] (π : Measure Θ) [IsProbabilityMeasure π] (risk : Θ → ℝ) (hrisk : Integrable risk π) (hbounded : BddAbove (range risk)) : ∫ θ, risk θ ∂π ≤ sSup (range risk) := by have hpoint : ∀ θ, risk θ ≤ sSup (range risk) := fun θ => le_csSup hbounded (Set.mem_range_self θ) have hconst : Integrable (fun _ : Θ => sSup (range risk)) π := integrable_const _ have hmono : (∫ θ, risk θ ∂π) ≤ ∫ _ : Θ, sSup (range risk) ∂π := integral_mono hrisk hconst hpoint simpa using hmono -
measurable_affinePullbackEstimatortheorem — If the observation rule is measurable and the target estimator is measurable, then undoing an affine change after pulling the estimator back through the observation rule is measurable.hypothesesX :sharedType uXY :sharedType uYX → Ya b :ℝtargetEst :Y → ℝhphi :Measurable phihtarget :Measurable targetEstconclusionMeasurable (affinePullbackEstimator phi a b targetEst)Proof (Lean source)
@[fun_prop] theorem measurable_affinePullbackEstimator {phi : X → Y} {a b : ℝ} {targetEst : Y → ℝ} (hphi : Measurable phi) (htarget : Measurable targetEst) : Measurable (affinePullbackEstimator phi a b targetEst) := by exact ((htarget.comp hphi).sub measurable_const).div measurable_const -
affine_sqLoss_pullback_identitytheorem — If the affine slope is nonzero, then the squared error of a target estimator after deterministic observation equals the squared error of its affine pullback multiplied by the squared slope, point by point.hypothesesX :sharedType uXY :sharedType uYX → Ya b theta :ℝtargetEst :Y → ℝha :a ≠ 0z :Xconclusiona ^ 2 * (affinePullbackEstimator phi a b targetEst z - theta) ^ 2= (targetEst (phi z) - (a * theta + b)) ^ 2Proof (Lean source)
theorem affine_sqLoss_pullback_identity {phi : X → Y} {a b theta : ℝ} {targetEst : Y → ℝ} (ha : a ≠ 0) (z : X) : a ^ 2 * (affinePullbackEstimator phi a b targetEst z - theta) ^ 2 = (targetEst (phi z) - (a * theta + b)) ^ 2 := by unfold affinePullbackEstimator field_simp ring -
sqRisk_map_affinePullbacktheorem — If the affine slope is nonzero, the deterministic observation rule is measurable, and the target estimator is measurable, then its squared risk under the pushed-forward law equals the pullback estimator's squared risk multiplied by the squared slope.hypothesesX :sharedType uXY :sharedType uYlaw :Measure Xphi :X → Ya b theta :ℝtargetEst :Y → ℝha :a ≠ 0hphi :Measurable phihtarget :Measurable targetEstconclusiona ^ 2 * sqRisk law (affinePullbackEstimator phi a b targetEst) theta= sqRisk (law.map phi) targetEst (a * theta + b)Proof (Lean source)
theorem sqRisk_map_affinePullback {law : Measure X} {phi : X → Y} {a b theta : ℝ} {targetEst : Y → ℝ} (ha : a ≠ 0) (hphi : Measurable phi) (htarget : Measurable targetEst) : a ^ 2 * sqRisk law (affinePullbackEstimator phi a b targetEst) theta = sqRisk (law.map phi) targetEst (a * theta + b) := by have hloss : Measurable (fun y => (targetEst y - (a * theta + b)) ^ 2) := (htarget.sub measurable_const).pow_const 2 unfold sqRisk calc a ^ 2 * (∫ z, (affinePullbackEstimator phi a b targetEst z - theta) ^ 2 ∂law) = ∫ z, a ^ 2 * (affinePullbackEstimator phi a b targetEst z - theta) ^ 2 ∂law := (integral_const_mul (a ^ 2) _).symm _ = ∫ z, (targetEst (phi z) - (a * theta + b)) ^ 2 ∂law := by exact integral_congr_ae (Filter.Eventually.of_forall fun z => affine_sqLoss_pullback_identity ha z) _ = ∫ y, (targetEst y - (a * theta + b)) ^ 2 ∂law.map phi := (integral_map hphi.aemeasurable hloss.aestronglyMeasurable).symm -
forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transporttheorem — Suppose the affine slope is nonzero, the observation rule is measurable, and each target-experiment law is the pushforward of its corresponding source law. If every measurable source estimator has squared risk at least a fixed level for some parameter, then every measurable target estimator has squared risk at least that level multiplied by the squared affine slope for some parameter, where the target parameter is transformed by the same affine map.hypothesesX :sharedType uXY :sharedType uYIota :sharedType uIP :Iota → Measure XQ :Iota → Measure Ytheta :Iota → ℝphi :X → Ya b L :ℝha :a ≠ 0hphi :Measurable phihQ :∀ j, Q j = (P j).map phihsource :∀ sourceEst : X → ℝifMeasurable sourceEstthen∃ j, L ≤ sqRisk (P j) sourceEst (theta j)targetEst :Y → ℝMeasurable targetEstconclusion∃ j, a ^ 2 * L ≤ sqRisk (Q j) targetEst (a * theta j + b)Proof (Lean source)
theorem forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport (P : Iota → Measure X) (Q : Iota → Measure Y) (theta : Iota → ℝ) (phi : X → Y) (a b L : ℝ) (ha : a ≠ 0) (hphi : Measurable phi) (hQ : ∀ j, Q j = (P j).map phi) (hsource : ∀ sourceEst : X → ℝ, Measurable sourceEst → ∃ j, L ≤ sqRisk (P j) sourceEst (theta j)) : ∀ targetEst : Y → ℝ, Measurable targetEst → ∃ j, a ^ 2 * L ≤ sqRisk (Q j) targetEst (a * theta j + b) := by intro targetEst htarget obtain ⟨j, hj⟩ := hsource (affinePullbackEstimator phi a b targetEst) (measurable_affinePullbackEstimator hphi htarget) refine ⟨j, ?_⟩ calc a ^ 2 * L ≤ a ^ 2 * sqRisk (P j) (affinePullbackEstimator phi a b targetEst) (theta j) := mul_le_mul_of_nonneg_left hj (sq_nonneg a) _ = sqRisk ((P j).map phi) targetEst (a * theta j + b) := sqRisk_map_affinePullback ha hphi htarget _ = sqRisk (Q j) targetEst (a * theta j + b) := by rw [hQ j] -
forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport_pitheorem — Suppose every source law is a probability law, the affine slope is nonzero, the observation rule is measurable, and each target marginal law is the pushforward of its corresponding source marginal. If every measurable estimator based on the finite source product experiment has squared risk at least a fixed level for some parameter, then every measurable estimator based on the corresponding target product experiment has squared risk at least that level multiplied by the squared affine slope for some parameter, including when the sample has no coordinates.hypothesesX :sharedType uXY :sharedType uYIota :sharedType uIn :ℕP :Iota → Measure XQ :Iota → Measure Y∀ j, IsProbabilityMeasure (P j)theta :Iota → ℝphi :X → Ya b L :ℝha :a ≠ 0hphi :Measurable phihQ :∀ j, Q j = (P j).map phihsource :∀ sourceEst : (Fin n → X) → ℝifMeasurable sourceEsttargetEst :(Fin n → Y) → ℝMeasurable targetEstProof (Lean source)
theorem forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport_pi (n : ℕ) (P : Iota → Measure X) (Q : Iota → Measure Y) [∀ j, IsProbabilityMeasure (P j)] (theta : Iota → ℝ) (phi : X → Y) (a b L : ℝ) (ha : a ≠ 0) (hphi : Measurable phi) (hQ : ∀ j, Q j = (P j).map phi) (hsource : ∀ sourceEst : (Fin n → X) → ℝ, Measurable sourceEst → ∃ j, L ≤ sqRisk (Measure.pi (fun _ : Fin n => P j)) sourceEst (theta j)) : ∀ targetEst : (Fin n → Y) → ℝ, Measurable targetEst → ∃ j, a ^ 2 * L ≤ sqRisk (Measure.pi (fun _ : Fin n => Q j)) targetEst (a * theta j + b) := by apply forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport (P := fun j => Measure.pi (fun _ : Fin n => P j)) (Q := fun j => Measure.pi (fun _ : Fin n => Q j)) (theta := theta) (phi := fun z i => phi (z i)) (a := a) (b := b) (L := L) ha (measurable_finCoordinatewise n hphi) ?_ hsource intro j calc Measure.pi (fun _ : Fin n => Q j) = Measure.pi (fun _ : Fin n => (P j).map phi) := by congr 1 funext i exact hQ j _ = (Measure.pi (fun _ : Fin n => P j)).map (fun z : Fin n → X => fun i => phi (z i)) := (map_pi_finCoordinatewise n (P j) hphi).symm
Pinsker 5 core · 2 supporting Proves Pinsker's inequality and its KL-form Le Cam minimax lower-bound corollaries. ★ pinskerBound_of_ac_of_ne_top★ pinskerBound_pi_iid★ klForm_two_point_lower_bound_of_pinsker★ klForm_two_point_lower_bound
Proves Pinsker's inequality and its KL-form Le Cam minimax lower-bound corollaries.
The file first proves the scalar inequality klFun_lower_bound, then packages
total-variation control as PinskerBound. The main bridge
pinskerBound_of_ac_of_ne_top derives that package for probability measures
μ ≪ ν with finite KL divergence. The product theorem
pinskerBound_pi_iid lifts the bridge to i.i.d. finite products, and the
klForm_two_point_lower_bound theorems turn the bridge into Le Cam lower bounds
whose right-hand divergence term is Kullback-Leibler rather than total variation.
def PinskerBound reviewed
⚑ flagged
Causalean.Stat
For a measurable sample space and two measures on that space, the Pinsker bound is the proposition that their total-variation distance is at most the square root of one half of the real-valued Kullback--Leibler divergence from the first measure to the second.
theorem pinskerBound_of_ac_of_ne_top reviewed
⚑ flagged
Causalean.Stat
Pinsker's inequality (unconditional). For probability measures μ and ν on the same space, if μ is absolutely continuous with respect to ν and their Kullback–Leibler divergence is finite, then the total variation distance between μ and ν is at most the square root of half their Kullback–Leibler divergence: tvDist μ ν ≤ √(klDiv(μ,ν)/2).
Formal statement
Proof (Lean source)
theorem pinskerBound_of_ac_of_ne_top (μ ν : Measure Ω)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν]
(hac : μ ≪ ν) (hfin : klDiv μ ν ≠ ⊤) :
PinskerBound μ ν := by
set p : Ω → ℝ := fun x => (μ.rnDeriv ν x).toReal with hp_def
set K : ℝ := (klDiv μ ν).toReal with hK_def
set g : Ω → ℝ := fun x => (p x - 1) ^ 2 / (p x + 2) with hg_def
set f₁ : Ω → ℝ := fun x => |p x - 1| / sqrt (p x + 2) with hf₁_def
set f₂ : Ω → ℝ := fun x => sqrt (p x + 2) with hf₂_def
have hp_nonneg : ∀ x, 0 ≤ p x := by
intro x
rw [hp_def]
exact ENNReal.toReal_nonneg
have hK_nonneg : 0 ≤ K := by
rw [hK_def]
exact ENNReal.toReal_nonneg
have hp_int : Integrable p ν := by
rw [hp_def]
exact Measure.integrable_toReal_rnDeriv
have hp_meas : AEStronglyMeasurable p ν := by
rw [hp_def]
exact (Measure.measurable_rnDeriv μ ν).ennreal_toReal.aestronglyMeasurable
have hp_integral_one : ∫ x, p x ∂ν = 1 := by
rw [hp_def, Measure.integral_toReal_rnDeriv hac]
rw [measureReal_def, measure_univ]
simp
have hllr_int : Integrable (llr μ ν) μ :=
(InformationTheory.klDiv_ne_top_iff.mp hfin).2
have hkl_int : Integrable (fun x => klFun (p x)) ν := by
rw [hp_def]
exact (InformationTheory.integrable_klFun_rnDeriv_iff hac).2 hllr_int
have hK_eq_integral : K = ∫ x, klFun (p x) ∂ν := by
rw [hK_def, hp_def]
exact InformationTheory.toReal_klDiv_eq_integral_klFun hac
have hg_nonneg : ∀ x, 0 ≤ g x := by
intro x
rw [hg_def]
exact pinsker_weight_nonneg (hp_nonneg x)
have hg_le : ∀ x, g x ≤ (2 / 3) * klFun (p x) := by
intro x
rw [hg_def]
exact pinsker_weight_le_klFun (hp_nonneg x)
have hg_meas : AEStronglyMeasurable g ν := by
rw [hg_def]
exact (by fun_prop : AEMeasurable (fun x => (p x - 1) ^ 2 / (p x + 2)) ν)
|>.aestronglyMeasurable
have hf₂_meas : AEStronglyMeasurable f₂ ν := by
rw [hf₂_def]
fun_prop
have hf₁_meas : AEStronglyMeasurable f₁ ν := by
rw [hf₁_def]
exact (by fun_prop : AEMeasurable (fun x => |p x - 1| / sqrt (p x + 2)) ν)
|>.aestronglyMeasurable
have hg_int : Integrable g ν := by
refine Integrable.mono' (hkl_int.const_mul (2 / 3)) hg_meas ?_
exact Filter.Eventually.of_forall fun x => by
rw [Real.norm_eq_abs, abs_of_nonneg (hg_nonneg x)]
exact hg_le x
have hg_integral_le : ∫ x, g x ∂ν ≤ (2 / 3) * K := by
have hdom_int : Integrable (fun x => (2 / 3) * klFun (p x)) ν :=
hkl_int.const_mul (2 / 3)
have hle_int : ∫ x, g x ∂ν ≤ ∫ x, (2 / 3) * klFun (p x) ∂ν :=
integral_mono_ae hg_int hdom_int (Filter.Eventually.of_forall hg_le)
calc
∫ x, g x ∂ν ≤ ∫ x, (2 / 3) * klFun (p x) ∂ν := hle_int
_ = (2 / 3) * K := by rw [integral_const_mul, ← hK_eq_integral]
have hf₁_sq_int : Integrable (fun x => f₁ x ^ 2) ν := by
refine hg_int.congr (Filter.Eventually.of_forall fun x => ?_)
rw [hf₁_def, hg_def]
exact (pinsker_abs_div_sqrt_sq (hp_nonneg x)).symm
have hf₂_sq_int : Integrable (fun x => f₂ x ^ 2) ν := by
refine (hp_int.add (integrable_const 2)).congr (Filter.Eventually.of_forall fun x => ?_)
rw [hf₂_def]
exact (Real.sq_sqrt (by linarith [hp_nonneg x] : 0 ≤ p x + 2)).symm
have hf₁L2 : MemLp f₁ (ofReal 2) ν := by
simpa using (memLp_two_iff_integrable_sq hf₁_meas).2 hf₁_sq_int
have hf₂L2 : MemLp f₂ (ofReal 2) ν := by
simpa using (memLp_two_iff_integrable_sq hf₂_meas).2 hf₂_sq_int
have hf₁_nonneg : 0 ≤ᵐ[ν] f₁ := Filter.Eventually.of_forall fun x => by
rw [hf₁_def]
exact div_nonneg (abs_nonneg _) (Real.sqrt_nonneg _)
have hf₂_nonneg : 0 ≤ᵐ[ν] f₂ := Filter.Eventually.of_forall fun x => by
rw [hf₂_def]
exact Real.sqrt_nonneg _
have hholder :
∫ x, f₁ x * f₂ x ∂ν
≤ (∫ x, f₁ x ^ (2 : ℝ) ∂ν) ^ (1 / (2 : ℝ))
* (∫ x, f₂ x ^ (2 : ℝ) ∂ν) ^ (1 / (2 : ℝ)) :=
integral_mul_le_Lp_mul_Lq_of_nonneg Real.HolderConjugate.two_two
hf₁_nonneg hf₂_nonneg hf₁L2 hf₂L2
have hLHS : ∫ x, f₁ x * f₂ x ∂ν = ∫ x, |p x - 1| ∂ν := by
apply integral_congr_ae
exact Filter.Eventually.of_forall fun x => by
rw [hf₁_def, hf₂_def]
exact pinsker_abs_div_sqrt_mul_sqrt (hp_nonneg x)
have hf₁_rpow : ∫ x, f₁ x ^ (2 : ℝ) ∂ν = ∫ x, g x ∂ν := by
apply integral_congr_ae
exact Filter.Eventually.of_forall fun x => by
rw [hf₁_def, hg_def]
change (|p x - 1| / √(p x + 2)) ^ (2 : ℝ) = (p x - 1) ^ 2 / (p x + 2)
rw [Real.rpow_two]
exact pinsker_abs_div_sqrt_sq (hp_nonneg x)
have hf₂_rpow : ∫ x, f₂ x ^ (2 : ℝ) ∂ν = 3 := by
have hsqrt_sq : ∫ x, f₂ x ^ (2 : ℝ) ∂ν = ∫ x, p x + 2 ∂ν := by
apply integral_congr_ae
exact Filter.Eventually.of_forall fun x => by
rw [hf₂_def]
change √(p x + 2) ^ (2 : ℝ) = p x + 2
rw [Real.rpow_two]
exact Real.sq_sqrt (by linarith [hp_nonneg x] : 0 ≤ p x + 2)
rw [hsqrt_sq]
calc
∫ x, p x + 2 ∂ν = (∫ x, p x ∂ν) + ∫ _ : Ω, (2 : ℝ) ∂ν :=
integral_add hp_int (integrable_const 2)
_ = 3 := by rw [hp_integral_one]; norm_num
have h_int_abs : ∫ x, |p x - 1| ∂ν ≤ sqrt (2 * K) := by
rw [hLHS, hf₁_rpow, hf₂_rpow] at hholder
have hrpow_g : (∫ x, g x ∂ν) ^ (1 / (2 : ℝ)) = sqrt (∫ x, g x ∂ν) :=
(Real.sqrt_eq_rpow (∫ x, g x ∂ν)).symm
have hrpow_three : (3 : ℝ) ^ (1 / (2 : ℝ)) = sqrt 3 :=
(Real.sqrt_eq_rpow 3).symm
rw [hrpow_g, hrpow_three] at hholder
have hg_int_nonneg : 0 ≤ ∫ x, g x ∂ν :=
integral_nonneg fun x => hg_nonneg x
have hsqrt_g_le : sqrt (∫ x, g x ∂ν) ≤ sqrt ((2 / 3) * K) :=
Real.sqrt_le_sqrt hg_integral_le
have hprod_le :
sqrt (∫ x, g x ∂ν) * sqrt 3
≤ sqrt ((2 / 3) * K) * sqrt 3 := by
exact mul_le_mul_of_nonneg_right hsqrt_g_le (Real.sqrt_nonneg 3)
calc
∫ x, |p x - 1| ∂ν
≤ sqrt (∫ x, g x ∂ν) * sqrt 3 := hholder
_ ≤ sqrt ((2 / 3) * K) * sqrt 3 := hprod_le
_ = sqrt (2 * K) := by
rw [← Real.sqrt_mul (by positivity : 0 ≤ (2 / 3) * K) (3 : ℝ)]
congr 1
ring
have hscheffe := tvDist_le_half_integral_abs_rnDeriv μ ν hac
unfold PinskerBound
have hscheffe_p : tvDist μ ν ≤ (1 / 2) * ∫ x, |p x - 1| ∂ν := by
simpa [hp_def] using hscheffe
calc
tvDist μ ν ≤ (1 / 2) * ∫ x, |p x - 1| ∂ν := hscheffe_p
_ ≤ (1 / 2) * sqrt (2 * K) := by
exact mul_le_mul_of_nonneg_left h_int_abs (by norm_num)
_ = sqrt (K / 2) := pinsker_half_sqrt_two_mul K hK_nonneg
_ = sqrt ((klDiv μ ν).toReal / 2) := by rw [hK_def]
theorem pinskerBound_pi_iid reviewed
⚑ flagged
Causalean.Stat
Pinsker's inequality for i.i.d. finite products (unconditional). For one-sample probability measures μ and ν, if μ is absolutely continuous with respect to ν and the log-likelihood ratio of μ against ν is integrable under μ, then the n-fold product measures μ^{⊗n} and ν^{⊗n} satisfy Pinsker's inequality: their total variation distance is at most the square root of half their Kullback–Leibler divergence.
Formal statement
Proof (Lean source)
theorem pinskerBound_pi_iid {α : Type*} [MeasurableSpace α]
(μ ν : Measure α) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν]
(hac : μ ≪ ν) (hint : Integrable (llr μ ν) μ) (n : ℕ) :
PinskerBound (Measure.pi (fun _ : Fin n => μ)) (Measure.pi (fun _ : Fin n => ν)) := by
have hac_pi :=
pi_iid_absolutelyContinuous μ ν hac n
have hint_pi :=
pi_iid_llr_integrable μ ν hac hint n
exact pinskerBound_of_ac_of_ne_top _ _ hac_pi
(InformationTheory.klDiv_ne_top hac_pi hint_pi)
theorem klForm_two_point_lower_bound_of_pinsker reviewed
⚑ flagged
Causalean.Stat
KL-form Le Cam two-point lower bound. Given a Pinsker-type bound h controlling the total variation distance between P₀ and P₁ by their Kullback–Leibler divergence, if est is a measurable estimator of the parameter and the parameter values θ₀ and θ₁ are separated by at least 2s, then the worse of the two error probabilities — that est misses θ₀ by at least s under P₀, or misses θ₁ by at least s under P₁ — is at least (1 - √(klDiv(P₀,P₁)/2))/2.
Formal statement
hypotheses
Θ :
Type u_2
h :
PinskerBound P₀ P₁
est :
Ω → Θ
hest :
Measurable est
θ₀ θ₁ :
Θ
s :
ℝ
hsep :
2 * s ≤ dist θ₀ θ₁
Proof (Lean source)
theorem klForm_two_point_lower_bound_of_pinsker (h : PinskerBound P₀ P₁)
{est : Ω → Θ} (hest : Measurable est) {θ₀ θ₁ : Θ} {s : ℝ}
(hsep : 2 * s ≤ dist θ₀ θ₁) :
(1 - sqrt ((klDiv P₀ P₁).toReal / 2)) / 2
≤ max (P₀.real {ω | s ≤ dist (est ω) θ₀}) (P₁.real {ω | s ≤ dist (est ω) θ₁}) := by
have hmax := half_one_sub_tvDist_le_max_error (P₀ := P₀) (P₁ := P₁) hest hsep
have htv : (1 - sqrt ((klDiv P₀ P₁).toReal / 2)) / 2
≤ (1 - tvDist P₀ P₁) / 2 := by
have := h
unfold PinskerBound at this
linarith
exact htv.trans hmax
theorem klForm_two_point_lower_bound reviewed
⚑ flagged
Causalean.Stat
KL-form Le Cam two-point lower bound (unconditional). For probability measures P₀ and P₁ and two points θ₀, θ₁ in a pseudometric parameter space, if P₀ is absolutely continuous with respect to P₁ and their Kullback–Leibler divergence is finite, while est is a measurable estimator of the parameter and θ₀ and θ₁ are separated by at least 2s, then the worse of the two error probabilities — that est misses θ₀ by at least s under P₀, or misses θ₁ by at least s under P₁ — is at least (1 - √(klDiv(P₀,P₁)/2))/2.
Formal statement
hypotheses
Θ :
Type u_2
hac :
P₀ ≪ P₁
hfin :
klDiv P₀ P₁ ≠ ⊤
est :
Ω → Θ
hest :
Measurable est
θ₀ θ₁ :
Θ
s :
ℝ
hsep :
2 * s ≤ dist θ₀ θ₁
Proof (Lean source)
theorem klForm_two_point_lower_bound (hac : P₀ ≪ P₁)
(hfin : klDiv P₀ P₁ ≠ ⊤)
{est : Ω → Θ} (hest : Measurable est) {θ₀ θ₁ : Θ} {s : ℝ}
(hsep : 2 * s ≤ dist θ₀ θ₁) :
(1 - sqrt ((klDiv P₀ P₁).toReal / 2)) / 2
≤ max (P₀.real {ω | s ≤ dist (est ω) θ₀}) (P₁.real {ω | s ≤ dist (est ω) θ₁}) :=
klForm_two_point_lower_bound_of_pinsker
(pinskerBound_of_ac_of_ne_top P₀ P₁ hac hfin) hest hsep
2 supporting declarations (lemmas, instances)
-
klFun_mul_lower_auxtheorem — Auxiliary: pinskerPhi x ≥ 0 for x ≥ 0, i.e. (x + 2) * klFun x - (3/2) * (x - 1)^2 ≥ 0.Proof (Lean source)
theorem klFun_mul_lower_aux {x : ℝ} (hx : 0 ≤ x) : 0 ≤ (x + 2) * klFun x - (3 / 2) * (x - 1) ^ 2 := by suffices h : 0 ≤ pinskerPhi x by simpa [pinskerPhi] using h rcases eq_or_lt_of_le hx with h0 | h0 · -- x = 0 rw [← h0]; norm_num [pinskerPhi, klFun] rcases le_total x 1 with hle | hge · -- 0 < x ≤ 1: φ antitone, so φ x ≥ φ 1 = 0 have := pinskerPhi_antitone_Ioc (Set.mem_Ioc.mpr ⟨h0, hle⟩) (Set.mem_Ioc.mpr ⟨one_pos, le_refl 1⟩) hle rw [pinskerPhi_one] at this exact this · -- 1 ≤ x: φ monotone, so φ x ≥ φ 1 = 0 have := pinskerPhi_monotone_Ici (Set.mem_Ici.mpr (le_refl 1)) (Set.mem_Ici.mpr hge) hge rw [pinskerPhi_one] at this exact this -
klFun_lower_boundtheorem — Pinsker scalar inequality. For x ≥ 0, (3/2) * (x - 1)^2 / (x + 2) ≤ klFun x. This is the elementary bound that powers Pinsker's inequality through Cauchy–Schwarz.Proof (Lean source)
theorem klFun_lower_bound {x : ℝ} (hx : 0 ≤ x) : (3 / 2) * (x - 1) ^ 2 / (x + 2) ≤ klFun x := by have hpos : (0 : ℝ) < x + 2 := by linarith rw [div_le_iff₀ hpos] have h := klFun_mul_lower_aux hx nlinarith [h]
BretagnolleHuber 1 core · 6 supporting This module builds the Bhattacharyya/Hellinger affinity layer used to prove the Bretagnolle-Huber inequality. ★ bretagnolle_huber_affinity
Bretagnolle-Huber inequality
This module builds the Bhattacharyya/Hellinger affinity layer used to prove the
Bretagnolle-Huber inequality. The auxiliary results
integral_min_le_one_sub_tvDist, sq_bhattacharyya_le_two_mul_integral_min,
and exp_neg_half_klDiv_le_bhattacharyya combine Scheffe, Cauchy-Schwarz, and
Jensen steps; the headline theorem bretagnolle_huber_affinity gives the
two-point testing floor 1 - tvDist μ ν >= (1/2) * exp(-KL(μ,ν)) for absolutely
continuous probability measures with finite KL divergence.
theorem bretagnolle_huber_affinity reviewed
⚑ flagged
Causalean.Stat
Bretagnolle–Huber inequality. For probability measures μ, ν such that μ is absolutely continuous with respect to ν and their Kullback–Leibler divergence is finite, the two-point testing affinity 1 − tvDist μ ν is at least ½·exp(-KL(μ‖ν)):
Formal statement
hypotheses
Proof (Lean source)
theorem bretagnolle_huber_affinity (μ ν : Measure Ω)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν]
(hac : μ ≪ ν) (hfin : klDiv μ ν ≠ ⊤) :
(1 / 2 : ℝ) * exp (-(klDiv μ ν).toReal) ≤ 1 - tvDist μ ν := by
set K : ℝ := (klDiv μ ν).toReal with hK_def
set ρ : ℝ := ∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν with hρ_def
have hint : Integrable (llr μ ν) μ := (klDiv_ne_top_iff.mp hfin).2
have hρ_nonneg : 0 ≤ ρ := by
rw [hρ_def]; exact integral_nonneg fun x => Real.sqrt_nonneg _
have hL3 : exp (-(1 / 2) * K) ≤ ρ :=
exp_neg_half_klDiv_le_bhattacharyya μ ν hac hint
-- square it: exp(-K) ≤ ρ²
have hexp_sq : exp (-K) = exp (-(1 / 2) * K) ^ 2 := by
have hsum : -K = -(1 / 2) * K + -(1 / 2) * K := by ring
rw [sq, ← Real.exp_add, ← hsum]
have hsq : exp (-K) ≤ ρ ^ 2 := by
rw [hexp_sq]
nlinarith [hL3, Real.exp_nonneg (-(1 / 2) * K), hρ_nonneg]
-- Cauchy–Schwarz + affinity
have hL2 : ρ ^ 2 ≤ 2 * ∫ x, min ((μ.rnDeriv ν x).toReal) 1 ∂ν := by
rw [hρ_def]; exact sq_bhattacharyya_le_two_mul_integral_min μ ν hac
have hL1 : ∫ x, min ((μ.rnDeriv ν x).toReal) 1 ∂ν ≤ 1 - tvDist μ ν :=
integral_min_le_one_sub_tvDist μ ν hac
rw [hK_def]
linarith [hsq, hL2, hL1]
6 supporting declarations (lemmas, instances)
-
mul_exp_neg_half_logtheorem — For y ≥ 0, y * exp(-(1/2)·log y) = √y.Proof (Lean source)
theorem mul_exp_neg_half_log {y : ℝ} (hy : 0 ≤ y) : y * exp (-(1 / 2) * log y) = sqrt y := by rcases eq_or_lt_of_le hy with h0 | h0 · simp [← h0] · -- y > 0 have hsqrt : sqrt y = exp ((1 / 2) * log y) := by rw [Real.sqrt_eq_rpow, Real.rpow_def_of_pos h0] ring_nf rw [hsqrt] nth_rewrite 1 [← Real.exp_log h0] rw [← Real.exp_add] ring_nf -
sqrt_le_half_add_onetheorem — √y ≤ (y + 1)/2 for y ≥ 0 (AM–GM); used to dominate √p by an integrable function. -
integrable_sqrt_rnDerivtheorem — √p is ν-integrable (dominated by (p+1)/2).hypothesesconclusionProof (Lean source)
@[fun_prop] theorem integrable_sqrt_rnDeriv : Integrable (fun x => sqrt ((μ.rnDeriv ν x).toReal)) ν := by have hp : Integrable (fun x => (μ.rnDeriv ν x).toReal) ν := Measure.integrable_toReal_rnDeriv have hmeas : AEStronglyMeasurable (fun x => sqrt ((μ.rnDeriv ν x).toReal)) ν := by apply Measurable.aestronglyMeasurable exact (Measure.measurable_rnDeriv μ ν).ennreal_toReal.sqrt refine Integrable.mono' (g := fun x => ((μ.rnDeriv ν x).toReal + 1) / 2) ((hp.add (integrable_const 1)).div_const 2) hmeas ?_ refine Filter.Eventually.of_forall fun x => ?_ rw [Real.norm_eq_abs, abs_of_nonneg (Real.sqrt_nonneg _)] exact sqrt_le_half_add_one ENNReal.toReal_nonneg -
integral_min_le_one_sub_tvDisttheorem — Affinity lower bound. ∫ min(p,1) ∂ν ≤ 1 - tvDist μ ν. Derived from Scheffé's ≤ inequality together with ∫ min(p,1) ∂ν = 1 - ½∫|p-1|∂ν.hypothesesProof (Lean source)
theorem integral_min_le_one_sub_tvDist (hac : μ ≪ ν) : ∫ x, min ((μ.rnDeriv ν x).toReal) 1 ∂ν ≤ 1 - tvDist μ ν := by set q : Ω → ℝ := fun x => (μ.rnDeriv ν x).toReal with hq_def have hp_int : Integrable q ν := Measure.integrable_toReal_rnDeriv have hp_one : ∫ x, q x ∂ν = 1 := by rw [hq_def, Measure.integral_toReal_rnDeriv hac, measureReal_def, measure_univ] simp have habs_int : Integrable (fun x => |q x - 1|) ν := (hp_int.sub (integrable_const 1)).abs -- min(q,1) = (q + 1 - |q - 1|)/2 pointwise have hmin_eq : ∀ x, min (q x) 1 = (q x + 1 - |q x - 1|) / 2 := by intro x; rcases le_total (q x) 1 with h | h · rw [min_eq_left h, abs_of_nonpos (by linarith)]; ring · rw [min_eq_right h, abs_of_nonneg (by linarith)]; ring have hmin_int : Integrable (fun x => min (q x) 1) ν := by refine (((hp_int.add (integrable_const 1)).sub habs_int).div_const 2).congr ?_ exact Filter.Eventually.of_forall fun x => (hmin_eq x).symm have hone : ∫ _ : Ω, (1 : ℝ) ∂ν = 1 := by rw [integral_const, measureReal_def, measure_univ]; simp have hint_min : ∫ x, min (q x) 1 ∂ν = 1 - (1/2) * ∫ x, |q x - 1| ∂ν := by have heq : (fun x => min (q x) 1) = fun x => (q x + 1) / 2 - |q x - 1| / 2 := by funext x; rw [hmin_eq x]; ring calc ∫ x, min (q x) 1 ∂ν = ∫ x, ((q x + 1) / 2 - |q x - 1| / 2) ∂ν := by rw [heq] _ = (∫ x, (q x + 1) / 2 ∂ν) - ∫ x, |q x - 1| / 2 ∂ν := integral_sub ((hp_int.add (integrable_const 1)).div_const 2) (habs_int.div_const 2) _ = ((∫ x, (q x + 1) ∂ν) / 2) - (∫ x, |q x - 1| ∂ν) / 2 := by rw [integral_div, integral_div] _ = 1 - (1/2) * ∫ x, |q x - 1| ∂ν := by rw [integral_add hp_int (integrable_const 1), hp_one, hone]; ring have hscheffe := tvDist_le_half_integral_abs_rnDeriv μ ν hac rw [hint_min] linarith [hscheffe] -
sq_bhattacharyya_le_two_mul_integral_mintheorem — Cauchy–Schwarz / Bhattacharyya step. (∫ √p ∂ν)² ≤ 2·∫ min(p,1) ∂ν.hypothesesProof (Lean source)
theorem sq_bhattacharyya_le_two_mul_integral_min (hac : μ ≪ ν) : (∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν) ^ 2 ≤ 2 * ∫ x, min ((μ.rnDeriv ν x).toReal) 1 ∂ν := by set q : Ω → ℝ := fun x => (μ.rnDeriv ν x).toReal with hq_def set f₁ : Ω → ℝ := fun x => sqrt (min (q x) 1) with hf₁_def set f₂ : Ω → ℝ := fun x => sqrt (max (q x) 1) with hf₂_def have hq_nonneg : ∀ x, 0 ≤ q x := by intro x rw [hq_def] exact ENNReal.toReal_nonneg have hmin_nonneg : ∀ x, 0 ≤ min (q x) 1 := fun x => le_min (hq_nonneg x) zero_le_one have hmax_nonneg : ∀ x, 0 ≤ max (q x) 1 := fun x => le_trans zero_le_one (le_max_right (q x) 1) have hp_int : Integrable q ν := by rw [hq_def] exact Measure.integrable_toReal_rnDeriv have hp_integral_one : ∫ x, q x ∂ν = 1 := by rw [hq_def, Measure.integral_toReal_rnDeriv hac, measureReal_def, measure_univ] simp have hmin_meas : AEStronglyMeasurable (fun x => min (q x) 1) ν := by rw [hq_def] exact ((Measure.measurable_rnDeriv μ ν).ennreal_toReal.min measurable_const) |>.aestronglyMeasurable have hmax_meas : AEStronglyMeasurable (fun x => max (q x) 1) ν := by rw [hq_def] exact ((Measure.measurable_rnDeriv μ ν).ennreal_toReal.max measurable_const) |>.aestronglyMeasurable have hmin_int : Integrable (fun x => min (q x) 1) ν := by refine Integrable.mono' hp_int hmin_meas ?_ exact Filter.Eventually.of_forall fun x => by have h : min (q x) 1 ≤ q x := min_le_left (q x) 1 simp [Real.norm_eq_abs, abs_of_nonneg (hmin_nonneg x), h] have hmax_int : Integrable (fun x => max (q x) 1) ν := by refine Integrable.mono' (hp_int.add (integrable_const 1)) hmax_meas ?_ exact Filter.Eventually.of_forall fun x => by have h : max (q x) 1 ≤ q x + 1 := max_le (by linarith [hq_nonneg x]) (by linarith [hq_nonneg x]) simpa [Real.norm_eq_abs, abs_of_nonneg (hmax_nonneg x), Pi.add_apply] using h have hf₁_meas : AEStronglyMeasurable f₁ ν := by rw [hf₁_def, hq_def] exact ((Measure.measurable_rnDeriv μ ν).ennreal_toReal.min measurable_const) |>.sqrt.aestronglyMeasurable have hf₂_meas : AEStronglyMeasurable f₂ ν := by rw [hf₂_def, hq_def] exact ((Measure.measurable_rnDeriv μ ν).ennreal_toReal.max measurable_const) |>.sqrt.aestronglyMeasurable have hf₁_sq_int : Integrable (fun x => f₁ x ^ 2) ν := by refine hmin_int.congr (Filter.Eventually.of_forall fun x => ?_) rw [hf₁_def] exact (Real.sq_sqrt (hmin_nonneg x)).symm have hf₂_sq_int : Integrable (fun x => f₂ x ^ 2) ν := by refine hmax_int.congr (Filter.Eventually.of_forall fun x => ?_) rw [hf₂_def] exact (Real.sq_sqrt (hmax_nonneg x)).symm have hf₁L2 : MemLp f₁ (ofReal 2) ν := by simpa using (memLp_two_iff_integrable_sq hf₁_meas).2 hf₁_sq_int have hf₂L2 : MemLp f₂ (ofReal 2) ν := by simpa using (memLp_two_iff_integrable_sq hf₂_meas).2 hf₂_sq_int have hf₁_nonneg : 0 ≤ᵐ[ν] f₁ := Filter.Eventually.of_forall fun x => by rw [hf₁_def] exact Real.sqrt_nonneg _ have hf₂_nonneg : 0 ≤ᵐ[ν] f₂ := Filter.Eventually.of_forall fun x => by rw [hf₂_def] exact Real.sqrt_nonneg _ have hholder : ∫ x, f₁ x * f₂ x ∂ν ≤ (∫ x, f₁ x ^ (2 : ℝ) ∂ν) ^ (1 / (2 : ℝ)) * (∫ x, f₂ x ^ (2 : ℝ) ∂ν) ^ (1 / (2 : ℝ)) := integral_mul_le_Lp_mul_Lq_of_nonneg Real.HolderConjugate.two_two hf₁_nonneg hf₂_nonneg hf₁L2 hf₂L2 have hprod : ∀ x, f₁ x * f₂ x = sqrt (q x) := by intro x rw [hf₁_def, hf₂_def, ← Real.sqrt_mul (hmin_nonneg x) (max (q x) 1)] have hminmax : min (q x) 1 * max (q x) 1 = q x := by rcases le_total (q x) 1 with h | h · rw [min_eq_left h, max_eq_right h, mul_one] · rw [min_eq_right h, max_eq_left h, one_mul] rw [hminmax] have hLHS : ∫ x, f₁ x * f₂ x ∂ν = ∫ x, sqrt (q x) ∂ν := by apply integral_congr_ae exact Filter.Eventually.of_forall hprod set A : ℝ := ∫ x, min (q x) 1 ∂ν with hA_def set B : ℝ := ∫ x, max (q x) 1 ∂ν with hB_def have hf₁_rpow : ∫ x, f₁ x ^ (2 : ℝ) ∂ν = A := by rw [hA_def] apply integral_congr_ae exact Filter.Eventually.of_forall fun x => by rw [hf₁_def] change sqrt (min (q x) 1) ^ (2 : ℝ) = min (q x) 1 rw [Real.rpow_two] exact Real.sq_sqrt (hmin_nonneg x) have hf₂_rpow : ∫ x, f₂ x ^ (2 : ℝ) ∂ν = B := by rw [hB_def] apply integral_congr_ae exact Filter.Eventually.of_forall fun x => by rw [hf₂_def] change sqrt (max (q x) 1) ^ (2 : ℝ) = max (q x) 1 rw [Real.rpow_two] exact Real.sq_sqrt (hmax_nonneg x) have hA_nonneg : 0 ≤ A := by rw [hA_def] exact integral_nonneg hmin_nonneg have hB_nonneg : 0 ≤ B := by rw [hB_def] exact integral_nonneg hmax_nonneg have hsqrt_bound : ∫ x, sqrt (q x) ∂ν ≤ sqrt A * sqrt B := by rw [hLHS, hf₁_rpow, hf₂_rpow] at hholder have hrpow_A : A ^ (1 / (2 : ℝ)) = sqrt A := (Real.sqrt_eq_rpow A).symm have hrpow_B : B ^ (1 / (2 : ℝ)) = sqrt B := (Real.sqrt_eq_rpow B).symm rw [hrpow_A, hrpow_B] at hholder exact hholder have hB_le_two : B ≤ 2 := by have hdom_int : Integrable (fun x => q x + 1) ν := hp_int.add (integrable_const 1) have hle_int : ∫ x, max (q x) 1 ∂ν ≤ ∫ x, q x + 1 ∂ν := integral_mono_ae hmax_int hdom_int (Filter.Eventually.of_forall fun x => by change max (q x) 1 ≤ q x + 1 exact max_le (by linarith [hq_nonneg x]) (by linarith [hq_nonneg x])) calc B = ∫ x, max (q x) 1 ∂ν := by rw [hB_def] _ ≤ ∫ x, q x + 1 ∂ν := hle_int _ = 2 := by rw [integral_add hp_int (integrable_const 1), hp_integral_one] norm_num have hsqrt_int_nonneg : 0 ≤ ∫ x, sqrt (q x) ∂ν := integral_nonneg fun x => Real.sqrt_nonneg _ have hsq_le : (∫ x, sqrt (q x) ∂ν) ^ 2 ≤ (sqrt A * sqrt B) ^ 2 := by nlinarith [hsqrt_bound, hsqrt_int_nonneg, Real.sqrt_nonneg A, Real.sqrt_nonneg B] have hsqrt_prod_sq : (sqrt A * sqrt B) ^ 2 = A * B := by rw [mul_pow, Real.sq_sqrt hA_nonneg, Real.sq_sqrt hB_nonneg] have hAB_le : A * B ≤ 2 * A := by nlinarith [hA_nonneg, hB_le_two] calc (∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν) ^ 2 = (∫ x, sqrt (q x) ∂ν) ^ 2 := by rw [hq_def] _ ≤ (sqrt A * sqrt B) ^ 2 := hsq_le _ = A * B := hsqrt_prod_sq _ ≤ 2 * A := hAB_le _ = 2 * ∫ x, min ((μ.rnDeriv ν x).toReal) 1 ∂ν := by rw [hA_def, hq_def] -
exp_neg_half_klDiv_le_bhattacharyyatheorem — Jensen / Bhattacharyya step. exp(-½·KL) ≤ ∫ √p ∂ν.hypotheseshac :μ ≪ νhint :Integrable (llr μ ν) μProof (Lean source)
theorem exp_neg_half_klDiv_le_bhattacharyya (hac : μ ≪ ν) (hint : Integrable (llr μ ν) μ) : exp (-(1 / 2) * (klDiv μ ν).toReal) ≤ ∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν := by have hK : (klDiv μ ν).toReal = ∫ a, llr μ ν a ∂μ := InformationTheory.toReal_klDiv_of_measure_eq hac (by simp [measure_univ]) have hCOV : ∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν = ∫ x, exp (-(1 / 2) * llr μ ν x) ∂μ := by have hcov0 : ∫ x, (μ.rnDeriv ν x).toReal • exp (-(1 / 2) * llr μ ν x) ∂ν = ∫ x, exp (-(1 / 2) * llr μ ν x) ∂μ := MeasureTheory.integral_rnDeriv_smul hac have hcov_lhs : ∫ x, (μ.rnDeriv ν x).toReal • exp (-(1 / 2) * llr μ ν x) ∂ν = ∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν := by apply integral_congr_ae exact Filter.Eventually.of_forall fun x => by change (μ.rnDeriv ν x).toReal * exp (-(1 / 2) * log (μ.rnDeriv ν x).toReal) = sqrt ((μ.rnDeriv ν x).toReal) exact mul_exp_neg_half_log ENNReal.toReal_nonneg calc ∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν = ∫ x, (μ.rnDeriv ν x).toReal • exp (-(1 / 2) * llr μ ν x) ∂ν := hcov_lhs.symm _ = ∫ x, exp (-(1 / 2) * llr μ ν x) ∂μ := hcov0 have hfi : Integrable (fun x => (-(1 / 2 : ℝ)) * llr μ ν x) μ := hint.const_mul (-(1 / 2 : ℝ)) have hgi : Integrable (fun x => exp (-(1 / 2) * llr μ ν x)) μ := by refine (integrable_rnDeriv_smul_iff hac).mp ?_ refine (integrable_sqrt_rnDeriv μ ν).congr (Filter.Eventually.of_forall fun x => ?_) change sqrt ((μ.rnDeriv ν x).toReal) = (μ.rnDeriv ν x).toReal * exp (-(1 / 2) * log (μ.rnDeriv ν x).toReal) exact (mul_exp_neg_half_log ENNReal.toReal_nonneg).symm have hfs : ∀ᵐ x ∂μ, (-(1 / 2 : ℝ)) * llr μ ν x ∈ (Set.univ : Set ℝ) := Filter.Eventually.of_forall fun x => Set.mem_univ _ have hJensen : exp (∫ x, (-(1 / 2 : ℝ)) * llr μ ν x ∂μ) ≤ ∫ x, exp (-(1 / 2) * llr μ ν x) ∂μ := by simpa only [Function.comp_apply] using (convexOn_exp.map_integral_le (μ := μ) (f := fun x => (-(1 / 2 : ℝ)) * llr μ ν x) Real.continuous_exp.continuousOn isClosed_univ hfs hfi (by simpa only [Function.comp_def] using hgi)) have harg : ∫ x, (-(1 / 2 : ℝ)) * llr μ ν x ∂μ = -(1 / 2) * (klDiv μ ν).toReal := by rw [integral_const_mul, ← hK] calc exp (-(1 / 2) * (klDiv μ ν).toReal) = exp (∫ x, (-(1 / 2 : ℝ)) * llr μ ν x ∂μ) := by rw [harg] _ ≤ ∫ x, exp (-(1 / 2) * llr μ ν x) ∂μ := hJensen _ = ∫ x, sqrt ((μ.rnDeriv ν x).toReal) ∂ν := hCOV.symm
ChiSquaredFinite 1 core · 2 supporting This file proves discrete and product-measure identities for chi-squared divergence on finite sample spaces. ★ finite_one_add_chiSqDiv
Finite Chi-Squared Identities
This file proves discrete and product-measure identities for chi-squared divergence on finite sample spaces. These identities turn Radon-Nikodym formulas into point-mass sums and product point masses for finite minimax constructions.
theorem finite_one_add_chiSqDiv reviewed
⚑ flagged
Causalean.Stat
Finite χ²-divergence formula. On a finite sample space, for probability measures μ, ν with μ absolutely continuous with respect to ν, the (shifted) χ²-divergence is the explicit sum of squared point-mass ratios: 1 + χ²(μ‖ν) = ∑ₓ (μ{x})² / (ν{x}). Combines chiSqDiv_eq, the finite integral formula integral_fintype, and the discrete RN bridge above.
Formal statement
Proof (Lean source)
theorem finite_one_add_chiSqDiv {Ω : Type*} [MeasurableSpace Ω] [Fintype Ω]
[MeasurableSingletonClass Ω] (μ ν : Measure Ω)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν) :
1 + chiSqDiv μ ν = ∑ x, (μ.real {x}) ^ 2 / (ν.real {x}) := by
have hexp : 1 + chiSqDiv μ ν = ∫ x, ((μ.rnDeriv ν x).toReal) ^ 2 ∂ν := by
rw [chiSqDiv_eq hac (Integrable.of_finite)]; ring
rw [hexp, integral_fintype (Integrable.of_finite)]
apply Finset.sum_congr rfl
intro x _
by_cases h : ν {x} = 0
· -- `ν {x} = 0` forces `μ {x} = 0` (absolute continuity); both sides vanish.
have hνr : ν.real {x} = 0 := by rw [measureReal_def, h, ENNReal.toReal_zero]
have hμr : μ.real {x} = 0 := by rw [measureReal_def, hac h, ENNReal.toReal_zero]
rw [hνr, hμr]; simp
· have hνtop : ν {x} ≠ ⊤ := measure_ne_top ν {x}
have hνr0 : ν.real {x} ≠ 0 := by
rw [measureReal_def]; exact ENNReal.toReal_ne_zero.2 ⟨h, hνtop⟩
-- Invert the density on the atom `{x}` via the discrete RN bridge.
have hrn : (μ.rnDeriv ν x).toReal = μ.real {x} / ν.real {x} := by
have hc := congrArg toReal (rnDeriv_mul_measure_singleton μ ν hac x)
rw [ENNReal.toReal_mul, ← measureReal_def, ← measureReal_def] at hc
rw [eq_div_iff hνr0, mul_comm, hc]
rw [hrn, smul_eq_mul]
field_simp
2 supporting declarations (lemmas, instances)
-
rnDeriv_mul_measure_singletontheorem — Discrete Radon–Nikodym bridge. On a measurable singleton {x}, the Radon–Nikodym density scaled by the base mass recovers the numerator mass: ν {x} · (dμ/dν)(x) = μ {x}, for μ ≪ ν. Proved by integrating the density over {x} (via setLIntegral_rnDeriv) and collapsing the singleton integral with restrict_singleton/lintegral_dirac'.hypothesesΩ :hac :μ ≪ νx :Ωconclusionν {x} * μ.rnDeriv ν x = μ {x}Proof (Lean source)
theorem rnDeriv_mul_measure_singleton {Ω : Type*} [MeasurableSpace Ω] [MeasurableSingletonClass Ω] (μ ν : Measure Ω) [SFinite ν] [μ.HaveLebesgueDecomposition ν] (hac : μ ≪ ν) (x : Ω) : ν {x} * μ.rnDeriv ν x = μ {x} := by have h1 : μ {x} = ∫⁻ y in {x}, μ.rnDeriv ν y ∂ν := (Measure.setLIntegral_rnDeriv hac {x}).symm have h2 : ∫⁻ y in {x}, μ.rnDeriv ν y ∂ν = ν {x} * μ.rnDeriv ν x := by rw [Measure.restrict_singleton, lintegral_smul_measure, lintegral_dirac' x (Measure.measurable_rnDeriv μ ν), smul_eq_mul] rw [h1, h2] -
pi_real_singletontheorem — Real-valued product point mass. Under the product measure Measure.pi μ of a family of probability measures, the mass of a single point factorizes as the product of the marginal point masses: (Measure.pi μ).real {ω} = ∏ᵢ (μ i).real {ωᵢ}. Rewrites the singleton as Set.univ.pi (fun i => {ωᵢ}), applies Measure.pi_pi, then distributes toReal over the finite product.hypothesesι :∀ i, MeasurableSpace (Ω i)∀ i, MeasurableSingletonClass (Ω i)μ :∀ i, Measure (Ω i)∀ i, IsProbabilityMeasure (μ i)ω :∀ i, Ω iProof (Lean source)
theorem pi_real_singleton {ι : Type*} [Fintype ι] {Ω : ι → Type*} [∀ i, MeasurableSpace (Ω i)] [∀ i, MeasurableSingletonClass (Ω i)] (μ : ∀ i, Measure (Ω i)) [∀ i, IsProbabilityMeasure (μ i)] (ω : ∀ i, Ω i) : (Measure.pi μ).real {ω} = ∏ i, (μ i).real {ω i} := by have hset : ({ω} : Set (∀ i, Ω i)) = Set.univ.pi (fun i => {ω i}) := by ext f simp [Set.mem_pi, funext_iff] rw [measureReal_def, hset, Measure.pi_pi, ENNReal.toReal_prod] simp only [measureReal_def]
ChiSquaredKernel 2 core · 1 supporting This module constructs a marked law that retains its base coordinate and samples its mark from a probability kernel. ★ one_add_chiSqDiv_attachKernel
Chi-squared divergence for retained-design kernel laws
This module constructs a marked law that retains its base coordinate and samples its mark from a probability kernel. It states the chi-squared disintegration formula for two such laws with the same base marginal.
def attachKernel reviewed
⚑ flagged
Causalean.Stat
Given a measurable design space, a measurable mark space, a measure on the design space, and a probability kernel from design points to mark laws, the attached marked-observation law draws a design point from the measure and then a mark from the corresponding kernel, retaining both coordinates.
Definition (Lean source)
parameters
def
given by
m ⊗ₘ kappa
theorem one_add_chiSqDiv_attachKernel reviewed
⚑ flagged
Causalean.Stat
For a probability design law, two measurable probability marking kernels, pointwise absolute continuity of the first mark law with respect to the second, and integrability of their jointly measurable squared density deviation, one plus the χ²-divergence of the resulting marked laws equals the design-average of one plus the conditional χ²-divergences.
Formal statement
hypotheses
D M :
m :
kappa eta :
hac :
forall x, kappa x ≪ eta x
hint :
conclusion
= ∫ x, (1 + chiSqDiv (kappa x) (eta x)) ∂m
Proof (Lean source)
theorem one_add_chiSqDiv_attachKernel
{D M : Type*} [MeasurableSpace D] [MeasurableSpace M]
[CountableOrCountablyGenerated D M]
(m : Measure D) [IsProbabilityMeasure m]
(kappa eta : Kernel D M) [IsMarkovKernel kappa] [IsMarkovKernel eta]
(hac : forall x, kappa x ≪ eta x)
(hint : Integrable
(fun p : D × M =>
((kappa.rnDeriv eta p.1 p.2).toReal - 1) ^ 2)
(attachKernel m eta)) :
1 + chiSqDiv (attachKernel m kappa) (attachKernel m eta) =
∫ x, (1 + chiSqDiv (kappa x) (eta x)) ∂m := by
let f : D × M → ℝ := fun p => ((kappa.rnDeriv eta p.1 p.2).toReal - 1) ^ 2
have hmeas : Measurable (fun p : D × M => kappa.rnDeriv eta p.1 p.2) := by
fun_prop
have hattach : attachKernel m kappa =
(attachKernel m eta).withDensity (fun p => kappa.rnDeriv eta p.1 p.2) := by
unfold attachKernel
calc
m ⊗ₘ kappa = m ⊗ₘ eta.withDensity (kappa.rnDeriv eta) :=
Measure.compProd_congr <| Filter.Eventually.of_forall fun x =>
(Kernel.withDensity_rnDeriv_eq (hac x)).symm
_ = (m ⊗ₘ eta).withDensity (fun p => kappa.rnDeriv eta p.1 p.2) :=
Measure.compProd_withDensity hmeas
have hrn : (attachKernel m kappa).rnDeriv (attachKernel m eta) =ᵐ[attachKernel m eta]
fun p => kappa.rnDeriv eta p.1 p.2 := by
rw [hattach]
exact Measure.rnDeriv_withDensity _ hmeas
have hchiJoint : chiSqDiv (attachKernel m kappa) (attachKernel m eta) =
∫ p, f p ∂(attachKernel m eta) := by
rw [chiSqDiv]
exact integral_congr_ae (hrn.mono fun p hp => by simp only [f, hp])
have hfiber (x : D) : chiSqDiv (kappa x) (eta x) =
∫ y, f (x, y) ∂(eta x) := by
rw [chiSqDiv]
exact integral_congr_ae
((Kernel.rnDeriv_eq_rnDeriv_measure (κ := kappa) (η := eta) (a := x)).mono
fun y hy => by simp only [f, hy]) |>.symm
have hintf : Integrable f (m ⊗ₘ eta) := by
simpa only [f, attachKernel] using hint
have hinner : Integrable (fun x => ∫ y, f (x, y) ∂(eta x)) m := by
have hnorm := (Measure.integrable_compProd_iff hintf.aestronglyMeasurable).mp hintf |>.2
simpa [f, Real.norm_eq_abs, abs_of_nonneg] using hnorm
calc
1 + chiSqDiv (attachKernel m kappa) (attachKernel m eta) =
1 + ∫ p, f p ∂(attachKernel m eta) := by rw [hchiJoint]
_ = 1 + ∫ x, ∫ y, f (x, y) ∂(eta x) ∂m := by
exact congrArg (fun z => 1 + z) (Measure.integral_compProd hintf)
_ = ∫ x, (1 + ∫ y, f (x, y) ∂(eta x)) ∂m := by
rw [integral_add (integrable_const 1) hinner]
simp
_ = ∫ x, (1 + chiSqDiv (kappa x) (eta x)) ∂m := by
exact integral_congr_ae <| Filter.Eventually.of_forall fun x => by
dsimp
rw [← hfiber x]
1 supporting declaration (lemmas, instances)
-
instIsProbabilityMeasureinstance — A marked-observation law formed from a probability base and a probability kernel is itself a probability law.parametersinstancegiven byby unfold attachKernel infer_instance
ChiSquaredTwoPoint 1 core · 1 supporting This module evaluates chi-squared divergence for the Bernoulli perturbation around one half and for its equivalent signed two-point representation. ★ chiSqDiv_twoPointMean_centerHalf
Explicit chi-squared divergence for centered two-point laws
This module evaluates chi-squared divergence for the Bernoulli perturbation around one half and for its equivalent signed two-point representation.
theorem chiSqDiv_twoPointMean_centerHalf reviewed
⚑ flagged
Causalean.Stat
For a mean perturbation whose magnitude is strictly below one half, the χ²-divergence of the symmetric two-point law with that mean, relative to the centered symmetric two-point law, is four times the squared perturbation.
Formal statement
hypotheses
conclusion
Proof (Lean source)
theorem chiSqDiv_twoPointMean_centerHalf {u : Real} (hu : abs u < 1 / 2) :
chiSqDiv (twoPointMean (1 / 2) u) (twoPointMean (1 / 2) 0) =
4 * u ^ 2 := by
let e : ℝ ≃ᵐ ℝ :=
(affineHomeomorph (2 * (1 / 2 : ℝ)) (-(1 / 2 : ℝ)) (by norm_num)).toMeasurableEquiv
have hub := abs_lt.mp hu
haveI : IsProbabilityMeasure (bernoulliLaw (1 / 2 + u)) :=
bernoulliLaw_isProbabilityMeasure (by linarith [hub.1]) (by linarith [hub.2])
haveI : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_isProbabilityMeasure (by norm_num) (by norm_num)
rw [twoPointMean_eq_map_bernoulli (1 / 2) u (by norm_num),
twoPointMean_eq_map_bernoulli (1 / 2) 0 (by norm_num)]
have hp : (1 + u / (1 / 2)) / 2 = (1 / 2 + u : ℝ) := by ring
have hq : (1 + 0 / (1 / 2)) / 2 = (1 / 2 : ℝ) := by ring
rw [hp, hq]
change chiSqDiv
(Measure.map e (bernoulliLaw (1 / 2 + u)))
(Measure.map e (bernoulliLaw (1 / 2))) = 4 * u ^ 2
rw [chiSqDiv_map_measurableEquiv]
exact chiSqDiv_bernoulliLaw_centerHalf hu
1 supporting declaration (lemmas, instances)
-
chiSqDiv_bernoulliLaw_centerHalftheorem — For a perturbation whose magnitude is strictly below one half, the χ²-divergence of the Bernoulli law with success probability one half plus that perturbation, relative to the centered Bernoulli law, is four times the squared perturbation.hypothesesconclusionProof (Lean source)
theorem chiSqDiv_bernoulliLaw_centerHalf {u : Real} (hu : abs u < 1 / 2) : chiSqDiv (bernoulliLaw (1 / 2 + u)) (bernoulliLaw (1 / 2)) = 4 * u ^ 2 := by classical let p : ℝ := 1 / 2 + u let q : ℝ := 1 / 2 have hub := abs_lt.mp hu have hp0 : 0 ≤ p := by dsimp [p]; linarith have hp1 : p ≤ 1 := by dsimp [p]; linarith have hq0 : 0 < q := by norm_num [q] have hq1 : q < 1 := by norm_num [q] haveI : IsProbabilityMeasure (bernoulliLaw q) := bernoulliLaw_isProbabilityMeasure hq0.le hq1.le let g : ℝ → ENNReal := fun x => if x = 1 then ofReal (p / q) else ofReal ((1 - p) / (1 - q)) have hg : Measurable g := by dsimp [g] exact Measurable.ite (measurableSet_singleton (1 : ℝ)) measurable_const measurable_const have hq_ne0 : ofReal q ≠ 0 := by intro h have hle := ENNReal.ofReal_eq_zero.mp h linarith have h1q_ne0 : ofReal (1 - q) ≠ 0 := by intro h have hle := ENNReal.ofReal_eq_zero.mp h linarith have hwd : bernoulliLaw p = (bernoulliLaw q).withDensity g := by ext s hs rw [withDensity_apply _ hs, ← lintegral_indicator hs g] unfold bernoulliLaw dsimp [g] rw [lintegral_add_measure, lintegral_smul_measure, lintegral_smul_measure] simp only [lintegral_dirac] by_cases h1 : (1 : ℝ) ∈ s · by_cases h0 : (0 : ℝ) ∈ s · simp [h1, h0, ENNReal.ofReal_div_of_pos hq0, ENNReal.ofReal_div_of_pos (sub_pos.mpr hq1), ENNReal.mul_div_cancel hq_ne0 ENNReal.ofReal_ne_top, ENNReal.mul_div_cancel h1q_ne0 ENNReal.ofReal_ne_top] · simp [h1, h0, ENNReal.ofReal_div_of_pos hq0, ENNReal.mul_div_cancel hq_ne0 ENNReal.ofReal_ne_top] · by_cases h0 : (0 : ℝ) ∈ s · simp [h1, h0, ENNReal.ofReal_div_of_pos (sub_pos.mpr hq1), ENNReal.mul_div_cancel h1q_ne0 ENNReal.ofReal_ne_top] · simp [h1, h0] have hrn : (bernoulliLaw p).rnDeriv (bernoulliLaw q) =ᵐ[bernoulliLaw q] g := by rw [hwd] exact Measure.rnDeriv_withDensity _ hg change chiSqDiv (bernoulliLaw p) (bernoulliLaw q) = 4 * u ^ 2 rw [chiSqDiv] trans ∫ x, ((g x).toReal - 1) ^ 2 ∂(bernoulliLaw q) · exact integral_congr_ae <| hrn.mono fun x hx => by dsimp rw [hx] rw [bernoulliLaw_integral hq0.le hq1.le] dsimp [g] have hpq_nonneg : 0 ≤ p / q := div_nonneg hp0 hq0.le have hcp_nonneg : 0 ≤ (1 - p) / (1 - q) := div_nonneg (sub_nonneg.mpr hp1) (sub_nonneg.mpr hq1.le) simp only [if_pos, if_neg zero_ne_one] rw [ENNReal.toReal_ofReal hpq_nonneg, ENNReal.toReal_ofReal hcp_nonneg] dsimp [p, q] ring
CoordinatewiseOverlap 7 core · 12 supporting This file states a world-independent decentralized testing bound. ★ coordinatewise_overlap_direct_product
Coordinatewise overlap direct-product bound
This file states a world-independent decentralized testing bound. Each decoder may inspect its compressed local coordinate, all other coordinates, and a common ancillary variable.
def compressedCoordinateLaw reviewed
⚑ flagged
Causalean.Stat
For a compression from a measurable observation space to a measurable summary space and a measure on the observation space, the compressed-coordinate law is the image measure of the summary obtained by applying the compression to an observation governed by that measure.
Definition (Lean source)
parameters
def
given by
Measure.map compress Q
def coordinatewiseSuccessProbability reviewed
⚑ flagged
Causalean.Stat
For a nonnegative number of coordinates, measurable raw-observation and summary spaces at every coordinate, a measurable ancillary space, two probability laws for each coordinate, indexed by its binary state, a probability law for a common ancillary variable, one compression for each coordinate, and a decoder for each coordinate that uses its compressed observation, the full raw observation vector, and the ancillary variable, the coordinatewise success probability is the average, over all binary state vectors, of the probability that every decoder recovers its corresponding state under the associated independent product experiment.
Definition (Lean source)
parameters
M :
ℕ
Fin M → Type*
A :
Type*
∀ j, MeasurableSpace (Z j)
∀ j, MeasurableSpace (S j)
R :
Measure A
∀ j b, IsProbabilityMeasure (Q j b)
compress :
∀ j
if
Z j
then
S j
def
coordinatewiseSuccessProbability Q R compress decoder :
ℝ≥0∞
def coordinateOverlap reviewed
⚑ flagged
Causalean.Stat
For two probability laws at each coordinate, indexed by its binary state, a compression at each coordinate, and one coordinate, the common-part overlap at that coordinate is one minus the total-variation distance between the two compressed laws of that coordinate.
Definition (Lean source)
parameters
M :
ℕ
Fin M → Type*
∀ j, MeasurableSpace (Z j)
∀ j, MeasurableSpace (S j)
compress :
∀ j
if
Z j
then
S j
j :
Fin M
def
coordinateOverlap Q compress j :
ℝ
given by
1
- tvDist (compressedCoordinateLaw (compress j) (Q j false)) (compressedCoordinateLaw (compress j) (Q j true))
def selectCoupledRaw reviewed
⚑ flagged
Causalean.Stat
For a binary hypercube vertex and a pair of raw observations at every coordinate, the selected raw observation vector takes the first member of each pair when the corresponding vertex bit is false and the second member when it is true.
def coupledDecoderGood reviewed
⚑ flagged
Causalean.Stat
For a nonnegative number of coordinates, one compression for each coordinate, one decoder for each coordinate, a binary hypercube vertex, a pair of raw observations at every coordinate, and an ancillary-variable value, the coupled decoder-good condition holds exactly when every decoder, applied to its compressed selected observation together with the full selected raw vector and the ancillary value, returns its corresponding vertex bit.
Definition (Lean source)
parameters
def
coupledDecoderGood compress decoder omega z a :
Prop
given by
decoder j (compress j (selectCoupledRaw omega z j)) (selectCoupledRaw omega z) a = omega j
def coupledGoodIndicator reviewed
⚑ flagged
Causalean.Stat
For a nonnegative number of coordinates, one compression for each coordinate, one decoder for each coordinate, a binary hypercube vertex, a pair of raw observations at every coordinate, and an ancillary-variable value, the coupled decoder-good condition holds exactly when every decoder, applied to its compressed selected observation together with the full selected raw vector and the ancillary value, returns its corresponding vertex bit.
Definition (Lean source)
parameters
def
coupledGoodIndicator compress decoder omega z a :
ℝ≥0∞
given by
by classical exact if coupledDecoderGood compress decoder omega z a then 1 else 0
lemma coordinatewise_overlap_direct_product reviewed
⚑ flagged
Causalean.Stat
Coordinatewise-overlap direct-product bound. In a conditionally independent hypercube experiment with at least one coordinate, per-coordinate candidate laws Q j, indexed by a bit, a per-coordinate compression map compress that is measurable at every coordinate, and per-coordinate decoders decoder built from the compressed local summary, the other coordinates' raw data, and shared randomness that are jointly measurable and depend on the raw sample at coordinate j only through its compressed summary, not directly on the raw value at j, then decentralized coordinate decoders cannot on average be correct more often than the common-part product bound built from the per-coordinate total-variation overlaps; and if every compressed adjacent KL divergence is at most κ log M, the displayed finite-M simultaneous-error certificate follows.
Formal statement
hypotheses
M :
ℕ
hM :
1 ≤ M
Z S :
Fin M → Type*
A :
Type*
∀ j, MeasurableSpace (Z j)
∀ j, StandardBorelSpace (Z j)
∀ j, MeasurableSpace (S j)
∀ j, StandardBorelSpace (S j)
R :
Measure A
∀ j b, IsProbabilityMeasure (Q j b)
compress :
∀ j
if
Z j
then
S j
hcompress :
∀ j, Measurable (compress j)
hdecoder :
∀ j, Measurable (fun p : S j × ((k : Fin M) → Z k) × A => decoder j p.1 p.2.1 p.2.2)
hlocal :
∀ j s z z' a
if
(∀ k, k ≠ j → z k = z' k)
then
decoder j s z a = decoder j s z' a
conclusion
conclusion 1
coordinatewiseSuccessProbability Q R compress decoder
≤ ofReal ((1 / 2 : ℝ) * (1 + ∏ j, (1 - coordinateOverlap Q compress j)))
conclusion 2
κ :
ℝ
κ < 1
(∀ j, klDiv (compressedCoordinateLaw (compress j) (Q j false)) (compressedCoordinateLaw (compress j) (Q j true)) ≤ ofReal (κ * log M))
≤ 1 - coordinatewiseSuccessProbability Q R compress decoder
Proof (Lean source)
lemma coordinatewise_overlap_direct_product
{M : ℕ} (hM : 1 ≤ M)
{Z S : Fin M → Type*} {A : Type*}
[∀ j, MeasurableSpace (Z j)] [∀ j, StandardBorelSpace (Z j)]
[∀ j, MeasurableSpace (S j)] [∀ j, StandardBorelSpace (S j)]
[MeasurableSpace A] [StandardBorelSpace A]
(Q : ∀ j, Bool → Measure (Z j)) (R : Measure A)
[∀ j b, IsProbabilityMeasure (Q j b)] [IsProbabilityMeasure R]
(compress : ∀ j, Z j → S j) (hcompress : ∀ j, Measurable (compress j))
(decoder : ∀ j, S j → ((k : Fin M) → Z k) → A → Bool)
(hdecoder : ∀ j, Measurable
(fun p : S j × ((k : Fin M) → Z k) × A => decoder j p.1 p.2.1 p.2.2))
(hlocal : ∀ j s z z' a,
(∀ k, k ≠ j → z k = z' k) → decoder j s z a = decoder j s z' a) :
coordinatewiseSuccessProbability Q R compress decoder ≤
ofReal
((1 / 2 : ℝ) *
(1 + ∏ j, (1 - coordinateOverlap Q compress j))) ∧
∀ κ : ℝ, κ < 1 →
(∀ j,
klDiv
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true)) ≤
ofReal (κ * log M)) →
ofReal
((1 / 2 : ℝ) *
(1 - exp (-((M : ℝ) ^ (1 - κ)) / 2))) ≤
1 - coordinatewiseSuccessProbability Q R compress decoder := by
letI mapProb (j : Fin M) (b : Bool) :
IsProbabilityMeasure (compressedCoordinateLaw (compress j) (Q j b)) :=
Measure.isProbabilityMeasure_map (hcompress j).aemeasurable
have hmain :
coordinatewiseSuccessProbability Q R compress decoder ≤
ofReal
((1 / 2 : ℝ) *
(1 + ∏ j, (1 - coordinateOverlap Q compress j))) := by
classical
let gamma : ∀ j, Measure (Z j × Z j) := fun j =>
compressionCoupling (Q j false) (Q j true) (compress j) (hcompress j)
let Gamma : Measure ((j : Fin M) → Z j × Z j) := Measure.pi gamma
let selected (omega : Fin M → Bool) :
((j : Fin M) → Z j × Z j) → ((j : Fin M) → Z j) :=
fun z => selectCoupledRaw omega z
have hcoord (omega : Fin M → Bool) (j : Fin M) :
MeasurePreserving
(fun p : Z j × Z j => if omega j then p.2 else p.1)
(gamma j) (Q j (omega j)) := by
cases hbit : omega j
· constructor
· simpa [hbit] using (measurable_fst : Measurable (Prod.fst : Z j × Z j → Z j))
· simpa [gamma, hbit] using
compressionCoupling_map_fst (Q j false) (Q j true)
(compress j) (hcompress j)
· constructor
· simpa [hbit] using (measurable_snd : Measurable (Prod.snd : Z j × Z j → Z j))
· simpa [gamma, hbit] using
compressionCoupling_map_snd (Q j false) (Q j true)
(compress j) (hcompress j)
have hselected (omega : Fin M → Bool) :
MeasurePreserving (selected omega) Gamma
(Measure.pi fun j => Q j (omega j)) := by
exact measurePreserving_pi gamma (fun j => Q j (omega j)) (hcoord omega)
have htargetMeas (omega : Fin M → Bool) : MeasurableSet
{data : A × ((j : Fin M) → Z j) | ∀ j,
decoder j (compress j (data.2 j)) data.2 data.1 = omega j} := by
rw [show {data : A × ((j : Fin M) → Z j) | ∀ j,
decoder j (compress j (data.2 j)) data.2 data.1 = omega j} =
⋂ j, {data | decoder j (compress j (data.2 j)) data.2 data.1 = omega j} by
ext data; simp]
apply MeasurableSet.iInter
intro j
apply measurableSet_eq_fun _ measurable_const
have hm := (hdecoder j).comp
(((hcompress j).comp ((measurable_pi_apply j).comp measurable_snd)).prodMk
(measurable_snd.prodMk measurable_fst))
simpa only [Function.comp_def] using hm
have hmeasure (omega : Fin M → Bool) :
(R.prod (Measure.pi fun j => Q j (omega j)))
{data | ∀ j,
decoder j (compress j (data.2 j)) data.2 data.1 = omega j} =
(R.prod Gamma)
{data | coupledDecoderGood compress decoder omega data.2 data.1} := by
let mp := (MeasurePreserving.id R).prod (hselected omega)
have happ := Measure.map_apply (μ := R.prod Gamma) mp.measurable (htargetMeas omega)
rw [mp.map_eq] at happ
rw [happ]
congr 1
have hsourceMeas (omega : Fin M → Bool) : MeasurableSet
{data : A × ((j : Fin M) → Z j × Z j) |
coupledDecoderGood compress decoder omega data.2 data.1} := by
exact (htargetMeas omega).preimage
(measurable_id.prodMap (hselected omega).measurable)
have hindMeas (omega : Fin M → Bool) : Measurable
(fun data : A × ((j : Fin M) → Z j × Z j) =>
coupledGoodIndicator compress decoder omega data.2 data.1) := by
let E : Set (A × ((j : Fin M) → Z j × Z j)) :=
{data | coupledDecoderGood compress decoder omega data.2 data.1}
rw [show (fun data : A × ((j : Fin M) → Z j × Z j) =>
coupledGoodIndicator compress decoder omega data.2 data.1) =
E.indicator (fun _ => (1 : ℝ≥0∞)) by
funext data
by_cases hgood : coupledDecoderGood compress decoder omega data.2 data.1 <;>
simp [coupledGoodIndicator, E, hgood]]
exact measurable_const.indicator (hsourceMeas omega)
have hindicator (omega : Fin M → Bool) :
∫⁻ data, coupledGoodIndicator compress decoder omega data.2 data.1
∂(R.prod Gamma) =
(R.prod Gamma)
{data | coupledDecoderGood compress decoder omega data.2 data.1} := by
let E : Set (A × ((j : Fin M) → Z j × Z j)) :=
{data | coupledDecoderGood compress decoder omega data.2 data.1}
have hE : MeasurableSet E := hsourceMeas omega
rw [← setLIntegral_one E]
rw [← lintegral_indicator hE]
apply lintegral_congr
intro data
by_cases hgood : coupledDecoderGood compress decoder omega data.2 data.1 <;>
simp [coupledGoodIndicator, E, hgood]
let N : Set ((j : Fin M) → Z j × Z j) :=
{z | ∀ j, compress j (z j).1 ≠ compress j (z j).2}
have hNj (j : Fin M) : MeasurableSet
{p : Z j × Z j | compress j p.1 ≠ compress j p.2} := by
letI : MeasurableEq (S j) := measurableEqOfStandardBorel (S j)
exact (measurableSet_eq_fun ((hcompress j).comp measurable_fst)
((hcompress j).comp measurable_snd)).compl
have hN : MeasurableSet N := by
rw [show N = Set.univ.pi fun j =>
{p : Z j × Z j | compress j p.1 ≠ compress j p.2} by
ext z; simp [N]]
exact MeasurableSet.univ_pi hNj
have hgammaN (j : Fin M) : gamma j
{p : Z j × Z j | compress j p.1 ≠ compress j p.2} ≤
ofReal (tvDist
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true))) := by
have heq := compressionCoupling_equal_compression_mass_ge
(Q j false) (Q j true) (compress j) (hcompress j)
have hEqMeas : MeasurableSet
{p : Z j × Z j | compress j p.1 = compress j p.2} := by
letI : MeasurableEq (S j) := measurableEqOfStandardBorel (S j)
exact measurableSet_eq_fun ((hcompress j).comp measurable_fst)
((hcompress j).comp measurable_snd)
have hcompl := tsub_le_tsub_left heq 1
rw [show {p : Z j × Z j | compress j p.1 ≠ compress j p.2} =
{p : Z j × Z j | compress j p.1 = compress j p.2}ᶜ by ext; simp,
measure_compl hEqMeas (measure_ne_top (gamma j) _), measure_univ]
calc
1 - gamma j {p : Z j × Z j | compress j p.1 = compress j p.2} ≤
1 - ofReal (1 - tvDist
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true))) := by
simpa [gamma, compressedCoordinateLaw] using hcompl
_ = ofReal (tvDist
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true))) := by
rw [← ENNReal.ofReal_one,
← ENNReal.ofReal_sub (1 : ℝ) (by
exact sub_nonneg.mpr tvDist_le_one)]
congr 1
ring
have hGammaN : Gamma N ≤ ofReal
(∏ j, tvDist
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true))) := by
change (Measure.pi gamma) N ≤ _
rw [show N = Set.univ.pi fun j =>
{p : Z j × Z j | compress j p.1 ≠ compress j p.2} by
ext z; simp [N], Measure.pi_pi]
calc
(∏ j, gamma j {p : Z j × Z j | compress j p.1 ≠ compress j p.2}) ≤
∏ j, ofReal (tvDist
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true))) :=
Finset.prod_le_prod (fun _ _ => bot_le) (fun j _ => hgammaN j)
_ = _ := by
rw [ENNReal.ofReal_prod_of_nonneg]
exact fun j _ => tvDist_nonneg
have hpoint (data : A × ((j : Fin M) → Z j × Z j)) :
(∑ omega : Fin M → Bool,
coupledGoodIndicator compress decoder omega data.2 data.1) ≤
(2 : ℝ≥0∞) ^ M / 2 * (1 + N.indicator (fun _ => (1 : ℝ≥0∞)) data.2) := by
by_cases hz : data.2 ∈ N
· have htriv : (∑ omega : Fin M → Bool,
coupledGoodIndicator compress decoder omega data.2 data.1) ≤
(2 : ℝ≥0∞) ^ M := by
calc
_ ≤ ∑ _omega : Fin M → Bool, (1 : ℝ≥0∞) :=
Finset.sum_le_sum fun omega _ => by
by_cases hg : coupledDecoderGood compress decoder omega data.2 data.1 <;>
simp [coupledGoodIndicator, hg]
_ = _ := by simp
have hcancel : (2 : ℝ≥0∞) ^ M / 2 * (1 + 1) = (2 : ℝ≥0∞) ^ M := by
rw [one_add_one_eq_two, ENNReal.div_mul_cancel (by norm_num) (by norm_num)]
simpa [Set.indicator_of_mem hz, hcancel] using htriv
· have hz' : ¬ ∀ j, compress j (data.2 j).1 ≠ compress j (data.2 j).2 := by
simpa [N] using hz
push_neg at hz'
obtain ⟨j, hj⟩ := hz'
have hhalf := coupledDecoderGood_count_le_half compress decoder hlocal
data.2 data.1 j hj
simpa [indicator, hz] using hhalf
have hsumMeasure :
(∑ omega : Fin M → Bool,
(R.prod (Measure.pi fun j => Q j (omega j)))
{data | ∀ j,
decoder j (compress j (data.2 j)) data.2 data.1 = omega j}) ≤
(2 : ℝ≥0∞) ^ M / 2 * (1 + Gamma N) := by
have hIndIntegral :
(∫⁻ data : A × ((j : Fin M) → Z j × Z j),
N.indicator (fun _ => (1 : ℝ≥0∞)) data.2 ∂(R.prod Gamma)) = Gamma N := by
rw [show (fun data : A × ((j : Fin M) → Z j × Z j) =>
N.indicator (fun _ => (1 : ℝ≥0∞)) data.2) =
(univ ×ˢ N).indicator (fun _ => (1 : ℝ≥0∞)) by
funext data
by_cases hz : data.2 ∈ N <;> simp [indicator, hz]]
rw [lintegral_indicator_const (MeasurableSet.univ.prod hN)]
rw [Measure.prod_apply (MeasurableSet.univ.prod hN)]
simp
simp_rw [hmeasure, ← hindicator]
rw [← lintegral_finset_sum univ (fun omega _ => hindMeas omega)]
calc
_ ≤ ∫⁻ data, (2 : ℝ≥0∞) ^ M / 2 *
(1 + N.indicator (fun _ => (1 : ℝ≥0∞)) data.2) ∂(R.prod Gamma) :=
lintegral_mono hpoint
_ = (2 : ℝ≥0∞) ^ M / 2 * (1 + Gamma N) := by
rw [lintegral_const_mul]
· rw [lintegral_add_left measurable_const]
rw [lintegral_const, measure_univ, hIndIntegral]
simp
· exact measurable_const.fun_add
((measurable_const.indicator hN).comp measurable_snd)
unfold coordinatewiseSuccessProbability
have hprodReal : 0 ≤ ∏ j, tvDist
(compressedCoordinateLaw (compress j) (Q j false))
(compressedCoordinateLaw (compress j) (Q j true)) :=
Finset.prod_nonneg fun j _ => tvDist_nonneg
calc
_ ≤ ((2 : ℝ≥0∞) ^ M / 2 * (1 + Gamma N)) / (2 : ℝ≥0∞) ^ M :=
ENNReal.div_le_div_right hsumMeasure _
-- … truncated; follow the source link for the rest …
12 supporting declarations (lemmas, instances)
-
compressedCoordinateLaw_klDiv_lelemma — Measurable finite-coordinate compression cannot increase KL divergence.Proof (Lean source)
-- @node: compressedCoordinateLaw_klDiv_le lemma compressedCoordinateLaw_klDiv_le {Z S : Type*} [MeasurableSpace Z] [MeasurableSpace S] (compress : Z → S) (hcompress : Measurable compress) (μ ν : Measure Z) [IsFiniteMeasure μ] [IsFiniteMeasure ν] : klDiv (compressedCoordinateLaw compress μ) (compressedCoordinateLaw compress ν) ≤ klDiv μ ν := by exact klDiv_map_le hcompress -
coupledDecoderGood_flip_exclusivelemma — If the two compressed versions agree at coordinate j, simultaneous correctness is impossible at both endpoints of the corresponding cube edge.hypothesesM :ℕFin M → Type*A :Type*compress :∀ jifZ jthenS jhlocal :∀ j s z z' aif(∀ k, k ≠ j → z k = z' k)thendecoder j s z a = decoder j s z' az :(j : Fin M) → Z j × Z ja :Aj :Fin Mheq :compress j (z j).1 = compress j (z j).2conclusion¬ (coupledDecoderGood compress decoder omega z a ∧ coupledDecoderGood compress decoder (flipBit j omega) z a)Proof (Lean source)
lemma coupledDecoderGood_flip_exclusive {M : ℕ} {Z S : Fin M → Type*} {A : Type*} (compress : ∀ j, Z j → S j) (decoder : ∀ j, S j → ((k : Fin M) → Z k) → A → Bool) (hlocal : ∀ j s z z' a, (∀ k, k ≠ j → z k = z' k) → decoder j s z a = decoder j s z' a) (omega : Fin M → Bool) (z : (j : Fin M) → Z j × Z j) (a : A) (j : Fin M) (heq : compress j (z j).1 = compress j (z j).2) : ¬ (coupledDecoderGood compress decoder omega z a ∧ coupledDecoderGood compress decoder (flipBit j omega) z a) := by rintro ⟨hgood, hflip⟩ have hother : ∀ k, k ≠ j → selectCoupledRaw omega z k = selectCoupledRaw (flipBit j omega) z k := by intro k hkj simp [selectCoupledRaw, flipBit, hkj] have hs : compress j (selectCoupledRaw omega z j) = compress j (selectCoupledRaw (flipBit j omega) z j) := by cases hbit : omega j <;> simp [selectCoupledRaw, flipBit, hbit, heq] have hout := hlocal j (compress j (selectCoupledRaw omega z j)) (selectCoupledRaw omega z) (selectCoupledRaw (flipBit j omega) z) a hother have h1 := hgood j have h2 := hflip j rw [← hs] at h2 rw [hout, h2, flipBit_self] at h1 cases hbit : omega j <;> simp [hbit] at h1 -
coupledDecoderGood_count_le_halflemma — Once one coupled coordinate has equal compressions, at most half of the hypercube vertices can be simultaneously decoded correctly.hypothesesM :ℕFin M → Type*A :Type*compress :∀ jifZ jthenS jhlocal :∀ j s z z' aif(∀ k, k ≠ j → z k = z' k)thendecoder j s z a = decoder j s z' az :(j : Fin M) → Z j × Z ja :Aj :Fin Mheq :compress j (z j).1 = compress j (z j).2conclusion≤ (2 : ℝ≥0∞) ^ M / 2Proof (Lean source)
lemma coupledDecoderGood_count_le_half {M : ℕ} {Z S : Fin M → Type*} {A : Type*} (compress : ∀ j, Z j → S j) (decoder : ∀ j, S j → ((k : Fin M) → Z k) → A → Bool) (hlocal : ∀ j s z z' a, (∀ k, k ≠ j → z k = z' k) → decoder j s z a = decoder j s z' a) (z : (j : Fin M) → Z j × Z j) (a : A) (j : Fin M) (heq : compress j (z j).1 = compress j (z j).2) : (∑ omega : Fin M → Bool, coupledGoodIndicator compress decoder omega z a) ≤ (2 : ℝ≥0∞) ^ M / 2 := by classical let I : (Fin M → Bool) → ℝ≥0∞ := fun omega => coupledGoodIndicator compress decoder omega z a have hpair : ∀ omega, I omega + I (flipBit j omega) ≤ 1 := by intro omega by_cases h1 : coupledDecoderGood compress decoder omega z a · have h2 : ¬ coupledDecoderGood compress decoder (flipBit j omega) z a := fun h => coupledDecoderGood_flip_exclusive compress decoder hlocal omega z a j heq ⟨h1, h⟩ simp [I, coupledGoodIndicator, h1, h2] · by_cases h2 : coupledDecoderGood compress decoder (flipBit j omega) z a <;> simp [I, coupledGoodIndicator, h1, h2] have hreindex : ∑ omega, I (flipBit j omega) = ∑ omega, I omega := Equiv.sum_comp (flipPerm j) I have hsum : 2 * ∑ omega, I omega ≤ ∑ _omega : Fin M → Bool, (1 : ℝ≥0∞) := by calc 2 * ∑ omega, I omega = (∑ omega, I omega) + ∑ omega, I (flipBit j omega) := by rw [hreindex]; ring _ = ∑ omega, (I omega + I (flipBit j omega)) := by rw [Finset.sum_add_distrib] _ ≤ ∑ _omega : Fin M → Bool, (1 : ℝ≥0∞) := Finset.sum_le_sum fun omega _ => hpair omega rw [show (∑ _omega : Fin M → Bool, (1 : ℝ≥0∞)) = (2 : ℝ≥0∞) ^ M by simp] at hsum rw [ENNReal.le_div_iff_mul_le (inr (by norm_num)) (inr (by norm_num))] simpa [mul_comm] using hsum -
half_integral_abs_rnDeriv_sub_le_tvDistlemma — For two probability measures dominated by a common finite measure, half the L¹ distance between their Radon--Nikodym densities is bounded by total variation. This is the reverse Scheffé inequality needed to construct the common submeasure in the maximal-coupling argument.hypothesesΩ :Type*μ ν ξ :hμξ :μ ≪ ξhνξ :ν ≪ ξProof (Lean source)
lemma half_integral_abs_rnDeriv_sub_le_tvDist {Ω : Type*} [MeasurableSpace Ω] (μ ν ξ : Measure Ω) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] [IsFiniteMeasure ξ] (hμξ : μ ≪ ξ) (hνξ : ν ≪ ξ) : (1 / 2 : ℝ) * ∫ x, |(μ.rnDeriv ξ x).toReal - (ν.rnDeriv ξ x).toReal| ∂ξ ≤ tvDist μ ν := by let p : Ω → ℝ := fun x => (μ.rnDeriv ξ x).toReal let q : Ω → ℝ := fun x => (ν.rnDeriv ξ x).toReal let d : Ω → ℝ := fun x => p x - q x let A : Set Ω := {x | 0 ≤ d x} have hp : Integrable p ξ := Measure.integrable_toReal_rnDeriv have hq : Integrable q ξ := Measure.integrable_toReal_rnDeriv have hd : Integrable d ξ := hp.sub hq have hA : MeasurableSet A := by dsimp [A, d, p, q] exact measurableSet_le measurable_const ((Measure.measurable_rnDeriv μ ξ).ennreal_toReal.sub (Measure.measurable_rnDeriv ν ξ).ennreal_toReal) have hd0 : ∫ x, d x ∂ξ = 0 := by rw [show d = fun x => p x - q x from rfl, integral_sub hp hq] rw [show p = fun x => (μ.rnDeriv ξ x).toReal from rfl, show q = fun x => (ν.rnDeriv ξ x).toReal from rfl, Measure.integral_toReal_rnDeriv hμξ, Measure.integral_toReal_rnDeriv hνξ] simp only [probReal_univ, sub_self] have hsplit : ∫ x in A, d x ∂ξ + ∫ x in Aᶜ, d x ∂ξ = 0 := by rw [MeasureTheory.integral_add_compl hA hd, hd0] have hpos : 0 ≤ ∫ x in A, d x ∂ξ := by apply integral_nonneg_of_ae filter_upwards [ae_restrict_mem hA] with x hx exact hx have habsA : ∫ x in A, |d x| ∂ξ = ∫ x in A, d x ∂ξ := by apply integral_congr_ae filter_upwards [ae_restrict_mem hA] with x hx rw [abs_of_nonneg] exact hx have habsAc : ∫ x in Aᶜ, |d x| ∂ξ = -(∫ x in Aᶜ, d x ∂ξ) := by rw [← integral_neg] apply integral_congr_ae filter_upwards [ae_restrict_mem hA.compl] with x hx change ¬ 0 ≤ d x at hx rw [not_le] at hx rw [abs_of_neg hx] have habs : ∫ x, |d x| ∂ξ = 2 * ∫ x in A, d x ∂ξ := by rw [← MeasureTheory.integral_add_compl hA hd.abs, habsA, habsAc] linarith have hgap : μ.real A - ν.real A = ∫ x in A, d x ∂ξ := by rw [show d = fun x => p x - q x from rfl, integral_sub hp.integrableOn hq.integrableOn, show p = fun x => (μ.rnDeriv ξ x).toReal from rfl, show q = fun x => (ν.rnDeriv ξ x).toReal from rfl, Measure.setIntegral_toReal_rnDeriv hμξ, Measure.setIntegral_toReal_rnDeriv hνξ] have htv := abs_measureReal_sub_le_tvDist (μ := μ) (ν := ν) hA rw [hgap, abs_of_nonneg hpos] at htv change (1 / 2 : ℝ) * ∫ x, |d x| ∂ξ ≤ _ rw [habs] linarith -
overlap_ge_exp_neg_klBudgetlemma — A nonnegative finite KL budget yields the corresponding Bretagnolle--Huber lower bound on testing overlap.hypothesesΩ :Type*μ ν :B :ℝhB :0 ≤ BProof (Lean source)
lemma overlap_ge_exp_neg_klBudget {Ω : Type*} [MeasurableSpace Ω] (μ ν : Measure Ω) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] {B : ℝ} (hB : 0 ≤ B) (hKL : klDiv μ ν ≤ ofReal B) : (1 / 2 : ℝ) * exp (-B) ≤ 1 - tvDist μ ν := by have hfin : klDiv μ ν ≠ ⊤ := ne_top_of_le_ne_top (ENNReal.ofReal_ne_top : ofReal B ≠ ⊤) hKL have hKLreal : (klDiv μ ν).toReal ≤ B := by rw [← ENNReal.toReal_ofReal hB] exact ENNReal.toReal_le_toReal hfin ENNReal.ofReal_ne_top |>.2 hKL calc (1 / 2 : ℝ) * exp (-B) ≤ (1 / 2 : ℝ) * exp (-(klDiv μ ν).toReal) := by gcongr _ ≤ 1 - tvDist μ ν := bretagnolle_huber_affinity μ ν (InformationTheory.klDiv_ne_top_iff.mp hfin).1 hfin -
prod_one_sub_le_exp_neg_sumlemma — A product of complementary overlap probabilities is bounded by the exponential of minus their sum.Proof (Lean source)
lemma prod_one_sub_le_exp_neg_sum {M : ℕ} (ρ : Fin M → ℝ) (hρ1 : ∀ j, ρ j ≤ 1) : (∏ j, (1 - ρ j)) ≤ exp (-∑ j, ρ j) := by calc (∏ j, (1 - ρ j)) ≤ ∏ j, exp (-ρ j) := by apply Finset.prod_le_prod · intro j _ linarith [hρ1 j] · intro j _ linarith [Real.add_one_le_exp (-ρ j)] _ = exp (-∑ j, ρ j) := by rw [← Real.exp_sum] congr 1 simp -
prod_one_sub_le_exp_neg_card_mullemma — A common coordinatewise overlap floor c bounds the complementary product by exp (-M * c).hypothesesconclusion(∏ j, (1 - ρ j)) ≤ exp (-(M : ℝ) * c)Proof (Lean source)
lemma prod_one_sub_le_exp_neg_card_mul {M : ℕ} (ρ : Fin M → ℝ) (c : ℝ) (hρ1 : ∀ j, ρ j ≤ 1) (hc : ∀ j, c ≤ ρ j) : (∏ j, (1 - ρ j)) ≤ exp (-(M : ℝ) * c) := by refine (prod_one_sub_le_exp_neg_sum ρ hρ1).trans ?_ rw [Real.exp_le_exp] have hsum : (M : ℝ) * c ≤ ∑ j, ρ j := by calc (M : ℝ) * c = ∑ _j : Fin M, c := by simp _ ≤ ∑ j, ρ j := Finset.sum_le_sum (fun j _ => hc j) linarith -
tvDist_eq_zero_of_klBudget_nonposlemma — A nonpositive real KL budget forces two probability measures to coincide, and hence forces their total variation distance to vanish.Proof (Lean source)
lemma tvDist_eq_zero_of_klBudget_nonpos {Ω : Type*} [MeasurableSpace Ω] (μ ν : Measure Ω) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] {B : ℝ} (hB : B ≤ 0) (hKL : klDiv μ ν ≤ ofReal B) : tvDist μ ν = 0 := by have hKL0 : klDiv μ ν = 0 := by apply le_antisymm · simpa [ENNReal.ofReal_eq_zero.mpr hB] using hKL · exact bot_le have hmeasure : μ = ν := InformationTheory.klDiv_eq_zero_iff.mp hKL0 subst ν simp [tvDist] -
card_mul_exp_neg_log_eq_rpowlemma — The exponential KL-overlap floor has the expected power-law scaling after multiplication by the number of coordinates.hypothesesM :ℕhM :1 ≤ Mκ :ℝProof (Lean source)
lemma card_mul_exp_neg_log_eq_rpow {M : ℕ} (hM : 1 ≤ M) (κ : ℝ) : (M : ℝ) * ((1 / 2 : ℝ) * exp (-(κ * log M))) = (M : ℝ) ^ (1 - κ) / 2 := by have hMr : (0 : ℝ) < M := by exact_mod_cast (lt_of_lt_of_le Nat.zero_lt_one hM) rw [Real.rpow_def_of_pos hMr] rw [show log (M : ℝ) * (1 - κ) = log (M : ℝ) + -(κ * log (M : ℝ)) by ring, Real.exp_add, Real.exp_log hMr] ring -
coordinateOverlap_product_le_of_nonnegative_kllemma — Under a nonnegative logarithmic KL budget, the product of coordinatewise total-variation factors has the finite-M exponential bound.hypothesesM :ℕhM :1 ≤ MZ S :∀ j b, IsProbabilityMeasure (Q j b)compress :∀ jifZ jthenS jhcompress :∀ j, Measurable (compress j)κ :ℝhκ0 :0 ≤ κhKL :∀ j,klDiv (compressedCoordinateLaw (compress j) (Q j false)) (compressedCoordinateLaw (compress j) (Q j true))conclusion(∏ j, (1 - coordinateOverlap Q compress j)) ≤ exp (-((M : ℝ) ^ (1 - κ)) / 2)Proof (Lean source)
lemma coordinateOverlap_product_le_of_nonnegative_kl {M : ℕ} (hM : 1 ≤ M) {Z S : Fin M → Type*} [∀ j, MeasurableSpace (Z j)] [∀ j, MeasurableSpace (S j)] (Q : ∀ j, Bool → Measure (Z j)) [∀ j b, IsProbabilityMeasure (Q j b)] (compress : ∀ j, Z j → S j) (hcompress : ∀ j, Measurable (compress j)) {κ : ℝ} (hκ0 : 0 ≤ κ) (hKL : ∀ j, klDiv (compressedCoordinateLaw (compress j) (Q j false)) (compressedCoordinateLaw (compress j) (Q j true)) ≤ ofReal (κ * log M)) : (∏ j, (1 - coordinateOverlap Q compress j)) ≤ exp (-((M : ℝ) ^ (1 - κ)) / 2) := by letI mapProb (j : Fin M) (b : Bool) : IsProbabilityMeasure (compressedCoordinateLaw (compress j) (Q j b)) := Measure.isProbabilityMeasure_map (hcompress j).aemeasurable have hlog : 0 ≤ log (M : ℝ) := Real.log_nonneg (by exact_mod_cast hM) have hover_le_one : ∀ j, coordinateOverlap Q compress j ≤ 1 := by intro j unfold coordinateOverlap linarith [tvDist_nonneg (μ := compressedCoordinateLaw (compress j) (Q j false)) (ν := compressedCoordinateLaw (compress j) (Q j true))] have hover_floor : ∀ j, (1 / 2 : ℝ) * exp (-(κ * log M)) ≤ coordinateOverlap Q compress j := by intro j exact overlap_ge_exp_neg_klBudget _ _ (mul_nonneg hκ0 hlog) (hKL j) have hprod := prod_one_sub_le_exp_neg_card_mul (fun j => coordinateOverlap Q compress j) ((1 / 2 : ℝ) * exp (-(κ * log M))) hover_le_one hover_floor rw [show -(M : ℝ) * ((1 / 2 : ℝ) * exp (-(κ * log M))) = -((M : ℝ) * ((1 / 2 : ℝ) * exp (-(κ * log M)))) by ring, card_mul_exp_neg_log_eq_rpow hM κ] at hprod simpa only [neg_div] using hprod -
ennreal_error_lower_bound_of_success_upper_boundlemma — A real-valued product bound below one converts an ENNReal simultaneous success upper bound into the complementary error lower bound.hypothesesconclusionofReal ((1 / 2 : ℝ) * (1 - e)) ≤ 1 - sProof (Lean source)
lemma ennreal_error_lower_bound_of_success_upper_bound {s : ℝ≥0∞} {p e : ℝ} (hp0 : 0 ≤ p) (he1 : e ≤ 1) (hpe : p ≤ e) (hs : s ≤ ofReal ((1 / 2 : ℝ) * (1 + p))) : ofReal ((1 / 2 : ℝ) * (1 - e)) ≤ 1 - s := by have ht : 0 ≤ (1 / 2 : ℝ) * (1 - e) := mul_nonneg (by norm_num) (sub_nonneg.mpr he1) have hu : 0 ≤ (1 / 2 : ℝ) * (1 + p) := mul_nonneg (by norm_num) (by linarith) apply ENNReal.le_sub_of_add_le_right (ne_top_of_le_ne_top ENNReal.ofReal_ne_top hs) calc ofReal ((1 / 2 : ℝ) * (1 - e)) + s ≤ ofReal ((1 / 2 : ℝ) * (1 - e)) + ofReal ((1 / 2 : ℝ) * (1 + p)) := add_le_add_right hs _ _ = ofReal ((1 / 2 : ℝ) * (1 - e) + (1 / 2 : ℝ) * (1 + p)) := (ENNReal.ofReal_add ht hu).symm _ ≤ ofReal 1 := ENNReal.ofReal_le_ofReal (by linarith) _ = 1 := ENNReal.ofReal_one -
coordinatewise_overlap_direct_product_asymptoticlemma — Along any sequence M_n → ∞ and for fixed κ < 1, the finite direct-product certificate tends to one half.hypothesesProof (Lean source)
lemma coordinatewise_overlap_direct_product_asymptotic (Mseq : ℕ → ℕ) (κ : ℝ) (hκ : κ < 1) (hMseq : Tendsto Mseq atTop atTop) : Tendsto (fun n => (1 / 2 : ℝ) * (1 - exp (-(((Mseq n : ℕ) : ℝ) ^ (1 - κ)) / 2))) atTop (nhds (1 / 2 : ℝ)) := by have hpow : Tendsto (fun n => ((Mseq n : ℝ) ^ (1 - κ))) atTop atTop := (tendsto_rpow_atTop (sub_pos.mpr hκ)).comp (tendsto_natCast_atTop_atTop.comp hMseq) have hneg : Tendsto (fun n => -((Mseq n : ℝ) ^ (1 - κ)) / 2) atTop atBot := by exact (tendsto_neg_atTop_atBot.comp hpow).atBot_div_const (by norm_num) have hexp : Tendsto (fun n => exp (-((Mseq n : ℝ) ^ (1 - κ)) / 2)) atTop (nhds 0) := Real.tendsto_exp_atBot.comp hneg convert (tendsto_const_nhds.mul (tendsto_const_nhds.sub hexp)) using 1 all_goals norm_num
Fano 3 core · 6 supporting This file proves a total-variation form of the multiple-hypothesis minimax lower bound. ★ fano_average_error★ fano_exists_error
Fano-Type Multiple-Hypothesis Bound
This file proves a total-variation form of the multiple-hypothesis minimax lower bound. It converts pairwise-separated parameter values and a common reference measure into an average error lower bound for any estimator.
def acceptanceRegion reviewed
⚑ flagged
Causalean.Stat
For a sample space, a pseudo-metric parameter space, an estimator from the sample space to the parameter space, a parameter value, and a real radius, the acceptance region is the set of sample points at which the estimator lies at distance strictly less than that radius from the parameter value.
theorem fano_average_error reviewed
⚑ flagged
Causalean.Stat
Fano average-error lower bound. For a measurable estimator est and a family of parameter values that are pairwise 2s-separated, the average probability of error over the N = card ι hypotheses is at least 1 − (1 + ∑ᵢ tvDist (P i₀) (P i)) / N.
Formal statement
hypotheses
Θ :
Type u_2
ι :
Type u_3
est :
Ω → Θ
hest :
Measurable est
θ :
ι → Θ
s :
ℝ
hsep :
∀ i k
if
i ≠ k
then
2 * s ≤ dist (θ i) (θ k)
i₀ :
ι
Proof (Lean source)
theorem fano_average_error {est : Ω → Θ} (hest : Measurable est) {θ : ι → Θ} {s : ℝ}
(hsep : ∀ i k, i ≠ k → 2 * s ≤ dist (θ i) (θ k)) (i₀ : ι) :
1 - (1 + ∑ i, tvDist (P i₀) (P i)) / (Fintype.card ι)
≤ (∑ i, (P i).real {ω | s ≤ dist (est ω) (θ i)}) / (Fintype.card ι) := by
set N : ℝ := (Fintype.card ι : ℝ) with hN
have hNpos : 0 < N := by
rw [hN, Nat.cast_pos]
exact Fintype.card_pos_iff.mpr ⟨i₀⟩
have hNne : N ≠ 0 := ne_of_gt hNpos
have herr : ∀ i, (P i).real {ω | s ≤ dist (est ω) (θ i)}
= 1 - (P i).real (acceptanceRegion est (θ i) s) := by
intro i
rw [← acceptanceRegion_compl est (θ i) s,
measureReal_compl (measurableSet_acceptanceRegion hest (θ i) s)]
simp [probReal_univ]
have hsumerr : ∑ i, (P i).real {ω | s ≤ dist (est ω) (θ i)}
= N - ∑ i, (P i).real (acceptanceRegion est (θ i) s) := by
simp_rw [herr]
rw [Finset.sum_sub_distrib]
congr 1
rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, mul_one, hN]
have hcorrect := sum_correct_le P hest hsep i₀
rw [hsumerr, le_div_iff₀ hNpos, sub_mul, div_mul_cancel₀ _ hNne, one_mul]
linarith [hcorrect]
theorem fano_exists_error reviewed
⚑ flagged
Causalean.Stat
Fano existence-of-bad-hypothesis bound (uniform β). For a measurable estimator est and a family of parameter values that are pairwise 2s-separated, if every hypothesis's law is within total variation β of the reference P i₀, then some hypothesis has error probability at least 1 − 1/N − β. This is the directly usable minimax statement: choosing the number of hypotheses N large and the divergence β small forces error.
Formal statement
hypotheses
Θ :
Type u_2
ι :
Type u_3
est :
Ω → Θ
hest :
Measurable est
θ :
ι → Θ
s :
ℝ
hsep :
∀ i k
if
i ≠ k
then
2 * s ≤ dist (θ i) (θ k)
i₀ :
ι
β :
ℝ
hβ :
∀ i, tvDist (P i₀) (P i) ≤ β
Proof (Lean source)
theorem fano_exists_error {est : Ω → Θ} (hest : Measurable est) {θ : ι → Θ} {s : ℝ}
(hsep : ∀ i k, i ≠ k → 2 * s ≤ dist (θ i) (θ k)) (i₀ : ι) {β : ℝ}
(hβ : ∀ i, tvDist (P i₀) (P i) ≤ β) :
∃ i, 1 - 1 / (Fintype.card ι) - β
≤ (P i).real {ω | s ≤ dist (est ω) (θ i)} := by
set N : ℝ := (Fintype.card ι : ℝ) with hN
have hNpos : 0 < N := by
rw [hN, Nat.cast_pos]; exact Fintype.card_pos_iff.mpr ⟨i₀⟩
have havg := fano_average_error P hest hsep i₀
rw [← hN] at havg
have hsumtv : ∑ i, tvDist (P i₀) (P i) ≤ N * β := by
calc ∑ i, tvDist (P i₀) (P i) ≤ ∑ _i : ι, β := Finset.sum_le_sum (fun i _ => hβ i)
_ = N * β := by rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, hN]
have hstep : (1 + ∑ i, tvDist (P i₀) (P i)) / N ≤ 1 / N + β := by
rw [add_div]
have hle : (∑ i, tvDist (P i₀) (P i)) / N ≤ β := by
rw [div_le_iff₀ hNpos]; linarith [hsumtv]
linarith
have hbound : 1 - 1 / N - β
≤ (∑ i, (P i).real {ω | s ≤ dist (est ω) (θ i)}) / N := by
refine le_trans ?_ havg
linarith [hstep]
by_contra hcon
push_neg at hcon
have hstrict : ∑ i, (P i).real {ω | s ≤ dist (est ω) (θ i)} < N * (1 - 1 / N - β) := by
calc ∑ i, (P i).real {ω | s ≤ dist (est ω) (θ i)}
< ∑ _i : ι, (1 - 1 / N - β) :=
Finset.sum_lt_sum_of_nonempty (Finset.univ_nonempty_iff.mpr ⟨i₀⟩)
(fun i _ => hcon i)
_ = N * (1 - 1 / N - β) := by
rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, hN]
rw [le_div_iff₀ hNpos] at hbound
nlinarith [hbound, hstrict]
6 supporting declarations (lemmas, instances)
-
measurableSet_acceptanceRegiontheorem — A measurable estimator has a measurable acceptance region around any target.hypothesesconclusionMeasurableSet (acceptanceRegion est θ s)Proof (Lean source)
theorem measurableSet_acceptanceRegion {est : Ω → Θ} (hest : Measurable est) (θ : Θ) (s : ℝ) : MeasurableSet (acceptanceRegion est θ s) := by have h : acceptanceRegion est θ s = {ω | s ≤ dist (est ω) θ}ᶜ := by ext ω; simp [acceptanceRegion, not_le] rw [h] exact (measurableSet_error hest θ s).compl -
acceptanceRegion_compltheorem — The acceptance region is the complement of the error region.hypothesesΩ :sharedType u_1Θ :sharedType u_2est :Ω → Θθ :Θs :ℝconclusion(acceptanceRegion est θ s)ᶜ = {ω | s ≤ dist (est ω) θ}Proof (Lean source)
theorem acceptanceRegion_compl (est : Ω → Θ) (θ : Θ) (s : ℝ) : (acceptanceRegion est θ s)ᶜ = {ω | s ≤ dist (est ω) θ} := by ext ω; simp [acceptanceRegion, not_lt] -
acceptanceRegion_disjointtheorem — Disjointness of acceptance regions. If two target values are 2s-separated, their acceptance regions are disjoint: a point within s of both would force the targets within 2s of each other.hypothesesconclusionProof (Lean source)
theorem acceptanceRegion_disjoint {est : Ω → Θ} {θ₀ θ₁ : Θ} {s : ℝ} (hsep : 2 * s ≤ dist θ₀ θ₁) : Disjoint (acceptanceRegion est θ₀ s) (acceptanceRegion est θ₁ s) := by rw [Set.disjoint_left] intro ω h0 h1 have h0' : dist (est ω) θ₀ < s := h0 have h1' : dist (est ω) θ₁ < s := h1 have htri : dist θ₀ θ₁ ≤ dist (est ω) θ₀ + dist (est ω) θ₁ := by simpa [dist_comm] using dist_triangle θ₀ (est ω) θ₁ linarith -
acceptanceRegion_pairwiseDisjointtheorem — Pairwise disjointness of the acceptance regions of a 2s-separated family.hypothesesΩ :sharedType u_1Θ :sharedType u_2ι :sharedType u_3est :Ω → Θθ :ι → Θs :ℝhsep :∀ i kifi ≠ kthen2 * s ≤ dist (θ i) (θ k)conclusionProof (Lean source)
theorem acceptanceRegion_pairwiseDisjoint {est : Ω → Θ} {θ : ι → Θ} {s : ℝ} (hsep : ∀ i k, i ≠ k → 2 * s ≤ dist (θ i) (θ k)) : Pairwise (onFun Disjoint (fun i => acceptanceRegion est (θ i) s)) := fun i k hik => acceptanceRegion_disjoint (hsep i k hik) -
sum_refReal_acceptance_letheorem — A reference hypothesis assigns total mass ≤ 1 across the disjoint acceptance regions, since their union has measure ≤ 1.hypothesesΘ :sharedType u_2ι :sharedType u_3est :Ω → Θhest :Measurable estθ :ι → Θs :ℝhsep :∀ i kifi ≠ kthen2 * s ≤ dist (θ i) (θ k)i₀ :ιconclusion∑ i, (P i₀).real (acceptanceRegion est (θ i) s) ≤ 1Proof (Lean source)
theorem sum_refReal_acceptance_le {est : Ω → Θ} (hest : Measurable est) {θ : ι → Θ} {s : ℝ} (hsep : ∀ i k, i ≠ k → 2 * s ≤ dist (θ i) (θ k)) (i₀ : ι) : ∑ i, (P i₀).real (acceptanceRegion est (θ i) s) ≤ 1 := by have hmeas : ∀ i, MeasurableSet (acceptanceRegion est (θ i) s) := fun i => measurableSet_acceptanceRegion hest (θ i) s have hdisj : Pairwise (onFun Disjoint (fun i => acceptanceRegion est (θ i) s)) := acceptanceRegion_pairwiseDisjoint (est := est) (θ := θ) (s := s) hsep have hunion : (P i₀) (⋃ i, acceptanceRegion est (θ i) s) = ∑ i, (P i₀) (acceptanceRegion est (θ i) s) := by rw [measure_iUnion hdisj hmeas, tsum_fintype] have hle : (P i₀) (⋃ i, acceptanceRegion est (θ i) s) ≤ 1 := by calc (P i₀) (⋃ i, acceptanceRegion est (θ i) s) ≤ (P i₀) univ := measure_mono (Set.subset_univ _) _ = 1 := measure_univ have hfin : ∀ i, (P i₀) (acceptanceRegion est (θ i) s) ≠ ⊤ := fun i => measure_ne_top _ _ have hsumeq : ∑ i, (P i₀).real (acceptanceRegion est (θ i) s) = ((P i₀) (⋃ i, acceptanceRegion est (θ i) s)).toReal := by rw [hunion, ENNReal.toReal_sum (fun i _ => hfin i)]; rfl rw [hsumeq] calc ((P i₀) (⋃ i, acceptanceRegion est (θ i) s)).toReal ≤ (1 : ENNReal).toReal := ENNReal.toReal_mono ENNReal.one_ne_top hle _ = 1 := ENNReal.toReal_one -
sum_correct_letheorem — Heart of Fano. The total probability of correct recovery, summed over the family, is at most 1 + ∑ᵢ tvDist (P i₀) (P i): comparing each Pᵢ(A i) to the reference P i₀(A i) costs one tvDist, and the reference masses sum to ≤ 1.hypothesesΘ :sharedType u_2ι :sharedType u_3est :Ω → Θhest :Measurable estθ :ι → Θs :ℝhsep :∀ i kifi ≠ kthen2 * s ≤ dist (θ i) (θ k)i₀ :ιconclusionProof (Lean source)
theorem sum_correct_le {est : Ω → Θ} (hest : Measurable est) {θ : ι → Θ} {s : ℝ} (hsep : ∀ i k, i ≠ k → 2 * s ≤ dist (θ i) (θ k)) (i₀ : ι) : ∑ i, (P i).real (acceptanceRegion est (θ i) s) ≤ 1 + ∑ i, tvDist (P i₀) (P i) := by have hterm : ∀ i, (P i).real (acceptanceRegion est (θ i) s) ≤ (P i₀).real (acceptanceRegion est (θ i) s) + tvDist (P i₀) (P i) := by intro i have hmeas : MeasurableSet (acceptanceRegion est (θ i) s) := measurableSet_acceptanceRegion hest (θ i) s have h := measureReal_sub_le_tvDist (μ := P i₀) (ν := P i) hmeas linarith calc ∑ i, (P i).real (acceptanceRegion est (θ i) s) ≤ ∑ i, ((P i₀).real (acceptanceRegion est (θ i) s) + tvDist (P i₀) (P i)) := Finset.sum_le_sum (fun i _ => hterm i) _ = (∑ i, (P i₀).real (acceptanceRegion est (θ i) s)) + ∑ i, tvDist (P i₀) (P i) := by rw [Finset.sum_add_distrib] _ ≤ 1 + ∑ i, tvDist (P i₀) (P i) := by have := sum_refReal_acceptance_le P hest hsep i₀ linarith
FinitePosteriorBayesRisk 11 core · 24 supporting This module conditions a finite prior through an arbitrary finite Markov observation kernel. ★ squaredRisk_eq_posteriorResidual_add★ finiteDesignBayesRisk_statewiseSquaredLoss★ realBayesRisk_statewiseSquaredLoss_eq_posteriorResidual
Finite posterior means and continuous-mixture Bayes risk
This module conditions a finite prior through an arbitrary finite Markov observation kernel. It provides zero-guarded posterior weights, exact finite disintegration and squared-loss completion, and identifies the resulting real-valued Bayes-risk infimum.
It also transports the construction through a continuous prior mixed into the finite latent space, reusing the induced finite design to transfer estimator-wise integral lower bounds to finite and continuous-mixture Bayes risks.
def kernelMass reviewed
⚑ flagged
Causalean.Stat
Given a measurable latent-state space, a measurable observation space, a Markov observation kernel from latent states to observations, a latent state, and an observation, the real-valued singleton observation mass is the kernel's mass at that observation conditional on that latent state.
def jointMass reviewed
⚑ flagged
Causalean.Stat
Given a finite measurable latent-state space, a measurable observation space, a finite probability design on the latent states, a Markov observation kernel from latent states to observations, a latent state, and an observation, the joint mass of that state and observation is the prior mass of the state times the conditional singleton mass of the observation.
Definition (Lean source)
def observationMass reviewed
⚑ flagged
Causalean.Stat
Given a finite measurable latent-state space, a measurable observation space, a finite probability design on the latent states, a Markov observation kernel from latent states to observations, and an observation, the observation marginal mass is the sum, over all latent states, of their joint masses with that observation.
Definition (Lean source)
parameters
def
observationMass ν L x :
ℝ
given by
∑ s, jointMass ν L s x
def posteriorWeight reviewed
⚑ flagged
Causalean.Stat
Given a finite measurable latent-state space, a measurable observation space, a finite probability design on the latent states, a Markov observation kernel from latent states to observations, an observation, and a latent state, the guarded posterior weight is zero when the observation has zero marginal mass and is otherwise the joint mass of the state and observation divided by that marginal mass.
Definition (Lean source)
parameters
def
posteriorWeight ν L x s :
ℝ
given by
if observationMass ν L x = 0 then 0 else jointMass ν L s x / observationMass ν L x
def posteriorMean reviewed
⚑ flagged
Causalean.Stat
Given a finite measurable latent-state space, a measurable observation space, a finite probability design on the latent states, a Markov observation kernel from latent states to observations, an observation-dependent real target, and an observation, the guarded posterior mean is the sum over latent states of the guarded posterior weight times that target at the state and observation.
Definition (Lean source)
parameters
def
posteriorMean ν L t x :
ℝ
given by
∑ s, posteriorWeight ν L x s * t s x
def squaredRisk reviewed
⚑ flagged
Causalean.Stat
Given a finite measurable latent-state space, a finite measurable observation space, a finite probability design on the latent states, a Markov observation kernel from latent states to observations, an observation-dependent real target, and a real-valued estimator based on the observation, the squared risk is the sum over all latent states and observations of their joint mass times the estimator's squared error relative to the target.
Definition (Lean source)
parameters
def
squaredRisk ν L t T :
ℝ
given by
∑ s, ∑ x, jointMass ν L s x * (T x - t s x) ^ 2
def posteriorResidual reviewed
⚑ flagged
Causalean.Stat
Given a finite measurable latent-state space, a finite measurable observation space, a finite probability design on the latent states, a Markov observation kernel from latent states to observations, and an observation-dependent real target, the posterior residual risk is the sum over observations of their marginal mass times the posterior-weighted squared deviation of the target from its guarded posterior mean.
Definition (Lean source)
parameters
def
posteriorResidual ν L t :
ℝ
given by
∑ x, observationMass ν L x * (∑ s, posteriorWeight ν L x s * (t s x - posteriorMean ν L t x) ^ 2)
theorem squaredRisk_eq_posteriorResidual_add reviewed
⚑ flagged
Causalean.Stat
Every estimator's squared risk equals posterior residual risk plus its marginally weighted squared distance from the posterior mean.
Formal statement
hypotheses
conclusion
squaredRisk ν L t T
= posteriorResidual ν L t + ∑ x, observationMass ν L x * (T x - posteriorMean ν L t x) ^ 2
Proof (Lean source)
theorem squaredRisk_eq_posteriorResidual_add (ν : FiniteDesign S) (L : Kernel S X)
(t : S → X → ℝ) (T : X → ℝ) :
squaredRisk ν L t T = posteriorResidual ν L t +
∑ x, observationMass ν L x * (T x - posteriorMean ν L t x) ^ 2 := by
-- Disintegrate the risk. On positive fibers, expand the square around the posterior mean and
-- kill the cross term with `posterior_centered_sum_of_pos`; null fibers vanish by their mass.
classical
rw [squaredRisk, disintegrate_sum ν L]
unfold posteriorResidual
rw [← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro x _
by_cases hx0 : observationMass ν L x = 0
· simp [hx0]
· have hx : 0 < observationMass ν L x :=
lt_of_le_of_ne (observationMass_nonneg ν L x) (Ne.symm hx0)
rw [← mul_add]
congr 1
calc
(∑ s, posteriorWeight ν L x s * (T x - t s x) ^ 2) =
∑ s, (posteriorWeight ν L x s *
(t s x - posteriorMean ν L t x) ^ 2 +
posteriorWeight ν L x s *
(T x - posteriorMean ν L t x) ^ 2 -
(2 * (T x - posteriorMean ν L t x)) *
(posteriorWeight ν L x s *
(t s x - posteriorMean ν L t x))) := by
apply Finset.sum_congr rfl
intro s _
ring
_ = (∑ s, posteriorWeight ν L x s *
(t s x - posteriorMean ν L t x) ^ 2) +
(∑ s, posteriorWeight ν L x s) *
(T x - posteriorMean ν L t x) ^ 2 -
(2 * (T x - posteriorMean ν L t x)) *
(∑ s, posteriorWeight ν L x s *
(t s x - posteriorMean ν L t x)) := by
rw [Finset.sum_sub_distrib, Finset.sum_add_distrib,
Finset.sum_mul, ← Finset.mul_sum]
_ = (∑ s, posteriorWeight ν L x s *
(t s x - posteriorMean ν L t x) ^ 2) +
(T x - posteriorMean ν L t x) ^ 2 := by
rw [posteriorWeight_sum_of_pos ν L hx,
posterior_centered_sum_of_pos ν L t hx]
ring
def statewiseSquaredLoss reviewed
⚑ flagged
Causalean.Stat
Given a measurable latent-state space, a finite measurable observation space, a Markov observation kernel from latent states to observations, an observation-dependent real target, a real-valued estimator based on the observation, and a latent state, the statewise squared loss is the sum over observations of their conditional singleton masses times the estimator's squared error relative to the target.
Definition (Lean source)
theorem finiteDesignBayesRisk_statewiseSquaredLoss reviewed
⚑ flagged
Causalean.Stat
The real-valued finite-design Bayes-risk infimum for observation-dependent squared loss is exactly the posterior residual risk.
Formal statement
hypotheses
conclusion
= posteriorResidual ν L t
Proof (Lean source)
theorem finiteDesignBayesRisk_statewiseSquaredLoss (ν : FiniteDesign S) (L : Kernel S X)
(t : S → X → ℝ) :
finiteDesignBayesRisk ν (statewiseSquaredLoss L t) =
posteriorResidual ν L t := by
-- Unfold the generic Bayes risk, rewrite each design expectation as `squaredRisk`, and apply
-- `iInf_squaredRisk_eq_posteriorResidual`.
simp only [finiteDesignBayesRisk,
finiteDesign_expectedLoss_statewiseSquaredLoss]
exact iInf_squaredRisk_eq_posteriorResidual ν L t
theorem realBayesRisk_statewiseSquaredLoss_eq_posteriorResidual reviewed
⚑ flagged
Causalean.Stat
The real Bayes risk of continuously mixed finite-state squared loss equals the posterior residual risk computed from the induced finite design.
Formal statement
hypotheses
Θ :
Type u_3
S :
Type u_4
X :
Type u_5
π :
K :
Kernel Θ S
L :
Kernel S X
t :
S → X → ℝ
conclusion
realBayesRisk π K (statewiseSquaredLoss L t)
= posteriorResidual (inducedFiniteDesign π K) L t
Proof (Lean source)
theorem realBayesRisk_statewiseSquaredLoss_eq_posteriorResidual
(π : Measure Θ) [IsProbabilityMeasure π]
(K : Kernel Θ S) [IsMarkovKernel K] (L : Kernel S X)
(t : S → X → ℝ) :
realBayesRisk π K (statewiseSquaredLoss L t) =
posteriorResidual (inducedFiniteDesign π K) L t := by
-- Chain `realBayesRisk_eq_inducedFiniteDesignBayesRisk` with the finite posterior theorem.
rw [realBayesRisk_eq_inducedFiniteDesignBayesRisk]
exact finiteDesignBayesRisk_statewiseSquaredLoss _ L t
24 supporting declarations (lemmas, instances)
-
kernelMass_nonnegtheorem — Every singleton observation mass emitted by a Markov kernel is nonnegative.hypothesesconclusion0 ≤ kernelMass L s xProof (Lean source)
theorem kernelMass_nonneg (L : Kernel S X) (s : S) (x : X) : 0 ≤ kernelMass L s x := by exact measureReal_nonneg -
kernelMass_sumtheorem — The singleton observation masses emitted in each latent state sum to one.hypothesesconclusion∑ x, kernelMass L s x = 1Proof (Lean source)
theorem kernelMass_sum (L : Kernel S X) [IsMarkovKernel L] (s : S) : ∑ x, kernelMass L s x = 1 := by -- Rewrite the finite singleton sum as the real mass of `univ`, then use Markov normalization. simpa [kernelMass] using (sum_measureReal_singleton (μ := L s) (Finset.univ : Finset X)) -
jointMass_nonnegtheorem — Every latent-state--observation joint mass is nonnegative.hypothesesconclusion0 ≤ jointMass ν L s xProof (Lean source)
theorem jointMass_nonneg (ν : FiniteDesign S) (L : Kernel S X) (s : S) (x : X) : 0 ≤ jointMass ν L s x := by exact mul_nonneg (ν.p_nonneg s) (kernelMass_nonneg L s x) -
jointMass_sumtheorem — The joint masses of a normalized finite prior and Markov observation kernel sum to one.hypothesesconclusion∑ s, ∑ x, jointMass ν L s x = 1Proof (Lean source)
theorem jointMass_sum (ν : FiniteDesign S) (L : Kernel S X) [IsMarkovKernel L] : ∑ s, ∑ x, jointMass ν L s x = 1 := by classical simp_rw [jointMass, ← Finset.mul_sum, kernelMass_sum, mul_one] exact ν.p_sum -
observationMass_nonnegtheorem — Every observation marginal mass is nonnegative.hypothesesconclusion0 ≤ observationMass ν L xProof (Lean source)
theorem observationMass_nonneg (ν : FiniteDesign S) (L : Kernel S X) (x : X) : 0 ≤ observationMass ν L x := by exact sum_nonneg fun s _ => jointMass_nonneg ν L s x -
observationMass_sumtheorem — The observation marginal masses sum to one.hypothesesconclusion∑ x, observationMass ν L x = 1Proof (Lean source)
theorem observationMass_sum (ν : FiniteDesign S) (L : Kernel S X) [IsMarkovKernel L] : ∑ x, observationMass ν L x = 1 := by classical simp_rw [observationMass] rw [Finset.sum_comm] exact jointMass_sum ν L -
posteriorWeight_of_ne_zerotheorem — On a nonzero observation fiber, the guarded posterior weight is the usual Bayes ratio.hypothesesS :sharedType u_1X :sharedType u_2ν :L :Kernel S Xx :Xhx :observationMass ν L x ≠ 0s :SconclusionposteriorWeight ν L x s = jointMass ν L s x / observationMass ν L xProof (Lean source)
theorem posteriorWeight_of_ne_zero (ν : FiniteDesign S) (L : Kernel S X) {x : X} (hx : observationMass ν L x ≠ 0) (s : S) : posteriorWeight ν L x s = jointMass ν L s x / observationMass ν L x := by simp [posteriorWeight, hx] -
posteriorWeight_of_eq_zerotheorem — On a null observation fiber, every guarded posterior weight is zero.hypothesesS :sharedType u_1X :sharedType u_2ν :L :Kernel S Xx :Xhx :observationMass ν L x = 0s :SconclusionposteriorWeight ν L x s = 0Proof (Lean source)
theorem posteriorWeight_of_eq_zero (ν : FiniteDesign S) (L : Kernel S X) {x : X} (hx : observationMass ν L x = 0) (s : S) : posteriorWeight ν L x s = 0 := by simp [posteriorWeight, hx] -
posteriorWeight_nonnegtheorem — Every guarded posterior weight is nonnegative.hypothesesconclusion0 ≤ posteriorWeight ν L x sProof (Lean source)
theorem posteriorWeight_nonneg (ν : FiniteDesign S) (L : Kernel S X) (x : X) (s : S) : 0 ≤ posteriorWeight ν L x s := by classical by_cases hx : observationMass ν L x = 0 · rw [posteriorWeight_of_eq_zero ν L hx] · rw [posteriorWeight_of_ne_zero ν L hx] exact div_nonneg (jointMass_nonneg ν L s x) (observationMass_nonneg ν L x) -
posteriorWeight_sum_of_postheorem — On a positive-mass observation fiber, the guarded posterior weights sum to one.hypothesesconclusion∑ s, posteriorWeight ν L x s = 1Proof (Lean source)
theorem posteriorWeight_sum_of_pos (ν : FiniteDesign S) (L : Kernel S X) {x : X} (hx : 0 < observationMass ν L x) : ∑ s, posteriorWeight ν L x s = 1 := by classical simp_rw [posteriorWeight_of_ne_zero ν L (ne_of_gt hx), div_eq_mul_inv] rw [← Finset.sum_mul] change observationMass ν L x * (observationMass ν L x)⁻¹ = 1 exact mul_inv_cancel₀ (ne_of_gt hx) -
posteriorWeight_sum_of_eq_zerotheorem — On a null observation fiber, the guarded posterior weights sum to zero.hypothesesconclusion∑ s, posteriorWeight ν L x s = 0Proof (Lean source)
theorem posteriorWeight_sum_of_eq_zero (ν : FiniteDesign S) (L : Kernel S X) {x : X} (hx : observationMass ν L x = 0) : ∑ s, posteriorWeight ν L x s = 0 := by simp [posteriorWeight_of_eq_zero ν L hx] -
observationMass_mul_posteriorWeighttheorem — Multiplying an observation marginal by its guarded posterior weight recovers the joint atom, including on null fibers.hypothesesconclusionobservationMass ν L x * posteriorWeight ν L x s = jointMass ν L s xProof (Lean source)
theorem observationMass_mul_posteriorWeight (ν : FiniteDesign S) (L : Kernel S X) (x : X) (s : S) : observationMass ν L x * posteriorWeight ν L x s = jointMass ν L s x := by -- Split on the marginal. The nonzero branch cancels; in the zero branch, nonnegativity and -- `sum_eq_zero_iff_of_nonneg` force every joint atom in the defining marginal sum to vanish. classical by_cases hx : observationMass ν L x = 0 · rw [hx, zero_mul] have hsum : (∑ s', jointMass ν L s' x) = 0 := by simpa [observationMass] using hx have hterm := (Finset.sum_eq_zero_iff_of_nonneg (fun s' _ => jointMass_nonneg ν L s' x)).mp hsum s (Finset.mem_univ s) exact hterm.symm · rw [posteriorWeight_of_ne_zero ν L hx] exact mul_div_cancel₀ _ hx -
disintegrate_sumtheorem — Every real test function has the same joint expectation as the marginally weighted guarded posterior expectation.hypothesesconclusion∑ s, ∑ x, jointMass ν L s x * f s x= ∑ x, observationMass ν L x * (∑ s, posteriorWeight ν L x s * f s x)Proof (Lean source)
theorem disintegrate_sum (ν : FiniteDesign S) (L : Kernel S X) (f : S → X → ℝ) : ∑ s, ∑ x, jointMass ν L s x * f s x = ∑ x, observationMass ν L x * (∑ s, posteriorWeight ν L x s * f s x) := by -- Distribute the marginal through the inner sum, use atomwise factorization, and swap sums. classical simp_rw [Finset.mul_sum, ← mul_assoc, observationMass_mul_posteriorWeight ν L] rw [Finset.sum_comm] -
observationMass_mul_posteriorMeantheorem — Observation mass times the guarded posterior mean equals the corresponding joint weighted target numerator, including on null fibers.hypothesesconclusionobservationMass ν L x * posteriorMean ν L t x = ∑ s, jointMass ν L s x * t s xProof (Lean source)
theorem observationMass_mul_posteriorMean (ν : FiniteDesign S) (L : Kernel S X) (t : S → X → ℝ) (x : X) : observationMass ν L x * posteriorMean ν L t x = ∑ s, jointMass ν L s x * t s x := by classical simp_rw [posteriorMean, Finset.mul_sum, ← mul_assoc, observationMass_mul_posteriorWeight ν L] -
posterior_centered_sum_of_postheorem — On a positive-mass observation fiber, posterior target residuals have weighted mean zero.hypothesesS :sharedType u_1X :sharedType u_2ν :L :Kernel S Xt :S → X → ℝx :Xhx :0 < observationMass ν L xconclusion∑ s, posteriorWeight ν L x s * (t s x - posteriorMean ν L t x) = 0Proof (Lean source)
theorem posterior_centered_sum_of_pos (ν : FiniteDesign S) (L : Kernel S X) (t : S → X → ℝ) {x : X} (hx : 0 < observationMass ν L x) : ∑ s, posteriorWeight ν L x s * (t s x - posteriorMean ν L t x) = 0 := by -- Expand subtraction and use positive-fiber posterior normalization plus the mean definition. classical simp_rw [mul_sub] rw [Finset.sum_sub_distrib, ← Finset.sum_mul] rw [posteriorWeight_sum_of_pos ν L hx, one_mul] rw [posteriorMean] exact sub_self _ -
squaredRisk_posteriorMeantheorem — The guarded posterior-mean estimator has squared risk exactly equal to posterior residual risk.hypothesesconclusionsquaredRisk ν L t (posteriorMean ν L t) = posteriorResidual ν L tProof (Lean source)
theorem squaredRisk_posteriorMean (ν : FiniteDesign S) (L : Kernel S X) (t : S → X → ℝ) : squaredRisk ν L t (posteriorMean ν L t) = posteriorResidual ν L t := by rw [squaredRisk_eq_posteriorResidual_add] simp -
posteriorMean_minimizestheorem — The guarded posterior mean minimizes squared risk among all real estimators.hypothesesconclusionsquaredRisk ν L t (posteriorMean ν L t) ≤ squaredRisk ν L t TProof (Lean source)
theorem posteriorMean_minimizes (ν : FiniteDesign S) (L : Kernel S X) (t : S → X → ℝ) (T : X → ℝ) : squaredRisk ν L t (posteriorMean ν L t) ≤ squaredRisk ν L t T := by rw [squaredRisk_posteriorMean, squaredRisk_eq_posteriorResidual_add] exact le_add_of_nonneg_right (sum_nonneg fun x _ => mul_nonneg (observationMass_nonneg ν L x) (sq_nonneg _)) -
iInf_squaredRisk_eq_posteriorResidualtheorem — The infimum over all real estimators of finite squared risk is exactly the posterior residual Bayes risk.hypothesesconclusion(⨅ T : X → ℝ, squaredRisk ν L t T) = posteriorResidual ν L tProof (Lean source)
theorem iInf_squaredRisk_eq_posteriorResidual (ν : FiniteDesign S) (L : Kernel S X) (t : S → X → ℝ) : (⨅ T : X → ℝ, squaredRisk ν L t T) = posteriorResidual ν L t := by -- The minimizer supplies the upper bound. For the lower bound use `le_ciInf` and -- `posteriorMean_minimizes`; no compactness or ENNReal conversion is needed. have hb : BddBelow (Set.range fun T : X → ℝ => squaredRisk ν L t T) := by refine ⟨posteriorResidual ν L t, ?_⟩ rintro _ ⟨T, rfl⟩ calc posteriorResidual ν L t = squaredRisk ν L t (posteriorMean ν L t) := (squaredRisk_posteriorMean ν L t).symm _ ≤ squaredRisk ν L t T := posteriorMean_minimizes ν L t T apply le_antisymm · calc (⨅ T : X → ℝ, squaredRisk ν L t T) ≤ squaredRisk ν L t (posteriorMean ν L t) := ciInf_le hb _ _ = posteriorResidual ν L t := squaredRisk_posteriorMean ν L t · exact le_ciInf fun T => by calc posteriorResidual ν L t = squaredRisk ν L t (posteriorMean ν L t) := (squaredRisk_posteriorMean ν L t).symm _ ≤ squaredRisk ν L t T := posteriorMean_minimizes ν L t T -
finiteDesign_expectedLoss_statewiseSquaredLosstheorem — Finite-design expected statewise squared loss is the joint-law squared risk.hypothesesconclusionν.E (statewiseSquaredLoss L t T) = squaredRisk ν L t TProof (Lean source)
theorem finiteDesign_expectedLoss_statewiseSquaredLoss (ν : FiniteDesign S) (L : Kernel S X) (t : S → X → ℝ) (T : X → ℝ) : ν.E (statewiseSquaredLoss L t T) = squaredRisk ν L t T := by simp only [FiniteDesign.E, statewiseSquaredLoss, squaredRisk, jointMass] simp_rw [Finset.mul_sum, ← mul_assoc] -
mixedKernelLoss_statewiseSquaredLoss_eq_inducedSquaredRisktheorem — The continuous-prior mixture of an estimator's finite-observation squared loss is the real-valued squared risk under the induced finite design.hypothesesΘ :sharedType u_3S :sharedType u_4X :sharedType u_5π :K :Kernel Θ SL :Kernel S Xt :S → X → ℝT :X → ℝconclusionmixedKernelLoss π K (statewiseSquaredLoss L t) T= squaredRisk (inducedFiniteDesign π K) L t TProof (Lean source)
theorem mixedKernelLoss_statewiseSquaredLoss_eq_inducedSquaredRisk (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (L : Kernel S X) (t : S → X → ℝ) (T : X → ℝ) : mixedKernelLoss π K (statewiseSquaredLoss L t) T = squaredRisk (inducedFiniteDesign π K) L t T := by rw [← inducedFiniteDesign_expectedLoss_eq_mixedKernelLoss] exact finiteDesign_expectedLoss_statewiseSquaredLoss _ L t T -
integratedLowerBound_le_inducedFiniteDesignBayesRisktheorem — If an integrated continuous-prior risk agrees estimator by estimator with the induced finite squared risk and has a common lower bound, that constant lower-bounds the real-valued finite-design Bayes-risk infimum.hypothesesΘ :sharedType u_3S :sharedType u_4X :sharedType u_5π :K :Kernel Θ SL :Kernel S Xt :S → X → ℝintegratedRisk :(X → ℝ) → ℝB :ℝhcompat :∀ T, integratedRisk T = squaredRisk (inducedFiniteDesign π K) L t Thlower :∀ T, B ≤ integratedRisk TconclusionBProof (Lean source)
theorem integratedLowerBound_le_inducedFiniteDesignBayesRisk (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (L : Kernel S X) (t : S → X → ℝ) (integratedRisk : (X → ℝ) → ℝ) (B : ℝ) (hcompat : ∀ T, integratedRisk T = squaredRisk (inducedFiniteDesign π K) L t T) (hlower : ∀ T, B ≤ integratedRisk T) : B ≤ finiteDesignBayesRisk (inducedFiniteDesign π K) (statewiseSquaredLoss L t) := by -- Rewrite the generic finite Bayes risk as the infimum of `squaredRisk`; apply `le_ciInf` to -- the lower bound after rewriting each estimator through `hcompat`. unfold finiteDesignBayesRisk simp_rw [finiteDesign_expectedLoss_statewiseSquaredLoss] exact le_ciInf fun T => (hlower T).trans_eq (hcompat T) -
integratedLowerBound_le_inducedPosteriorResidualtheorem — If an integrated continuous-prior risk agrees estimator by estimator with the induced finite squared risk and has a common lower bound, that bound is at most the posterior residual risk of the induced finite experiment.hypothesesΘ :sharedType u_3S :sharedType u_4X :sharedType u_5π :K :Kernel Θ SL :Kernel S Xt :S → X → ℝintegratedRisk :(X → ℝ) → ℝB :ℝhcompat :∀ T, integratedRisk T = squaredRisk (inducedFiniteDesign π K) L t Thlower :∀ T, B ≤ integratedRisk TconclusionB ≤ posteriorResidual (inducedFiniteDesign π K) L tProof (Lean source)
theorem integratedLowerBound_le_inducedPosteriorResidual (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (L : Kernel S X) (t : S → X → ℝ) (integratedRisk : (X → ℝ) → ℝ) (B : ℝ) (hcompat : ∀ T, integratedRisk T = squaredRisk (inducedFiniteDesign π K) L t T) (hlower : ∀ T, B ≤ integratedRisk T) : B ≤ posteriorResidual (inducedFiniteDesign π K) L t := by rw [← finiteDesignBayesRisk_statewiseSquaredLoss] exact integratedLowerBound_le_inducedFiniteDesignBayesRisk π K L t integratedRisk B hcompat hlower -
mixedIntegratedLowerBound_le_inducedFiniteDesignBayesRisktheorem — If an integrated risk agrees estimator by estimator with the generic continuous mixture and has a common lower bound, that bound transfers to the corresponding real-valued induced finite-design Bayes risk.hypothesesΘ :sharedType u_3S :sharedType u_4X :sharedType u_5π :K :Kernel Θ SL :Kernel S Xt :S → X → ℝintegratedRisk :(X → ℝ) → ℝB :ℝhcompat :∀ T, integratedRisk T = mixedKernelLoss π K (statewiseSquaredLoss L t) Thlower :∀ T, B ≤ integratedRisk TconclusionBProof (Lean source)
theorem mixedIntegratedLowerBound_le_inducedFiniteDesignBayesRisk (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (L : Kernel S X) (t : S → X → ℝ) (integratedRisk : (X → ℝ) → ℝ) (B : ℝ) (hcompat : ∀ T, integratedRisk T = mixedKernelLoss π K (statewiseSquaredLoss L t) T) (hlower : ∀ T, B ≤ integratedRisk T) : B ≤ finiteDesignBayesRisk (inducedFiniteDesign π K) (statewiseSquaredLoss L t) := by -- Convert mixture compatibility estimator-by-estimator using the induced-design bridge, then -- invoke `integratedLowerBound_le_inducedFiniteDesignBayesRisk`. apply integratedLowerBound_le_inducedFiniteDesignBayesRisk π K L t integratedRisk B _ hlower intro T exact (hcompat T).trans (mixedKernelLoss_statewiseSquaredLoss_eq_inducedSquaredRisk π K L t T) -
mixedIntegratedLowerBound_le_realBayesRisktheorem — If an integrated risk agrees estimator by estimator with the generic continuous mixture and has a common lower bound, that bound transfers to the real-valued continuous-mixture Bayes risk.hypothesesΘ :sharedType u_3S :sharedType u_4X :sharedType u_5π :K :Kernel Θ SL :Kernel S Xt :S → X → ℝintegratedRisk :(X → ℝ) → ℝB :ℝhcompat :∀ T, integratedRisk T = mixedKernelLoss π K (statewiseSquaredLoss L t) Thlower :∀ T, B ≤ integratedRisk TconclusionB ≤ realBayesRisk π K (statewiseSquaredLoss L t)Proof (Lean source)
theorem mixedIntegratedLowerBound_le_realBayesRisk (π : Measure Θ) [IsProbabilityMeasure π] (K : Kernel Θ S) [IsMarkovKernel K] (L : Kernel S X) (t : S → X → ℝ) (integratedRisk : (X → ℝ) → ℝ) (B : ℝ) (hcompat : ∀ T, integratedRisk T = mixedKernelLoss π K (statewiseSquaredLoss L t) T) (hlower : ∀ T, B ≤ integratedRisk T) : B ≤ realBayesRisk π K (statewiseSquaredLoss L t) := by rw [realBayesRisk_eq_inducedFiniteDesignBayesRisk] exact mixedIntegratedLowerBound_le_inducedFiniteDesignBayesRisk π K L t integratedRisk B hcompat hlower
FuzzyHypotheses 5 core · 3 supporting This module converts target concentration under two priors and total-variation closeness of their predictive mixtures into estimator-wise, worst-case, and minimax squared-risk bounds. ★ twoFuzzyHypotheses_minimax_lower_standard
Squared-loss lower bounds from two fuzzy hypotheses
This module converts target concentration under two priors and total-variation closeness of
their predictive mixtures into estimator-wise, worst-case, and minimax squared-risk bounds.
Risks use ℝ≥0∞ lower integrals, so arbitrary measurable estimators require no artificial
integrability or boundedness assumptions.
def squaredRisk reviewed
⚑ flagged
Causalean.Stat.Minimax.FuzzyHypotheses
Given an experiment kernel, a real-valued target, an estimator, and a parameter value, the squared risk is the expected squared estimation error under the experiment law at that parameter, represented as a nonnegative lower integral.
def bayesSquaredRisk reviewed
⚑ flagged
Causalean.Stat.Minimax.FuzzyHypotheses
Given a prior, an experiment kernel, a real-valued target, and an estimator, the Bayes squared risk averages parameterwise squared risk against the prior.
def worstCaseSquaredRisk reviewed
⚑ flagged
Causalean.Stat.Minimax.FuzzyHypotheses
Given an experiment kernel, a real-valued target, and an estimator, the worst-case squared risk is the supremum of parameterwise risk over the model's parameter class.
Definition (Lean source)
parameters
def
worstCaseSquaredRisk K target estimator :
ℝ≥0∞
given by
⨆ θ, squaredRisk K target estimator θ
def minimaxSquaredRisk reviewed
⚑ flagged
Causalean.Stat.Minimax.FuzzyHypotheses
Given an experiment kernel and a real-valued target, the measurable-estimator minimax squared risk is the infimum of worst-case risk over all measurable real-valued estimators.
Definition (Lean source)
parameters
def
minimaxSquaredRisk K target :
ℝ≥0∞
given by
⨅ estimator : {f : X → ℝ // Measurable f}, worstCaseSquaredRisk K target estimator.1
theorem twoFuzzyHypotheses_minimax_lower_standard reviewed
⚑ flagged
Causalean.Stat.Minimax.FuzzyHypotheses
Given two probability priors, a probability experiment kernel, a measurable real-valued target, two target centers, and a positive center separation, if the centers have at least that separation, each prior puts at most one eighth of its mass beyond one quarter of the separation from its center, and the predictive mixtures are within one sixteenth in total variation, then the minimax squared risk over all measurable estimators is at least eleven times the squared separation divided by 512.
Formal statement
hypotheses
Θ :
Type u_1
X :
Type u_2
π0 π1 :
Measure Θ
K :
Kernel Θ X
target :
Θ → ℝ
hK :
∀ θ, IsProbabilityMeasure (K θ)
htarget :
Measurable target
center0 center1 Delta :
ℝ
hDelta :
0 < Delta
hsep :
Delta ≤ center1 - center0
hmass0 :
π0.real {θ | Delta / 4 < |target θ - center0|} ≤ 1 / 8
hmass1 :
π1.real {θ | Delta / 4 < |target θ - center1|} ≤ 1 / 8
htv :
≤ 1 / 16
conclusion
ofReal (11 * Delta ^ 2 / 512) ≤ minimaxSquaredRisk K target
Proof (Lean source)
theorem twoFuzzyHypotheses_minimax_lower_standard
(π0 π1 : Measure Θ) (K : Kernel Θ X) (target : Θ → ℝ)
[IsProbabilityMeasure π0] [IsProbabilityMeasure π1]
(hK : ∀ θ, IsProbabilityMeasure (K θ)) (htarget : Measurable target)
(center0 center1 Delta : ℝ) (hDelta : 0 < Delta)
(hsep : Delta ≤ center1 - center0)
(hmass0 : π0.real {θ | Delta / 4 < |target θ - center0|} ≤ 1 / 8)
(hmass1 : π1.real {θ | Delta / 4 < |target θ - center1|} ≤ 1 / 8)
(htv : tvDist (priorPredictive π0 K) (priorPredictive π1 K) ≤ 1 / 16) :
ofReal (11 * Delta ^ 2 / 512) ≤ minimaxSquaredRisk K target := by
unfold minimaxSquaredRisk
apply le_iInf
intro estimator
exact twoFuzzyHypotheses_worstCase_lower_standard π0 π1 K target hK estimator.1
estimator.2 htarget center0 center1 Delta hDelta hsep hmass0 hmass1 htv
3 supporting declarations (lemmas, instances)
-
twoFuzzyHypotheses_bayesRisk_lowertheorem — Given two probability priors, a probability experiment kernel, a measurable real-valued target, a measurable estimator, two target centers, a nonnegative center separation, a nonnegative concentration radius smaller than half the separation, the stated separation of the centers, nonnegative prior-tail bounds, and a nonnegative predictive-distance bound, if the first prior concentrates around its center, the second prior concentrates around its center, and the predictive mixtures satisfy the distance bound, then the larger Bayes squared risk is at least half the squared residual separation times one minus the three error terms.hypothesesΘ :sharedType u_1X :sharedType u_2π0 π1 :Measure ΘK :Kernel Θ Xtarget :Θ → ℝhK :∀ θ, IsProbabilityMeasure (K θ)estimator :X → ℝhestimator :Measurable estimatorhtarget :Measurable targetcenter0 center1 Delta r alpha0 alpha1 beta :ℝhDelta :0 ≤ Deltahr :0 ≤ rhrhalf :r < Delta / 2hsep :Delta ≤ center1 - center0halpha0 :0 ≤ alpha0halpha1 :0 ≤ alpha1hbeta :0 ≤ betahmass0 :π0.real {θ | r < |target θ - center0|} ≤ alpha0hmass1 :π1.real {θ | r < |target θ - center1|} ≤ alpha1htv :≤ betaconclusionofReal (((Delta / 2 - r) ^ 2 * (1 - beta - alpha0 - alpha1)) / 2)Proof (Lean source)
theorem twoFuzzyHypotheses_bayesRisk_lower (π0 π1 : Measure Θ) (K : Kernel Θ X) (target : Θ → ℝ) [IsProbabilityMeasure π0] [IsProbabilityMeasure π1] (hK : ∀ θ, IsProbabilityMeasure (K θ)) (estimator : X → ℝ) (hestimator : Measurable estimator) (htarget : Measurable target) (center0 center1 Delta r alpha0 alpha1 beta : ℝ) (hDelta : 0 ≤ Delta) (hr : 0 ≤ r) (hrhalf : r < Delta / 2) (hsep : Delta ≤ center1 - center0) (halpha0 : 0 ≤ alpha0) (halpha1 : 0 ≤ alpha1) (hbeta : 0 ≤ beta) (hmass0 : π0.real {θ | r < |target θ - center0|} ≤ alpha0) (hmass1 : π1.real {θ | r < |target θ - center1|} ≤ alpha1) (htv : tvDist (priorPredictive π0 K) (priorPredictive π1 K) ≤ beta) : ofReal (((Delta / 2 - r) ^ 2 * (1 - beta - alpha0 - alpha1)) / 2) ≤ max (bayesSquaredRisk π0 K target estimator) (bayesSquaredRisk π1 K target estimator) := by let midpoint := (center0 + center1) / 2 let A : Set X := {x | midpoint ≤ estimator x} let B0 : Set Θ := {θ | r < |target θ - center0|} let B1 : Set Θ := {θ | r < |target θ - center1|} let c : ℝ≥0∞ := ofReal ((Delta / 2 - r) ^ 2) let R0 := bayesSquaredRisk π0 K target estimator let R1 := bayesSquaredRisk π1 K target estimator have hA : MeasurableSet A := by dsimp [A, midpoint] exact measurableSet_le measurable_const hestimator have hB0 : MeasurableSet B0 := by dsimp [B0] exact measurableSet_lt measurable_const (continuous_abs.measurable.comp (htarget.sub measurable_const)) have hB1 : MeasurableSet B1 := by dsimp [B1] exact measurableSet_lt measurable_const (continuous_abs.measurable.comp (htarget.sub measurable_const)) letI : IsProbabilityMeasure (priorPredictive π0 K) := Causalean.Stat.Minimax.MomentMatchedMixture.priorPredictive_isProbability π0 K hK letI : IsProbabilityMeasure (priorPredictive π1 K) := Causalean.Stat.Minimax.MomentMatchedMixture.priorPredictive_isProbability π1 K hK have hpoint0 (θ : Θ) : c * K θ A ≤ squaredRisk K target estimator θ + B0.indicator (fun _ => c) θ := by by_cases hθ : θ ∈ B0 · rw [Set.indicator_of_mem hθ] letI : IsProbabilityMeasure (K θ) := hK θ have hKA : K θ A ≤ 1 := by calc K θ A ≤ K θ univ := measure_mono (Set.subset_univ A) _ = 1 := measure_univ calc c * K θ A ≤ c := by simpa only [mul_one] using (mul_le_mul (le_refl c) hKA bot_le bot_le) _ ≤ squaredRisk K target estimator θ + c := by simp · have hgood : |target θ - center0| ≤ r := by simpa [B0, not_lt] using hθ have hdom : ∀ x ∈ A, (Delta / 2 - r) ^ 2 ≤ (estimator x - target θ) ^ 2 := by intro x hx have hx' : midpoint ≤ estimator x := hx have ht : target θ ≤ center0 + r := by linarith [(abs_le.mp hgood).2] have hd : 0 < Delta / 2 - r := sub_pos.mpr hrhalf dsimp [midpoint] at hx' nlinarith have hnot : θ ∉ B0 := hθ simp only [Set.indicator_of_notMem hnot, add_zero] unfold squaredRisk calc c * K θ A = ∫⁻ x, A.indicator (fun _ => c) x ∂K θ := by simp [hA] _ ≤ ∫⁻ x, ofReal ((estimator x - target θ) ^ 2) ∂K θ := by apply lintegral_mono intro x by_cases hx : x ∈ A · simp only [Set.indicator_of_mem hx] exact ENNReal.ofReal_le_ofReal (hdom x hx) · simp [Set.indicator_of_notMem hx] have hpoint1 (θ : Θ) : c * K θ Aᶜ ≤ squaredRisk K target estimator θ + B1.indicator (fun _ => c) θ := by by_cases hθ : θ ∈ B1 · rw [Set.indicator_of_mem hθ] letI : IsProbabilityMeasure (K θ) := hK θ have hKA : K θ Aᶜ ≤ 1 := by calc K θ Aᶜ ≤ K θ univ := measure_mono (Set.subset_univ Aᶜ) _ = 1 := measure_univ calc c * K θ Aᶜ ≤ c := by simpa only [mul_one] using (mul_le_mul (le_refl c) hKA bot_le bot_le) _ ≤ squaredRisk K target estimator θ + c := by simp · have hgood : |target θ - center1| ≤ r := by simpa [B1, not_lt] using hθ have hdom : ∀ x ∈ Aᶜ, (Delta / 2 - r) ^ 2 ≤ (estimator x - target θ) ^ 2 := by intro x hx have hx' : estimator x < midpoint := by simpa [A] using hx have ht : center1 - r ≤ target θ := by have := (abs_le.mp hgood).1 linarith have hd : 0 < Delta / 2 - r := sub_pos.mpr hrhalf dsimp [midpoint] at hx' nlinarith have hnot : θ ∉ B1 := hθ simp only [Set.indicator_of_notMem hnot, add_zero] unfold squaredRisk calc c * K θ Aᶜ = ∫⁻ x, Aᶜ.indicator (fun _ => c) x ∂K θ := by simp [hA.compl] _ ≤ ∫⁻ x, ofReal ((estimator x - target θ) ^ 2) ∂K θ := by apply lintegral_mono intro x by_cases hx : x ∈ Aᶜ · simp only [Set.indicator_of_mem hx] exact ENNReal.ofReal_le_ofReal (hdom x hx) · simp [Set.indicator_of_notMem hx] have hmaster0 : c * priorPredictive π0 K A ≤ R0 + c * π0 B0 := by calc c * priorPredictive π0 K A = ∫⁻ θ, c * K θ A ∂π0 := by rw [lintegral_const_mul'' c ((Kernel.measurable_coe K hA).aemeasurable)] rw [← priorPredictive_apply π0 K hA] _ ≤ ∫⁻ θ, (squaredRisk K target estimator θ + B0.indicator (fun _ => c) θ) ∂π0 := lintegral_mono hpoint0 _ = R0 + c * π0 B0 := by rw [lintegral_add_right _ (measurable_const.indicator hB0)] simp [R0, bayesSquaredRisk, hB0] have hmaster1 : c * priorPredictive π1 K Aᶜ ≤ R1 + c * π1 B1 := by calc c * priorPredictive π1 K Aᶜ = ∫⁻ θ, c * K θ Aᶜ ∂π1 := by rw [lintegral_const_mul'' c ((Kernel.measurable_coe K hA.compl).aemeasurable)] rw [← priorPredictive_apply π1 K hA.compl] _ ≤ ∫⁻ θ, (squaredRisk K target estimator θ + B1.indicator (fun _ => c) θ) ∂π1 := lintegral_mono hpoint1 _ = R1 + c * π1 B1 := by rw [lintegral_add_right _ (measurable_const.indicator hB1)] simp [R1, bayesSquaredRisk, hB1] have htest := one_sub_tvDist_le_test (μ := priorPredictive π0 K) (ν := priorPredictive π1 K) hA have herr : 1 - beta ≤ (priorPredictive π0 K).real A + (priorPredictive π1 K).real Aᶜ := by linarith by_cases hR : max R0 R1 = ∞ · simp [R0, R1, hR] have hR0 : R0 ≠ ∞ := fun h => hR (by simp [h]) have hR1 : R1 ≠ ∞ := fun h => hR (by simp [h]) have hc : c ≠ ∞ := by simp [c] have hcbad0 : c * π0 B0 ≠ ∞ := ENNReal.mul_ne_top hc (measure_ne_top π0 B0) have hcbad1 : c * π1 B1 ≠ ∞ := ENNReal.mul_ne_top hc (measure_ne_top π1 B1) have hmaster0Real := ENNReal.toReal_mono (ENNReal.add_ne_top.mpr ⟨hR0, hcbad0⟩) hmaster0 have hmaster1Real := ENNReal.toReal_mono (ENNReal.add_ne_top.mpr ⟨hR1, hcbad1⟩) hmaster1 rw [ENNReal.toReal_mul, ENNReal.toReal_add hR0 hcbad0, ENNReal.toReal_mul, ← measureReal_def, ← measureReal_def] at hmaster0Real rw [ENNReal.toReal_mul, ENNReal.toReal_add hR1 hcbad1, ENNReal.toReal_mul, ← measureReal_def, ← measureReal_def] at hmaster1Real have hcReal : c.toReal = (Delta / 2 - r) ^ 2 := by simp [c, ENNReal.toReal_ofReal (sq_nonneg _)] have hR0max : R0.toReal ≤ (max R0 R1).toReal := ENNReal.toReal_mono hR (le_max_left _ _) have hR1max : R1.toReal ≤ (max R0 R1).toReal := ENNReal.toReal_mono hR (le_max_right _ _) rw [ENNReal.ofReal_le_iff_le_toReal hR] rw [hcReal] at hmaster0Real hmaster1Real nlinarith [sq_nonneg (Delta / 2 - r)] -
bayesSquaredRisk_le_worstCasetheorem — Given a probability prior, an experiment kernel, a real-valued target, and an estimator, Bayes squared risk is at most worst-case squared risk over the parameter class.hypothesesΘ :sharedType u_1X :sharedType u_2π :Measure ΘK :Kernel Θ Xtarget :Θ → ℝestimator :X → ℝconclusionbayesSquaredRisk π K target estimator ≤ worstCaseSquaredRisk K target estimatorProof (Lean source)
theorem bayesSquaredRisk_le_worstCase (π : Measure Θ) (K : Kernel Θ X) (target : Θ → ℝ) (estimator : X → ℝ) [IsProbabilityMeasure π] : bayesSquaredRisk π K target estimator ≤ worstCaseSquaredRisk K target estimator := by unfold bayesSquaredRisk worstCaseSquaredRisk apply lintegral_le_const filter_upwards with θ exact le_iSup (fun θ => squaredRisk K target estimator θ) θ -
twoFuzzyHypotheses_worstCase_lower_standardtheorem — Given two probability priors, a probability experiment kernel, a measurable real-valued target, a measurable estimator, two target centers, and a positive center separation, if the centers have at least that separation, each prior puts at most one eighth of its mass beyond one quarter of the separation from its center, and the predictive mixtures are within one sixteenth in total variation, then the estimator's worst-case squared risk is at least eleven times the squared separation divided by 512.hypothesesΘ :sharedType u_1X :sharedType u_2π0 π1 :Measure ΘK :Kernel Θ Xtarget :Θ → ℝhK :∀ θ, IsProbabilityMeasure (K θ)estimator :X → ℝhestimator :Measurable estimatorhtarget :Measurable targetcenter0 center1 Delta :ℝhDelta :0 < Deltahsep :Delta ≤ center1 - center0hmass0 :π0.real {θ | Delta / 4 < |target θ - center0|} ≤ 1 / 8hmass1 :π1.real {θ | Delta / 4 < |target θ - center1|} ≤ 1 / 8htv :≤ 1 / 16conclusionofReal (11 * Delta ^ 2 / 512) ≤ worstCaseSquaredRisk K target estimatorProof (Lean source)
theorem twoFuzzyHypotheses_worstCase_lower_standard (π0 π1 : Measure Θ) (K : Kernel Θ X) (target : Θ → ℝ) [IsProbabilityMeasure π0] [IsProbabilityMeasure π1] (hK : ∀ θ, IsProbabilityMeasure (K θ)) (estimator : X → ℝ) (hestimator : Measurable estimator) (htarget : Measurable target) (center0 center1 Delta : ℝ) (hDelta : 0 < Delta) (hsep : Delta ≤ center1 - center0) (hmass0 : π0.real {θ | Delta / 4 < |target θ - center0|} ≤ 1 / 8) (hmass1 : π1.real {θ | Delta / 4 < |target θ - center1|} ≤ 1 / 8) (htv : tvDist (priorPredictive π0 K) (priorPredictive π1 K) ≤ 1 / 16) : ofReal (11 * Delta ^ 2 / 512) ≤ worstCaseSquaredRisk K target estimator := by have hbayes := twoFuzzyHypotheses_bayesRisk_lower π0 π1 K target hK estimator hestimator htarget center0 center1 Delta (Delta / 4) (1 / 8) (1 / 8) (1 / 16) (le_of_lt hDelta) (by positivity) (by linarith) hsep (by norm_num) (by norm_num) (by norm_num) hmass0 hmass1 htv have hconstant : (((Delta / 2 - Delta / 4) ^ 2 * (1 - (1 / 16 : ℝ) - 1 / 8 - 1 / 8)) / 2) = 11 * Delta ^ 2 / 512 := by ring rw [hconstant] at hbayes exact hbayes.trans (max_le (bayesSquaredRisk_le_worstCase π0 K target estimator) (bayesSquaredRisk_le_worstCase π1 K target estimator))
HellingerAffinity 3 core · 3 supporting This file develops the Hellinger/Bhattacharyya affinity of two nonnegative densities taken against a single dominating measure, its identity with the squared Hellinger distance, the Cauchy–Schwarz bound of total variatio ★ tvDist_le_sqrt_two_mul_one_sub_affinity
Hellinger Affinity for Common-Measure Densities
This file develops the Hellinger/Bhattacharyya affinity of two nonnegative densities taken against a single dominating measure, its identity with the squared Hellinger distance, the Cauchy–Schwarz bound of total variation by the affinity defect, and the tensorization of affinity over finite products. These are the ingredients of a product-construction Le Cam two-point (or multi-point) lower bound.
def densityAffinity reviewed
⚑ flagged
Causalean.Stat
Given a measurable sample space, a measure on that space, and two real-valued functions on it, the Hellinger--Bhattacharyya affinity of the functions relative to the measure is the integral of the square root of their pointwise product.
Definition (Lean source)
def hellingerSqDensity reviewed
⚑ flagged
Causalean.Stat
Given a measurable sample space, a measure on that space, and two real-valued functions on it, the unhalved squared Hellinger discrepancy relative to the measure is the integral of the squared difference between their pointwise square roots.
Definition (Lean source)
parameters
def
hellingerSqDensity μ f g :
ℝ
lemma tvDist_le_sqrt_two_mul_one_sub_affinity reviewed
⚑ flagged
Causalean.Stat
Cauchy–Schwarz on the Hellinger affinity. For a dominating measure μ and functions f, g such that f is μ-integrable, g is μ-integrable, f is pointwise nonnegative, g is pointwise nonnegative, f integrates to 1 against μ, and g integrates to 1 against μ — so that f dμ and g dμ are probability densities — the total variation distance between the two weighted laws is at most the square root of twice the affinity defect, √(2(1 − densityAffinity μ f g)). This is the Cauchy–Schwarz half of the standard total-variation–Hellinger comparison, and it is what converts an affinity computation into a Le Cam two-point bound.
Formal statement
hypotheses
α :
Type*
μ :
Measure α
f g :
α → ℝ
hf :
Integrable f μ
hg :
Integrable g μ
hf0 :
0 ≤ f
hg0 :
0 ≤ g
hf1 :
∫ x, f x ∂μ = 1
hg1 :
∫ x, g x ∂μ = 1
conclusion
≤ sqrt (2 * (1 - densityAffinity μ f g))
Proof (Lean source)
lemma tvDist_le_sqrt_two_mul_one_sub_affinity
{α : Type*} [MeasurableSpace α] (μ : Measure α)
(f g : α → ℝ)
(hf : Integrable f μ) (hg : Integrable g μ)
(hf0 : 0 ≤ f) (hg0 : 0 ≤ g)
(hf1 : ∫ x, f x ∂μ = 1) (hg1 : ∫ x, g x ∂μ = 1) :
tvDist
(μ.withDensity (fun x => ofReal (f x)))
(μ.withDensity (fun x => ofReal (g x))) ≤
sqrt (2 * (1 - densityAffinity μ f g)) := by
let P := μ.withDensity (fun x => ofReal (f x))
let Q := μ.withDensity (fun x => ofReal (g x))
have hreal_f {A : Set α} (hA : MeasurableSet A) :
P.real A = ∫ x in A, f x ∂μ := by
rw [measureReal_def, withDensity_apply _ hA]
rw [← MeasureTheory.ofReal_integral_eq_lintegral_ofReal hf.integrableOn]
· rw [ENNReal.toReal_ofReal]
exact integral_nonneg_of_ae
(Filter.Eventually.of_forall fun x => hf0 x)
· exact Filter.Eventually.of_forall fun x => hf0 x
have hreal_g {A : Set α} (hA : MeasurableSet A) :
Q.real A = ∫ x in A, g x ∂μ := by
rw [measureReal_def, withDensity_apply _ hA]
rw [← MeasureTheory.ofReal_integral_eq_lintegral_ofReal hg.integrableOn]
· rw [ENNReal.toReal_ofReal]
exact integral_nonneg_of_ae
(Filter.Eventually.of_forall fun x => hg0 x)
· exact Filter.Eventually.of_forall fun x => hg0 x
have hfg : Integrable (fun x => f x - g x) μ := hf.sub hg
have hfg0 : ∫ x, (f x - g x) ∂μ = 0 := by
integral_linearity
rw [hf1, hg1]
ring
have htv :
tvDist P Q ≤
(1 / 2 : ℝ) * ∫ x, |f x - g x| ∂μ := by
unfold tvDist
refine ciSup_le fun A => ?_
rw [hreal_f A.2, hreal_g A.2,
← integral_sub hf.integrableOn hg.integrableOn]
exact
abs_setIntegral_le_half_integral_abs_of_integral_eq_zero
hfg hfg0 A.2
have hsf_asm :
AEStronglyMeasurable (fun x => sqrt (f x)) μ :=
(Real.continuous_sqrt.measurable.comp_aemeasurable
hf.aestronglyMeasurable.aemeasurable).aestronglyMeasurable
have hsg_asm :
AEStronglyMeasurable (fun x => sqrt (g x)) μ :=
(Real.continuous_sqrt.measurable.comp_aemeasurable
hg.aestronglyMeasurable.aemeasurable).aestronglyMeasurable
have hsf : MemLp (fun x => sqrt (f x)) 2 μ := by
rw [memLp_two_iff_integrable_sq hsf_asm]
convert hf using 1
funext x
exact Real.sq_sqrt (hf0 x)
have hsg : MemLp (fun x => sqrt (g x)) 2 μ := by
rw [memLp_two_iff_integrable_sq hsg_asm]
convert hg using 1
funext x
exact Real.sq_sqrt (hg0 x)
let u : α → ℝ := fun x => sqrt (f x) - sqrt (g x)
let v : α → ℝ := fun x => sqrt (f x) + sqrt (g x)
have hu : MemLp u 2 μ := hsf.sub hsg
have hv : MemLp v 2 μ := hsf.add hsg
have hfact : ∀ x, |f x - g x| = |u x| * |v x| := by
intro x
rw [← abs_mul]
dsimp [u, v]
rw [show
(sqrt (f x) - sqrt (g x)) *
(sqrt (f x) + sqrt (g x)) =
sqrt (f x) ^ 2 - sqrt (g x) ^ 2 by ring]
simp only [Real.sq_sqrt (hf0 x), Real.sq_sqrt (hg0 x)]
have hholder :
∫ x, |u x| * |v x| ∂μ ≤
(∫ x, |u x| ^ (2 : ℝ) ∂μ) ^ (1 / (2 : ℝ)) *
(∫ x, |v x| ^ (2 : ℝ) ∂μ) ^ (1 / (2 : ℝ)) := by
apply integral_mul_le_Lp_mul_Lq_of_nonneg
(p := (2 : ℝ)) (q := (2 : ℝ))
· exact Real.HolderConjugate.two_two
· exact Filter.Eventually.of_forall fun x => abs_nonneg _
· exact Filter.Eventually.of_forall fun x => abs_nonneg _
· rw [show (ofReal 2 : ℝ≥0∞) = 2 from by norm_num]
exact hu.abs
· rw [show (ofReal 2 : ℝ≥0∞) = 2 from by norm_num]
exact hv.abs
have hsfg : Integrable
(fun x => sqrt (f x) * sqrt (g x)) μ :=
hsf.integrable_mul hsg
have hu_sq :
∫ x, |u x| ^ (2 : ℝ) ∂μ =
2 * (1 - densityAffinity μ f g) := by
simp_rw [Real.rpow_two, sq_abs]
have hpoint : (fun x => u x ^ 2) =
(fun x => f x + g x -
2 * sqrt (f x * g x)) := by
funext x
dsimp [u]
rw [show sqrt (f x * g x) =
sqrt (f x) * sqrt (g x) by
rw [Real.sqrt_mul (hf0 x)]]
nlinarith [Real.sq_sqrt (hf0 x), Real.sq_sqrt (hg0 x)]
rw [hpoint]
have hsqrtfg : Integrable (fun x => sqrt (f x * g x)) μ := by
convert hsfg using 1
funext x
rw [Real.sqrt_mul (hf0 x)]
change ∫ x, (f + g) x - (2 * (fun x => sqrt (f x * g x)) x) ∂μ =
2 * (1 - densityAffinity μ f g)
rw [integral_sub (hf.add hg) (hsqrtfg.const_mul 2)]
change (∫ x, f x + g x ∂μ) -
∫ x, 2 * sqrt (f x * g x) ∂μ =
2 * (1 - densityAffinity μ f g)
rw [integral_add hf hg, integral_const_mul, hf1, hg1]
simp [densityAffinity]
ring
have hv_sq_plain :
∫ x, v x ^ 2 ∂μ ≤ 4 := by
calc
∫ x, v x ^ 2 ∂μ ≤ ∫ x, (2 * f x + 2 * g x) ∂μ := by
apply integral_mono_ae hv.integrable_sq
((hf.const_mul 2).add (hg.const_mul 2))
exact Filter.Eventually.of_forall fun x => by
dsimp [v]
have hsf0 := Real.sqrt_nonneg (f x)
have hsg0 := Real.sqrt_nonneg (g x)
nlinarith [Real.sq_sqrt (hf0 x), Real.sq_sqrt (hg0 x),
sq_nonneg (sqrt (f x) - sqrt (g x))]
_ = 4 := by
rw [integral_add (hf.const_mul 2) (hg.const_mul 2),
integral_const_mul, integral_const_mul, hf1, hg1]
ring
have hv_sq :
∫ x, |v x| ^ (2 : ℝ) ∂μ ≤ 4 := by
simpa [Real.rpow_two, sq_abs] using hv_sq_plain
have hu_sq_nonneg : 0 ≤ 2 * (1 - densityAffinity μ f g) := by
rw [← hu_sq]
exact integral_nonneg_of_ae
(Filter.Eventually.of_forall fun x => by positivity)
exact htv.trans (by
rw [show (fun x => |f x - g x|) =
(fun x => |u x| * |v x|) by funext x; exact hfact x]
calc
(1 / 2 : ℝ) * ∫ x, |u x| * |v x| ∂μ ≤
(1 / 2 : ℝ) *
((∫ x, |u x| ^ (2 : ℝ) ∂μ) ^ (1 / (2 : ℝ)) *
(∫ x, |v x| ^ (2 : ℝ) ∂μ) ^ (1 / (2 : ℝ))) := by
gcongr
_ ≤ sqrt (2 * (1 - densityAffinity μ f g)) := by
rw [hu_sq, ← Real.sqrt_eq_rpow]
have hv_sq_nonneg :
0 ≤ ∫ x, |v x| ^ (2 : ℝ) ∂μ :=
integral_nonneg_of_ae
(Filter.Eventually.of_forall fun x => by positivity)
rw [← Real.sqrt_eq_rpow]
have hsqrtv :
sqrt (∫ x, |v x| ^ (2 : ℝ) ∂μ) ≤ 2 := by
rw [Real.sqrt_le_iff]
norm_num
simpa [Real.rpow_two, sq_abs] using hv_sq
nlinarith [Real.sqrt_nonneg
(2 * (1 - densityAffinity μ f g))])
3 supporting declarations (lemmas, instances)
-
hellingerSqDensity_eq_two_mul_one_sub_affinitylemma — For two integrable nonnegative densities that each integrate to one, the unhalved squared Hellinger distance is exactly twice the affinity defect, i.e. twice one minus the affinity. This is the algebraic identity that lets an affinity computation be read as a Hellinger distance and back.hypothesesα :Type*μ :Measure αf g :α → ℝhf :Integrable f μhg :Integrable g μhf0 :0 ≤ fhg0 :0 ≤ ghf1 :∫ x, f x ∂μ = 1hg1 :∫ x, g x ∂μ = 1conclusionhellingerSqDensity μ f g = 2 * (1 - densityAffinity μ f g)Proof (Lean source)
lemma hellingerSqDensity_eq_two_mul_one_sub_affinity {α : Type*} [MeasurableSpace α] (μ : Measure α) (f g : α → ℝ) (hf : Integrable f μ) (hg : Integrable g μ) (hf0 : 0 ≤ f) (hg0 : 0 ≤ g) (hf1 : ∫ x, f x ∂μ = 1) (hg1 : ∫ x, g x ∂μ = 1) : hellingerSqDensity μ f g = 2 * (1 - densityAffinity μ f g) := by have hsf_asm : AEStronglyMeasurable (fun x => sqrt (f x)) μ := (Real.continuous_sqrt.measurable.comp_aemeasurable hf.aestronglyMeasurable.aemeasurable).aestronglyMeasurable have hsg_asm : AEStronglyMeasurable (fun x => sqrt (g x)) μ := (Real.continuous_sqrt.measurable.comp_aemeasurable hg.aestronglyMeasurable.aemeasurable).aestronglyMeasurable have hsf : MemLp (fun x => sqrt (f x)) 2 μ := by rw [memLp_two_iff_integrable_sq hsf_asm] convert hf using 1 funext x exact Real.sq_sqrt (hf0 x) have hsg : MemLp (fun x => sqrt (g x)) 2 μ := by rw [memLp_two_iff_integrable_sq hsg_asm] convert hg using 1 funext x exact Real.sq_sqrt (hg0 x) have hcross : Integrable (fun x => sqrt (f x * g x)) μ := by convert hsf.integrable_mul hsg using 1 funext x simp only [Pi.mul_apply] rw [Real.sqrt_mul (hf0 x)] unfold hellingerSqDensity densityAffinity rw [show (fun x => (sqrt (f x) - sqrt (g x)) ^ 2) = (fun x => f x + g x - 2 * sqrt (f x * g x)) by funext x rw [Real.sqrt_mul (hf0 x)] nlinarith [Real.sq_sqrt (hf0 x), Real.sq_sqrt (hg0 x)]] change ∫ x, (f + g) x - 2 * sqrt (f x * g x) ∂μ = 2 * (1 - ∫ x, sqrt (f x * g x) ∂μ) rw [integral_sub (hf.add hg) (hcross.const_mul 2), show (∫ x, (f + g) x ∂μ) = ∫ x, f x + g x ∂μ by rfl, integral_add hf hg, integral_const_mul, hf1, hg1] ring -
one_sub_prod_le_sumlemma — For finitely many numbers in the unit interval, the amount by which their product falls short of one is at most the total shortfall of the individual factors. Applied to coordinate affinities this is the union-bound step that turns a tensorized affinity into a sum of coordinate defects.hypothesesconclusion1 - ∏ i, a i ≤ ∑ i, (1 - a i)Proof (Lean source)
lemma one_sub_prod_le_sum {ι : Type*} [Fintype ι] (a : ι → ℝ) (ha0 : ∀ i, 0 ≤ a i) (ha1 : ∀ i, a i ≤ 1) : 1 - ∏ i, a i ≤ ∑ i, (1 - a i) := by classical have hbounds (s : Finset ι) : 0 ≤ ∏ i ∈ s, a i ∧ ∏ i ∈ s, a i ≤ 1 := by induction s using Finset.induction_on with | empty => simp | @insert i s hi ih => rw [Finset.prod_insert hi] constructor <;> nlinarith [ha0 i, ha1 i, ih.1, ih.2] have hprod (s : Finset ι) : 1 - ∏ i ∈ s, a i ≤ ∑ i ∈ s, (1 - a i) := by induction s using Finset.induction_on with | empty => simp | @insert i s hi ih => rw [Finset.prod_insert hi, Finset.sum_insert hi] have hP0 : 0 ≤ ∏ j ∈ s, a j := (hbounds s).1 have hP1 : ∏ j ∈ s, a j ≤ 1 := (hbounds s).2 have hsum0 : 0 ≤ ∑ j ∈ s, (1 - a j) := by exact sum_nonneg fun j _ => sub_nonneg.mpr (ha1 j) nlinarith [ha0 i, ha1 i] simpa using hprod univ -
densityAffinity_pilemma — Affinity tensorizes. On a finite product of σ-finite measure spaces, the affinity of two densities that each factor coordinatewise is the product of the coordinate affinities. This is what makes an n-fold product construction tractable: a single coordinate defect computation is enough.hypothesesι :∀ i, MeasurableSpace (E i)μ :∀ i, Measure (E i)∀ i, SigmaFinite (μ i)f g :∀ iifE ithenℝhf0 :∀ i u, 0 ≤ f i uhg0 :∀ i u, 0 ≤ g i uconclusiondensityAffinity (Measure.pi μ) (fun x => ∏ i, f i (x i)) (fun x => ∏ i, g i (x i))= ∏ i, ∫ u, sqrt (f i u * g i u) ∂(μ i)Proof (Lean source)
lemma densityAffinity_pi {ι : Type*} [Fintype ι] {E : ι → Type*} [∀ i, MeasurableSpace (E i)] (μ : ∀ i, Measure (E i)) [∀ i, SigmaFinite (μ i)] (f g : ∀ i, E i → ℝ) (hf0 : ∀ i u, 0 ≤ f i u) (hg0 : ∀ i u, 0 ≤ g i u) : densityAffinity (Measure.pi μ) (fun x => ∏ i, f i (x i)) (fun x => ∏ i, g i (x i)) = ∏ i, ∫ u, sqrt (f i u * g i u) ∂(μ i) := by classical unfold densityAffinity rw [show (fun x : ∀ i, E i => sqrt ((∏ i, f i (x i)) * ∏ i, g i (x i))) = (fun x => ∏ i, sqrt (f i (x i) * g i (x i))) by funext x rw [← Finset.prod_mul_distrib] rw [Real.sqrt_prod] intro i hi exact mul_nonneg (hf0 i (x i)) (hg0 i (x i))] exact MeasureTheory.integral_fintype_prod_eq_prod (fun i u => sqrt (f i u * g i u))
HonestConfidenceSet 4 core · 6 supporting This module provides model-free measure and order lemmas for honest random confidence sets. ★ coverage_tv_expectedRestrictedVolume_lower
Honest confidence sets and frontier risk
This module provides model-free measure and order lemmas for honest random confidence sets. It relates expected restricted volume to pointwise inclusion probabilities, transfers coverage through total variation, and packages uniform asymptotic coverage and frontier-risk bounds over arbitrary model classes. It also fixes the worst-case coverage convention for empty model classes.
def restrictedSetVolume reviewed
⚑ flagged
Causalean.Stat
For a parameter region on the real line and a set on the real line, the restricted set volume is the real-valued Lebesgue volume of their intersection, with infinite volume represented by zero.
theorem coverage_tv_expectedRestrictedVolume_lower reviewed
⚑ flagged
Causalean.Stat
Coverage-to-expected-restricted-volume bound. For a family of laws Q u indexed by u : ℝ, a random set C, a subset I of a parameter region region, and a reference point reference, suppose every Q u is a probability measure, C covers u with probability at least coverage, for every u in I, Q u is within total variation tv of the reference law Q reference, for every u in I, the graph {(ω, u) | u ∈ C ω} is measurable, region is measurable, region has finite Lebesgue volume, I is measurable, and I is contained in region. Then the expected restricted volume of C under the reference law Q reference is at least (volume I) · (coverage − tv).
Formal statement
hypotheses
Ω :
Type*
Q :
ℝ → Measure Ω
C :
Ω → Set ℝ
region I :
Set ℝ
reference coverage tv :
ℝ
hQ :
∀ u, IsProbabilityMeasure (Q u)
hcover :
∀ u ∈ I, coverage ≤ (Q u {ω | u ∈ C ω}).toReal
htv :
∀ u ∈ I, tvDist (Q u) (Q reference) ≤ tv
hgraph :
MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1}
hregion :
MeasurableSet region
hregionFinite :
volume region ≠ ⊤
hI :
hI_sub :
I ⊆ region
conclusion
Proof (Lean source)
theorem coverage_tv_expectedRestrictedVolume_lower
{Ω : Type*} [MeasurableSpace Ω]
(Q : ℝ → Measure Ω) (C : Ω → Set ℝ) (region I : Set ℝ)
(reference coverage tv : ℝ)
(hQ : ∀ u, IsProbabilityMeasure (Q u))
(hcover : ∀ u ∈ I, coverage ≤ (Q u {ω | u ∈ C ω}).toReal)
(htv : ∀ u ∈ I, tvDist (Q u) (Q reference) ≤ tv)
(hgraph : MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1})
(hregion : MeasurableSet region) (hregionFinite : volume region ≠ ⊤)
(hI : MeasurableSet I) (hI_sub : I ⊆ region) :
(volume I).toReal * (coverage - tv) ≤
∫ ω, restrictedSetVolume region (C ω) ∂Q reference := by
letI : IsProbabilityMeasure (Q reference) := hQ reference
have hpoint : ∀ u ∈ I,
coverage - tv ≤ (Q reference {ω | u ∈ C ω}).toReal := by
intro u hu
letI : IsProbabilityMeasure (Q u) := hQ u
have hE : MeasurableSet {ω | u ∈ C ω} :=
hgraph.preimage (measurable_id.prodMk measurable_const)
have hgap := measureReal_sub_le_tvDist
(μ := Q reference) (ν := Q u) hE
change (Q u {ω | u ∈ C ω}).toReal -
(Q reference {ω | u ∈ C ω}).toReal ≤
tvDist (Q reference) (Q u) at hgap
rw [tvDist_symm] at hgap
linarith [hcover u hu, htv u hu]
rw [expected_restrictedSetVolume_eq_integral_inclusion
(Q := Q reference) C region hgraph hregion hregionFinite]
have hmeas : Measurable fun u =>
(Q reference {ω | u ∈ C ω}).toReal :=
Measurable.ennreal_toReal (measurable_measure_prodMk_right hgraph)
have hIfinite : volume I ≠ ⊤ := ne_top_of_le_ne_top hregionFinite
(measure_mono hI_sub)
have hrhsint : IntegrableOn
(fun u => (Q reference {ω | u ∈ C ω}).toReal) I := by
letI : IsFiniteMeasure (volume.restrict I) :=
isFiniteMeasure_restrict.mpr hIfinite
exact Integrable.of_bound
hmeas.aestronglyMeasurable.restrict 1
(Filter.Eventually.of_forall fun u => by
rw [Real.norm_eq_abs, abs_of_nonneg ENNReal.toReal_nonneg]
simpa only [← measureReal_def] using
(measureReal_le_one (μ := Q reference)
(s := {ω | u ∈ C ω})))
have hrhsRegion : IntegrableOn
(fun u => (Q reference {ω | u ∈ C ω}).toReal) region := by
letI : IsFiniteMeasure (volume.restrict region) :=
isFiniteMeasure_restrict.mpr hregionFinite
exact Integrable.of_bound hmeas.aestronglyMeasurable.restrict 1
(Filter.Eventually.of_forall fun u => by
rw [Real.norm_eq_abs, abs_of_nonneg ENNReal.toReal_nonneg]
simpa only [← measureReal_def] using
(measureReal_le_one (μ := Q reference)
(s := {ω | u ∈ C ω})))
calc
(volume I).toReal * (coverage - tv) =
∫ _u in I, coverage - tv := by
rw [setIntegral_const]
simp [measureReal_def]
_ ≤ ∫ u in I, (Q reference {ω | u ∈ C ω}).toReal :=
setIntegral_mono_on (integrableOn_const hIfinite) hrhsint hI hpoint
_ ≤ ∫ u in region, (Q reference {ω | u ∈ C ω}).toReal :=
setIntegral_mono_set hrhsRegion
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(Filter.Eventually.of_forall hI_sub)
def classFrontierRisk reviewed
⚑ flagged
Causalean.Stat
For a sequence of model classes, a real-valued strength and a real-valued expected-length criterion for each sample size and model, and a real strength threshold, the class frontier risk is the limit superior, across sample sizes, of the supremum expected length over the models in that class whose strength is at least the threshold.
def coverageInfOrOne reviewed
⚑ flagged
Causalean.Stat
For a real-valued criterion indexed by an arbitrary collection, the worst-case criterion with the empty-collection convention is its infimum when the collection is nonempty and is one when it is empty.
Definition (Lean source)
parameters
ι :
Sort*
f :
ι → ℝ
def
coverageInfOrOne f :
ℝ
given by
by classical exact if Nonempty ι then ⨅ i, f i else 1
6 supporting declarations (lemmas, instances)
-
expected_restrictedSetVolume_eq_integral_inclusiontheorem — The expected restricted volume of a jointly measurable random set equals the integral, over the parameter region, of its pointwise inclusion probabilities.hypothesesΩ :Type*Q :C :Ω → Set ℝregion :Set ℝhgraph :MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1}hregion :MeasurableSet regionhregionFinite :volume region ≠ ⊤conclusion(∫ ω, restrictedSetVolume region (C ω) ∂Q) = ∫ u in region, (Q {ω | u ∈ C ω}).toRealProof (Lean source)
theorem expected_restrictedSetVolume_eq_integral_inclusion {Ω : Type*} [MeasurableSpace Ω] (Q : Measure Ω) [IsFiniteMeasure Q] (C : Ω → Set ℝ) (region : Set ℝ) (hgraph : MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1}) (hregion : MeasurableSet region) (hregionFinite : volume region ≠ ⊤) : (∫ ω, restrictedSetVolume region (C ω) ∂Q) = ∫ u in region, (Q {ω | u ∈ C ω}).toReal := by let S : Set (Ω × ℝ) := {p | p.2 ∈ C p.1 ∧ p.2 ∈ region} let f : Ω × ℝ → ℝ := S.indicator (fun _ => 1) have hS : MeasurableSet S := hgraph.inter (hregion.preimage measurable_snd) have hSsub : S ⊆ univ ×ˢ region := by intro p hp exact ⟨Set.mem_univ _, hp.2⟩ have hSfinite : (Q.prod volume) S ≠ ⊤ := by apply ne_of_lt calc (Q.prod volume) S ≤ (Q.prod volume) (univ ×ˢ region) := measure_mono hSsub _ = Q univ * volume region := by rw [Measure.prod_prod] _ < ⊤ := ENNReal.mul_lt_top (measure_lt_top Q univ) (lt_top_iff_ne_top.mpr hregionFinite) have hfint : Integrable f (Q.prod volume) := by rw [show f = S.indicator (fun _ => (1 : ℝ)) from rfl] exact (integrableOn_const hSfinite).integrable_indicator hS calc (∫ ω, restrictedSetVolume region (C ω) ∂Q) = ∫ ω, (∫ u, f (ω, u) ∂volume) ∂Q := by apply integral_congr_ae filter_upwards with ω rw [show restrictedSetVolume region (C ω) = (volume (C ω ∩ region)).toReal from rfl] rw [show (fun u => f (ω, u)) = (C ω ∩ region).indicator (fun _ => (1 : ℝ)) by funext u simp only [f, S, indicator] split_ifs <;> simp_all] exact (integral_indicator_one (hgraph.preimage (measurable_const.prodMk measurable_id) |>.inter hregion)).symm _ = ∫ u, (∫ ω, f (ω, u) ∂Q) ∂volume := by calc _ = ∫ z, f z ∂Q.prod volume := (integral_prod (μ := Q) (ν := volume) f hfint).symm _ = _ := integral_prod_symm (μ := Q) (ν := volume) f hfint _ = ∫ u in region, (Q {ω | u ∈ C ω}).toReal := by rw [← MeasureTheory.integral_indicator hregion] apply integral_congr_ae filter_upwards with u by_cases hu : u ∈ region · rw [show (fun ω => f (ω, u)) = {ω | u ∈ C ω}.indicator (fun _ => (1 : ℝ)) by funext ω simp only [f, S, indicator] split_ifs <;> simp_all] rw [Set.indicator_of_mem hu] exact integral_indicator_one (μ := Q) (hgraph.preimage (measurable_id.prodMk measurable_const)) · simp [f, S, hu] -
classCoverage_liminftheorem — Pointwise coverage rows with a vanishing uniform error imply asymptotic uniform coverage over any eventually inhabited sequence of model classes.hypothesesModel :Type*cls :ℕ → Model → Propcoverage :ℕ → Model → ℝalpha :ℝdelta :ℕ → ℝhInhab :∀ᶠ n in atTop, ∃ P, cls n Phcoverage :∀ n Pifcls n Pthen0 ≤ coverage n P ∧ coverage n P ≤ 1hrow :∀ n Pifcls n Pthen1 - alpha - delta n ≤ coverage n PProof (Lean source)
theorem classCoverage_liminf {Model : Type*} (cls : ℕ → Model → Prop) (coverage : ℕ → Model → ℝ) (alpha : ℝ) (delta : ℕ → ℝ) (hdelta : Tendsto delta atTop (𝓝 0)) (hInhab : ∀ᶠ n in atTop, ∃ P, cls n P) (hcoverage : ∀ n P, cls n P → 0 ≤ coverage n P ∧ coverage n P ≤ 1) (hrow : ∀ n P, cls n P → 1 - alpha - delta n ≤ coverage n P) : 1 - alpha ≤ liminf (fun n => ⨅ P : {P : Model // cls n P}, coverage n P) atTop := by let row : ℕ → ℝ := fun n => ⨅ P : {P : Model // cls n P}, coverage n P have hrows : ∀ᶠ n in atTop, 1 - alpha - delta n ≤ row n := by filter_upwards [hInhab] with n hn letI : Nonempty {P : Model // cls n P} := ⟨⟨Classical.choose hn, Classical.choose_spec hn⟩⟩ apply le_ciInf intro P exact hrow n P P.2 have hrowUpper : ∀ᶠ n in atTop, row n ≤ 1 := by filter_upwards [hInhab] with n hn obtain ⟨P₀, hP₀⟩ := hn have hbdd : BddBelow (Set.range fun P : {P : Model // cls n P} => coverage n P) := by refine ⟨0, ?_⟩ rintro y ⟨P, rfl⟩ exact (hcoverage n P P.2).1 exact (ciInf_le hbdd ⟨P₀, hP₀⟩).trans (hcoverage n P₀ hP₀).2 have hlowerBounded : IsBoundedUnder (· ≥ ·) atTop (fun n => 1 - alpha - delta n) := by change ∃ b, ∀ᶠ n in atTop, b ≤ 1 - alpha - delta n have hdeltalt : ∀ᶠ n in atTop, delta n < 1 := (tendsto_order.1 hdelta).2 1 zero_lt_one exact ⟨-alpha, hdeltalt.mono fun _ hn => by linarith⟩ have hrowCobounded : IsCoboundedUnder (· ≥ ·) atTop row := by change ∃ b, ∀ a, (∀ᶠ n in atTop, a ≤ row n) → a ≤ b refine ⟨1, fun a ha => ?_⟩ obtain ⟨n, han, hn1⟩ := (ha.and hrowUpper).exists exact han.trans hn1 have hlowerTendsto : Tendsto (fun n => 1 - alpha - delta n) atTop (𝓝 (1 - alpha)) := by simpa using ((tendsto_const_nhds.sub tendsto_const_nhds).sub hdelta) change 1 - alpha ≤ liminf row atTop rw [← hlowerTendsto.liminf_eq] exact Filter.liminf_le_liminf hrows hlowerBounded hrowCobounded -
inverseSqrtCap_antitheorem — The capped inverse-square-root rate is antitone on positive strengths.hypothesest0 t :ℝht0 :0 < t0htt :t0 ≤ tProof (Lean source)
theorem inverseSqrtCap_anti {t0 t : ℝ} (ht0 : 0 < t0) (htt : t0 ≤ t) : min 1 (t ^ (-1 / 2 : ℝ)) ≤ min 1 (t0 ^ (-1 / 2 : ℝ)) := by apply min_le_min_left exact Real.rpow_le_rpow_of_nonpos ht0 htt (by norm_num) -
classFrontierRisk_letheorem — A pointwise capped inverse-square-root expected-length bound passes through both the class supremum and asymptotic limsup at the threshold value.hypothesesModel :Type*cls :ℕ → Model → Propstrength expectedLength :ℕ → Model → ℝC0 t0 :ℝhC0 :0 ≤ C0ht0 :0 < t0hLengthNonneg :∀ n Pifcls n Pthen0 ≤ expectedLength n Phpoint :∀ n Pifcls n PthenexpectedLength n P ≤ C0 * min 1 (strength n P ^ (-1 / 2 : ℝ))conclusionclassFrontierRisk cls strength expectedLength t0 ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ))Proof (Lean source)
theorem classFrontierRisk_le {Model : Type*} (cls : ℕ → Model → Prop) (strength expectedLength : ℕ → Model → ℝ) (C0 t0 : ℝ) (hC0 : 0 ≤ C0) (ht0 : 0 < t0) (hLengthNonneg : ∀ n P, cls n P → 0 ≤ expectedLength n P) (hpoint : ∀ n P, cls n P → expectedLength n P ≤ C0 * min 1 (strength n P ^ (-1 / 2 : ℝ))) : classFrontierRisk cls strength expectedLength t0 ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) := by let bound : ℝ := C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) have hmin0 : 0 ≤ min 1 (t0 ^ (-1 / 2 : ℝ)) := le_min (by norm_num) (Real.rpow_nonneg ht0.le _) have hbound0 : 0 ≤ bound := mul_nonneg hC0 hmin0 let row : ℕ → ℝ := fun n => ⨆ P : {P : Model // cls n P ∧ t0 ≤ strength n P}, expectedLength n P have hrowUpper : ∀ n, row n ≤ bound := by intro n let I := {P : Model // cls n P ∧ t0 ≤ strength n P} cases isEmpty_or_nonempty I with | inl hEmpty => letI : IsEmpty I := hEmpty change (⨆ P : I, expectedLength n P) ≤ bound simpa using hbound0 | inr hNonempty => letI : Nonempty I := hNonempty apply ciSup_le intro P exact (hpoint n P P.2.1).trans (mul_le_mul_of_nonneg_left (inverseSqrtCap_anti ht0 P.2.2) hC0) have hrowLower : ∀ n, 0 ≤ row n := by intro n let I := {P : Model // cls n P ∧ t0 ≤ strength n P} cases isEmpty_or_nonempty I with | inl hEmpty => letI : IsEmpty I := hEmpty change 0 ≤ ⨆ P : I, expectedLength n P simp | inr hNonempty => letI : Nonempty I := hNonempty obtain ⟨P⟩ := hNonempty have hbdd : BddAbove (Set.range fun Q : I => expectedLength n Q) := by refine ⟨bound, ?_⟩ rintro y ⟨Q, rfl⟩ exact (hpoint n Q Q.2.1).trans (mul_le_mul_of_nonneg_left (inverseSqrtCap_anti ht0 Q.2.2) hC0) exact (hLengthNonneg n P P.2.1).trans (le_ciSup hbdd P) have hcob : IsCoboundedUnder (· ≤ ·) atTop row := Filter.isCoboundedUnder_le_of_le atTop hrowLower change limsup row atTop ≤ bound exact Filter.limsup_le_of_le hcob (Filter.Eventually.of_forall hrowUpper) -
coverageInfOrOne_of_nonemptytheorem — On a nonempty model class, worst-case coverage with the empty-class convention is the ordinary infimum of coverage across the class.Proof (Lean source)
theorem coverageInfOrOne_of_nonempty {ι : Sort*} [Nonempty ι] (f : ι → ℝ) : coverageInfOrOne f = ⨅ i, f i := by have hne : Nonempty ι := inferInstance simp [coverageInfOrOne, hne] -
coverageInfOrOne_of_isEmptytheorem — On an empty model class, worst-case coverage with the empty-class convention is one, expressing that the coverage requirement is vacuously satisfied.Proof (Lean source)
theorem coverageInfOrOne_of_isEmpty {ι : Sort*} [IsEmpty ι] (f : ι → ℝ) : coverageInfOrOne f = 1 := by simp [coverageInfOrOne, not_nonempty_iff.mpr inferInstance]
LeCam 2 core · 1 supporting This file proves the two-point minimax lower bound that reduces estimation risk to binary testing. ★ one_sub_tvDist_le_error_sum★ half_one_sub_tvDist_le_max_error
Le Cam Two-Point Method
This file proves the two-point minimax lower bound that reduces estimation risk to binary testing. It supplies the real-error and worst-case probability inequalities used by later minimax-risk modules.
theorem one_sub_tvDist_le_error_sum reviewed
⚑ flagged
Causalean.Stat
Le Cam two-point bound (summed form). For a measurable estimator est, if the parameter values θ₀, θ₁ are 2s-separated, then the two error probabilities sum to at least 1 − tvDist P₀ P₁.
Formal statement
hypotheses
Proof (Lean source)
theorem one_sub_tvDist_le_error_sum {est : Ω → Θ} (hest : Measurable est)
{θ₀ θ₁ : Θ} {s : ℝ} (hsep : 2 * s ≤ dist θ₀ θ₁) :
1 - tvDist P₀ P₁
≤ P₀.real {ω | s ≤ dist (est ω) θ₀} + P₁.real {ω | s ≤ dist (est ω) θ₁} := by
set A := {ω | s ≤ dist (est ω) θ₀} with hA
set B := {ω | s ≤ dist (est ω) θ₁} with hB
have hAmeas : MeasurableSet A := measurableSet_error hest θ₀ s
have hBmeas : MeasurableSet B := measurableSet_error hest θ₁ s
-- Outside the first error region, the second error region is forced.
have hsub : Aᶜ ⊆ B := by
intro ω hω
have hlt : dist (est ω) θ₀ < s := by
simpa [hA, Set.mem_compl_iff, not_le] using hω
have htri : dist θ₀ θ₁ ≤ dist (est ω) θ₀ + dist (est ω) θ₁ := by
simpa [dist_comm] using dist_triangle θ₀ (est ω) θ₁
have : s ≤ dist (est ω) θ₁ := by nlinarith [htri, hsep, hlt]
simpa [hB] using this
-- Monotonicity transports the testing bound from `Aᶜ` to `B`.
have hmono : P₁.real Aᶜ ≤ P₁.real B :=
measureReal_mono hsub (measure_ne_top P₁ _)
have htest := one_sub_tvDist_le_test (μ := P₀) (ν := P₁) hAmeas
linarith [htest, hmono]
theorem half_one_sub_tvDist_le_max_error reviewed
⚑ flagged
Causalean.Stat
Le Cam two-point bound (max form). For a measurable estimator est, under 2s-separation of θ₀, θ₁, the worst-case error probability is at least ½ (1 − tvDist P₀ P₁).
Formal statement
hypotheses
Proof (Lean source)
theorem half_one_sub_tvDist_le_max_error {est : Ω → Θ} (hest : Measurable est)
{θ₀ θ₁ : Θ} {s : ℝ} (hsep : 2 * s ≤ dist θ₀ θ₁) :
(1 - tvDist P₀ P₁) / 2
≤ max (P₀.real {ω | s ≤ dist (est ω) θ₀}) (P₁.real {ω | s ≤ dist (est ω) θ₁}) := by
have hsum := one_sub_tvDist_le_error_sum (P₀ := P₀) (P₁ := P₁) hest hsep
have h0 : P₀.real {ω | s ≤ dist (est ω) θ₀}
≤ max (P₀.real {ω | s ≤ dist (est ω) θ₀}) (P₁.real {ω | s ≤ dist (est ω) θ₁}) :=
le_max_left _ _
have h1 : P₁.real {ω | s ≤ dist (est ω) θ₁}
≤ max (P₀.real {ω | s ≤ dist (est ω) θ₀}) (P₁.real {ω | s ≤ dist (est ω) θ₁}) :=
le_max_right _ _
linarith [hsum, h0, h1]
1 supporting declaration (lemmas, instances)
-
measurableSet_errortheorem — The error region {ω | s ≤ dist (est ω) θ} of an estimator is measurable.hypothesesconclusionMeasurableSet {ω | s ≤ dist (est ω) θ}Proof (Lean source)
theorem measurableSet_error {est : Ω → Θ} (hest : Measurable est) (θ : Θ) (s : ℝ) : MeasurableSet {ω | s ≤ dist (est ω) θ} := by have hclosed : IsClosed {x : Θ | s ≤ dist x θ} := isClosed_le continuous_const (continuous_id.dist continuous_const) exact hest hclosed.measurableSet
LeCamTwoPoint 2 core · 0 supporting This file packages two reusable finite-sample Le Cam risk lower bounds for estimating a real-valued functional from independent product observations. ★ leCam_two_point_L1_lower★ le_cam_two_point_mse
Le Cam two-point lower bound for finite L¹ risk
This file packages two reusable finite-sample Le Cam risk lower bounds for estimating a real-valued functional from independent product observations.
The L¹ theorem leCam_two_point_L1_lower converts a two-point separation
δ ≤ |θP - θQ|, an n-sample KL budget, a supplied product-KL comparison, and a
supplied product Pinsker bridge into a positive lower bound on the worst-case
Bochner L¹ risk of any measurable estimator. The MSE theorem
le_cam_two_point_mse uses the Bretagnolle-Huber testing floor instead of
Pinsker, so every finite KL budget K yields a positive constant
exp(-K) / 32 for the corresponding worst-case squared-error risk.
The private helper event_mul_measureReal_le_integral is the Markov-type
event-to-integral step used by the L¹ theorem; the MSE theorem uses the
project-wide squared-loss event bound.
theorem leCam_two_point_L1_lower reviewed
⚑ flagged
Causalean.Stat.Minimax
Le Cam two-point lower bound on finite Bochner L¹ risk. Fix a strictly positive KL budget C with C at most 1/2. Then there is a universal positive constant c₀ such that, for every sample size, every pair of single-observation laws whose scaled KL divergence is bounded by C and whose n-fold product KL divergence is controlled by a supplied product-KL comparison and a supplied product-form Pinsker bound, every pair of separated real targets, and every measurable estimator with integrable absolute loss under both n-fold product laws, the worst-case Bochner L¹ risk is at least c₀ times the target separation.
Formal statement
Proof (Lean source)
theorem leCam_two_point_L1_lower
{Ω : Type*} [MeasurableSpace Ω]
(C : ℝ) (_hC_pos : 0 < C) (hC_small : C ≤ 1 / 2) :
∃ c₀ : ℝ, 0 < c₀ ∧
∀ (n : ℕ), 1 ≤ n →
∀ (P Q : Measure Ω) [IsProbabilityMeasure P] [IsProbabilityMeasure Q]
(θP θQ δ : ℝ),
(n : ℝ) * (_root_.InformationTheory.klDiv P Q).toReal ≤ C →
(_root_.InformationTheory.klDiv
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q))).toReal
≤ (n : ℝ) * (_root_.InformationTheory.klDiv P Q).toReal →
PinskerBound
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q)) →
0 ≤ δ → δ ≤ |θP - θQ| →
∀ (T : (Fin n → Ω) → ℝ), Measurable T →
Integrable (fun ω : Fin n → Ω => |T ω - θP|)
(Measure.pi (fun _ : Fin n => P)) →
Integrable (fun ω : Fin n → Ω => |T ω - θQ|)
(Measure.pi (fun _ : Fin n => Q)) →
c₀ * δ ≤ max
(∫ ω, |T ω - θP| ∂Measure.pi (fun _ : Fin n => P))
(∫ ω, |T ω - θQ| ∂Measure.pi (fun _ : Fin n => Q)) := by
refine ⟨(1 / 8 : ℝ), by norm_num, ?_⟩
intro n _hn P Q _ _ θP θQ δ hKLbound hKLprod hPinsker hδnonneg hδsep
T hT hIntp hIntq
have hsep : 2 * (δ / 2) ≤ dist θP θQ := by
rw [Real.dist_eq]
linarith
have hprob := klForm_two_point_lower_bound_of_pinsker
(P₀ := Measure.pi (fun _ : Fin n => P))
(P₁ := Measure.pi (fun _ : Fin n => Q))
(Θ := ℝ) hPinsker hT hsep
have hklprodC :
(_root_.InformationTheory.klDiv
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q))).toReal ≤ C :=
hKLprod.trans hKLbound
have hprobLower :
(1 / 4 : ℝ) ≤
max ((Measure.pi (fun _ : Fin n => P)).real
{ω | δ / 2 ≤ dist (T ω) θP})
((Measure.pi (fun _ : Fin n => Q)).real
{ω | δ / 2 ≤ dist (T ω) θQ}) := by
have hkl_nonneg :
0 ≤ (_root_.InformationTheory.klDiv
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q))).toReal := ENNReal.toReal_nonneg
have hpre :
(1 / 4 : ℝ) ≤
(1 - sqrt
(((_root_.InformationTheory.klDiv
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q))).toReal) / 2)) / 2 := by
have hdiv_le :
((_root_.InformationTheory.klDiv
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q))).toReal) / 2 ≤
(1 / 4 : ℝ) := by
nlinarith
have hsqrt_le :
sqrt
(((_root_.InformationTheory.klDiv
(Measure.pi (fun _ : Fin n => P))
(Measure.pi (fun _ : Fin n => Q))).toReal) / 2) ≤
(1 / 2 : ℝ) := by
have h := Real.sqrt_le_sqrt hdiv_le
convert h using 1
rw [show (1 / 4 : ℝ) = (1 / 2 : ℝ) ^ 2 by norm_num]
rw [Real.sqrt_sq (by norm_num : 0 ≤ (1 / 2 : ℝ))]
nlinarith
exact hpre.trans hprob
have hmeas_p : Measurable (fun ω : Fin n → Ω => |T ω - θP|) := by
fun_prop
have hmeas_q : Measurable (fun ω : Fin n → Ω => |T ω - θQ|) := by
fun_prop
have hp_event := event_mul_measureReal_le_integral
(μ := Measure.pi (fun _ : Fin n => P))
(f := fun ω : Fin n → Ω => |T ω - θP|)
hIntp hmeas_p (s := δ / 2)
(Filter.Eventually.of_forall fun _ => abs_nonneg _)
have hq_event := event_mul_measureReal_le_integral
(μ := Measure.pi (fun _ : Fin n => Q))
(f := fun ω : Fin n → Ω => |T ω - θQ|)
hIntq hmeas_q (s := δ / 2)
(Filter.Eventually.of_forall fun _ => abs_nonneg _)
have hp_event' :
(δ / 2) *
(Measure.pi (fun _ : Fin n => P)).real
{ω | δ / 2 ≤ dist (T ω) θP} ≤
∫ ω, |T ω - θP| ∂Measure.pi (fun _ : Fin n => P) := by
simpa [Real.dist_eq] using hp_event
have hq_event' :
(δ / 2) *
(Measure.pi (fun _ : Fin n => Q)).real
{ω | δ / 2 ≤ dist (T ω) θQ} ≤
∫ ω, |T ω - θQ| ∂Measure.pi (fun _ : Fin n => Q) := by
simpa [Real.dist_eq] using hq_event
have hmax_event :
(δ / 2) *
max ((Measure.pi (fun _ : Fin n => P)).real
{ω | δ / 2 ≤ dist (T ω) θP})
((Measure.pi (fun _ : Fin n => Q)).real
{ω | δ / 2 ≤ dist (T ω) θQ}) ≤
max (∫ ω, |T ω - θP| ∂Measure.pi (fun _ : Fin n => P))
(∫ ω, |T ω - θQ| ∂Measure.pi (fun _ : Fin n => Q)) := by
by_cases hpq :
(Measure.pi (fun _ : Fin n => P)).real
{ω | δ / 2 ≤ dist (T ω) θP} ≤
(Measure.pi (fun _ : Fin n => Q)).real
{ω | δ / 2 ≤ dist (T ω) θQ}
· rw [max_eq_right hpq]
exact hq_event'.trans (le_max_right _ _)
· have hqp :
(Measure.pi (fun _ : Fin n => Q)).real
{ω | δ / 2 ≤ dist (T ω) θQ} ≤
(Measure.pi (fun _ : Fin n => P)).real
{ω | δ / 2 ≤ dist (T ω) θP} :=
le_of_not_ge hpq
rw [max_eq_left hqp]
exact hp_event'.trans (le_max_left _ _)
have hleft : (1 / 8 : ℝ) * δ ≤ (δ / 2) * (1 / 4 : ℝ) := by
ring_nf
rfl
have hmid :
(δ / 2) * (1 / 4 : ℝ) ≤
(δ / 2) *
max ((Measure.pi (fun _ : Fin n => P)).real
{ω | δ / 2 ≤ dist (T ω) θP})
((Measure.pi (fun _ : Fin n => Q)).real
{ω | δ / 2 ≤ dist (T ω) θQ}) :=
mul_le_mul_of_nonneg_left hprobLower (by linarith)
exact hleft.trans (hmid.trans hmax_event)
lemma le_cam_two_point_mse reviewed
⚑ flagged
Causalean.Stat.Minimax
Le Cam two-point reduction in mean-squared-error form, uniform over a finite KL budget. For every Kullback–Leibler budget K, there is a single positive constant c_K (here exp(−K)/32), chosen before the laws, such that for every pair of probability laws Q₀, Q₁ whose divergence obeys KL(Q₀, Q₁) ≤ K, any measurable estimator with integrable squared loss under both laws has worst-case mean-squared error at least c_K times the squared separation of the two candidate parameter values. The finite-budget hypothesis is encoded as the ℝ≥0∞ inequality klDiv Q₀ Q₁ ≤ ENNReal.ofReal K, which forces a finite divergence (so it cannot be met vacuously by an infinite divergence collapsing under .toReal) and pins c_K to K alone.
Formal statement
hypotheses
K :
ℝ
conclusion
∃ cK : ℝ,
0 < cK
conclusion 1
S :
Type*
Q0 Q1 :
Measure S
theta0 theta1 :
ℝ
T :
S → ℝ
Integrable (fun s => (T s - theta0) ^ 2) Q0
Integrable (fun s => (T s - theta1) ^ 2) Q1
cK * (theta1 - theta0) ^ 2
≤ max (∫ s, (T s - theta0) ^ 2 ∂Q0) (∫ s, (T s - theta1) ^ 2 ∂Q1)
Proof (Lean source)
lemma le_cam_two_point_mse (K : ℝ) :
∃ cK : ℝ, 0 < cK ∧
∀ {S : Type*} [MeasurableSpace S]
(Q0 Q1 : Measure S) [IsProbabilityMeasure Q0] [IsProbabilityMeasure Q1]
(theta0 theta1 : ℝ),
InformationTheory.klDiv Q0 Q1 ≤ ENNReal.ofReal K →
∀ T : S → ℝ, Measurable T →
Integrable (fun s => (T s - theta0) ^ 2) Q0 →
Integrable (fun s => (T s - theta1) ^ 2) Q1 →
cK * (theta1 - theta0) ^ 2
≤ max (∫ s, (T s - theta0) ^ 2 ∂Q0) (∫ s, (T s - theta1) ^ 2 ∂Q1) := by
refine ⟨Real.exp (-(ENNReal.ofReal K).toReal) / 32, ?_, ?_⟩
· positivity
intro S _ Q0 Q1 _ _ theta0 theta1 hKL T hT hInt0 hInt1
let r : ℝ := |theta1 - theta0| / 2
have hr_nonneg : 0 ≤ r := by
dsimp [r]
positivity
have hsep : 2 * r ≤ |theta0 - theta1| := by
dsimp [r]
rw [abs_sub_comm]
linarith [abs_nonneg (theta1 - theta0)]
have hsep_dist : 2 * r ≤ dist theta0 theta1 := by
rwa [Real.dist_eq]
have hprob_dist := half_one_sub_tvDist_le_max_error
(P₀ := Q0) (P₁ := Q1) (Θ := ℝ) hT hsep_dist
have hprob :
(1 - tvDist Q0 Q1) / 2
≤ max (Q0.real {s | r ≤ |T s - theta0|})
(Q1.real {s | r ≤ |T s - theta1|}) := by
simpa only [Real.dist_eq] using hprob_dist
have hkl_toReal_le :
(InformationTheory.klDiv Q0 Q1).toReal ≤ (ENNReal.ofReal K).toReal :=
ENNReal.toReal_mono ENNReal.ofReal_ne_top hKL
have hexp_budget :
exp (-(ENNReal.ofReal K).toReal)
≤ exp (-(InformationTheory.klDiv Q0 Q1).toReal) := by
exact Real.exp_le_exp.mpr (by linarith)
-- The finite-budget hypothesis pins down both BH side-conditions.
have hfin : InformationTheory.klDiv Q0 Q1 ≠ ⊤ :=
ne_top_of_le_ne_top ENNReal.ofReal_ne_top hKL
have hac : Q0 ≪ Q1 := (InformationTheory.klDiv_ne_top_iff.mp hfin).1
have hBH :=
Causalean.Stat.bretagnolle_huber_affinity Q0 Q1 hac hfin
have hprob_floor :
exp (-(ENNReal.ofReal K).toReal) / 4
≤ max (Q0.real {s | r ≤ |T s - theta0|})
(Q1.real {s | r ≤ |T s - theta1|}) := by
calc
exp (-(ENNReal.ofReal K).toReal) / 4
≤ ((1 / 2 : ℝ) * exp (-(InformationTheory.klDiv Q0 Q1).toReal)) / 2 := by
nlinarith [hexp_budget, Real.exp_pos (-(ENNReal.ofReal K).toReal),
Real.exp_pos (-(InformationTheory.klDiv Q0 Q1).toReal)]
_ ≤ (1 - tvDist Q0 Q1) / 2 := by
nlinarith [hBH]
_ ≤ max (Q0.real {s | r ≤ |T s - theta0|})
(Q1.real {s | r ≤ |T s - theta1|}) := hprob
have hset0 :
{s : S | r ≤ |T s - theta0|} =
{s : S | r ^ 2 ≤ (T s - theta0) ^ 2} := by
ext s
simp only [Set.mem_setOf_eq]
constructor <;> intro hs <;>
nlinarith [hr_nonneg, abs_nonneg (T s - theta0), sq_abs (T s - theta0),
sq_nonneg (T s - theta0)]
have hset1 :
{s : S | r ≤ |T s - theta1|} =
{s : S | r ^ 2 ≤ (T s - theta1) ^ 2} := by
ext s
simp only [Set.mem_setOf_eq]
constructor <;> intro hs <;>
nlinarith [hr_nonneg, abs_nonneg (T s - theta1), sq_abs (T s - theta1),
sq_nonneg (T s - theta1)]
have hmse0 :
r ^ 2 * Q0.real {s | r ≤ |T s - theta0|}
≤ ∫ s, (T s - theta0) ^ 2 ∂Q0 := by
rw [hset0]
exact mul_meas_ge_le_integral_of_nonneg
(μ := Q0) (f := fun s => (T s - theta0) ^ 2)
(Filter.Eventually.of_forall fun s => sq_nonneg (T s - theta0)) hInt0 (r ^ 2)
have hmse1 :
r ^ 2 * Q1.real {s | r ≤ |T s - theta1|}
≤ ∫ s, (T s - theta1) ^ 2 ∂Q1 := by
rw [hset1]
exact mul_meas_ge_le_integral_of_nonneg
(μ := Q1) (f := fun s => (T s - theta1) ^ 2)
(Filter.Eventually.of_forall fun s => sq_nonneg (T s - theta1)) hInt1 (r ^ 2)
have hmse_max :
r ^ 2 * max (Q0.real {s | r ≤ |T s - theta0|})
(Q1.real {s | r ≤ |T s - theta1|})
≤ max (∫ s, (T s - theta0) ^ 2 ∂Q0) (∫ s, (T s - theta1) ^ 2 ∂Q1) := by
by_cases h01 :
Q0.real {s | r ≤ |T s - theta0|} ≤ Q1.real {s | r ≤ |T s - theta1|}
· rw [max_eq_right h01]
exact hmse1.trans (le_max_right _ _)
· have h10 :
Q1.real {s | r ≤ |T s - theta1|} ≤ Q0.real {s | r ≤ |T s - theta0|} :=
le_of_not_ge h01
rw [max_eq_left h10]
exact hmse0.trans (le_max_left _ _)
have hrate :
r ^ 2 * (exp (-(ENNReal.ofReal K).toReal) / 4)
≤ max (∫ s, (T s - theta0) ^ 2 ∂Q0) (∫ s, (T s - theta1) ^ 2 ∂Q1) := by
calc
r ^ 2 * (exp (-(ENNReal.ofReal K).toReal) / 4)
≤ r ^ 2 * max (Q0.real {s | r ≤ |T s - theta0|})
(Q1.real {s | r ≤ |T s - theta1|}) :=
mul_le_mul_of_nonneg_left hprob_floor (sq_nonneg r)
_ ≤ max (∫ s, (T s - theta0) ^ 2 ∂Q0)
(∫ s, (T s - theta1) ^ 2 ∂Q1) := hmse_max
have hcoef :
(exp (-(ENNReal.ofReal K).toReal) / 32) * (theta1 - theta0) ^ 2
≤ r ^ 2 * (exp (-(ENNReal.ofReal K).toReal) / 4) := by
dsimp [r]
nlinarith [Real.exp_pos (-(ENNReal.ofReal K).toReal), sq_abs (theta1 - theta0),
sq_nonneg (theta1 - theta0)]
exact hcoef.trans hrate
MarkovKernelTransport 9 core · 22 supporting This module packages the Blackwell comparison for squared loss: randomizing an experiment through a Markov kernel cannot improve the best attainable squared-error risk. ★ forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport_pi★ sqRisk_raoBlackwellStatistic_le
Squared-risk transport through Markov kernels
This module packages the Blackwell comparison for squared loss: randomizing an experiment through a Markov kernel cannot improve the best attainable squared-error risk. It constructs the Rao--Blackwell pullback of a bounded estimator, proves its risk comparison under an affine change of target, and exports the resulting minimax-hardness transport for both one observation and finite independent samples, including the empty sample.
Main results
* forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport transfers a quantified
squared-risk lower bound through a randomized experiment and an affine target change.
* finProductKernel_comp_pi identifies the image of an independent product experiment under
the coordinatewise product kernel.
* forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport_pi gives the finite-product
form of the randomized transport theorem.
def UniformlyBounded reviewed
⚑ flagged
Causalean.Stat
For an arbitrary domain and a real-valued function on that domain, the uniform boundedness property holds exactly when there exists a nonnegative real constant that bounds the function's absolute value at every input.
Definition (Lean source)
parameters
A :
Type*
f :
A → ℝ
def
UniformlyBounded f :
Prop
given by
∃ M : ℝ,
0 ≤ M
clause 1
x :
|f x| ≤ M
def kernelMean reviewed
⚑ flagged
Causalean.Stat
For an input measurable space, an output measurable space, and a Markov kernel from the input space to the output space and a real-valued function on the output space, the kernel mean assigns to each input the expectation of that function under the output distribution selected by the kernel.
Definition (Lean source)
parameters
def
kernelMean K T :
X → ℝ
given by
fun x => ∫ y, T y ∂K x
def kernelAffinePullback reviewed
⚑ flagged
Causalean.Stat
For an input measurable space, an output measurable space, and a Markov kernel from the input space to the output space, a real slope, a real offset, and a real-valued estimator on the output space, the affine kernel pullback assigns to each input the kernel mean of the estimator, minus the offset, divided by the slope.
Definition (Lean source)
parameters
def
kernelAffinePullback K a b targetEst :
X → ℝ
given by
fun x => (kernelMean K targetEst x - b) / a
def finProductKernel reviewed
⚑ flagged
Causalean.Stat
For an input measurable space, an output measurable space, a nonnegative integer number of coordinates, and a Markov kernel from the input space to the output space, the finite product kernel maps an input vector to the unique point-mass kernel at zero coordinates, and otherwise to independent kernel outputs coordinate by coordinate.
Definition (Lean source)
parameters
given by
match n with | 0
=> Kernel.deterministic (fun _ => fun i => elim0 i) measurable_const | n
+ 1
=> let eX := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => X) 0 let eY := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => Y) 0 ((K ∥ₖ finProductKernel n K).comap eX eX.measurable).map eY.symm
theorem forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport_pi reviewed
⚑ flagged
Causalean.Stat
Suppose the affine slope is nonzero, each one-coordinate target law is obtained by applying the common Markov kernel to its source law, and every measurable uniformly bounded estimator on the finite source product incurs squared risk at least a fixed level for some parameter index. Then every measurable uniformly bounded estimator on the target product incurs at least the source level multiplied by the squared slope for some parameter index, including when the sample has no coordinates.
Formal statement
hypotheses
X :
Type uX
Y :
Type uY
Iota :
Type uI
n :
ℕ
P :
Iota → Measure X
Q :
Iota → Measure Y
∀ j, IsProbabilityMeasure (P j)
K :
Kernel X Y
theta :
Iota → ℝ
a b L :
ℝ
ha :
a ≠ 0
hQ :
∀ j, Q j = K ∘ₘ P j
hsource :
∀ sourceEst : (Fin n → X) → ℝ
if
Measurable sourceEst
and
UniformlyBounded sourceEst
targetEst :
(Fin n → Y) → ℝ
Measurable targetEst
UniformlyBounded targetEst
Proof (Lean source)
theorem forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport_pi
(n : ℕ) (P : Iota → Measure X) (Q : Iota → Measure Y)
[∀ j, IsProbabilityMeasure (P j)]
(K : Kernel X Y) [IsMarkovKernel K]
(theta : Iota → ℝ) (a b L : ℝ) (ha : a ≠ 0)
(hQ : ∀ j, Q j = K ∘ₘ P j)
(hsource : ∀ sourceEst : (Fin n → X) → ℝ, Measurable sourceEst →
UniformlyBounded sourceEst →
∃ j, L ≤ sqRisk (Measure.pi (fun _ : Fin n => P j)) sourceEst (theta j)) :
∀ targetEst : (Fin n → Y) → ℝ, Measurable targetEst →
UniformlyBounded targetEst →
∃ j, a ^ 2 * L ≤ sqRisk (Measure.pi (fun _ : Fin n => Q j))
targetEst (a * theta j + b) := by
apply forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport
(P := fun j => Measure.pi (fun _ : Fin n => P j))
(Q := fun j => Measure.pi (fun _ : Fin n => Q j))
(K := finProductKernel n K) theta a b L ha
· intro j
rw [finProductKernel_comp_pi, hQ j]
· exact hsource
def cappedStatistic reviewed
⚑ flagged
Causalean.Stat
Given a statistic on finite samples, an overflow value, a fixed array, and an auxiliary count, the randomized capped statistic evaluates T on the requested prefix off overflow and equals the specified value on overflow.
Definition (Lean source)
parameters
def
cappedStatistic T zOver x m :
ℝ
given by
if h : m ≤ n then T (prefixOfLE x m h) else zOver
def auxPoissonCountKernel reviewed
⚑ flagged
Causalean.Stat
Given a Poisson mean, the auxiliary-count Markov kernel sends each fixed array to that same array paired with an independent Poisson count.
Definition (Lean source)
parameters
X :
Type u_1
n :
ℕ
lambda :
ℝ≥0
given by
def raoBlackwellStatistic reviewed
⚑ flagged
Causalean.Stat
Given a Poisson mean, a statistic, and an overflow value, the fixed-sample Rao--Blackwell statistic is the mean of the capped statistic under the auxiliary-count kernel.
Definition (Lean source)
parameters
def
given by
theorem sqRisk_raoBlackwellStatistic_le reviewed
⚑ flagged
Causalean.Stat
For an observation probability law, a Poisson mean, and a fixed sample size, if the statistic is measurable, the interval is ordered, the statistic stays in the interval, the target is in the interval, and the overflow value is in the interval, then the fixed-iid squared risk of the Rao--Blackwell statistic is at most the uncapped finite-Poisson risk plus the squared interval diameter times the Poisson overflow probability.
Formal statement
hypotheses
X :
Type u_1
P :
lambda :
ℝ≥0
n :
ℕ
T :
FiniteSample X → ℝ
hT :
a b theta zOver :
ℝ
hab :
a ≤ b
hTmem :
∀ s, T s ∈ Icc a b
htheta :
theta ∈ Icc a b
hzOver :
zOver ∈ Icc a b
conclusion
≤ sqRisk (finitePoissonSampleLaw P lambda) T theta
Proof (Lean source)
theorem sqRisk_raoBlackwellStatistic_le
(P : Measure X) [IsProbabilityMeasure P] (lambda : ℝ≥0) (n : ℕ)
{T : FiniteSample X → ℝ} (hT : Measurable T)
{a b theta zOver : ℝ} (hab : a ≤ b)
(hTmem : ∀ s, T s ∈ Icc a b) (htheta : theta ∈ Icc a b)
(hzOver : zOver ∈ Icc a b) :
sqRisk (Measure.pi (fun _ : Fin n ↦ P))
(raoBlackwellStatistic lambda T zOver) theta ≤
sqRisk (finitePoissonSampleLaw P lambda) T theta +
(b - a) ^ 2 * (poissonMeasure lambda).real (Ioi n) := by
-- Apply `sqRisk_kernelMean_le_comp` to the bounded capped statistic, rewrite
-- the composed law with `auxPoissonCountKernel_comp_pi`, and split its risk
-- integral over `m ≤ n` and `m > n`. Rewrite the first restriction with
-- `map_totalizedPrefix_restrict_nonoverflow`; bound the overflow loss by
-- the squared interval diameter before integrating its indicator.
have hTb : UniformlyBounded T := by
refine ⟨max |a| |b|, ?_, ?_⟩
· exact (abs_nonneg a).trans (le_max_left _ _)
· intro s
exact abs_le_max_abs_abs (hTmem s).1 (hTmem s).2
have hCappedBound : UniformlyBounded
(fun z : (Fin n → X) × ℕ ↦ cappedStatistic T zOver z.1 z.2) := by
refine ⟨max |a| |b|, ?_, ?_⟩
· exact (abs_nonneg a).trans (le_max_left _ _)
· intro z
by_cases h : z.2 ≤ n
· simp only [cappedStatistic, h, ↓reduceDIte]
exact abs_le_max_abs_abs (hTmem _).1 (hTmem _).2
· simp only [cappedStatistic, h, ↓reduceDIte]
exact abs_le_max_abs_abs hzOver.1 hzOver.2
let μ : Measure ((Fin n → X) × ℕ) :=
(Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)
let s : Set ((Fin n → X) × ℕ) := Prod.snd ⁻¹' Set.Iic n
have hs : MeasurableSet s := measurable_snd measurableSet_Iic
obtain ⟨M, hM, hCappedBoundM⟩ := hCappedBound
have hlossInt : Integrable
(fun z : (Fin n → X) × ℕ =>
(cappedStatistic T zOver z.1 z.2 - theta) ^ 2) μ := by
refine (integrable_const ((M + |theta|) ^ 2)).mono'
(((measurable_cappedStatistic hT zOver).sub measurable_const).pow_const 2
).aestronglyMeasurable ?_
filter_upwards [] with z
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _), sq_le_sq,
abs_of_nonneg (add_nonneg hM (abs_nonneg theta))]
exact (abs_sub _ _).trans (add_le_add (hCappedBoundM z) le_rfl)
have hsplit :
sqRisk μ (fun z : (Fin n → X) × ℕ ↦
cappedStatistic T zOver z.1 z.2) theta =
sqRisk (μ.restrict s) (fun z : (Fin n → X) × ℕ ↦
cappedStatistic T zOver z.1 z.2) theta +
sqRisk (μ.restrict sᶜ) (fun z : (Fin n → X) × ℕ ↦
cappedStatistic T zOver z.1 z.2) theta := by
unfold sqRisk
conv_lhs => rw [← Measure.restrict_add_restrict_compl (μ := μ) hs,
integral_add_measure hlossInt.restrict hlossInt.restrict]
have hsc : sᶜ = Prod.snd ⁻¹' Ioi n := by
rw [← Set.preimage_compl, Set.compl_Iic]
have hoverflowSq : (zOver - theta) ^ 2 ≤ (b - a) ^ 2 := by
have hdist : |zOver - theta| ≤ b - a := by
simpa [Real.dist_eq] using Real.dist_le_of_mem_Icc hzOver htheta
rw [← sq_abs (zOver - theta)]
exact (sq_le_sq₀ (abs_nonneg _) (sub_nonneg.mpr hab)).2 hdist
have hoverflowProb : 0 ≤ (poissonMeasure lambda).real (Ioi n) := by
positivity
calc
sqRisk (Measure.pi (fun _ : Fin n ↦ P))
(raoBlackwellStatistic lambda T zOver) theta ≤
sqRisk (auxPoissonCountKernel lambda ∘ₘ
Measure.pi (fun _ : Fin n ↦ P))
(fun z : (Fin n → X) × ℕ ↦ cappedStatistic T zOver z.1 z.2) theta := by
exact sqRisk_kernelMean_le_comp
(Measure.pi (fun _ : Fin n ↦ P)) (auxPoissonCountKernel lambda)
(measurable_cappedStatistic hT zOver)
⟨M, hM, hCappedBoundM⟩ theta
_ = sqRisk μ (fun z : (Fin n → X) × ℕ ↦
cappedStatistic T zOver z.1 z.2) theta := by
rw [auxPoissonCountKernel_comp_pi]
_ = sqRisk (μ.restrict s) (fun z : (Fin n → X) × ℕ ↦
cappedStatistic T zOver z.1 z.2) theta +
sqRisk (μ.restrict sᶜ) (fun z : (Fin n → X) × ℕ ↦
cappedStatistic T zOver z.1 z.2) theta := hsplit
_ = sqRisk
((finitePoissonSampleLaw P lambda).restrict
(FiniteSample.count ⁻¹' Set.Iic n)) T theta +
(zOver - theta) ^ 2 * (poissonMeasure lambda).real (Ioi n) := by
rw [show μ.restrict s =
((Measure.pi (fun _ : Fin n ↦ P)).prod
(poissonMeasure lambda)).restrict (Prod.snd ⁻¹' Set.Iic n) from rfl,
sqRisk_cappedStatistic_restrict_nonoverflow_eq P lambda n hT zOver theta,
hsc]
exact congrArg
(fun r => sqRisk
((finitePoissonSampleLaw P lambda).restrict
(FiniteSample.count ⁻¹' Set.Iic n)) T theta + r)
(sqRisk_cappedStatistic_restrict_overflow_eq
P lambda n T zOver theta)
_ ≤ sqRisk (finitePoissonSampleLaw P lambda) T theta +
(b - a) ^ 2 * (poissonMeasure lambda).real (Ioi n) :=
add_le_add
(sqRisk_finitePoisson_restrict_nonoverflow_le
P lambda n hT hTb theta)
(mul_le_mul_of_nonneg_right hoverflowSq hoverflowProb)
22 supporting declarations (lemmas, instances)
-
measurable_kernelMeantheorem — Measurability of a real-valued estimator ensures that its expectation under each kernel output distribution varies measurably with the kernel input.hypothesesconclusionMeasurable (kernelMean K T)Proof (Lean source)
@[fun_prop] theorem measurable_kernelMean (K : Kernel X Y) {T : Y → ℝ} (hT : Measurable T) : Measurable (kernelMean K T) := by exact hT.stronglyMeasurable.integral_kernel.measurable -
abs_kernelMean_letheorem — If a proposed bound is nonnegative and bounds the estimator in absolute value at every output, then the kernel mean obeys the same absolute bound at every input when each kernel output is a probability distribution.hypothesesX :sharedType uXY :sharedType uYK :Kernel X YT :Y → ℝM :ℝhM :0 ≤ MhT :∀ y, |T y| ≤ Mx :conclusion|kernelMean K T x| ≤ MProof (Lean source)
theorem abs_kernelMean_le (K : Kernel X Y) [IsMarkovKernel K] {T : Y → ℝ} {M : ℝ} (hM : 0 ≤ M) (hT : ∀ y, |T y| ≤ M) : ∀ x, |kernelMean K T x| ≤ M := by intro x haveI : IsProbabilityMeasure (K x) := inferInstance simpa [kernelMean, Real.norm_eq_abs] using (norm_integral_le_of_norm_le_const (μ := K x) (f := T) (C := M) (Filter.Eventually.of_forall hT)) -
uniformlyBounded_kernelMeantheorem — Uniform boundedness of an estimator implies that averaging it against a Markov kernel is uniformly bounded by the same witness.hypothesesconclusionUniformlyBounded (kernelMean K T)Proof (Lean source)
theorem uniformlyBounded_kernelMean (K : Kernel X Y) [IsMarkovKernel K] {T : Y → ℝ} (hT : UniformlyBounded T) : UniformlyBounded (kernelMean K T) := by obtain ⟨M, hM, hT⟩ := hT exact ⟨M, hM, abs_kernelMean_le K hM hT⟩ -
measurable_kernelAffinePullbacktheorem — Measurability of a target estimator ensures that its affine kernel pullback is measurable on the source experiment.hypothesesX :sharedType uXY :sharedType uYK :Kernel X Ya b :ℝtargetEst :Y → ℝhtarget :Measurable targetEstconclusionMeasurable (kernelAffinePullback K a b targetEst)Proof (Lean source)
@[fun_prop] theorem measurable_kernelAffinePullback (K : Kernel X Y) {a b : ℝ} {targetEst : Y → ℝ} (htarget : Measurable targetEst) : Measurable (kernelAffinePullback K a b targetEst) := by exact ((measurable_kernelMean K htarget).sub measurable_const).div measurable_const -
uniformlyBounded_kernelAffinePullbacktheorem — If the affine slope is nonzero and the target estimator is uniformly bounded, then the affine kernel pullback is uniformly bounded on the source experiment.hypothesesX :sharedType uXY :sharedType uYK :Kernel X Ya b :ℝha :a ≠ 0targetEst :Y → ℝhtarget :UniformlyBounded targetEstconclusionUniformlyBounded (kernelAffinePullback K a b targetEst)Proof (Lean source)
theorem uniformlyBounded_kernelAffinePullback (K : Kernel X Y) [IsMarkovKernel K] {a b : ℝ} (ha : a ≠ 0) {targetEst : Y → ℝ} (htarget : UniformlyBounded targetEst) : UniformlyBounded (kernelAffinePullback K a b targetEst) := by obtain ⟨M, hM, htarget⟩ := htarget refine ⟨(M + |b|) / |a|, div_nonneg (add_nonneg hM (abs_nonneg b)) (abs_nonneg a), ?_⟩ intro x rw [kernelAffinePullback, abs_div] exact div_le_div_of_nonneg_right ((abs_sub _ _).trans (add_le_add (abs_kernelMean_le K hM htarget x) le_rfl)) (abs_nonneg a) -
integral_kernelMean_eq_integral_comptheorem — If the target estimator is measurable and uniformly bounded, then its expectation after taking the kernel mean under a source probability law equals its expectation under the garbled law.hypothesesconclusion∫ x, kernelMean K T x ∂P = ∫ y, T y ∂(K ∘ₘ P)Proof (Lean source)
theorem integral_kernelMean_eq_integral_comp (P : Measure X) [IsProbabilityMeasure P] (K : Kernel X Y) [IsMarkovKernel K] {T : Y → ℝ} (hTmeas : Measurable T) (hTbound : UniformlyBounded T) : ∫ x, kernelMean K T x ∂P = ∫ y, T y ∂(K ∘ₘ P) := by obtain ⟨M, _hM, hT⟩ := hTbound have hTint : Integrable T (K ∘ₘ P) := (integrable_const M).mono' hTmeas.aestronglyMeasurable (Filter.Eventually.of_forall fun y => by simpa [Real.norm_eq_abs] using hT y) simpa [kernelMean] using (integral_bind K.measurable hTint).symm -
sqLoss_kernelMean_letheorem — If the target estimator is measurable and uniformly bounded, then the squared error of its kernel mean is no greater than the kernel average of its squared error at each source observation.hypothesesconclusion(kernelMean K T x - c) ^ 2 ≤ ∫ y, (T y - c) ^ 2 ∂K xProof (Lean source)
theorem sqLoss_kernelMean_le (K : Kernel X Y) [IsMarkovKernel K] {T : Y → ℝ} (hTmeas : Measurable T) (hTbound : UniformlyBounded T) (c : ℝ) (x : X) : (kernelMean K T x - c) ^ 2 ≤ ∫ y, (T y - c) ^ 2 ∂K x := by haveI : IsProbabilityMeasure (K x) := inferInstance obtain ⟨M, hM, hT⟩ := hTbound have hTint : Integrable T (K x) := (integrable_const M).mono' hTmeas.aestronglyMeasurable (Filter.Eventually.of_forall fun y => by simpa [Real.norm_eq_abs] using hT y) have hdiffint : Integrable (fun y => T y - c) (K x) := hTint.sub (integrable_const c) have hlossint : Integrable (fun y => (T y - c) ^ 2) (K x) := by refine (integrable_const ((M + |c|) ^ 2)).mono' ((hTmeas.sub measurable_const).pow_const 2).aestronglyMeasurable ?_ filter_upwards [] with y rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _), sq_le_sq, abs_of_nonneg (add_nonneg hM (abs_nonneg c))] exact (abs_sub _ _).trans (add_le_add (hT y) le_rfl) have hjensen := ((show Even 2 by norm_num).convexOn_pow : ConvexOn ℝ univ (fun z : ℝ => z ^ 2)).map_integral_le (continuousOn_pow 2) isClosed_univ (Filter.Eventually.of_forall fun _ => Set.mem_univ _) hdiffint hlossint simpa [kernelMean, integral_sub hTint (integrable_const c), integral_const] using hjensen -
sqRisk_kernelMean_le_comptheorem — If the target estimator is measurable and uniformly bounded, then Rao--Blackwellizing it through a Markov kernel cannot increase squared risk: the source-law risk of the kernel mean is at most the target-law risk after garbling.hypothesesconclusionProof (Lean source)
theorem sqRisk_kernelMean_le_comp (P : Measure X) [IsProbabilityMeasure P] (K : Kernel X Y) [IsMarkovKernel K] {T : Y → ℝ} (hTmeas : Measurable T) (hTbound : UniformlyBounded T) (c : ℝ) : sqRisk P (kernelMean K T) c ≤ sqRisk (K ∘ₘ P) T c := by obtain ⟨M, hM, hT⟩ := hTbound have hlossMeas : Measurable (fun y => (T y - c) ^ 2) := (hTmeas.sub measurable_const).pow_const 2 have hlossBound : UniformlyBounded (fun y => (T y - c) ^ 2) := by refine ⟨(M + |c|) ^ 2, sq_nonneg _, ?_⟩ intro y rw [abs_of_nonneg (sq_nonneg _), sq_le_sq, abs_of_nonneg (add_nonneg hM (abs_nonneg c))] exact (abs_sub _ _).trans (add_le_add (hT y) le_rfl) obtain ⟨N, _hN, hmeanLoss⟩ := uniformlyBounded_kernelMean K hlossBound have hmeanLossInt : Integrable (kernelMean K fun y => (T y - c) ^ 2) P := (integrable_const N).mono' (measurable_kernelMean K hlossMeas).aestronglyMeasurable (Filter.Eventually.of_forall fun x => by simpa [Real.norm_eq_abs] using hmeanLoss x) unfold sqRisk calc (∫ x, (kernelMean K T x - c) ^ 2 ∂P) ≤ ∫ x, kernelMean K (fun y => (T y - c) ^ 2) x ∂P := integral_mono_of_nonneg (Filter.Eventually.of_forall fun _ => sq_nonneg _) hmeanLossInt (Filter.Eventually.of_forall fun x => sqLoss_kernelMean_le K hTmeas ⟨M, hM, hT⟩ c x) _ = ∫ y, (T y - c) ^ 2 ∂(K ∘ₘ P) := integral_kernelMean_eq_integral_comp P K hlossMeas hlossBound -
sqRisk_kernelAffinePullback_le_comptheorem — If the affine slope is nonzero, the target estimator is measurable, and the target estimator is uniformly bounded, then the source risk of the affine kernel pullback, multiplied by the squared slope, is at most the target risk under the garbled law.hypothesesX :sharedType uXY :sharedType uYP :K :Kernel X Ya b theta :ℝha :a ≠ 0targetEst :Y → ℝhtargetMeas :Measurable targetEsthtargetBound :UniformlyBounded targetEstconclusiona ^ 2 * sqRisk P (kernelAffinePullback K a b targetEst) theta≤ sqRisk (K ∘ₘ P) targetEst (a * theta + b)Proof (Lean source)
theorem sqRisk_kernelAffinePullback_le_comp (P : Measure X) [IsProbabilityMeasure P] (K : Kernel X Y) [IsMarkovKernel K] {a b theta : ℝ} (ha : a ≠ 0) {targetEst : Y → ℝ} (htargetMeas : Measurable targetEst) (htargetBound : UniformlyBounded targetEst) : a ^ 2 * sqRisk P (kernelAffinePullback K a b targetEst) theta ≤ sqRisk (K ∘ₘ P) targetEst (a * theta + b) := by have hpull : ∀ x, a ^ 2 * (kernelAffinePullback K a b targetEst x - theta) ^ 2 = (kernelMean K targetEst x - (a * theta + b)) ^ 2 := by intro x unfold kernelAffinePullback field_simp ring calc a ^ 2 * sqRisk P (kernelAffinePullback K a b targetEst) theta = ∫ x, a ^ 2 * (kernelAffinePullback K a b targetEst x - theta) ^ 2 ∂P := by exact (integral_const_mul (a ^ 2) _).symm _ = sqRisk P (kernelMean K targetEst) (a * theta + b) := by unfold sqRisk exact integral_congr_ae (Filter.Eventually.of_forall hpull) _ ≤ sqRisk (K ∘ₘ P) targetEst (a * theta + b) := sqRisk_kernelMean_le_comp P K htargetMeas htargetBound _ -
forall_estimator_exists_sqRisk_ge_of_kernel_affine_transporttheorem — Suppose the affine slope is nonzero, each target law is obtained by passing its source law through the common Markov kernel, and every measurable uniformly bounded source estimator incurs squared risk at least a fixed level for some parameter index. Then every measurable uniformly bounded target estimator incurs at least the source level multiplied by the squared slope for some parameter index, with the target parameter transformed by the same affine map.hypothesesX :sharedType uXY :sharedType uYIota :sharedType uIP :Iota → Measure XQ :Iota → Measure Y∀ j, IsProbabilityMeasure (P j)K :Kernel X Ytheta :Iota → ℝa b L :ℝha :a ≠ 0hQ :∀ j, Q j = K ∘ₘ P jhsource :∀ sourceEst : X → ℝtargetEst :Y → ℝMeasurable targetEstUniformlyBounded targetEstconclusion∃ j, a ^ 2 * L ≤ sqRisk (Q j) targetEst (a * theta j + b)Proof (Lean source)
theorem forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport (P : Iota → Measure X) (Q : Iota → Measure Y) [∀ j, IsProbabilityMeasure (P j)] (K : Kernel X Y) [IsMarkovKernel K] (theta : Iota → ℝ) (a b L : ℝ) (ha : a ≠ 0) (hQ : ∀ j, Q j = K ∘ₘ P j) (hsource : ∀ sourceEst : X → ℝ, Measurable sourceEst → UniformlyBounded sourceEst → ∃ j, L ≤ sqRisk (P j) sourceEst (theta j)) : ∀ targetEst : Y → ℝ, Measurable targetEst → UniformlyBounded targetEst → ∃ j, a ^ 2 * L ≤ sqRisk (Q j) targetEst (a * theta j + b) := by intro targetEst htargetMeas htargetBound obtain ⟨j, hj⟩ := hsource (kernelAffinePullback K a b targetEst) (measurable_kernelAffinePullback K htargetMeas) (uniformlyBounded_kernelAffinePullback K ha htargetBound) refine ⟨j, ?_⟩ calc a ^ 2 * L ≤ a ^ 2 * sqRisk (P j) (kernelAffinePullback K a b targetEst) (theta j) := mul_le_mul_of_nonneg_left hj (sq_nonneg a) _ ≤ sqRisk (K ∘ₘ P j) targetEst (a * theta j + b) := sqRisk_kernelAffinePullback_le_comp (P j) K ha htargetMeas htargetBound _ = sqRisk (Q j) targetEst (a * theta j + b) := by rw [hQ j] -
finProductKernel_applytheorem — Each fibre of the finite product kernel equals the independent product of its coordinate output laws, including the unique empty product fibre.hypothesesconclusionfinProductKernel n K x = Measure.pi (fun i : Fin n => K (x i))Proof (Lean source)
theorem finProductKernel_apply (n : ℕ) (K : Kernel X Y) [IsMarkovKernel K] (x : Fin n → X) : finProductKernel n K x = Measure.pi (fun i : Fin n => K (x i)) := by induction n with | zero => rw [finProductKernel, Kernel.deterministic_apply] exact (Measure.pi_of_empty _ _).symm | succ n ih => let _ : IsMarkovKernel (finProductKernel n K) := ⟨fun z => ⟨by rw [ih z]; exact measure_univ⟩⟩ let eX := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => X) 0 let eY := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => Y) 0 rw [finProductKernel, Kernel.map_apply _ eY.symm.measurable, Kernel.comap_apply, Kernel.parallelComp_apply, ih] exact (MeasureTheory.measurePreserving_piFinSuccAbove (fun i : Fin (n + 1) => K (x i)) 0).symm.map_eq -
instIsMarkovKernelFinProductKernelinstance — For every nonnegative integer number of coordinates and every Markov kernel between measurable input and output spaces, the independent product kernel over that many coordinates is a Markov kernel, including when there are zero coordinates. -
finProductKernel_comp_pitheorem — Passing an independent finite product law through the coordinatewise product kernel produces the product of the one-coordinate garbled law, including when there are no coordinates.hypothesesconclusionfinProductKernel n K ∘ₘ Measure.pi (fun _ : Fin n => P)= Measure.pi (fun _ : Fin n => K ∘ₘ P)Proof (Lean source)
theorem finProductKernel_comp_pi (n : ℕ) (P : Measure X) [IsProbabilityMeasure P] (K : Kernel X Y) [IsMarkovKernel K] : finProductKernel n K ∘ₘ Measure.pi (fun _ : Fin n => P) = Measure.pi (fun _ : Fin n => K ∘ₘ P) := by induction n with | zero => rw [finProductKernel, Measure.deterministic_comp_eq_map, Measure.pi_of_empty, Measure.map_dirac] exact (Measure.pi_of_empty _ _).symm | succ n ih => let eX := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => X) 0 let eY := MeasurableEquiv.piFinSuccAbove (fun _ : Fin (n + 1) => Y) 0 have hsource : (Measure.pi (fun _ : Fin (n + 1) => P)).map eX = P.prod (Measure.pi (fun _ : Fin n => P)) := (MeasureTheory.measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => P) 0).map_eq have hparallel : (K ∥ₖ finProductKernel n K) ∘ₘ (P.prod (Measure.pi (fun _ : Fin n => P))) = (K ∘ₘ P).prod (finProductKernel n K ∘ₘ Measure.pi (fun _ : Fin n => P)) := by calc (K ∥ₖ finProductKernel n K) ∘ₘ (P.prod (Measure.pi (fun _ : Fin n => P))) = (K ∥ₖ Kernel.id) ∘ₘ ((Kernel.id ∥ₖ finProductKernel n K) ∘ₘ (P.prod (Measure.pi (fun _ : Fin n => P)))) := by rw [Measure.comp_assoc, Kernel.parallelComp_comp_parallelComp, Kernel.comp_id, Kernel.id_comp] _ = (K ∥ₖ Kernel.id) ∘ₘ (P.prod (finProductKernel n K ∘ₘ Measure.pi (fun _ : Fin n => P))) := by rw [← Measure.prod_comp_right] _ = (K ∘ₘ P).prod (finProductKernel n K ∘ₘ Measure.pi (fun _ : Fin n => P)) := by rw [Measure.prod_comp_left] calc finProductKernel (n + 1) K ∘ₘ Measure.pi (fun _ : Fin (n + 1) => P) = (((K ∥ₖ finProductKernel n K) ∘ₘ (Measure.pi (fun _ : Fin (n + 1) => P)).map eX).map eY.symm) := by rw [finProductKernel, ← Kernel.deterministic_comp_eq_map eY.symm.measurable, ← Measure.comp_assoc, ← Kernel.comp_deterministic_eq_comap, ← Measure.comp_assoc, Measure.deterministic_comp_eq_map, Measure.deterministic_comp_eq_map] _ = ((K ∘ₘ P).prod (Measure.pi (fun _ : Fin n => K ∘ₘ P))).map eY.symm := by rw [hsource, hparallel, ih] _ = Measure.pi (fun _ : Fin (n + 1) => K ∘ₘ P) := (MeasureTheory.measurePreserving_piFinSuccAbove (fun _ : Fin (n + 1) => K ∘ₘ P) 0).symm.map_eq -
sqRisk_finProductKernel_affinePullback_letheorem — If the affine slope is nonzero, a finite-sample target estimator is measurable, and that estimator is uniformly bounded, then the source-product risk of its affine product-kernel pullback, multiplied by the squared slope, is at most its target-product risk.hypothesesX :sharedType uXY :sharedType uYn :ℕP :K :Kernel X Ya b theta :ℝha :a ≠ 0targetEst :(Fin n → Y) → ℝhtargetMeas :Measurable targetEsthtargetBound :UniformlyBounded targetEstconclusiona ^ 2 * sqRisk (Measure.pi (fun _ : Fin n => P)) (kernelAffinePullback (finProductKernel n K) a b targetEst) thetaProof (Lean source)
theorem sqRisk_finProductKernel_affinePullback_le (n : ℕ) (P : Measure X) [IsProbabilityMeasure P] (K : Kernel X Y) [IsMarkovKernel K] {a b theta : ℝ} (ha : a ≠ 0) {targetEst : (Fin n → Y) → ℝ} (htargetMeas : Measurable targetEst) (htargetBound : UniformlyBounded targetEst) : a ^ 2 * sqRisk (Measure.pi (fun _ : Fin n => P)) (kernelAffinePullback (finProductKernel n K) a b targetEst) theta ≤ sqRisk (Measure.pi (fun _ : Fin n => K ∘ₘ P)) targetEst (a * theta + b) := by calc a ^ 2 * sqRisk (Measure.pi (fun _ : Fin n => P)) (kernelAffinePullback (finProductKernel n K) a b targetEst) theta ≤ sqRisk (finProductKernel n K ∘ₘ Measure.pi (fun _ : Fin n => P)) targetEst (a * theta + b) := sqRisk_kernelAffinePullback_le_comp (Measure.pi (fun _ : Fin n => P)) (finProductKernel n K) ha htargetMeas htargetBound _ = sqRisk (Measure.pi (fun _ : Fin n => K ∘ₘ P)) targetEst (a * theta + b) := by rw [finProductKernel_comp_pi] -
measurable_cappedStatistictheorem — If the finite-sample statistic is measurable, then the capped statistic is jointly measurable in the fixed array and auxiliary count.hypothesesconclusionProof (Lean source)
@[fun_prop] theorem measurable_cappedStatistic {n : ℕ} {T : FiniteSample X → ℝ} (hT : Measurable T) (zOver : ℝ) : Measurable (fun z : (Fin n → X) × ℕ ↦ cappedStatistic T zOver z.1 z.2) := by apply measurable_from_prod_countable_left intro m unfold cappedStatistic change Measurable (fun x : Fin n → X ↦ if h : m ≤ n then T (prefixOfLE x m h) else zOver) split · exact hT.comp (measurable_prefixOfLE (X := X) ‹m ≤ n›) · exact measurable_const -
isMarkovKernelinstance — The auxiliary-count kernel is a Markov kernel.parametersX :sharedType u_1n :ℕlambda :ℝ≥0instancegiven by -
auxPoissonCountKernel_comp_pitheorem — For an observation probability law, a Poisson mean, and a fixed sample size, mixing the auxiliary-count kernel over the fixed iid sample law gives exactly the product of that iid law and the Poisson count law.hypothesesconclusionauxPoissonCountKernel (X := X) (n := n) lambda ∘ₘ Measure.pi (fun _ : Fin n ↦ P)Proof (Lean source)
theorem auxPoissonCountKernel_comp_pi (P : Measure X) [IsProbabilityMeasure P] (lambda : ℝ≥0) (n : ℕ) : auxPoissonCountKernel (X := X) (n := n) lambda ∘ₘ Measure.pi (fun _ : Fin n ↦ P) = (Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda) := by -- Use `map_graph_prod_eq_compProd` with the identity mechanism, then take -- the second marginal (`Measure.snd_compProd`). have hgraph : Measure.map (fun p : (Fin n → X) × ℕ => (p.1, p)) ((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)) = (Measure.pi (fun _ : Fin n ↦ P)).compProd (auxPoissonCountKernel lambda) := by simpa [auxPoissonCountKernel] using (map_graph_prod_eq_compProd (Measure.pi (fun _ : Fin n ↦ P)) (poissonMeasure lambda) (show Measurable (fun z : (Fin n → X) × ℕ ↦ z) from measurable_id)) have hsnd := congrArg (Measure.map snd) hgraph rw [Measure.map_map measurable_snd (show Measurable (fun p : (Fin n → X) × ℕ => (p.1, p)) from measurable_fst.prodMk measurable_id)] at hsnd change _ = ((Measure.pi (fun _ : Fin n ↦ P)) ⊗ₘ auxPoissonCountKernel lambda).snd at hsnd rw [Measure.snd_compProd] at hsnd simpa [Function.comp_def] using hsnd.symm -
raoBlackwellStatistic_eq_integraltheorem — Given a Poisson mean, a finite-sample statistic, its measurability, an overflow value, and a fixed array, the Rao--Blackwell statistic equals the Poisson integral of the capped prefix statistic.hypothesesconclusionraoBlackwellStatistic lambda T zOver x= ∫ m : ℕ, cappedStatistic T zOver x m ∂poissonMeasure lambdaProof (Lean source)
theorem raoBlackwellStatistic_eq_integral {n : ℕ} (lambda : ℝ≥0) (T : FiniteSample X → ℝ) (hT : Measurable T) (zOver : ℝ) (x : Fin n → X) : raoBlackwellStatistic lambda T zOver x = ∫ m : ℕ, cappedStatistic T zOver x m ∂poissonMeasure lambda := by -- Unfold the kernel mean, rewrite with `mechanismKernel_apply`, and use -- `integral_map` with `measurable_cappedStatistic hT zOver`. unfold raoBlackwellStatistic kernelMean auxPoissonCountKernel rw [mechanismKernel_apply _ (show Measurable (fun z : (Fin n → X) × ℕ ↦ z) from measurable_id) x] exact integral_map measurable_prodMk_left.aemeasurable (measurable_cappedStatistic hT zOver).aestronglyMeasurable -
sqRisk_cappedStatistic_restrict_nonoverflow_eqtheorem — For an observation probability law, a Poisson mean, a fixed sample size, a measurable finite-sample statistic, an overflow value, and a target, the squared risk of the capped statistic on the nonoverflow restriction equals the squared risk of the original statistic under the correspondingly restricted finite-Poisson law.hypothesesconclusionsqRisk (((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)).restrict (snd ⁻¹' Iic n)) (fun z ↦ cappedStatistic T zOver z.1 z.2) thetaProof (Lean source)
theorem sqRisk_cappedStatistic_restrict_nonoverflow_eq (P : Measure X) [IsProbabilityMeasure P] (lambda : ℝ≥0) (n : ℕ) {T : FiniteSample X → ℝ} (hT : Measurable T) (zOver theta : ℝ) : sqRisk (((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)).restrict (snd ⁻¹' Iic n)) (fun z ↦ cappedStatistic T zOver z.1 z.2) theta = sqRisk ((finitePoissonSampleLaw P lambda).restrict (FiniteSample.count ⁻¹' Iic n)) T theta := by -- On the restricted source, `cappedStatistic` is the squared loss pulled -- back along `totalizedPrefix`; use `integral_map` and -- `map_totalizedPrefix_restrict_nonoverflow`. let overflow : FiniteSample X := ⟨0, fun i => elim0 i⟩ unfold sqRisk rw [← map_totalizedPrefix_restrict_nonoverflow P lambda n overflow] rw [integral_map (μ := ((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)).restrict (snd ⁻¹' Iic n)) (φ := fun z : (Fin n → X) × ℕ ↦ totalizedPrefix overflow z.1 z.2) (f := fun s : FiniteSample X ↦ (T s - theta) ^ 2) (measurable_totalizedPrefix overflow).aemeasurable ((hT.sub measurable_const).pow_const 2).aestronglyMeasurable] apply integral_congr_ae filter_upwards [ae_restrict_mem (measurable_snd measurableSet_Iic)] with z hz have hzn : z.2 ≤ n := hz simp [cappedStatistic, totalizedPrefix, cappedPrefix, hzn] -
sqRisk_cappedStatistic_restrict_overflow_eqtheorem — For an observation probability law, a Poisson mean, a fixed sample size, a statistic, an overflow value, and a target, the squared risk of the capped statistic on the overflow restriction is its constant overflow loss times the Poisson overflow probability.hypothesesconclusionsqRisk (((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)).restrict (snd ⁻¹' Ioi n)) (fun z ↦ cappedStatistic T zOver z.1 z.2) thetaProof (Lean source)
theorem sqRisk_cappedStatistic_restrict_overflow_eq (P : Measure X) [IsProbabilityMeasure P] (lambda : ℝ≥0) (n : ℕ) (T : FiniteSample X → ℝ) (zOver theta : ℝ) : sqRisk (((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)).restrict (Prod.snd ⁻¹' Ioi n)) (fun z ↦ cappedStatistic T zOver z.1 z.2) theta = (zOver - theta) ^ 2 * (poissonMeasure lambda).real (Ioi n) := by -- The capped statistic is constant on `m > n`; evaluate the restricted -- integral and the product-event mass using that the iid law has mass one. unfold sqRisk have hae : ∀ᵐ z ∂ ((Measure.pi (fun _ : Fin n ↦ P)).prod (poissonMeasure lambda)).restrict (Prod.snd ⁻¹' Ioi n), (cappedStatistic T zOver z.1 z.2 - theta) ^ 2 = (zOver - theta) ^ 2 := by filter_upwards [ae_restrict_mem (measurable_snd measurableSet_Ioi)] with z hz change n < z.2 at hz simp [cappedStatistic, not_le_of_gt hz] have hset : (Prod.snd : ((Fin n → X) × ℕ) → ℕ) ⁻¹' Ioi n = (Set.univ : Set (Fin n → X)) ×ˢ Ioi n := by ext z simp rw [integral_congr_ae hae, integral_const] simp only [smul_eq_mul, measureReal_def] rw [Measure.restrict_apply_univ, hset, Measure.prod_apply (MeasurableSet.univ.prod measurableSet_Ioi)] simp [mul_comm] -
sqRisk_finitePoisson_restrict_nonoverflow_letheorem — For an observation probability law, a Poisson mean, a fixed sample size, a measurable uniformly bounded statistic, and a target, restricting the finite-Poisson law to counts at most the fixed sample size cannot increase squared risk.hypothesesconclusion≤ sqRisk (finitePoissonSampleLaw P lambda) T thetaProof (Lean source)
theorem sqRisk_finitePoisson_restrict_nonoverflow_le (P : Measure X) [IsProbabilityMeasure P] (lambda : ℝ≥0) (n : ℕ) {T : FiniteSample X → ℝ} (hT : Measurable T) (hTb : UniformlyBounded T) (theta : ℝ) : sqRisk ((finitePoissonSampleLaw P lambda).restrict (FiniteSample.count ⁻¹' Set.Iic n)) T theta ≤ sqRisk (finitePoissonSampleLaw P lambda) T theta := by -- Split the full integral over the measurable count event and its -- complement, then discard the nonnegative complementary squared loss. obtain ⟨M, hM, hTb⟩ := hTb let μ : Measure (FiniteSample X) := finitePoissonSampleLaw P lambda let s : Set (FiniteSample X) := FiniteSample.count ⁻¹' Set.Iic n have hs : MeasurableSet s := by exact measurable_finiteSample_count measurableSet_Iic have hlossInt : Integrable (fun z => (T z - theta) ^ 2) μ := by refine (integrable_const ((M + |theta|) ^ 2)).mono' ((hT.sub measurable_const).pow_const 2).aestronglyMeasurable ?_ filter_upwards [] with z rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _), sq_le_sq, abs_of_nonneg (add_nonneg hM (abs_nonneg theta))] exact (abs_sub _ _).trans (add_le_add (hTb z) le_rfl) have hsInt : Integrable (fun z => (T z - theta) ^ 2) (μ.restrict s) := hlossInt.restrict have hscInt : Integrable (fun z => (T z - theta) ^ 2) (μ.restrict sᶜ) := hlossInt.restrict unfold sqRisk change (∫ z, (T z - theta) ^ 2 ∂μ.restrict s) ≤ ∫ z, (T z - theta) ^ 2 ∂μ conv_rhs => rw [← Measure.restrict_add_restrict_compl (μ := μ) hs, integral_add_measure hsInt hscInt] exact le_add_of_nonneg_right (integral_nonneg fun z => sq_nonneg (T z - theta)) -
sqRisk_raoBlackwellStatistic_unitInterval_letheorem — For an observation probability law, a Poisson mean, and a fixed sample size, under measurability of the statistic, unit-interval bounds on the statistic, target, and overflow value, the fixed-iid Rao--Blackwell risk is at most the uncapped finite-Poisson risk plus exactly the overflow probability.hypothesesX :sharedType u_1P :lambda :ℝ≥0n :ℕT :FiniteSample X → ℝhT :theta zOver :ℝhTmem :∀ s, T s ∈ Icc (0 : ℝ) 1htheta :theta ∈ Icc (0 : ℝ) 1hzOver :zOver ∈ Icc (0 : ℝ) 1conclusion≤ sqRisk (finitePoissonSampleLaw P lambda) T thetaProof (Lean source)
theorem sqRisk_raoBlackwellStatistic_unitInterval_le (P : Measure X) [IsProbabilityMeasure P] (lambda : ℝ≥0) (n : ℕ) {T : FiniteSample X → ℝ} (hT : Measurable T) {theta zOver : ℝ} (hTmem : ∀ s, T s ∈ Icc (0 : ℝ) 1) (htheta : theta ∈ Icc (0 : ℝ) 1) (hzOver : zOver ∈ Icc (0 : ℝ) 1) : sqRisk (Measure.pi (fun _ : Fin n ↦ P)) (raoBlackwellStatistic lambda T zOver) theta ≤ sqRisk (finitePoissonSampleLaw P lambda) T theta + (poissonMeasure lambda).real (Ioi n) := by -- Specialize the interval theorem to `[0,1]` and normalize `(1 - 0)^2`. simpa using (sqRisk_raoBlackwellStatistic_le P lambda n hT (a := (0 : ℝ)) (b := 1) (theta := theta) (zOver := zOver) (by norm_num) hTmem htheta hzOver)
MaximalCoupling 5 core · 13 supporting This file develops the common-submeasure and coupling construction used by the coordinatewise direct-product argument. ★ maximalCoupling_eq_mass_ge
Maximal couplings through a measurable compression
This file develops the common-submeasure and coupling construction used by the coordinatewise direct-product argument.
def measurableEqOfStandardBorel reviewed
⚑ flagged
Causalean.Stat
For a standard Borel measurable space, the measurability of its equality relation is defined.
Definition (Lean source)
parameters
X :
def
given by
by let e : X → ℝ := MeasureTheory.embeddingReal X have he : MeasurableEmbedding e := MeasureTheory.measurableEmbedding_embeddingReal X constructor rw [show Set.diagonal X = (map e e) ⁻¹' Set.diagonal ℝ by ext p simp only [Set.mem_diagonal_iff, Set.mem_preimage, Prod.map_apply] exact he.injective.eq_iff.symm] exact measurableSet_diagonal.preimage (he.measurable.prodMap he.measurable)
def rnCommonPart reviewed
⚑ flagged
Causalean.Stat
Given a measurable sample space and three measures on it, consisting of two target measures and a reference measure, the common Radon--Nikodym submeasure is the reference measure weighted by the pointwise minimum of the two target measures' Radon--Nikodym densities relative to that reference measure.
def maximalCoupling reviewed
⚑ flagged
Causalean.Stat
Given a measurable sample space and two probability measures on it, the maximal coupling is the measure on pairs whose first branch is the diagonal coupling when their total-variation distance is zero, and whose second branch otherwise combines their common part on the diagonal with the normalized product of their residual measures.
Definition (Lean source)
parameters
X :
Type*
mu nu :
def
given by
if hzero : tvDist mu nu
= 0 then mu.map fun x => (x, x) else let xi := mu
+ nu let c := rnCommonPart mu nu xi let rmu := mu
- c let rnu := nu
- c c.map (fun x => (x, x))
lemma maximalCoupling_eq_mass_ge reviewed
⚑ flagged
Causalean.Stat
For two probability measures mu and nu on a standard Borel space X, the two coordinates of their maximal coupling agree with probability at least one minus their total variation distance.
Formal statement
hypotheses
X :
mu nu :
conclusion
Proof (Lean source)
lemma maximalCoupling_eq_mass_ge
{X : Type*} [MeasurableSpace X] [MeasurableEq X]
(mu nu : Measure X) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu] :
ofReal (1 - tvDist mu nu) ≤
maximalCoupling mu nu {p | p.1 = p.2} := by
let D : Set (X × X) := {p | p.1 = p.2}
have hD : MeasurableSet D := measurableSet_eq_fun measurable_fst measurable_snd
change ofReal (1 - tvDist mu nu) ≤
maximalCoupling mu nu D
by_cases hzero : tvDist mu nu = 0
· rw [maximalCoupling, dif_pos hzero, Measure.map_apply
(μ := mu) (f := fun x : X => (x, x))
(measurable_id.prodMk measurable_id) hD]
simp [D, hzero]
· let xi := mu + nu
let c := rnCommonPart mu nu xi
have hmu : mu ≪ xi := Measure.AbsolutelyContinuous.rfl.add_right _
have hnu : nu ≪ xi := Measure.AbsolutelyContinuous.rfl.add_right' _
have hcmass : c univ =
ofReal (1 - tvDist mu nu) :=
rnCommonPart_mass_eq_one_sub_tvDist mu nu xi hmu hnu
rw [maximalCoupling, dif_neg hzero]
dsimp only
rw [Measure.add_apply]
calc
ofReal (1 - tvDist mu nu) = c univ := hcmass.symm
_ = c.map (fun x => (x, x)) D := by
rw [Measure.map_apply (μ := c) (f := fun x : X => (x, x))
(measurable_id.prodMk measurable_id) hD]
congr 1
ext x
simp [D]
_ ≤ c.map (fun x => (x, x)) D +
((ofReal (tvDist mu nu))⁻¹ •
((mu - c).prod (nu - c))) D := le_add_right le_rfl
def compressionCoupling reviewed
⚑ flagged
Causalean.Stat
Given two standard Borel measurable spaces, an observation space and a compressed-state space, two probability measures on the observation space, and a measurable compression map from observations to compressed states, the compression coupling is the joint law obtained by maximally coupling the two compressed laws and then, conditional on each coupled compressed state, drawing each observation from its corresponding regular conditional distribution.
Definition (Lean source)
parameters
def
given by
by letI : Nonempty Z := nonempty_of_isProbabilityMeasure Q0 letI mapProb0 : IsProbabilityMeasure (Q0.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable letI mapProb1 : IsProbabilityMeasure (Q1.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable let K0 := condDistrib id compress Q0 let K1 := condDistrib id compress Q1 let gamma := maximalCoupling (Q0.map compress) (Q1.map compress) exact ((K0.comap fst measurable_fst) ×ₖ (K1.comap snd measurable_snd)) ∘ₘ gamma
13 supporting declarations (lemmas, instances)
-
tvDist_eq_half_integral_abs_rnDeriv_sublemma — Scheffé's identity with both probability laws dominated by an arbitrary finite reference measure.hypothesesX :Type*mu nu xi :hmu :mu ≪ xihnu :nu ≪ xiProof (Lean source)
lemma tvDist_eq_half_integral_abs_rnDeriv_sub {X : Type*} [MeasurableSpace X] (mu nu xi : Measure X) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu] [IsFiniteMeasure xi] (hmu : mu ≪ xi) (hnu : nu ≪ xi) : tvDist mu nu = (1 / 2 : ℝ) * ∫ x, |(mu.rnDeriv xi x).toReal - (nu.rnDeriv xi x).toReal| ∂xi := by let p : X → ℝ := fun x => (mu.rnDeriv xi x).toReal let q : X → ℝ := fun x => (nu.rnDeriv xi x).toReal let d : X → ℝ := fun x => p x - q x have hp : Integrable p xi := Measure.integrable_toReal_rnDeriv have hq : Integrable q xi := Measure.integrable_toReal_rnDeriv have hd : Integrable d xi := hp.sub hq have hd0 : ∫ x, d x ∂xi = 0 := by rw [show d = fun x => p x - q x from rfl, integral_sub hp hq] rw [show p = fun x => (mu.rnDeriv xi x).toReal from rfl, show q = fun x => (nu.rnDeriv xi x).toReal from rfl, Measure.integral_toReal_rnDeriv hmu, Measure.integral_toReal_rnDeriv hnu] simp apply le_antisymm · unfold tvDist apply ciSup_le rintro ⟨A, hA⟩ have hgap : mu.real A - nu.real A = ∫ x in A, d x ∂xi := by rw [show d = fun x => p x - q x from rfl, integral_sub hp.integrableOn hq.integrableOn, show p = fun x => (mu.rnDeriv xi x).toReal from rfl, show q = fun x => (nu.rnDeriv xi x).toReal from rfl, Measure.setIntegral_toReal_rnDeriv hmu, Measure.setIntegral_toReal_rnDeriv hnu] rw [hgap] exact Causalean.Stat.abs_setIntegral_le_half_integral_abs_of_integral_eq_zero hd hd0 hA · let A : Set X := {x | 0 ≤ d x} have hA : MeasurableSet A := by dsimp [A, d, p, q] exact measurableSet_le measurable_const ((Measure.measurable_rnDeriv mu xi).ennreal_toReal.sub (Measure.measurable_rnDeriv nu xi).ennreal_toReal) have hsplit : ∫ x in A, d x ∂xi + ∫ x in Aᶜ, d x ∂xi = 0 := by rw [MeasureTheory.integral_add_compl hA hd, hd0] have hpos : 0 ≤ ∫ x in A, d x ∂xi := by apply integral_nonneg_of_ae filter_upwards [ae_restrict_mem hA] with x hx exact hx have habs : ∫ x, |d x| ∂xi = 2 * ∫ x in A, d x ∂xi := by rw [← MeasureTheory.integral_add_compl hA hd.abs] have h1 : ∫ x in A, |d x| ∂xi = ∫ x in A, d x ∂xi := by apply integral_congr_ae filter_upwards [ae_restrict_mem hA] with x hx exact abs_of_nonneg hx have h2 : ∫ x in Aᶜ, |d x| ∂xi = -(∫ x in Aᶜ, d x ∂xi) := by rw [← integral_neg] apply integral_congr_ae filter_upwards [ae_restrict_mem hA.compl] with x hx exact abs_of_nonpos (le_of_not_ge hx) rw [h1, h2] linarith have hgap : mu.real A - nu.real A = ∫ x in A, d x ∂xi := by rw [show d = fun x => p x - q x from rfl, integral_sub hp.integrableOn hq.integrableOn, show p = fun x => (mu.rnDeriv xi x).toReal from rfl, show q = fun x => (nu.rnDeriv xi x).toReal from rfl, Measure.setIntegral_toReal_rnDeriv hmu, Measure.setIntegral_toReal_rnDeriv hnu] have htv := abs_measureReal_sub_le_tvDist (μ := mu) (ν := nu) hA change (1 / 2 : ℝ) * ∫ x, |d x| ∂xi ≤ _ rw [habs] rw [hgap, abs_of_nonneg hpos] at htv linarith -
rnCommonPart_le_leftlemma — The RN common part is dominated by its first law.hypothesesconclusionrnCommonPart mu nu xi ≤ muProof (Lean source)
lemma rnCommonPart_le_left {X : Type*} [MeasurableSpace X] (mu nu xi : Measure X) [IsFiniteMeasure mu] [IsFiniteMeasure xi] (hmu : mu ≪ xi) : rnCommonPart mu nu xi ≤ mu := by calc rnCommonPart mu nu xi ≤ xi.withDensity (mu.rnDeriv xi) := by apply withDensity_mono exact Filter.Eventually.of_forall fun x => min_le_left _ _ _ = mu := Measure.withDensity_rnDeriv_eq mu xi hmu -
rnCommonPart_le_rightlemma — The RN common part is dominated by its second law.hypothesesconclusionrnCommonPart mu nu xi ≤ nuProof (Lean source)
lemma rnCommonPart_le_right {X : Type*} [MeasurableSpace X] (mu nu xi : Measure X) [IsFiniteMeasure nu] [IsFiniteMeasure xi] (hnu : nu ≪ xi) : rnCommonPart mu nu xi ≤ nu := by calc rnCommonPart mu nu xi ≤ xi.withDensity (nu.rnDeriv xi) := by apply withDensity_mono exact Filter.Eventually.of_forall fun x => min_le_right _ _ _ = nu := Measure.withDensity_rnDeriv_eq nu xi hnu -
rnCommonPart_mass_eq_one_sub_tvDistlemma — The mass of the RN common part is exactly one minus total variation.hypothesesX :Type*mu nu xi :hmu :mu ≪ xihnu :nu ≪ xiconclusionProof (Lean source)
lemma rnCommonPart_mass_eq_one_sub_tvDist {X : Type*} [MeasurableSpace X] (mu nu xi : Measure X) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu] [IsFiniteMeasure xi] (hmu : mu ≪ xi) (hnu : nu ≪ xi) : rnCommonPart mu nu xi univ = ofReal (1 - tvDist mu nu) := by let p : X → ℝ := fun x => (mu.rnDeriv xi x).toReal let q : X → ℝ := fun x => (nu.rnDeriv xi x).toReal have hp : Integrable p xi := Measure.integrable_toReal_rnDeriv have hq : Integrable q xi := Measure.integrable_toReal_rnDeriv have hformula : ∀ x, min (p x) (q x) = (p x + q x - |p x - q x|) / 2 := by intro x rcases le_total (p x) (q x) with h | h · rw [min_eq_left h, abs_of_nonpos (sub_nonpos.mpr h)] ring · rw [min_eq_right h, abs_of_nonneg (sub_nonneg.mpr h)] ring have hmin : Integrable (fun x => min (p x) (q x)) xi := by refine ((hp.add hq).sub (hp.sub hq).abs |>.div_const 2).congr ?_ exact Filter.Eventually.of_forall fun x => (hformula x).symm have hmin_nonneg : 0 ≤ᵐ[xi] fun x => min (p x) (q x) := by filter_upwards [] with x exact le_min ENNReal.toReal_nonneg ENNReal.toReal_nonneg have hp_one : ∫ x, p x ∂xi = 1 := by rw [show p = fun x => (mu.rnDeriv xi x).toReal from rfl, Measure.integral_toReal_rnDeriv hmu] simp have hq_one : ∫ x, q x ∂xi = 1 := by rw [show q = fun x => (nu.rnDeriv xi x).toReal from rfl, Measure.integral_toReal_rnDeriv hnu] simp have hreal : ∫ x, min (p x) (q x) ∂xi = 1 - tvDist mu nu := by have hid : (fun x => min (p x) (q x)) = fun x => (p x + q x - |p x - q x|) / 2 := by funext x; exact hformula x rw [hid, integral_div] rw [integral_sub (f := fun x => p x + q x) (g := fun x => |p x - q x|) (hp.add hq) (hp.sub hq).abs] rw [integral_add hp hq, hp_one, hq_one, tvDist_eq_half_integral_abs_rnDeriv_sub mu nu xi hmu hnu] ring rw [rnCommonPart, withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ] calc ∫⁻ x, min (mu.rnDeriv xi x) (nu.rnDeriv xi x) ∂xi = ∫⁻ x, ofReal (min (p x) (q x)) ∂xi := by apply lintegral_congr_ae filter_upwards [Measure.rnDeriv_lt_top mu xi, Measure.rnDeriv_lt_top nu xi] with x hpx hqx simp only [ENNReal.ofReal_min] rw [ENNReal.ofReal_toReal hpx.ne, ENNReal.ofReal_toReal hqx.ne] _ = ofReal (∫ x, min (p x) (q x) ∂xi) := by exact (ofReal_integral_eq_lintegral_ofReal hmin hmin_nonneg).symm _ = _ := by rw [hreal] -
measure_eq_of_tvDist_eq_zerolemma — Probability measures at total-variation distance zero are equal.Proof (Lean source)
lemma measure_eq_of_tvDist_eq_zero {X : Type*} [MeasurableSpace X] (mu nu : Measure X) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu] (h : tvDist mu nu = 0) : mu = nu := by ext A hA have hgap := abs_measureReal_sub_le_tvDist (μ := mu) (ν := nu) hA rw [h] at hgap have hre : mu.real A = nu.real A := by have := abs_nonneg (mu.real A - nu.real A) rw [abs_nonpos_iff] at hgap linarith rw [← ENNReal.toReal_eq_toReal_iff' (measure_ne_top mu A) (measure_ne_top nu A)] exact hre -
maximalCoupling_map_fstlemma — The first marginal of the maximal coupling is the first law.hypothesesX :Type*mu nu :conclusionProof (Lean source)
lemma maximalCoupling_map_fst {X : Type*} [MeasurableSpace X] (mu nu : Measure X) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu] : (maximalCoupling mu nu).map Prod.fst = mu := by by_cases hzero : tvDist mu nu = 0 · rw [maximalCoupling, dif_pos hzero, Measure.map_map (μ := mu) (f := fun x : X => (x, x)) (g := Prod.fst) measurable_fst (measurable_id.prodMk measurable_id)] change mu.map id = mu exact Measure.map_id · let xi := mu + nu let c := rnCommonPart mu nu xi let rmu := mu - c let rnu := nu - c have hmu : mu ≪ xi := Measure.AbsolutelyContinuous.rfl.add_right _ have hc : c ≤ mu := rnCommonPart_le_left mu nu xi hmu letI : IsFiniteMeasure c := isFiniteMeasure_of_le mu hc have hrnu : rnu univ = ofReal (tvDist mu nu) := residual_mass_right mu nu rw [maximalCoupling, dif_neg hzero] dsimp only rw [Measure.map_add, Measure.map_smul, Measure.map_map, Measure.map_fst_prod, hrnu] rw [← mul_smul, ENNReal.inv_mul_cancel (ENNReal.ofReal_ne_zero_iff.mpr (lt_of_le_of_ne (tvDist_nonneg (μ := mu) (ν := nu)) (Ne.symm hzero))) ENNReal.ofReal_ne_top, one_smul] rw [show (Prod.fst ∘ fun x : X => (x, x)) = id by rfl, Measure.map_id, add_comm, Measure.sub_add_cancel_of_le hc] all_goals fun_prop -
maximalCoupling_map_sndlemma — The second marginal of the maximal coupling is the second law.hypothesesX :Type*mu nu :conclusionProof (Lean source)
lemma maximalCoupling_map_snd {X : Type*} [MeasurableSpace X] (mu nu : Measure X) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu] : (maximalCoupling mu nu).map Prod.snd = nu := by by_cases hzero : tvDist mu nu = 0 · have heq : mu = nu := measure_eq_of_tvDist_eq_zero mu nu hzero rw [maximalCoupling, dif_pos hzero, Measure.map_map (μ := mu) (f := fun x : X => (x, x)) (g := Prod.snd) measurable_snd (measurable_id.prodMk measurable_id)] change mu.map id = nu rw [Measure.map_id, heq] · let xi := mu + nu let c := rnCommonPart mu nu xi let rmu := mu - c let rnu := nu - c have hnu : nu ≪ xi := Measure.AbsolutelyContinuous.rfl.add_right' _ have hc : c ≤ nu := rnCommonPart_le_right mu nu xi hnu letI : IsFiniteMeasure c := isFiniteMeasure_of_le nu hc have hrmu : rmu univ = ofReal (tvDist mu nu) := residual_mass mu nu rw [maximalCoupling, dif_neg hzero] dsimp only rw [Measure.map_add, Measure.map_smul, Measure.map_map, Measure.map_snd_prod, hrmu] rw [← mul_smul, ENNReal.inv_mul_cancel (ENNReal.ofReal_ne_zero_iff.mpr (lt_of_le_of_ne (tvDist_nonneg (μ := mu) (ν := nu)) (Ne.symm hzero))) ENNReal.ofReal_ne_top, one_smul] rw [show (Prod.snd ∘ fun x : X => (x, x)) = id by rfl, Measure.map_id, add_comm, Measure.sub_add_cancel_of_le hc] all_goals fun_prop -
instIsProbabilityMeasureinstanceparametersX :Type*mu nu :instancegiven byby constructor have h := congrArg (fun m : Measure X => m univ) (maximalCoupling_map_fst mu nu) change ((maximalCoupling mu nu).map fst) univ= mu univ at h rw [Measure.map_apply measurable_fst MeasurableSet.univ] at h simpa using h -
compressionCoupling_map_fstlemma — The first marginal of the lifted compression coupling is the first raw law.Proof (Lean source)
lemma compressionCoupling_map_fst {Z S : Type*} [MeasurableSpace Z] [StandardBorelSpace Z] [MeasurableSpace S] [StandardBorelSpace S] (Q0 Q1 : Measure Z) [IsProbabilityMeasure Q0] [IsProbabilityMeasure Q1] (compress : Z → S) (hcompress : Measurable compress) : (compressionCoupling Q0 Q1 compress hcompress).map fst = Q0 := by letI : Nonempty Z := nonempty_of_isProbabilityMeasure Q0 letI mapProb0 : IsProbabilityMeasure (Q0.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable letI mapProb1 : IsProbabilityMeasure (Q1.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable let K0 := condDistrib id compress Q0 let K1 := condDistrib id compress Q1 let gamma := maximalCoupling (Q0.map compress) (Q1.map compress) rw [compressionCoupling, Measure.map_comp] change (((K0.comap fst measurable_fst) ×ₖ (K1.comap snd measurable_snd)).map fst) ∘ₘ gamma = Q0 rw [← Kernel.fst_eq, Kernel.fst_prod] rw [comap_comp_map_measure gamma fst measurable_fst K0, maximalCoupling_map_fst] exact condDistrib_id_comp_map_compress Q0 compress hcompress all_goals fun_prop -
compressionCoupling_map_sndlemma — The second marginal of the lifted compression coupling is the second raw law.Proof (Lean source)
lemma compressionCoupling_map_snd {Z S : Type*} [MeasurableSpace Z] [StandardBorelSpace Z] [MeasurableSpace S] [StandardBorelSpace S] (Q0 Q1 : Measure Z) [IsProbabilityMeasure Q0] [IsProbabilityMeasure Q1] (compress : Z → S) (hcompress : Measurable compress) : (compressionCoupling Q0 Q1 compress hcompress).map snd = Q1 := by letI : Nonempty Z := nonempty_of_isProbabilityMeasure Q0 letI mapProb0 : IsProbabilityMeasure (Q0.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable letI mapProb1 : IsProbabilityMeasure (Q1.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable let K0 := condDistrib id compress Q0 let K1 := condDistrib id compress Q1 let gamma := maximalCoupling (Q0.map compress) (Q1.map compress) rw [compressionCoupling, Measure.map_comp] change (((K0.comap fst measurable_fst) ×ₖ (K1.comap snd measurable_snd)).map snd) ∘ₘ gamma = Q1 rw [← Kernel.snd_eq, Kernel.snd_prod] rw [comap_comp_map_measure gamma snd measurable_snd K1, maximalCoupling_map_snd] exact condDistrib_id_comp_map_compress Q1 compress hcompress all_goals fun_prop -
compressionCoupling_map_compress_pairlemma — Compressing both coordinates of the lifted coupling recovers the maximal coupling of the compressed laws.hypothesesZ S :Q0 Q1 :compress :Z → Shcompress :Measurable compressIsProbabilityMeasure (Q0.map compress)IsProbabilityMeasure (Q1.map compress)conclusion= maximalCoupling (Q0.map compress) (Q1.map compress)Proof (Lean source)
lemma compressionCoupling_map_compress_pair {Z S : Type*} [MeasurableSpace Z] [StandardBorelSpace Z] [MeasurableSpace S] [StandardBorelSpace S] (Q0 Q1 : Measure Z) [IsProbabilityMeasure Q0] [IsProbabilityMeasure Q1] (compress : Z → S) (hcompress : Measurable compress) [IsProbabilityMeasure (Q0.map compress)] [IsProbabilityMeasure (Q1.map compress)] : (compressionCoupling Q0 Q1 compress hcompress).map (map compress compress) = maximalCoupling (Q0.map compress) (Q1.map compress) := by letI : Nonempty Z := nonempty_of_isProbabilityMeasure Q0 letI : Nonempty S := nonempty_of_isProbabilityMeasure (Q0.map compress) letI mapProb0 : IsProbabilityMeasure (Q0.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable letI mapProb1 : IsProbabilityMeasure (Q1.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable let K0 := condDistrib id compress Q0 let K1 := condDistrib id compress Q1 let gamma := maximalCoupling (Q0.map compress) (Q1.map compress) have hK0 : K0.map compress =ᵐ[Q0.map compress] Kernel.id := by exact (ProbabilityTheory.condDistrib_comp (μ := Q0) (Y := id) compress aemeasurable_id hcompress).symm.trans (ProbabilityTheory.condDistrib_self (μ := Q0) compress) have hK1 : K1.map compress =ᵐ[Q1.map compress] Kernel.id := by exact (ProbabilityTheory.condDistrib_comp (μ := Q1) (Y := id) compress aemeasurable_id hcompress).symm.trans (ProbabilityTheory.condDistrib_self (μ := Q1) compress) have hK0gamma : ∀ᵐ p ∂gamma, K0.map compress p.1 = Kernel.id p.1 := by rw [← maximalCoupling_map_fst (Q0.map compress) (Q1.map compress)] at hK0 exact ae_of_ae_map measurable_fst.aemeasurable hK0 have hK1gamma : ∀ᵐ p ∂gamma, K1.map compress p.2 = Kernel.id p.2 := by rw [← maximalCoupling_map_snd (Q0.map compress) (Q1.map compress)] at hK1 exact ae_of_ae_map measurable_snd.aemeasurable hK1 rw [compressionCoupling] rw [Measure.map_comp _ _ (hcompress.prodMap hcompress)] calc _ = Kernel.id ∘ₘ gamma := by apply Measure.comp_congr filter_upwards [hK0gamma, hK1gamma] with p hp0 hp1 change (((K0.comap fst measurable_fst) ×ₖ (K1.comap snd measurable_snd)).map (map compress compress)) p = Kernel.id p rw [← Kernel.map_prod_map] rw [Kernel.prod_apply, Kernel.map_apply _ hcompress, Kernel.map_apply _ hcompress, Kernel.comap_apply, Kernel.comap_apply, Kernel.id_apply] rw [Kernel.map_apply _ hcompress] at hp0 hp1 rw [hp0, hp1] exact Measure.dirac_prod_dirac all_goals fun_prop _ = gamma := Measure.id_comp -
instIsProbabilityMeasureinstanceparametersinstanceinstIsProbabilityMeasure Q0 Q1 compress hcompress :IsProbabilityMeasure (compressionCoupling Q0 Q1 compress hcompress)given byby constructor have h := congrArg (fun m : Measure Z => m univ) (compressionCoupling_map_fst Q0 Q1 compress hcompress) change ((compressionCoupling Q0 Q1 compress hcompress).map fst) univ= Q0 univ at h rw [Measure.map_apply measurable_fst MeasurableSet.univ] at h simpa using h -
compressionCoupling_equal_compression_mass_gelemma — In the lifted coupling, the compressed observations agree with probability at least one minus the total variation of their compressed laws.hypothesesconclusion≤ compressionCoupling Q0 Q1 compress hcompress {p | compress p.1 = compress p.2}Proof (Lean source)
lemma compressionCoupling_equal_compression_mass_ge {Z S : Type*} [MeasurableSpace Z] [StandardBorelSpace Z] [MeasurableSpace S] [StandardBorelSpace S] (Q0 Q1 : Measure Z) [IsProbabilityMeasure Q0] [IsProbabilityMeasure Q1] (compress : Z → S) (hcompress : Measurable compress) : ofReal (1 - tvDist (Q0.map compress) (Q1.map compress)) ≤ compressionCoupling Q0 Q1 compress hcompress {p | compress p.1 = compress p.2} := by letI mapProb0 : IsProbabilityMeasure (Q0.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable letI mapProb1 : IsProbabilityMeasure (Q1.map compress) := Measure.isProbabilityMeasure_map hcompress.aemeasurable letI : MeasurableEq S := measurableEqOfStandardBorel S let D : Set (S × S) := {p | p.1 = p.2} have hD : MeasurableSet D := measurableSet_eq_fun measurable_fst measurable_snd have hmap := compressionCoupling_map_compress_pair Q0 Q1 compress hcompress have happ := congrArg (fun m : Measure (S × S) => m D) hmap change ((compressionCoupling Q0 Q1 compress hcompress).map (map compress compress)) D = maximalCoupling (Q0.map compress) (Q1.map compress) D at happ rw [Measure.map_apply (hcompress.prodMap hcompress) hD] at happ have hpre : (map compress compress) ⁻¹' D = {p : Z × Z | compress p.1 = compress p.2} := by ext p rfl rw [hpre] at happ rw [happ] exact maximalCoupling_eq_mass_ge (Q0.map compress) (Q1.map compress)
Scheffe 1 core · 2 supporting This file proves the one-sided Scheffe inequality relating total variation distance to the integral absolute deviation of a Radon-Nikodym density from one. ★ tvDist_le_half_integral_abs_rnDeriv
Scheffe Bound for Total Variation
This file proves the one-sided Scheffe inequality relating total variation distance to the integral absolute deviation of a Radon-Nikodym density from one. It supplies the analytic bridge used to convert density-based divergence bounds into minimax testing bounds.
theorem tvDist_le_half_integral_abs_rnDeriv reviewed
⚑ flagged
Causalean.Stat
Scheffé's inequality (≤ direction). For probability measures μ and ν on the same space with μ absolutely continuous with respect to ν, the total variation distance between μ and ν is at most half the L¹(ν)-distance of the Radon–Nikodym density dμ/dν to the constant 1.
Formal statement
hypotheses
μ ν :
hac :
μ ≪ ν
Proof (Lean source)
theorem tvDist_le_half_integral_abs_rnDeriv (μ ν : Measure Ω)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν] (hac : μ ≪ ν) :
tvDist μ ν ≤ (1/2) * ∫ x, |(μ.rnDeriv ν x).toReal - 1| ∂ν := by
set f : Ω → ℝ := fun x => (μ.rnDeriv ν x).toReal - 1 with hf_def
have hint_p : Integrable (fun x => (μ.rnDeriv ν x).toReal) ν :=
Measure.integrable_toReal_rnDeriv
have hf : Integrable f ν := hint_p.sub (integrable_const 1)
-- `∫ f = μ.real univ − ν.real univ = 0`
have hf0 : ∫ x, f x ∂ν = 0 := by
rw [hf_def]
rw [integral_sub hint_p (integrable_const 1)]
rw [Measure.integral_toReal_rnDeriv hac]
simp only [integral_const, smul_eq_mul, mul_one]
rw [measureReal_def]
simp [measure_univ]
-- supremum is over measurable sets; bound each term
refine ciSup_le fun A => ?_
obtain ⟨A, hA⟩ := A
have hgap : μ.real A - ν.real A = ∫ x in A, f x ∂ν :=
measureReal_sub_eq_setIntegral_rnDeriv_sub_one hac hA
rw [hgap]
exact abs_setIntegral_le_half_integral_abs_of_integral_eq_zero hf hf0 hA
2 supporting declarations (lemmas, instances)
-
abs_setIntegral_le_half_integral_abs_of_integral_eq_zerotheorem — For an integrable function f whose integral vanishes, the integral over any measurable set is bounded in absolute value by half the L¹ norm of f.hypothesesconclusion|∫ x in A, f x ∂ν| ≤ (1/2) * ∫ x, |f x| ∂νProof (Lean source)
theorem abs_setIntegral_le_half_integral_abs_of_integral_eq_zero {f : Ω → ℝ} (hf : Integrable f ν) (hf0 : ∫ x, f x ∂ν = 0) {A : Set Ω} (hA : MeasurableSet A) : |∫ x in A, f x ∂ν| ≤ (1/2) * ∫ x, |f x| ∂ν := by have hfA : IntegrableOn f A ν := hf.integrableOn have hfAc : IntegrableOn f Aᶜ ν := hf.integrableOn -- `∫_A f + ∫_{Aᶜ} f = 0` have hsplit : ∫ x in A, f x ∂ν + ∫ x in Aᶜ, f x ∂ν = 0 := by rw [MeasureTheory.integral_add_compl hA hf, hf0] have hcompl : ∫ x in Aᶜ, f x ∂ν = -(∫ x in A, f x ∂ν) := by linarith -- `∫|f| = ∫_A |f| + ∫_{Aᶜ} |f|` have habs : Integrable (fun x => |f x|) ν := hf.abs have hsplitabs : ∫ x, |f x| ∂ν = (∫ x in A, |f x| ∂ν) + ∫ x in Aᶜ, |f x| ∂ν := (MeasureTheory.integral_add_compl hA habs).symm -- bound each piece of `f` by `|f|` have hbA : ∫ x in A, f x ∂ν ≤ ∫ x in A, |f x| ∂ν := integral_mono_ae hfA habs.integrableOn (Filter.Eventually.of_forall fun x => le_abs_self _) have hbAc : ∫ x in Aᶜ, f x ∂ν ≤ ∫ x in Aᶜ, |f x| ∂ν := integral_mono_ae hfAc habs.integrableOn (Filter.Eventually.of_forall fun x => le_abs_self _) have hbAneg : -(∫ x in A, f x ∂ν) ≤ ∫ x in A, |f x| ∂ν := by have : ∫ x in A, (-f x) ∂ν ≤ ∫ x in A, |f x| ∂ν := integral_mono_ae hfA.neg habs.integrableOn (Filter.Eventually.of_forall fun x => (neg_le_abs _)) rwa [integral_neg] at this have hbAcneg : -(∫ x in Aᶜ, f x ∂ν) ≤ ∫ x in Aᶜ, |f x| ∂ν := by have : ∫ x in Aᶜ, (-f x) ∂ν ≤ ∫ x in Aᶜ, |f x| ∂ν := integral_mono_ae hfAc.neg habs.integrableOn (Filter.Eventually.of_forall fun x => (neg_le_abs _)) rwa [integral_neg] at this rw [abs_le, hsplitabs] constructor · -- `-(½(∫_A|f| + ∫_{Aᶜ}|f|)) ≤ ∫_A f` nlinarith [hbAneg, hbAc, hcompl] · -- `∫_A f ≤ ½(∫_A|f| + ∫_{Aᶜ}|f|)` nlinarith [hbA, hbAcneg, hcompl] -
measureReal_sub_eq_setIntegral_rnDeriv_sub_onetheorem — With density p = dμ/dν and μ ≪ ν, the signed gap μ.real A − ν.real A equals ∫ x in A, (p x − 1) ∂ν.hypothesesconclusionμ.real A - ν.real A = ∫ x in A, ((μ.rnDeriv ν x).toReal - 1) ∂νProof (Lean source)
theorem measureReal_sub_eq_setIntegral_rnDeriv_sub_one (hac : μ ≪ ν) {A : Set Ω} (_hA : MeasurableSet A) : μ.real A - ν.real A = ∫ x in A, ((μ.rnDeriv ν x).toReal - 1) ∂ν := by have hp : ∫ x in A, (μ.rnDeriv ν x).toReal ∂ν = μ.real A := Measure.setIntegral_toReal_rnDeriv hac A have hint : IntegrableOn (fun x => (μ.rnDeriv ν x).toReal) A ν := (Measure.integrable_toReal_rnDeriv).integrableOn have hc : IntegrableOn (fun _ : Ω => (1:ℝ)) A ν := (integrable_const 1).integrableOn have h1 : ∫ _ in A, (1:ℝ) ∂ν = ν.real A := by rw [setIntegral_const, smul_eq_mul, mul_one, measureReal_def] rw [integral_sub hint hc, hp, h1]