Formalization: Treatment Effect Estimation with Auxiliary Records and Diminishing Overlap
The complete Lean development behind this paper — every definition, lemma, and theorem of its module, including helpers the paper text never cites. Identifiers link within this page, into the Causalean library, or out to the official Mathlib docs. Note: this index was generated at a different commit than the paper's audited pin (70df51f) — statements may have moved since the audit.
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Basic 61 declarations Finite two-channel ATE experiment and its rare-overlap frontier.
Finite two-channel ATE experiment and its rare-overlap frontier.
def Obs
Complete record with binary treatment and outcome.
def AuxObs
An auxiliary record retains covariates and treatment without the outcome.
The unknown observable population is a PMF on complete-record atoms.
instance instMeasurableSpaceDiscreteLaw
The simplex carries its coordinate Borel sigma algebra.
Definition (Lean source)
instance {d : Nat} : MeasurableSpace (DiscreteLaw d) :=
comap (fun P => fun z => (P.pmf z).toReal) inferInstance
def obsLaw
For a finite covariate dimension and an observable population law, the observed-data sampling law is the probability measure represented by that law's PMF.
Definition (Lean source)
def jointMass
The real mass of one complete-record atom is read from the population PMF.
Covariate masses sum the complete-record probabilities over both binary coordinates.
def armMass
Arm-cell masses sum the two possible outcome marks.
def markedMass
The outcome-marked mass is the probability of a success in the specified arm and cell.
The occupied-cell treatment probability uses the arm-to-cell ratio; null cells use one half.
Outcome regression is the success-to-arm ratio, with zero on null arm cells.
Definition (Lean source)
instance instMeasurableSpacePMFAuxObs
Marginal PMFs carry the Borel sigma algebra of their real atom-mass coordinates.
Definition (Lean source)
instance {d : Nat} : MeasurableSpace (PMF (AuxObs d)) :=
comap (fun p => fun z => (p z).toReal) inferInstance
The auxiliary PMF is the actual treatment-covariate projection of the observed law.
Definition (Lean source)
parameters
d :
P :
given by
P.pmf.map (fun z => (z.1, z.2.1))
def labeledProductLaw
The ordered complete records have the iid product probability law.
Definition (Lean source)
parameters
given by
Measure.pi (fun _ => obsLaw P)
def auxProductLaw
The ordered auxiliary records have the iid law of the actual treatment-covariate marginal.
Definition (Lean source)
parameters
given by
Measure.pi (fun _ => (auxMarginal P).toMeasure)
def Sample
For the complete-sample size, the auxiliary-sample size, and the covariate dimension, the experiment's sample space consists of an ordered complete-record array paired with an ordered auxiliary-record array.
def annotationLaw
The two ordered iid arrays are independent and share the population marginal.
Definition (Lean source)
parameters
def
given by
def seedLaw
The randomization seed is uniform on the closed unit interval.
def Rule
Original-record rules are jointly measurable and take values in the closed interval from minus one to one.
Definition (Lean source)
parameters
n m d :
def
Rule n m d :
Type
given by
A data-only statistic becomes a randomized-domain rule by ignoring the independent seed.
def Overlap
Both arm probabilities exceed the public floor on occupied cells.
Definition (Lean source)
parameters
def
Overlap eps P :
Prop
given by
clause 1
eps ≤ propensity P j
clause 2
propensity P j ≤ 1 - eps
def DataProduct
The sample law is the canonical independent product of the two channels.
Definition (Lean source)
theorem annotationLaw_dataProduct
Under the stated inputs and conditions, The canonical experiment realizes the structural independent-sampling atom.
Formal statement
Proof (Lean source)
-- @node: annotationLaw_dataProduct
lemma annotationLaw_dataProduct {d : Nat} (P : DiscreteLaw d) (n m : Nat) :
DataProduct P n m (annotationLaw P n m) := by
rfl
structure ModelClass
For a covariate dimension, an overlap floor, and a population law, the model class is the proposition that the law satisfies the occupied-cell overlap condition.
Definition (Lean source)
A model law pairs its population PMF with occupied-cell overlap membership.
Definition (Lean source)
def ateFunctional
The signed ATE sums cell mass times the treated-minus-control conditional outcome mean.
Definition (Lean source)
parameters
d :
P :
def
given by
theorem jointMass_nonneg
Under the stated inputs and conditions, Every observable atom has nonnegative real mass. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: jointMass_nonneg
lemma jointMass_nonneg {d : Nat} (P : DiscreteLaw d) (j : Fin d) (a y : Bool) :
0 ≤ jointMass P j a y := ENNReal.toReal_nonneg
theorem armMass_nonneg
Under the stated inputs and conditions, Arm masses are nonnegative, including on null cells. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: armMass_nonneg
lemma armMass_nonneg {d : Nat} (P : DiscreteLaw d) (j : Fin d) (a : Bool) :
0 ≤ armMass P j a := Finset.sum_nonneg (fun y _ => jointMass_nonneg P j a y)
theorem cellMass_nonneg
Under the stated inputs and conditions, Covariate masses are nonnegative. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: cellMass_nonneg
lemma cellMass_nonneg {d : Nat} (P : DiscreteLaw d) (j : Fin d) :
0 ≤ cellMass P j := Finset.sum_nonneg (fun a _ => armMass_nonneg P j a)
theorem outcomeMean_mem_Icc
Under the stated inputs and conditions, Binary outcome regressions lie in the unit interval, with zero on null arms. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: outcomeMean_mem_Icc
lemma outcomeMean_mem_Icc {d : Nat} (P : DiscreteLaw d) (a : Bool) (j : Fin d) :
outcomeMean P a j ∈ Icc (0 : Real) 1 := by
have hq : 0 ≤ markedMass P j a := jointMass_nonneg P j a true
have hs := armMass_nonneg P j a
have hqs : markedMass P j a ≤ armMass P j a := by
simp only [armMass, markedMass, Fintype.sum_bool]
linarith [jointMass_nonneg P j a false]
constructor
· exact div_nonneg hq hs
· by_cases hz : armMass P j a = 0
· simp [outcomeMean, hz]
· exact (div_le_one (lt_of_le_of_ne hs (Ne.symm hz))).mpr hqs
theorem sum_cellMass
Under the stated inputs and conditions, The covariate masses exhaust the population probability. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: sum_cellMass
lemma sum_cellMass {d : Nat} (P : DiscreteLaw d) : ∑ j, cellMass P j = 1 := by
have hsum : ∑ z : Obs d, (P.pmf z).toReal = 1 := by
rw [← ENNReal.toReal_sum (fun z _ => P.pmf.apply_ne_top z)]
have hprob : ∑ z : Obs d, P.pmf z = 1 := by
simpa only [tsum_fintype] using P.pmf.tsum_coe
rw [hprob, ENNReal.toReal_one]
simpa only [cellMass, jointMass, Fintype.sum_prod_type] using hsum
theorem ateFunctional_mem_Icc
Under the stated inputs and conditions, The observable signed contrast lies between minus one and one, including null cells.
Formal statement
Proof (Lean source)
-- @node: ateFunctional_mem_Icc
lemma ateFunctional_mem_Icc {d : Nat} (P : DiscreteLaw d) :
ateFunctional P ∈ Icc (-1 : Real) 1 := by
have hdiff (j : Fin d) :
-1 ≤ outcomeMean P true j - outcomeMean P false j ∧
outcomeMean P true j - outcomeMean P false j ≤ 1 := by
have h1 := outcomeMean_mem_Icc P true j
have h0 := outcomeMean_mem_Icc P false j
constructor <;> linarith [h1.1, h1.2, h0.1, h0.2]
constructor
· calc
-1 = ∑ j, cellMass P j * (-1) := by simp [sum_cellMass]
_ ≤ ateFunctional P := Finset.sum_le_sum (fun j _ =>
mul_le_mul_of_nonneg_left (hdiff j).1 (cellMass_nonneg P j))
· calc
ateFunctional P ≤ ∑ j, cellMass P j * 1 := Finset.sum_le_sum (fun j _ =>
mul_le_mul_of_nonneg_left (hdiff j).2 (cellMass_nonneg P j))
_ = 1 := by simp [sum_cellMass]
Squared-error risk integrates over both sample channels and the independent uniform seed.
Definition (Lean source)
parameters
def
given by
Worst-case squared-error risk takes the supremum over the overlap model class.
def minimaxRisk
The original-experiment minimax risk infimizes worst-case risk over clipped Borel randomized rules.
def FullObs
A full atom augments the observed record with binary control and treated potential outcomes.
A potential-outcome law is a PMF on all five coordinates.
def fullMass
The real probability of a full potential-outcome atom is read from its PMF.
Definition (Lean source)
def poAtom
Marginal potential-outcome atom masses sum out the factual outcome.
A full law induces its observed complete-record marginal by coordinate projection.
Definition (Lean source)
parameters
d :
H :
def
given by
⟨H.pmf.map (fun z => (z.1, z.2.1, z.2.2.1))⟩
The causal contrast is the finite expectation of treated minus control potential outcome.
def Consistency
Full atoms with factual outcome different from the selected potential outcome have zero mass.
Definition (Lean source)
def ConditionalExchangeability
The joint potential-outcome pair is independent of treatment given covariates, including null cells.
def AuxTable
Exact auxiliary tables use the product Borel space of real coordinate masses.
def auxTable
The supplied side-information table is read from the actual auxiliary marginal PMF.
Definition (Lean source)
def KnownRule
Exact-table rules are jointly Borel in complete records, the supplied table, and the seed, with clipped output.
def knownRuleRisk
An exact-table rule receives the true population marginal in its squared-error risk.
Definition (Lean source)
parameters
def
given by
The exact-marginal minimax benchmark infimizes worst-case risk over all clipped jointly Borel rules.
Rare-label information is the complete-record budget multiplied by the overlap floor.
The logarithmic diagnostic is the natural logarithm of exp one plus rare-label information.
The rare-label benchmark caps the inverse rare-label information at one.
Definition (Lean source)
def fixedOverlapBenchmark
The fixed-overlap benchmark combines inverse labeled size with the squared annotation approximation term.
def frontierRate
The four-index frontier caps the sum of inverse rare-label scale and the squared marginal approximation ratio.
theorem frontierRate_pos
Under the stated inputs and conditions, The evaluated frontier is strictly positive on the public experiment domain.
Formal statement
Proof (Lean source)
-- @node: frontierRate_pos
lemma frontierRate_pos (n m d : Nat) (eps : Real) (hn : 1 ≤ n) (heps : 0 < eps) :
0 < frontierRate n m d eps := by
have hnreal : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hS : 0 < labelScale n eps := mul_pos hnreal heps
exact lt_min (by norm_num) (add_pos_of_pos_of_nonneg (one_div_pos.mpr hS) (sq_nonneg _))
theorem frontierRate_le_one
Under the stated inputs and conditions, The evaluated frontier is capped at one.
Formal statement
Proof (Lean source)
-- @node: frontierRate_le_one
lemma frontierRate_le_one (n m d : Nat) (eps : Real) : frontierRate n m d eps ≤ 1 := by
exact min_le_left _ _
Allowed public indices; scalar ranges are not additional model fields.
An experiment sequence is an arbitrary sequence of allowed public indices.
Definition (Lean source)
def
ExperimentSeq :
Type
given by
def risk
The risk of a public index evaluates the canonical original-experiment minimax value.
Definition (Lean source)
def consistencyPhase
The consistency phase consists of sequences with minimax risk tending to zero.
Definition (Lean source)
def
The oracle phase requires diverging rare-label information and an eventually bounded risk-to-benchmark ratio.
Definition (Lean source)
def
given by
{v | Tendsto (fun k => labelScale (v k).n (v k).eps) atTop atTop ∧ IsBoundedUnder (· ≤ ·) atTop (fun k => (v k).risk / labelBenchmark (v k).n (v k).eps)}
Strict auxiliary improvement means risk divided by the supervised risk tends to zero.
Definition (Lean source)
def
given by
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.AtomHistogram 20 declarations Singleton factorization and structural zero support for the original raw atom histograms, connecting the embedded count experiment to independent atom counts.
Singleton factorization and structural zero support for the original raw atom histograms, connecting the embedded count experiment to independent atom counts.
Independent atom counts in both original raw channels.
Definition (Lean source)
parameters
n m d :
eps :
sigma :
hyp :
lat :
given by
(Measure.pi fun z : Obs d => poissonMeasure (toNNReal ((affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat z))).prod (Measure.pi fun z : AuxObs d => poissonMeasure (toNNReal ((affineTuning n m d eps).w * ∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y))))
Under the stated inputs and conditions, Raw complete counts have no positive control outcomes. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_rawTable_control_zero
lemma affine_rawTable_control_zero (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) : rawTable hyp n m d eps sigma lat (j, false, true) = 0 := by
simp [rawTable, bernoulliMass]
theorem affine_rawTable_positive_outcome_zero Lemma affine_rawTable_positive_outcome_zero in the paper ↗
Under the stated inputs and conditions, Outside the rare cells, all complete positive outcome atoms are structural zeros. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_rawTable_positive_outcome_zero
lemma affine_rawTable_positive_outcome_zero (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) (hj : (affineTuning n m d eps).Kstar ≤ j.val) (a : Bool) :
rawTable hyp n m d eps sigma lat (j, a, true) = 0 := by
simp [rawTable, not_lt.mpr hj, bernoulliMass]
Under the stated inputs and conditions, Labels beyond the reservoir have zero complete and auxiliary raw mass. This gives the stated result.
Formal statement
hypotheses
conclusion
rawTable hyp n m d eps sigma lat (j, a, y) = 0
Proof (Lean source)
-- @node: affine_rawTable_null
lemma affine_rawTable_null (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) (hj : (affineTuning n m d eps).Kstar < j.val) (a y : Bool) :
rawTable hyp n m d eps sigma lat (j, a, y) = 0 := by
simp [rawTable, not_lt.mpr hj.le, ne_of_gt hj]
theorem affineRawHistogramLaw_singleton
Under the stated inputs and conditions, Singleton probabilities of the full original histogram factor over all atoms. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
conclusion
affineRawHistogramLaw n m d eps sigma hyp lat {h}
= (∏ z : Obs d, poissonMeasure (toNNReal ((affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat z)) {h.1 z}) * (∏ z : AuxObs d, poissonMeasure (toNNReal ((affineTuning n m d eps).w * ∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y))) {h.2 z})
Proof (Lean source)
-- @node: affineRawHistogramLaw_singleton
lemma affineRawHistogramLaw_singleton (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(h : (Obs d → Nat) × (AuxObs d → Nat)) :
affineRawHistogramLaw n m d eps sigma hyp lat {h} =
(∏ z : Obs d, poissonMeasure
(toNNReal ((affineTuning n m d eps).u *
rawTable hyp n m d eps sigma lat z)) {h.1 z}) *
(∏ z : AuxObs d, poissonMeasure
(toNNReal ((affineTuning n m d eps).w *
∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y))) {h.2 z}) := by
unfold affineRawHistogramLaw
rw [← Set.singleton_prod_singleton, Measure.prod_prod,
Measure.pi_singleton, Measure.pi_singleton]
Under the stated inputs and conditions, A zero-intensity atom contributes one exactly at count zero. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_poisson_zero_singleton
lemma affine_poisson_zero_singleton (r : Nat) :
poissonMeasure 0 {r} = if r = 0 then 1 else 0 := by
by_cases hr : r = 0
· subst r; simp [poissonMeasure_singleton]
· simp [poissonMeasure_singleton, hr, zero_pow hr]
The six atom factors at a single original covariate label.
Definition (Lean source)
parameters
def
given by
(∏ ay : Bool × Bool, poissonMeasure (toNNReal ((affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat (j, ay))) {h.1 (j, ay)}) * (∏ a : Bool, poissonMeasure (toNNReal ((affineTuning n m d eps).w * ∑ y : Bool, rawTable hyp n m d eps sigma lat (j, a, y))) {h.2 (j, a)})
theorem affineRawHistogramLaw_singleton_by_label
Under the stated inputs and conditions, Grouping by label preserves every original complete and auxiliary atom. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
conclusion
affineRawHistogramLaw n m d eps sigma hyp lat {h}
= ∏ j, affineRawCellSingleton n m d eps sigma hyp lat h j
Proof (Lean source)
-- @node: affineRawHistogramLaw_singleton_by_label
lemma affineRawHistogramLaw_singleton_by_label (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(h : (Obs d → Nat) × (AuxObs d → Nat)) :
affineRawHistogramLaw n m d eps sigma hyp lat {h} =
∏ j, affineRawCellSingleton n m d eps sigma hyp lat h j := by
rw [affineRawHistogramLaw_singleton]
simp only [Obs, AuxObs, Fintype.prod_prod_type]
rw [← Finset.prod_mul_distrib]
simp only [affineRawCellSingleton, Fintype.prod_prod_type]
theorem affineRawCellSingleton_null
Under the stated inputs and conditions, Every unused label contributes only deterministic zero counts. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
hs :
affineHistogramSupported d (affineTuning n m d eps).Kstar h
j :
Fin d
hj :
(affineTuning n m d eps).Kstar < j.val
conclusion
affineRawCellSingleton n m d eps sigma hyp lat h j = 1
Proof (Lean source)
-- @node: affineRawCellSingleton_null
lemma affineRawCellSingleton_null (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(h : (Obs d → Nat) × (AuxObs d → Nat))
(hs : affineHistogramSupported d (affineTuning n m d eps).Kstar h)
(j : Fin d) (hj : (affineTuning n m d eps).Kstar < j.val) :
affineRawCellSingleton n m d eps sigma hyp lat h j = 1 := by
simp [affineRawCellSingleton, affine_rawTable_null n m d eps sigma hyp lat j hj,
(hs.2.2 j hj).1, (hs.2.2 j hj).2, affine_poisson_zero_singleton]
Under the stated inputs and conditions, A finite label product with unit tail consists of its rare prefix and reservoir. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_label_product_prefix
lemma affine_label_product_prefix {M : Type*} [CommMonoid M] (d k : Nat)
(hk : k < d) (f : Fin d → M) (ht : ∀ j, k < j.val → f j = 1) :
(∏ j, f j) = (∏ i : Fin k, f ⟨i.val, lt_trans i.isLt hk⟩) * f ⟨k, hk⟩ := by
let e : Fin (k + 1) → Fin d := fun i => ⟨i.val, lt_of_lt_of_le i.isLt hk⟩
have he : Injective e := fun i j hij => Fin.ext (congrArg (fun z : Fin d => z.val) hij)
have htail : ∀ j ∉ range e, f j = 1 := by
intro j hj
apply ht
by_contra hle
apply hj
exact ⟨⟨j.val, by omega⟩, rfl⟩
rw [← Fintype.prod_of_injective e he (fun i => f (e i)) f htail (fun _ => rfl),
Fin.prod_univ_castSucc]
rfl
theorem affineRawCellSingleton_rare
Under the stated inputs and conditions, At a rare label the original atom factors are exactly its five-coordinate law. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
hk :
(affineTuning n m d eps).Kstar < d
hs :
affineHistogramSupported d (affineTuning n m d eps).Kstar h
i :
Fin (affineTuning n m d eps).Kstar
conclusion
affineRawCellSingleton n m d eps sigma hyp lat h ⟨i.val, lt_trans i.isLt hk⟩
= affineCellFullPoissonLaw sigma (affineTuning n m d eps).u (affineTuning n m d eps).w hyp (lat i) {(affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h).1 i}
Proof (Lean source)
-- @node: affineRawCellSingleton_rare
lemma affineRawCellSingleton_rare (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(hk : (affineTuning n m d eps).Kstar < d)
(h : (Obs d → Nat) × (AuxObs d → Nat))
(hs : affineHistogramSupported d (affineTuning n m d eps).Kstar h)
(i : Fin (affineTuning n m d eps).Kstar) :
affineRawCellSingleton n m d eps sigma hyp lat h ⟨i.val, lt_trans i.isLt hk⟩ =
affineCellFullPoissonLaw sigma (affineTuning n m d eps).u
(affineTuning n m d eps).w hyp (lat i)
{(affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h).1 i} := by
rw [affineCellFullPoissonLaw_raw_atoms n m d eps sigma hyp lat
⟨i.val, lt_trans i.isLt hk⟩ i.isLt]
simp only [affineHistogramCoordinates, affineRawCellSingleton,
Fintype.prod_prod_type, Fintype.prod_bool,
← Set.singleton_prod_singleton, Measure.prod_prod]
simp only [affine_rawTable_control_zero, mul_zero, Real.toNNReal_zero,
hs.1, affine_poisson_zero_singleton, if_pos rfl, one_mul]
ac_rfl
theorem affineRawCellSingleton_reservoir
Under the stated inputs and conditions, At the reservoir label the original atom factors retain both channels and arms. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
hk :
(affineTuning n m d eps).Kstar < d
hs :
affineHistogramSupported d (affineTuning n m d eps).Kstar h
conclusion
affineRawCellSingleton n m d eps sigma hyp lat h ⟨(affineTuning n m d eps).Kstar, hk⟩
= affineReservoirCountLaw n m d eps sigma {(affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h).2}
Proof (Lean source)
-- @node: affineRawCellSingleton_reservoir
lemma affineRawCellSingleton_reservoir (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(hk : (affineTuning n m d eps).Kstar < d)
(h : (Obs d → Nat) × (AuxObs d → Nat))
(hs : affineHistogramSupported d (affineTuning n m d eps).Kstar h) :
affineRawCellSingleton n m d eps sigma hyp lat h ⟨(affineTuning n m d eps).Kstar, hk⟩ =
affineReservoirCountLaw n m d eps sigma
{(affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h).2} := by
rw [affineReservoirCountLaw_raw_atoms n m d eps sigma hyp lat
⟨(affineTuning n m d eps).Kstar, hk⟩ rfl]
simp only [affineHistogramCoordinates, affineRawCellSingleton,
Fintype.prod_prod_type, Fintype.prod_bool,
← Set.singleton_prod_singleton, Measure.prod_prod]
simp only [affine_rawTable_positive_outcome_zero n m d eps sigma hyp lat
⟨(affineTuning n m d eps).Kstar, hk⟩ (le_refl _) , mul_zero,
Real.toNNReal_zero, hs.1, hs.2.1 ⟨(affineTuning n m d eps).Kstar, hk⟩ (le_refl _),
affine_poisson_zero_singleton, if_pos rfl, one_mul]
ac_rfl
theorem affineHistogramKernel_raw_supported_singleton
Under the stated inputs and conditions, Supported histogram singleton probabilities agree in both count constructions. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
hk :
(affineTuning n m d eps).Kstar < d
hs :
affineHistogramSupported d (affineTuning n m d eps).Kstar h
conclusion
affineHistogramKernel n m d eps sigma hyp lat {h}
= affineRawHistogramLaw n m d eps sigma hyp lat {h}
Proof (Lean source)
-- @node: affineHistogramKernel_raw_supported_singleton
lemma affineHistogramKernel_raw_supported_singleton (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(hk : (affineTuning n m d eps).Kstar < d)
(h : (Obs d → Nat) × (AuxObs d → Nat))
(hs : affineHistogramSupported d (affineTuning n m d eps).Kstar h) :
affineHistogramKernel n m d eps sigma hyp lat {h} =
affineRawHistogramLaw n m d eps sigma hyp lat {h} := by
rw [affineHistogramKernel_supported_singleton n m d eps sigma hyp lat hk h hs,
affineRawHistogramLaw_singleton_by_label,
affine_label_product_prefix d _ hk _ (affineRawCellSingleton_null n m d eps sigma hyp lat h hs),
affineRawCellSingleton_reservoir n m d eps sigma hyp lat hk h hs]
congr 1
apply Finset.prod_congr rfl
intro i _
exact (affineRawCellSingleton_rare n m d eps sigma hyp lat hk h hs i).symm
theorem affineRawHistogramLaw_complete_zero
Under the stated inputs and conditions, A nonzero count at a zero complete intensity makes the whole atom mass zero. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
z :
Obs d
hz :
rawTable hyp n m d eps sigma lat z = 0
hc :
h.1 z ≠ 0
conclusion
affineRawHistogramLaw n m d eps sigma hyp lat {h} = 0
Proof (Lean source)
-- @node: affineRawHistogramLaw_complete_zero
lemma affineRawHistogramLaw_complete_zero (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(h : (Obs d → Nat) × (AuxObs d → Nat)) (z : Obs d)
(hz : rawTable hyp n m d eps sigma lat z = 0) (hc : h.1 z ≠ 0) :
affineRawHistogramLaw n m d eps sigma hyp lat {h} = 0 := by
rw [affineRawHistogramLaw_singleton]
have hp : (∏ z : Obs d, poissonMeasure
(toNNReal ((affineTuning n m d eps).u *
rawTable hyp n m d eps sigma lat z)) {h.1 z}) = 0 := by
apply Finset.prod_eq_zero (Finset.mem_univ z)
simp [hz, affine_poisson_zero_singleton, hc]
rw [hp, zero_mul]
theorem affineRawHistogramLaw_auxiliary_zero
Under the stated inputs and conditions, A nonzero count at a zero auxiliary intensity makes the whole atom mass zero. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
z :
AuxObs d
hz :
∀ y, rawTable hyp n m d eps sigma lat (z.1, z.2, y) = 0
hc :
h.2 z ≠ 0
conclusion
affineRawHistogramLaw n m d eps sigma hyp lat {h} = 0
Proof (Lean source)
-- @node: affineRawHistogramLaw_auxiliary_zero
lemma affineRawHistogramLaw_auxiliary_zero (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(h : (Obs d → Nat) × (AuxObs d → Nat)) (z : AuxObs d)
(hz : ∀ y, rawTable hyp n m d eps sigma lat (z.1, z.2, y) = 0)
(hc : h.2 z ≠ 0) :
affineRawHistogramLaw n m d eps sigma hyp lat {h} = 0 := by
rw [affineRawHistogramLaw_singleton]
have hp : (∏ z : AuxObs d, poissonMeasure
(toNNReal ((affineTuning n m d eps).w *
∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y))) {h.2 z}) = 0 := by
apply Finset.prod_eq_zero (Finset.mem_univ z)
simp [hz, affine_poisson_zero_singleton, hc]
rw [hp, mul_zero]
theorem affineRawHistogramLaw_unsupported_singleton
Under the stated inputs and conditions, Unsupported histograms have probability zero also in the independent atom law. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
hs :
¬ affineHistogramSupported d (affineTuning n m d eps).Kstar h
conclusion
affineRawHistogramLaw n m d eps sigma hyp lat {h} = 0
Proof (Lean source)
-- @node: affineRawHistogramLaw_unsupported_singleton
lemma affineRawHistogramLaw_unsupported_singleton (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(h : (Obs d → Nat) × (AuxObs d → Nat))
(hs : ¬ affineHistogramSupported d (affineTuning n m d eps).Kstar h) :
affineRawHistogramLaw n m d eps sigma hyp lat {h} = 0 := by
by_contra hne
apply hs
refine ⟨?_, ?_, ?_⟩
· intro j
by_contra hc
exact hne (affineRawHistogramLaw_complete_zero n m d eps sigma hyp lat h
(j, false, true) (affine_rawTable_control_zero n m d eps sigma hyp lat j) hc)
· intro j hj
by_contra hc
exact hne (affineRawHistogramLaw_complete_zero n m d eps sigma hyp lat h
(j, true, true) (affine_rawTable_positive_outcome_zero n m d eps sigma hyp lat j hj true) hc)
· intro j hj
constructor
· intro a y
by_contra hc
exact hne (affineRawHistogramLaw_complete_zero n m d eps sigma hyp lat h
(j, a, y) (affine_rawTable_null n m d eps sigma hyp lat j hj a y) hc)
· intro a
by_contra hc
exact hne (affineRawHistogramLaw_auxiliary_zero n m d eps sigma hyp lat h
(j, a) (fun y => affine_rawTable_null n m d eps sigma hyp lat j hj a y) hc)
theorem affineHistogramKernel_eq_raw_atom_law
Under the stated inputs and conditions, The embedded conditional law equals the full independent raw atom histogram law. This gives the stated result.
Formal statement
hypotheses
conclusion
affineHistogramKernel n m d eps sigma hyp lat
= affineRawHistogramLaw n m d eps sigma hyp lat
Proof (Lean source)
-- @node: affineHistogramKernel_eq_raw_atom_law
lemma affineHistogramKernel_eq_raw_atom_law (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(hk : (affineTuning n m d eps).Kstar < d) :
affineHistogramKernel n m d eps sigma hyp lat =
affineRawHistogramLaw n m d eps sigma hyp lat := by
apply Measure.ext_of_singleton
intro h
by_cases hs : affineHistogramSupported d (affineTuning n m d eps).Kstar h
· exact affineHistogramKernel_raw_supported_singleton n m d eps sigma hyp lat hk h hs
· rw [affineHistogramKernel_singleton n m d eps sigma hyp lat hk h, if_neg hs,
affineRawHistogramLaw_unsupported_singleton n m d eps sigma hyp lat h hs]
Under the stated inputs and conditions, The selected reservoir fits inside the original public alphabet. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_tuning_Kstar_lt_dimension
lemma affine_tuning_Kstar_lt_dimension (n m d : Nat) (eps : Real) (hd : 2 ≤ d) :
(affineTuning n m d eps).Kstar < d := by
have hk : (affineTuning n m d eps).Kstar ≤ d - 1 := Nat.min_le_left _ _
omega
theorem affineHistogramKernel_eq_normalized_counts
Under the stated inputs and conditions, The conditional histogram is exactly the normalized random-scale atom count law. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
t :
affineTuning n m d eps
P :
normalizedLaw hyp n m d eps hd sigma lat
Q :
rawNormalizer n m d eps sigma lat
affineHistogramKernel n m d eps sigma hyp lat
= (independentPoissonCountLaw P.pmf.toMeasure (toNNReal (t.u * Q))).prod (independentPoissonCountLaw (auxMarginal P).toMeasure (toNNReal (t.w * Q)))
Proof (Lean source)
-- @node: affineHistogramKernel_eq_normalized_counts
lemma affineHistogramKernel_eq_normalized_counts (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
let t := affineTuning n m d eps
let P := normalizedLaw hyp n m d eps hd sigma lat
let Q := rawNormalizer n m d eps sigma lat
affineHistogramKernel n m d eps sigma hyp lat =
(independentPoissonCountLaw
P.pmf.toMeasure (toNNReal (t.u * Q))).prod
(independentPoissonCountLaw
(auxMarginal P).toMeasure (toNNReal (t.w * Q))) := by
dsimp only
rw [affine_complete_countLaw_readback n m d eps hn hd heps heps' sigma hyp lat,
affine_auxiliary_countLaw_readback n m d eps hn hd heps heps' sigma hyp lat]
exact affineHistogramKernel_eq_raw_atom_law n m d eps sigma hyp lat
(affine_tuning_Kstar_lt_dimension n m d eps hd)
theorem affineHistogramKernel_ordered_apply
Under the stated inputs and conditions, Ordering the actual conditional histogram gives precisely the two independent normalized iid channels at their shared random scale. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
t :
affineTuning n m d eps
P :
normalizedLaw hyp n m d eps hd sigma lat
Q :
rawNormalizer n m d eps sigma lat
(pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₖ affineHistogramKernel n m d eps sigma hyp) lat
= (finitePoissonSampleLaw P.pmf.toMeasure (toNNReal (t.u * Q))).prod (finitePoissonSampleLaw (auxMarginal P).toMeasure (toNNReal (t.w * Q)))
Proof (Lean source)
-- @node: affineHistogramKernel_ordered_apply
lemma affineHistogramKernel_ordered_apply (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
let t := affineTuning n m d eps
let P := normalizedLaw hyp n m d eps hd sigma lat
let Q := rawNormalizer n m d eps sigma lat
(pairedHistogramReconstructionKernel
(Obs d) (AuxObs d) ∘ₖ affineHistogramKernel n m d eps sigma hyp) lat =
(finitePoissonSampleLaw
P.pmf.toMeasure (toNNReal (t.u * Q))).prod
(finitePoissonSampleLaw
(auxMarginal P).toMeasure (toNNReal (t.w * Q))) := by
dsimp only
rw [Kernel.comp_apply, affineHistogramKernel_eq_raw_atom_law n m d eps sigma hyp lat
(affine_tuning_Kstar_lt_dimension n m d eps hd)]
exact affine_raw_atom_counts_reconstruction n m d eps hn hd heps heps' sigma hyp lat
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.AtomLaw 5 declarations Exact raw atom intensities for the rare-cell and reservoir factors, and identification of the normalized random-scale independent atom count laws.
Exact raw atom intensities for the rare-cell and reservoir factors, and identification of the normalized random-scale independent atom count laws.
theorem affineCellFullPoissonLaw_raw_atoms
Under the stated inputs and conditions, The five independent coordinates of a rare cell have exactly the raw complete and auxiliary atom intensities at its original label. This gives the stated result.
Formal statement
hypotheses
conclusion
t :
affineTuning n m d eps
raw :
rawTable hyp n m d eps sigma lat
affineCellFullPoissonLaw sigma t.u t.w hyp (lat ⟨j.val, hj⟩)
= ((poissonMeasure (toNNReal (t.u * raw (j, true, true)))).prod (poissonMeasure (toNNReal (t.u * raw (j, true, false))))).prod ((poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, true, y)))).prod ((poissonMeasure (toNNReal (t.u * raw (j, false, false)))).prod (poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, false, y))))))
Proof (Lean source)
-- @node: affineCellFullPoissonLaw_raw_atoms
lemma affineCellFullPoissonLaw_raw_atoms (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) (hj : j.val < (affineTuning n m d eps).Kstar) :
let t := affineTuning n m d eps
let raw := rawTable hyp n m d eps sigma lat
affineCellFullPoissonLaw sigma t.u t.w hyp (lat ⟨j.val, hj⟩) =
((poissonMeasure (toNNReal (t.u * raw (j, true, true)))).prod
(poissonMeasure (toNNReal (t.u * raw (j, true, false))))).prod
((poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, true, y)))).prod
((poissonMeasure (toNNReal (t.u * raw (j, false, false)))).prod
(poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, false, y)))))) := by
dsimp only
have halpha : (affineTuning n m d eps).alpha =
(affineTuning n m d eps).B / (100 * ((affineTuning n m d eps).L : Real) ^ 2) := rfl
have hb0 : (affineTuning n m d eps).b0 =
(affineTuning n m d eps).alpha / eps := rfl
cases hlat : lat ⟨j.val, hj⟩ <;>
simp [affineCellFullPoissonLaw, rawTable, hj, bernoulliMass,
← halpha, ← hb0, hlat, mul_assoc]
congr 2
congr 1
congr 1
ring
theorem affineReservoirCountLaw_raw_atoms
Under the stated inputs and conditions, The independent reservoir coordinates have the same raw atom intensities in both hypotheses, including the full auxiliary arm marginal. This gives the stated result.
Formal statement
hypotheses
conclusion
t :
affineTuning n m d eps
raw :
rawTable hyp n m d eps sigma lat
affineReservoirCountLaw n m d eps sigma
= ((poissonMeasure (toNNReal (t.u * raw (j, true, false)))).prod (poissonMeasure (toNNReal (t.u * raw (j, false, false))))).prod ((poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, true, y)))).prod (poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, false, y)))))
Proof (Lean source)
-- @node: affineReservoirCountLaw_raw_atoms
lemma affineReservoirCountLaw_raw_atoms (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) (hj : j.val = (affineTuning n m d eps).Kstar) :
let t := affineTuning n m d eps
let raw := rawTable hyp n m d eps sigma lat
affineReservoirCountLaw n m d eps sigma =
((poissonMeasure (toNNReal (t.u * raw (j, true, false)))).prod
(poissonMeasure (toNNReal (t.u * raw (j, false, false))))).prod
((poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, true, y)))).prod
(poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (j, false, y))))) := by
dsimp only
simp [affineReservoirCountLaw, rawTable, hj, bernoulliMass]
Under the stated hypotheses, Scaling the normalized complete atom law by its random total gives the independent Poisson product at the original raw complete intensities. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
independentPoissonCountLaw (normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure (toNNReal ((affineTuning n m d eps).u * rawNormalizer n m d eps sigma lat))
= Measure.pi (fun z : Obs d => poissonMeasure (toNNReal ((affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat z)))
Proof (Lean source)
-- @node: affine_complete_countLaw_readback
lemma affine_complete_countLaw_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
independentPoissonCountLaw (normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure
(toNNReal ((affineTuning n m d eps).u * rawNormalizer n m d eps sigma lat)) =
Measure.pi (fun z : Obs d => poissonMeasure
(toNNReal ((affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat z))) := by
unfold independentPoissonCountLaw
congr 1
funext z
congr 1
rw [PMF.toMeasure_apply_singleton _ _ (measurableSet_singleton _)]
apply NNReal.eq
rw [NNReal.coe_mul]
have hQ := affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat
have hu : (affineTuning n m d eps).u = 128 * (n : Real) := rfl
rw [Real.coe_toNNReal _ (by rw [hu]; positivity),
Real.coe_toNNReal _ (mul_nonneg (by rw [hu]; positivity)
(affine_rawTable_nonneg n m d eps hn hd heps heps' sigma hyp lat z))]
exact affine_complete_intensity_readback n m d eps hn hd heps heps' sigma hyp lat z
theorem affine_auxiliary_countLaw_readback
Under the stated hypotheses, Scaling the normalized auxiliary law by the same total retains both arm marginals and gives the independent Poisson product at their raw intensities. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
independentPoissonCountLaw (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure (toNNReal ((affineTuning n m d eps).w * rawNormalizer n m d eps sigma lat))
= Measure.pi (fun z : AuxObs d => poissonMeasure (toNNReal ((affineTuning n m d eps).w * ∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y))))
Proof (Lean source)
-- @node: affine_auxiliary_countLaw_readback
lemma affine_auxiliary_countLaw_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
independentPoissonCountLaw (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure
(toNNReal ((affineTuning n m d eps).w * rawNormalizer n m d eps sigma lat)) =
Measure.pi (fun z : AuxObs d => poissonMeasure
(toNNReal ((affineTuning n m d eps).w *
∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y)))) := by
unfold independentPoissonCountLaw
congr 1
funext z
congr 1
rw [PMF.toMeasure_apply_singleton _ _ (measurableSet_singleton _)]
apply NNReal.eq
rw [NNReal.coe_mul]
have hQ := affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat
have hw : (affineTuning n m d eps).w = 128 * ((n : Real) + m) := rfl
rw [Real.coe_toNNReal _ (by rw [hw]; positivity),
Real.coe_toNNReal _ (mul_nonneg (by rw [hw]; positivity)
(Finset.sum_nonneg fun y _ =>
affine_rawTable_nonneg n m d eps hn hd heps heps' sigma hyp lat (z.1, z.2, y)))]
exact affine_auxiliary_intensity_readback n m d eps hn hd heps heps' sigma hyp lat z
theorem affine_raw_atom_counts_reconstruction
Under the stated inputs and conditions, Uniformly ordering the independent raw atom counts recovers the two independent normalized iid Poisson channels at their actual random totals. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
t :
affineTuning n m d eps
P :
normalizedLaw hyp n m d eps hd sigma lat
Q :
rawNormalizer n m d eps sigma lat
raw :
rawTable hyp n m d eps sigma lat
pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₘ ((Measure.pi fun z : Obs d => poissonMeasure (toNNReal (t.u * raw z))).prod (Measure.pi fun z : AuxObs d => poissonMeasure (toNNReal (t.w * ∑ y : Bool, raw (z.1, z.2, y)))))
= (finitePoissonSampleLaw P.pmf.toMeasure (toNNReal (t.u * Q))).prod (finitePoissonSampleLaw (auxMarginal P).toMeasure (toNNReal (t.w * Q)))
Proof (Lean source)
-- @node: affine_raw_atom_counts_reconstruction
lemma affine_raw_atom_counts_reconstruction (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
let t := affineTuning n m d eps
let P := normalizedLaw hyp n m d eps hd sigma lat
let Q := rawNormalizer n m d eps sigma lat
let raw := rawTable hyp n m d eps sigma lat
pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₘ
((Measure.pi fun z : Obs d => poissonMeasure (toNNReal (t.u * raw z))).prod
(Measure.pi fun z : AuxObs d => poissonMeasure
(toNNReal (t.w * ∑ y : Bool, raw (z.1, z.2, y))))) =
(finitePoissonSampleLaw
P.pmf.toMeasure (toNNReal (t.u * Q))).prod
(finitePoissonSampleLaw
(auxMarginal P).toMeasure (toNNReal (t.w * Q))) := by
dsimp only
rw [← affine_complete_countLaw_readback n m d eps hn hd heps heps' sigma hyp lat,
← affine_auxiliary_countLaw_readback n m d eps hn hd heps heps' sigma hyp lat]
exact pairedIndependentPoissonCountLaw_comp_reconstruction _ _ _ _
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.CellPredictive 14 declarations The actual finite core/filler mixture for one rare cell in the raw Poisson experiment, with singleton readback and exact signed outcome cancellation.
The actual finite core/filler mixture for one rare cell in the raw Poisson experiment, with singleton readback and exact signed outcome cancellation.
The four counts retain both treated outcome marks, auxiliary treated counts, and the combined control count. The filler has zero treated outcome mean.
Definition (Lean source)
parameters
def
given by
alpha :
B / (100 * (L : Real) ^ 2)
s1 :
alpha + (1 - eps) * latentNode sigma z
s0 :
(1 - eps) * (alpha / eps) + eps * latentNode sigma z
mean :
match z with | none
=> 0 | some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2
((poissonMeasure (toNNReal (u * s1 * mean))).prod (poissonMeasure (toNNReal (u * s1 * (1 - mean))))).prod ((poissonMeasure (toNNReal (w * s1))).prod (poissonMeasure (toNNReal ((u + w) * s0))))
instance affineCellPoissonLaw_isProbabilityMeasure
Each conditional raw cell law is a probability measure.
Definition (Lean source)
-- @node: affineCellPoissonLaw_isProbabilityMeasure
instance affineCellPoissonLaw_isProbabilityMeasure {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) (z : Latent L) :
IsProbabilityMeasure (affineCellPoissonLaw sigma u w hyp z) := by
unfold affineCellPoissonLaw
infer_instance
Mix the conditional raw count law over the specified finite latent PMF.
Definition (Lean source)
parameters
def
given by
instance affineCellPoissonPredictive_isProbabilityMeasure
The finite latent mixture has total mass one.
Definition (Lean source)
-- @node: affineCellPoissonPredictive_isProbabilityMeasure
instance affineCellPoissonPredictive_isProbabilityMeasure {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) :
IsProbabilityMeasure (affineCellPoissonPredictive sigma u w hyp) := by
constructor
simp only [affineCellPoissonPredictive, Measure.finsetSum_apply,
Measure.smul_apply, measure_univ, smul_eq_mul, mul_one]
exact (tsum_fintype _).symm.trans (latentPMF sigma).tsum_coe
theorem affineCellPoissonPredictive_real_singleton
Under the stated inputs and conditions, Singleton masses of the finite mixture are the finite weighted sums of conditional singleton masses. This gives the stated result.
Formal statement
hypotheses
conclusion
(affineCellPoissonPredictive sigma u w hyp).real {o}
Proof (Lean source)
-- @node: affineCellPoissonPredictive_real_singleton
lemma affineCellPoissonPredictive_real_singleton {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) (o : MarkedPoissonObservation) :
(affineCellPoissonPredictive sigma u w hyp).real {o} =
∑ z : Latent L, (latentPMF sigma z).toReal *
(affineCellPoissonLaw sigma u w hyp z).real {o} := by
simp only [affineCellPoissonPredictive, measureReal_def, Measure.finsetSum_apply,
Measure.smul_apply, smul_eq_mul]
rw [ENNReal.toReal_sum (fun z _ => by
exact ENNReal.mul_ne_top (PMF.apply_ne_top _ _) (measure_ne_top _ _))]
simp only [ENNReal.toReal_mul]
theorem affineCellPoissonLaw_filler_eq
Under the stated inputs and conditions, The filler is identical under the two hypotheses. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affineCellPoissonLaw_filler_eq
lemma affineCellPoissonLaw_filler_eq {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) :
affineCellPoissonLaw sigma u w true none =
affineCellPoissonLaw sigma u w false none := by
rfl
theorem affineCellPoissonLaw_core_target_sub
Under the stated inputs and conditions, On core atoms, a nonzero treated count in one outcome channel has the signed polar discrepancy times the three unmarked Poisson singleton masses. This gives the stated result.
Formal statement
hypotheses
conclusion
= (if first then latentPolar sigma (some i) else -latentPolar sigma (some i)) * (poissonMeasure (toNNReal (u * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {r + 1} * (poissonMeasure (toNNReal (w * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {g} * (poissonMeasure (toNNReal ((u + w) * ((1 - eps) * (B / (100 * (L : Real) ^ 2) / eps) + eps * sigma.nodes i)))).real {c}
Proof (Lean source)
-- @node: affineCellPoissonLaw_core_target_sub
lemma affineCellPoissonLaw_core_target_sub {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (i : Fin (L + 2))
(first : Bool) (r g c : Nat) :
(affineCellPoissonLaw sigma u w true (some i)).real
{palmSplitTarget first r g c} -
(affineCellPoissonLaw sigma u w false (some i)).real
{palmSplitTarget first r g c} =
(if first then latentPolar sigma (some i) else -latentPolar sigma (some i)) *
(poissonMeasure (toNNReal
(u * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {r + 1} *
(poissonMeasure (toNNReal
(w * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {g} *
(poissonMeasure (toNNReal ((u + w) *
((1 - eps) * (B / (100 * (L : Real) ^ 2) / eps) + eps * sigma.nodes i)))).real {c} := by
cases first <;>
simp only [affineCellPoissonLaw, latentNode, latentPolar, Bool.false_eq_true,
if_false, if_true] <;> split_ifs <;>
simp [palmSplitTarget, prod_real_singleton, poissonMeasure_real_singleton,
zero_pow (by omega : 1 + r ≠ 0)] <;> ring
theorem affineCellPoissonPredictive_target_sub
Under the stated inputs and conditions, Mixing the core atoms cancels the filler and restores the signed certificate weights. This is the measure-level signed identity behind equation (6). This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
ha :
0 < B / (100 * (L : Real) ^ 2)
he :
eps < 1
first :
r g c :
conclusion
= (if first then 1 else -1) * ∑ i, sigma.weights i * ((B / (100 * (L : Real) ^ 2) / (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)) * (poissonMeasure (toNNReal (u * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {r + 1} * (poissonMeasure (toNNReal (w * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {g} * (poissonMeasure (toNNReal ((u + w) * ((1 - eps) * (B / (100 * (L : Real) ^ 2) / eps) + eps * sigma.nodes i)))).real {c})
Proof (Lean source)
-- @node: affineCellPoissonPredictive_target_sub
lemma affineCellPoissonPredictive_target_sub {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : eps < 1)
(first : Bool) (r g c : Nat) :
(affineCellPoissonPredictive sigma u w true).real {palmSplitTarget first r g c} -
(affineCellPoissonPredictive sigma u w false).real {palmSplitTarget first r g c} =
(if first then 1 else -1) * ∑ i, sigma.weights i *
((B / (100 * (L : Real) ^ 2) /
(B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)) *
(poissonMeasure (toNNReal
(u * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {r + 1} *
(poissonMeasure (toNNReal
(w * (B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)))).real {g} *
(poissonMeasure (toNNReal ((u + w) *
((1 - eps) * (B / (100 * (L : Real) ^ 2) / eps) + eps * sigma.nodes i)))).real {c}) := by
rw [affineCellPoissonPredictive_real_singleton, affineCellPoissonPredictive_real_singleton,
← Finset.sum_sub_distrib]
simp only [Fintype.sum_option, affineCellPoissonLaw_filler_eq, sub_self, zero_add]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
rw [← mul_sub, affineCellPoissonLaw_core_target_sub,
affine_latent_mass_readback sigma ha he]
dsimp only [latentCoreMass]
conv_rhs => rw [← affine_latent_polar_weight sigma i]
cases first <;> simp only [Bool.false_eq_true, if_false, if_true] <;> ring
theorem affineCellPoissonLaw_no_labeled_eq
Under the stated inputs and conditions, With no complete treated observations, both core hypotheses have the same likelihood, including the shared auxiliary and control counts. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineCellPoissonLaw_no_labeled_eq
lemma affineCellPoissonLaw_no_labeled_eq {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (z : Latent L) (g c : Nat) :
(affineCellPoissonLaw sigma u w true z).real {((0, 0), (g, c))} =
(affineCellPoissonLaw sigma u w false z).real {((0, 0), (g, c))} := by
cases z with
| none => rfl
| some i =>
simp only [affineCellPoissonLaw, latentNode, latentPolar, Bool.false_eq_true,
if_false, if_true]
split_ifs <;>
simp [prod_real_singleton, poissonMeasure_real_singleton] <;> ring
theorem affineCellPoissonPredictive_no_labeled_eq
Under the stated inputs and conditions, With zero treated complete counts, the predictive singleton masses agree. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineCellPoissonPredictive_no_labeled_eq
lemma affineCellPoissonPredictive_no_labeled_eq {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (g c : Nat) :
(affineCellPoissonPredictive sigma u w true).real {((0, 0), (g, c))} =
(affineCellPoissonPredictive sigma u w false).real {((0, 0), (g, c))} := by
simp only [affineCellPoissonPredictive_real_singleton,
affineCellPoissonLaw_no_labeled_eq]
theorem affineCellPoissonLaw_mixed_zero
Under the stated inputs and conditions, Deterministic marks forbid a positive count in both complete treated outcome channels, under every core or filler draw. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineCellPoissonLaw_mixed_zero
lemma affineCellPoissonLaw_mixed_zero {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) (z : Latent L)
(r s g c : Nat) :
(affineCellPoissonLaw sigma u w hyp z).real {((r + 1, s + 1), (g, c))} = 0 := by
cases z with
| none =>
simp [affineCellPoissonLaw, prod_real_singleton, poissonMeasure_real_singleton,
zero_pow (by omega : r + 1 ≠ 0)]
| some i =>
cases hyp <;>
simp only [affineCellPoissonLaw, latentNode, latentPolar, Bool.false_eq_true,
if_false, if_true] <;> split_ifs <;>
simp [prod_real_singleton, poissonMeasure_real_singleton,
zero_pow (by omega : r + 1 ≠ 0), zero_pow (by omega : s + 1 ≠ 0)]
theorem affineCellPoissonPredictive_mixed_zero
Under the stated inputs and conditions, Mixed positive complete treated outcome counts have zero predictive mass. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineCellPoissonPredictive_mixed_zero
lemma affineCellPoissonPredictive_mixed_zero {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) (r s g c : Nat) :
(affineCellPoissonPredictive sigma u w hyp).real {((r + 1, s + 1), (g, c))} = 0 := by
simp only [affineCellPoissonPredictive_real_singleton,
affineCellPoissonLaw_mixed_zero, mul_zero, Finset.sum_const_zero]
theorem affine_three_poisson_singleton_readback Lemma affine_three_poisson_singleton_readback in the paper ↗
Under the stated inputs and conditions, The three independent unmarked Poisson singleton masses give the exponential times powers likelihood in equation (6). This gives the stated result.
Formal statement
hypotheses
b0 eps u w v :
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
hv :
0 ≤ v
r g c :
conclusion
(poissonMeasure (toNNReal (u * (eps * b0 + (1 - eps) * v)))).real {r + 1} * (poissonMeasure (toNNReal (w * (eps * b0 + (1 - eps) * v)))).real {g} * (poissonMeasure (toNNReal ((u + w) * ((1 - eps) * b0 + eps * v)))).real {c}
Proof (Lean source)
-- @node: affine_three_poisson_singleton_readback
lemma affine_three_poisson_singleton_readback (b0 eps u w v : Real)
(hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (hv : 0 ≤ v) (r g c : Nat) :
(poissonMeasure (toNNReal (u * (eps * b0 + (1 - eps) * v)))).real {r + 1} *
(poissonMeasure (toNNReal (w * (eps * b0 + (1 - eps) * v)))).real {g} *
(poissonMeasure (toNNReal ((u + w) * ((1 - eps) * b0 + eps * v)))).real {c} =
exp (-(u + w) * (b0 + v)) *
(u * (eps * b0 + (1 - eps) * v)) ^ (r + 1) *
(w * (eps * b0 + (1 - eps) * v)) ^ g *
((u + w) * ((1 - eps) * b0 + eps * v)) ^ c /
(((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) := by
have hs1 : 0 ≤ eps * b0 + (1 - eps) * v := by positivity
have hs0 : 0 ≤ (1 - eps) * b0 + eps * v := by positivity
simp only [poissonMeasure_real_singleton,
Real.coe_toNNReal _ (mul_nonneg hu hs1), Real.coe_toNNReal _ (mul_nonneg hw hs1),
Real.coe_toNNReal _ (mul_nonneg (add_nonneg hu hw) hs0)]
have hexp : exp (-(u + w) * (b0 + v)) =
exp (-(u * (eps * b0 + (1 - eps) * v))) *
exp (-(w * (eps * b0 + (1 - eps) * v))) *
exp (-((u + w) * ((1 - eps) * b0 + eps * v))) := by
rw [← Real.exp_add, ← Real.exp_add]
congr 1
ring
rw [hexp]
ring
theorem affineCellPoissonPredictive_equation_six
Under the stated inputs and conditions, The actual predictive discrepancy at either nonzero deterministic outcome pattern is exactly the signed count kernel from equation (6). This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
ha :
0 < B / (100 * (L : Real) ^ 2)
he :
0 < eps
he' :
eps < 1
hu :
0 ≤ u
hw :
0 ≤ w
first :
r g c :
conclusion
b0 :
B / (100 * (L : Real) ^ 2) / eps
= (if first then 1 else -1) * ∑ i, sigma.weights i * (exp (-(u + w) * (b0 + sigma.nodes i)) * (u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (r + 1) * (w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ g * ((u + w) * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ c / (((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) * (eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i)))
Proof (Lean source)
-- @node: affineCellPoissonPredictive_equation_six
lemma affineCellPoissonPredictive_equation_six {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps < 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (first : Bool) (r g c : Nat) :
let b0 := B / (100 * (L : Real) ^ 2) / eps
(affineCellPoissonPredictive sigma u w true).real {palmSplitTarget first r g c} -
(affineCellPoissonPredictive sigma u w false).real {palmSplitTarget first r g c} =
(if first then 1 else -1) * ∑ i, sigma.weights i *
(exp (-(u + w) * (b0 + sigma.nodes i)) *
(u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (r + 1) *
(w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ g *
((u + w) * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ c /
(((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) *
(eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i))) := by
intro b0
have hab : B / (100 * (L : Real) ^ 2) = eps * b0 := by
dsimp [b0]
field_simp
rw [affineCellPoissonPredictive_target_sub sigma u w ha he']
apply congrArg ((if first then 1 else -1) * ·)
apply Finset.sum_congr rfl
intro i _
rw [hab]
simp only [mul_div_cancel_left₀ _ he.ne']
have h := affine_three_poisson_singleton_readback b0 eps u w (sigma.nodes i)
(div_pos ha he).le he.le he'.le hu hw (sigma.mem i).1 r g c
linear_combination sigma.weights i *
(eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i)) * h
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.CellTV 6 declarations Absolute singleton discrepancy bounds for the actual core/filler rare-cell predictive measures, connecting their likelihoods to the signed Taylor envelope.
Absolute singleton discrepancy bounds for the actual core/filler rare-cell predictive measures, connecting their likelihoods to the signed Taylor envelope.
theorem affineCellPoissonPredictive_abs_target_sub
Under the stated inputs and conditions, Either deterministic marked pattern has the same absolute signed discrepancy. The core/filler likelihood is read back before taking the absolute value. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
ha :
0 < B / (100 * (L : Real) ^ 2)
he :
0 < eps
he' :
eps < 1
hu :
0 ≤ u
hw :
0 ≤ w
first :
r g c :
conclusion
b0 :
B / (100 * (L : Real) ^ 2) / eps
= |∑ i, sigma.weights i * (exp (-(u + w) * (b0 + sigma.nodes i)) * (u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (r + 1) * (w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ g * ((u + w) * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ c / (((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) * (eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i)))|
Proof (Lean source)
-- @node: affineCellPoissonPredictive_abs_target_sub
lemma affineCellPoissonPredictive_abs_target_sub {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps < 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (first : Bool) (r g c : Nat) :
let b0 := B / (100 * (L : Real) ^ 2) / eps
|(affineCellPoissonPredictive sigma u w true).real {palmSplitTarget first r g c} -
(affineCellPoissonPredictive sigma u w false).real {palmSplitTarget first r g c}| =
|∑ i, sigma.weights i *
(exp (-(u + w) * (b0 + sigma.nodes i)) *
(u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (r + 1) *
(w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ g *
((u + w) * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ c /
(((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) *
(eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i)))| := by
intro b0
rw [affineCellPoissonPredictive_equation_six sigma u w ha he he' hu hw]
cases first <;> simp [b0]
theorem affineCellPoissonPredictive_target_tail_bound
Under the stated inputs and conditions, The actual one-cell predictive singleton discrepancies are absolutely summable, and their sum over one deterministic mark pattern has equation (7)'s bound. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
ha :
0 < B / (100 * (L : Real) ^ 2)
he :
0 < eps
he' :
eps < 1
hu :
0 ≤ u
hw :
0 ≤ w
first :
conclusion
D :
fun v
- (affineCellPoissonPredictive sigma u w false).real {palmSplitTarget first v.2.1 v.2.2 v.1}| (: Nat × (Nat × Nat) → Real)
Proof (Lean source)
-- @node: affineCellPoissonPredictive_target_tail_bound
lemma affineCellPoissonPredictive_target_tail_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps < 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (first : Bool) :
let D : Nat × (Nat × Nat) → Real := fun v =>
|(affineCellPoissonPredictive sigma u w true).real
{palmSplitTarget first v.2.1 v.2.2 v.1} -
(affineCellPoissonPredictive sigma u w false).real
{palmSplitTarget first v.2.1 v.2.2 v.1}|
Summable D ∧ (∑' v, D v) ≤
u * (B / (100 * (L : Real) ^ 2)) *
∑' h : Nat, if L < h then (2 * (u + w) * B) ^ h / (h.factorial : Real) else 0 := by
intro D
have hab : eps * (B / (100 * (L : Real) ^ 2) / eps) =
B / (100 * (L : Real) ^ 2) := mul_div_cancel₀ _ he.ne'
have h := affine_signed_raw_count_tail_bound sigma
(B / (100 * (L : Real) ^ 2) / eps) u w (u + w)
(div_pos ha he) he he'.le hu hw rfl
simpa only [D, affineCellPoissonPredictive_abs_target_sub sigma u w ha he he' hu hw,
hab] using h
theorem affineCellPoissonPredictive_joint_target_small
Under the stated inputs and conditions, At the paper's tuning the K-cell budget for actual marked-pattern singleton discrepancies is strictly below one sixty-fourth. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
first :
conclusion
t :
affineTuning n m d eps
(t.Kstar : Real) * (∑' v : Nat × (Nat × Nat), |(affineCellPoissonPredictive sigma t.u t.w true).real {palmSplitTarget first v.2.1 v.2.2 v.1} - (affineCellPoissonPredictive sigma t.u t.w false).real {palmSplitTarget first v.2.1 v.2.2 v.1}|) < 1 / 64
Proof (Lean source)
-- @node: affineCellPoissonPredictive_joint_target_small
lemma affineCellPoissonPredictive_joint_target_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(first : Bool) :
let t := affineTuning n m d eps
(t.Kstar : Real) * (∑' v : Nat × (Nat × Nat),
|(affineCellPoissonPredictive sigma t.u t.w true).real
{palmSplitTarget first v.2.1 v.2.2 v.1} -
(affineCellPoissonPredictive sigma t.u t.w false).real
{palmSplitTarget first v.2.1 v.2.2 v.1}|) < 1 / 64 := by
intro t
obtain ⟨_, _, _, _, _, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hu : 0 ≤ t.u := by change 0 ≤ 128 * (n : Real); positivity
have hw : 0 ≤ t.w := by change 0 ≤ 128 * ((n : Real) + m); positivity
have h := (affineCellPoissonPredictive_target_tail_bound sigma t.u t.w
ha heps (by linarith) hu hw first).2
have hM : t.u + t.w = t.M := rfl
have hA : t.B / (100 * (t.L : Real) ^ 2) = t.alpha := rfl
change (∑' v : Nat × (Nat × Nat),
|(affineCellPoissonPredictive sigma t.u t.w true).real
{palmSplitTarget first v.2.1 v.2.2 v.1} -
(affineCellPoissonPredictive sigma t.u t.w false).real
{palmSplitTarget first v.2.1 v.2.2 v.1}|) ≤
t.u * (t.B / (100 * (t.L : Real) ^ 2)) *
∑' h : Nat, if t.L < h then (2 * (t.u + t.w) * t.B) ^ h /
(h.factorial : Real) else 0 at h
rw [hM, hA] at h
have hbudget := affine_poisson_joint_tail_small n m d eps hn hd heps heps'
change (t.Kstar : Real) * t.u * t.alpha *
(∑' h : Nat, if t.L < h then (2 * t.M * t.B) ^ h /
(h.factorial : Real) else 0) < 1 / 64 at hbudget
apply lt_of_le_of_lt (mul_le_mul_of_nonneg_left h (Nat.cast_nonneg _))
simpa only [mul_assoc] using hbudget
theorem affineCellPoissonPredictive_l1_eq
Under the stated inputs and conditions, The two disjoint deterministic mark patterns exhaust the predictive discrepancy; null treated counts and mixed positive marks contribute zero. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
ha :
0 < B / (100 * (L : Real) ^ 2)
he :
0 < eps
he' :
eps < 1
hu :
0 ≤ u
hw :
0 ≤ w
conclusion
(∑' o : MarkedPoissonObservation, |(affineCellPoissonPredictive sigma u w true).real {o} - (affineCellPoissonPredictive sigma u w false).real {o}|)
= (∑' v : Nat × (Nat × Nat), |(affineCellPoissonPredictive sigma u w true).real {palmSplitTarget true v.2.1 v.2.2 v.1} - (affineCellPoissonPredictive sigma u w false).real {palmSplitTarget true v.2.1 v.2.2 v.1}|)
+ (∑' v : Nat × (Nat × Nat), |(affineCellPoissonPredictive sigma u w true).real {palmSplitTarget false v.2.1 v.2.2 v.1} - (affineCellPoissonPredictive sigma u w false).real {palmSplitTarget false v.2.1 v.2.2 v.1}|)
Proof (Lean source)
-- @node: affineCellPoissonPredictive_l1_eq
lemma affineCellPoissonPredictive_l1_eq {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps < 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) :
(∑' o : MarkedPoissonObservation,
|(affineCellPoissonPredictive sigma u w true).real {o} -
(affineCellPoissonPredictive sigma u w false).real {o}|) =
(∑' v : Nat × (Nat × Nat),
|(affineCellPoissonPredictive sigma u w true).real
{palmSplitTarget true v.2.1 v.2.2 v.1} -
(affineCellPoissonPredictive sigma u w false).real
{palmSplitTarget true v.2.1 v.2.2 v.1}|) +
(∑' v : Nat × (Nat × Nat),
|(affineCellPoissonPredictive sigma u w true).real
{palmSplitTarget false v.2.1 v.2.2 v.1} -
(affineCellPoissonPredictive sigma u w false).real
{palmSplitTarget false v.2.1 v.2.2 v.1}|) := by
classical
let D : MarkedPoissonObservation → Real := fun o =>
|(affineCellPoissonPredictive sigma u w true).real {o} -
(affineCellPoissonPredictive sigma u w false).real {o}|
let embed : Bool × (Nat × (Nat × Nat)) → MarkedPoissonObservation :=
fun z => palmSplitTarget z.1 z.2.2.1 z.2.2.2 z.2.1
have hi : Injective embed := by
rintro ⟨a, c, r, g⟩ ⟨b, c', r', g'⟩ h
cases a <;> cases b <;> simp [embed, palmSplitTarget] at h ⊢ <;> omega
have hcover : support D ⊆ range embed := by
rintro ⟨⟨a, b⟩, g, c⟩ ho
cases a with
| zero =>
cases b with
| zero =>
exfalso
exact ho (by simp [D, affineCellPoissonPredictive_no_labeled_eq])
| succ b => exact ⟨(false, c, b, g), rfl⟩
| succ a =>
cases b with
| zero => exact ⟨(true, c, a, g), rfl⟩
| succ b =>
exfalso
exact ho (by simp [D, affineCellPoissonPredictive_mixed_zero])
have hsum : (∑' o, D o) = ∑' z : Bool × (Nat × (Nat × Nat)), D (embed z) := by
apply tsum_eq_tsum_of_ne_zero_bij (fun z => embed z.val)
· exact hi.comp val_injective
· intro o ho
change D o ≠ 0 at ho
obtain ⟨z, hz⟩ := hcover ho
refine ⟨⟨z, ?_⟩, hz⟩
change D (embed z) ≠ 0
simpa only [hz] using ho
· intro z; rfl
have hs : Summable (fun z : Bool × (Nat × (Nat × Nat)) => D (embed z)) := by
apply (summable_prod_of_nonneg (fun z => abs_nonneg _)).2
refine ⟨?_, (hasSum_fintype _).summable⟩
intro first
exact (affineCellPoissonPredictive_target_tail_bound sigma u w ha he he' hu hw first).1
rw [hsum, hs.tsum_prod' hs.prod_factor]
simp only [tsum_fintype, Fintype.sum_bool, D, embed]
theorem affineCellPoissonPredictive_tv_le
Under the stated inputs and conditions, The actual rare-cell predictive total variation is bounded by the signed Taylor tail, retaining both observed channels and the core/filler mixture. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineCellPoissonPredictive_tv_le
lemma affineCellPoissonPredictive_tv_le {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps < 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) :
tvDist (affineCellPoissonPredictive sigma u w true)
(affineCellPoissonPredictive sigma u w false) ≤
u * (B / (100 * (L : Real) ^ 2)) *
∑' h : Nat, if L < h then (2 * (u + w) * B) ^ h / (h.factorial : Real) else 0 := by
have htv := tvDist_le_half_tsum_singleton_abs
(affineCellPoissonPredictive sigma u w true)
(affineCellPoissonPredictive sigma u w false)
rw [affineCellPoissonPredictive_l1_eq sigma u w ha he he' hu hw] at htv
have ht := (affineCellPoissonPredictive_target_tail_bound sigma u w ha he he' hu hw true).2
have hf := (affineCellPoissonPredictive_target_tail_bound sigma u w ha he he' hu hw false).2
linarith
theorem affineCellPoissonPredictive_joint_tv_small
Under the stated inputs and conditions, The tuned sum of rare-cell total variation bounds is below the paper's one-sixty-fourth budget. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
t :
affineTuning n m d eps
(t.Kstar : Real) * tvDist (affineCellPoissonPredictive sigma t.u t.w true) (affineCellPoissonPredictive sigma t.u t.w false) < 1 / 64
Proof (Lean source)
-- @node: affineCellPoissonPredictive_joint_tv_small
lemma affineCellPoissonPredictive_joint_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let t := affineTuning n m d eps
(t.Kstar : Real) * tvDist
(affineCellPoissonPredictive sigma t.u t.w true)
(affineCellPoissonPredictive sigma t.u t.w false) < 1 / 64 := by
intro t
obtain ⟨_, _, _, _, _, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hu : 0 ≤ t.u := by change 0 ≤ 128 * (n : Real); positivity
have hw : 0 ≤ t.w := by change 0 ≤ 128 * ((n : Real) + m); positivity
have h := affineCellPoissonPredictive_tv_le sigma t.u t.w ha heps (by linarith) hu hw
change tvDist
(affineCellPoissonPredictive sigma t.u t.w true)
(affineCellPoissonPredictive sigma t.u t.w false) ≤
t.u * t.alpha * ∑' h : Nat, if t.L < h then
(2 * t.M * t.B) ^ h / (h.factorial : Real) else 0 at h
have hbudget := affine_poisson_joint_tail_small n m d eps hn hd heps heps'
change (t.Kstar : Real) * t.u * t.alpha *
(∑' h : Nat, if t.L < h then (2 * t.M * t.B) ^ h /
(h.factorial : Real) else 0) < 1 / 64 at hbudget
apply lt_of_le_of_lt (mul_le_mul_of_nonneg_left h (Nat.cast_nonneg _))
simpa only [mul_assoc] using hbudget
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Certificate 2 declarations The shrinking-cone dual as a normalized atomic signed-moment certificate.
The shrinking-cone dual as a normalized atomic signed-moment certificate.
Retain the nodes, weights, and matching moments of the paper's dual witness.
Definition (Lean source)
parameters
def
theorem toNormalizedCertificate_support Lemma ConeDual.toNormalizedCertificate_support in the paper ↗
Under the stated inputs and conditions, The certificate variation is supported on the original shrinking cone. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: ConeDual.toNormalizedCertificate_support
lemma ConeDual.toNormalizedCertificate_support {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) :
∀ᵐ v ∂sigma.toNormalizedCertificate.signedMeasure.variation, v ∈ Icc 0 B := by
rw [NormalizedFiniteSignedMomentCertificate.variation_eq_absoluteMeasure,
NormalizedFiniteSignedMomentCertificate.absoluteMeasure, ae_finsetSum_measure_iff]
intro i hi
apply Measure.ae_smul_measure
exact (ae_dirac_iff measurableSet_Icc).2 (sigma.mem i)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Concentration 18 declarations Exact normalized target readback and the deterministic error decomposition used for affine prior concentration.
Exact normalized target readback and the deterministic error decomposition used for affine prior concentration. The filler, reservoir, and null cells contribute zero target.
The raw target is the sum of the treated rare-cell contributions.
Definition (Lean source)
theorem affine_control_mean_zero
Under the stated inputs and conditions, Every control mean is zero, including the reservoir and null cells. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
j :
Fin d
conclusion
Proof (Lean source)
-- @node: affine_control_mean_zero
lemma affine_control_mean_zero (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) : outcomeMean (normalizedLaw hyp n m d eps hd sigma lat) false j = 0 := by
unfold outcomeMean markedMass
rw [affine_jointMass_readback n m d eps hn hd heps heps' sigma hyp lat]
have hz : rawTable hyp n m d eps sigma lat (j, false, true) = 0 := by
simp [rawTable, bernoulliMass]
rw [hz, zero_div, zero_div]
theorem affine_treated_mean_readback
Under the stated inputs and conditions, Rare treated means are exactly their assigned binary marks; other means vanish. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
j :
Fin d
conclusion
= if hj : j.val < (affineTuning n m d eps).Kstar then match lat ⟨j.val, hj⟩ with | none => 0 | some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2 else 0
Proof (Lean source)
-- @node: affine_treated_mean_readback
lemma affine_treated_mean_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) : outcomeMean (normalizedLaw hyp n m d eps hd sigma lat) true j =
if hj : j.val < (affineTuning n m d eps).Kstar then
match lat ⟨j.val, hj⟩ with
| none => 0
| some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2
else 0 := by
have hQ : rawNormalizer n m d eps sigma lat ≠ 0 :=
ne_of_gt (lt_trans (by norm_num)
(affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat))
obtain ⟨_, _, _, _, _, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
unfold outcomeMean markedMass
rw [affine_jointMass_readback n m d eps hn hd heps heps' sigma hyp lat,
affine_armMass_readback n m d eps hn hd heps heps' sigma hyp lat]
by_cases hj : j.val < (affineTuning n m d eps).Kstar
· have hv : 0 ≤ latentNode sigma (lat ⟨j.val, hj⟩) := by
cases lat ⟨j.val, hj⟩ with
| none => exact le_refl 0
| some i => exact (sigma.mem i).1
have hs : (affineTuning n m d eps).alpha +
(1 - eps) * latentNode sigma (lat ⟨j.val, hj⟩) ≠ 0 :=
ne_of_gt (add_pos_of_pos_of_nonneg ha (mul_nonneg (by linarith) hv))
simp only [rawTable, dif_pos hj, if_true, bernoulliMass]
field_simp
rfl
· simp [rawTable, hj, bernoulliMass]
theorem affine_normalized_target_readback
Under the stated inputs and conditions, The normalized ATE is precisely the raw target divided by the common total mass. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat)
= affineRawTarget n m d eps sigma hyp lat / rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_normalized_target_readback
lemma affine_normalized_target_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) =
affineRawTarget n m d eps sigma hyp lat / rawNormalizer n m d eps sigma lat := by
let K := (affineTuning n m d eps).Kstar
have hKd : K ≤ d := by
have h := Nat.min_le_left (d - 1) (floor (1 / (100 * (affineTuning n m d eps).b0)))
change K ≤ d - 1 at h
omega
unfold ateFunctional
simp_rw [affine_control_mean_zero n m d eps hn hd heps heps' sigma hyp lat,
affine_treated_mean_readback n m d eps hn hd heps heps' sigma hyp lat,
affine_cellMass_readback n m d eps hn hd heps heps' sigma hyp lat, sub_zero]
have hcell (j : Fin d) :
(if hj : j.val < K then (affineTuning n m d eps).b0 + latentNode sigma (lat ⟨j.val, hj⟩)
else if j.val = K then 1 - (K : Real) * latentMeanMass sigma else 0) /
rawNormalizer n m d eps sigma lat *
(if hj : j.val < K then
match lat ⟨j.val, hj⟩ with
| none => 0
| some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2
else 0) =
if hj : j.val < K then
affineCellTarget sigma hyp (lat ⟨j.val, hj⟩) / rawNormalizer n m d eps sigma lat
else 0 := by
by_cases hj : j.val < K
· simp only [dif_pos hj, affineCellTarget]
rw [div_mul_eq_mul_div]
cases lat ⟨j.val, hj⟩
· simp
· dsimp only [affineTuning]
· simp [hj]
dsimp only [K] at hcell
simp_rw [hcell]
rw [affine_sum_prefix d (affineTuning n m d eps).Kstar hKd
(fun j => affineCellTarget sigma hyp (lat j) / rawNormalizer n m d eps sigma lat)]
exact (Finset.sum_div _ _ _).symm
theorem affine_raw_target_bounds
Under the stated inputs and conditions, The raw target lies between zero and the shared normalizer. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
conclusion 1
0 ≤ affineRawTarget n m d eps sigma hyp lat
conclusion 2
affineRawTarget n m d eps sigma hyp lat ≤ rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_raw_target_bounds
lemma affine_raw_target_bounds (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
0 ≤ affineRawTarget n m d eps sigma hyp lat ∧
affineRawTarget n m d eps sigma hyp lat ≤ rawNormalizer n m d eps sigma lat := by
obtain ⟨_, _, _, _, _, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have ha' : 0 < (affineTuning n m d eps).B /
(100 * ((affineTuning n m d eps).L : Real) ^ 2) := ha
have hb (j : Fin (affineTuning n m d eps).Kstar) :=
affine_cell_target_bounds sigma ha' heps hyp (lat j)
constructor
· exact Finset.sum_nonneg (fun j _ => (hb j).1)
· have hs : affineRawTarget n m d eps sigma hyp lat ≤
∑ j, ((affineTuning n m d eps).b0 + latentNode sigma (lat j)) := by
apply Finset.sum_le_sum
intro j _
exact (hb j).2
have hr := affine_reservoir_pos n m d eps hn hd heps heps' sigma
unfold rawNormalizer
linarith
theorem affine_normalized_target_bounds
Under the stated inputs and conditions, Both normalized affine targets lie in the unit interval. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
conclusion
conclusion 1
0 ≤ ateFunctional (normalizedLaw hyp n m d eps hd sigma lat)
conclusion 2
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) ≤ 1
Proof (Lean source)
-- @node: affine_normalized_target_bounds
lemma affine_normalized_target_bounds (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
0 ≤ ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) ∧
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) ≤ 1 := by
rw [affine_normalized_target_readback n m d eps hn hd heps heps' sigma hyp lat]
have hQ : 0 < rawNormalizer n m d eps sigma lat :=
lt_trans (by norm_num) (affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat)
obtain ⟨h0, h1⟩ := affine_raw_target_bounds n m d eps hn hd heps heps' sigma hyp lat
exact ⟨div_nonneg h0 hQ.le, (div_le_one hQ).mpr h1⟩
theorem affine_normalized_target_error_identity
Under the stated inputs and conditions, Normalization separates the raw target error from the total-mass error exactly. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
t :
conclusion
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t
= (affineRawTarget n m d eps sigma hyp lat - t)
+ ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) * (1 - rawNormalizer n m d eps sigma lat)
Proof (Lean source)
-- @node: affine_normalized_target_error_identity
lemma affine_normalized_target_error_identity (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(t : Real) :
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t =
(affineRawTarget n m d eps sigma hyp lat - t) +
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) *
(1 - rawNormalizer n m d eps sigma lat) := by
rw [affine_normalized_target_readback n m d eps hn hd heps heps' sigma hyp lat]
have hQ : rawNormalizer n m d eps sigma lat ≠ 0 :=
ne_of_gt (lt_trans (by norm_num)
(affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat))
field_simp
ring
theorem affine_normalized_target_sq_error_bound
Under the stated inputs and conditions, The squared normalized error is controlled by the raw target and normalizer errors. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
t :
conclusion
(ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2
≤ 2 * (affineRawTarget n m d eps sigma hyp lat - t) ^ 2
+ 2 * (1 - rawNormalizer n m d eps sigma lat) ^ 2
Proof (Lean source)
-- @node: affine_normalized_target_sq_error_bound
lemma affine_normalized_target_sq_error_bound (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(t : Real) :
(ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2 ≤
2 * (affineRawTarget n m d eps sigma hyp lat - t) ^ 2 +
2 * (1 - rawNormalizer n m d eps sigma lat) ^ 2 := by
obtain ⟨h0, h1⟩ := affine_normalized_target_bounds n m d eps hn hd heps heps' sigma hyp lat
have ht : ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) ^ 2 ≤ 1 := by
nlinarith
have hm := mul_le_mul_of_nonneg_right ht
(sq_nonneg (1 - rawNormalizer n m d eps sigma lat))
rw [affine_normalized_target_error_identity n m d eps hn hd heps heps' sigma hyp lat t]
have hs := add_sq_le (a := affineRawTarget n m d eps sigma hyp lat - t)
(b := ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) *
(1 - rawNormalizer n m d eps sigma lat))
rw [mul_pow] at hs
nlinarith
Under the stated inputs and conditions, Finite latent product expectations of additive statistics equal K times their cell mean. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_latent_integral_sum
lemma affine_latent_integral_sum {L : Nat} {B eps : Real} (sigma : ConeDual L B eps)
(K : Nat) (f : Latent L → Real) :
(∫ lat : Fin K → Latent L, ∑ j, f (lat j)
∂Measure.pi (fun _ : Fin K => (latentPMF sigma).toMeasure)) =
(K : Real) * ∑ z, (latentPMF sigma z).toReal * f z := by
have hf : Integrable f (latentPMF sigma).toMeasure := Integrable.of_finite
rw [integral_finsetSum]
· simp_rw [integral_comp_eval (μ := fun _ : Fin K => (latentPMF sigma).toMeasure)
hf.aestronglyMeasurable, PMF.integral_eq_sum, smul_eq_mul]
simp
ring
· intro j _
exact integrable_comp_eval hf
theorem affine_raw_target_mean
Under the stated hypotheses, Expected raw targets are K times their one-cell expectations. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
conclusion
(∫ lat, affineRawTarget n m d eps sigma hyp lat ∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
= ((affineTuning n m d eps).Kstar : Real) * ∑ z, (latentPMF sigma z).toReal * affineCellTarget sigma hyp z
Proof (Lean source)
-- @node: affine_raw_target_mean
lemma affine_raw_target_mean (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) (hyp : Bool) :
(∫ lat, affineRawTarget n m d eps sigma hyp lat
∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)) =
((affineTuning n m d eps).Kstar : Real) *
∑ z, (latentPMF sigma z).toReal * affineCellTarget sigma hyp z :=
affine_latent_integral_sum sigma _ _
theorem affine_raw_target_mean_gap
Under the stated hypotheses, Independence lifts the signed one-cell gap to the expected raw targets. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
≤ (∫ lat, affineRawTarget n m d eps sigma true lat ∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
Proof (Lean source)
-- @node: affine_raw_target_mean_gap
lemma affine_raw_target_mean_gap (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 / 12 ≤
(∫ lat, affineRawTarget n m d eps sigma true lat
∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)) -
(∫ lat, affineRawTarget n m d eps sigma false lat
∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar =>
(latentPMF sigma).toMeasure)) := by
obtain ⟨_, _, hL, _, hB, _, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hell : 1 ≤ logScale n eps := by
have h := Real.log_le_log (Real.exp_pos 1)
(show exp 1 ≤ exp 1 + labelScale n eps by
apply le_add_of_nonneg_right; unfold labelScale; positivity)
simpa only [logScale, Real.log_exp] using h
have hLn : 2 ≤ (affineTuning n m d eps).L := by
have : (2 : Real) ≤ (affineTuning n m d eps).L := by linarith
exact_mod_cast this
have hg := affine_cell_target_mean_gap _ _ eps hLn hB heps heps' sigma
rw [affine_raw_target_mean, affine_raw_target_mean, ← mul_sub]
have hgK := mul_le_mul_of_nonneg_left hg
(Nat.cast_nonneg (affineTuning n m d eps).Kstar : (0 : Real) ≤ _)
convert hgK using 1
· rfl
· dsimp only [affineTuning]
ring
Under the stated inputs and conditions, Independence makes the variance of any additive finite latent mark exactly K-fold. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_latent_variance_sum
lemma affine_latent_variance_sum {L : Nat} {B eps : Real} (sigma : ConeDual L B eps)
(K : Nat) (f : Latent L → Real) :
variance (fun lat : Fin K → Latent L => ∑ j, f (lat j))
(Measure.pi (fun _ : Fin K => (latentPMF sigma).toMeasure)) =
(K : Real) * variance f (latentPMF sigma).toMeasure := by
have hf : ∀ _ : Fin K, MemLp f 2 (latentPMF sigma).toMeasure :=
fun _ => MemLp.of_discrete
have h := variance_sum_pi hf
convert h using 1
· congr 1
ext lat
simp
· simp
theorem affine_raw_target_variance_bound
Under the stated hypotheses, The raw target variance is bounded using its one-cell second moment. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
conclusion
variance (affineRawTarget n m d eps sigma hyp) (Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
≤ ((affineTuning n m d eps).Kstar : Real) * (2 * (affineTuning n m d eps).b0 ^ 2 + (8 / 3) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha)
Proof (Lean source)
-- @node: affine_raw_target_variance_bound
lemma affine_raw_target_variance_bound (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) (hyp : Bool) :
variance (affineRawTarget n m d eps sigma hyp)
(Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)) ≤
((affineTuning n m d eps).Kstar : Real) *
(2 * (affineTuning n m d eps).b0 ^ 2 +
(8 / 3) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha) := by
obtain ⟨_, _, _, _, hB, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have ha' : 0 < (affineTuning n m d eps).B /
(100 * ((affineTuning n m d eps).L : Real) ^ 2) := ha
have hs := affine_cell_target_second_moment_bound sigma hB.le ha' heps (by linarith) hyp
have hv := variance_le_expectation_sq (μ := (latentPMF sigma).toMeasure)
(X := affineCellTarget sigma hyp) (by fun_prop)
rw [PMF.integral_eq_sum] at hv
simp only [Pi.pow_apply, smul_eq_mul] at hv
have he : 0 < 1 - eps := by linarith
have hr : (affineTuning n m d eps).alpha / (1 - eps) ≤
(4 / 3) * (affineTuning n m d eps).alpha := by
apply (div_le_iff₀ he).mpr
nlinarith
have hc : variance (affineCellTarget sigma hyp) (latentPMF sigma).toMeasure ≤
2 * (affineTuning n m d eps).b0 ^ 2 +
(8 / 3) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha := by
have hm := mul_le_mul_of_nonneg_left hr (show 0 ≤ 2 * (affineTuning n m d eps).B by positivity)
have hs' : (∑ z, (latentPMF sigma z).toReal * affineCellTarget sigma hyp z ^ 2) ≤
2 * (affineTuning n m d eps).b0 ^ 2 +
2 * (affineTuning n m d eps).B * ((affineTuning n m d eps).alpha / (1 - eps)) := hs
linarith
rw [show affineRawTarget n m d eps sigma hyp =
(fun lat => ∑ j, affineCellTarget sigma hyp (lat j)) from rfl,
affine_latent_variance_sum]
exact mul_le_mul_of_nonneg_left hc (Nat.cast_nonneg _)
Under the stated inputs and conditions, The common normalizer has expectation exactly one under the latent product prior. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ lat, rawNormalizer n m d eps sigma lat ∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
= 1
Proof (Lean source)
-- @node: affine_raw_normalizer_mean
lemma affine_raw_normalizer_mean (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
(∫ lat, rawNormalizer n m d eps sigma lat
∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar =>
(latentPMF sigma).toMeasure)) = 1 := by
unfold rawNormalizer
rw [integral_add (integrable_const _) Integrable.of_finite, integral_const,
affine_latent_integral_sum sigma _ (fun z => (affineTuning n m d eps).b0 + latentNode sigma z)]
simp only [probReal_univ, one_smul]
have hb : (affineTuning n m d eps).b0 =
(affineTuning n m d eps).B / (100 * ((affineTuning n m d eps).L : Real) ^ 2) / eps := by
dsimp only [affineTuning]
rw [hb]
change 1 - _ * latentMeanMass sigma + _ * latentMeanMass sigma = 1
ring
theorem affine_raw_normalizer_variance_bound Lemma affine_raw_normalizer_variance_bound in the paper ↗
Under the stated hypotheses, The normalizer variance is at most four thirds K B alpha. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
variance (rawNormalizer n m d eps sigma) (Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
≤ (4 / 3) * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha
Proof (Lean source)
-- @node: affine_raw_normalizer_variance_bound
lemma affine_raw_normalizer_variance_bound (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
variance (rawNormalizer n m d eps sigma)
(Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)) ≤
(4 / 3) * ((affineTuning n m d eps).Kstar : Real) *
(affineTuning n m d eps).B * (affineTuning n m d eps).alpha := by
obtain ⟨_, _, _, _, hB, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have ha' : 0 < (affineTuning n m d eps).B /
(100 * ((affineTuning n m d eps).L : Real) ^ 2) := ha
have hs := affine_latent_node_second_moment_bound sigma hB.le ha' (by linarith)
have hv := variance_le_expectation_sq (μ := (latentPMF sigma).toMeasure)
(X := latentNode sigma) (by fun_prop)
rw [PMF.integral_eq_sum] at hv
simp only [Pi.pow_apply, smul_eq_mul] at hv
have he : 0 < 1 - eps := by linarith
have hr : (affineTuning n m d eps).alpha / (1 - eps) ≤
(4 / 3) * (affineTuning n m d eps).alpha := by
apply (div_le_iff₀ he).mpr
nlinarith
have hc : variance (latentNode sigma) (latentPMF sigma).toMeasure ≤
(4 / 3) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha := by
have hm := mul_le_mul_of_nonneg_left hr hB.le
have hs' : (∑ z, (latentPMF sigma z).toReal * latentNode sigma z ^ 2) ≤
(affineTuning n m d eps).B * ((affineTuning n m d eps).alpha / (1 - eps)) := hs
linarith
unfold rawNormalizer
rw [variance_const_add AEStronglyMeasurable.of_discrete,
affine_latent_variance_sum sigma _ (fun z => (affineTuning n m d eps).b0 + latentNode sigma z),
variance_const_add (by fun_prop)]
have hk := mul_le_mul_of_nonneg_left hc
(Nat.cast_nonneg (affineTuning n m d eps).Kstar : (0 : Real) ≤ _)
nlinarith
theorem affine_normalized_target_mean_sq_bound
Under the stated inputs and conditions, The normalized target's mean squared error about its raw mean has the roadmap bound. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
conclusion
nu :
t :
∫ lat, affineRawTarget n m d eps sigma hyp lat ∂nu
(∫ lat, (ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2 ∂nu)
+ 8 * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha
Proof (Lean source)
-- @node: affine_normalized_target_mean_sq_bound
lemma affine_normalized_target_mean_sq_bound (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) (hyp : Bool) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let t := ∫ lat, affineRawTarget n m d eps sigma hyp lat ∂nu
(∫ lat, (ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2 ∂nu) ≤
4 * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 ^ 2 +
8 * ((affineTuning n m d eps).Kstar : Real) *
(affineTuning n m d eps).B * (affineTuning n m d eps).alpha := by
intro nu t
have hi : (∫ lat, (ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2 ∂nu) ≤
∫ lat, 2 * (affineRawTarget n m d eps sigma hyp lat - t) ^ 2 +
2 * (1 - rawNormalizer n m d eps sigma lat) ^ 2 ∂nu := by
apply integral_mono Integrable.of_finite Integrable.of_finite
intro lat
exact affine_normalized_target_sq_error_bound n m d eps hn hd heps heps' sigma hyp lat t
rw [integral_add Integrable.of_finite Integrable.of_finite,
integral_const_mul, integral_const_mul] at hi
have ht : (∫ lat, (affineRawTarget n m d eps sigma hyp lat - t) ^ 2 ∂nu) =
variance (affineRawTarget n m d eps sigma hyp) nu := by
rw [variance_eq_integral (by fun_prop)]
have hQ : (∫ lat, (1 - rawNormalizer n m d eps sigma lat) ^ 2 ∂nu) =
variance (rawNormalizer n m d eps sigma) nu := by
rw [variance_eq_integral (by fun_prop), affine_raw_normalizer_mean]
apply integral_congr_ae
exact ae_of_all _ (fun lat => by ring)
rw [ht, hQ] at hi
have hv := affine_raw_target_variance_bound n m d eps hn hd heps heps' sigma hyp
have hw := affine_raw_normalizer_variance_bound n m d eps hn hd heps heps' sigma
change variance _ nu ≤ _ at hv hw
nlinarith
theorem affine_normalized_target_tail_bound
Under the stated inputs and conditions, Markov's inequality turns the normalized target MSE into a two-sided tail bound. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
a :
ha :
0 < a
conclusion
nu :
t :
∫ lat, affineRawTarget n m d eps sigma hyp lat ∂nu
nu.real {lat | a ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t|}
≤ (4 * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 ^ 2 + 8 * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).B * (affineTuning n m d eps).alpha) / a ^ 2
Proof (Lean source)
-- @node: affine_normalized_target_tail_bound
lemma affine_normalized_target_tail_bound (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (a : Real) (ha : 0 < a) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let t := ∫ lat, affineRawTarget n m d eps sigma hyp lat ∂nu
nu.real {lat | a ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t|} ≤
(4 * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 ^ 2 +
8 * ((affineTuning n m d eps).Kstar : Real) *
(affineTuning n m d eps).B * (affineTuning n m d eps).alpha) / a ^ 2 := by
intro nu t
have h := mul_meas_ge_le_integral_of_nonneg (μ := nu)
(f := fun lat => (ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2)
(ae_of_all _ (fun lat => sq_nonneg _)) Integrable.of_finite (a ^ 2)
have hset : {lat | a ^ 2 ≤ (ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t) ^ 2} =
{lat | a ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t|} := by
ext lat
dsimp
rw [← sq_abs (ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t)]
exact sq_le_sq₀ ha.le (abs_nonneg _)
rw [hset] at h
apply (le_div_iff₀ (sq_pos_of_pos ha)).mpr
rw [mul_comm]
exact h.trans (affine_normalized_target_mean_sq_bound n m d eps hn hd heps heps' sigma hyp)
theorem affine_normalized_target_gap_tail
Under the stated inputs and conditions, The two affine target priors have the roadmap's gap-relative concentration bound. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
sigma :
hyp :
conclusion
nu :
t :
fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
nu.real {lat | gap / 8 ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t hyp|}
≤ 2 ^ 23 * (1 + eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2) / ((affineTuning n m d eps).Kstar : Real)
Proof (Lean source)
-- @node: affine_normalized_target_gap_tail
lemma affine_normalized_target_gap_tail (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) (hyp : Bool) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let t := fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
let gap := t true - t false
nu.real {lat | gap / 8 ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t hyp|} ≤
2 ^ 23 * (1 + eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2) /
((affineTuning n m d eps).Kstar : Real) := by
intro nu t gap
have hK : (0 : Real) < (affineTuning n m d eps).Kstar := by
exact_mod_cast affine_tuning_Kstar_pos n m d eps hn hd heps heps' hS
obtain ⟨_, _, _, _, _, _, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hg : ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 / 12 ≤ gap :=
affine_raw_target_mean_gap n m d eps hn hd heps heps' sigma
have hgp : 0 < gap := lt_of_lt_of_le (by positivity) hg
have ht := affine_normalized_target_tail_bound n m d eps hn hd heps heps' sigma hyp
(gap / 8) (by positivity)
change nu.real _ ≤ _ at ht
refine ht.trans ?_
rw [show 8 * ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).B *
(affineTuning n m d eps).alpha = 8 * ((affineTuning n m d eps).Kstar : Real) *
((affineTuning n m d eps).B * (affineTuning n m d eps).alpha) by ring,
affine_tuning_B_alpha_identity n m d eps heps]
apply (div_le_iff₀ (sq_pos_of_pos (by positivity : 0 < gap / 8))).mpr
rw [div_mul_eq_mul_div]
apply (le_div_iff₀ hK).mpr
have hs : (((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 / 12) ^ 2 ≤
gap ^ 2 := pow_le_pow_left₀ (by positivity) hg 2
have hz : 0 ≤ eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2 := by positivity
have hm := mul_le_mul_of_nonneg_left hs (show 0 ≤ 2 ^ 23 * (1 + eps ^ 2 *
((affineTuning n m d eps).L : Real) ^ 2) by positivity)
have hp : 0 ≤ (((affineTuning n m d eps).Kstar : Real) *
(affineTuning n m d eps).b0) ^ 2 *
(eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2) := by positivity
nlinarith [sq_nonneg (((affineTuning n m d eps).Kstar : Real) *
(affineTuning n m d eps).b0)]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.ControlSplit 18 declarations The common binomial reconstruction of combined control counts into the two original channels, with exact independent-Poisson likelihood readback.
The common binomial reconstruction of combined control counts into the two original channels, with exact independent-Poisson likelihood readback.
Under the stated inputs and conditions, Splitting a combined Poisson count by the channel intensity ratio gives exactly the two independent channel likelihoods. This gives the stated result.
Formal statement
hypotheses
u v r :
ℝ
hu :
0 ≤ u
hv :
0 ≤ v
hr :
0 ≤ r
ht :
0 < u + v
k s :
ℕ
conclusion
(poissonMeasure (toNNReal ((u + v) * r))).real {k + s} * (binomial (k + s) (⟨u / (u + v), by constructor · positivity · rw [div_le_one ht] linarith⟩ : unitInterval)).real {k}
Proof (Lean source)
-- @node: affine_poisson_binomial_split
lemma affine_poisson_binomial_split
(u v r : ℝ) (hu : 0 ≤ u) (hv : 0 ≤ v) (hr : 0 ≤ r)
(ht : 0 < u + v) (k s : ℕ) :
(poissonMeasure (toNNReal ((u + v) * r))).real {k + s} *
(binomial (k + s)
(⟨u / (u + v), by
constructor
· positivity
· rw [div_le_one ht]
linarith⟩ : unitInterval)).real {k} =
(poissonMeasure (toNNReal (u * r))).real {k} *
(poissonMeasure (toNNReal (v * r))).real {s} := by
rw [poissonMeasure_real_singleton, binomial_real_singleton,
poissonMeasure_real_singleton, poissonMeasure_real_singleton]
have hfacNat :=
Nat.choose_mul_factorial_mul_factorial (le_add_right k s)
rw [Nat.add_sub_cancel_left] at hfacNat
have hfac :
((k + s).choose k : ℝ) * (k.factorial : ℝ) * (s.factorial : ℝ) =
((k + s).factorial : ℝ) := by
exact_mod_cast hfacNat
simp only [Nat.add_sub_cancel_left]
rw [Real.coe_toNNReal _ (mul_nonneg (add_nonneg hu hv) hr),
Real.coe_toNNReal _ (mul_nonneg hu hr),
Real.coe_toNNReal _ (mul_nonneg hv hr)]
rw [show (1 - u / (u + v) : ℝ) = v / (u + v) by
field_simp
ring]
have hexp : exp (-((u + v) * r)) =
exp (-(u * r)) * exp (-(v * r)) := by
rw [← Real.exp_add]
congr 1
ring
rw [hexp, pow_add, mul_pow, mul_pow, div_pow, div_pow]
field_simp
rw [← hfac]
ring
Conditional channel split of a combined control count.
instance affineControlSplitLaw_isProbabilityMeasure
The split law is a probability law for every combined count.
Definition (Lean source)
-- @node: affineControlSplitLaw_isProbabilityMeasure
instance affineControlSplitLaw_isProbabilityMeasure (q : unitInterval) (c : Nat) :
IsProbabilityMeasure (affineControlSplitLaw q c) := by
unfold affineControlSplitLaw
exact Measure.isProbabilityMeasure_map (measurable_of_countable _).aemeasurable
theorem affineControlSplitLaw_singleton
Under the stated inputs and conditions, A split singleton has nonzero likelihood only at the matching total. This gives the stated result.
Formal statement
hypotheses
q :
c k s :
conclusion
affineControlSplitLaw q c {(k, s)} = if c = k + s then (binomial c q) {k} else 0
Proof (Lean source)
-- @node: affineControlSplitLaw_singleton
lemma affineControlSplitLaw_singleton (q : unitInterval) (c k s : Nat) :
affineControlSplitLaw q c {(k, s)} =
if c = k + s then (binomial c q) {k} else 0 := by
rw [affineControlSplitLaw, Measure.map_apply (measurable_of_countable _)
(measurableSet_singleton _)]
by_cases hc : c = k + s
· rw [if_pos hc]
congr 1
ext j
simp only [Set.mem_preimage, Set.mem_singleton_iff, Prod.mk.injEq]
constructor
· exact fun h => h.1
· intro h
subst j
exact ⟨rfl, by omega⟩
· rw [if_neg hc]
have hsub : (fun j => (j, c - j)) ⁻¹' {(k, s)} ⊆ {k} := by
intro j hj
exact (Prod.mk.inj hj).1
by_cases hk : k ≤ c
· have hempty : (fun j => (j, c - j)) ⁻¹' {(k, s)} = ∅ := by
ext j
simp only [Set.mem_preimage, Set.mem_singleton_iff, Prod.mk.injEq,
Set.mem_empty_iff_false, iff_false, not_and]
intro hj
subst j
omega
rw [hempty, measure_empty]
· have hz : (binomial c q) {k} = 0 := by
rw [binomial_singleton, Nat.choose_eq_zero_of_lt (by omega)]
simp
exact le_antisymm ((measure_mono hsub).trans_eq hz) zero_le
The channel split is a common Markov kernel, independent of latent masses and of the hypothesis's outcome marks.
Definition (Lean source)
instance affineControlSplitKernel_isMarkovKernel
Each control reconstruction has total mass one.
Definition (Lean source)
-- @node: affineControlSplitKernel_isMarkovKernel
instance affineControlSplitKernel_isMarkovKernel (q : unitInterval) :
IsMarkovKernel (affineControlSplitKernel q) where
isProbabilityMeasure c := by
change IsProbabilityMeasure (affineControlSplitLaw q c)
infer_instance
theorem affineControlSplitKernel_poisson
Under the stated inputs and conditions, Applying the common control split to a combined Poisson count recovers both independent Poisson control channels, without dropping either channel. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affineControlSplitKernel_poisson
lemma affineControlSplitKernel_poisson
(u w r : Real) (hu : 0 ≤ u) (hw : 0 ≤ w) (hr : 0 ≤ r)
(huw : 0 < u + w) :
let q : unitInterval := ⟨u / (u + w), by
constructor
· positivity
· rw [div_le_one huw]; linarith⟩
(affineControlSplitKernel q) ∘ₘ poissonMeasure (toNNReal ((u + w) * r)) =
(poissonMeasure (toNNReal (u * r))).prod
(poissonMeasure (toNNReal (w * r))) := by
intro q
apply Measure.ext_of_singleton
rintro ⟨k, s⟩
rw [Measure.bind_apply (measurableSet_singleton _) (by fun_prop)]
change (∫⁻ c : Nat, affineControlSplitLaw q c {(k, s)}
∂poissonMeasure (toNNReal ((u + w) * r))) = _
simp only [affineControlSplitLaw_singleton]
rw [lintegral_countable']
simp only [ite_mul, zero_mul]
rw [tsum_eq_single (k + s)]
· rw [if_pos rfl]
apply (ENNReal.toReal_eq_toReal_iff' (by finiteness) (by finiteness)).mp
rw [ENNReal.toReal_mul, ← measureReal_def, ← measureReal_def,
← Set.singleton_prod_singleton, Measure.prod_prod, ENNReal.toReal_mul,
← measureReal_def, ← measureReal_def]
rw [mul_comm ((binomial (k + s) q).real {k})]
exact affine_poisson_binomial_split u w r hu hw hr huw k s
· intro c hc
rw [if_neg hc]
theorem affineControlSplitKernel_with_common
Under the stated inputs and conditions, Keeping independent existing coordinates while splitting a control count preserves their laws and restores the independent channel controls. This gives the stated result.
Formal statement
hypotheses
X :
Type*
u w r :
hu :
0 ≤ u
hw :
0 ≤ w
hr :
0 ≤ r
huw :
0 < u + w
conclusion
q :
⟨u / (u + w), by constructor · positivity · rw [div_le_one huw] (: unitInterval)
linarith⟩ (Kernel.id ∥ₖ affineControlSplitKernel q) ∘ₘ (mu.prod (poissonMeasure (toNNReal ((u + w) * r))))
Proof (Lean source)
-- @node: affineControlSplitKernel_with_common
lemma affineControlSplitKernel_with_common
{X : Type*} [MeasurableSpace X] (mu : Measure X) [SFinite mu]
(u w r : Real) (hu : 0 ≤ u) (hw : 0 ≤ w) (hr : 0 ≤ r)
(huw : 0 < u + w) :
let q : unitInterval := ⟨u / (u + w), by
constructor
· positivity
· rw [div_le_one huw]; linarith⟩
(Kernel.id ∥ₖ affineControlSplitKernel q) ∘ₘ
(mu.prod (poissonMeasure (toNNReal ((u + w) * r)))) =
mu.prod ((poissonMeasure (toNNReal (u * r))).prod
(poissonMeasure (toNNReal (w * r)))) := by
intro q
rw [Measure.comp_eq_comp_const_apply, ← Kernel.prod_const,
Kernel.parallelComp_comp_prod, Kernel.id_comp, Kernel.comp_const,
Kernel.prod_apply, Kernel.const_apply, Kernel.const_apply,
affineControlSplitKernel_poisson u w r hu hw hr huw]
Reconstruct the full rare-cell control channels while keeping both complete treated outcome counts and the auxiliary treated count.
Definition (Lean source)
instance affineCellControlSplitKernel_isMarkovKernel
Full rare-cell control reconstruction is a common probability kernel.
Definition (Lean source)
-- @node: affineCellControlSplitKernel_isMarkovKernel
instance affineCellControlSplitKernel_isMarkovKernel (q : unitInterval) :
IsMarkovKernel (affineCellControlSplitKernel q) := by
unfold affineCellControlSplitKernel
infer_instance
Under the stated inputs and conditions, Independent coordinate reconstruction commutes with the product law. This gives the stated result.
Formal statement
hypotheses
X Y Z W :
mu :
Measure X
nu :
Measure Y
SFinite mu
SFinite nu
kappa :
Kernel X Z
eta :
Kernel Y W
IsSFiniteKernel kappa
IsSFiniteKernel eta
conclusion
(kappa ∥ₖ eta) ∘ₘ mu.prod nu = (kappa ∘ₘ mu).prod (eta ∘ₘ nu)
Proof (Lean source)
-- @node: affine_parallelComp_prod
lemma affine_parallelComp_prod
{X Y Z W : Type*} [MeasurableSpace X] [MeasurableSpace Y]
[MeasurableSpace Z] [MeasurableSpace W]
(mu : Measure X) (nu : Measure Y) [SFinite mu] [SFinite nu]
(kappa : Kernel X Z) (eta : Kernel Y W)
[IsSFiniteKernel kappa] [IsSFiniteKernel eta] :
(kappa ∥ₖ eta) ∘ₘ mu.prod nu = (kappa ∘ₘ mu).prod (eta ∘ₘ nu) := by
rw [Measure.comp_eq_comp_const_apply, ← Kernel.prod_const,
Kernel.parallelComp_comp_prod, Kernel.comp_const, Kernel.comp_const,
Kernel.prod_apply, Kernel.const_apply, Kernel.const_apply]
theorem affineCellControlSplitKernel_law
Under the stated inputs and conditions, Splitting an actual conditional rare-cell law restores the independent complete and auxiliary control counts, while leaving all treated counts intact. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
hyp :
z :
Latent L
ha :
0 ≤ B / (100 * (L : Real) ^ 2)
he :
0 < eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
huw :
0 < u + w
conclusion
q :
⟨u / (u + w), by constructor · positivity · rw [div_le_one huw] (: unitInterval)
linarith⟩ let alpha := B / (100 * (L : Real) ^ 2) let s1 := alpha
+ (1 - eps) * latentNode sigma z let s0 := (1 - eps) * (alpha / eps)
+ eps * latentNode sigma z let mean := match z with | none
=> (0 : Real) | some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2 affineCellControlSplitKernel q ∘ₘ affineCellPoissonLaw sigma u w hyp z = ((poissonMeasure (toNNReal (u * s1 * mean))).prod (poissonMeasure (toNNReal (u * s1 * (1 - mean))))).prod ((poissonMeasure (toNNReal (w * s1))).prod ((poissonMeasure (toNNReal (u * s0))).prod (poissonMeasure (toNNReal (w * s0)))))
Proof (Lean source)
-- @node: affineCellControlSplitKernel_law
lemma affineCellControlSplitKernel_law {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) (z : Latent L)
(ha : 0 ≤ B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (huw : 0 < u + w) :
let q : unitInterval := ⟨u / (u + w), by
constructor
· positivity
· rw [div_le_one huw]; linarith⟩
let alpha := B / (100 * (L : Real) ^ 2)
let s1 := alpha + (1 - eps) * latentNode sigma z
let s0 := (1 - eps) * (alpha / eps) + eps * latentNode sigma z
let mean := match z with
| none => (0 : Real)
| some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2
affineCellControlSplitKernel q ∘ₘ affineCellPoissonLaw sigma u w hyp z =
((poissonMeasure (toNNReal (u * s1 * mean))).prod
(poissonMeasure (toNNReal (u * s1 * (1 - mean))))).prod
((poissonMeasure (toNNReal (w * s1))).prod
((poissonMeasure (toNNReal (u * s0))).prod
(poissonMeasure (toNNReal (w * s0))))) := by
intro q alpha s1 s0 mean
have hv : 0 ≤ latentNode sigma z := by
cases z with
| none => exact le_rfl
| some i => exact (sigma.mem i).1
have hs0 : 0 ≤ s0 := by
dsimp only [s0]
exact add_nonneg (mul_nonneg (sub_nonneg.mpr he') (div_nonneg ha he.le))
(mul_nonneg he.le hv)
unfold affineCellControlSplitKernel affineCellPoissonLaw
rw [affine_parallelComp_prod, affineControlSplitKernel_with_common _ u w _ hu hw hs0 huw]
rw [Measure.id_comp]
rfl
The conditional rare-cell law with separate controls in the original two channels. All five potentially nonzero atom counts are retained.
Definition (Lean source)
parameters
given by
alpha :
B / (100 * (L : Real) ^ 2)
s1 :
alpha + (1 - eps) * latentNode sigma z
s0 :
(1 - eps) * (alpha / eps) + eps * latentNode sigma z
mean :
match z with | none
=> (0 : Real) | some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2
((poissonMeasure (toNNReal (u * s1 * mean))).prod (poissonMeasure (toNNReal (u * s1 * (1 - mean))))).prod ((poissonMeasure (toNNReal (w * s1))).prod ((poissonMeasure (toNNReal (u * s0))).prod (poissonMeasure (toNNReal (w * s0)))))
instance affineCellFullPoissonLaw_isProbabilityMeasure
The full conditional cell law has total mass one.
Definition (Lean source)
-- @node: affineCellFullPoissonLaw_isProbabilityMeasure
instance affineCellFullPoissonLaw_isProbabilityMeasure {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) (z : Latent L) :
IsProbabilityMeasure (affineCellFullPoissonLaw sigma u w hyp z) := by
unfold affineCellFullPoissonLaw
infer_instance
The actual core/filler predictive law with control channels restored.
instance affineCellFullPoissonPredictive_isProbabilityMeasure
Mixing full conditional cell laws gives a probability measure.
Definition (Lean source)
-- @node: affineCellFullPoissonPredictive_isProbabilityMeasure
instance affineCellFullPoissonPredictive_isProbabilityMeasure {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool) :
IsProbabilityMeasure (affineCellFullPoissonPredictive sigma u w hyp) := by
constructor
simp only [affineCellFullPoissonPredictive, Measure.finsetSum_apply,
Measure.smul_apply, measure_univ, smul_eq_mul, mul_one]
exact (tsum_fintype _).symm.trans (latentPMF sigma).tsum_coe
theorem affineCellControlSplitKernel_predictive
Under the stated inputs and conditions, The common channel reconstruction identifies the actual full core/filler predictive law, rather than merely bounding an abstract postprocessing. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
u w :
hyp :
ha :
0 ≤ B / (100 * (L : Real) ^ 2)
he :
0 < eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
huw :
0 < u + w
conclusion
q :
⟨u / (u + w), by constructor · positivity · rw [div_le_one huw] (: unitInterval)
linarith⟩ affineCellControlSplitKernel q ∘ₘ affineCellPoissonPredictive sigma u w hyp
= affineCellFullPoissonPredictive sigma u w hyp
Proof (Lean source)
-- @node: affineCellControlSplitKernel_predictive
lemma affineCellControlSplitKernel_predictive {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (u w : Real) (hyp : Bool)
(ha : 0 ≤ B / (100 * (L : Real) ^ 2)) (he : 0 < eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (huw : 0 < u + w) :
let q : unitInterval := ⟨u / (u + w), by
constructor
· positivity
· rw [div_le_one huw]; linarith⟩
affineCellControlSplitKernel q ∘ₘ affineCellPoissonPredictive sigma u w hyp =
affineCellFullPoissonPredictive sigma u w hyp := by
intro q
have hcond (z : Latent L) :
affineCellControlSplitKernel q ∘ₘ affineCellPoissonLaw sigma u w hyp z =
affineCellFullPoissonLaw sigma u w hyp z :=
affineCellControlSplitKernel_law sigma u w hyp z ha he he' hu hw huw
ext s hs
rw [Measure.bind_apply hs (affineCellControlSplitKernel q).aemeasurable]
simp only [affineCellPoissonPredictive, lintegral_finsetSum_measure,
lintegral_smul_measure, affineCellFullPoissonPredictive, Measure.finsetSum_apply,
Measure.smul_apply, smul_eq_mul]
apply Finset.sum_congr rfl
intro z _
congr 1
rw [← Measure.bind_apply hs (affineCellControlSplitKernel q).aemeasurable]
exact congrArg (fun mu => mu s) (hcond z)
theorem affineCellFullPoissonPredictive_product_tv_small
Under the stated inputs and conditions, Restoring both control channels preserves the calibrated joint TV budget for the independent product of the actual full rare-cell mixtures. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
t :
affineTuning n m d eps
tvDist (Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w true) (Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w false) < 1 / 64
Proof (Lean source)
-- @node: affineCellFullPoissonPredictive_product_tv_small
lemma affineCellFullPoissonPredictive_product_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let t := affineTuning n m d eps
tvDist
(Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w true)
(Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w false) <
1 / 64 := by
intro t
obtain ⟨_, _, _, _, _, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hu : 0 ≤ t.u := by change 0 ≤ 128 * (n : Real); positivity
have hw : 0 ≤ t.w := by change 0 ≤ 128 * ((n : Real) + m); positivity
have huw : 0 < t.u + t.w := by
change 0 < 128 * (n : Real) + 128 * ((n : Real) + m)
have hnR : (0 : Real) < n := by exact_mod_cast (by omega : 0 < n)
positivity
let q : unitInterval := ⟨t.u / (t.u + t.w), by
constructor
· positivity
· rw [div_le_one huw]; linarith⟩
have htv : tvDist
(affineCellFullPoissonPredictive sigma t.u t.w true)
(affineCellFullPoissonPredictive sigma t.u t.w false) ≤
tvDist (affineCellPoissonPredictive sigma t.u t.w true)
(affineCellPoissonPredictive sigma t.u t.w false) := by
rw [← affineCellControlSplitKernel_predictive sigma t.u t.w true ha.le heps
(by linarith) hu hw huw,
← affineCellControlSplitKernel_predictive sigma t.u t.w false ha.le heps
(by linarith) hu hw huw]
exact tvDist_bind_le _ _ (affineCellControlSplitKernel q)
exact (tvDist_pi_iid_le t.Kstar _ _).trans_lt
((mul_le_mul_of_nonneg_left htv (Nat.cast_nonneg t.Kstar)).trans_lt
(affineCellPoissonPredictive_joint_tv_small n m d eps hn hd heps heps' sigma))
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.CountEnvelope 18 declarations Countwise nonnegative polynomial envelopes for equation (6), the shifted labeled factorial comparison, absolutely convergent coefficientwise count sums, and exact cancellation of the common control baseline.
Countwise nonnegative polynomial envelopes for equation (6), the shifted labeled factorial comparison, absolutely convergent coefficientwise count sums, and exact cancellation of the common control baseline.
Under the stated inputs and conditions, Products of nonnegative affine powers have nonnegative Taylor coefficients. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_arm_power_coeff_nonneg
lemma affine_arm_power_coeff_nonneg (alpha b0 eps : Real) (r c j : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1) :
0 ≤ ((C alpha + C (1 - eps) * X) ^ r *
(C ((1 - eps) * b0) + C eps * X) ^ c).coeff j := by
have hmul (p q : Polynomial Real) (hp : ∀ j, 0 ≤ p.coeff j)
(hq : ∀ j, 0 ≤ q.coeff j) (j : Nat) : 0 ≤ (p * q).coeff j := by
rw [Polynomial.coeff_mul]
exact Finset.sum_nonneg fun k _ => mul_nonneg (hp k.1) (hq k.2)
have hpow (p : Polynomial Real) (hp : ∀ j, 0 ≤ p.coeff j) (n j : Nat) :
0 ≤ (p ^ n).coeff j := by
induction n generalizing j with
| zero => simp only [pow_zero, Polynomial.coeff_one]; split_ifs <;> norm_num
| succ n ih =>
rw [pow_succ]
exact hmul (p ^ n) p ih hp j
have hlin (a b : Real) (ha : 0 ≤ a) (hb : 0 ≤ b) (j : Nat) :
0 ≤ (C a + C b * X).coeff j := by
simp only [Polynomial.coeff_add, Polynomial.coeff_C_mul, Polynomial.coeff_C,
Polynomial.coeff_X]
split_ifs <;> positivity
exact hmul _ _ (hpow _ (hlin _ _ ha (by linarith)) r)
(hpow _ (hlin _ _ (mul_nonneg (by linarith) hb) he) c) j
theorem affine_signed_count_polynomial_bound Lemma affine_signed_count_polynomial_bound in the paper ↗
Under the stated inputs and conditions, Each count kernel is bounded by positive coefficients with only total degrees above the matching degree retained, including counts exceeding that degree. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_signed_count_polynomial_bound
lemma affine_signed_count_polynomial_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (alpha b0 M : Real) (r c : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hM : 0 ≤ M) :
let p : Polynomial Real :=
(C alpha + C (1 - eps) * X) ^ r *
(C ((1 - eps) * b0) + C eps * X) ^ c
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)| ≤
∑ j ∈ range (p.natDegree + 1), p.coeff j * B ^ j *
∑' k : Nat, (M * B) ^ (k + (if j ≤ L then L - j + 1 else 0)) /
((k + (if j ≤ L then L - j + 1 else 0)).factorial : Real) := by
intro p
have h := affine_signed_exp_polynomial_bound sigma M hM p
have hcoeff (j : Nat) : |p.coeff j| = p.coeff j :=
abs_of_nonneg (affine_arm_power_coeff_nonneg alpha b0 eps r c j ha hb he he')
simpa only [hcoeff, p, Polynomial.eval_mul, Polynomial.eval_pow,
Polynomial.eval_add, Polynomial.eval_C, Polynomial.eval_X, mul_assoc] using h
Under the stated inputs and conditions, The shifted exponential tail is exactly the series filtered by total degree. This form retains the cancellation cutoff when polynomial and count sums are exchanged. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_total_degree_tail_eq
lemma affine_total_degree_tail_eq (z : Real) (hz : 0 ≤ z) (L j : Nat) :
(∑' k : Nat, z ^ (k + (if j ≤ L then L - j + 1 else 0)) /
((k + (if j ≤ L then L - j + 1 else 0)).factorial : Real)) =
∑' k : Nat, if L < k + j then z ^ k / (k.factorial : Real) else 0 := by
by_cases hj : j ≤ L
· simp only [if_pos hj]
have hs : Summable (fun k : Nat =>
if L < k + j then z ^ k / (k.factorial : Real) else 0) := by
apply (Real.summable_pow_div_factorial z).of_nonneg_of_le
· intro k; split_ifs <;> positivity
· intro k; split_ifs <;> first | exact le_rfl | positivity
have hzero : (∑ k ∈ range (L - j + 1),
if L < k + j then z ^ k / (k.factorial : Real) else 0) = 0 := by
apply Finset.sum_eq_zero
intro k hk
rw [if_neg (by have := Finset.mem_range.mp hk; omega)]
have hshift (k : Nat) : L < k + (L - j + 1) + j := by omega
simpa only [hzero, zero_add, hshift, if_true] using
hs.sum_add_tsum_nat_add (L - j + 1)
· simp only [if_neg hj, Nat.add_zero,
show ∀ k : Nat, L < k + j by omega, if_true]
theorem affine_signed_count_total_degree_bound Lemma affine_signed_count_total_degree_bound in the paper ↗
Under the stated inputs and conditions, The signed count kernel has a nonnegative envelope filtered by total degree, ready for exchanging the polynomial, exponential, and count sums. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
alpha b0 M :
r c :
ha :
0 ≤ alpha
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hM :
0 ≤ M
conclusion
Proof (Lean source)
-- @node: affine_signed_count_total_degree_bound
lemma affine_signed_count_total_degree_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (alpha b0 M : Real) (r c : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hM : 0 ≤ M) :
let p : Polynomial Real :=
(C alpha + C (1 - eps) * X) ^ r *
(C ((1 - eps) * b0) + C eps * X) ^ c
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)| ≤
∑ j ∈ range (p.natDegree + 1), p.coeff j * B ^ j *
∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0 := by
intro p
have hB : 0 ≤ B := (sigma.mem 0).1.trans (sigma.mem 0).2
have h := affine_signed_count_polynomial_bound sigma alpha b0 M r c ha hb he he' hM
simpa only [affine_total_degree_tail_eq (M * B) (mul_nonneg hM hB)] using h
Under the stated inputs and conditions, Shifting the labeled count by one decreases every nonnegative factorial term. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_labeled_factorial_term_le
lemma affine_labeled_factorial_term_le (z : Real) (hz : 0 ≤ z) (h : Nat) :
z ^ h / ((h + 1).factorial : Real) ≤ z ^ h / (h.factorial : Real) := by
apply div_le_div_of_nonneg_left (pow_nonneg hz h) (by positivity)
exact_mod_cast Nat.factorial_le (le_succ h)
Under the stated inputs and conditions, The labeled Palm count series is dominated by the ordinary exponential series. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_labeled_factorial_series_le
lemma affine_labeled_factorial_series_le (z : Real) (hz : 0 ≤ z) :
(∑' h : Nat, z ^ h / ((h + 1).factorial : Real)) ≤ exp z := by
have hs := Real.summable_pow_div_factorial z
have ht : Summable (fun h : Nat => z ^ h / ((h + 1).factorial : Real)) :=
hs.of_nonneg_of_le (fun h => by positivity)
(affine_labeled_factorial_term_le z hz)
calc
_ ≤ ∑' h : Nat, z ^ h / (h.factorial : Real) :=
Summable.tsum_le_tsum (affine_labeled_factorial_term_le z hz) ht hs
_ = exp z := by rw [Real.exp_eq_exp_ℝ, NormedSpace.exp_eq_tsum_div]
The positive count generating functions cancel the entire common baseline, so their envelope contains only twice the latent intensity.
Formal statement
Proof (Lean source)
-- @node: affine_count_exp_baseline_cancel
lemma affine_count_exp_baseline_cancel (u w b0 eps v : Real) :
exp ((u + w) * v - (u + w) * b0) *
exp (u * (eps * b0 + (1 - eps) * v)) *
exp (w * (eps * b0 + (1 - eps) * v)) *
exp ((u + w) * ((1 - eps) * b0 + eps * v)) =
exp (2 * (u + w) * v) := by
rw [← Real.exp_add, ← Real.exp_add, ← Real.exp_add]
congr 1
ring
The binomial coefficient of an affine arm power, including zero slopes.
Formal statement
Proof (Lean source)
-- @node: affine_linear_pow_coeff
lemma affine_linear_pow_coeff (a b : Real) (r j : Nat) :
((C a + C b * X) ^ r).coeff j =
b ^ j * a ^ (r - j) * (r.choose j : Real) := by
rw [add_comm (C a), add_pow, ← Polynomial.lcoeff_apply, map_sum]
simp only [Polynomial.lcoeff_apply, mul_pow, ← Polynomial.C_pow,
← Polynomial.C_eq_natCast, Polynomial.coeff_mul_C, Polynomial.coeff_C_mul]
rw [Finset.sum_eq_single j, Polynomial.coeff_X_pow_self]
· ring
· intro k hk hkj
simp [Polynomial.coeff_X_pow, hkj.symm]
· intro hj
have hrj : r < j := by simp only [Finset.mem_range] at hj; omega
simp [Nat.choose_eq_zero_of_lt hrj]
theorem affine_linear_coeff_factorial_shift Lemma affine_linear_coeff_factorial_shift in the paper ↗
After shifting by the coefficient degree, a count term separates into a fixed slope coefficient and an ordinary exponential term.
Formal statement
Proof (Lean source)
-- @node: affine_linear_coeff_factorial_shift
lemma affine_linear_coeff_factorial_shift (a b t : Real) (k j : Nat) :
t ^ (k + j) / ((k + j).factorial : Real) *
((C a + C b * X) ^ (k + j)).coeff j =
((t * b) ^ j / (j.factorial : Real)) *
((t * a) ^ k / (k.factorial : Real)) := by
rw [affine_linear_pow_coeff, Nat.add_sub_cancel, pow_add, mul_pow, mul_pow]
have hfact : ((k + j).choose j : Real) * (j.factorial : Real) *
(k.factorial : Real) = ((k + j).factorial : Real) := by
simpa only [mul_assoc, mul_comm, mul_left_comm] using
(show ((k + j).choose j : Real) * (k.factorial : Real) *
(j.factorial : Real) = ((k + j).factorial : Real) by
exact_mod_cast Nat.add_choose_mul_factorial_mul_factorial k j)
have hk : (k.factorial : Real) ≠ 0 := by positivity
have hj : (j.factorial : Real) ≠ 0 := by positivity
have hkj : ((k + j).factorial : Real) ≠ 0 := by positivity
field_simp
calc
_ = (t ^ k * t ^ j * b ^ j * a ^ k) *
(((k + j).choose j : Real) * (j.factorial : Real) * (k.factorial : Real)) := by ring
_ = _ := by rw [hfact]; ring
theorem affine_linear_coeff_series_summable Lemma affine_linear_coeff_series_summable in the paper ↗
Under the stated inputs and conditions, Each fixed affine Taylor coefficient is absolutely summable over the Poisson count. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_linear_coeff_series_summable
lemma affine_linear_coeff_series_summable (a b t : Real) (j : Nat) :
Summable (fun r : Nat => t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff j) := by
apply Summable.comp_nat_add (k := j)
simp_rw [affine_linear_coeff_factorial_shift]
exact (Real.summable_pow_div_factorial (t * a)).mul_left _
Under the stated inputs and conditions, Summing a fixed Taylor coefficient over all counts reads back the affine exponential exactly; the common constant is retained rather than bounded. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_linear_coeff_series_eq
lemma affine_linear_coeff_series_eq (a b t : Real) (j : Nat) :
(∑' r : Nat, t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff j) =
exp (t * a) * (t * b) ^ j / (j.factorial : Real) := by
have hs := affine_linear_coeff_series_summable a b t j
have hzero : (∑ r ∈ range j, t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff j) = 0 := by
apply Finset.sum_eq_zero
intro r hr
rw [affine_linear_pow_coeff, Nat.choose_eq_zero_of_lt (Finset.mem_range.mp hr)]
simp
rw [← hs.sum_add_tsum_nat_add j, hzero, zero_add]
simp_rw [affine_linear_coeff_factorial_shift]
have he : (∑' k : Nat, (t * a) ^ k / (k.factorial : Real)) = exp (t * a) := by
rw [Real.exp_eq_exp_ℝ, NormedSpace.exp_eq_tsum_div]
rw [tsum_mul_left, he]
ring
theorem affine_exponential_coeff_convolution Lemma affine_exponential_coeff_convolution in the paper ↗
Under the stated inputs and conditions, The finite convolution of two exponential Taylor coefficients is the coefficient for the sum of their slopes. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_exponential_coeff_convolution
lemma affine_exponential_coeff_convolution (x y : Real) (j : Nat) :
(∑ k ∈ Finset.antidiagonal j,
x ^ k.1 / (k.1.factorial : Real) * y ^ k.2 / (k.2.factorial : Real)) =
(x + y) ^ j / (j.factorial : Real) := by
rw [(Commute.all x y).add_pow', Finset.sum_div]
apply Finset.sum_congr rfl
intro k hk
have hkj : k.1 + k.2 = j := Finset.mem_antidiagonal.mp hk
have hfact : (j.choose k.1 : Real) * (k.1.factorial : Real) *
(k.2.factorial : Real) = (j.factorial : Real) := by
have h := Nat.choose_mul_factorial_mul_factorial
(show k.1 ≤ j by omega)
have hsub : j - k.1 = k.2 := by omega
rw [hsub] at h
exact_mod_cast h
simp only [nsmul_eq_mul]
field_simp
calc
_ = (x ^ k.1 * y ^ k.2) * (j.factorial : Real) := by ring
_ = _ := by rw [← hfact]; ring
Under the stated inputs and conditions, The two independent arm-count sums exchange with the finite coefficient convolution and retain their common constant exactly. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_two_arm_coeff_series_eq
lemma affine_two_arm_coeff_series_eq (a b c d t s : Real) (j : Nat) :
(∑' r : Nat, ∑' q : Nat, t ^ r / (r.factorial : Real) *
(s ^ q / (q.factorial : Real)) *
(((C a + C b * X) ^ r) *
((C c + C d * X) ^ q)).coeff j) =
exp (t * a + s * c) * (t * b + s * d) ^ j / (j.factorial : Real) := by
have hinner (r : Nat) :
(∑' q : Nat, t ^ r / (r.factorial : Real) * (s ^ q / (q.factorial : Real)) *
(((C a + C b * X) ^ r) *
((C c + C d * X) ^ q)).coeff j) =
∑ k ∈ Finset.antidiagonal j,
(t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff k.1) *
(exp (s * c) * (s * d) ^ k.2 / (k.2.factorial : Real)) := by
calc
_ = ∑' q : Nat, ∑ k ∈ Finset.antidiagonal j,
(t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff k.1) *
(s ^ q / (q.factorial : Real) *
((C c + C d * X) ^ q).coeff k.2) := by
apply tsum_congr
intro q
rw [Polynomial.coeff_mul, Finset.mul_sum]
apply Finset.sum_congr rfl
intro k _
ring
_ = _ := by
have hsum (k : Nat × Nat) (_ : k ∈ Finset.antidiagonal j) :
Summable (fun q : Nat =>
(t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff k.1) *
(s ^ q / (q.factorial : Real) *
((C c + C d * X) ^ q).coeff k.2)) :=
(affine_linear_coeff_series_summable c d s k.2).mul_left _
rw [Summable.tsum_finsetSum hsum]
simp_rw [tsum_mul_left, affine_linear_coeff_series_eq]
simp_rw [hinner]
have hsum (k : Nat × Nat) (_ : k ∈ Finset.antidiagonal j) :
Summable (fun r : Nat =>
(t ^ r / (r.factorial : Real) *
((C a + C b * X) ^ r).coeff k.1) *
(exp (s * c) * (s * d) ^ k.2 / (k.2.factorial : Real))) :=
(affine_linear_coeff_series_summable a b t k.1).mul_right _
rw [Summable.tsum_finsetSum hsum]
simp_rw [tsum_mul_right, affine_linear_coeff_series_eq]
calc
_ = exp (t * a + s * c) *
∑ k ∈ Finset.antidiagonal j,
(t * b) ^ k.1 / (k.1.factorial : Real) *
(s * d) ^ k.2 / (k.2.factorial : Real) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro k _
rw [Real.exp_add]
ring
_ = _ := by rw [affine_exponential_coeff_convolution]; ring
Under the stated inputs and conditions, The Palm shift in the labeled count decreases each coefficient of the nonnegative two-arm polynomial envelope. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_labeled_count_coeff_le
lemma affine_labeled_count_coeff_le (alpha b0 eps u : Real) (r c j : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) :
u ^ r / ((r + 1).factorial : Real) *
(((C alpha + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j ≤
u ^ r / (r.factorial : Real) *
(((C alpha + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j := by
exact mul_le_mul_of_nonneg_right (affine_labeled_factorial_term_le u hu r)
(affine_arm_power_coeff_nonneg alpha b0 eps r c j ha hb he he')
theorem affine_arm_count_coeff_baseline_cancel Lemma affine_arm_count_coeff_baseline_cancel in the paper ↗
Under the stated inputs and conditions, Summing the combined treated and control counts cancels the common baseline at every coefficient, without any bound on its size. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_arm_count_coeff_baseline_cancel
lemma affine_arm_count_coeff_baseline_cancel (b0 eps M : Real) (j : Nat) :
exp (-M * b0) *
(∑' r : Nat, ∑' c : Nat, M ^ r / (r.factorial : Real) *
(M ^ c / (c.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j) =
M ^ j / (j.factorial : Real) := by
rw [affine_two_arm_coeff_series_eq]
have hbase : M * (eps * b0) + M * ((1 - eps) * b0) = M * b0 := by ring
have hslope : M * (1 - eps) + M * eps = M := by ring
rw [hbase, hslope]
rw [← mul_div_assoc, ← mul_assoc, ← Real.exp_add]
simp
theorem affine_full_count_envelope_coeff_eq Lemma affine_full_count_envelope_coeff_eq in the paper ↗
Under the stated inputs and conditions, Convolving the coefficientwise count sum with the positive Taylor series of exp(Mv) leaves exactly the coefficients of exp(2Mv). This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ k ∈ Finset.antidiagonal h, M ^ k.1 / (k.1.factorial : Real) * (exp (-M * b0) * (∑' r : Nat, ∑' c : Nat, M ^ r / (r.factorial : Real) * (M ^ c / (c.factorial : Real)) * (((C (eps * b0) + C (1 - eps) * X) ^ r) * ((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff k.2)))
= (2 * M) ^ h / (h.factorial : Real)
Proof (Lean source)
-- @node: affine_full_count_envelope_coeff_eq
lemma affine_full_count_envelope_coeff_eq (b0 eps M : Real) (h : Nat) :
(∑ k ∈ Finset.antidiagonal h, M ^ k.1 / (k.1.factorial : Real) *
(exp (-M * b0) *
(∑' r : Nat, ∑' c : Nat, M ^ r / (r.factorial : Real) *
(M ^ c / (c.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff k.2))) =
(2 * M) ^ h / (h.factorial : Real) := by
simp_rw [affine_arm_count_coeff_baseline_cancel]
have hc := affine_exponential_coeff_convolution M M h
simpa only [mul_div_assoc, show M + M = 2 * M by ring] using hc
Under the stated inputs and conditions, Grouping the labeled and auxiliary treated counts by their sum gives the ordinary combined-intensity coefficient; the polynomial power is unchanged. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_treated_count_antidiagonal
lemma affine_treated_count_antidiagonal (u w : Real) (p : Polynomial Real) (R j : Nat) :
(∑ k ∈ Finset.antidiagonal R, u ^ k.1 / (k.1.factorial : Real) *
w ^ k.2 / (k.2.factorial : Real) * (p ^ (k.1 + k.2)).coeff j) =
(u + w) ^ R / (R.factorial : Real) * (p ^ R).coeff j := by
calc
_ = (∑ k ∈ Finset.antidiagonal R,
u ^ k.1 / (k.1.factorial : Real) * w ^ k.2 / (k.2.factorial : Real)) *
(p ^ R).coeff j := by
rw [Finset.sum_mul]
apply Finset.sum_congr rfl
intro k hk
rw [Finset.mem_antidiagonal.mp hk]
_ = _ := by rw [affine_exponential_coeff_convolution]
Under the stated inputs and conditions, At the support endpoint, the full coefficientwise count envelope has exactly the Taylor tail used in equation (7). No common baseline enters the bound. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑' h : Nat, if L < h then B ^ h * (∑ k ∈ Finset.antidiagonal h, M ^ k.1 / (k.1.factorial : Real) * (exp (-M * b0) * (∑' r : Nat, ∑' c : Nat, M ^ r / (r.factorial : Real) * (M ^ c / (c.factorial : Real)) * (((C (eps * b0) + C (1 - eps) * X) ^ r) * ((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff k.2))) else 0)
Proof (Lean source)
-- @node: affine_full_count_endpoint_tail_eq
lemma affine_full_count_endpoint_tail_eq (b0 eps M B : Real) (L : Nat) :
(∑' h : Nat, if L < h then B ^ h *
(∑ k ∈ Finset.antidiagonal h, M ^ k.1 / (k.1.factorial : Real) *
(exp (-M * b0) *
(∑' r : Nat, ∑' c : Nat, M ^ r / (r.factorial : Real) *
(M ^ c / (c.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff k.2)))
else 0) =
∑' h : Nat, if L < h then (2 * M * B) ^ h / (h.factorial : Real) else 0 := by
apply tsum_congr
intro h
rw [affine_full_count_envelope_coeff_eq]
split_ifs
· rw [mul_pow]
ring
· rfl
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.CountSummation 8 declarations Absolutely convergent count sums and the Palm-shifted coefficient envelope for one rare cell.
Absolutely convergent count sums and the Palm-shifted coefficient envelope for one rare cell. These bounds retain the common control baseline exactly.
theorem affine_two_arm_coeff_series_summable Lemma affine_two_arm_coeff_series_summable in the paper ↗
Under the stated inputs and conditions, Absolute convergence justifies regrouping the two count indices at any fixed Taylor coefficient. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_two_arm_coeff_series_summable
lemma affine_two_arm_coeff_series_summable (a b c d t s : Real) (j : Nat) :
Summable (fun k : Nat × Nat => t ^ k.1 / (k.1.factorial : Real) *
(s ^ k.2 / (k.2.factorial : Real)) *
(((C a + C b * X) ^ k.1) *
((C c + C d * X) ^ k.2)).coeff j) := by
have hs (v : Nat × Nat) (_ : v ∈ Finset.antidiagonal j) :
Summable (fun k : Nat × Nat =>
(t ^ k.1 / (k.1.factorial : Real) *
((C a + C b * X) ^ k.1).coeff v.1) *
(s ^ k.2 / (k.2.factorial : Real) *
((C c + C d * X) ^ k.2).coeff v.2)) := by
exact summable_mul_of_summable_norm
(summable_norm_iff.mpr (affine_linear_coeff_series_summable a b t v.1))
(summable_norm_iff.mpr (affine_linear_coeff_series_summable c d s v.2))
convert summable_sum hs using 1
ext k
rw [Polynomial.coeff_mul, Finset.mul_sum]
apply Finset.sum_congr rfl
intro v _
ring
theorem affine_labeled_two_arm_coeff_series_summable Lemma affine_labeled_two_arm_coeff_series_summable in the paper ↗
Under the stated hypotheses, The labeled Palm shift preserves absolute convergence of the joint count sum. This gives the stated result.
Formal statement
hypotheses
alpha b0 eps u M :
j :
ha :
0 ≤ alpha
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hM :
0 ≤ M
Proof (Lean source)
-- @node: affine_labeled_two_arm_coeff_series_summable
lemma affine_labeled_two_arm_coeff_series_summable (alpha b0 eps u M : Real) (j : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hM : 0 ≤ M) :
Summable (fun k : Nat × Nat => u ^ k.1 / ((k.1 + 1).factorial : Real) *
(M ^ k.2 / (k.2.factorial : Real)) *
(((C alpha + C (1 - eps) * X) ^ k.1) *
((C ((1 - eps) * b0) + C eps * X) ^ k.2)).coeff j) := by
apply (affine_two_arm_coeff_series_summable alpha (1 - eps)
((1 - eps) * b0) eps u M j).of_nonneg_of_le
· intro k
exact mul_nonneg (by positivity)
(affine_arm_power_coeff_nonneg alpha b0 eps k.1 k.2 j ha hb he he')
· intro k
have h := affine_labeled_count_coeff_le alpha b0 eps u k.1 k.2 j ha hb he he' hu
have hc : 0 ≤ M ^ k.2 / (k.2.factorial : Real) := by positivity
convert mul_le_mul_of_nonneg_right h hc using 1 <;> first | rfl | ring
theorem affine_labeled_two_arm_coeff_series_le Lemma affine_labeled_two_arm_coeff_series_le in the paper ↗
Under the stated inputs and conditions, After summing both counts, the Palm-shifted coefficients are dominated by those of the ordinary two-arm exponential. This gives the stated result.
Formal statement
hypotheses
alpha b0 eps u M :
j :
ha :
0 ≤ alpha
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hM :
0 ≤ M
Proof (Lean source)
-- @node: affine_labeled_two_arm_coeff_series_le
lemma affine_labeled_two_arm_coeff_series_le (alpha b0 eps u M : Real) (j : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hM : 0 ≤ M) :
(∑' r : Nat, ∑' c : Nat, u ^ r / ((r + 1).factorial : Real) *
(M ^ c / (c.factorial : Real)) *
(((C alpha + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j) ≤
exp (u * alpha + M * ((1 - eps) * b0)) *
(u * (1 - eps) + M * eps) ^ j / (j.factorial : Real) := by
rw [← affine_two_arm_coeff_series_eq alpha (1 - eps) ((1 - eps) * b0) eps u M j]
have hl := affine_labeled_two_arm_coeff_series_summable alpha b0 eps u M j
ha hb he he' hu hM
have hr := affine_two_arm_coeff_series_summable alpha (1 - eps) ((1 - eps) * b0) eps u M j
rw [← hl.tsum_prod' hl.prod_factor, ← hr.tsum_prod' hr.prod_factor]
apply hl.tsum_le_tsum _ hr
intro k
have h := affine_labeled_count_coeff_le alpha b0 eps u k.1 k.2 j ha hb he he' hu
have hc : 0 ≤ M ^ k.2 / (k.2.factorial : Real) := by positivity
convert mul_le_mul_of_nonneg_right h hc using 1 <;> first | rfl | ring
theorem affine_labeled_count_coeff_baseline_le Lemma affine_labeled_count_coeff_baseline_le in the paper ↗
Under the stated inputs and conditions, When treated channels have been combined, the Palm envelope cancels the common baseline, coefficient by coefficient. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_labeled_count_coeff_baseline_le
lemma affine_labeled_count_coeff_baseline_le (b0 eps M : Real) (j : Nat)
(hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1) (hM : 0 ≤ M) :
exp (-M * b0) *
(∑' r : Nat, ∑' c : Nat, M ^ r / ((r + 1).factorial : Real) *
(M ^ c / (c.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * X) ^ r) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j) ≤
M ^ j / (j.factorial : Real) := by
have h := mul_le_mul_of_nonneg_left
(affine_labeled_two_arm_coeff_series_le (eps * b0) b0 eps M M j
(mul_nonneg he hb) hb he he' hM hM) (Real.exp_pos (-M * b0)).le
have hbase : M * (eps * b0) + M * ((1 - eps) * b0) = M * b0 := by ring
have hslope : M * (1 - eps) + M * eps = M := by ring
rw [hbase, hslope, ← mul_div_assoc, ← mul_assoc, ← Real.exp_add] at h
simpa using h
theorem affine_treated_count_series_summable Lemma affine_treated_count_series_summable in the paper ↗
Under the stated inputs and conditions, Absolute convergence of both treated channels remains valid after multiplying by a fixed control-count polynomial. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_treated_count_series_summable
lemma affine_treated_count_series_summable (a b u w : Real)
(q : Polynomial Real) (j : Nat) :
Summable (fun k : Nat × Nat => u ^ k.1 / (k.1.factorial : Real) *
(w ^ k.2 / (k.2.factorial : Real)) *
(((C a + C b * X) ^ (k.1 + k.2)) * q).coeff j) := by
have hs (v : Nat × Nat) (_ : v ∈ Finset.antidiagonal j) :
Summable (fun k : Nat × Nat =>
(u ^ k.1 / (k.1.factorial : Real) * (w ^ k.2 / (k.2.factorial : Real)) *
(((C a + C b * X) ^ k.1) *
((C a + C b * X) ^ k.2)).coeff v.1) *
q.coeff v.2) :=
(affine_two_arm_coeff_series_summable a b a b u w v.1).mul_right _
convert summable_sum hs using 1
ext k
rw [pow_add, Polynomial.coeff_mul, Finset.mul_sum]
apply Finset.sum_congr rfl
intro v _
ring
Under the stated inputs and conditions, Regrouping the complete and auxiliary treated counts gives intensity u+w, with every fixed control count retained. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_treated_count_series_eq
lemma affine_treated_count_series_eq (a b u w : Real)
(q : Polynomial Real) (j : Nat) :
(∑' r : Nat, ∑' g : Nat, u ^ r / (r.factorial : Real) *
(w ^ g / (g.factorial : Real)) *
(((C a + C b * Polynomial.X) ^ (r + g)) * q).coeff j) =
∑' R : Nat, (u + w) ^ R / (R.factorial : Real) *
(((C a + C b * Polynomial.X) ^ R) * q).coeff j := by
let p : Polynomial Real := C a + C b * Polynomial.X
let f : Nat × Nat → Real := fun k => u ^ k.1 / (k.1.factorial : Real) *
(w ^ k.2 / (k.2.factorial : Real)) * ((p ^ (k.1 + k.2)) * q).coeff j
have hf : Summable f := affine_treated_count_series_summable a b u w q j
rw [← hf.tsum_prod' hf.prod_factor]
have hs : Summable (fun k : Σ R : Nat, Finset.antidiagonal R => f k.2) :=
Finset.HasAntidiagonal.sigmaAntidiagonalEquivProd.summable_iff.mpr hf
rw [← Finset.HasAntidiagonal.sigmaAntidiagonalEquivProd.tsum_eq f]
simp only [Finset.HasAntidiagonal.sigmaAntidiagonalEquivProd_apply]
rw [hs.tsum_sigma' (fun R => (hasSum_fintype _).summable)]
apply tsum_congr
intro R
rw [tsum_fintype]
change (∑ k : Finset.antidiagonal R, f k) = _
rw [Finset.sum_coe_sort]
calc
_ = (∑ k ∈ Finset.antidiagonal R,
u ^ k.1 / (k.1.factorial : Real) * (w ^ k.2 / (k.2.factorial : Real))) *
((p ^ R) * q).coeff j := by
rw [Finset.sum_mul]
apply Finset.sum_congr rfl
intro k hk
simp only [f, Finset.mem_antidiagonal.mp hk]
_ = _ := by
have hc : (∑ k ∈ Finset.antidiagonal R,
u ^ k.1 / (k.1.factorial : Real) * (w ^ k.2 / (k.2.factorial : Real))) =
(u + w) ^ R / (R.factorial : Real) := by
simpa only [mul_div_assoc] using affine_exponential_coeff_convolution u w R
rw [← hc]
Under the stated inputs and conditions, The labeled factorial shift decreases the joint treated-channel coefficient sum, while retaining the entire control-count polynomial. This gives the stated result.
Formal statement
hypotheses
alpha b0 eps u w :
c j :
ha :
0 ≤ alpha
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
Proof (Lean source)
-- @node: affine_labeled_treated_series_le
lemma affine_labeled_treated_series_le (alpha b0 eps u w : Real) (c j : Nat)
(ha : 0 ≤ alpha) (hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) :
(∑' r : Nat, ∑' g : Nat, u ^ r / ((r + 1).factorial : Real) *
(w ^ g / (g.factorial : Real)) *
(((C alpha + C (1 - eps) * Polynomial.X) ^ (r + g)) *
((C ((1 - eps) * b0) + C eps * Polynomial.X) ^ c)).coeff j) ≤
∑' R : Nat, (u + w) ^ R / (R.factorial : Real) *
(((C alpha + C (1 - eps) * Polynomial.X) ^ R) *
((C ((1 - eps) * b0) + C eps * Polynomial.X) ^ c)).coeff j := by
let p : Polynomial Real := C alpha + C (1 - eps) * Polynomial.X
let q : Polynomial Real := C ((1 - eps) * b0) + C eps * Polynomial.X
let f : Nat × Nat → Real := fun k => u ^ k.1 / (k.1.factorial : Real) *
(w ^ k.2 / (k.2.factorial : Real)) * ((p ^ (k.1 + k.2)) * q ^ c).coeff j
let g : Nat × Nat → Real := fun k => u ^ k.1 / ((k.1 + 1).factorial : Real) *
(w ^ k.2 / (k.2.factorial : Real)) * ((p ^ (k.1 + k.2)) * q ^ c).coeff j
have hcoeff (k : Nat × Nat) : 0 ≤ ((p ^ (k.1 + k.2)) * q ^ c).coeff j :=
affine_arm_power_coeff_nonneg alpha b0 eps (k.1 + k.2) c j ha hb he he'
have hnonneg (k : Nat × Nat) : 0 ≤ g k := mul_nonneg (by positivity) (hcoeff k)
have hle (k : Nat × Nat) : g k ≤ f k := by
have h := mul_le_mul_of_nonneg_right (affine_labeled_factorial_term_le u hu k.1)
(mul_nonneg (show 0 ≤ w ^ k.2 / (k.2.factorial : Real) by positivity) (hcoeff k))
simpa only [g, f, mul_assoc] using h
have hf : Summable f := affine_treated_count_series_summable alpha (1 - eps) u w (q ^ c) j
have hg : Summable g := hf.of_nonneg_of_le hnonneg hle
rw [← affine_treated_count_series_eq alpha (1 - eps) u w (q ^ c) j]
change (∑' r, ∑' g', g (r, g')) ≤ ∑' r, ∑' g', f (r, g')
rw [← hg.tsum_prod' hg.prod_factor, ← hf.tsum_prod' hf.prod_factor]
exact hg.tsum_le_tsum hle hf
theorem affine_three_count_coeff_baseline_le Lemma affine_three_count_coeff_baseline_le in the paper ↗
Under the stated inputs and conditions, The three count indices have the coefficient envelope asserted in equation (7): the two treated intensities combine, and the baseline cancels exactly. This gives the stated result.
Formal statement
hypotheses
b0 eps u w M :
j :
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
hM :
M = u + w
Proof (Lean source)
-- @node: affine_three_count_coeff_baseline_le
lemma affine_three_count_coeff_baseline_le (b0 eps u w M : Real) (j : Nat)
(hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (hM : M = u + w) :
exp (-M * b0) *
(∑' c : Nat, M ^ c / (c.factorial : Real) *
(∑' r : Nat, ∑' g : Nat, u ^ r / ((r + 1).factorial : Real) *
(w ^ g / (g.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * Polynomial.X) ^ (r + g)) *
((C ((1 - eps) * b0) + C eps * Polynomial.X) ^ c)).coeff j)) ≤
M ^ j / (j.factorial : Real) := by
let p : Polynomial Real := C (eps * b0) + C (1 - eps) * Polynomial.X
let q : Polynomial Real := C ((1 - eps) * b0) + C eps * Polynomial.X
let F : Nat → Real := fun c => M ^ c / (c.factorial : Real) *
(∑' r : Nat, ∑' g : Nat, u ^ r / ((r + 1).factorial : Real) *
(w ^ g / (g.factorial : Real)) * ((p ^ (r + g)) * q ^ c).coeff j)
let G : Nat → Real := fun c => M ^ c / (c.factorial : Real) *
(∑' R : Nat, M ^ R / (R.factorial : Real) * ((p ^ R) * q ^ c).coeff j)
have hM0 : 0 ≤ M := by rw [hM]; positivity
have hle (c : Nat) : F c ≤ G c := by
have h := affine_labeled_treated_series_le (eps * b0) b0 eps u w c j
(mul_nonneg he hb) hb he he' hu hw
rw [← hM] at h
exact mul_le_mul_of_nonneg_left h (by positivity)
have hn (c : Nat) : 0 ≤ F c := by
apply mul_nonneg (by positivity)
apply tsum_nonneg
intro r
apply tsum_nonneg
intro g
exact mul_nonneg (by positivity)
(affine_arm_power_coeff_nonneg (eps * b0) b0 eps (r + g) c j
(mul_nonneg he hb) hb he he')
have hr : Summable (fun k : Nat × Nat => M ^ k.1 / (k.1.factorial : Real) *
(M ^ k.2 / (k.2.factorial : Real)) * ((p ^ k.1) * q ^ k.2).coeff j) :=
affine_two_arm_coeff_series_summable (eps * b0) (1 - eps)
((1 - eps) * b0) eps M M j
have hG : Summable G := by
apply hr.prod_symm.prod.congr
intro c
simp only [G, Prod.swap_prod_mk, ← tsum_mul_left]
apply tsum_congr
intro R
ring
have hF : Summable F := hG.of_nonneg_of_le hn hle
have heq : exp (-M * b0) * (∑' c, G c) = M ^ j / (j.factorial : Real) := by
have hc := Summable.tsum_comm (f := fun R c : Nat =>
M ^ R / (R.factorial : Real) * (M ^ c / (c.factorial : Real)) *
((p ^ R) * q ^ c).coeff j) hr
have hsum : (∑' c, G c) = ∑' R : Nat, ∑' c : Nat, M ^ R / (R.factorial : Real) *
(M ^ c / (c.factorial : Real)) * ((p ^ R) * q ^ c).coeff j := by
rw [← hc]
simp only [G, ← tsum_mul_left]
apply tsum_congr
intro c
apply tsum_congr
intro R
ring
rw [hsum]
exact affine_arm_count_coeff_baseline_cancel b0 eps M j
change exp (-M * b0) * (∑' c, F c) ≤ _
rw [← heq]
exact mul_le_mul_of_nonneg_left (hF.tsum_le_tsum hle hG) (Real.exp_pos _).le
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.CountTail 4 declarations Exchange of the count-summed coefficient envelope with the positive exponential series.
Exchange of the count-summed coefficient envelope with the positive exponential series. The total-degree cutoff reads back the one-cell Taylor tail without a bound on the common control baseline.
theorem affine_filtered_exponential_product_summable Lemma affine_filtered_exponential_product_summable in the paper ↗
Under the stated hypotheses, The positive two-series envelope is absolutely summable even after the matching-degree cutoff is imposed. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_filtered_exponential_product_summable
lemma affine_filtered_exponential_product_summable (z : Real) (hz : 0 ≤ z) (L : Nat) :
Summable (fun k : Nat × Nat =>
if L < k.1 + k.2 then
z ^ k.1 / (k.1.factorial : Real) * z ^ k.2 / (k.2.factorial : Real)
else 0) := by
have hs := summable_mul_of_summable_norm
(summable_norm_iff.mpr (Real.summable_pow_div_factorial z))
(summable_norm_iff.mpr (Real.summable_pow_div_factorial z))
apply hs.of_nonneg_of_le
· intro k; split_ifs <;> positivity
· intro k
split_ifs
· simp only [mul_div_assoc]
exact le_rfl
· positivity
theorem affine_exponential_total_degree_tail_eq Lemma affine_exponential_total_degree_tail_eq in the paper ↗
Under the stated inputs and conditions, Regrouping the two positive exponential series by total degree gives the Taylor tail of exp(2z), retaining precisely the degrees above L. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_exponential_total_degree_tail_eq
lemma affine_exponential_total_degree_tail_eq (z : Real) (hz : 0 ≤ z) (L : Nat) :
(∑' j : Nat, ∑' k : Nat, if L < j + k then
z ^ j / (j.factorial : Real) * z ^ k / (k.factorial : Real) else 0) =
∑' h : Nat, if L < h then (2 * z) ^ h / (h.factorial : Real) else 0 := by
let f : Nat × Nat → Real := fun k => if L < k.1 + k.2 then
z ^ k.1 / (k.1.factorial : Real) * z ^ k.2 / (k.2.factorial : Real) else 0
have hf : Summable f := affine_filtered_exponential_product_summable z hz L
rw [← hf.tsum_prod' hf.prod_factor]
have hs : Summable (fun k : Σ h : Nat, Finset.antidiagonal h => f k.2) :=
Finset.HasAntidiagonal.sigmaAntidiagonalEquivProd.summable_iff.mpr hf
rw [← Finset.HasAntidiagonal.sigmaAntidiagonalEquivProd.tsum_eq f]
simp only [Finset.HasAntidiagonal.sigmaAntidiagonalEquivProd_apply]
rw [hs.tsum_sigma' (fun h => (hasSum_fintype _).summable)]
apply tsum_congr
intro h
rw [tsum_fintype, Finset.sum_coe_sort]
by_cases hh : L < h
· calc
_ = ∑ k ∈ Finset.antidiagonal h,
z ^ k.1 / (k.1.factorial : Real) * z ^ k.2 / (k.2.factorial : Real) := by
apply Finset.sum_congr rfl
intro k hk
simp only [f, Finset.mem_antidiagonal.mp hk, if_pos hh]
_ = _ := by
simpa only [if_pos hh, mul_div_assoc, show z + z = 2 * z by ring] using
affine_exponential_coeff_convolution z z h
· simp only [if_neg hh]
apply Finset.sum_eq_zero
intro k hk
simp only [f, Finset.mem_antidiagonal.mp hk, if_neg hh]
theorem affine_dominated_coeff_total_degree_tail_le Lemma affine_dominated_coeff_total_degree_tail_le in the paper ↗
Under the stated inputs and conditions, Any nonnegative coefficient envelope dominated by the baseline-canceled Poisson coefficients has total-degree tail at most that of exp(2MB). This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_dominated_coeff_total_degree_tail_le
lemma affine_dominated_coeff_total_degree_tail_le (A : Nat → Real) (M B : Real)
(L : Nat) (hM : 0 ≤ M) (hB : 0 ≤ B)
(hA : ∀ j, 0 ≤ A j) (hle : ∀ j, A j ≤ M ^ j / (j.factorial : Real)) :
(∑' j : Nat, A j * B ^ j *
∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0) ≤
∑' h : Nat, if L < h then (2 * M * B) ^ h / (h.factorial : Real) else 0 := by
let F : Nat × Nat → Real := fun v => if L < v.1 + v.2 then
A v.1 * B ^ v.1 * (M * B) ^ v.2 / (v.2.factorial : Real) else 0
let G : Nat × Nat → Real := fun v => if L < v.1 + v.2 then
(M * B) ^ v.1 / (v.1.factorial : Real) *
(M * B) ^ v.2 / (v.2.factorial : Real) else 0
have hn (v : Nat × Nat) : 0 ≤ F v := by
have hAv := hA v.1
dsimp [F]; split_ifs <;> positivity
have hFG (v : Nat × Nat) : F v ≤ G v := by
dsimp [F, G]
split_ifs
· have h := mul_le_mul_of_nonneg_right (hle v.1)
(show 0 ≤ B ^ v.1 * ((M * B) ^ v.2 / (v.2.factorial : Real)) by positivity)
convert h using 1 <;> first | rfl | (try simp only [mul_pow]; ring)
· exact le_rfl
have hG : Summable G :=
affine_filtered_exponential_product_summable (M * B) (mul_nonneg hM hB) L
have hF : Summable F := hG.of_nonneg_of_le hn hFG
have heq : (∑' j : Nat, A j * B ^ j *
∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0) =
∑' j : Nat, ∑' k : Nat, F (j, k) := by
apply tsum_congr
intro j
rw [← tsum_mul_left]
apply tsum_congr
intro k
dsimp [F]
rw [Nat.add_comm j k]
split_ifs <;> simp [mul_div_assoc]
rw [heq, ← hF.tsum_prod' hF.prod_factor]
calc
_ ≤ ∑' v, G v := hF.tsum_le_tsum hFG hG
_ = ∑' j : Nat, ∑' k : Nat, G (j, k) := hG.tsum_prod' hG.prod_factor
_ = _ := by
simpa only [G, mul_assoc] using
affine_exponential_total_degree_tail_eq (M * B) (mul_nonneg hM hB) L
theorem affine_three_count_total_degree_tail_le Lemma affine_three_count_total_degree_tail_le in the paper ↗
Under the stated inputs and conditions, Equation (7)'s count-summed Taylor envelope is bounded by the one-cell exponential tail. Both treated channels and all control counts are retained. This gives the stated result.
Formal statement
hypotheses
b0 eps u w M B :
L :
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
hM :
M = u + w
hB :
0 ≤ B
conclusion
(∑' j : Nat, (exp (-M * b0) * (∑' c : Nat, M ^ c / (c.factorial : Real) * (∑' r : Nat, ∑' g : Nat, u ^ r / ((r + 1).factorial : Real) * (w ^ g / (g.factorial : Real)) * (((C (eps * b0) + C (1 - eps) * X) ^ (r + g)) * ((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j))) * B ^ j * ∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0)
Proof (Lean source)
-- @node: affine_three_count_total_degree_tail_le
lemma affine_three_count_total_degree_tail_le (b0 eps u w M B : Real) (L : Nat)
(hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (hM : M = u + w) (hB : 0 ≤ B) :
(∑' j : Nat, (exp (-M * b0) *
(∑' c : Nat, M ^ c / (c.factorial : Real) *
(∑' r : Nat, ∑' g : Nat, u ^ r / ((r + 1).factorial : Real) *
(w ^ g / (g.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * X) ^ (r + g)) *
((C ((1 - eps) * b0) + C eps * X) ^ c)).coeff j))) *
B ^ j * ∑' k : Nat, if L < k + j then
(M * B) ^ k / (k.factorial : Real) else 0) ≤
∑' h : Nat, if L < h then (2 * M * B) ^ h / (h.factorial : Real) else 0 := by
have hM0 : 0 ≤ M := by rw [hM]; positivity
apply affine_dominated_coeff_total_degree_tail_le _ M B L hM0 hB
· intro j
apply mul_nonneg (Real.exp_pos _).le
apply tsum_nonneg
intro c
apply mul_nonneg (by positivity)
apply tsum_nonneg
intro r
apply tsum_nonneg
intro g
exact mul_nonneg (by positivity)
(affine_arm_power_coeff_nonneg (eps * b0) b0 eps (r + g) c j
(mul_nonneg he hb) hb he he')
· intro j
exact affine_three_count_coeff_baseline_le b0 eps u w M j hb he he' hu hw hM
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.FixedExperiment 18 declarations The tagged affine parameter's original fixed experiment, exact ordered prefix retention with its independent seed, and the conditional and predictive failure budgets.
The tagged affine parameter's original fixed experiment, exact ordered prefix retention with its independent seed, and the conditional and predictive failure budgets. These are the experiment leaves for the final minimax transfer.
def affineTaggedCompleteKernel
The complete mark kernel retains the hypothesis and its latent vector.
Definition (Lean source)
parameters
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
def
given by
Kernel.ofFunOfCountable fun theta
=> (normalizedLaw theta.1 n m d eps hd sigma theta.2).pmf.toMeasure
def affineTaggedAuxiliaryKernel
The auxiliary mark kernel is the full normalized arm marginal.
Definition (Lean source)
parameters
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
def
given by
Kernel.ofFunOfCountable fun theta
=> (auxMarginal (normalizedLaw theta.1 n m d eps hd sigma theta.2)).toMeasure
instance affineTaggedCompleteKernel_isProbabilityMeasure
Every complete mark law is a probability measure.
Definition (Lean source)
-- @node: affineTaggedCompleteKernel_isProbabilityMeasure
instance affineTaggedCompleteKernel_isProbabilityMeasure (theta : AffineRawParameter n m d eps) :
IsProbabilityMeasure (affineTaggedCompleteKernel n m d eps hd sigma theta) := by
change IsProbabilityMeasure (normalizedLaw theta.1 n m d eps hd sigma theta.2).pmf.toMeasure
infer_instance
instance affineTaggedAuxiliaryKernel_isProbabilityMeasure
Every auxiliary mark law is a probability measure.
Definition (Lean source)
-- @node: affineTaggedAuxiliaryKernel_isProbabilityMeasure
instance affineTaggedAuxiliaryKernel_isProbabilityMeasure (theta : AffineRawParameter n m d eps) :
IsProbabilityMeasure (affineTaggedAuxiliaryKernel n m d eps hd sigma theta) := by
change IsProbabilityMeasure (auxMarginal (normalizedLaw theta.1 n m d eps hd sigma theta.2)).toMeasure
infer_instance
def affineTaggedOrderedKernel
The unseeded tagged raw experiment has the actual ordered conditional laws.
Definition (Lean source)
parameters
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
def
affineTaggedOrderedKernel n m d eps hd sigma :
given by
Kernel.ofFunOfCountable fun theta
=> affineOrderedRawKernel n m d eps hd sigma theta.1 theta.2
instance affineTaggedOrderedKernel_isMarkovKernel
The ordered tagged raw laws are probability measures.
Definition (Lean source)
-- @node: affineTaggedOrderedKernel_isMarkovKernel
instance affineTaggedOrderedKernel_isMarkovKernel :
IsMarkovKernel (affineTaggedOrderedKernel n m d eps hd sigma) where
isProbabilityMeasure theta := by
change IsProbabilityMeasure (affineOrderedRawKernel n m d eps hd sigma theta.1 theta.2)
infer_instance
def affineTaggedFixedKernel
The fixed experiment keeps both full original ordered arrays.
Definition (Lean source)
parameters
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
def
given by
Kernel.ofFunOfCountable fun theta
=> annotationLaw (normalizedLaw theta.1 n m d eps hd sigma theta.2) n m
theorem affineTaggedFixedKernel_isFixedTwoPool
The fixed tagged experiment satisfies the exact unequal two-pool interface.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
conclusion
IsFixedTwoPoolExperiment n m (affineTaggedFixedKernel n m d eps hd sigma) (affineTaggedCompleteKernel n m d eps hd sigma) (affineTaggedAuxiliaryKernel n m d eps hd sigma)
Proof (Lean source)
-- @node: affineTaggedFixedKernel_isFixedTwoPool
lemma affineTaggedFixedKernel_isFixedTwoPool :
IsFixedTwoPoolExperiment n m (affineTaggedFixedKernel n m d eps hd sigma)
(affineTaggedCompleteKernel n m d eps hd sigma)
(affineTaggedAuxiliaryKernel n m d eps hd sigma) := by
intro theta
rfl
theorem affineTaggedOrderedKernel_isRandomScale
Retaining the hypothesis tag changes neither scaled intensity nor mark law.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
conclusion
IsRandomScaleTwoPoolExperiment (affineTaggedOrderedKernel n m d eps hd sigma) (affineTaggedCompleteKernel n m d eps hd sigma) (affineTaggedAuxiliaryKernel n m d eps hd sigma) (fun theta => toNNReal (rawNormalizer n m d eps sigma theta.2)) (toNNReal (affineTuning n m d eps).u) (toNNReal (affineTuning n m d eps).w)
Proof (Lean source)
-- @node: affineTaggedOrderedKernel_isRandomScale
lemma affineTaggedOrderedKernel_isRandomScale :
IsRandomScaleTwoPoolExperiment (affineTaggedOrderedKernel n m d eps hd sigma)
(affineTaggedCompleteKernel n m d eps hd sigma)
(affineTaggedAuxiliaryKernel n m d eps hd sigma)
(fun theta => toNNReal (rawNormalizer n m d eps sigma theta.2))
(toNNReal (affineTuning n m d eps).u)
(toNNReal (affineTuning n m d eps).w) := by
intro theta
exact affineOrderedRawKernel_isRandomScale n m d eps hd sigma theta.1 theta.2
theorem affineTaggedOrderedKernel_retention_success
Under the stated inputs and conditions, Exact retention on joint success gives the original fixed data law times its success probability, for every parameter rather than merely on average. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
fallbackX :
Fin n → Obs d
theta :
AffineRawParameter n m d eps
conclusion
((affineTaggedOrderedKernel n m d eps hd sigma theta).restrict (retentionSuccessSet n m)).map (twoPoolOrderedRetention fallbackX fallbackY)
= (poissonMeasure (toNNReal ((affineTuning n m d eps).u * rawNormalizer n m d eps sigma theta.2)) (Ici n) * poissonMeasure (toNNReal ((affineTuning n m d eps).w * rawNormalizer n m d eps sigma theta.2)) (Ici m)) • affineTaggedFixedKernel n m d eps hd sigma theta
Proof (Lean source)
-- @node: affineTaggedOrderedKernel_retention_success
lemma affineTaggedOrderedKernel_retention_success
(fallbackX : Fin n → Obs d) (fallbackY : Fin m → AuxObs d)
(theta : AffineRawParameter n m d eps) :
((affineTaggedOrderedKernel n m d eps hd sigma theta).restrict
(retentionSuccessSet n m)).map (twoPoolOrderedRetention fallbackX fallbackY) =
(poissonMeasure (toNNReal ((affineTuning n m d eps).u *
rawNormalizer n m d eps sigma theta.2)) (Ici n) *
poissonMeasure (toNNReal ((affineTuning n m d eps).w *
rawNormalizer n m d eps sigma theta.2)) (Ici m)) •
affineTaggedFixedKernel n m d eps hd sigma theta := by
exact map_twoPoolOrderedRetention_restrict_success
(normalizedLaw theta.1 n m d eps hd sigma theta.2).pmf.toMeasure
(auxMarginal (normalizedLaw theta.1 n m d eps hd sigma theta.2)).toMeasure
_ _ n m fallbackX fallbackY
theorem affineRawTaggedKernel_retention_success
Under the stated inputs and conditions, The same exact retention identity keeps the independent uniform seed. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
fallbackX :
Fin n → Obs d
theta :
AffineRawParameter n m d eps
conclusion
((affineRawTaggedKernel n m d eps hd sigma theta).restrict ((retentionSuccessSet (X := Obs d) (Y := AuxObs d) n m) ×ˢ univ)).map (map (twoPoolOrderedRetention fallbackX fallbackY) id)
= (poissonMeasure (toNNReal ((affineTuning n m d eps).u * rawNormalizer n m d eps sigma theta.2)) (Ici n) * poissonMeasure (toNNReal ((affineTuning n m d eps).w * rawNormalizer n m d eps sigma theta.2)) (Ici m)) • ((affineTaggedFixedKernel n m d eps hd sigma theta).prod seedLaw)
Proof (Lean source)
-- @node: affineRawTaggedKernel_retention_success
lemma affineRawTaggedKernel_retention_success
(fallbackX : Fin n → Obs d) (fallbackY : Fin m → AuxObs d)
(theta : AffineRawParameter n m d eps) :
((affineRawTaggedKernel n m d eps hd sigma theta).restrict
((retentionSuccessSet (X := Obs d) (Y := AuxObs d) n m) ×ˢ univ)).map
(map (twoPoolOrderedRetention fallbackX fallbackY) id) =
(poissonMeasure (toNNReal ((affineTuning n m d eps).u *
rawNormalizer n m d eps sigma theta.2)) (Ici n) *
poissonMeasure (toNNReal ((affineTuning n m d eps).w *
rawNormalizer n m d eps sigma theta.2)) (Ici m)) •
((affineTaggedFixedKernel n m d eps hd sigma theta).prod seedLaw) := by
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
change (((affineTaggedOrderedKernel n m d eps hd sigma theta).prod seedLaw).restrict
(_ ×ˢ univ)).map _ = _
rw [← Measure.prod_restrict, Measure.restrict_univ,
← Measure.map_prod_map _ _ (measurable_twoPoolOrderedRetention fallbackX fallbackY)
measurable_id, Measure.map_id,
affineTaggedOrderedKernel_retention_success, Measure.prod_smul_left]
theorem affineTaggedOrderedKernel_failure_gap
Under the stated hypotheses, Conditional count failure fits the half-gap budget for every raw table. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hn :
1 ≤ n
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
theta :
AffineRawParameter n m d eps
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
affineTaggedOrderedKernel n m d eps hd sigma theta (retentionFailureSet n m)
≤ ofReal (gap ^ 2 / 128)
Proof (Lean source)
-- @node: affineTaggedOrderedKernel_failure_gap
lemma affineTaggedOrderedKernel_failure_gap
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(theta : AffineRawParameter n m d eps) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
affineTaggedOrderedKernel n m d eps hd sigma theta (retentionFailureSet n m) ≤
ofReal (gap ^ 2 / 128) := by
intro nu gap
have htail := conditional_retentionFailure_le_poissonLowerTails
(affineTaggedOrderedKernel n m d eps hd sigma)
(affineTaggedCompleteKernel n m d eps hd sigma)
(affineTaggedAuxiliaryKernel n m d eps hd sigma)
(fun theta => toNNReal (rawNormalizer n m d eps sigma theta.2))
(toNNReal (affineTuning n m d eps).u)
(toNNReal (affineTuning n m d eps).w)
(affineTaggedOrderedKernel_isRandomScale n m d eps hd sigma) n m theta
rw [← Real.toNNReal_mul (by change 0 ≤ 128 * (n : Real); positivity),
← Real.toNNReal_mul (by change 0 ≤ 128 * ((n : Real) + m); positivity)] at htail
exact htail.trans (affine_prefix_failure_sum_gap n m d eps hn hd heps heps' hS hx sigma theta.2)
theorem affineTaggedOrderedKernel_predictive_failure_gap
Under the stated hypotheses, Integrating the uniform tablewise budget gives the same predictive budget under either tagged prior; no low-scale exception is needed. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hn :
1 ≤ n
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
hyp :
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
priorPredictive (affineRawTaggedPrior n m d eps sigma hyp) (affineTaggedOrderedKernel n m d eps hd sigma) (retentionFailureSet n m)
≤ ofReal (gap ^ 2 / 128)
Proof (Lean source)
-- @node: affineTaggedOrderedKernel_predictive_failure_gap
lemma affineTaggedOrderedKernel_predictive_failure_gap
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) (hyp : Bool) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
priorPredictive (affineRawTaggedPrior n m d eps sigma hyp)
(affineTaggedOrderedKernel n m d eps hd sigma) (retentionFailureSet n m) ≤
ofReal (gap ^ 2 / 128) := by
intro nu gap
rw [priorPredictive_apply _ _ (measurableSet_retentionFailureSet n m)]
calc
_ ≤ ∫⁻ _theta, ofReal (gap ^ 2 / 128)
∂affineRawTaggedPrior n m d eps sigma hyp := lintegral_mono fun theta =>
affineTaggedOrderedKernel_failure_gap n m d eps hd sigma hn heps heps' hS hx theta
_ = _ := by simp
theorem affineRawTaggedKernel_failure_gap
Under the stated hypotheses, The independent seed has unit mass, so it adds no count failure. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hn :
1 ≤ n
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
theta :
AffineRawParameter n m d eps
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
affineRawTaggedKernel n m d eps hd sigma theta ((retentionFailureSet (X := Obs d) (Y := AuxObs d) n m) ×ˢ univ)
≤ ofReal (gap ^ 2 / 128)
Proof (Lean source)
-- @node: affineRawTaggedKernel_failure_gap
lemma affineRawTaggedKernel_failure_gap
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(theta : AffineRawParameter n m d eps) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
affineRawTaggedKernel n m d eps hd sigma theta
((retentionFailureSet (X := Obs d) (Y := AuxObs d) n m) ×ˢ univ) ≤
ofReal (gap ^ 2 / 128) := by
intro nu gap
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
change ((affineTaggedOrderedKernel n m d eps hd sigma theta).prod seedLaw) _ ≤ _
rw [Measure.prod_prod, measure_univ, mul_one]
exact affineTaggedOrderedKernel_failure_gap n m d eps hd sigma hn heps heps' hS hx theta
A randomized original rule uses the first requested records when both prefixes exist, and returns zero on failure, keeping the same seed.
Definition (Lean source)
parameters
def
given by
fun z
=> if z.1 ∈ retentionSuccessSet n m then T (twoPoolOrderedRetention fallbackX fallbackY z.1, z.2) else 0
theorem affineTransferredRule_measurable
Under the stated inputs and conditions, Prefix transfer preserves joint Borel measurability with the seed. This gives the stated result.
Formal statement
hypotheses
conclusion
Measurable (affineTransferredRule n m d T fallbackX fallbackY)
Proof (Lean source)
-- @node: affineTransferredRule_measurable
lemma affineTransferredRule_measurable (T : Sample n m d × Real → Real)
(hT : Measurable T) (fallbackX : Fin n → Obs d) (fallbackY : Fin m → AuxObs d) :
Measurable (affineTransferredRule n m d T fallbackX fallbackY) := by
unfold affineTransferredRule
apply Measurable.ite ((measurableSet_retentionSuccessSet n m).preimage measurable_fst)
· exact hT.comp (((measurable_twoPoolOrderedRetention fallbackX fallbackY).comp
measurable_fst).prodMk measurable_snd)
· exact measurable_const
theorem affineTransferredRule_mem_Icc
Under the stated inputs and conditions, Prefix transfer preserves the clipping constraint, including on failure. This gives the stated result.
Formal statement
hypotheses
conclusion
affineTransferredRule n m d T.1 fallbackX fallbackY z ∈ Icc (-1) 1
Proof (Lean source)
-- @node: affineTransferredRule_mem_Icc
lemma affineTransferredRule_mem_Icc (T : Rule n m d)
(fallbackX : Fin n → Obs d) (fallbackY : Fin m → AuxObs d)
(z : RawTwoPool (Obs d) (AuxObs d) × Real) :
affineTransferredRule n m d T.1 fallbackX fallbackY z ∈ Icc (-1) 1 := by
unfold affineTransferredRule
split
· exact T.2.2 _
· constructor <;> norm_num
theorem affineTransferredRule_conditional_risk_le
Under the stated hypotheses, For each actual normalized table, successful ordered retention pays exactly its fixed-experiment risk; zero fallback costs at most the count-failure mass. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hn :
1 ≤ n
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
T :
Rule n m d
fallbackX :
Fin n → Obs d
theta :
AffineRawParameter n m d eps
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
target :
ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)
(∫⁻ z, ofReal ((affineTransferredRule n m d T.1 fallbackX fallbackY z - target) ^ 2) ∂affineRawTaggedKernel n m d eps hd sigma theta)
≤ (∫⁻ z, ofReal ((T.1 z - target) ^ 2) ∂((affineTaggedFixedKernel n m d eps hd sigma theta).prod seedLaw))
+ ofReal (gap ^ 2 / 128)
Proof (Lean source)
-- @node: affineTransferredRule_conditional_risk_le
lemma affineTransferredRule_conditional_risk_le
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(T : Rule n m d) (fallbackX : Fin n → Obs d) (fallbackY : Fin m → AuxObs d)
(theta : AffineRawParameter n m d eps) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
let target := ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)
(∫⁻ z, ofReal ((affineTransferredRule n m d T.1 fallbackX fallbackY z - target) ^ 2)
∂affineRawTaggedKernel n m d eps hd sigma theta) ≤
(∫⁻ z, ofReal ((T.1 z - target) ^ 2)
∂((affineTaggedFixedKernel n m d eps hd sigma theta).prod seedLaw)) +
ofReal (gap ^ 2 / 128) := by
intro nu gap target
let E : Set (RawTwoPool (Obs d) (AuxObs d) × Real) :=
retentionSuccessSet n m ×ˢ Set.univ
have hE : MeasurableSet E := (measurableSet_retentionSuccessSet n m).prod MeasurableSet.univ
let mu := affineRawTaggedKernel n m d eps hd sigma theta
have hsuccess : (∫⁻ z in E,
ofReal ((affineTransferredRule n m d T.1 fallbackX fallbackY z - target) ^ 2) ∂mu) ≤
∫⁻ z, ofReal ((T.1 z - target) ^ 2)
∂((affineTaggedFixedKernel n m d eps hd sigma theta).prod seedLaw) := by
have heq : (∫⁻ z in E,
ofReal ((affineTransferredRule n m d T.1 fallbackX fallbackY z - target) ^ 2) ∂mu) =
∫⁻ z in E, ofReal ((T.1
(map (twoPoolOrderedRetention fallbackX fallbackY) id z) - target) ^ 2) ∂mu := by
apply setLIntegral_congr_fun hE
intro z hz
simp only [affineTransferredRule, if_pos hz.1]
rfl
have hT := T.2.1
rw [heq, ← lintegral_map (f := fun z : Sample n m d × Real =>
ofReal ((T.1 z - target) ^ 2)) (by fun_prop)
((measurable_twoPoolOrderedRetention fallbackX fallbackY).prodMap measurable_id)]
change (∫⁻ z, ofReal ((T.1 z - target) ^ 2)
∂((affineRawTaggedKernel n m d eps hd sigma theta).restrict
(retentionSuccessSet n m ×ˢ Set.univ)).map _) ≤ _
rw [affineRawTaggedKernel_retention_success, lintegral_smul_measure, smul_eq_mul]
apply mul_le_of_le_one_left (by positivity)
exact (mul_le_mul prob_le_one prob_le_one (by positivity) (by positivity)).trans_eq (by simp)
have hfailure : (∫⁻ z in Eᶜ,
ofReal ((affineTransferredRule n m d T.1 fallbackX fallbackY z - target) ^ 2) ∂mu) ≤
ofReal (gap ^ 2 / 128) := by
have hb := affine_normalized_target_bounds n m d eps hn hd heps heps' sigma theta.1 theta.2
have htarget : target ^ 2 ≤ 1 := by nlinarith [hb.1, hb.2]
have heq : Eᶜ = retentionFailureSet (X := Obs d) (Y := AuxObs d) n m ×ˢ Set.univ := by
ext z
simp only [E, Set.mem_compl_iff, Set.mem_prod, Set.mem_univ, and_true,
retentionSuccessSet, retentionFailureSet, Set.mem_setOf_eq, Set.mem_union]
omega
calc
_ ≤ ∫⁻ _z in Eᶜ, (1 : ENNReal) ∂mu := by
apply setLIntegral_mono measurable_const
intro z hz
have hz' : z.1 ∉ retentionSuccessSet (X := Obs d) (Y := AuxObs d) n m := by
simpa [E] using hz
simp only [affineTransferredRule, if_neg hz', zero_sub, neg_sq]
exact (ENNReal.ofReal_le_ofReal htarget).trans_eq ENNReal.ofReal_one
_ = mu Eᶜ := by simp
_ ≤ _ := by
rw [heq]
exact affineRawTaggedKernel_failure_gap n m d eps hd sigma hn heps heps' hS hx theta
rw [← lintegral_add_compl _ hE]
exact add_le_add hsuccess hfailure
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.FixedRisk 9 declarations Real and extended squared-risk readback for clipped original-experiment rules, and the affine prior's fixed-size minimax assembly.
Real and extended squared-risk readback for clipped original-experiment rules, and the affine prior's fixed-size minimax assembly.
theorem affine_rule_loss_bound
Under the stated inputs and conditions, Clipping the action and target bounds squared loss by four. This gives the stated result.
Formal statement
hypotheses
conclusion
(T.1 z - ateFunctional P) ^ 2 ≤ 4
Proof (Lean source)
-- @node: affine_rule_loss_bound
lemma affine_rule_loss_bound {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d)
(z : Sample n m d × Real) : (T.1 z - ateFunctional P) ^ 2 ≤ 4 := by
have hT := T.2.2 z
have hP := ateFunctional_mem_Icc P
have hlo : -2 ≤ T.1 z - ateFunctional P := by linarith [hT.1, hP.2]
have hhi : T.1 z - ateFunctional P ≤ 2 := by linarith [hT.2, hP.1]
nlinarith [mul_nonneg (by linarith : 0 ≤ T.1 z - ateFunctional P + 2)
(by linarith : 0 ≤ 2 - (T.1 z - ateFunctional P))]
theorem affine_rule_loss_integrable
Under the stated inputs and conditions, The bounded measurable squared loss is integrable under the original experiment. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affine_rule_loss_integrable
lemma affine_rule_loss_integrable {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d) :
Integrable (fun z => (T.1 z - ateFunctional P) ^ 2)
((annotationLaw P n m).prod seedLaw) := by
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw obsLaw
infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have hT := T.2.1
apply Integrable.of_bound (by fun_prop) 4
filter_upwards [] with z
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
exact affine_rule_loss_bound T P z
theorem affine_ruleRisk_ofReal
Under the stated inputs and conditions, The original real risk agrees with the extended nonnegative loss integral. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affine_ruleRisk_ofReal
lemma affine_ruleRisk_ofReal {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d) :
ofReal (ruleRisk T.1 P) =
∫⁻ z, ofReal ((T.1 z - ateFunctional P) ^ 2)
∂((annotationLaw P n m).prod seedLaw) := by
exact ofReal_integral_eq_lintegral_ofReal (affine_rule_loss_integrable T P)
(Filter.Eventually.of_forall fun _ => sq_nonneg _)
theorem affine_ruleRisk_bounds
Under the stated inputs and conditions, Every clipped original rule has nonnegative risk bounded by four. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_ruleRisk_bounds
lemma affine_ruleRisk_bounds {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d) :
0 ≤ ruleRisk T.1 P ∧ ruleRisk T.1 P ≤ 4 := by
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw obsLaw
infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
constructor
· exact integral_nonneg fun _ => sq_nonneg _
· calc
_ ≤ ∫ _z, (4 : Real) ∂((annotationLaw P n m).prod seedLaw) :=
integral_mono (affine_rule_loss_integrable T P) (integrable_const _)
(affine_rule_loss_bound T P)
_ = 4 := by simp
theorem affine_ruleRisk_le_worstRisk
Under the stated inputs and conditions, A legal table's risk is dominated by the bounded real worst-case supremum. This gives the stated result.
Formal statement
hypotheses
conclusion
ruleRisk T.1 P ≤ worstRisk T.1 eps
Proof (Lean source)
-- @node: affine_ruleRisk_le_worstRisk
lemma affine_ruleRisk_le_worstRisk {n m d : Nat} (T : Rule n m d)
(eps : Real) (P : DiscreteLaw d) (hP : ModelClass d eps P) :
ruleRisk T.1 P ≤ worstRisk T.1 eps := by
apply le_ciSup (show BddAbove (range (fun Q : ClassLaw d eps => ruleRisk T.1 Q.1)) from
⟨4, by rintro _ ⟨Q, rfl⟩; exact (affine_ruleRisk_bounds T Q.1).2⟩) ⟨P, hP⟩
theorem affineTransferredRule_bayes_le
Under the stated inputs and conditions, Averaging the conditional transfer under either tagged prior preserves the worst-case bound and the half-gap failure budget. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
sigma :
T :
Rule n m d
fallbackX :
Fin n → Obs d
hyp :
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma hyp) (affineRawTaggedKernel n m d eps hd sigma) (fun theta => ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)) (affineTransferredRule n m d T.1 fallbackX fallbackY)
Proof (Lean source)
-- @node: affineTransferredRule_bayes_le
lemma affineTransferredRule_bayes_le (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(T : Rule n m d) (fallbackX : Fin n → Obs d) (fallbackY : Fin m → AuxObs d)
(hyp : Bool) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma hyp)
(affineRawTaggedKernel n m d eps hd sigma)
(fun theta => ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2))
(affineTransferredRule n m d T.1 fallbackX fallbackY) ≤
ofReal (worstRisk T.1 eps) + ofReal (gap ^ 2 / 128) := by
intro nu gap
unfold bayesSquaredRisk
calc
_ ≤ ∫⁻ _theta, ofReal (worstRisk T.1 eps) + ofReal (gap ^ 2 / 128)
∂affineRawTaggedPrior n m d eps sigma hyp := by
apply lintegral_mono
intro theta
have ht := affineTransferredRule_conditional_risk_le n m d eps hd sigma
hn heps heps' hS hx T fallbackX fallbackY theta
apply ht.trans
apply add_le_add ?_ (le_refl _)
change (∫⁻ z, ofReal ((T.1 z -
ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)) ^ 2)
∂((annotationLaw (normalizedLaw theta.1 n m d eps hd sigma theta.2) n m).prod seedLaw)) ≤ _
rw [← affine_ruleRisk_ofReal T]
apply ENNReal.ofReal_le_ofReal
exact affine_ruleRisk_le_worstRisk T eps _
(((affine_support_and_normalizer n m d eps hn hd heps heps' sigma).2 theta.2).2.1 theta.1)
_ = _ := by simp
theorem affine_fixed_rule_gap_lower
Under the stated inputs and conditions, The genuine raw Bayes bound and prefix transfer force each original clipped Borel rule to pay at least one one-hundred-twenty-eighth of the squared gap. This gives the stated result.
Formal statement
hypotheses
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
gap ^ 2 / 128 ≤ worstRisk T.1 eps
Proof (Lean source)
-- @node: affine_fixed_rule_gap_lower
lemma affine_fixed_rule_gap_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(T : Rule n m d) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
gap ^ 2 / 128 ≤ worstRisk T.1 eps := by
intro nu gap
have hS1 : 1 ≤ (n : Real) * eps := by linarith [Real.add_one_le_exp (4096 : Real)]
let fallbackX : Fin n → Obs d := fun _ => (⟨0, by omega⟩, false, false)
let fallbackY : Fin m → AuxObs d := fun _ => (⟨0, by omega⟩, false)
have hlower := affineRawTaggedKernel_bayes_lower n m d eps hd sigma hn heps heps' hS hx
(affineTransferredRule n m d T.1 fallbackX fallbackY)
(affineTransferredRule_measurable n m d T.1 T.2.1 fallbackX fallbackY)
have hupper (hyp : Bool) := affineTransferredRule_bayes_le n m d eps hn hd heps heps'
hS1 hx sigma T fallbackX fallbackY hyp
have hraw : ofReal (gap ^ 2 / 64) ≤
ofReal (worstRisk T.1 eps) + ofReal (gap ^ 2 / 128) :=
hlower.trans (max_le (hupper false) (hupper true))
have hW : 0 ≤ worstRisk T.1 eps := by
let lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L :=
fun _ => none
exact (affine_ruleRisk_bounds T (normalizedLaw false n m d eps hd sigma lat)).1.trans
(affine_ruleRisk_le_worstRisk T eps _
(((affine_support_and_normalizer n m d eps hn hd heps heps' sigma).2 lat).2.1 false))
rw [← ENNReal.ofReal_add hW (by positivity)] at hraw
have hr := (ENNReal.ofReal_le_ofReal_iff (by positivity : 0 ≤
worstRisk T.1 eps + gap ^ 2 / 128)).mp hraw
linarith
theorem affine_fixed_rule_rate_lower
Under the stated inputs and conditions, Squaring the quantitative target separation gives the displayed affine rate, with the same numerical constant as the paper. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_fixed_rule_rate_lower
lemma affine_fixed_rule_rate_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(T : Rule n m d) :
(10 ^ (-20 : Int) / 128 : Real) *
min 1 (((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) ≤
worstRisk T.1 eps := by
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
let x := (d : Real) / (((n : Real) + m) * eps * logScale n eps)
have hS1 : 1 ≤ (n : Real) * eps := by linarith [Real.add_one_le_exp (4096 : Real)]
have hell := affine_logScale_one_le n eps heps.le
have hx0 : 0 ≤ x := by dsimp [x]; positivity
have hclip : 0 ≤ min x 1 := le_min hx0 (by norm_num)
have hgap : min x 1 / 10000000000 ≤ gap :=
affine_raw_target_mean_gap_uniform n m d eps hn hd heps heps' hS1 sigma
have hsq := pow_le_pow_left₀ (div_nonneg hclip (by norm_num)) hgap 2
have hmin : (min x 1) ^ 2 = min 1 (x ^ 2) := by
by_cases hx1 : x ≤ 1
· rw [min_eq_left hx1, min_eq_right (by nlinarith)]
· rw [min_eq_right (le_of_not_ge hx1), min_eq_left (by nlinarith), one_pow]
have hfixed : gap ^ 2 / 128 ≤ worstRisk T.1 eps :=
affine_fixed_rule_gap_lower n m d eps hn hd heps heps' hS hx sigma T
have hscaled : (10 ^ (-20 : Int) / 128 : Real) * min 1 (x ^ 2) ≤ gap ^ 2 / 128 := by
rw [div_pow, hmin] at hsq
norm_num at hsq ⊢
linarith
exact hscaled.trans hfixed
theorem affine_fixed_minimax_rate_lower
Under the stated inputs and conditions, Taking the infimum over all original randomized Borel rules preserves the fixed-size affine lower bound. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
Proof (Lean source)
-- @node: affine_fixed_minimax_rate_lower
lemma affine_fixed_minimax_rate_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
(10 ^ (-20 : Int) / 128 : Real) *
min 1 (((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) ≤
minimaxRisk n m d eps := by
haveI : Nonempty (Rule n m d) := ⟨⟨fun _ => 0, measurable_const,
fun _ => by constructor <;> norm_num⟩⟩
apply le_ciInf
intro T
exact affine_fixed_rule_rate_lower n m d eps hn hd heps heps' hS hx sigma T
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Histogram 21 declarations Embedding the five rare-cell count coordinates and the four common reservoir counts into both original atom histograms, followed by uniform channel ordering.
Embedding the five rare-cell count coordinates and the four common reservoir counts into both original atom histograms, followed by uniform channel ordering.
Place rare-cell counts at their original labels, reservoir counts at label k, and zeros at unused labels. Complete control outcomes are always zero.
Definition (Lean source)
parameters
given by
(fun z => if hj : z.1.val < k then let c := counts.1 ⟨z.1.val, hj⟩ if z.2.1 then (if z.2.2 then c.1.1 else c.1.2) else (if z.2.2 then 0 else c.2.2.1) else if z.1.val = k then if z.2.2 then 0 else if z.2.1 then counts.2.1.1 else counts.2.1.2 else 0, fun z => if hj : z.1.val < k then let c := counts.1 ⟨z.1.val, hj⟩ if z.2 then c.2.1 else c.2.2.2 else if z.1.val = k then if z.2 then counts.2.2.1 else counts.2.2.2 else 0)
theorem affineFullCountHistograms_measurable
Both histogram channels are measurable functions of the finite count coordinates.
Formal statement
Proof (Lean source)
-- @node: affineFullCountHistograms_measurable
@[fun_prop]
lemma affineFullCountHistograms_measurable (d k : Nat) :
Measurable (affineFullCountHistograms d k) := by
exact measurable_of_countable _
theorem affineFullCountHistograms_rare
Under the stated inputs and conditions, The five rare-cell coordinates occupy exactly their original complete and auxiliary atoms, with no positive complete control outcomes. This gives the stated result.
Formal statement
hypotheses
conclusion
c :
counts.1 ⟨j.val, hj⟩
(affineFullCountHistograms d k counts).2 (j, true) = c.2.1
(affineFullCountHistograms d k counts).2 (j, false) = c.2.2.2
Proof (Lean source)
-- @node: affineFullCountHistograms_rare
lemma affineFullCountHistograms_rare (d k : Nat)
(counts : (Fin k → ((Nat × Nat) × (Nat × (Nat × Nat)))) ×
((Nat × Nat) × (Nat × Nat))) (j : Fin d) (hj : j.val < k) :
let c := counts.1 ⟨j.val, hj⟩
(affineFullCountHistograms d k counts).1 (j, true, true) = c.1.1 ∧
(affineFullCountHistograms d k counts).1 (j, true, false) = c.1.2 ∧
(affineFullCountHistograms d k counts).1 (j, false, false) = c.2.2.1 ∧
(affineFullCountHistograms d k counts).1 (j, false, true) = 0 ∧
(affineFullCountHistograms d k counts).2 (j, true) = c.2.1 ∧
(affineFullCountHistograms d k counts).2 (j, false) = c.2.2.2 := by
simp [affineFullCountHistograms, hj]
theorem affineFullCountHistograms_reservoir
Under the stated inputs and conditions, All four reservoir coordinates are retained, and its complete outcomes are zero. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
conclusion 2
conclusion 3
conclusion 4
conclusion 5
(affineFullCountHistograms d k counts).2 (j, true) = counts.2.2.1
conclusion 6
(affineFullCountHistograms d k counts).2 (j, false) = counts.2.2.2
Proof (Lean source)
-- @node: affineFullCountHistograms_reservoir
lemma affineFullCountHistograms_reservoir (d k : Nat)
(counts : (Fin k → ((Nat × Nat) × (Nat × (Nat × Nat)))) ×
((Nat × Nat) × (Nat × Nat))) (j : Fin d) (hj : j.val = k) :
(affineFullCountHistograms d k counts).1 (j, true, false) = counts.2.1.1 ∧
(affineFullCountHistograms d k counts).1 (j, false, false) = counts.2.1.2 ∧
(affineFullCountHistograms d k counts).1 (j, true, true) = 0 ∧
(affineFullCountHistograms d k counts).1 (j, false, true) = 0 ∧
(affineFullCountHistograms d k counts).2 (j, true) = counts.2.2.1 ∧
(affineFullCountHistograms d k counts).2 (j, false) = counts.2.2.2 := by
simp [affineFullCountHistograms, hj]
theorem affineFullCountHistograms_null
Under the stated inputs and conditions, Labels beyond the reservoir contribute no counts in either channel. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineFullCountHistograms_null
lemma affineFullCountHistograms_null (d k : Nat)
(counts : (Fin k → ((Nat × Nat) × (Nat × (Nat × Nat)))) ×
((Nat × Nat) × (Nat × Nat))) (j : Fin d) (hj : k < j.val) (a y : Bool) :
(affineFullCountHistograms d k counts).1 (j, a, y) = 0 ∧
(affineFullCountHistograms d k counts).2 (j, a) = 0 := by
simp [affineFullCountHistograms, not_lt.mpr hj.le, ne_of_gt hj]
The reservoir has independent zero-outcome complete and auxiliary counts in both arms, each with half the common reservoir mass.
Definition (Lean source)
parameters
given by
t :
affineTuning n m d eps
pstar :
1 - (t.Kstar : Real) * latentMeanMass sigma
complete :
poissonMeasure (toNNReal (t.u * (pstar / 2)))
auxiliary :
poissonMeasure (toNNReal (t.w * (pstar / 2)))
(complete.prod complete).prod (auxiliary.prod auxiliary)
instance affineReservoirCountLaw_isProbabilityMeasure
The common reservoir count law has total mass one.
Definition (Lean source)
-- @node: affineReservoirCountLaw_isProbabilityMeasure
instance affineReservoirCountLaw_isProbabilityMeasure (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
IsProbabilityMeasure (affineReservoirCountLaw n m d eps sigma) := by
unfold affineReservoirCountLaw
infer_instance
Embed the independent full rare-cell mixtures and the construction's common reservoir into the two complete atom histograms.
Definition (Lean source)
parameters
n m d :
eps :
sigma :
hyp :
given by
t :
affineTuning n m d eps
Kernel.deterministic (affineFullCountHistograms d t.Kstar) (affineFullCountHistograms_measurable d t.Kstar) ∘ₘ ((Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w hyp).prod (affineReservoirCountLaw n m d eps sigma))
instance affineHistogramPredictive_isProbabilityMeasure
The embedded histogram mixture is a probability law in both channels.
Definition (Lean source)
-- @node: affineHistogramPredictive_isProbabilityMeasure
instance affineHistogramPredictive_isProbabilityMeasure (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) : IsProbabilityMeasure (affineHistogramPredictive n m d eps sigma hyp) := by
unfold affineHistogramPredictive
infer_instance
theorem affineHistogramPredictive_tv_small
Under the stated inputs and conditions, The embedded paired histogram experiment satisfies the full joint TV budget. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
tvDist (affineHistogramPredictive n m d eps sigma true) (affineHistogramPredictive n m d eps sigma false) < 1 / 64
Proof (Lean source)
-- @node: affineHistogramPredictive_tv_small
lemma affineHistogramPredictive_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
tvDist (affineHistogramPredictive n m d eps sigma true)
(affineHistogramPredictive n m d eps sigma false) < 1 / 64 := by
let t := affineTuning n m d eps
have hprod := tvDist_prod_le_add
(Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w true)
(Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w false)
(affineReservoirCountLaw n m d eps sigma) (affineReservoirCountLaw n m d eps sigma)
have hself : tvDist (affineReservoirCountLaw n m d eps sigma)
(affineReservoirCountLaw n m d eps sigma) = 0 := by
simp [tvDist]
rw [hself, add_zero] at hprod
exact (tvDist_bind_le _ _
(Kernel.deterministic (affineFullCountHistograms d t.Kstar)
(affineFullCountHistograms_measurable d t.Kstar))).trans_lt
(hprod.trans_lt
(affineCellFullPoissonPredictive_product_tv_small n m d eps hn hd heps heps' sigma))
theorem affineHistogramPredictive_ordered_tv_small
Under the stated inputs and conditions, Uniform ordering independently restores both channels without increasing the distance of the embedded full count mixtures. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
tvDist (pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₘ affineHistogramPredictive n m d eps sigma true) (pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₘ affineHistogramPredictive n m d eps sigma false) < 1 / 64
Proof (Lean source)
-- @node: affineHistogramPredictive_ordered_tv_small
lemma affineHistogramPredictive_ordered_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
tvDist
(pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₘ
affineHistogramPredictive n m d eps sigma true)
(pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₘ
affineHistogramPredictive n m d eps sigma false) < 1 / 64 := by
exact (tvDist_bind_le _ _
(pairedHistogramReconstructionKernel (Obs d) (AuxObs d))).trans_lt
(affineHistogramPredictive_tv_small n m d eps hn hd heps heps' sigma)
The conditional product of the five full cell coordinates over the actual latent vector, before adding the common reservoir.
Definition (Lean source)
parameters
def
given by
Kernel.ofFunOfCountable fun lat
=> Measure.pi fun i : Fin k => affineCellFullPoissonLaw sigma u w hyp (lat i)
instance affineFullCellProductKernel_isMarkovKernel
Each full conditional product is a probability kernel.
Definition (Lean source)
-- @node: affineFullCellProductKernel_isMarkovKernel
instance affineFullCellProductKernel_isMarkovKernel {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (k : Nat) (u w : Real) (hyp : Bool) :
IsMarkovKernel (affineFullCellProductKernel sigma k u w hyp) where
isProbabilityMeasure lat := by
change IsProbabilityMeasure
(Measure.pi fun i : Fin k => affineCellFullPoissonLaw sigma u w hyp (lat i))
infer_instance
theorem affineFullCellProductKernel_predictive_eq
Under the stated inputs and conditions, Mixing the full conditional cell products over independent core/filler latent draws gives exactly the product used by the histogram experiment. This gives the stated result.
Formal statement
hypotheses
conclusion
priorPredictive (Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure) (affineFullCellProductKernel sigma k u w hyp)
= Measure.pi fun _ : Fin k => affineCellFullPoissonPredictive sigma u w hyp
Proof (Lean source)
-- @node: affineFullCellProductKernel_predictive_eq
lemma affineFullCellProductKernel_predictive_eq {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (k : Nat) (u w : Real) (hyp : Bool) :
priorPredictive (Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure)
(affineFullCellProductKernel sigma k u w hyp) =
Measure.pi fun _ : Fin k => affineCellFullPoissonPredictive sigma u w hyp := by
classical
apply Measure.ext_of_singleton
intro o
rw [priorPredictive_apply _ _ (measurableSet_singleton o), lintegral_fintype]
change (∑ lat : Fin k → Latent L,
(Measure.pi fun i : Fin k => affineCellFullPoissonLaw sigma u w hyp (lat i)) {o} *
(Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure) {lat}) = _
simp only [Measure.pi_singleton,
PMF.toMeasure_apply_singleton _ _ (measurableSet_singleton _)]
simp only [affineCellFullPoissonPredictive, Measure.finsetSum_apply, Measure.smul_apply,
smul_eq_mul]
rw [Fintype.prod_sum]
apply Finset.sum_congr rfl
intro lat _
rw [← Finset.prod_mul_distrib]
apply Finset.prod_congr rfl
intro i _
exact mul_comm _ _
Attach the actual common reservoir to the full conditional cell experiment.
Definition (Lean source)
parameters
n m d :
eps :
sigma :
hyp :
def
given by
t :
affineTuning n m d eps
(affineFullCellProductKernel sigma t.Kstar t.u t.w hyp).prod (Kernel.const _ (affineReservoirCountLaw n m d eps sigma))
instance affineFullCountKernel_isMarkovKernel
The conditional full count law is a probability kernel.
Definition (Lean source)
-- @node: affineFullCountKernel_isMarkovKernel
instance affineFullCountKernel_isMarkovKernel (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) : IsMarkovKernel (affineFullCountKernel n m d eps sigma hyp) := by
unfold affineFullCountKernel
infer_instance
theorem affineFullCountKernel_predictive_eq
Under the stated inputs and conditions, The independent reservoir factors out of the actual full latent mixture. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
conclusion
t :
affineTuning n m d eps
priorPredictive (Measure.pi fun _ : Fin t.Kstar => (latentPMF sigma).toMeasure) (affineFullCountKernel n m d eps sigma hyp)
= (Measure.pi fun _ : Fin t.Kstar => affineCellFullPoissonPredictive sigma t.u t.w hyp).prod (affineReservoirCountLaw n m d eps sigma)
Proof (Lean source)
-- @node: affineFullCountKernel_predictive_eq
lemma affineFullCountKernel_predictive_eq (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) :
let t := affineTuning n m d eps
priorPredictive (Measure.pi fun _ : Fin t.Kstar => (latentPMF sigma).toMeasure)
(affineFullCountKernel n m d eps sigma hyp) =
(Measure.pi fun _ : Fin t.Kstar =>
affineCellFullPoissonPredictive sigma t.u t.w hyp).prod
(affineReservoirCountLaw n m d eps sigma) := by
intro t
refine (Measure.prod_eq ?_).symm
intro s r hs hr
rw [priorPredictive_apply _ _ (hs.prod hr)]
simp only [affineFullCountKernel, Kernel.prod_apply_prod, Kernel.const_apply]
change (∫⁻ lat, (affineFullCellProductKernel sigma t.Kstar t.u t.w hyp lat) s *
(affineReservoirCountLaw n m d eps sigma) r
∂Measure.pi fun _ : Fin t.Kstar => (latentPMF sigma).toMeasure) = _
rw [lintegral_mul_const _ ((affineFullCellProductKernel sigma t.Kstar t.u t.w hyp).measurable_coe hs),
← priorPredictive_apply _ _ hs, affineFullCellProductKernel_predictive_eq]
The concrete histogram embedding of the actual conditional full count law.
Definition (Lean source)
parameters
n m d :
eps :
sigma :
hyp :
def
affineHistogramKernel n m d eps sigma hyp :
Kernel (Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) ((Obs d → Nat) × (AuxObs d → Nat))
given by
Kernel.deterministic (affineFullCountHistograms d (affineTuning n m d eps).Kstar) (affineFullCountHistograms_measurable d (affineTuning n m d eps).Kstar) ∘ₖ affineFullCountKernel n m d eps sigma hyp
instance affineHistogramKernel_isMarkovKernel
The conditional histogram embedding is a probability kernel.
Definition (Lean source)
-- @node: affineHistogramKernel_isMarkovKernel
instance affineHistogramKernel_isMarkovKernel (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) : IsMarkovKernel (affineHistogramKernel n m d eps sigma hyp) := by
unfold affineHistogramKernel
infer_instance
theorem affineHistogramKernel_predictive_eq
Under the stated inputs and conditions, The histogram predictive law is the mixture of the actual conditional core/filler count embeddings, rather than an assumed identification. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
conclusion
priorPredictive (Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure) (affineHistogramKernel n m d eps sigma hyp)
= affineHistogramPredictive n m d eps sigma hyp
Proof (Lean source)
-- @node: affineHistogramKernel_predictive_eq
lemma affineHistogramKernel_predictive_eq (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) :
priorPredictive
(Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
(affineHistogramKernel n m d eps sigma hyp) =
affineHistogramPredictive n m d eps sigma hyp := by
rw [affineHistogramKernel, kernel_comp_priorPredictive, affineFullCountKernel_predictive_eq]
rfl
theorem affineHistogramKernel_ordered_predictive_tv_small
Under the stated inputs and conditions, The ordered, mixed conditional histogram experiment obeys the joint TV budget in the orientation used by the two-prior testing inequality. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
nu :
order :
pairedHistogramReconstructionKernel (Obs d) (AuxObs d)
tvDist (priorPredictive nu (order ∘ₖ affineHistogramKernel n m d eps sigma false)) (priorPredictive nu (order ∘ₖ affineHistogramKernel n m d eps sigma true)) < 1 / 64
Proof (Lean source)
-- @node: affineHistogramKernel_ordered_predictive_tv_small
lemma affineHistogramKernel_ordered_predictive_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let order := pairedHistogramReconstructionKernel (Obs d) (AuxObs d)
tvDist
(priorPredictive nu (order ∘ₖ affineHistogramKernel n m d eps sigma false))
(priorPredictive nu (order ∘ₖ affineHistogramKernel n m d eps sigma true)) < 1 / 64 := by
intro nu order
rw [kernel_comp_priorPredictive, kernel_comp_priorPredictive,
affineHistogramKernel_predictive_eq, affineHistogramKernel_predictive_eq,
tvDist_symm]
exact affineHistogramPredictive_ordered_tv_small n m d eps hn hd heps heps' sigma
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.HistogramCoordinates 14 declarations The full histogram embedding is injective and its image consists precisely of histograms with zero counts at the structural zero atoms.
The full histogram embedding is injective and its image consists precisely of histograms with zero counts at the structural zero atoms. Singleton probabilities therefore reduce to the unique original rare-cell and reservoir count vector.
Read every rare-cell and reservoir coordinate from the original histograms.
Definition (Lean source)
parameters
def
theorem affineHistogramCoordinates_leftInverse
Under the stated inputs and conditions, The full embedding loses none of the nine types of stochastic coordinates. This gives the stated result.
Formal statement
hypotheses
d k :
hk :
k < d
conclusion
Proof (Lean source)
-- @node: affineHistogramCoordinates_leftInverse
lemma affineHistogramCoordinates_leftInverse (d k : Nat) (hk : k < d) :
LeftInverse (affineHistogramCoordinates d k hk)
(affineFullCountHistograms d k) := by
intro counts
apply Prod.ext
· funext i
simp [affineHistogramCoordinates, affineFullCountHistograms]
· simp [affineHistogramCoordinates, affineFullCountHistograms]
theorem affineFullCountHistograms_injective
Under the stated inputs and conditions, A full count vector is uniquely determined by its original atom histograms. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineFullCountHistograms_injective
lemma affineFullCountHistograms_injective (d k : Nat) (hk : k < d) :
Injective (affineFullCountHistograms d k) :=
(affineHistogramCoordinates_leftInverse d k hk).injective
Structural zero coordinates of the two-channel raw affine experiment.
Definition (Lean source)
theorem affineFullCountHistograms_supported
Under the stated inputs and conditions, Every embedded count vector vanishes at exactly the structural zero coordinates. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affineFullCountHistograms_supported
lemma affineFullCountHistograms_supported (d k : Nat)
(counts : (Fin k → ((Nat × Nat) × (Nat × (Nat × Nat)))) ×
((Nat × Nat) × (Nat × Nat))) :
affineHistogramSupported d k (affineFullCountHistograms d k counts) := by
refine ⟨?_, ?_, ?_⟩
· intro j
simp [affineFullCountHistograms]
· intro j hj
simp [affineFullCountHistograms, not_lt.mpr hj]
· intro j hj
exact ⟨fun a y => (affineFullCountHistograms_null d k counts j hj a y).1,
fun a => (affineFullCountHistograms_null d k counts j hj a false).2⟩
theorem affineHistogramCoordinates_rightInverse
Under the stated inputs and conditions, Reading and re-embedding any supported histogram returns that histogram. This gives the stated result.
Formal statement
hypotheses
conclusion
affineFullCountHistograms d k (affineHistogramCoordinates d k hk h) = h
Proof (Lean source)
-- @node: affineHistogramCoordinates_rightInverse
lemma affineHistogramCoordinates_rightInverse (d k : Nat) (hk : k < d)
(h : (Obs d → Nat) × (AuxObs d → Nat)) (hs : affineHistogramSupported d k h) :
affineFullCountHistograms d k (affineHistogramCoordinates d k hk h) = h := by
rcases hs with ⟨hcontrol, htreated, hnull⟩
apply Prod.ext
· funext z
rcases z with ⟨j, a, y⟩
by_cases hj : j.val < k
· cases a <;> cases y <;>
simp [affineFullCountHistograms, affineHistogramCoordinates, hj, hcontrol]
· by_cases heq : j.val = k
· have hjEq : j = ⟨k, hk⟩ := Fin.ext heq
subst j
cases a <;> cases y <;>
simp [affineFullCountHistograms, affineHistogramCoordinates, hcontrol,
htreated ⟨k, hk⟩ (le_refl k)]
· have hgt : k < j.val := lt_of_le_of_ne (not_lt.mp hj) (Ne.symm heq)
simp [affineFullCountHistograms, hj, heq, (hnull j hgt).1 a y]
· funext z
rcases z with ⟨j, a⟩
by_cases hj : j.val < k
· cases a <;> simp [affineFullCountHistograms, affineHistogramCoordinates, hj]
· by_cases heq : j.val = k
· have hjEq : j = ⟨k, hk⟩ := Fin.ext heq
subst j
cases a <;> simp [affineFullCountHistograms, affineHistogramCoordinates]
· have hgt : k < j.val := lt_of_le_of_ne (not_lt.mp hj) (Ne.symm heq)
simp [affineFullCountHistograms, hj, heq, (hnull j hgt).2 a]
theorem affineFullCountHistograms_range
Under the stated inputs and conditions, The structural zero conditions exactly characterize the histogram image. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affineFullCountHistograms_range
lemma affineFullCountHistograms_range (d k : Nat) (hk : k < d)
(h : (Obs d → Nat) × (AuxObs d → Nat)) :
h ∈ range (affineFullCountHistograms d k) ↔ affineHistogramSupported d k h := by
constructor
· rintro ⟨counts, rfl⟩
exact affineFullCountHistograms_supported d k counts
· intro hs
exact ⟨affineHistogramCoordinates d k hk h,
affineHistogramCoordinates_rightInverse d k hk h hs⟩
theorem affineFullCountHistograms_preimage_singleton
Under the stated inputs and conditions, A supported histogram has precisely one preimage, the explicit coordinate readback. This gives the stated result.
Formal statement
hypotheses
conclusion
affineFullCountHistograms d k ⁻¹' {h} = {affineHistogramCoordinates d k hk h}
Proof (Lean source)
-- @node: affineFullCountHistograms_preimage_singleton
lemma affineFullCountHistograms_preimage_singleton (d k : Nat) (hk : k < d)
(h : (Obs d → Nat) × (AuxObs d → Nat)) (hs : affineHistogramSupported d k h) :
affineFullCountHistograms d k ⁻¹' {h} = {affineHistogramCoordinates d k hk h} := by
ext counts
simp only [Set.mem_preimage, Set.mem_singleton_iff]
constructor
· intro hc
apply affineFullCountHistograms_injective d k hk
exact hc.trans (affineHistogramCoordinates_rightInverse d k hk h hs).symm
· rintro rfl
exact affineHistogramCoordinates_rightInverse d k hk h hs
theorem affineFullCountHistograms_preimage_singleton_empty
Under the stated inputs and conditions, Unsupported histograms have no preimage under the full embedding. This gives the stated result.
Formal statement
hypotheses
conclusion
affineFullCountHistograms d k ⁻¹' {h} = ∅
Proof (Lean source)
-- @node: affineFullCountHistograms_preimage_singleton_empty
lemma affineFullCountHistograms_preimage_singleton_empty (d k : Nat)
(h : (Obs d → Nat) × (AuxObs d → Nat)) (hs : ¬ affineHistogramSupported d k h) :
affineFullCountHistograms d k ⁻¹' {h} = ∅ := by
apply Set.eq_empty_iff_forall_notMem.mpr
intro counts hc
apply hs
have heq : affineFullCountHistograms d k counts = h := hc
rw [← heq]
exact affineFullCountHistograms_supported d k counts
theorem affineFullCountHistograms_map_singleton
Under the stated inputs and conditions, Singleton masses after embedding are explicit coordinate masses on the image, and zero off the image. This applies to conditional laws and to their mixtures. This gives the stated result.
Formal statement
hypotheses
d k :
hk :
k < d
conclusion
mu.map (affineFullCountHistograms d k) {h}
= if affineHistogramSupported d k h then mu {affineHistogramCoordinates d k hk h} else 0
Proof (Lean source)
-- @node: affineFullCountHistograms_map_singleton
lemma affineFullCountHistograms_map_singleton (d k : Nat) (hk : k < d)
(mu : Measure ((Fin k → ((Nat × Nat) × (Nat × (Nat × Nat)))) ×
((Nat × Nat) × (Nat × Nat))))
(h : (Obs d → Nat) × (AuxObs d → Nat)) :
mu.map (affineFullCountHistograms d k) {h} =
if affineHistogramSupported d k h then mu {affineHistogramCoordinates d k hk h}
else 0 := by
classical
rw [Measure.map_apply (affineFullCountHistograms_measurable d k)
(measurableSet_singleton h)]
split_ifs with hs
· rw [affineFullCountHistograms_preimage_singleton d k hk h hs]
· rw [affineFullCountHistograms_preimage_singleton_empty d k h hs, measure_empty]
theorem affineHistogramKernel_apply_eq_map
Under the stated inputs and conditions, The actual conditional histogram law is the pushforward of the full count law. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
conclusion
affineHistogramKernel n m d eps sigma hyp lat
= (affineFullCountKernel n m d eps sigma hyp lat).map (affineFullCountHistograms d (affineTuning n m d eps).Kstar)
Proof (Lean source)
-- @node: affineHistogramKernel_apply_eq_map
lemma affineHistogramKernel_apply_eq_map (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
affineHistogramKernel n m d eps sigma hyp lat =
(affineFullCountKernel n m d eps sigma hyp lat).map
(affineFullCountHistograms d (affineTuning n m d eps).Kstar) := by
rw [affineHistogramKernel, Kernel.deterministic_comp_eq_map, Kernel.map_apply]
exact affineFullCountHistograms_measurable d _
theorem affineHistogramKernel_singleton
Under the stated inputs and conditions, The conditional histogram singleton mass is the unique original count mass; no untracked coordinates or histogram multiplicities remain. This gives the stated result.
Formal statement
hypotheses
conclusion
affineHistogramKernel n m d eps sigma hyp lat {h}
= if affineHistogramSupported d (affineTuning n m d eps).Kstar h then affineFullCountKernel n m d eps sigma hyp lat {affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h} else 0
Proof (Lean source)
-- @node: affineHistogramKernel_singleton
lemma affineHistogramKernel_singleton (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(hk : (affineTuning n m d eps).Kstar < d)
(h : (Obs d → Nat) × (AuxObs d → Nat)) :
affineHistogramKernel n m d eps sigma hyp lat {h} =
if affineHistogramSupported d (affineTuning n m d eps).Kstar h then
affineFullCountKernel n m d eps sigma hyp lat
{affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h}
else 0 := by
rw [affineHistogramKernel_apply_eq_map]
exact affineFullCountHistograms_map_singleton d _ hk _ h
theorem affineFullCountKernel_singleton
Under the stated inputs and conditions, Conditional full count singleton probabilities factor over the independent rare cells and the independent common reservoir. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
conclusion
affineFullCountKernel n m d eps sigma hyp lat {counts}
= (∏ i, affineCellFullPoissonLaw sigma (affineTuning n m d eps).u (affineTuning n m d eps).w hyp (lat i) {counts.1 i}) * affineReservoirCountLaw n m d eps sigma {counts.2}
Proof (Lean source)
-- @node: affineFullCountKernel_singleton
lemma affineFullCountKernel_singleton (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(counts : (Fin (affineTuning n m d eps).Kstar →
((Nat × Nat) × (Nat × (Nat × Nat)))) × ((Nat × Nat) × (Nat × Nat))) :
affineFullCountKernel n m d eps sigma hyp lat {counts} =
(∏ i, affineCellFullPoissonLaw sigma (affineTuning n m d eps).u
(affineTuning n m d eps).w hyp (lat i) {counts.1 i}) *
affineReservoirCountLaw n m d eps sigma {counts.2} := by
rw [affineFullCountKernel, Kernel.prod_apply, Kernel.const_apply]
change ((Measure.pi fun i => affineCellFullPoissonLaw sigma
(affineTuning n m d eps).u (affineTuning n m d eps).w hyp (lat i)).prod
(affineReservoirCountLaw n m d eps sigma)) {(counts.1, counts.2)} = _
rw [← Set.singleton_prod_singleton, Measure.prod_prod, Measure.pi_singleton]
theorem affineHistogramKernel_supported_singleton
Under the stated inputs and conditions, A supported original histogram has the product probability of its read-back rare-cell counts and reservoir counts in the actual conditional experiment. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
sigma :
hyp :
lat :
hk :
(affineTuning n m d eps).Kstar < d
hs :
affineHistogramSupported d (affineTuning n m d eps).Kstar h
conclusion
counts :
affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h
affineHistogramKernel n m d eps sigma hyp lat {h}
= (∏ i, affineCellFullPoissonLaw sigma (affineTuning n m d eps).u (affineTuning n m d eps).w hyp (lat i) {counts.1 i}) * affineReservoirCountLaw n m d eps sigma {counts.2}
Proof (Lean source)
-- @node: affineHistogramKernel_supported_singleton
lemma affineHistogramKernel_supported_singleton (n m d : Nat) (eps : Real)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(hk : (affineTuning n m d eps).Kstar < d)
(h : (Obs d → Nat) × (AuxObs d → Nat))
(hs : affineHistogramSupported d (affineTuning n m d eps).Kstar h) :
let counts := affineHistogramCoordinates d (affineTuning n m d eps).Kstar hk h
affineHistogramKernel n m d eps sigma hyp lat {h} =
(∏ i, affineCellFullPoissonLaw sigma (affineTuning n m d eps).u
(affineTuning n m d eps).w hyp (lat i) {counts.1 i}) *
affineReservoirCountLaw n m d eps sigma {counts.2} := by
rw [affineHistogramKernel_singleton n m d eps sigma hyp lat hk h, if_pos hs]
exact affineFullCountKernel_singleton n m d eps sigma hyp lat _
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Membership 4 declarations Positivity and support of every normalized affine table under the common latent coupling.
Positivity and support of every normalized affine table under the common latent coupling.
Under the stated inputs and conditions, Every normalized affine atom is its raw mass divided by the common normalizer. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
j :
Fin d
a y :
conclusion
jointMass (normalizedLaw hyp n m d eps hd sigma lat) j a y
= rawTable hyp n m d eps sigma lat (j, a, y) / rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_jointMass_readback
lemma affine_jointMass_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) (a y : Bool) :
jointMass (normalizedLaw hyp n m d eps hd sigma lat) j a y =
rawTable hyp n m d eps sigma lat (j, a, y) / rawNormalizer n m d eps sigma lat := by
have hp := affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat
have hs := affine_rawTable_sum n m d eps hd heps sigma hyp lat
dsimp only [jointMass, normalizedLaw]
rw [affine_normalizedPMF_toReal _ _
(affine_rawTable_nonneg n m d eps hn hd heps heps' sigma hyp lat)
(by rw [hs]; linarith), hs]
Under the stated hypotheses, Summing the outcome marks leaves arm masses independent of the hypothesis. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
j :
Fin d
a :
conclusion
armMass (normalizedLaw hyp n m d eps hd sigma lat) j a
= (if hj : j.val < (affineTuning n m d eps).Kstar then (if a then (affineTuning n m d eps).alpha + (1 - eps) * latentNode sigma (lat ⟨j.val, hj⟩) else (1 - eps) * (affineTuning n m d eps).b0 + eps * latentNode sigma (lat ⟨j.val, hj⟩)) else if j.val = (affineTuning n m d eps).Kstar then (1 - ((affineTuning n m d eps).Kstar : Real) * latentMeanMass sigma) / 2 else 0) / rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_armMass_readback
lemma affine_armMass_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) (a : Bool) :
armMass (normalizedLaw hyp n m d eps hd sigma lat) j a =
(if hj : j.val < (affineTuning n m d eps).Kstar then
(if a then (affineTuning n m d eps).alpha + (1 - eps) * latentNode sigma (lat ⟨j.val, hj⟩)
else (1 - eps) * (affineTuning n m d eps).b0 + eps * latentNode sigma (lat ⟨j.val, hj⟩))
else if j.val = (affineTuning n m d eps).Kstar then
(1 - ((affineTuning n m d eps).Kstar : Real) * latentMeanMass sigma) / 2
else 0) / rawNormalizer n m d eps sigma lat := by
dsimp only [armMass]
simp_rw [affine_jointMass_readback n m d eps hn hd heps heps' sigma hyp lat]
rw [← Finset.sum_div]
congr 1
by_cases hj : j.val < (affineTuning n m d eps).Kstar
· cases a <;> simp [rawTable, hj, Fintype.sum_bool, bernoulliMass] <;> ring
· by_cases hr : j.val = (affineTuning n m d eps).Kstar <;>
simp [rawTable, hj, hr, Fintype.sum_bool, bernoulliMass]
theorem affine_cellMass_readback
Under the stated hypotheses, The normalized cell mass is the sum of its two normalized arm masses. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
j :
Fin d
conclusion
cellMass (normalizedLaw hyp n m d eps hd sigma lat) j
= (if hj : j.val < (affineTuning n m d eps).Kstar then (affineTuning n m d eps).b0 + latentNode sigma (lat ⟨j.val, hj⟩) else if j.val = (affineTuning n m d eps).Kstar then 1 - ((affineTuning n m d eps).Kstar : Real) * latentMeanMass sigma else 0) / rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_cellMass_readback
lemma affine_cellMass_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(j : Fin d) :
cellMass (normalizedLaw hyp n m d eps hd sigma lat) j =
(if hj : j.val < (affineTuning n m d eps).Kstar then
(affineTuning n m d eps).b0 + latentNode sigma (lat ⟨j.val, hj⟩)
else if j.val = (affineTuning n m d eps).Kstar then
1 - ((affineTuning n m d eps).Kstar : Real) * latentMeanMass sigma
else 0) / rawNormalizer n m d eps sigma lat := by
have hab : (affineTuning n m d eps).alpha = eps * (affineTuning n m d eps).b0 := by
dsimp only [affineTuning]; field_simp
change (∑ a : Bool, armMass _ j a) = _
simp_rw [affine_armMass_readback n m d eps hn hd heps heps' sigma hyp lat]
rw [← Finset.sum_div]
congr 1
by_cases hj : j.val < (affineTuning n m d eps).Kstar
· simp [hj, Fintype.sum_bool, hab]; ring
· by_cases hr : j.val = (affineTuning n m d eps).Kstar <;>
simp [hj, hr, Fintype.sum_bool] <;> ring
theorem affine_support_and_normalizer
Under the stated inputs and conditions, Reservoir and normalizer positivity, legal support, and pointwise common marginal are derived.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
conclusion 1
conclusion 2
lat :
1 / 2 < rawNormalizer n m d eps sigma lat
hyp :
ModelClass d eps (normalizedLaw hyp n m d eps hd sigma lat)
Proof (Lean source)
-- @node: affine_support_and_normalizer
lemma affine_support_and_normalizer (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
1 / 2 < 1 - ((affineTuning n m d eps).Kstar : Real) * latentMeanMass sigma ∧
∀ lat, 1 / 2 < rawNormalizer n m d eps sigma lat ∧
(∀ hyp, ModelClass d eps (normalizedLaw hyp n m d eps hd sigma lat)) ∧
auxMarginal (normalizedLaw true n m d eps hd sigma lat) =
auxMarginal (normalizedLaw false n m d eps hd sigma lat) := by
refine ⟨affine_reservoir_pos n m d eps hn hd heps heps' sigma, ?_⟩
intro lat
have hQ := affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat
have hQpos : 0 < rawNormalizer n m d eps sigma lat := by linarith
obtain ⟨_, _, _, _, _, _, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
refine ⟨hQ, ?_, ?_⟩
· intro hyp
constructor
intro j hjpos
have hjne := ne_of_gt hjpos
rw [propensity, if_neg hjne]
rw [le_div_iff₀ hjpos, div_le_iff₀ hjpos]
rw [affine_armMass_readback n m d eps hn hd heps heps' sigma hyp lat j true,
affine_cellMass_readback n m d eps hn hd heps heps' sigma hyp lat j]
have hab : (affineTuning n m d eps).alpha = eps * (affineTuning n m d eps).b0 := by
dsimp only [affineTuning]; field_simp
by_cases hj : j.val < (affineTuning n m d eps).Kstar
· simp only [dif_pos hj, if_true, hab]
have hv : 0 ≤ latentNode sigma (lat ⟨j.val, hj⟩) := by
cases lat ⟨j.val, hj⟩ with
| none => exact le_refl 0
| some i => exact (sigma.mem i).1
simp only [← mul_div_assoc]
constructor <;> apply (div_le_div_iff_of_pos_right hQpos).mpr
· nlinarith
· nlinarith
· simp only [dif_neg hj]
by_cases hr : j.val = (affineTuning n m d eps).Kstar
· simp only [if_pos hr]
have hp := affine_reservoir_pos n m d eps hn hd heps heps' sigma
simp only [← mul_div_assoc]
constructor <;> apply (div_le_div_iff_of_pos_right hQpos).mpr <;> nlinarith
· simp [hr]
· apply PMF.ext
intro z
apply (ENNReal.toReal_eq_toReal_iff'
((auxMarginal _).apply_ne_top z) ((auxMarginal _).apply_ne_top z)).mp
rcases z with ⟨j, a⟩
rw [auxMarginal_toReal_armMass, auxMarginal_toReal_armMass,
affine_armMass_readback n m d eps hn hd heps heps' sigma true lat,
affine_armMass_readback n m d eps hn hd heps heps' sigma false lat]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.NumericalConcentration 7 declarations Uniform numerical degree and rare-cell count estimates for concentration of the affine priors.
Uniform numerical degree and rare-cell count estimates for concentration of the affine priors.
Under the stated inputs and conditions, Half-scale exponential growth dominates the numerical linear budget above the threshold. This gives the stated result.
Proof (Lean source)
-- @node: affine_exp_half_linear_budget
lemma affine_exp_half_linear_budget (y : Real) (hy : 4096 ≤ y) :
180 * 2 ^ 30 * y ≤ exp (y / 2) := by
have hbase : (2 : Real) ^ 50 ≤ exp (y / 4) := by
have hp := pow_le_pow_left₀ (by norm_num : (0 : Real) ≤ 2)
Real.exp_one_gt_two.le 50
rw [← Real.exp_nat_mul] at hp
norm_num at hp ⊢
exact hp.trans (Real.exp_le_exp.mpr (show (50 : Real) ≤ y / 4 by linarith))
have hlin : y / 4 ≤ exp (y / 4) := by
linarith [Real.add_one_le_exp (y / 4)]
have hprod := mul_le_mul hbase hlin (by linarith : 0 ≤ y / 4)
(Real.exp_pos (y / 4)).le
rw [← Real.exp_add] at hprod
have hid : y / 4 + y / 4 = y / 2 := by ring
rw [hid] at hprod
norm_num at hprod ⊢
nlinarith
theorem affine_logScale_le_log_add_one
Under the stated inputs and conditions, Above label scale two, the scheduled logarithm is within one of the ordinary logarithm. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_logScale_le_log_add_one
lemma affine_logScale_le_log_add_one (n : Nat) (eps : Real)
(hS : 2 ≤ (n : Real) * eps) :
logScale n eps ≤ log ((n : Real) * eps) + 1 := by
have hbound : exp 1 + (n : Real) * eps ≤ exp 1 * ((n : Real) * eps) := by
nlinarith [Real.exp_one_gt_two]
have hlog := Real.log_le_log (by positivity : 0 < exp 1 + (n : Real) * eps) hbound
rw [Real.log_mul (Real.exp_pos 1).ne' (by linarith : (n : Real) * eps ≠ 0),
Real.log_exp] at hlog
simpa only [logScale, labelScale, add_comm] using hlog
Under the stated inputs and conditions, Above the affine threshold the polynomial degree is bounded by ten times log S. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_tuning_degree_le_log
lemma affine_tuning_degree_le_log (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps) :
4096 ≤ log ((n : Real) * eps) ∧
((affineTuning n m d eps).L : Real) ≤ 10 * log ((n : Real) * eps) := by
have htwo : (2 : Real) ≤ (n : Real) * eps := by
have h := Real.add_one_le_exp (4096 : Real)
linarith
have hy : 4096 ≤ log ((n : Real) * eps) := by
have h := Real.log_le_log (Real.exp_pos 4096) hS
simpa only [Real.log_exp] using h
have hell := affine_logScale_le_log_add_one n eps htwo
have hL := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.1
exact ⟨hy, by linarith⟩
Under the stated inputs and conditions, The floor of the reciprocal mass budget retains at least 500 M ε L cells. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affine_tuning_floor_count_lower
lemma affine_tuning_floor_count_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps) :
500 * (affineTuning n m d eps).M * eps * ((affineTuning n m d eps).L : Real) ≤
(floor (1 / (100 * (affineTuning n m d eps).b0)) : Real) := by
have hb := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.2.2.2.1
have hmass := affine_tuning_floor_mass_lower n m d eps hn hd heps heps' hS
have h := (div_le_iff₀ hb).mpr hmass
have hid : (1 / 200 : Real) / (affineTuning n m d eps).b0 =
500 * (affineTuning n m d eps).M * eps * ((affineTuning n m d eps).L : Real) := by
rw [affine_tuning_b0_identity n m d eps hn hd heps heps']
field_simp
ring
rw [hid] at h
exact h
theorem affine_tuning_count_exp_lower
Under the stated inputs and conditions, Both alphabet and floor budgets exceed the square-root-scale count in the affine regime. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ ((affineTuning n m d eps).Kstar : Real)
Proof (Lean source)
-- @node: affine_tuning_count_exp_lower
lemma affine_tuning_count_exp_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) :
exp (log ((n : Real) * eps) / 2) *
((affineTuning n m d eps).L : Real) / 18 ≤
((affineTuning n m d eps).Kstar : Real) := by
let S := (n : Real) * eps
let r := exp (log S / 2)
let ell := logScale n eps
let D := ((n : Real) + m) * eps * ell
have hSp : 0 < S := by dsimp [S]; linarith
have hr : 0 < r := Real.exp_pos _
have hrsq : r ^ 2 = S := by
dsimp [r]
rw [pow_two, ← Real.exp_add, show log S / 2 + log S / 2 = log S by ring,
Real.exp_log hSp]
have hr1 : 1 ≤ r := by
have hlog : 0 ≤ log S := Real.log_nonneg hS
exact Real.one_le_exp (by positivity)
have hrS : r ≤ S := by nlinarith
have hell : 1 ≤ ell := affine_logScale_one_le n eps heps.le
have hN : S ≤ ((n : Real) + m) * eps := by
dsimp [S]; nlinarith [Nat.cast_nonneg m (α := Real)]
have hDp : 0 < D := by dsimp [D]; nlinarith
have hdell : r * ell < (d : Real) := by
by_contra h
have hle : (d : Real) ≤ r * ell := le_of_not_gt h
have hnum : (d : Real) * r ≤ D := by
have ha := mul_le_mul_of_nonneg_right hle hr.le
have hb := mul_le_mul_of_nonneg_right hN (by linarith : 0 ≤ ell)
dsimp [D]
nlinarith [hrsq]
have hxle : (d : Real) / D ≤ 1 / r :=
(div_le_div_iff₀ hDp hr).mpr (by simpa using hnum)
have hs := pow_le_pow_left₀ (div_nonneg (Nat.cast_nonneg d) hDp.le) hxle 2
simp only [div_pow, one_pow, hrsq] at hs
apply (not_le_of_gt hx)
simpa only [div_pow] using hs
have hdhalf : (d : Real) / 2 ≤ ((d - 1 : Nat) : Real) := by
rw [Nat.cast_sub (by omega : 1 ≤ d), Nat.cast_one]
have hdR : (2 : Real) ≤ d := by exact_mod_cast hd
linarith
obtain ⟨hM, _, hLlo, hLhi, _, _, _, _⟩ :=
affine_tuning_bounds n m d eps hn hd heps heps'
have hL0 : 0 ≤ ((affineTuning n m d eps).L : Real) := Nat.cast_nonneg _
have halphabet : r * ((affineTuning n m d eps).L : Real) / 18 ≤ ((d - 1 : Nat) : Real) := by
have h := mul_le_mul_of_nonneg_left hLhi hr.le
change _ ≤ 9 * ell at hLhi
nlinarith
have hMe : 128 * S ≤ (affineTuning n m d eps).M * eps := by
have h := mul_le_mul_of_nonneg_right hM heps.le
dsimp [S]
nlinarith [mul_nonneg (Nat.cast_nonneg m) heps.le]
have hcap : r * ((affineTuning n m d eps).L : Real) / 18 ≤
(floor (1 / (100 * (affineTuning n m d eps).b0)) : Real) := by
have hf := affine_tuning_floor_count_lower n m d eps hn hd heps heps' hS
have ha := mul_le_mul_of_nonneg_right hMe hL0
have hb := mul_le_mul_of_nonneg_right hrS hL0
nlinarith
change r * ((affineTuning n m d eps).L : Real) / 18 ≤
((min (d - 1) (floor (1 / (100 * (affineTuning n m d eps).b0))) : Nat) : Real)
rw [Nat.cast_min]
exact le_min halphabet hcap
theorem affine_tuning_count_concentration_budget
Under the stated inputs and conditions, The scheduled cell count dominates the entire degree-dependent concentration budget. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
conclusion
2 ^ 30 * (1 + eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2)
≤ ((affineTuning n m d eps).Kstar : Real)
Proof (Lean source)
-- @node: affine_tuning_count_concentration_budget
lemma affine_tuning_count_concentration_budget (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) :
2 ^ 30 * (1 + eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2) ≤
((affineTuning n m d eps).Kstar : Real) := by
obtain ⟨hy, hLhi⟩ := affine_tuning_degree_le_log n m d eps hn hd heps heps' hS
have hS1 : 1 ≤ (n : Real) * eps := by
linarith [Real.add_one_le_exp (4096 : Real)]
have hK := affine_tuning_count_exp_lower n m d eps hn hd heps heps' hS1 hx
have hexp := affine_exp_half_linear_budget (log ((n : Real) * eps)) hy
have hLlo := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.1
have hell := affine_logScale_one_le n eps heps.le
have hL : (8 : Real) ≤ (affineTuning n m d eps).L := by linarith
have hprod := mul_le_mul_of_nonneg_right hexp
(show 0 ≤ ((affineTuning n m d eps).L : Real) by positivity)
have hKstrong : 10 * 2 ^ 30 * log ((n : Real) * eps) *
((affineTuning n m d eps).L : Real) ≤ ((affineTuning n m d eps).Kstar : Real) := by
nlinarith
have hepssq : eps ^ 2 ≤ (1 / 16 : Real) := by
have h := pow_le_pow_left₀ heps.le heps' 2
norm_num at h
exact h
have hdegree := mul_le_mul_of_nonneg_right hLhi
(show 0 ≤ ((affineTuning n m d eps).L : Real) by positivity)
have hweight := mul_le_mul_of_nonneg_right hepssq
(sq_nonneg ((affineTuning n m d eps).L : Real))
have hunit : 1 ≤ log ((n : Real) * eps) * ((affineTuning n m d eps).L : Real) := by
nlinarith
have hbudget : 1 + eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2 ≤
10 * log ((n : Real) * eps) * ((affineTuning n m d eps).L : Real) := by
nlinarith
have hscaled := mul_le_mul_of_nonneg_left hbudget (by norm_num : (0 : Real) ≤ 2 ^ 30)
nlinarith
theorem affine_normalized_target_tail_small
Under the stated inputs and conditions, Each normalized affine target has bad-event probability at most 1/128. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
conclusion
nu :
t :
fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
nu.real {lat | gap / 8 ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t hyp|}
≤ 1 / 128
Proof (Lean source)
-- @node: affine_normalized_target_tail_small
lemma affine_normalized_target_tail_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) (hyp : Bool) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let t := fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
let gap := t true - t false
nu.real {lat | gap / 8 ≤ |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t hyp|} ≤
1 / 128 := by
intro nu t gap
have hS1 : 1 ≤ (n : Real) * eps := by
linarith [Real.add_one_le_exp (4096 : Real)]
have htail := affine_normalized_target_gap_tail n m d eps hn hd heps heps' hS1 sigma hyp
change nu.real _ ≤ _ at htail
refine htail.trans ?_
have hK := affine_tuning_count_concentration_budget n m d eps hn hd heps heps' hS hx
have hKp : (0 : Real) < (affineTuning n m d eps).Kstar := by
exact_mod_cast affine_tuning_Kstar_pos n m d eps hn hd heps heps' hS1
apply (div_le_iff₀ hKp).mpr
norm_num at hK ⊢
linarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.NumericalSeparation 5 declarations Uniform numerical target separation for the affine priors, as in equation (10).
Uniform numerical target separation for the affine priors, as in equation (10).
theorem affine_logScale_one_le
Under the stated inputs and conditions, The logarithmic scale is at least one for nonnegative overlap. This gives the stated result.
Proof (Lean source)
-- @node: affine_logScale_one_le
lemma affine_logScale_one_le (n : Nat) (eps : Real) (heps : 0 ≤ eps) :
1 ≤ logScale n eps := by
have h := Real.log_le_log (Real.exp_pos 1)
(show exp 1 ≤ exp 1 + labelScale n eps by
apply le_add_of_nonneg_right; unfold labelScale; positivity)
simpa only [logScale, Real.log_exp] using h
theorem affine_tuning_b0_lower
Under the stated inputs and conditions, The scheduled baseline mass is bounded below in terms of the public indices. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affine_tuning_b0_lower
lemma affine_tuning_b0_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
1 / (230400000 * ((n : Real) + m) * eps * logScale n eps) ≤
(affineTuning n m d eps).b0 := by
obtain ⟨hMlo, hMhi, hLlo, hLhi, _, _, _, _⟩ :=
affine_tuning_bounds n m d eps hn hd heps heps'
have hnR : (1 : Real) ≤ n := by exact_mod_cast hn
have hmR : (0 : Real) ≤ m := Nat.cast_nonneg m
have hell : 0 < logScale n eps := lt_of_lt_of_le (by norm_num)
(affine_logScale_one_le n eps heps.le)
have hMp : 0 < (affineTuning n m d eps).M := by linarith
have hLp : 0 < ((affineTuning n m d eps).L : Real) := by linarith
rw [affine_tuning_b0_identity n m d eps hn hd heps heps']
apply one_div_le_one_div_of_le (by positivity)
have hprod := mul_le_mul hMhi hLhi (by positivity : 0 ≤ ((affineTuning n m d eps).L : Real))
(by positivity : 0 ≤ 256 * ((n : Real) + m))
nlinarith [mul_le_mul_of_nonneg_right hprod heps.le]
Under the stated inputs and conditions, The floor cap retains at least half of its reciprocal mass budget. This gives the stated result.
Formal statement
hypotheses
conclusion
1 / 200
Proof (Lean source)
-- @node: affine_tuning_floor_mass_lower
lemma affine_tuning_floor_mass_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps) :
1 / 200 ≤ (floor (1 / (100 * (affineTuning n m d eps).b0)) : Real) *
(affineTuning n m d eps).b0 := by
have hb := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.2.2.2.1
have hK := affine_tuning_Kstar_pos n m d eps hn hd heps heps' hS
have hf : 1 ≤ floor (1 / (100 * (affineTuning n m d eps).b0)) := by
change 0 < min (d - 1) (floor (1 / (100 * (affineTuning n m d eps).b0))) at hK
omega
have hfR : (1 : Real) ≤ floor (1 / (100 * (affineTuning n m d eps).b0)) := by
exact_mod_cast hf
have hlt := Nat.lt_floor_add_one (1 / (100 * (affineTuning n m d eps).b0))
have hhalf : 1 / (100 * (affineTuning n m d eps).b0) ≤
2 * (floor (1 / (100 * (affineTuning n m d eps).b0)) : Real) := by linarith
have hmul := mul_le_mul_of_nonneg_right hhalf hb.le
have hid : (1 / (100 * (affineTuning n m d eps).b0)) *
(affineTuning n m d eps).b0 = 1 / 100 := by field_simp
rw [hid] at hmul
linarith
theorem affine_tuning_total_mass_lower
Under the stated inputs and conditions, The actual rare-cell mass budget dominates the clipped public affine scale. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affine_tuning_total_mass_lower
lemma affine_tuning_total_mass_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps) :
min ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) 1 / 460800000 ≤
((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).b0 := by
have hb := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.2.2.2.1
have hblo := affine_tuning_b0_lower n m d eps hn hd heps heps'
have hfloor := affine_tuning_floor_mass_lower n m d eps hn hd heps heps' hS
have hdR : (d : Real) / 2 ≤ (d - 1 : Nat) := by
rw [Nat.cast_sub (by omega : 1 ≤ d), Nat.cast_one]
have : (2 : Real) ≤ d := by exact_mod_cast hd
linarith
have hnR : (1 : Real) ≤ n := by exact_mod_cast hn
have hmR : (0 : Real) ≤ m := Nat.cast_nonneg m
have hell : 0 < logScale n eps := lt_of_lt_of_le (by norm_num)
(affine_logScale_one_le n eps heps.le)
have hcell : (d : Real) / (((n : Real) + m) * eps * logScale n eps) / 460800000 ≤
((d - 1 : Nat) : Real) * (affineTuning n m d eps).b0 := by
have hprod := mul_le_mul hdR hblo (by positivity) (by positivity)
calc
_ = (d : Real) / 2 * (1 / (230400000 * ((n : Real) + m) * eps * logScale n eps)) := by
field_simp
<;> ring
_ ≤ _ := hprod
change _ ≤ ((min (d - 1) (floor (1 / (100 * (affineTuning n m d eps).b0))) : Nat) : Real) * _
by_cases hcap : d - 1 ≤ floor (1 / (100 * (affineTuning n m d eps).b0))
· rw [min_eq_left hcap]
exact (div_le_div_of_nonneg_right (min_le_left _ _) (by norm_num)).trans hcell
· rw [min_eq_right (le_of_not_ge hcap)]
have hclip : min ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) 1 /
460800000 ≤ (1 : Real) / 200 := by
have := min_le_right ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) 1
linarith
exact hclip.trans hfloor
theorem affine_raw_target_mean_gap_uniform
Under the stated hypotheses, The raw target mean gap has the uniform numerical scale claimed in equation (10). This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
sigma :
conclusion
≤ (∫ lat, affineRawTarget n m d eps sigma true lat ∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure))
Proof (Lean source)
-- @node: affine_raw_target_mean_gap_uniform
lemma affine_raw_target_mean_gap_uniform (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
min ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) 1 / 10000000000 ≤
(∫ lat, affineRawTarget n m d eps sigma true lat
∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)) -
(∫ lat, affineRawTarget n m d eps sigma false lat
∂Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)) := by
have hmass := affine_tuning_total_mass_lower n m d eps hn hd heps heps' hS
have hgap := affine_raw_target_mean_gap n m d eps hn hd heps heps' sigma
have hell : 0 < logScale n eps := lt_of_lt_of_le (by norm_num)
(affine_logScale_one_le n eps heps.le)
have hclip : 0 ≤ min ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) 1 := by
apply le_min <;> positivity
linarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.NumericalTV 6 declarations Uniform numerical bound on the joint raw-mixture Taylor envelope in equation (8).
Uniform numerical bound on the joint raw-mixture Taylor envelope in equation (8).
Under the stated inputs and conditions, The scheduled Taylor argument is exactly the degree divided by 500. This gives the stated result.
Formal statement
hypotheses
conclusion
2 * (affineTuning n m d eps).M * (affineTuning n m d eps).B
= ((affineTuning n m d eps).L : Real) / 500
Proof (Lean source)
-- @node: affine_tuning_taylor_argument
lemma affine_tuning_taylor_argument (n m d : Nat) (eps : Real) (hn : 1 ≤ n) :
2 * (affineTuning n m d eps).M * (affineTuning n m d eps).B =
((affineTuning n m d eps).L : Real) / 500 := by
have hnR : (1 : Real) ≤ n := by exact_mod_cast hn
have hmR : (0 : Real) ≤ m := Nat.cast_nonneg m
have hM : (affineTuning n m d eps).M ≠ 0 := by
dsimp [affineTuning]
linarith
change 2 * (affineTuning n m d eps).M *
(((affineTuning n m d eps).L : Real) / (1000 * (affineTuning n m d eps).M)) = _
field_simp
<;> ring
theorem affine_tuning_marked_intensity_bound Lemma affine_tuning_marked_intensity_bound in the paper ↗
Under the stated inputs and conditions, The summed marked intensity is controlled by the capped rare-cell mass budget. This gives the stated result.
Formal statement
hypotheses
conclusion
((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).u * (affineTuning n m d eps).alpha
Proof (Lean source)
-- @node: affine_tuning_marked_intensity_bound
lemma affine_tuning_marked_intensity_bound (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).u *
(affineTuning n m d eps).alpha ≤ (32 / 25 : Real) * ((n : Real) * eps) := by
have hbudget := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.2.2.2.2
have hab : (affineTuning n m d eps).alpha = eps * (affineTuning n m d eps).b0 := by
change _ = eps * ((affineTuning n m d eps).alpha / eps)
field_simp
rw [hab]
have hu : (affineTuning n m d eps).u = 128 * (n : Real) := rfl
rw [hu]
have h := mul_le_mul_of_nonneg_right hbudget
(show 0 ≤ 128 * (n : Real) * eps by positivity)
nlinarith
Under the stated inputs and conditions, Degree calibration makes the label-scale weighted exponential tail uniformly tiny. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_tuning_weighted_exp_tail
lemma affine_tuning_weighted_exp_tail (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
((n : Real) * eps) * exp (-5 * ((affineTuning n m d eps).L : Real)) ≤
(1 : Real) / 256 := by
have hL := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.1
have hell := affine_logScale_one_le n eps heps.le
have hS : (n : Real) * eps ≤ exp (logScale n eps) := by
rw [logScale, Real.exp_log (by unfold labelScale; positivity)]
unfold labelScale
linarith [Real.exp_pos 1]
have he8 : (256 : Real) ≤ exp 8 := by
have h := pow_le_pow_left₀ (by norm_num : (0 : Real) ≤ 2)
(Real.exp_one_gt_two.le) 8
rw [← Real.exp_nat_mul] at h
norm_num at h ⊢
exact h
have hneg : exp (-8 : Real) ≤ (1 : Real) / 256 := by
rw [Real.exp_neg, ← one_div]
exact one_div_le_one_div_of_le (by norm_num) he8
calc
_ ≤ exp (logScale n eps) *
exp (-5 * ((affineTuning n m d eps).L : Real)) :=
mul_le_mul_of_nonneg_right hS (Real.exp_pos _).le
_ = exp (logScale n eps - 5 * ((affineTuning n m d eps).L : Real)) := by
rw [← Real.exp_add]; congr 1; ring
_ ≤ exp (-39 * logScale n eps) := Real.exp_le_exp.mpr (by linarith)
_ ≤ exp (-8) := Real.exp_le_exp.mpr (by linarith)
_ ≤ _ := hneg
Under the stated inputs and conditions, The full K-cell Taylor envelope is below the testing distance budget of 1/64. This gives the stated result.
Formal statement
hypotheses
conclusion
((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).u * (affineTuning n m d eps).alpha * (∑' h : Nat, if (affineTuning n m d eps).L < h then (2 * (affineTuning n m d eps).M * (affineTuning n m d eps).B) ^ h / (h.factorial : Real) else 0) < 1 / 64
Proof (Lean source)
-- @node: affine_poisson_joint_tail_small
lemma affine_poisson_joint_tail_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).u *
(affineTuning n m d eps).alpha *
(∑' h : Nat, if (affineTuning n m d eps).L < h then
(2 * (affineTuning n m d eps).M * (affineTuning n m d eps).B) ^ h /
(h.factorial : Real) else 0) < 1 / 64 := by
obtain ⟨_, _, hL, _, _, ha, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hell := affine_logScale_one_le n eps heps.le
have hLn : 2 ≤ (affineTuning n m d eps).L := by
have : (2 : Real) ≤ (affineTuning n m d eps).L := by linarith
exact_mod_cast this
rw [affine_tuning_taylor_argument n m d eps hn]
have htail := affine_poisson_taylor_tail (affineTuning n m d eps).L hLn
have hintensity := affine_tuning_marked_intensity_bound n m d eps hn hd heps heps'
have hexp := affine_tuning_weighted_exp_tail n m d eps hn hd heps heps'
have hI : 0 ≤ ((affineTuning n m d eps).Kstar : Real) *
(affineTuning n m d eps).u * (affineTuning n m d eps).alpha := by
have hu : (affineTuning n m d eps).u = 128 * (n : Real) := rfl
rw [hu]; positivity
calc
_ ≤ ((affineTuning n m d eps).Kstar : Real) * (affineTuning n m d eps).u *
(affineTuning n m d eps).alpha *
(2 * exp (-5 * ((affineTuning n m d eps).L : Real))) :=
mul_le_mul_of_nonneg_left htail hI
_ ≤ ((32 / 25 : Real) * ((n : Real) * eps)) *
(2 * exp (-5 * ((affineTuning n m d eps).L : Real))) :=
mul_le_mul_of_nonneg_right hintensity (by positivity)
_ ≤ (1 : Real) / 100 := by nlinarith
_ < _ := by norm_num
theorem affine_three_count_joint_envelope_small Lemma affine_three_count_joint_envelope_small in the paper ↗
Under the stated inputs and conditions, The full three-count coefficient envelope, weighted by the K-cell marked intensity, meets the raw testing budget directly. This gives the stated result.
Formal statement
hypotheses
conclusion
t :
affineTuning n m d eps
(t.Kstar : Real) * t.u * t.alpha * (∑' j : Nat, (exp (-t.M * t.b0) * (∑' c : Nat, t.M ^ c / (c.factorial : Real) * (∑' r : Nat, ∑' g : Nat, t.u ^ r / ((r + 1).factorial : Real) * (t.w ^ g / (g.factorial : Real)) * (((C (eps * t.b0) + C (1 - eps) * X) ^ (r + g)) * ((C ((1 - eps) * t.b0) + C eps * X) ^ c)).coeff j))) * t.B ^ j * ∑' k : Nat, if t.L < k + j then (t.M * t.B) ^ k / (k.factorial : Real) else 0) < 1 / 64
Proof (Lean source)
-- @node: affine_three_count_joint_envelope_small
lemma affine_three_count_joint_envelope_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
let t := affineTuning n m d eps
(t.Kstar : Real) * t.u * t.alpha *
(∑' j : Nat, (exp (-t.M * t.b0) *
(∑' c : Nat, t.M ^ c / (c.factorial : Real) *
(∑' r : Nat, ∑' g : Nat, t.u ^ r / ((r + 1).factorial : Real) *
(t.w ^ g / (g.factorial : Real)) *
(((C (eps * t.b0) +
C (1 - eps) * X) ^ (r + g)) *
((C ((1 - eps) * t.b0) +
C eps * X) ^ c)).coeff j))) *
t.B ^ j * ∑' k : Nat, if t.L < k + j then
(t.M * t.B) ^ k / (k.factorial : Real) else 0) < 1 / 64 := by
intro t
obtain ⟨_, _, _, _, hB, ha, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hu : 0 ≤ t.u := by change 0 ≤ 128 * (n : Real); positivity
have hw : 0 ≤ t.w := by change 0 ≤ 128 * ((n : Real) + m); positivity
have hM : t.M = t.u + t.w := rfl
have hI : 0 ≤ (t.Kstar : Real) * t.u * t.alpha := by
exact mul_nonneg (mul_nonneg (Nat.cast_nonneg _) hu) ha.le
exact lt_of_le_of_lt
(mul_le_mul_of_nonneg_left
(affine_three_count_total_degree_tail_le t.b0 eps t.u t.w t.M t.B t.L
hb.le heps.le (by linarith) hu hw hM hB.le) hI)
(affine_poisson_joint_tail_small n m d eps hn hd heps heps')
theorem affine_signed_raw_joint_count_small Lemma affine_signed_raw_joint_count_small in the paper ↗
Under the stated inputs and conditions, The K-cell sum of the absolute equation (6) signed masses meets the testing budget in equation (8), with no bound on the common control baseline. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
t :
affineTuning n m d eps
(t.Kstar : Real) * (∑' v : Nat × (Nat × Nat), |∑ i, sigma.weights i * (exp (-t.M * (t.b0 + sigma.nodes i)) * (t.u * (eps * t.b0 + (1 - eps) * sigma.nodes i)) ^ (v.2.1 + 1) * (t.w * (eps * t.b0 + (1 - eps) * sigma.nodes i)) ^ v.2.2 * (t.M * ((1 - eps) * t.b0 + eps * sigma.nodes i)) ^ v.1 / (((v.2.1 + 1).factorial : Real) * (v.2.2.factorial : Real) * (v.1.factorial : Real)) * (eps * t.b0 / (eps * t.b0 + (1 - eps) * sigma.nodes i)))|) < 1 / 64
Proof (Lean source)
-- @node: affine_signed_raw_joint_count_small
lemma affine_signed_raw_joint_count_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let t := affineTuning n m d eps
(t.Kstar : Real) * (∑' v : Nat × (Nat × Nat),
|∑ i, sigma.weights i * (exp (-t.M * (t.b0 + sigma.nodes i)) *
(t.u * (eps * t.b0 + (1 - eps) * sigma.nodes i)) ^ (v.2.1 + 1) *
(t.w * (eps * t.b0 + (1 - eps) * sigma.nodes i)) ^ v.2.2 *
(t.M * ((1 - eps) * t.b0 + eps * sigma.nodes i)) ^ v.1 /
(((v.2.1 + 1).factorial : Real) * (v.2.2.factorial : Real) *
(v.1.factorial : Real)) *
(eps * t.b0 / (eps * t.b0 + (1 - eps) * sigma.nodes i)))|) < 1 / 64 := by
intro t
obtain ⟨_, _, _, _, _, _, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hu : 0 ≤ t.u := by change 0 ≤ 128 * (n : Real); positivity
have hw : 0 ≤ t.w := by change 0 ≤ 128 * ((n : Real) + m); positivity
have h := (affine_signed_raw_count_tail_bound sigma t.b0 t.u t.w t.M
hb heps (by linarith) hu hw rfl).2
have ha : t.alpha = eps * t.b0 := by
change t.alpha = eps * (t.alpha / eps)
field_simp
calc
_ ≤ (t.Kstar : Real) * (t.u * (eps * t.b0) *
∑' h : Nat, if t.L < h then (2 * t.M * t.B) ^ h / (h.factorial : Real) else 0) :=
mul_le_mul_of_nonneg_left h (Nat.cast_nonneg _)
_ = (t.Kstar : Real) * t.u * t.alpha *
(∑' h : Nat, if t.L < h then (2 * t.M * t.B) ^ h / (h.factorial : Real) else 0) := by
rw [ha]; ring
_ < _ := affine_poisson_joint_tail_small n m d eps hn hd heps heps'
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.NumericalTransfer 4 declarations Numerical prefix-failure and squared-gap bounds for the fixed-size affine transfer.
Numerical prefix-failure and squared-gap bounds for the fixed-size affine transfer. These are the final numerical estimates following equations (10)--(12).
theorem affine_weighted_prefix_failure_bound Lemma affine_weighted_prefix_failure_bound in the paper ↗
Under the stated inputs and conditions, Above unit label scale, the weighted failure envelope is bounded at scale one. This gives the stated result.
Proof (Lean source)
-- @node: affine_weighted_prefix_failure_bound
lemma affine_weighted_prefix_failure_bound (S : Real) (hS : 1 ≤ S) :
S * exp (-124 * S) ≤ exp (-124) := by
have hlin : S ≤ exp (124 * (S - 1)) := by
have h := Real.add_one_le_exp (124 * (S - 1))
linarith
calc
_ ≤ exp (124 * (S - 1)) * exp (-124 * S) :=
mul_le_mul_of_nonneg_right hlin (Real.exp_pos _).le
_ = _ := by rw [← Real.exp_add]; congr 1; ring
Under the stated inputs and conditions, The two prefix failures cost at most 10⁻²⁴ divided by the label scale. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_fixed_size_failure_small
lemma affine_fixed_size_failure_small (n : Nat) (eps : Real)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) (hS : 1 ≤ (n : Real) * eps) :
2 * exp (-31 * (n : Real)) ≤
(1 : Real) / 1000000000000000000000000 / ((n : Real) * eps) := by
have hn : 4 * ((n : Real) * eps) ≤ n := by
have h := mul_le_mul_of_nonneg_left heps' (Nat.cast_nonneg n : (0 : Real) ≤ n)
linarith
have he124 : (2 : Real) ^ 124 ≤ exp 124 := by
have h := pow_le_pow_left₀ (by norm_num : (0 : Real) ≤ 2)
(le_of_lt Real.exp_one_gt_two) 124
rw [← Real.exp_nat_mul] at h
norm_num at h ⊢
exact h
have hsmall : 2 * exp (-124 : Real) ≤
(1 : Real) / 1000000000000000000000000 := by
rw [Real.exp_neg, ← one_div]
rw [← mul_div_assoc, mul_one]
apply (div_le_iff₀ (Real.exp_pos 124)).mpr
norm_num at he124 ⊢
linarith
apply (le_div_iff₀ (by linarith : 0 < (n : Real) * eps)).mpr
have hmono := Real.exp_le_exp.mpr (show -31 * (n : Real) ≤
-124 * ((n : Real) * eps) by linarith)
have hweighted := affine_weighted_prefix_failure_bound ((n : Real) * eps) hS
have hm := mul_le_mul_of_nonneg_right hmono (show 0 ≤ (n : Real) * eps by positivity)
nlinarith
theorem affine_raw_target_mean_gap_sq_lower
Under the stated inputs and conditions, In the affine regime, the squared raw target gap dominates 10⁻²⁰/S. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
sigma :
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
Proof (Lean source)
-- @node: affine_raw_target_mean_gap_sq_lower
lemma affine_raw_target_mean_gap_sq_lower (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
(1 : Real) / 100000000000000000000 / ((n : Real) * eps) ≤ gap ^ 2 := by
intro nu gap
let x := (d : Real) / (((n : Real) + m) * eps * logScale n eps)
have hgap : min x 1 / 10000000000 ≤ gap :=
affine_raw_target_mean_gap_uniform n m d eps hn hd heps heps' hS sigma
have hell := affine_logScale_one_le n eps heps.le
have hx0 : 0 ≤ x := by dsimp [x]; positivity
have hclip : 0 ≤ min x 1 := le_min hx0 (by norm_num)
have hsq := pow_le_pow_left₀ (div_nonneg hclip (by norm_num)) hgap 2
have hmin : 1 / ((n : Real) * eps) ≤ (min x 1) ^ 2 := by
by_cases hx1 : x ≤ 1
· rw [min_eq_left hx1]; exact hx.le
· rw [min_eq_right (le_of_not_ge hx1)]
have h := one_div_le_one_div_of_le (by norm_num : (0 : Real) < 1) hS
simpa using h
have hscaled := mul_le_mul_of_nonneg_left hmin
(by norm_num : (0 : Real) ≤ 1 / 100000000000000000000)
simp only [div_eq_mul_inv] at hscaled hsq ⊢
norm_num at hscaled hsq ⊢
nlinarith
theorem affine_fixed_size_failure_gap
Under the stated inputs and conditions, Prefix failures consume at most half of the raw testing lower bound. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
sigma :
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
Proof (Lean source)
-- @node: affine_fixed_size_failure_gap
lemma affine_fixed_size_failure_gap (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
2 * exp (-31 * (n : Real)) ≤ gap ^ 2 / 128 := by
intro nu gap
have hf := affine_fixed_size_failure_small n eps heps heps' hS
have hg := affine_raw_target_mean_gap_sq_lower n m d eps hn hd heps heps' hS hx sigma
change (1 : Real) / 100000000000000000000 / ((n : Real) * eps) ≤ gap ^ 2 at hg
have hpos : 0 ≤ 1 / ((n : Real) * eps) := by positivity
simp only [div_eq_mul_inv] at hf hg hpos ⊢
norm_num at hf hg hpos ⊢
nlinarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.OrderedExperiment 6 declarations The actual ordered random-scale experiment and its two-channel mixture distance, including the independent uniform seed used by randomized rules.
The actual ordered random-scale experiment and its two-channel mixture distance, including the independent uniform seed used by randomized rules.
Conditional on the latent table, both ordered channels have their normalized mark laws and independent Poisson totals at the shared raw scale.
Definition (Lean source)
parameters
n m d :
eps :
hd :
2 ≤ d
sigma :
hyp :
def
affineOrderedRawKernel n m d eps hd sigma hyp :
Kernel (Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) (RawTwoPool (Obs d) (AuxObs d))
given by
Kernel.ofFunOfCountable fun lat
=> let t := affineTuning n m d eps let P := normalizedLaw hyp n m d eps hd sigma lat let Q := rawNormalizer n m d eps sigma lat (finitePoissonSampleLaw P.pmf.toMeasure (toNNReal (t.u * Q))).prod (finitePoissonSampleLaw (auxMarginal P).toMeasure (toNNReal (t.w * Q)))
instance affineOrderedRawKernel_isMarkovKernel
The actual ordered experiment is a probability kernel.
Definition (Lean source)
-- @node: affineOrderedRawKernel_isMarkovKernel
instance affineOrderedRawKernel_isMarkovKernel (n m d : Nat) (eps : Real) (hd : 2 ≤ d)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) : IsMarkovKernel (affineOrderedRawKernel n m d eps hd sigma hyp) where
isProbabilityMeasure lat := by
change IsProbabilityMeasure ((finitePoissonSampleLaw _ _).prod (finitePoissonSampleLaw _ _))
infer_instance
theorem affineOrderedRawKernel_isRandomScale
Under the stated inputs and conditions, The exact normalized mark laws and raw normalizer instantiate the shared-scale experiment property required by the fixed-size transfer theorem. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hd :
2 ≤ d
sigma :
hyp :
conclusion
P :
Kernel.ofFunOfCountable fun lat
=> (normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure
Q :
Kernel.ofFunOfCountable fun lat
=> (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure
letI : ∀ lat, IsProbabilityMeasure (P lat)
Proof (Lean source)
-- @node: affineOrderedRawKernel_isRandomScale
lemma affineOrderedRawKernel_isRandomScale (n m d : Nat) (eps : Real) (hd : 2 ≤ d)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) :
let P := Kernel.ofFunOfCountable fun lat =>
(normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure
let Q := Kernel.ofFunOfCountable fun lat =>
(auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure
letI : ∀ lat, IsProbabilityMeasure (P lat) := by
intro lat
change IsProbabilityMeasure (normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure
infer_instance
letI : ∀ lat, IsProbabilityMeasure (Q lat) := by
intro lat
change IsProbabilityMeasure (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure
infer_instance
let S := fun lat => toNNReal (rawNormalizer n m d eps sigma lat)
IsRandomScaleTwoPoolExperiment (affineOrderedRawKernel n m d eps hd sigma hyp)
P Q S (toNNReal (affineTuning n m d eps).u)
(toNNReal (affineTuning n m d eps).w) := by
intro P Q S lat
change (finitePoissonSampleLaw (normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure
(toNNReal (_ * _))).prod
(finitePoissonSampleLaw (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure
(toNNReal (_ * _))) =
(finitePoissonSampleLaw (normalizedLaw hyp n m d eps hd sigma lat).pmf.toMeasure
(toNNReal _ * toNNReal _)).prod
(finitePoissonSampleLaw (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat)).toMeasure
(toNNReal _ * toNNReal _))
rw [Real.toNNReal_mul (by change 0 ≤ 128 * (n : Real); positivity),
Real.toNNReal_mul (by change 0 ≤ 128 * ((n : Real) + m); positivity)]
theorem affineOrderedRawKernel_eq_reconstruction
Under the stated inputs and conditions, The atom-histogram reconstruction equals the actual ordered conditional law. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
conclusion
affineOrderedRawKernel n m d eps hd sigma hyp
= pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₖ affineHistogramKernel n m d eps sigma hyp
Proof (Lean source)
-- @node: affineOrderedRawKernel_eq_reconstruction
lemma affineOrderedRawKernel_eq_reconstruction (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) :
affineOrderedRawKernel n m d eps hd sigma hyp =
pairedHistogramReconstructionKernel (Obs d) (AuxObs d) ∘ₖ
affineHistogramKernel n m d eps sigma hyp := by
ext lat : 1
exact (affineHistogramKernel_ordered_apply n m d eps hn hd heps heps' sigma hyp lat).symm
theorem affineOrderedRawKernel_predictive_tv_small
Under the stated inputs and conditions, The distance budget holds for the actual normalized ordered random-scale experiment, retaining both original observation channels. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
nu :
tvDist (priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma false)) (priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma true)) < 1 / 64
Proof (Lean source)
-- @node: affineOrderedRawKernel_predictive_tv_small
lemma affineOrderedRawKernel_predictive_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
tvDist
(priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma false))
(priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma true)) < 1 / 64 := by
intro nu
rw [affineOrderedRawKernel_eq_reconstruction n m d eps hn hd heps heps' sigma false,
affineOrderedRawKernel_eq_reconstruction n m d eps hn hd heps heps' sigma true]
exact affineHistogramKernel_ordered_predictive_tv_small n m d eps hn hd heps heps' sigma
theorem affineOrderedRawKernel_randomized_tv_small
Under the stated inputs and conditions, A common independent uniform seed preserves the ordered mixture TV budget. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
nu :
tvDist ((priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma false)).prod seedLaw) ((priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma true)).prod seedLaw) < 1 / 64
Proof (Lean source)
-- @node: affineOrderedRawKernel_randomized_tv_small
lemma affineOrderedRawKernel_randomized_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
tvDist
((priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma false)).prod seedLaw)
((priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma true)).prod seedLaw) <
1 / 64 := by
intro nu
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
let hp (hyp : Bool) : IsProbabilityMeasure
(priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma hyp)) :=
priorPredictive_isProbability nu _ (fun _ => inferInstance)
let := hp false
let := hp true
have hprod := tvDist_prod_le_add
(priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma false))
(priorPredictive nu (affineOrderedRawKernel n m d eps hd sigma true)) seedLaw seedLaw
have hself : tvDist seedLaw seedLaw = 0 := by
simp [tvDist]
rw [hself, add_zero] at hprod
exact hprod.trans_lt
(affineOrderedRawKernel_predictive_tv_small n m d eps hn hd heps heps' sigma)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.ProductExperiment 8 declarations Identification of the independent rare-cell predictive law with the mixture of conditional raw cell products over the construction's independent latent draws.
Identification of the independent rare-cell predictive law with the mixture of conditional raw cell products over the construction's independent latent draws.
Conditional independent raw count laws of the rare cells, before restoring control-channel splits and ordering the records.
Definition (Lean source)
parameters
def
affineRareCellProductKernel sigma k u w hyp :
given by
Kernel.ofFunOfCountable fun lat
=> Measure.pi fun i : Fin k => affineCellPoissonLaw sigma u w hyp (lat i)
instance affineRareCellProductKernel_isMarkovKernel
Every conditional rare-cell product is a probability law.
Definition (Lean source)
-- @node: affineRareCellProductKernel_isMarkovKernel
instance affineRareCellProductKernel_isMarkovKernel {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (k : Nat) (u w : Real) (hyp : Bool) :
IsMarkovKernel (affineRareCellProductKernel sigma k u w hyp) where
isProbabilityMeasure lat := by
change IsProbabilityMeasure
(Measure.pi fun i : Fin k => affineCellPoissonLaw sigma u w hyp (lat i))
infer_instance
theorem affineRareCellProductKernel_predictive_eq
Under the stated inputs and conditions, Mixing independent conditional raw cells over the actual latent product prior gives exactly the product of the actual core/filler predictive measures. This gives the stated result.
Formal statement
hypotheses
conclusion
priorPredictive (Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure) (affineRareCellProductKernel sigma k u w hyp)
= Measure.pi fun _ : Fin k => affineCellPoissonPredictive sigma u w hyp
Proof (Lean source)
-- @node: affineRareCellProductKernel_predictive_eq
lemma affineRareCellProductKernel_predictive_eq {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (k : Nat) (u w : Real) (hyp : Bool) :
priorPredictive
(Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure)
(affineRareCellProductKernel sigma k u w hyp) =
Measure.pi fun _ : Fin k => affineCellPoissonPredictive sigma u w hyp := by
classical
apply Measure.ext_of_singleton
intro o
rw [priorPredictive_apply _ _ (measurableSet_singleton o), lintegral_fintype]
change (∑ lat : Fin k → Latent L,
(Measure.pi fun i : Fin k => affineCellPoissonLaw sigma u w hyp (lat i)) {o} *
(Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure) {lat}) = _
simp only [Measure.pi_singleton,
PMF.toMeasure_apply_singleton _ _ (measurableSet_singleton _)]
simp only [affineCellPoissonPredictive, Measure.finsetSum_apply, Measure.smul_apply,
smul_eq_mul]
rw [Fintype.prod_sum]
apply Finset.sum_congr rfl
intro lat _
rw [← Finset.prod_mul_distrib]
apply Finset.prod_congr rfl
intro i _
exact mul_comm _ _
theorem affineRareCellProductKernel_predictive_tv_small
Under the stated inputs and conditions, The raw rare-cell mixture itself inherits the joint one-sixty-fourth TV budget; the product identity eliminates any identification hypothesis. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
t :
affineTuning n m d eps
tvDist (priorPredictive nu (affineRareCellProductKernel sigma t.Kstar t.u t.w true)) (priorPredictive nu (affineRareCellProductKernel sigma t.Kstar t.u t.w false)) < 1 / 64
Proof (Lean source)
-- @node: affineRareCellProductKernel_predictive_tv_small
lemma affineRareCellProductKernel_predictive_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let t := affineTuning n m d eps
let nu := Measure.pi fun _ : Fin t.Kstar => (latentPMF sigma).toMeasure
tvDist
(priorPredictive nu (affineRareCellProductKernel sigma t.Kstar t.u t.w true))
(priorPredictive nu (affineRareCellProductKernel sigma t.Kstar t.u t.w false)) <
1 / 64 := by
intro t nu
rw [affineRareCellProductKernel_predictive_eq, affineRareCellProductKernel_predictive_eq]
exact affineCellPoissonPredictive_product_tv_small n m d eps hn hd heps heps' sigma
Attach a common independent reservoir to the conditional rare-cell counts.
instance affineRareCellReservoirKernel_isMarkovKernel
A probability reservoir preserves the conditional probability law.
Definition (Lean source)
-- @node: affineRareCellReservoirKernel_isMarkovKernel
instance affineRareCellReservoirKernel_isMarkovKernel {R : Type*} [MeasurableSpace R]
(rho : Measure R) [IsProbabilityMeasure rho] {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (k : Nat) (u w : Real) (hyp : Bool) :
IsMarkovKernel (affineRareCellReservoirKernel rho sigma k u w hyp) := by
unfold affineRareCellReservoirKernel
infer_instance
theorem affineRareCellReservoirKernel_predictive_eq
Under the stated inputs and conditions, The common reservoir factors out of the actual finite latent mixture. This gives the stated result.
Formal statement
hypotheses
conclusion
priorPredictive (Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure) (affineRareCellReservoirKernel rho sigma k u w hyp)
Proof (Lean source)
-- @node: affineRareCellReservoirKernel_predictive_eq
lemma affineRareCellReservoirKernel_predictive_eq {R : Type*} [MeasurableSpace R]
(rho : Measure R) [IsProbabilityMeasure rho] {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (k : Nat) (u w : Real) (hyp : Bool) :
priorPredictive
(Measure.pi fun _ : Fin k => (latentPMF sigma).toMeasure)
(affineRareCellReservoirKernel rho sigma k u w hyp) =
(Measure.pi fun _ : Fin k => affineCellPoissonPredictive sigma u w hyp).prod rho := by
refine (Measure.prod_eq ?_).symm
intro s r hs hr
rw [priorPredictive_apply _ _ (hs.prod hr)]
simp only [affineRareCellReservoirKernel, Kernel.prod_apply_prod, Kernel.const_apply]
rw [lintegral_mul_const _ ((affineRareCellProductKernel sigma k u w hyp).measurable_coe hs),
← priorPredictive_apply _ _ hs, affineRareCellProductKernel_predictive_eq]
theorem affineRareCellReservoirKernel_reconstruction_tv_small
Under the stated inputs and conditions, A common reconstruction channel, including control splits, ordering and a common random seed, preserves the TV budget of the mixed raw cells and reservoir. This gives the stated result.
Formal statement
hypotheses
R X :
rho :
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
reconstruct :
IsMarkovKernel reconstruct
conclusion
t :
affineTuning n m d eps
tvDist (priorPredictive nu (reconstruct ∘ₖ affineRareCellReservoirKernel rho sigma t.Kstar t.u t.w true)) (priorPredictive nu (reconstruct ∘ₖ affineRareCellReservoirKernel rho sigma t.Kstar t.u t.w false)) < 1 / 64
Proof (Lean source)
-- @node: affineRareCellReservoirKernel_reconstruction_tv_small
lemma affineRareCellReservoirKernel_reconstruction_tv_small
{R X : Type*} [MeasurableSpace R] [MeasurableSpace X]
(rho : Measure R) [IsProbabilityMeasure rho]
(n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(reconstruct : Kernel
((Fin (affineTuning n m d eps).Kstar → MarkedPoissonObservation) × R) X)
[IsMarkovKernel reconstruct] :
let t := affineTuning n m d eps
let nu := Measure.pi fun _ : Fin t.Kstar => (latentPMF sigma).toMeasure
tvDist
(priorPredictive nu
(reconstruct ∘ₖ affineRareCellReservoirKernel rho sigma t.Kstar t.u t.w true))
(priorPredictive nu
(reconstruct ∘ₖ affineRareCellReservoirKernel rho sigma t.Kstar t.u t.w false)) <
1 / 64 := by
intro t nu
rw [kernel_comp_priorPredictive, kernel_comp_priorPredictive,
affineRareCellReservoirKernel_predictive_eq,
affineRareCellReservoirKernel_predictive_eq]
exact (tvDist_bind_le _ _ reconstruct).trans_lt
(affineCellPoissonPredictive_reservoir_tv_small rho n m d eps hn hd heps heps' sigma)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.ProductTV 2 declarations Tensorization of the actual affine core/filler predictive measures.
Tensorization of the actual affine core/filler predictive measures. Independent rare cells obey the joint Taylor budget, and a common reservoir law adds no distance.
theorem affineCellPoissonPredictive_product_tv_small
Under the stated inputs and conditions, The independent rare-cell product has total variation below one sixty-fourth. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
t :
affineTuning n m d eps
tvDist (Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w true) (Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w false) < 1 / 64
Proof (Lean source)
-- @node: affineCellPoissonPredictive_product_tv_small
lemma affineCellPoissonPredictive_product_tv_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let t := affineTuning n m d eps
tvDist
(Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w true)
(Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w false) <
1 / 64 := by
intro t
exact (tvDist_pi_iid_le t.Kstar _ _).trans_lt
(affineCellPoissonPredictive_joint_tv_small n m d eps hn hd heps heps' sigma)
theorem affineCellPoissonPredictive_reservoir_tv_small
Under the stated inputs and conditions, Adding any common independent reservoir factor preserves the joint distance budget. This gives the stated result.
Formal statement
hypotheses
R :
Type*
rho :
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
t :
affineTuning n m d eps
tvDist ((Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w true).prod rho) ((Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w false).prod rho) < 1 / 64
Proof (Lean source)
-- @node: affineCellPoissonPredictive_reservoir_tv_small
lemma affineCellPoissonPredictive_reservoir_tv_small {R : Type*} [MeasurableSpace R]
(rho : Measure R) [IsProbabilityMeasure rho]
(n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
let t := affineTuning n m d eps
tvDist
((Measure.pi fun _ : Fin t.Kstar =>
affineCellPoissonPredictive sigma t.u t.w true).prod rho)
((Measure.pi fun _ : Fin t.Kstar =>
affineCellPoissonPredictive sigma t.u t.w false).prod rho) < 1 / 64 := by
intro t
have h := tvDist_prod_le_add
(Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w true)
(Measure.pi fun _ : Fin t.Kstar => affineCellPoissonPredictive sigma t.u t.w false)
rho rho
have hself : tvDist rho rho = 0 := by
simp [tvDist]
rw [hself, add_zero] at h
exact h.trans_lt
(affineCellPoissonPredictive_product_tv_small n m d eps hn hd heps heps' sigma)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.RawBayes 9 declarations Two-prior testing for the actual ordered raw experiment.
Two-prior testing for the actual ordered raw experiment. A hypothesis tag retains its raw scale together with the latent vector, so the same parameter space can also be used by the random-scale fixed-size transfer.
The raw parameter retains the hypothesis and its finite latent draw.
Definition (Lean source)
parameters
def
AffineRawParameter n m d eps :
Type
given by
def affineRawTaggedPrior
The tagged prior draws the common latent vector with a fixed hypothesis.
Definition (Lean source)
parameters
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hyp :
def
given by
(Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure).map (fun lat => (hyp, lat))
instance affineRawTaggedPrior_isProbabilityMeasure
Both tagged priors are probability measures.
Definition (Lean source)
-- @node: affineRawTaggedPrior_isProbabilityMeasure
instance affineRawTaggedPrior_isProbabilityMeasure (hyp : Bool) :
IsProbabilityMeasure (affineRawTaggedPrior n m d eps sigma hyp) := by
unfold affineRawTaggedPrior
apply Measure.isProbabilityMeasure_map
fun_prop
def affineRawTaggedKernel
The tagged experiment keeps both ordered channels and the independent seed.
Definition (Lean source)
parameters
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
def
affineRawTaggedKernel n m d eps hd sigma :
given by
Kernel.ofFunOfCountable fun theta
instance affineRawTaggedKernel_isMarkovKernel
Each tagged conditional law is a probability law.
Definition (Lean source)
-- @node: affineRawTaggedKernel_isMarkovKernel
instance affineRawTaggedKernel_isMarkovKernel :
IsMarkovKernel (affineRawTaggedKernel n m d eps hd sigma) where
isProbabilityMeasure theta := by
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
change IsProbabilityMeasure ((_ : Measure (RawTwoPool (Obs d) (AuxObs d))).prod seedLaw)
infer_instance
theorem affineRawTaggedKernel_predictive
Under the stated inputs and conditions, Mixing the tagged experiment reads back to the seeded ordered raw mixture. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hyp :
conclusion
priorPredictive (affineRawTaggedPrior n m d eps sigma hyp) (affineRawTaggedKernel n m d eps hd sigma)
= (priorPredictive (Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure) (affineOrderedRawKernel n m d eps hd sigma hyp)).prod seedLaw
Proof (Lean source)
-- @node: affineRawTaggedKernel_predictive
lemma affineRawTaggedKernel_predictive (hyp : Bool) :
priorPredictive (affineRawTaggedPrior n m d eps sigma hyp)
(affineRawTaggedKernel n m d eps hd sigma) =
(priorPredictive
(Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
(affineOrderedRawKernel n m d eps hd sigma hyp)).prod seedLaw := by
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
ext A hA : 1
rw [priorPredictive_apply _ _ hA]
unfold affineRawTaggedPrior
rw [lintegral_map (measurable_of_countable _) (measurable_of_countable _)]
change (∫⁻ lat, ((affineOrderedRawKernel n m d eps hd sigma hyp lat).prod seedLaw) A
∂Measure.pi _) = _
rw [Measure.prod_apply hA]
rw [lintegral_priorPredictive _ _ _]
· apply lintegral_congr
intro lat
rw [Measure.prod_apply hA]
· exact measurable_measure_prodMk_left hA
theorem affineRawTaggedPrior_target_tail
Under the stated hypotheses, The tagged normalized target has the paper's concentration budget. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hn :
1 ≤ n
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hyp :
conclusion
nu :
t :
fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
target :
fun theta : AffineRawParameter n m d eps
=> ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)
(affineRawTaggedPrior n m d eps sigma hyp).real {theta | (t true - t false) / 8 < |target theta - t hyp|}
≤ 1 / 128
Proof (Lean source)
-- @node: affineRawTaggedPrior_target_tail
lemma affineRawTaggedPrior_target_tail
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) (hyp : Bool) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let t := fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
let target := fun theta : AffineRawParameter n m d eps =>
ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)
(affineRawTaggedPrior n m d eps sigma hyp).real
{theta | (t true - t false) / 8 < |target theta - t hyp|} ≤ 1 / 128 := by
intro nu t target
unfold affineRawTaggedPrior Measure.real
rw [Measure.map_apply (measurable_of_countable _) (Set.to_countable _).measurableSet]
have ht := affine_normalized_target_tail_small n m d eps hn hd heps heps' hS hx sigma hyp
apply le_trans (measureReal_mono ?_) ht
intro lat hlat
change (t true - t false) / 8 <
|ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t hyp| at hlat
exact hlat.le
theorem affineRawTaggedKernel_bayes_readback
Under the stated inputs and conditions, The tagged Bayes risk is exactly the latent average of the actual seeded conditional squared risk; tagging changes no observation or loss. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hyp :
estimator :
conclusion
bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma hyp) (affineRawTaggedKernel n m d eps hd sigma) (fun theta => ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)) estimator
= ∫⁻ lat, ∫⁻ z, ofReal ((estimator z - ateFunctional (normalizedLaw hyp n m d eps hd sigma lat)) ^ 2) ∂((affineOrderedRawKernel n m d eps hd sigma hyp lat).prod seedLaw) ∂(Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
Proof (Lean source)
-- @node: affineRawTaggedKernel_bayes_readback
lemma affineRawTaggedKernel_bayes_readback (hyp : Bool)
(estimator : RawTwoPool (Obs d) (AuxObs d) × Real → Real) :
bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma hyp)
(affineRawTaggedKernel n m d eps hd sigma)
(fun theta => ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)) estimator =
∫⁻ lat, ∫⁻ z,
ofReal ((estimator z -
ateFunctional (normalizedLaw hyp n m d eps hd sigma lat)) ^ 2)
∂((affineOrderedRawKernel n m d eps hd sigma hyp lat).prod seedLaw)
∂(Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure) := by
unfold bayesSquaredRisk affineRawTaggedPrior
rw [lintegral_map (measurable_of_countable _) (measurable_of_countable _)]
rfl
theorem affineRawTaggedKernel_bayes_lower
Under the stated hypotheses, Every randomized measurable rule in the actual raw experiment has a Bayes risk at least one sixty-fourth of the squared raw target gap. This gives the stated result.
Formal statement
hypotheses
n :
ℕ
m :
ℕ
d :
ℕ
eps :
ℝ
hd :
2 ≤ d
hn :
1 ≤ n
heps :
0 < eps
heps' :
eps ≤ 1 / 4
estimator :
hestimator :
Measurable estimator
conclusion
nu :
t :
fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
target :
fun theta : AffineRawParameter n m d eps
=> ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)
ofReal (gap ^ 2 / 64)
≤ max (bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma false) (affineRawTaggedKernel n m d eps hd sigma) target estimator) (bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma true) (affineRawTaggedKernel n m d eps hd sigma) target estimator)
Proof (Lean source)
-- @node: affineRawTaggedKernel_bayes_lower
lemma affineRawTaggedKernel_bayes_lower
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(estimator : RawTwoPool (Obs d) (AuxObs d) × Real → Real)
(hestimator : Measurable estimator) :
let nu := Measure.pi fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure
let t := fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
let gap := t true - t false
let target := fun theta : AffineRawParameter n m d eps =>
ateFunctional (normalizedLaw theta.1 n m d eps hd sigma theta.2)
ofReal (gap ^ 2 / 64) ≤
max (bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma false)
(affineRawTaggedKernel n m d eps hd sigma) target estimator)
(bayesSquaredRisk (affineRawTaggedPrior n m d eps sigma true)
(affineRawTaggedKernel n m d eps hd sigma) target estimator) := by
intro nu t gap target
have hS1 : 1 ≤ (n : Real) * eps := by
linarith [Real.add_one_le_exp (4096 : Real)]
have hKpos : (0 : Real) < (affineTuning n m d eps).Kstar := by
exact_mod_cast affine_tuning_Kstar_pos n m d eps hn hd heps heps' hS1
have hb := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.2.2.2.1
have hgapbound := affine_raw_target_mean_gap n m d eps hn hd heps heps' sigma
have hgap : 0 < gap := lt_of_lt_of_le (by positivity) hgapbound
have htv : tvDist
(priorPredictive (affineRawTaggedPrior n m d eps sigma false)
(affineRawTaggedKernel n m d eps hd sigma))
(priorPredictive (affineRawTaggedPrior n m d eps sigma true)
(affineRawTaggedKernel n m d eps hd sigma)) ≤ 1 / 64 := by
rw [affineRawTaggedKernel_predictive, affineRawTaggedKernel_predictive]
exact (affineOrderedRawKernel_randomized_tv_small n m d eps hn hd heps heps' sigma).le
have hlower := twoFuzzyHypotheses_bayesRisk_lower
(affineRawTaggedPrior n m d eps sigma false)
(affineRawTaggedPrior n m d eps sigma true)
(affineRawTaggedKernel n m d eps hd sigma) target (fun _ => inferInstance)
estimator hestimator (measurable_of_countable _)
(t false) (t true) gap (gap / 8) (1 / 128) (1 / 128) (1 / 64)
(by linarith) (le_refl _)
(affineRawTaggedPrior_target_tail n m d eps hd sigma hn heps heps' hS hx false)
(affineRawTaggedPrior_target_tail n m d eps hd sigma hn heps heps' hS hx true) htv
apply le_trans (ENNReal.ofReal_le_ofReal ?_) hlower
nlinarith [sq_nonneg gap]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.RawMass 10 declarations Nonnegative affine raw tables and their exact total mass before normalization.
Nonnegative affine raw tables and their exact total mass before normalization.
Under the stated inputs and conditions, Reciprocal tilting gives nonnegative core probabilities whose sum is at most one. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_latent_core_bounds
lemma affine_latent_core_bounds {L : Nat} {B eps : Real} (sigma : ConeDual L B eps)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : eps < 1) :
(∀ i, 0 ≤ latentCoreMass sigma i) ∧ (∑ i, latentCoreMass sigma i) ≤ 1 := by
have hden i : 0 < B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i :=
add_pos_of_pos_of_nonneg ha (mul_nonneg (by linarith) (sigma.mem i).1)
constructor
· intro i
exact mul_nonneg (div_nonneg ha.le (hden i).le) (abs_nonneg _)
· calc
(∑ i, latentCoreMass sigma i) ≤ ∑ i, |sigma.weights i| := by
apply Finset.sum_le_sum
intro i _
apply mul_le_of_le_one_left (abs_nonneg _)
exact (div_le_one (hden i)).mpr
(le_add_of_nonneg_right (mul_nonneg (by linarith) (sigma.mem i).1))
_ = 1 := sigma.abs_sum
Under the stated inputs and conditions, The latent PMF has exactly the specified core and filler masses. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_latent_mass_readback
lemma affine_latent_mass_readback {L : Nat} {B eps : Real} (sigma : ConeDual L B eps)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : eps < 1) (z : Latent L) :
(latentPMF sigma z).toReal =
(match z with | none => 1 - ∑ i, latentCoreMass sigma i | some i => latentCoreMass sigma i) := by
obtain ⟨hn, hs⟩ := affine_latent_core_bounds sigma ha he
apply normalizedPMF_toReal
· intro z
cases z with
| none => exact sub_nonneg.mpr hs
| some i => exact hn i
· simp only [Fintype.sum_option]
ring
Under the stated inputs and conditions, The expected latent node is at most alpha divided by one minus the overlap floor. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_latent_node_mean_bound
lemma affine_latent_node_mean_bound {L : Nat} {B eps : Real} (sigma : ConeDual L B eps)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : eps < 1) :
(∑ z : Latent L, (latentPMF sigma z).toReal * latentNode sigma z) ≤
(B / (100 * (L : Real) ^ 2)) / (1 - eps) := by
simp_rw [affine_latent_mass_readback sigma ha he]
simp only [Fintype.sum_option, latentNode, mul_zero, zero_add]
calc
(∑ i, latentCoreMass sigma i * sigma.nodes i) ≤
∑ i, (B / (100 * (L : Real) ^ 2) / (1 - eps)) * |sigma.weights i| := by
apply Finset.sum_le_sum
intro i _
have hepos : 0 < 1 - eps := by linarith
have hd : 0 < B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i :=
add_pos_of_pos_of_nonneg ha (mul_nonneg hepos.le (sigma.mem i).1)
have hb : (B / (100 * (L : Real) ^ 2)) /
(B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i) * sigma.nodes i ≤
(B / (100 * (L : Real) ^ 2)) / (1 - eps) := by
rw [div_mul_eq_mul_div, div_le_div_iff₀ hd hepos]
nlinarith [sq_nonneg (B / (100 * (L : Real) ^ 2))]
simpa only [latentCoreMass, mul_assoc, mul_comm, mul_left_comm] using
mul_le_mul_of_nonneg_right hb (abs_nonneg (sigma.weights i))
_ = _ := by rw [← Finset.mul_sum, sigma.abs_sum, mul_one]
theorem affine_latent_node_second_moment_bound Lemma affine_latent_node_second_moment_bound in the paper ↗
Under the stated inputs and conditions, The latent second moment is bounded by the bandwidth times its first moment. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_latent_node_second_moment_bound
lemma affine_latent_node_second_moment_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (hB : 0 ≤ B)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (he : eps < 1) :
(∑ z : Latent L, (latentPMF sigma z).toReal * latentNode sigma z ^ 2) ≤
B * ((B / (100 * (L : Real) ^ 2)) / (1 - eps)) := by
calc
_ ≤ ∑ z : Latent L, (latentPMF sigma z).toReal *
(B * latentNode sigma z) := by
apply Finset.sum_le_sum
intro z _
apply mul_le_mul_of_nonneg_left _ ENNReal.toReal_nonneg
have hv : latentNode sigma z ∈ Icc 0 B := by
cases z with
| none => exact ⟨le_rfl, hB⟩
| some i => exact sigma.mem i
nlinarith [hv.1, hv.2]
_ = B * ∑ z : Latent L, (latentPMF sigma z).toReal * latentNode sigma z := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro z _
ring
_ ≤ _ := mul_le_mul_of_nonneg_left
(affine_latent_node_mean_bound sigma ha he) hB
Under the stated inputs and conditions, The reservoir has mass greater than one half uniformly in all legal public indices. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
Proof (Lean source)
-- @node: affine_reservoir_pos
lemma affine_reservoir_pos (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
1 / 2 < 1 - ((affineTuning n m d eps).Kstar : Real) * latentMeanMass sigma := by
obtain ⟨_, _, _, _, _, ha, hb, hK⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have he : eps < 1 := by linarith
have hm := affine_latent_node_mean_bound sigma ha he
change (∑ z, (latentPMF sigma z).toReal * latentNode sigma z) ≤
(affineTuning n m d eps).alpha / (1 - eps) at hm
have hprob : ∑ z : Latent (affineTuning n m d eps).L, (latentPMF sigma z).toReal = 1 :=
by
simp_rw [affine_latent_mass_readback sigma ha he]
simp only [Fintype.sum_option]
ring
have hmean : latentMeanMass sigma ≤ (4 / 3) * (affineTuning n m d eps).b0 := by
dsimp only [latentMeanMass]
simp_rw [mul_add]
rw [Finset.sum_add_distrib, ← Finset.sum_mul, hprob, one_mul]
have hfrac : (affineTuning n m d eps).alpha / (1 - eps) ≤
(1 / 3) * (affineTuning n m d eps).b0 := by
apply (div_le_iff₀ (by linarith : 0 < 1 - eps)).mpr
have hid : (affineTuning n m d eps).alpha = eps * (affineTuning n m d eps).b0 := by
dsimp only [affineTuning]; field_simp
nlinarith
change (affineTuning n m d eps).b0 + _ ≤ _
linarith
have hprod := mul_le_mul_of_nonneg_left hmean
(Nat.cast_nonneg (affineTuning n m d eps).Kstar)
nlinarith
Under the stated inputs and conditions, Extending a finite cell table by zero preserves its total mass. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_sum_prefix
lemma affine_sum_prefix (d K : Nat) (hK : K ≤ d) (f : Fin K → Real) :
(∑ j : Fin d, if h : j.val < K then f ⟨j.val, h⟩ else 0) = ∑ j, f j := by
obtain ⟨b, rfl⟩ := Nat.exists_eq_add_of_le hK
rw [Fin.sum_trunc _ (by intro j; simp [Fin.val_natAdd])]
simp
Under the stated inputs and conditions, Every raw affine atom is nonnegative, including the reservoir and null cells. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
z :
Obs d
conclusion
0 ≤ rawTable hyp n m d eps sigma lat z
Proof (Lean source)
-- @node: affine_rawTable_nonneg
lemma affine_rawTable_nonneg (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(z : Obs d) : 0 ≤ rawTable hyp n m d eps sigma lat z := by
obtain ⟨_, _, _, _, _, ha, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hp := affine_reservoir_pos n m d eps hn hd heps heps' sigma
have hv (j) : 0 ≤ latentNode sigma (lat j) := by
cases lat j with
| none => exact le_refl 0
| some i => exact (sigma.mem i).1
dsimp only [rawTable]
by_cases hj : z.1.val < (affineTuning n m d eps).Kstar
· rw [dif_pos hj]
apply mul_nonneg
· cases ha' : z.2.1 <;> simp only [ha', Bool.false_eq_true, if_false, if_true]
· nlinarith [hv ⟨z.1.val, hj⟩]
· nlinarith [hv ⟨z.1.val, hj⟩]
· cases ha' : z.2.1
· cases z.2.2 <;> simp [ha', bernoulliMass]
· cases hl : lat ⟨z.1.val, hj⟩ with
| none => cases z.2.2 <;> simp [ha', hl, bernoulliMass]
| some i =>
cases hyp <;> cases z.2.2 <;>
simp [ha', hl, latentPolar, bernoulliMass] <;> split_ifs <;> norm_num
· rw [dif_neg hj]
by_cases hr : z.1.val = (affineTuning n m d eps).Kstar
· rw [if_pos hr]
cases z.2.2 <;> simp [bernoulliMass] <;> linarith
· rw [if_neg hr]
Under the stated inputs and conditions, Summing the raw table gives the shared reservoir-plus-cells normalizer. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hd :
2 ≤ d
heps :
0 < eps
sigma :
hyp :
lat :
conclusion
(∑ z : Obs d, rawTable hyp n m d eps sigma lat z) = rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_rawTable_sum
lemma affine_rawTable_sum (n m d : Nat) (eps : Real) (hd : 2 ≤ d) (heps : 0 < eps)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
(∑ z : Obs d, rawTable hyp n m d eps sigma lat z) = rawNormalizer n m d eps sigma lat := by
let K := (affineTuning n m d eps).Kstar
have hKd : K < d := by
have h := Nat.min_le_left (d - 1) (floor (1 / (100 * (affineTuning n m d eps).b0)))
change K ≤ d - 1 at h
omega
have hab : (affineTuning n m d eps).alpha = eps * (affineTuning n m d eps).b0 := by
dsimp only [affineTuning]
field_simp
simp only [Fintype.sum_prod_type]
have hcell (j : Fin d) :
(∑ a : Bool, ∑ y : Bool, rawTable hyp n m d eps sigma lat (j, a, y)) =
(if h : j.val < K then (affineTuning n m d eps).b0 + latentNode sigma (lat ⟨j.val, h⟩)
else 0) + (if j.val = K then 1 - (K : Real) * latentMeanMass sigma else 0) := by
by_cases hj : j.val < K
· have hr : j.val ≠ K := by omega
simp [rawTable, K, hj, hr, Fintype.sum_bool, bernoulliMass]
nlinarith [hab]
· by_cases hr : j.val = K <;>
simp [rawTable, K, hj, hr, Fintype.sum_bool, bernoulliMass] <;> ring
simp_rw [hcell]
rw [Finset.sum_add_distrib, affine_sum_prefix d K hKd.le
(fun j => (affineTuning n m d eps).b0 + latentNode sigma (lat j))]
have hres : (∑ j : Fin d, if j.val = K then 1 - (K : Real) * latentMeanMass sigma else 0) =
1 - (K : Real) * latentMeanMass sigma := by
have heq (j : Fin d) : j.val = K ↔ j = ⟨K, hKd⟩ := by
constructor
· intro h; exact Fin.ext h
· intro h; subst j; rfl
simp_rw [heq]
simp
rw [hres]
exact add_comm _ _
Under the stated inputs and conditions, Positive rare-cell masses preserve the reservoir lower bound on the normalizer. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
lat :
conclusion
1 / 2 < rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_rawNormalizer_pos
lemma affine_rawNormalizer_pos (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
1 / 2 < rawNormalizer n m d eps sigma lat := by
have hp := affine_reservoir_pos n m d eps hn hd heps heps' sigma
obtain ⟨_, _, _, _, _, _, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hs : 0 ≤ ∑ j, ((affineTuning n m d eps).b0 + latentNode sigma (lat j)) := by
apply Finset.sum_nonneg
intro j _
apply add_nonneg hb.le
cases lat j with
| none => exact le_refl 0
| some i => exact (sigma.mem i).1
dsimp only [rawNormalizer]
linarith
Under the stated inputs and conditions, Normalizing any finite nonnegative table divides each atom by its positive total. This gives the stated result.
Formal statement
hypotheses
alpha :
Type
Fintype alpha
mass :
alpha → Real
fallback :
PMF alpha
hn :
∀ z, 0 ≤ mass z
hs :
0 < ∑ z, mass z
z :
alpha
conclusion
(normalizedPMF mass fallback z).toReal = mass z / (∑ z, mass z)
Proof (Lean source)
-- @node: affine_normalizedPMF_toReal
lemma affine_normalizedPMF_toReal {alpha : Type} [Fintype alpha]
(mass : alpha → Real) (fallback : PMF alpha) (hn : ∀ z, 0 ≤ mass z)
(hs : 0 < ∑ z, mass z) (z : alpha) :
(normalizedPMF mass fallback z).toReal = mass z / (∑ z, mass z) := by
have ht : (∑' z, ofReal (mass z)) = ofReal (∑ z, mass z) := by
rw [tsum_fintype, ENNReal.ofReal_sum_of_nonneg (fun z _ => hn z)]
have hne : ofReal (∑ z, mass z) ≠ 0 := by
exact ne_of_gt (ENNReal.ofReal_pos.mpr hs)
have hok : (∑' z, ofReal (mass z)) ≠ 0 ∧
(∑' z, ofReal (mass z)) ≠ ⊤ := by
rw [ht]
exact ⟨hne, ENNReal.ofReal_ne_top⟩
dsimp only [normalizedPMF]
rw [dif_pos hok]
simp only [PMF.normalize_apply, ht,
ENNReal.toReal_mul, ENNReal.toReal_inv,
ENNReal.toReal_ofReal (hn z), ENNReal.toReal_ofReal hs.le, div_eq_mul_inv]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.RawPoissonTV 11 declarations Moment cancellation and exact Taylor-remainder readback for the affine one-cell marked-Poisson kernel, together with its calibrated exponential tail bound.
Moment cancellation and exact Taylor-remainder readback for the affine one-cell marked-Poisson kernel, together with its calibrated exponential tail bound.
Under the stated inputs and conditions, The Taylor remainder at 2MB=L/500 is geometrically small in the degree.
Formal statement
Proof (Lean source)
-- @node: affine_poisson_taylor_tail
lemma affine_poisson_taylor_tail (L : Nat) (hL : 2 ≤ L) :
(∑' h : Nat, if L < h then ((L : Real) / 500) ^ h / (h.factorial : Real) else 0) ≤
2 * exp (-5 * (L : Real)) := by
have he6 : exp (6 : Real) ≤ 500 := by
have he : exp 1 ≤ (11 / 4 : Real) := by linarith [Real.exp_one_lt_d9]
have hp := pow_le_pow_left₀ (Real.exp_pos 1).le he 6
rw [← Real.exp_nat_mul] at hp
norm_num at hp ⊢
linarith
have hq : exp 1 / 500 ≤ exp (-5) := by
apply (div_le_iff₀ (by norm_num : (0 : Real) < 500)).2
have h := mul_le_mul_of_nonneg_left he6 (Real.exp_pos (-5)).le
rw [← Real.exp_add] at h
norm_num at h
simpa [mul_comm] using h
have hterm (h : Nat) (hh : L < h) :
((L : Real) / 500) ^ h / (h.factorial : Real) ≤ exp (-5) ^ h := by
have hLh : (L : Real) ≤ h := by exact_mod_cast hh.le
have hbound := Real.pow_div_factorial_le_exp (h : Real) (Nat.cast_nonneg h) h
calc
((L : Real) / 500) ^ h / (h.factorial : Real) ≤
((h : Real) / 500) ^ h / (h.factorial : Real) := by gcongr
_ = (1 / 500 : Real) ^ h * ((h : Real) ^ h / (h.factorial : Real)) := by
simp only [div_pow]; norm_num; ring
_ ≤ (1 / 500 : Real) ^ h * exp (h : Real) :=
mul_le_mul_of_nonneg_left hbound (by positivity)
_ = (exp 1 / 500) ^ h := by
rw [show (h : Real) = (h : Real) * 1 by ring, Real.exp_nat_mul]
simp only [div_pow, one_pow]; ring
_ ≤ exp (-5) ^ h := pow_le_pow_left₀ (by positivity) hq h
have hqhalf : exp (-5) ≤ (1 / 2 : Real) := by
rw [Real.exp_neg, ← one_div]
apply (div_le_iff₀ (Real.exp_pos 5)).2
have h := Real.exp_le_exp.mpr (show (1 : Real) ≤ 5 by norm_num)
linarith [Real.exp_one_gt_two]
have hqs : Summable (fun h : Nat => exp (-5) ^ h) :=
summable_geometric_of_norm_lt_one (by
rw [Real.norm_eq_abs, abs_of_pos (Real.exp_pos _)]; linarith)
have hgs : Summable (fun h : Nat => if L < h then exp (-5) ^ h else 0) := by
apply hqs.of_nonneg_of_le
· intro h; positivity
· intro h; split_ifs
· rfl
· positivity
have hfs : Summable (fun h : Nat =>
if L < h then ((L : Real) / 500) ^ h / (h.factorial : Real) else 0) := by
apply hgs.of_nonneg_of_le
· intro h; positivity
· intro h; split_ifs with hh
· exact hterm h hh
· rfl
have hzero : (∑ h ∈ range (L + 1),
if L < h then exp (-5) ^ h else 0) = 0 := by
apply Finset.sum_eq_zero
intro h hh
rw [if_neg (by have := Finset.mem_range.mp hh; omega)]
calc
_ ≤ ∑' h : Nat, if L < h then exp (-5) ^ h else 0 := by
apply Summable.tsum_le_tsum _ hfs hgs
intro h; split_ifs with hh
· exact hterm h hh
· rfl
_ = exp (-5) ^ (L + 1) * (1 - exp (-5))⁻¹ := by
rw [← hgs.sum_add_tsum_nat_add (L + 1), hzero, zero_add]
simp only [show ∀ h : Nat, L < h + (L + 1) by omega, if_true, pow_add]
rw [tsum_mul_right, tsum_geometric_of_norm_lt_one (by
rw [Real.norm_eq_abs, abs_of_pos (Real.exp_pos _)]; linarith)]
ring
_ ≤ 2 * exp (-5) ^ L := by
have hi : (1 - exp (-5))⁻¹ ≤ (2 : Real) := by
rw [← one_div]
apply (div_le_iff₀ (by linarith : 0 < 1 - exp (-5))).2
linarith
rw [pow_succ]
have hp : 0 ≤ exp (-5) ^ L := by positivity
nlinarith [mul_le_mul_of_nonneg_left hi
(show 0 ≤ exp (-5) ^ L * exp (-5) by positivity)]
_ = 2 * exp (-5 * (L : Real)) := by rw [← Real.exp_nat_mul]; congr 2; ring
Under the stated inputs and conditions, Every polynomial through the matching degree cancels against the cone dual. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ i, sigma.weights i * p.eval (sigma.nodes i)) = 0
Proof (Lean source)
-- @node: ConeDual.sum_polynomial_eq_zero
lemma ConeDual.sum_polynomial_eq_zero {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (p : Polynomial Real) (hp : p.natDegree ≤ L) :
(∑ i, sigma.weights i * p.eval (sigma.nodes i)) = 0 := by
simp_rw [Polynomial.eval_eq_sum_range, Finset.mul_sum]
rw [Finset.sum_comm]
apply Finset.sum_eq_zero
intro k hk
have hkL : k ≤ L := by have := Finset.mem_range.mp hk; omega
calc
(∑ i, sigma.weights i * (p.coeff k * sigma.nodes i ^ k)) =
p.coeff k * ∑ i, sigma.weights i * sigma.nodes i ^ k := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
ring
_ = 0 := by rw [sigma.moments k hkL, mul_zero]
Under the stated inputs and conditions, The low-degree terms of the exponential times the two affine arm powers cancel. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_signed_mixed_moment_zero
lemma affine_signed_mixed_moment_zero {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (alpha b0 : Real) (k r c : Nat)
(hdegree : k + r + c ≤ L) :
(∑ i, sigma.weights i * (sigma.nodes i ^ k *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)) = 0 := by
let p : Polynomial Real := X ^ k *
(C alpha + C (1 - eps) * X) ^ r *
(C ((1 - eps) * b0) + C eps * X) ^ c
have hlin (a b : Real) :
(C a + C b * X).natDegree ≤ 1 := by
apply Polynomial.natDegree_add_le_of_degree_le
· simp
· exact (Polynomial.natDegree_C_mul_le _ _).trans Polynomial.natDegree_X_le
have hp : p.natDegree ≤ L := by
apply le_trans (Polynomial.natDegree_mul_le)
apply le_trans (Nat.add_le_add (Polynomial.natDegree_mul_le) le_rfl)
have hk : (X ^ k : Polynomial Real).natDegree ≤ k := by
simp
have hr : ((C alpha + C (1 - eps) * X) ^ r).natDegree ≤ r :=
Polynomial.natDegree_pow_le.trans (by
simpa using Nat.mul_le_mul_left r (hlin alpha (1 - eps)))
have hc :
((C ((1 - eps) * b0) + C eps * X) ^ c).natDegree ≤ c :=
Polynomial.natDegree_pow_le.trans (by
simpa using Nat.mul_le_mul_left c (hlin ((1 - eps) * b0) eps))
exact (Nat.add_le_add (Nat.add_le_add hk hr) hc).trans hdegree
simpa [p] using sigma.sum_polynomial_eq_zero p hp
theorem affine_signed_polynomial_remainder_bound Lemma affine_signed_polynomial_remainder_bound in the paper ↗
Under the stated inputs and conditions, Subtracting any canceled polynomial bounds the signed sum by its uniform remainder, using the dual's variation mass one. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_signed_polynomial_remainder_bound
lemma affine_signed_polynomial_remainder_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (f : Real → Real) (p : Polynomial Real)
(hp : p.natDegree ≤ L) (R : Real)
(hR : ∀ i, |f (sigma.nodes i) - p.eval (sigma.nodes i)| ≤ R) :
|∑ i, sigma.weights i * f (sigma.nodes i)| ≤ R := by
have hcancel := sigma.sum_polynomial_eq_zero p hp
calc
|∑ i, sigma.weights i * f (sigma.nodes i)| =
|∑ i, sigma.weights i * (f (sigma.nodes i) - p.eval (sigma.nodes i))| := by
simp_rw [mul_sub]
rw [Finset.sum_sub_distrib, hcancel, sub_zero]
_ ≤ ∑ i, |sigma.weights i * (f (sigma.nodes i) - p.eval (sigma.nodes i))| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ i, |sigma.weights i| * R := by
apply Finset.sum_le_sum
intro i _
rw [abs_mul]
exact mul_le_mul_of_nonneg_left (hR i) (abs_nonneg _)
_ = R := by rw [← Finset.sum_mul, sigma.abs_sum, one_mul]
Under the stated inputs and conditions, Every retained Taylor term of the one-cell kernel has zero signed integral. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_signed_exp_truncation_zero
lemma affine_signed_exp_truncation_zero {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (alpha b0 M : Real) (r c : Nat)
(hdegree : r + c ≤ L) :
(∑ i, sigma.weights i *
((∑ k ∈ range (L - (r + c) + 1),
(-M) ^ k / (k.factorial : Real) * sigma.nodes i ^ k) *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)) = 0 := by
simp_rw [Finset.sum_mul, Finset.mul_sum]
rw [Finset.sum_comm]
apply Finset.sum_eq_zero
intro k hk
have hkL : k + r + c ≤ L := by
have := Finset.mem_range.mp hk
omega
calc
(∑ i, sigma.weights i *
((-M) ^ k / (k.factorial : Real) * sigma.nodes i ^ k *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)) =
((-M) ^ k / (k.factorial : Real)) *
∑ i, sigma.weights i * (sigma.nodes i ^ k *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
ring
_ = 0 := by rw [affine_signed_mixed_moment_zero sigma alpha b0 k r c hkL, mul_zero]
theorem affine_signed_exp_remainder_identity Lemma affine_signed_exp_remainder_identity in the paper ↗
Under the stated inputs and conditions, The signed one-cell exponential kernel equals its surviving Taylor remainder exactly. This removes all terms of total degree at most L before bounding absolute coefficients. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * (alpha + (1 - eps) * sigma.nodes i) ^ r * ((1 - eps) * b0 + eps * sigma.nodes i) ^ c))
Proof (Lean source)
-- @node: affine_signed_exp_remainder_identity
lemma affine_signed_exp_remainder_identity {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (alpha b0 M : Real) (r c : Nat)
(hdegree : r + c ≤ L) :
(∑ i, sigma.weights i *
(exp (-M * sigma.nodes i) *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)) =
∑ i, sigma.weights i *
((exp (-M * sigma.nodes i) -
∑ k ∈ range (L - (r + c) + 1),
(-M) ^ k / (k.factorial : Real) * sigma.nodes i ^ k) *
(alpha + (1 - eps) * sigma.nodes i) ^ r *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c) := by
have hc := affine_signed_exp_truncation_zero sigma alpha b0 M r c hdegree
simp_rw [sub_mul, mul_sub] at hc ⊢
rw [Finset.sum_sub_distrib, hc, sub_zero]
Under the stated inputs and conditions, The absolute exponential remainder on the cone is bounded by its positive Taylor tail at the support endpoint; no bound on the common baseline is used. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_exp_remainder_abs_le
lemma affine_exp_remainder_abs_le (M B v : Real) (s : Nat)
(hM : 0 ≤ M) (hv : 0 ≤ v) (hvB : v ≤ B) :
|exp (-M * v) - ∑ k ∈ range s,
(-M) ^ k / (k.factorial : Real) * v ^ k| ≤
∑' k : Nat, (M * B) ^ (k + s) / ((k + s).factorial : Real) := by
have hB : 0 ≤ B := hv.trans hvB
have heq : exp (-M * v) = ∑' k : Nat,
(-M * v) ^ k / (k.factorial : Real) := by
rw [Real.exp_eq_exp_ℝ, NormedSpace.exp_eq_tsum_div]
have hsplit := (Real.summable_pow_div_factorial (-M * v)).sum_add_tsum_nat_add s
have hfin : (∑ k ∈ range s, (-M * v) ^ k / (k.factorial : Real)) =
∑ k ∈ range s, (-M) ^ k / (k.factorial : Real) * v ^ k := by
apply Finset.sum_congr rfl
intro k _
rw [mul_pow]
ring
rw [← heq, hfin] at hsplit
rw [show exp (-M * v) - ∑ k ∈ range s,
(-M) ^ k / (k.factorial : Real) * v ^ k =
∑' k : Nat, (-M * v) ^ (k + s) / ((k + s).factorial : Real) by linarith [hsplit]]
have hnorm (k : Nat) :
‖(-M * v) ^ (k + s) / ((k + s).factorial : Real)‖ =
(M * v) ^ (k + s) / ((k + s).factorial : Real) := by
simp [Real.norm_eq_abs, abs_div, abs_pow, abs_mul, abs_of_nonneg hM,
abs_of_nonneg hv]
have hsum := (Real.summable_pow_div_factorial (M * v)).comp_injective
(show Injective (fun k : Nat => k + s) from fun _ _ h => Nat.add_right_cancel h)
have hsumB := (Real.summable_pow_div_factorial (M * B)).comp_injective
(show Injective (fun k : Nat => k + s) from fun _ _ h => Nat.add_right_cancel h)
change Summable (fun k : Nat => (M * v) ^ (k + s) / ((k + s).factorial : Real)) at hsum
change Summable (fun k : Nat => (M * B) ^ (k + s) / ((k + s).factorial : Real)) at hsumB
have hsumNorm : Summable (fun k : Nat =>
‖(-M * v) ^ (k + s) / ((k + s).factorial : Real)‖) := by
simpa only [hnorm] using hsum
calc
_ ≤ ∑' k : Nat, ‖(-M * v) ^ (k + s) / ((k + s).factorial : Real)‖ := by
simpa only [Real.norm_eq_abs] using norm_tsum_le_tsum_norm hsumNorm
_ = ∑' k : Nat, (M * v) ^ (k + s) / ((k + s).factorial : Real) := tsum_congr hnorm
_ ≤ _ := by
apply Summable.tsum_le_tsum _ hsum hsumB
intro k
exact div_le_div_of_nonneg_right
(pow_le_pow_left₀ (mul_nonneg hM hv) (mul_le_mul_of_nonneg_left hvB hM) _)
(Nat.cast_nonneg _)
theorem affine_signed_monomial_exp_truncation_zero Lemma affine_signed_monomial_exp_truncation_zero in the paper ↗
Under the stated inputs and conditions, Matching moments cancel the monomial times every retained exponential term. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_signed_monomial_exp_truncation_zero
lemma affine_signed_monomial_exp_truncation_zero {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (M : Real) (j : Nat) (hj : j ≤ L) :
(∑ i, sigma.weights i *
((∑ k ∈ range (L - j + 1),
(-M) ^ k / (k.factorial : Real) * sigma.nodes i ^ k) * sigma.nodes i ^ j)) = 0 := by
simp_rw [Finset.sum_mul, Finset.mul_sum]
rw [Finset.sum_comm]
apply Finset.sum_eq_zero
intro k hk
have hkj : k + j ≤ L := by have := Finset.mem_range.mp hk; omega
calc
_ = ((-M) ^ k / (k.factorial : Real)) *
∑ i, sigma.weights i * sigma.nodes i ^ (k + j) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
rw [pow_add]
ring
_ = 0 := by rw [sigma.moments (k + j) hkj, mul_zero]
Under the stated inputs and conditions, A signed exponential monomial retains only terms above the matching total degree. The variation mass one bounds the remainder by its positive endpoint tail. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_signed_exp_monomial_bound
lemma affine_signed_exp_monomial_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (M : Real) (hM : 0 ≤ M)
(j : Nat) (hj : j ≤ L) :
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * sigma.nodes i ^ j)| ≤
B ^ j * ∑' k : Nat,
(M * B) ^ (k + (L - j + 1)) / ((k + (L - j + 1)).factorial : Real) := by
let R := ∑' k : Nat,
(M * B) ^ (k + (L - j + 1)) / ((k + (L - j + 1)).factorial : Real)
have hB : 0 ≤ B := (sigma.mem 0).1.trans (sigma.mem 0).2
have hR : 0 ≤ R := tsum_nonneg (fun k => by positivity)
have hc := affine_signed_monomial_exp_truncation_zero sigma M j hj
have heq : (∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * sigma.nodes i ^ j)) =
∑ i, sigma.weights i *
((exp (-M * sigma.nodes i) - ∑ k ∈ range (L - j + 1),
(-M) ^ k / (k.factorial : Real) * sigma.nodes i ^ k) * sigma.nodes i ^ j) := by
simp_rw [sub_mul, mul_sub]
rw [Finset.sum_sub_distrib, hc, sub_zero]
rw [heq]
calc
_ ≤ ∑ i, |sigma.weights i *
((exp (-M * sigma.nodes i) - ∑ k ∈ range (L - j + 1),
(-M) ^ k / (k.factorial : Real) * sigma.nodes i ^ k) * sigma.nodes i ^ j)| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ i, |sigma.weights i| * (R * B ^ j) := by
apply Finset.sum_le_sum
intro i _
rw [abs_mul, abs_mul, abs_pow, abs_of_nonneg (sigma.mem i).1]
apply mul_le_mul_of_nonneg_left _ (abs_nonneg _)
exact mul_le_mul
(affine_exp_remainder_abs_le M B (sigma.nodes i) (L - j + 1)
hM (sigma.mem i).1 (sigma.mem i).2)
(pow_le_pow_left₀ (sigma.mem i).1 (sigma.mem i).2 j)
(pow_nonneg (sigma.mem i).1 j) hR
_ = _ := by rw [← Finset.sum_mul, sigma.abs_sum, one_mul]; ring
theorem affine_signed_exp_monomial_full_bound Lemma affine_signed_exp_monomial_full_bound in the paper ↗
Under the stated inputs and conditions, Without a matching-degree restriction, the full positive Taylor series still bounds the signed exponential monomial. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_signed_exp_monomial_full_bound
lemma affine_signed_exp_monomial_full_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (M : Real) (hM : 0 ≤ M) (j : Nat) :
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * sigma.nodes i ^ j)| ≤
B ^ j * ∑' k : Nat, (M * B) ^ k / (k.factorial : Real) := by
let R := ∑' k : Nat, (M * B) ^ k / (k.factorial : Real)
have hB : 0 ≤ B := (sigma.mem 0).1.trans (sigma.mem 0).2
have hR : 0 ≤ R := tsum_nonneg (fun k => by positivity)
calc
_ ≤ ∑ i, |sigma.weights i *
(exp (-M * sigma.nodes i) * sigma.nodes i ^ j)| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ i, |sigma.weights i| * (R * B ^ j) := by
apply Finset.sum_le_sum
intro i _
rw [abs_mul, abs_mul, abs_pow, abs_of_nonneg (sigma.mem i).1]
apply mul_le_mul_of_nonneg_left _ (abs_nonneg _)
apply mul_le_mul _ (pow_le_pow_left₀ (sigma.mem i).1 (sigma.mem i).2 j)
(pow_nonneg (sigma.mem i).1 j) hR
simpa [R] using affine_exp_remainder_abs_le M B (sigma.nodes i) 0
hM (sigma.mem i).1 (sigma.mem i).2
_ = _ := by rw [← Finset.sum_mul, sigma.abs_sum, one_mul]; ring
Under the stated inputs and conditions, Polynomial coefficients give a total-degree Taylor envelope for the signed kernel. This preserves each monomial's cancellation degree before the count sums are exchanged. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_signed_exp_polynomial_bound
lemma affine_signed_exp_polynomial_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (M : Real) (hM : 0 ≤ M)
(p : Polynomial Real) :
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * p.eval (sigma.nodes i))| ≤
∑ j ∈ range (p.natDegree + 1), |p.coeff j| * B ^ j *
∑' k : Nat, (M * B) ^ (k + (if j ≤ L then L - j + 1 else 0)) /
((k + (if j ≤ L then L - j + 1 else 0)).factorial : Real) := by
simp_rw [Polynomial.eval_eq_sum_range, Finset.mul_sum]
rw [Finset.sum_comm]
calc
_ = |∑ j ∈ range (p.natDegree + 1), p.coeff j *
∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * sigma.nodes i ^ j)| := by
congr 1
apply Finset.sum_congr rfl
intro j _
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
ring
_ ≤ ∑ j ∈ range (p.natDegree + 1), |p.coeff j *
∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * sigma.nodes i ^ j)| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ _ := by
apply Finset.sum_le_sum
intro j hj
rw [abs_mul, mul_assoc]
apply mul_le_mul_of_nonneg_left _ (abs_nonneg _)
by_cases hjL : j ≤ L
· simpa only [if_pos hjL] using affine_signed_exp_monomial_bound sigma M hM j hjL
· simpa only [if_neg hjL, Nat.add_zero] using affine_signed_exp_monomial_full_bound sigma M hM j
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Separation 7 declarations Signed reciprocal-certificate separation before reservoir normalization.
Signed reciprocal-certificate separation before reservoir normalization.
Under the stated inputs and conditions, The signed one-cell raw target gap is at least one twelfth of b0.
Formal statement
Proof (Lean source)
-- @node: affine_signed_target_gap
lemma affine_signed_target_gap (L : Nat) (B eps : Real) (hL : 2 ≤ L)
(hB : 0 < B) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) (sigma : ConeDual L B eps) :
let alpha := B / (100 * (L : Real) ^ 2)
let b0 := alpha / eps
b0 / 12 ≤ ∑ i, sigma.weights i *
((b0 + sigma.nodes i) * alpha / (alpha + (1 - eps) * sigma.nodes i)) := by
let alpha := B / (100 * (L : Real) ^ 2)
let b0 := alpha / eps
have hLR : (0 : Real) < L := by exact_mod_cast (show 0 < L by omega)
have ha : 0 < alpha := div_pos hB (by positivity)
have hb : 0 < b0 := div_pos ha heps
have he : 0 < 1 - eps := by linarith
have hab : alpha = eps * b0 := by dsimp [b0]; field_simp
have hzero : ∑ i, sigma.weights i = 0 := by
simpa using sigma.moments 0 (by omega)
have hid (i : Fin (L + 2)) :
(b0 + sigma.nodes i) * alpha / (alpha + (1 - eps) * sigma.nodes i) =
alpha / (1 - eps) + (b0 * (1 - 2 * eps) / (1 - eps)) *
(alpha / (alpha + (1 - eps) * sigma.nodes i)) := by
have hden : 0 < alpha + (1 - eps) * sigma.nodes i :=
add_pos_of_pos_of_nonneg ha (mul_nonneg he.le (sigma.mem i).1)
have hden' : alpha + sigma.nodes i * (1 - eps) ≠ 0 := by
simpa only [mul_comm] using hden.ne'
field_simp [ha.ne', he.ne', hden.ne', hden']
nlinarith [hab]
have hsum : (∑ i, sigma.weights i *
((b0 + sigma.nodes i) * alpha / (alpha + (1 - eps) * sigma.nodes i))) =
(b0 * (1 - 2 * eps) / (1 - eps)) *
∑ i, sigma.weights i * (alpha / (alpha + (1 - eps) * sigma.nodes i)) := by
simp_rw [hid, mul_add]
rw [Finset.sum_add_distrib, ← Finset.sum_mul, hzero, zero_mul, zero_add]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i hi
ring
have hcoef : b0 * (2 / 3) ≤ b0 * (1 - 2 * eps) / (1 - eps) := by
apply (le_div_iff₀ he).2
nlinarith
have hcpos : 0 ≤ b0 * (1 - 2 * eps) / (1 - eps) :=
div_nonneg (mul_nonneg hb.le (by linarith)) he.le
have hgap := mul_le_mul_of_nonneg_left sigma.gap hcpos
change b0 / 12 ≤ _
rw [hsum]
change (b0 * (1 - 2 * eps) / (1 - eps)) * (1 / 8) ≤ _ at hgap
linarith
A single rare cell's raw contribution to the target, including the zero filler.
Definition (Lean source)
parameters
def
given by
(B / (100 * (L : Real) ^ 2) / eps + latentNode sigma z) * (match z with | none => 0 | some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2)
Under the stated inputs and conditions, Multiplying the polar sign by the absolute weight restores the signed weight. This gives the stated result.
Formal statement
hypotheses
conclusion
|sigma.weights i| * latentPolar sigma (some i) = sigma.weights i
Proof (Lean source)
-- @node: affine_latent_polar_weight
lemma affine_latent_polar_weight {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (i : Fin (L + 2)) :
|sigma.weights i| * latentPolar sigma (some i) = sigma.weights i := by
dsimp only [latentPolar]
split_ifs with h
· rw [abs_of_nonneg h, mul_one]
· rw [abs_of_neg (lt_of_not_ge h)]
ring
theorem affine_cell_target_mean_difference
Under the stated inputs and conditions, The expected target difference is exactly the reciprocal signed functional. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: affine_cell_target_mean_difference
lemma affine_cell_target_mean_difference {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (ha : 0 < B / (100 * (L : Real) ^ 2))
(he : eps < 1) :
(∑ z : Latent L, (latentPMF sigma z).toReal * affineCellTarget sigma true z) -
(∑ z : Latent L, (latentPMF sigma z).toReal * affineCellTarget sigma false z) =
∑ i, sigma.weights i *
((B / (100 * (L : Real) ^ 2) / eps + sigma.nodes i) *
(B / (100 * (L : Real) ^ 2)) /
(B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)) := by
rw [← Finset.sum_sub_distrib]
simp_rw [affine_latent_mass_readback sigma ha he]
simp only [Fintype.sum_option, affineCellTarget, latentNode,
Bool.false_eq_true, if_false, if_true, mul_zero, sub_self, zero_add]
apply Finset.sum_congr rfl
intro i _
calc
_ = |sigma.weights i| * latentPolar sigma (some i) *
((B / (100 * (L : Real) ^ 2) / eps + sigma.nodes i) *
(B / (100 * (L : Real) ^ 2)) /
(B / (100 * (L : Real) ^ 2) + (1 - eps) * sigma.nodes i)) := by
dsimp only [latentCoreMass]
ring
_ = _ := by rw [affine_latent_polar_weight]
theorem affine_cell_target_mean_gap
Under the stated inputs and conditions, The two hypotheses separate the expected one-cell raw targets by b0/12. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_cell_target_mean_gap
lemma affine_cell_target_mean_gap (L : Nat) (B eps : Real) (hL : 2 ≤ L)
(hB : 0 < B) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual L B eps) :
(B / (100 * (L : Real) ^ 2) / eps) / 12 ≤
(∑ z : Latent L, (latentPMF sigma z).toReal * affineCellTarget sigma true z) -
(∑ z : Latent L, (latentPMF sigma z).toReal * affineCellTarget sigma false z) := by
have hLR : (0 : Real) < L := by exact_mod_cast (show 0 < L by omega)
rw [affine_cell_target_mean_difference sigma (by positivity) (by linarith)]
exact affine_signed_target_gap L B eps hL hB heps heps' sigma
theorem affine_cell_target_bounds
Under the stated inputs and conditions, Each raw cell target is between zero and the cell's total raw mass. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
ha :
0 < B / (100 * (L : Real) ^ 2)
heps :
0 < eps
hyp :
z :
Latent L
Proof (Lean source)
-- @node: affine_cell_target_bounds
lemma affine_cell_target_bounds {L : Nat} {B eps : Real} (sigma : ConeDual L B eps)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (heps : 0 < eps)
(hyp : Bool) (z : Latent L) :
0 ≤ affineCellTarget sigma hyp z ∧
affineCellTarget sigma hyp z ≤
B / (100 * (L : Real) ^ 2) / eps + latentNode sigma z := by
have hb : 0 ≤ B / (100 * (L : Real) ^ 2) / eps := (div_pos ha heps).le
cases z with
| none => simpa [affineCellTarget, latentNode] using hb
| some i =>
have hw : 0 ≤ B / (100 * (L : Real) ^ 2) / eps + sigma.nodes i :=
add_nonneg hb (sigma.mem i).1
cases hyp <;>
simp only [affineCellTarget, latentNode, latentPolar,
Bool.false_eq_true, if_false, if_true] <;>
split_ifs <;> constructor <;> nlinarith
theorem affine_cell_target_second_moment_bound
Under the stated inputs and conditions, The cell target second moment has the bound used in the prior concentration step. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_cell_target_second_moment_bound
lemma affine_cell_target_second_moment_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (hB : 0 ≤ B)
(ha : 0 < B / (100 * (L : Real) ^ 2)) (heps : 0 < eps) (he : eps < 1)
(hyp : Bool) :
(∑ z : Latent L, (latentPMF sigma z).toReal * affineCellTarget sigma hyp z ^ 2) ≤
2 * (B / (100 * (L : Real) ^ 2) / eps) ^ 2 +
2 * B * ((B / (100 * (L : Real) ^ 2)) / (1 - eps)) := by
let b := B / (100 * (L : Real) ^ 2) / eps
have hp : ∑ z : Latent L, (latentPMF sigma z).toReal = 1 := by
simp_rw [affine_latent_mass_readback sigma ha he]
simp only [Fintype.sum_option]
ring
calc
_ ≤ ∑ z : Latent L, (latentPMF sigma z).toReal *
(2 * b ^ 2 + 2 * latentNode sigma z ^ 2) := by
apply Finset.sum_le_sum
intro z _
apply mul_le_mul_of_nonneg_left _ ENNReal.toReal_nonneg
obtain ⟨h0, h1⟩ := affine_cell_target_bounds sigma ha heps hyp z
have hw : 0 ≤ b + latentNode sigma z := h0.trans h1
have hs := sq_le_sq₀ h0 hw |>.mpr h1
nlinarith [sq_nonneg (b - latentNode sigma z)]
_ = 2 * b ^ 2 + 2 * ∑ z : Latent L,
(latentPMF sigma z).toReal * latentNode sigma z ^ 2 := by
simp_rw [mul_add]
rw [Finset.sum_add_distrib, ← Finset.sum_mul, hp, one_mul, Finset.mul_sum]
congr 1
apply Finset.sum_congr rfl
intro z _
ring
_ ≤ _ := by
have hs := affine_latent_node_second_moment_bound sigma hB ha he
change 2 * b ^ 2 + _ ≤ 2 * b ^ 2 + _
linarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.SignedCountTail 6 declarations Absolute convergence and count-summed Taylor bounds for the signed rare-cell kernel.
Absolute convergence and count-summed Taylor bounds for the signed rare-cell kernel. These justify exchanging the count indices with the polynomial envelope.
theorem affine_coefficient_tail_majorant_summable Lemma affine_coefficient_tail_majorant_summable in the paper ↗
Under the stated hypotheses, The coefficient majorant remains summable after attaching the matching-degree tail. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_coefficient_tail_majorant_summable
lemma affine_coefficient_tail_majorant_summable (M B : Real) (L : Nat)
(hM : 0 ≤ M) (hB : 0 ≤ B) :
Summable (fun j : Nat => M ^ j / (j.factorial : Real) * B ^ j *
∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0) := by
have hs := (affine_filtered_exponential_product_summable (M * B)
(mul_nonneg hM hB) L).prod
apply hs.congr
intro j
rw [← tsum_mul_left]
apply tsum_congr
intro k
simp only [Nat.add_comm j k]
split_ifs <;> simp [mul_pow, mul_div_assoc, mul_assoc] <;> ring
theorem affine_signed_polynomial_family_tail_bound Lemma affine_signed_polynomial_family_tail_bound in the paper ↗
Under the stated inputs and conditions, A family of nonnegative polynomial coefficients with summable columns and an exponential column bound has an absolutely summable signed-kernel envelope. Moment cancellation is applied to each polynomial before the count exchange. This gives the stated result.
Formal statement
hypotheses
I :
Type*
L :
B eps :
sigma :
ConeDual L B eps
M :
hM :
0 ≤ M
p :
I → Polynomial Real
a :
I → Real
ha :
∀ i, 0 ≤ a i
hp :
∀ i j, 0 ≤ (p i).coeff j
hc :
∀ j, Summable (fun i => a i * (p i).coeff j)
hle :
∀ j, (∑' i, a i * (p i).coeff j) ≤ M ^ j / (j.factorial : Real)
conclusion
conclusion 1
Proof (Lean source)
-- @node: affine_signed_polynomial_family_tail_bound
lemma affine_signed_polynomial_family_tail_bound {I : Type*} {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (M : Real) (hM : 0 ≤ M)
(p : I → Polynomial Real) (a : I → Real) (ha : ∀ i, 0 ≤ a i)
(hp : ∀ i j, 0 ≤ (p i).coeff j)
(hc : ∀ j, Summable (fun i => a i * (p i).coeff j))
(hle : ∀ j, (∑' i, a i * (p i).coeff j) ≤ M ^ j / (j.factorial : Real)) :
Summable (fun i => a i *
|∑ t, sigma.weights t * (exp (-M * sigma.nodes t) * (p i).eval (sigma.nodes t))|) ∧
(∑' i, a i *
|∑ t, sigma.weights t * (exp (-M * sigma.nodes t) * (p i).eval (sigma.nodes t))|) ≤
∑' h : Nat, if L < h then (2 * M * B) ^ h / (h.factorial : Real) else 0 := by
have hB : 0 ≤ B := (sigma.mem 0).1.trans (sigma.mem 0).2
let T : Nat → Real := fun j => B ^ j *
∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0
have hT (j : Nat) : 0 ≤ T j := by
dsimp [T]; apply mul_nonneg (by positivity)
exact tsum_nonneg fun k => by split_ifs <;> positivity
let F : Nat × I → Real := fun v => (a v.2 * (p v.2).coeff v.1) * T v.1
have hn (v : Nat × I) : 0 ≤ F v :=
mul_nonneg (mul_nonneg (ha _) (hp _ _)) (hT _)
have hcols (j : Nat) : Summable (fun i => F (j, i)) := (hc j).mul_right (T j)
have hsum : Summable (fun j => ∑' i, F (j, i)) := by
apply (affine_coefficient_tail_majorant_summable M B L hM hB).of_nonneg_of_le
· intro j; exact tsum_nonneg fun i => hn (j, i)
· intro j
change (∑' i, (a i * (p i).coeff j) * T j) ≤ _
rw [tsum_mul_right]
simpa only [T, mul_assoc] using mul_le_mul_of_nonneg_right (hle j) (hT j)
have hF : Summable F := (summable_prod_of_nonneg hn).2 ⟨hcols, hsum⟩
let G : I → Real := fun i => a i *
|∑ t, sigma.weights t * (exp (-M * sigma.nodes t) * (p i).eval (sigma.nodes t))|
have hG (i : I) : 0 ≤ G i := mul_nonneg (ha i) (abs_nonneg _)
have hGF (i : I) : G i ≤ ∑' j, F (j, i) := by
have h := affine_signed_exp_polynomial_bound sigma M hM (p i)
simp_rw [abs_of_nonneg (hp i _),
affine_total_degree_tail_eq (M * B) (mul_nonneg hM hB)] at h
calc
G i ≤ a i * ∑ j ∈ range ((p i).natDegree + 1), (p i).coeff j * T j :=
mul_le_mul_of_nonneg_left (by simpa only [T, mul_assoc] using h) (ha i)
_ = ∑ j ∈ range ((p i).natDegree + 1), F (j, i) := by
rw [Finset.mul_sum]; apply Finset.sum_congr rfl; intro j _; dsimp [F]; ring
_ ≤ ∑' j, F (j, i) := (hF.prod_symm.prod_factor i).sum_le_tsum _
(fun j _ => hn (j, i))
have hGs : Summable G := hF.prod_symm.prod.of_nonneg_of_le hG hGF
refine ⟨hGs, ?_⟩
calc
(∑' i, G i) ≤ ∑' i, ∑' j, F (j, i) :=
hGs.tsum_le_tsum hGF hF.prod_symm.prod
_ = ∑' j, ∑' i, F (j, i) := Summable.tsum_comm (f := fun j i => F (j, i)) hF
_ = ∑' j, (∑' i, a i * (p i).coeff j) * B ^ j *
∑' k : Nat, if L < k + j then (M * B) ^ k / (k.factorial : Real) else 0 := by
simp only [F, tsum_mul_right]
apply tsum_congr
intro j
dsimp [T]
ring
_ ≤ _ := affine_dominated_coeff_total_degree_tail_le _ M B L hM hB
(fun j => tsum_nonneg fun i => mul_nonneg (ha i) (hp i j)) hle
theorem affine_three_count_coeff_series_summable Lemma affine_three_count_coeff_series_summable in the paper ↗
Under the stated hypotheses, The three original count indices are jointly absolutely summable at every fixed Taylor coefficient, including the Palm factorial shift. This gives the stated result.
Formal statement
hypotheses
b0 eps u w M :
j :
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
hM :
M = u + w
Proof (Lean source)
-- @node: affine_three_count_coeff_series_summable
lemma affine_three_count_coeff_series_summable (b0 eps u w M : Real) (j : Nat)
(hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (hM : M = u + w) :
Summable (fun v : Nat × (Nat × Nat) => M ^ v.1 / (v.1.factorial : Real) *
(u ^ v.2.1 / ((v.2.1 + 1).factorial : Real) *
(w ^ v.2.2 / (v.2.2.factorial : Real)) *
(((C (eps * b0) + C (1 - eps) * Polynomial.X) ^
(v.2.1 + v.2.2)) *
((C ((1 - eps) * b0) + C eps * Polynomial.X) ^ v.1)).coeff j)) := by
let p : Polynomial Real := C (eps * b0) + C (1 - eps) * Polynomial.X
let q : Polynomial Real := C ((1 - eps) * b0) + C eps * Polynomial.X
let f (c : Nat) (k : Nat × Nat) : Real := u ^ k.1 / ((k.1 + 1).factorial : Real) *
(w ^ k.2 / (k.2.factorial : Real)) * ((p ^ (k.1 + k.2)) * q ^ c).coeff j
let g (c : Nat) (k : Nat × Nat) : Real := u ^ k.1 / (k.1.factorial : Real) *
(w ^ k.2 / (k.2.factorial : Real)) * ((p ^ (k.1 + k.2)) * q ^ c).coeff j
have hM0 : 0 ≤ M := by rw [hM]; positivity
have hcoeff (c : Nat) (k : Nat × Nat) : 0 ≤ ((p ^ (k.1 + k.2)) * q ^ c).coeff j :=
affine_arm_power_coeff_nonneg (eps * b0) b0 eps (k.1 + k.2) c j
(mul_nonneg he hb) hb he he'
have hn (c : Nat) (k : Nat × Nat) : 0 ≤ f c k :=
mul_nonneg (by positivity) (hcoeff c k)
have hle (c : Nat) (k : Nat × Nat) : f c k ≤ g c k := by
have h := mul_le_mul_of_nonneg_right (affine_labeled_factorial_term_le u hu k.1)
(mul_nonneg (show 0 ≤ w ^ k.2 / (k.2.factorial : Real) by positivity) (hcoeff c k))
simpa only [f, g, mul_assoc] using h
have hs (c : Nat) : Summable (f c) :=
(affine_treated_count_series_summable (eps * b0) (1 - eps) u w (q ^ c) j).of_nonneg_of_le
(hn c) (hle c)
let G : Nat → Real := fun c => M ^ c / (c.factorial : Real) *
(∑' R : Nat, M ^ R / (R.factorial : Real) * ((p ^ R) * q ^ c).coeff j)
have hr := affine_two_arm_coeff_series_summable (eps * b0) (1 - eps)
((1 - eps) * b0) eps M M j
have hG : Summable G := by
apply hr.prod_symm.prod.congr
intro c
simp only [G, Prod.swap_prod_mk, ← tsum_mul_left]
apply tsum_congr
intro R
ring
change Summable (fun v : Nat × (Nat × Nat) =>
M ^ v.1 / (v.1.factorial : Real) * f v.1 v.2)
apply (summable_prod_of_nonneg (fun v =>
mul_nonneg (show 0 ≤ M ^ v.1 / (v.1.factorial : Real) by positivity) (hn v.1 v.2))).2
constructor
· intro c
exact (hs c).mul_left (M ^ c / (c.factorial : Real))
· apply hG.of_nonneg_of_le
· intro c
exact tsum_nonneg fun k => mul_nonneg (by positivity) (hn c k)
· intro c
change (∑' k, M ^ c / (c.factorial : Real) * f c k) ≤ G c
rw [tsum_mul_left, (hs c).tsum_prod' (hs c).prod_factor]
apply mul_le_mul_of_nonneg_left _ (by positivity)
have h := affine_labeled_treated_series_le (eps * b0) b0 eps u w c j
(mul_nonneg he hb) hb he he' hu hw
simpa only [← hM] using h
theorem affine_signed_three_count_tail_bound Lemma affine_signed_three_count_tail_bound in the paper ↗
Under the stated inputs and conditions, The absolute signed equation (6) kernels, summed over all three counts, have precisely the equation (7) tail bound. The common baseline cancels before any bound is taken, and the convergence claim justifies the exchange. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
b0 u w M :
hb :
0 ≤ b0
he :
0 ≤ eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
hM :
M = u + w
conclusion
F :
fun v
=> exp (-M * b0) * (M ^ v.1 / (v.1.factorial : Real)) * (u ^ v.2.1 / ((v.2.1 + 1).factorial : Real)) * (w ^ v.2.2 / (v.2.2.factorial : Real)) * |∑ i, sigma.weights i * (exp (-M * sigma.nodes i) * (eps * b0 + (1 - eps) * sigma.nodes i) ^ (v.2.1 + v.2.2) * ((1 - eps) * b0 + eps * sigma.nodes i) ^ v.1)| (: Nat × (Nat × Nat) → Real)
Proof (Lean source)
-- @node: affine_signed_three_count_tail_bound
lemma affine_signed_three_count_tail_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (b0 u w M : Real)
(hb : 0 ≤ b0) (he : 0 ≤ eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (hM : M = u + w) :
let F : Nat × (Nat × Nat) → Real := fun v =>
exp (-M * b0) * (M ^ v.1 / (v.1.factorial : Real)) *
(u ^ v.2.1 / ((v.2.1 + 1).factorial : Real)) *
(w ^ v.2.2 / (v.2.2.factorial : Real)) *
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) *
(eps * b0 + (1 - eps) * sigma.nodes i) ^ (v.2.1 + v.2.2) *
((1 - eps) * b0 + eps * sigma.nodes i) ^ v.1)|
Summable F ∧ (∑' v, F v) ≤
∑' h : Nat, if L < h then (2 * M * B) ^ h / (h.factorial : Real) else 0 := by
intro F
let p : Nat × (Nat × Nat) → Polynomial Real := fun v =>
(C (eps * b0) + C (1 - eps) * X) ^ (v.2.1 + v.2.2) *
(C ((1 - eps) * b0) + C eps * X) ^ v.1
let a : Nat × (Nat × Nat) → Real := fun v =>
exp (-M * b0) * (M ^ v.1 / (v.1.factorial : Real)) *
(u ^ v.2.1 / ((v.2.1 + 1).factorial : Real)) *
(w ^ v.2.2 / (v.2.2.factorial : Real))
have hM0 : 0 ≤ M := by rw [hM]; positivity
have hs (j : Nat) := affine_three_count_coeff_series_summable b0 eps u w M j hb he he' hu hw hM
have hc (j : Nat) : Summable (fun v => a v * (p v).coeff j) := by
apply ((hs j).mul_left (exp (-M * b0))).congr
intro v
dsimp [a, p]
ring
have hread (j : Nat) : (∑' v, a v * (p v).coeff j) =
exp (-M * b0) *
(∑' c : Nat, M ^ c / (c.factorial : Real) *
(∑' r : Nat, ∑' g : Nat, u ^ r / ((r + 1).factorial : Real) *
(w ^ g / (g.factorial : Real)) * (p (c, (r, g))).coeff j)) := by
rw [(hc j).tsum_prod' (hc j).prod_factor]
simp_rw [← tsum_mul_left]
apply tsum_congr
intro c
have hinner := (hc j).prod_factor c
rw [hinner.tsum_prod' hinner.prod_factor]
apply tsum_congr
intro r
apply tsum_congr
intro g
dsimp [a, p]
ring
have hle (j : Nat) : (∑' v, a v * (p v).coeff j) ≤ M ^ j / (j.factorial : Real) := by
rw [hread]
exact affine_three_count_coeff_baseline_le b0 eps u w M j hb he he' hu hw hM
have h := affine_signed_polynomial_family_tail_bound sigma M hM0 p a
(fun v => by dsimp [a]; positivity)
(fun v j => affine_arm_power_coeff_nonneg (eps * b0) b0 eps
(v.2.1 + v.2.2) v.1 j (mul_nonneg he hb) hb he he') hc hle
simpa only [F, a, p, Polynomial.eval_mul, Polynomial.eval_pow,
Polynomial.eval_add, Polynomial.eval_C, Polynomial.eval_X, mul_assoc] using h
theorem affine_signed_raw_count_factorization Lemma affine_signed_raw_count_factorization in the paper ↗
Under the stated inputs and conditions, Factoring the original signed count difference in equation (6) gives the Palm intensity times the kernel controlled by the count-summed Taylor bound. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ i, sigma.weights i * (exp (-M * (b0 + sigma.nodes i)) * (u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (r + 1) * (w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ g * (M * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ c / (((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) * (eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i))))
Proof (Lean source)
-- @node: affine_signed_raw_count_factorization
lemma affine_signed_raw_count_factorization {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (b0 u w M : Real) (r g c : Nat)
(hb : 0 < b0) (he : 0 < eps) (he' : eps ≤ 1) :
(∑ i, sigma.weights i * (exp (-M * (b0 + sigma.nodes i)) *
(u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (r + 1) *
(w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ g *
(M * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ c /
(((r + 1).factorial : Real) * (g.factorial : Real) * (c.factorial : Real)) *
(eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i)))) =
u * (eps * b0) * exp (-M * b0) *
(M ^ c / (c.factorial : Real)) * (u ^ r / ((r + 1).factorial : Real)) *
(w ^ g / (g.factorial : Real)) *
(∑ i, sigma.weights i * (exp (-M * sigma.nodes i) *
(eps * b0 + (1 - eps) * sigma.nodes i) ^ (r + g) *
((1 - eps) * b0 + eps * sigma.nodes i) ^ c)) := by
simp only [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
have hS : eps * b0 + (1 - eps) * sigma.nodes i ≠ 0 := by
apply ne_of_gt
exact add_pos_of_pos_of_nonneg (mul_pos he hb)
(mul_nonneg (sub_nonneg.mpr he') (sigma.mem i).1)
rw [show -M * (b0 + sigma.nodes i) = -M * b0 + -M * sigma.nodes i by ring,
Real.exp_add]
generalize hsdef : eps * b0 + (1 - eps) * sigma.nodes i = s at hS ⊢
simp only [mul_pow, pow_succ, pow_add]
field_simp [hS]
Under the stated inputs and conditions, The equation (6) signed masses are absolutely summable and their absolute sum obeys the equation (7) one-cell bound, with both channels retained. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
sigma :
ConeDual L B eps
b0 u w M :
hb :
0 < b0
he :
0 < eps
he' :
eps ≤ 1
hu :
0 ≤ u
hw :
0 ≤ w
hM :
M = u + w
conclusion
D :
fun v
=> ∑ i, sigma.weights i * (exp (-M * (b0 + sigma.nodes i)) * (u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (v.2.1 + 1) * (w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ v.2.2 * (M * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ v.1 / (((v.2.1 + 1).factorial : Real) * (v.2.2.factorial : Real) * (v.1.factorial : Real)) * (eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i))) (: Nat × (Nat × Nat) → Real)
Proof (Lean source)
-- @node: affine_signed_raw_count_tail_bound
lemma affine_signed_raw_count_tail_bound {L : Nat} {B eps : Real}
(sigma : ConeDual L B eps) (b0 u w M : Real)
(hb : 0 < b0) (he : 0 < eps) (he' : eps ≤ 1)
(hu : 0 ≤ u) (hw : 0 ≤ w) (hM : M = u + w) :
let D : Nat × (Nat × Nat) → Real := fun v =>
∑ i, sigma.weights i * (exp (-M * (b0 + sigma.nodes i)) *
(u * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ (v.2.1 + 1) *
(w * (eps * b0 + (1 - eps) * sigma.nodes i)) ^ v.2.2 *
(M * ((1 - eps) * b0 + eps * sigma.nodes i)) ^ v.1 /
(((v.2.1 + 1).factorial : Real) * (v.2.2.factorial : Real) *
(v.1.factorial : Real)) * (eps * b0 / (eps * b0 + (1 - eps) * sigma.nodes i)))
Summable (fun v => |D v|) ∧ (∑' v, |D v|) ≤
u * (eps * b0) *
∑' h : Nat, if L < h then (2 * M * B) ^ h / (h.factorial : Real) else 0 := by
intro D
have hM0 : 0 ≤ M := by rw [hM]; positivity
let F : Nat × (Nat × Nat) → Real := fun v =>
exp (-M * b0) * (M ^ v.1 / (v.1.factorial : Real)) *
(u ^ v.2.1 / ((v.2.1 + 1).factorial : Real)) *
(w ^ v.2.2 / (v.2.2.factorial : Real)) *
|∑ i, sigma.weights i * (exp (-M * sigma.nodes i) *
(eps * b0 + (1 - eps) * sigma.nodes i) ^ (v.2.1 + v.2.2) *
((1 - eps) * b0 + eps * sigma.nodes i) ^ v.1)|
obtain ⟨hs, ht⟩ := affine_signed_three_count_tail_bound sigma b0 u w M
hb.le he.le he' hu hw hM
have hread (v : Nat × (Nat × Nat)) : |D v| = u * (eps * b0) * F v := by
dsimp only [D]
rw [affine_signed_raw_count_factorization sigma b0 u w M v.2.1 v.2.2 v.1 hb he he']
simp only [abs_mul, abs_of_nonneg hu, abs_of_nonneg he.le, abs_of_nonneg hb.le,
abs_of_pos (Real.exp_pos _), abs_of_nonneg (pow_nonneg hM0 _),
abs_of_nonneg (pow_nonneg hu _), abs_of_nonneg (pow_nonneg hw _),
abs_div, abs_of_nonneg (Nat.cast_nonneg (α := Real) _)]
dsimp only [F]
ring
simp_rw [hread]
constructor
· exact hs.mul_left (u * (eps * b0))
· rw [tsum_mul_left]
exact mul_le_mul_of_nonneg_left ht (mul_nonneg hu (mul_nonneg he.le hb.le))
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Testing 4 declarations Normalized affine-prior concentration and the two-prior testing step in equation (11).
Normalized affine-prior concentration and the two-prior testing step in equation (11). The experiment-distance estimate is a separate input, to be proved for the raw experiment.
theorem affine_ateFunctional_measurable
Under the stated inputs and conditions, The ATE is Borel measurable in the finite table of observable atom probabilities. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_ateFunctional_measurable
@[fun_prop]
lemma affine_ateFunctional_measurable (d : Nat) :
Measurable (ateFunctional (d := d)) := by
have hj (j : Fin d) (a y : Bool) :
Measurable (fun P : DiscreteLaw d => jointMass P j a y) := by
exact (measurable_pi_apply (j, a, y)).comp
(comap_measurable (fun P : DiscreteLaw d => fun o => (P.pmf o).toReal))
unfold ateFunctional cellMass outcomeMean markedMass armMass
fun_prop
theorem affine_productPrior_isProbabilityMeasure Lemma affine_productPrior_isProbabilityMeasure in the paper ↗
Under the stated inputs and conditions, Each normalized affine prior is a probability measure, including its zero filler. This gives the stated result.
Formal statement
hypotheses
hyp :
n m d :
eps :
hd :
2 ≤ d
sigma :
conclusion
IsProbabilityMeasure (affineProductPrior hyp n m d eps hd sigma)
Proof (Lean source)
-- @node: affine_productPrior_isProbabilityMeasure
lemma affine_productPrior_isProbabilityMeasure (hyp : Bool) (n m d : Nat)
(eps : Real) (hd : 2 ≤ d)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
IsProbabilityMeasure (affineProductPrior hyp n m d eps hd sigma) := by
unfold affineProductPrior
apply Measure.isProbabilityMeasure_map
fun_prop
theorem affine_productPrior_target_tail_small
Under the stated inputs and conditions, Pushing forward the latent tail bound gives concentration for the actual law prior. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
Proof (Lean source)
-- @node: affine_productPrior_target_tail_small
lemma affine_productPrior_target_tail_small (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let t := fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
let gap := t true - t false
(affineProductPrior hyp n m d eps hd sigma).real
{P | gap / 8 < |ateFunctional P - t hyp|} ≤ 1 / 128 := by
intro nu t gap
have hmap : Measurable (normalizedLaw hyp n m d eps hd sigma) := by fun_prop
have hset : MeasurableSet {P : DiscreteLaw d | gap / 8 < |ateFunctional P - t hyp|} := by
exact measurableSet_lt measurable_const
(continuous_abs.measurable.comp
((affine_ateFunctional_measurable d).sub measurable_const))
change ((nu.map (normalizedLaw hyp n m d eps hd sigma))
{P | gap / 8 < |ateFunctional P - t hyp|}).toReal ≤ _
rw [Measure.map_apply hmap hset]
have htail := affine_normalized_target_tail_small n m d eps hn hd heps heps' hS hx sigma hyp
change nu.real _ ≤ _ at htail
apply le_trans (measureReal_mono ?_) htail
intro lat hlat
change gap / 8 < |ateFunctional (normalizedLaw hyp n m d eps hd sigma lat) - t hyp| at hlat
exact hlat.le
theorem affine_two_prior_bayes_lower
Under the stated inputs and conditions, With the raw-mixture distance budget, every measurable rule has Bayes risk at least one sixty-fourth of the squared affine target gap. This is equation (11). This gives the stated result.
Formal statement
hypotheses
X :
Type*
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
K :
Kernel (DiscreteLaw d) X
hK :
∀ P, IsProbabilityMeasure (K P)
estimator :
X → Real
hestimator :
Measurable estimator
htv :
tvDist (priorPredictive (affineProductPrior false n m d eps hd sigma) K) (priorPredictive (affineProductPrior true n m d eps hd sigma) K)
≤ 1 / 64
conclusion
nu :
t :
fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
ofReal (gap ^ 2 / 64)
≤ max (bayesSquaredRisk (affineProductPrior false n m d eps hd sigma) K ateFunctional estimator) (bayesSquaredRisk (affineProductPrior true n m d eps hd sigma) K ateFunctional estimator)
Proof (Lean source)
-- @node: affine_two_prior_bayes_lower
lemma affine_two_prior_bayes_lower {X : Type*} [MeasurableSpace X]
(n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(K : Kernel (DiscreteLaw d) X) (hK : ∀ P, IsProbabilityMeasure (K P))
(estimator : X → Real) (hestimator : Measurable estimator)
(htv : tvDist
(priorPredictive (affineProductPrior false n m d eps hd sigma) K)
(priorPredictive (affineProductPrior true n m d eps hd sigma) K) ≤ 1 / 64) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let t := fun i => ∫ lat, affineRawTarget n m d eps sigma i lat ∂nu
let gap := t true - t false
ofReal (gap ^ 2 / 64) ≤
max (bayesSquaredRisk (affineProductPrior false n m d eps hd sigma) K ateFunctional estimator)
(bayesSquaredRisk (affineProductPrior true n m d eps hd sigma)
K ateFunctional estimator) := by
intro nu t gap
let := affine_productPrior_isProbabilityMeasure false n m d eps hd sigma
let := affine_productPrior_isProbabilityMeasure true n m d eps hd sigma
have hS1 : 1 ≤ (n : Real) * eps := by
linarith [Real.add_one_le_exp (4096 : Real)]
have hKpos : (0 : Real) < (affineTuning n m d eps).Kstar := by
exact_mod_cast affine_tuning_Kstar_pos n m d eps hn hd heps heps' hS1
have hb := (affine_tuning_bounds n m d eps hn hd heps heps').2.2.2.2.2.2.1
have hgapbound := affine_raw_target_mean_gap n m d eps hn hd heps heps' sigma
have hgap : 0 < gap := lt_of_lt_of_le (by positivity) hgapbound
have hlower := twoFuzzyHypotheses_bayesRisk_lower
(affineProductPrior false n m d eps hd sigma)
(affineProductPrior true n m d eps hd sigma) K ateFunctional hK
estimator hestimator (affine_ateFunctional_measurable d)
(t false) (t true) gap (gap / 8) (1 / 128) (1 / 128) (1 / 64)
(by linarith) (le_refl _)
(affine_productPrior_target_tail_small n m d eps hn hd heps heps' hS hx sigma false)
(affine_productPrior_target_tail_small n m d eps hn hd heps heps' hS hx sigma true) htv
apply le_trans (ENNReal.ofReal_le_ofReal ?_) hlower
nlinarith [sq_nonneg gap]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Transfer 8 declarations Raw-to-normalized likelihood and intensity readback for both channels, and the Poisson prefix-failure budgets for the random-scale two-channel transfer.
Raw-to-normalized likelihood and intensity readback for both channels, and the Poisson prefix-failure budgets for the random-scale two-channel transfer.
Under the stated inputs and conditions, Normalized complete-record atom masses read back to raw masses divided by the shared Q.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
z :
Obs d
conclusion
((normalizedLaw hyp n m d eps hd sigma lat).pmf z).toReal
= rawTable hyp n m d eps sigma lat z / rawNormalizer n m d eps sigma lat
Proof (Lean source)
-- @node: affine_normalized_mass_readback
lemma affine_normalized_mass_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(z : Obs d) :
((normalizedLaw hyp n m d eps hd sigma lat).pmf z).toReal =
rawTable hyp n m d eps sigma lat z / rawNormalizer n m d eps sigma lat := by
have hsum := affine_rawTable_sum n m d eps hd heps sigma hyp lat
have hpos : 0 < ∑ z : Obs d, rawTable hyp n m d eps sigma lat z := by
rw [hsum]
exact lt_trans (by norm_num) (affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat)
change (normalizedPMF (rawTable hyp n m d eps sigma lat) _ z).toReal = _
rw [affine_normalizedPMF_toReal _ _
(affine_rawTable_nonneg n m d eps hn hd heps heps' sigma hyp lat) hpos, hsum]
theorem affine_normalized_aux_mass_readback
Under the stated inputs and conditions, The auxiliary channel retains the full arm marginal of the raw table, divided by Q. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
z :
AuxObs d
conclusion
Proof (Lean source)
-- @node: affine_normalized_aux_mass_readback
lemma affine_normalized_aux_mass_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(z : AuxObs d) :
(auxMarginal (normalizedLaw hyp n m d eps hd sigma lat) z).toReal =
(∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y)) /
rawNormalizer n m d eps sigma lat := by
rw [auxMarginal_toReal_armMass]
unfold armMass jointMass
simp_rw [affine_normalized_mass_readback n m d eps hn hd heps heps' sigma hyp lat]
rw [Finset.sum_div]
Under the stated inputs and conditions, Multiplying the normalized complete-channel probabilities by the raw total recovers every complete Poisson intensity exactly. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
z :
Obs d
conclusion
(affineTuning n m d eps).u * rawNormalizer n m d eps sigma lat * ((normalizedLaw hyp n m d eps hd sigma lat).pmf z).toReal
= (affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat z
Proof (Lean source)
-- @node: affine_complete_intensity_readback
lemma affine_complete_intensity_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(z : Obs d) :
(affineTuning n m d eps).u * rawNormalizer n m d eps sigma lat *
((normalizedLaw hyp n m d eps hd sigma lat).pmf z).toReal =
(affineTuning n m d eps).u * rawTable hyp n m d eps sigma lat z := by
rw [affine_normalized_mass_readback n m d eps hn hd heps heps' sigma hyp lat]
have hQ : rawNormalizer n m d eps sigma lat ≠ 0 :=
ne_of_gt (lt_trans (by norm_num) (affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat))
field_simp
theorem affine_auxiliary_intensity_readback
Under the stated inputs and conditions, The random-scale auxiliary pool has exactly the raw arm intensities, including all controls. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
hyp :
lat :
z :
AuxObs d
conclusion
(affineTuning n m d eps).w * rawNormalizer n m d eps sigma lat * (auxMarginal (normalizedLaw hyp n m d eps hd sigma lat) z).toReal
Proof (Lean source)
-- @node: affine_auxiliary_intensity_readback
lemma affine_auxiliary_intensity_readback (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(hyp : Bool) (lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L)
(z : AuxObs d) :
(affineTuning n m d eps).w * rawNormalizer n m d eps sigma lat *
(auxMarginal (normalizedLaw hyp n m d eps hd sigma lat) z).toReal =
(affineTuning n m d eps).w * ∑ y : Bool, rawTable hyp n m d eps sigma lat (z.1, z.2, y) := by
rw [affine_normalized_aux_mass_readback n m d eps hn hd heps heps' sigma hyp lat]
have hQ : rawNormalizer n m d eps sigma lat ≠ 0 :=
ne_of_gt (lt_trans (by norm_num) (affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat))
field_simp
Under the stated inputs and conditions, A Poisson pool with mean at least 64 times the requested size has the exponential prefix-failure bound in equation (12). This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_poisson_prefix_tail
lemma affine_poisson_prefix_tail (k : Nat) (rate : NNReal)
(hrate : 64 * (k : Real) ≤ (rate : Real)) :
poissonMeasure rate (Iio k) ≤ ofReal (exp (-31 * (k : Real))) := by
have hsub : Iio k ⊆ {j : Nat | (j : Real) ≤ (4 * (k : Real)) / 4} := by
intro j hj
have hjk : (j : Real) ≤ k := by exact_mod_cast (show j ≤ k from hj.le)
change (j : Real) ≤ (4 * (k : Real)) / 4
linarith
have htail := poisson_pilot_lower_tail
rate (4 * (k : Real))
have hlog : log (4 : Real) ≤ 3 := by
have h := Real.log_le_sub_one_of_pos (by norm_num : (0 : Real) < 4)
norm_num at h
exact h
have hexp : 4 * (k : Real) * log 4 / 4 - 3 * (rate : Real) / 4 ≤
-31 * (k : Real) := by
have h := mul_le_mul_of_nonneg_left hlog (Nat.cast_nonneg k : (0 : Real) ≤ k)
nlinarith
have hreal : (poissonMeasure rate).real (Iio k) ≤ exp (-31 * (k : Real)) :=
(measureReal_mono hsub).trans (htail.trans (Real.exp_le_exp.mpr hexp))
rw [← ENNReal.ofReal_toReal (measure_ne_top (poissonMeasure rate) (Iio k))]
exact ENNReal.ofReal_le_ofReal hreal
Under the stated inputs and conditions, The complete raw pool has the uniform exponential prefix budget, since Q exceeds one half. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
lat :
conclusion
Proof (Lean source)
-- @node: affine_complete_prefix_tail
lemma affine_complete_prefix_tail (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
poissonMeasure (toNNReal ((affineTuning n m d eps).u *
rawNormalizer n m d eps sigma lat)) (Iio n) ≤
ofReal (exp (-31 * (n : Real))) := by
have hQ := affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat
have hu : (affineTuning n m d eps).u = 128 * (n : Real) := rfl
apply affine_poisson_prefix_tail
rw [Real.coe_toNNReal _ (by rw [hu]; positivity), hu]
nlinarith [Nat.cast_nonneg n (α := Real)]
Under the stated inputs and conditions, Retaining m auxiliary records costs at most the exponential budget calibrated to n+m. When m is zero, monotonicity still applies to the empty failure event. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
lat :
conclusion
Proof (Lean source)
-- @node: affine_auxiliary_prefix_tail
lemma affine_auxiliary_prefix_tail (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
poissonMeasure (toNNReal ((affineTuning n m d eps).w *
rawNormalizer n m d eps sigma lat)) (Iio m) ≤
ofReal (exp (-31 * ((n : Real) + m))) := by
have hQ := affine_rawNormalizer_pos n m d eps hn hd heps heps' sigma lat
have hw : (affineTuning n m d eps).w = 128 * ((n : Real) + m) := rfl
have hrate : 64 * ((n + m : Nat) : Real) ≤
(toNNReal ((affineTuning n m d eps).w *
rawNormalizer n m d eps sigma lat) : Real) := by
rw [Real.coe_toNNReal _ (by rw [hw]; positivity), hw, Nat.cast_add]
nlinarith [Nat.cast_nonneg n (α := Real), Nat.cast_nonneg m (α := Real)]
exact (measure_mono (show Iio m ⊆ Iio (n + m) by intro j hj; exact lt_of_lt_of_le hj (Nat.le_add_left m n))).trans
(by simpa only [Nat.cast_add] using affine_poisson_prefix_tail (n + m) _ hrate)
theorem affine_prefix_failure_sum_gap
Under the stated inputs and conditions, The sum of the two actual Poisson failure probabilities fits the gap budget required by the fixed-size transfer. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hS :
1 ≤ (n : Real) * eps
sigma :
lat :
conclusion
nu :
gap :
(∫ lat, affineRawTarget n m d eps sigma true lat ∂nu)
- (∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
+ poissonMeasure (toNNReal ((affineTuning n m d eps).w * rawNormalizer n m d eps sigma lat)) (Iio m)
≤ ofReal (gap ^ 2 / 128)
Proof (Lean source)
-- @node: affine_prefix_failure_sum_gap
lemma affine_prefix_failure_sum_gap (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps)
(lat : Fin (affineTuning n m d eps).Kstar → Latent (affineTuning n m d eps).L) :
let nu := Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)
let gap := (∫ lat, affineRawTarget n m d eps sigma true lat ∂nu) -
(∫ lat, affineRawTarget n m d eps sigma false lat ∂nu)
poissonMeasure (toNNReal ((affineTuning n m d eps).u *
rawNormalizer n m d eps sigma lat)) (Iio n) +
poissonMeasure (toNNReal ((affineTuning n m d eps).w *
rawNormalizer n m d eps sigma lat)) (Iio m) ≤
ofReal (gap ^ 2 / 128) := by
intro nu gap
have hxTail := affine_complete_prefix_tail n m d eps hn hd heps heps' sigma lat
have hyTail := affine_auxiliary_prefix_tail n m d eps hn hd heps heps' sigma lat
have hmono : exp (-31 * ((n : Real) + m)) ≤ exp (-31 * (n : Real)) := by
apply Real.exp_le_exp.mpr
linarith [Nat.cast_nonneg m (α := Real)]
have hgap := affine_fixed_size_failure_gap n m d eps hn hd heps heps' hS hx sigma
calc
_ ≤ ofReal (exp (-31 * (n : Real))) +
ofReal (exp (-31 * (n : Real))) :=
add_le_add hxTail (hyTail.trans (ENNReal.ofReal_le_ofReal hmono))
_ = ofReal (2 * exp (-31 * (n : Real))) := by
rw [← ENNReal.ofReal_add (Real.exp_pos _).le (Real.exp_pos _).le]
congr 1
ring
_ ≤ _ := ENNReal.ofReal_le_ofReal hgap
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Affine.Tuning 5 declarations Affine-prior public schedule and the shared raw total mass.
Affine-prior public schedule and the shared raw total mass.
theorem affine_tuning_bounds
Under the stated inputs and conditions, The affine schedule has uniform total-intensity, degree, and rare-cell mass bounds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: affine_tuning_bounds
lemma affine_tuning_bounds (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
let tun := affineTuning n m d eps
128 * ((n : Real) + m) ≤ tun.M ∧ tun.M ≤ 256 * ((n : Real) + m) ∧
8 * logScale n eps ≤ (tun.L : Real) ∧ (tun.L : Real) ≤ 9 * logScale n eps ∧
0 < tun.B ∧ 0 < tun.alpha ∧ 0 < tun.b0 ∧ (tun.Kstar : Real) * tun.b0 ≤ 1 / 100 := by
have hnR : (1 : Real) ≤ n := by exact_mod_cast hn
have hmR : (0 : Real) ≤ m := Nat.cast_nonneg m
have hell : 1 ≤ logScale n eps := by
have h := Real.log_le_log (Real.exp_pos 1)
(show exp 1 ≤ exp 1 + labelScale n eps by
exact le_add_of_nonneg_right (by dsimp [labelScale]; positivity))
simpa only [Real.log_exp, logScale] using h
have hLlo := Nat.le_ceil (8 * logScale n eps)
have hLhi := Nat.ceil_lt_add_one
(show 0 ≤ 8 * logScale n eps by linarith)
have hLpos : 0 < (ceil (8 * logScale n eps) : Real) := by linarith
have hMpos : 0 < 128 * (n : Real) + 128 * ((n : Real) + m) := by linarith
have hBpos : 0 < (affineTuning n m d eps).B := by
dsimp [affineTuning]; exact div_pos hLpos (mul_pos (by norm_num) hMpos)
have hapos : 0 < (affineTuning n m d eps).alpha := by
dsimp only [affineTuning] at hBpos ⊢
exact div_pos hBpos (mul_pos (by norm_num) (pow_pos hLpos 2))
have hbpos : 0 < (affineTuning n m d eps).b0 := div_pos hapos heps
refine ⟨?_, ?_, hLlo, ?_, hBpos, hapos, hbpos, ?_⟩
· dsimp [affineTuning]; linarith
· dsimp [affineTuning]; linarith
· dsimp [affineTuning]; linarith
· have hfloor := Nat.floor_le
(show 0 ≤ 1 / (100 * (affineTuning n m d eps).b0) by positivity)
have hmin : (affineTuning n m d eps).Kstar ≤
floor (1 / (100 * (affineTuning n m d eps).b0)) := Nat.min_le_right _ _
have hcast : ((affineTuning n m d eps).Kstar : Real) ≤
(floor (1 / (100 * (affineTuning n m d eps).b0)) : Real) := by
exact_mod_cast hmin
have hbound := mul_le_mul_of_nonneg_right (hcast.trans hfloor) hbpos.le
have hid : (1 / (100 * (affineTuning n m d eps).b0)) *
(affineTuning n m d eps).b0 = 1 / 100 := by field_simp
exact hid ▸ hbound
Under the stated inputs and conditions, The baseline mass has the explicit reciprocal form used in the numerical bounds. This gives the stated result.
Formal statement
hypotheses
conclusion
(affineTuning n m d eps).b0
= 1 / (100000 * (affineTuning n m d eps).M * eps * (affineTuning n m d eps).L)
Proof (Lean source)
-- @node: affine_tuning_b0_identity
lemma affine_tuning_b0_identity (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
(affineTuning n m d eps).b0 =
1 / (100000 * (affineTuning n m d eps).M * eps * (affineTuning n m d eps).L) := by
obtain ⟨hM, _, hL, _, _, _, _, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hell : 1 ≤ logScale n eps := by
have h := Real.log_le_log (Real.exp_pos 1)
(show exp 1 ≤ exp 1 + labelScale n eps by
apply le_add_of_nonneg_right; dsimp [labelScale]; positivity)
simpa only [logScale, Real.log_exp] using h
have hLp : 0 < ((affineTuning n m d eps).L : Real) := by linarith
have hMp : 0 < (affineTuning n m d eps).M := by
have hnR : (1 : Real) ≤ n := by exact_mod_cast hn
have hmR : (0 : Real) ≤ m := Nat.cast_nonneg m
linarith
change ((affineTuning n m d eps).L : Real) /
(1000 * (affineTuning n m d eps).M) /
(100 * ((affineTuning n m d eps).L : Real) ^ 2) / eps = _
field_simp
<;> ring
Under the stated inputs and conditions, The bandwidth contribution to concentration equals the stated dimensionless scale. This gives the stated result.
Formal statement
hypotheses
conclusion
(affineTuning n m d eps).B * (affineTuning n m d eps).alpha
Proof (Lean source)
-- @node: affine_tuning_B_alpha_identity
lemma affine_tuning_B_alpha_identity (n m d : Nat) (eps : Real) (heps : 0 < eps) :
(affineTuning n m d eps).B * (affineTuning n m d eps).alpha =
100 * eps ^ 2 * ((affineTuning n m d eps).L : Real) ^ 2 *
(affineTuning n m d eps).b0 ^ 2 := by
have hb : eps * (affineTuning n m d eps).b0 = (affineTuning n m d eps).alpha := by
change eps * ((affineTuning n m d eps).alpha / eps) = (affineTuning n m d eps).alpha
rw [mul_comm]
exact div_mul_cancel₀ _ heps.ne'
have ha : (affineTuning n m d eps).alpha *
(100 * ((affineTuning n m d eps).L : Real) ^ 2) = (affineTuning n m d eps).B := by
change ((affineTuning n m d eps).B / (100 * ((affineTuning n m d eps).L : Real) ^ 2)) *
(100 * ((affineTuning n m d eps).L : Real) ^ 2) = (affineTuning n m d eps).B
by_cases hL : (affineTuning n m d eps).L = 0
· have hB : (affineTuning n m d eps).B = 0 := by
change ((affineTuning n m d eps).L : Real) / (1000 * (affineTuning n m d eps).M) = 0
rw [hL]; simp
simp [hL, hB]
· exact div_mul_cancel₀ _ (by positivity)
rw [← ha, ← hb]
ring
Under the stated inputs and conditions, At label scale at least one, the affine schedule includes a rare cell. This gives the stated result.
Formal statement
hypotheses
conclusion
0 < (affineTuning n m d eps).Kstar
Proof (Lean source)
-- @node: affine_tuning_Kstar_pos
lemma affine_tuning_Kstar_pos (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : 1 ≤ (n : Real) * eps) : 0 < (affineTuning n m d eps).Kstar := by
obtain ⟨hM, _, hL, _, _, _, hb, _⟩ := affine_tuning_bounds n m d eps hn hd heps heps'
have hell : 1 ≤ logScale n eps := by
have h := Real.log_le_log (Real.exp_pos 1)
(show exp 1 ≤ exp 1 + labelScale n eps by
apply le_add_of_nonneg_right; dsimp [labelScale]; positivity)
simpa only [logScale, Real.log_exp] using h
have hMe : 128 ≤ (affineTuning n m d eps).M * eps := by
have hm : (0 : Real) ≤ m := Nat.cast_nonneg m
have h := mul_le_mul_of_nonneg_right hM heps.le
nlinarith
have hden : 100 ≤ 100000 * (affineTuning n m d eps).M * eps *
((affineTuning n m d eps).L : Real) := by
have h := mul_le_mul hMe hL (by linarith : 0 ≤ 8 * logScale n eps) (by linarith)
nlinarith
have hbsmall : (affineTuning n m d eps).b0 ≤ 1 / 100 := by
rw [affine_tuning_b0_identity n m d eps hn hd heps heps']
apply (div_le_iff₀ (by linarith : 0 < 100000 * (affineTuning n m d eps).M * eps *
((affineTuning n m d eps).L : Real))).mpr
linarith
have hinv : (1 : Real) ≤ 1 / (100 * (affineTuning n m d eps).b0) := by
apply (le_div_iff₀ (by positivity)).mpr
linarith
have hfloor : 1 ≤ floor (1 / (100 * (affineTuning n m d eps).b0)) :=
(Nat.le_floor_iff (by positivity)).mpr (by exact_mod_cast hinv)
change 0 < min (d - 1) (floor (1 / (100 * (affineTuning n m d eps).b0)))
exact lt_min (by omega) (by omega)
def rawNormalizer
For the complete-sample size, the auxiliary-sample size, the covariate dimension, the overlap floor, a cone-dual certificate, and the latent assignments of rare cells, the raw normalizer is the remaining reservoir mass after the affine rare-cell masses are assigned.
Definition (Lean source)
parameters
n m d :
eps :
sigma :
lat :
def
given by
1
+ ∑ j, ((affineTuning n m d eps).b0 + latentNode sigma (lat j))
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.AffineTesting 5 declarations Support, common normalized marginal, and original-experiment affine-prior testing bound.
Support, common normalized marginal, and original-experiment affine-prior testing bound.
theorem affine_auxMarginal_measurable
Under the stated inputs and conditions, The auxiliary marginal is Borel in the complete-record mass coordinates. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: affine_auxMarginal_measurable
@[fun_prop]
lemma affine_auxMarginal_measurable (d : Nat) :
Measurable (auxMarginal (d := d)) := by
apply measurable_comap_iff.mpr
apply measurable_pi_lambda
intro z
change Measurable (fun P : DiscreteLaw d => (auxMarginal P z).toReal)
simp_rw [show ∀ P : DiscreteLaw d, (auxMarginal P z).toReal = armMass P z.1 z.2 from
fun P => auxMarginal_toReal_armMass P z.1 z.2]
unfold armMass jointMass
apply Finset.measurable_sum
intro y _
exact (measurable_pi_apply (z.1, z.2, y)).comp
(comap_measurable (fun P : DiscreteLaw d => fun o => (P.pmf o).toReal))
theorem affine_productPrior_common_marginal
Under the stated inputs and conditions, Coupling the same finite latent vector gives identical normalized marginal laws. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
Proof (Lean source)
-- @node: affine_productPrior_common_marginal
lemma affine_productPrior_common_marginal (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
(affineProductPrior true n m d eps hd sigma).map auxMarginal =
(affineProductPrior false n m d eps hd sigma).map auxMarginal := by
have hm (hyp : Bool) : Measurable (normalizedLaw hyp n m d eps hd sigma) := by
fun_prop
unfold affineProductPrior
rw [Measure.map_map (affine_auxMarginal_measurable d) (hm true),
Measure.map_map (affine_auxMarginal_measurable d) (hm false)]
congr 1
funext lat
exact (affine_support_and_normalizer n m d eps hn hd heps heps' sigma).2 lat |>.2.2
theorem affine_prior_support_and_common_marginal
Under the stated inputs and conditions, Both affine hypotheses have legal support and the same marginal under the latent coupling. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
sigma :
conclusion
conclusion 2
lat :
conclusion 3
Proof (Lean source)
-- @node: affine_prior_support_and_common_marginal
lemma affine_prior_support_and_common_marginal (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) :
(∀ hyp lat, ModelClass d eps (normalizedLaw hyp n m d eps hd sigma lat)) ∧
(∀ lat, auxMarginal (normalizedLaw true n m d eps hd sigma lat) =
auxMarginal (normalizedLaw false n m d eps hd sigma lat)) ∧
(affineProductPrior true n m d eps hd sigma).map auxMarginal =
(affineProductPrior false n m d eps hd sigma).map auxMarginal := by
have hs := (affine_support_and_normalizer n m d eps hn hd heps heps' sigma).2
exact ⟨fun hyp lat => (hs lat).2.1 hyp, fun lat => (hs lat).2.2,
affine_productPrior_common_marginal n m d eps hn hd heps heps' sigma⟩
theorem affine_selectedPrior_common_marginal
Under the stated inputs and conditions, In the affine regime the selected prior handle preserves the common marginal law. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
conclusion
Proof (Lean source)
-- @node: affine_selectedPrior_common_marginal
lemma affine_selectedPrior_common_marginal (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hS : exp 4096 ≤ (n : Real) * eps)
(hx : 1 / ((n : Real) * eps) <
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) :
(affinePrior true n m d eps hn hd heps heps').map auxMarginal =
(affinePrior false n m d eps hn hd heps heps').map auxMarginal := by
have hbranch : ¬ (labelScale n eps < exp 4096 ∨
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2 ≤
1 / labelScale n eps) := by
exact not_or.mpr ⟨not_lt.mpr hS, not_le.mpr hx⟩
simp only [affinePrior, hbranch, if_false]
exact affine_productPrior_common_marginal n m d eps hn hd heps heps' _
theorem normalized_affine_testing
Under the stated inputs and conditions, The normalized affine priors have legal support and a common auxiliary-table distribution, and prove the all-rule lower bound. This gives the stated conclusion.
Formal statement
conclusion
∃ c : Real,
0 < c
conclusion 1
n m d :
eps :
hn :
1 ≤ n
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
1 / ((n : Real) * eps) < x ^ 2
sigma :
c * min 1 (x ^ 2) ≤ minimaxRisk n m d eps
Proof (Lean source)
lemma normalized_affine_testing :
∃ c : Real, 0 < c ∧ ∀ (n m d : Nat) (eps : Real),
∀ hn : 1 ≤ n, ∀ hd : 2 ≤ d, ∀ heps : 0 < eps, ∀ heps' : eps ≤ 1 / 4,
exp 4096 ≤ (n : Real) * eps →
let x := (d : Real) / (((n : Real) + m) * eps * logScale n eps)
1 / ((n : Real) * eps) < x ^ 2 →
(∀ sigma : ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps,
(∀ hyp lat, ModelClass d eps (normalizedLaw hyp n m d eps hd sigma lat)) ∧
(∀ lat, auxMarginal (normalizedLaw true n m d eps hd sigma lat) =
auxMarginal (normalizedLaw false n m d eps hd sigma lat)) ∧
(affineProductPrior true n m d eps hd sigma).map auxMarginal =
(affineProductPrior false n m d eps hd sigma).map auxMarginal) ∧
(affinePrior true n m d eps hn hd heps heps').map auxMarginal =
(affinePrior false n m d eps hn hd heps heps').map auxMarginal ∧
c * min 1 (x ^ 2) ≤ minimaxRisk n m d eps := by
refine ⟨10 ^ (-20 : Int) / 128, by positivity, ?_⟩
intro n m d eps hn hd heps heps' hS x hx
refine ⟨fun sigma => affine_prior_support_and_common_marginal n m d eps hn hd heps heps' sigma,
affine_selectedPrior_common_marginal n m d eps hn hd heps heps' hS hx, ?_⟩
obtain ⟨sigma⟩ := affine_schedule_dual n m d eps hn heps heps'
exact affine_fixed_minimax_rate_lower n m d eps hn hd heps heps' hS hx sigma
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Baseline 4 declarations Deterministic inverse-count baseline with independent two-pool prefix averaging.
Deterministic inverse-count baseline with independent two-pool prefix averaging.
def baselinePrefixStatistic
The baseline prefix statistic clips the treated-minus-control inverse-count sum.
Definition (Lean source)
parameters
def
The inverse-count baseline averages valid two-pool prefixes and outputs zero on overflow or small sample sizes.
Definition (Lean source)
parameters
n m d :
def
theorem baselineEstimator_measurable
The baseline is measurable on the original finite sample arrays.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: baselineEstimator_measurable
lemma baselineEstimator_measurable (n m d : Nat) : Measurable (baselineEstimator n m d) := by
fun_prop
theorem baselineEstimator_mem_Icc
The total baseline takes values between minus one and one.
Formal statement
Proof (Lean source)
-- @node: baselineEstimator_mem_Icc
lemma baselineEstimator_mem_Icc (n m d : Nat) (s : Sample n m d) :
baselineEstimator n m d s ∈ Icc (-1) 1 := by
classical
by_cases hsmall : n < 6
· simp [baselineEstimator, hsmall]
· rw [baselineEstimator, if_neg hsmall]
let h := n / 2
let g := n - h + m
let u : Real := h / 8
let t : Real := g / 8
have hu : 0 ≤ u := by dsimp [u]; positivity
have ht : 0 ≤ t := by dsimp [t]; positivity
have hclip (r v : Nat) : baselinePrefixStatistic r v s ∈ Icc (-1) 1 := by
constructor
· exact le_max_left _ _
· exact max_le (by norm_num) (min_le_left _ _)
have havg := finitePoissonAverage_mem_Icc u hu h _ (fun r =>
finitePoissonAverage_mem_Icc t ht g _ (hclip r))
convert havg using 1
simp only [u, t, g, h, Finset.mul_sum]
apply Finset.sum_congr rfl
intro r _
apply Finset.sum_congr rfl
intro v _
rw [sub_eq_add_neg, Real.exp_add]
ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.BaselinePoissonRisk 7 declarations Squared-risk assembly for the inverse-count Poisson baseline and numerical pool bounds.
Squared-risk assembly for the inverse-count Poisson baseline and numerical pool bounds.
theorem baseline_poisson_arm_memLp
Under the stated hypotheses, Independent Poisson cell statistics have a square-integrable arm sum. This gives the stated result.
Formal statement
hypotheses
d :
P :
a :
u t :
hu :
0 < u
Om :
Type
mu :
hmeas :
hZ :
hK :
hW :
Proof (Lean source)
-- @node: baseline_poisson_arm_memLp
lemma baseline_poisson_arm_memLp {d : Nat} (P : DiscreteLaw d) (a : Bool)
(u t : NNReal) (hu : 0 < u) {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (Z K W : Fin d → Om → Nat)
(hmeas : ∀ j, Measurable (Z j) ∧ Measurable (K j) ∧ Measurable (W j))
(hZ : ∀ j, mu.map (Z j) = poissonMeasure (u * toNNReal (markedMass P j a)))
(hK : ∀ j, mu.map (K j) = poissonMeasure (t * toNNReal (armMass P j a)))
(hW : ∀ j, mu.map (W j) = poissonMeasure (t * toNNReal (armMass P j (!a))))
(hind : iIndepFun
(fun i : Fin 3 × Fin d => if i.1 = 0 then Z i.2 else if i.1 = 1 then K i.2 else W i.2) mu) :
MemLp (fun om => ∑ j, (Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1))) 2 mu := by
apply memLp_finsetSum
intro j _
let X : Fin 3 → Om → Nat :=
fun i => if i = 0 then Z j else if i = 1 then K j else W j
let lam : Fin 3 → NNReal :=
fun i => if i = 0 then u * toNNReal (markedMass P j a)
else if i = 1 then t * toNNReal (armMass P j a)
else t * toNNReal (armMass P j (!a))
have hx (i) : Measurable (X i) := by
dsimp [X]
split_ifs <;> first
| exact (hmeas j).1
| exact (hmeas j).2.1
| exact (hmeas j).2.2
have hl (i) : mu.map (X i) = poissonMeasure (lam i) := by
dsimp [X, lam]
split_ifs <;> first | exact hZ j | exact hK j | exact hW j
have hind' : iIndepFun X mu :=
hind.precomp (g := fun i : Fin 3 => (i, j))
(fun i i' h => (Prod.mk.inj h).1)
have h := inverse_count_poisson_cell_variance mu X lam u hx hl hind'
have hq : 0 ≤ markedMass P j a := jointMass_nonneg P j a true
have hu' : (u : Real) ≠ 0 := (NNReal.coe_pos.mpr hu).ne'
have hscale : (u : Real) * markedMass P j a / (u : Real) ^ 2 =
markedMass P j a / (u : Real) := by
field_simp
simpa [X, lam, NNReal.coe_mul,
Real.coe_toNNReal _ hq,
Real.coe_toNNReal _ (armMass_nonneg P j a),
Real.coe_toNNReal _ (armMass_nonneg P j (!a)),
mul_div_cancel_left₀ _ hu', hscale] using h.1
Under the stated inputs and conditions, Squared loss around any constant is variance plus squared bias. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: baseline_mse_eq_variance_add_bias
lemma baseline_mse_eq_variance_add_bias {Om : Type} [MeasurableSpace Om]
{mu : Measure Om} [IsProbabilityMeasure mu] {H : Om → Real}
(hH : MemLp H 2 mu) (theta : Real) :
(∫ om, (H om - theta) ^ 2 ∂mu) =
variance H mu + ((∫ om, H om ∂mu) - theta) ^ 2 := by
have hshift := hH.sub (memLp_const theta)
have hv := variance_eq_sub hshift
change variance (fun om => H om - theta) mu =
(∫ om, (H om - theta) ^ 2 ∂mu) - (∫ om, H om - theta ∂mu) ^ 2 at hv
rw [variance_sub_const hH.aestronglyMeasurable theta] at hv
have hm : (∫ om, H om - theta ∂mu) = (∫ om, H om ∂mu) - theta := by
rw [integral_sub (hH.integrable (by norm_num)) (integrable_const theta)]
simp
rw [hm] at hv
linarith
theorem baseline_poisson_arm_mse
Under the stated inputs and conditions, The proved arm bias and variance bounds imply the uncapped squared-risk bound. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
d :
eps :
P :
a :
u t :
Om :
Type
mu :
Measure Om
2 ≤ d
0 < eps
eps ≤ 1 / 4
ModelClass d eps P
0 < u
0 < t
Hs :
psi :
Proof (Lean source)
-- @node: baseline_poisson_arm_mse
lemma baseline_poisson_arm_mse :
∃ C : Real, 0 < C ∧
∀ (d : Nat) (eps : Real) (P : DiscreteLaw d) (a : Bool) (u t : NNReal)
(Om : Type) [MeasurableSpace Om] (mu : Measure Om) [IsProbabilityMeasure mu]
(Z K W : Fin d → Om → Nat),
2 ≤ d → 0 < eps → eps ≤ 1 / 4 → ModelClass d eps P → 0 < u → 0 < t →
(∀ j, Measurable (Z j) ∧ Measurable (K j) ∧ Measurable (W j)) →
(∀ j, mu.map (Z j) = poissonMeasure (u * toNNReal (markedMass P j a))) →
(∀ j, mu.map (K j) = poissonMeasure (t * toNNReal (armMass P j a))) →
(∀ j, mu.map (W j) = poissonMeasure (t * toNNReal (armMass P j (!a)))) →
iIndepFun
(fun i : Fin 3 × Fin d => if i.1 = 0 then Z i.2 else if i.1 = 1 then K i.2 else W i.2) mu →
let Hs : Om → Real := fun om => ∑ j : Fin d,
(Z j om : Real) / (u : Real) * (1 + (W j om : Real) / ((K j om : Real) + 1))
let psi : Real := ∑ j : Fin d, cellMass P j * outcomeMean P a j
(∫ om, (Hs om - psi) ^ 2 ∂mu) ≤
C * (1 / ((u : Real) * eps) + 1 / ((t : Real) * eps)) +
((d : Real) / (exp 1 * (t : Real) * eps)) ^ 2 := by
obtain ⟨C, hC, hbound⟩ := inverse_count_arm_risk
refine ⟨C, hC, ?_⟩
intro d eps P a u t Om _ mu _ Z K W hd heps heps' hP hu ht hmeas hZ hK hW hind
dsimp only
have hLp := baseline_poisson_arm_memLp P a u t hu mu Z K W hmeas hZ hK hW hind
obtain ⟨hbias, hvar⟩ :=
hbound d eps P a u t Om mu Z K W hd heps heps' hP hu ht hmeas hZ hK hW hind
rw [baseline_mse_eq_variance_add_bias hLp]
apply add_le_add hvar
have hb := hbias.trans (min_le_right _ _)
have hs := pow_le_pow_left₀ (abs_nonneg _) hb 2
simpa only [sq_abs] using hs
theorem baseline_poisson_contrast_risk
Under the stated inputs and conditions, The clipped Poisson contrast has risk at most four times the arm envelope; independence between the two arms is unnecessary. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
d :
eps :
P :
u t :
Om :
Type
mu :
Measure Om
2 ≤ d
0 < eps
eps ≤ 1 / 4
ModelClass d eps P
0 < u
0 < t
(∀ a j, Measurable (Z a j) ∧ Measurable (K a j))
Proof (Lean source)
-- @node: baseline_poisson_contrast_risk
lemma baseline_poisson_contrast_risk :
∃ C : Real, 0 < C ∧
∀ (d : Nat) (eps : Real) (P : DiscreteLaw d) (u t : NNReal)
(Om : Type) [MeasurableSpace Om] (mu : Measure Om) [IsProbabilityMeasure mu]
(Z K : Bool → Fin d → Om → Nat),
2 ≤ d → 0 < eps → eps ≤ 1 / 4 → ModelClass d eps P → 0 < u → 0 < t →
(∀ a j, Measurable (Z a j) ∧ Measurable (K a j)) →
(∀ a j, mu.map (Z a j) = poissonMeasure (u * toNNReal (markedMass P j a))) →
(∀ a j, mu.map (K a j) = poissonMeasure (t * toNNReal (armMass P j a))) →
(∀ a, iIndepFun
(fun i : Fin 3 × Fin d => if i.1 = 0 then Z a i.2
else if i.1 = 1 then K a i.2 else K (!a) i.2) mu) →
let H : Bool → Om → Real := fun a om => ∑ j : Fin d,
(Z a j om : Real) / (u : Real) *
(1 + (K (!a) j om : Real) / ((K a j om : Real) + 1))
(∫ om, (max (-1) (min 1 (H true om - H false om)) - ateFunctional P) ^ 2 ∂mu) ≤
C * (1 / ((u : Real) * eps) + 1 / ((t : Real) * eps)) +
4 * ((d : Real) / (exp 1 * (t : Real) * eps)) ^ 2 := by
obtain ⟨C, hC, hbound⟩ := baseline_poisson_arm_mse
refine ⟨4 * C, by positivity, ?_⟩
intro d eps P u t Om _ mu _ Z K hd heps heps' hP hu ht hmeas hZ hK hind
let H : Bool → Om → Real := fun a om => ∑ j : Fin d,
(Z a j om : Real) / (u : Real) *
(1 + (K (!a) j om : Real) / ((K a j om : Real) + 1))
let psi : Bool → Real := fun a => ∑ j : Fin d, cellMass P j * outcomeMean P a j
have hmeas' (a j) : Measurable (Z a j) ∧ Measurable (K a j) ∧
Measurable (K (!a) j) := ⟨(hmeas a j).1, (hmeas a j).2, (hmeas (!a) j).2⟩
have hLp (a) : MemLp (H a) 2 mu :=
baseline_poisson_arm_memLp P a u t hu mu (Z a) (K a) (K (!a))
(hmeas' a) (hZ a) (hK a) (hK (!a)) (hind a)
have hMSE (a) : (∫ om, (H a om - psi a) ^ 2 ∂mu) ≤
C * (1 / ((u : Real) * eps) + 1 / ((t : Real) * eps)) +
((d : Real) / (exp 1 * (t : Real) * eps)) ^ 2 :=
hbound d eps P a u t Om mu (Z a) (K a) (K (!a)) hd heps heps' hP hu ht
(hmeas' a) (hZ a) (hK a) (hK (!a)) (hind a)
have htau : ateFunctional P = psi true - psi false := by
simp only [ateFunctional, psi, Finset.sum_sub_distrib, mul_sub]
have hint (a) : Integrable (fun om => (H a om - psi a) ^ 2) mu :=
((hLp a).sub (memLp_const (psi a))).integrable_sq
have hcontrast : Integrable (fun om =>
(H true om - H false om - ateFunctional P) ^ 2) mu :=
(((hLp true).sub (hLp false)).sub (memLp_const (ateFunctional P))).integrable_sq
change (∫ om, (max (-1) (min 1 (H true om - H false om)) - ateFunctional P) ^ 2 ∂mu) ≤ _
calc
_ ≤ ∫ om, (H true om - H false om - ateFunctional P) ^ 2 ∂mu :=
integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
hcontrast (Filter.Eventually.of_forall (fun om =>
knownMarginal_clip_loss _ _ (ateFunctional_mem_Icc P)))
_ ≤ ∫ om, (2 * (H true om - psi true) ^ 2 +
2 * (H false om - psi false) ^ 2) ∂mu := by
apply integral_mono hcontrast ((hint true).const_mul 2 |>.add ((hint false).const_mul 2))
intro om
change (H true om - H false om - ateFunctional P) ^ 2 ≤
2 * (H true om - psi true) ^ 2 + 2 * (H false om - psi false) ^ 2
rw [htau]
nlinarith [sq_nonneg ((H true om - psi true) + (H false om - psi false))]
_ = 2 * (∫ om, (H true om - psi true) ^ 2 ∂mu) +
2 * (∫ om, (H false om - psi false) ^ 2 ∂mu) := by
rw [integral_add ((hint true).const_mul 2) ((hint false).const_mul 2),
integral_const_mul, integral_const_mul]
_ ≤ _ := by nlinarith [hMSE true, hMSE false]
Under the stated inputs and conditions, The baseline's two disjoint pools have the lower sizes required by the roadmap. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: baseline_pool_sizes
lemma baseline_pool_sizes (n m : Nat) (hn : 6 ≤ n) :
(n : Real) / 3 ≤ (n / 2 : Nat) ∧
((n : Real) + m) / 2 ≤ (n - n / 2 + m : Nat) := by
have hh : n ≤ 3 * (n / 2) := by omega
have hg : n + m ≤ 2 * (n - n / 2 + m) := by omega
have hh' : (n : Real) ≤ 3 * (n / 2 : Nat) := by exact_mod_cast hh
have hg' : (n : Real) + m ≤ 2 * (n - n / 2 + m : Nat) := by exact_mod_cast hg
constructor <;> linarith
Under the stated inputs and conditions, Zero on overflow adds at most two inverse label scales. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: baseline_pool_overflow_bound
lemma baseline_pool_overflow_bound (n m : Nat) (eps : Real)
(hn : 6 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
exp (-(n / 2 : Nat)) + exp (-(n - n / 2 + m : Nat)) ≤
2 / ((n : Real) * eps) := by
obtain ⟨hh, hg⟩ := baseline_pool_sizes n m hn
have hn0 : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hS : 0 < (n : Real) * eps := mul_pos hn0 heps
have htail : exp (-(n / 2 : Nat)) + exp (-(n - n / 2 + m : Nat)) ≤
2 * exp (-(n : Real) / 3) := by
have h1 := Real.exp_le_exp.mpr (show -(n / 2 : Nat) ≤ -(n : Real) / 3 by linarith)
have h2 := Real.exp_le_exp.mpr
(show -(n - n / 2 + m : Nat) ≤ -(n : Real) / 3 by
linarith [Nat.cast_nonneg (α := Real) m])
linarith
have hx := Real.mul_exp_neg_le_exp_neg_one ((n : Real) / 3)
rw [← neg_div] at hx
have he : exp (-1 : Real) ≤ 1 := by
exact (Real.exp_le_exp.mpr (by norm_num : (-1 : Real) ≤ 0)).trans_eq Real.exp_zero
have hprod : (n : Real) * eps * (2 * exp (-(n : Real) / 3)) ≤ 2 := by
have hne : (n : Real) * eps ≤ (n : Real) / 4 := by nlinarith
have hmul := mul_le_mul_of_nonneg_right hne (Real.exp_pos (-(n : Real) / 3)).le
nlinarith
exact htail.trans ((le_div_iff₀ hS).2 (by nlinarith [hprod]))
Under the stated inputs and conditions, Substituting the pool intensities and absorbing overflow gives a numerical public-rate bound. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: baseline_pool_rate_bound
lemma baseline_pool_rate_bound (n m d : Nat) (eps C : Real)
(hn : 6 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) (hC : 0 < C) :
C * (1 / (((n / 2 : Nat) : Real) / 8 * eps) +
1 / (((n - n / 2 + m : Nat) : Real) / 8 * eps)) +
4 * ((d : Real) / (exp 1 * ((n - n / 2 + m : Nat) : Real) / 8 * eps)) ^ 2 +
(exp (-(n / 2 : Nat)) + exp (-(n - n / 2 + m : Nat))) ≤
(40 * C + 1026) * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps)) ^ 2) := by
obtain ⟨hh, hg⟩ := baseline_pool_sizes n m hn
have hn0 : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hS : 0 < (n : Real) * eps := mul_pos hn0 heps
have hN : 0 < ((n : Real) + m) * eps := by positivity
have hh0 : (0 : Real) < (n / 2 : Nat) := by linarith
have hg0 : (0 : Real) < (n - n / 2 + m : Nat) := by
linarith [Nat.cast_nonneg (α := Real) m]
have hu : 0 < ((n / 2 : Nat) : Real) / 8 * eps := by positivity
have ht : 0 < ((n - n / 2 + m : Nat) : Real) / 8 * eps := by positivity
have hU : 1 / (((n / 2 : Nat) : Real) / 8 * eps) ≤ 24 / ((n : Real) * eps) := by
apply (div_le_div_iff₀ hu hS).2
nlinarith
have hT : 1 / (((n - n / 2 + m : Nat) : Real) / 8 * eps) ≤
16 / ((n : Real) * eps) := by
apply (div_le_div_iff₀ ht hS).2
nlinarith [Nat.cast_nonneg (α := Real) m]
have he : 1 ≤ exp (1 : Real) := by
exact Real.exp_zero.symm.trans_le (Real.exp_le_exp.mpr (by norm_num : (0 : Real) ≤ 1))
have hden : ((n : Real) + m) * eps ≤
16 * (exp 1 * ((n - n / 2 + m : Nat) : Real) / 8 * eps) := by
have hm := mul_le_mul_of_nonneg_right he ht.le
nlinarith
have hratio : (d : Real) / (exp 1 * ((n - n / 2 + m : Nat) : Real) / 8 * eps) ≤
16 * ((d : Real) / (((n : Real) + m) * eps)) := by
calc
_ = (16 * (d : Real)) /
(16 * (exp 1 * ((n - n / 2 + m : Nat) : Real) / 8 * eps)) := by ring
_ ≤ (16 * (d : Real)) / (((n : Real) + m) * eps) :=
div_le_div_of_nonneg_left (by positivity) hN hden
_ = _ := by ring
have hsq := pow_le_pow_left₀ (by positivity) hratio 2
have hov := baseline_pool_overflow_bound n m eps hn heps heps'
have hvar := mul_le_mul_of_nonneg_left (add_le_add hU hT) hC.le
have hi : 0 ≤ 1 / ((n : Real) * eps) := by positivity
have hs : 0 ≤ ((d : Real) / (((n : Real) + m) * eps)) ^ 2 := sq_nonneg _
have hCs := mul_nonneg hC.le hs
simp only [div_eq_mul_inv] at hvar hov hsq hi hs hCs ⊢
nlinarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.BaselinePoolEncoding 19 declarations Exact identifications of the baseline's array block counts with ordered-prefix histograms.
Exact identifications of the baseline's array block counts with ordered-prefix histograms. These identities provide the deterministic count bridge for finite-pool risk transfer.
An ordered block is represented as a finite sample without changing its order.
Definition (Lean source)
parameters
def
given by
⟨r, fun i => s ⟨offset + i.val, by omega⟩⟩
theorem baseline_block_histogram
Under the stated inputs and conditions, Histogram counts of an ordered block equal the corresponding array filter counts. This gives the stated result.
Formal statement
hypotheses
conclusion
= (Finset.univ.filter fun i => offset ≤ i.val ∧ i.val < offset + r ∧ s i = x).card
Proof (Lean source)
-- @node: baseline_block_histogram
lemma baseline_block_histogram {X : Type*} [MeasurableSpace X] [DecidableEq X] {n : Nat}
(s : Fin n → X) (offset r : Nat) (hr : offset + r ≤ n) (x : X) :
finiteSampleHistogram (baselineBlockPrefix s offset r hr).points x =
(Finset.univ.filter fun i => offset ≤ i.val ∧ i.val < offset + r ∧ s i = x).card := by
classical
rw [← Fintype.card_subtype]
unfold finiteSampleHistogram
change card {i : Fin r // s ⟨offset + i.val, by omega⟩ = x} = _
apply Fintype.card_congr
refine
{ toFun := fun i => ⟨⟨offset + i.1.val, by omega⟩, ?_⟩
invFun := fun i => ⟨⟨i.1.val - offset, ?_⟩, ?_⟩
left_inv := ?_
right_inv := ?_ }
· dsimp
exact ⟨by omega, by omega, i.2⟩
· have hi := i.2
omega
· have hi := i.2
simpa [baselineBlockPrefix, Nat.add_sub_of_le hi.1] using hi.2.2
· intro i
apply Subtype.ext
apply Fin.ext
simp
· intro i
apply Subtype.ext
apply Fin.ext
exact Nat.add_sub_of_le i.2.1
theorem baseline_labeled_block_histogram
Under the stated inputs and conditions, The baseline's complete-record block counter is the exact ordered-block histogram. This gives the stated result.
Formal statement
hypotheses
conclusion
= labeledBlockCount s offset r j a y
Proof (Lean source)
-- @node: baseline_labeled_block_histogram
lemma baseline_labeled_block_histogram {n d : Nat} (s : Fin n → Obs d)
(offset r : Nat) (hr : offset + r ≤ n) (j : Fin d) (a y : Bool) :
finiteSampleHistogram (baselineBlockPrefix s offset r hr).points (j,a,y) =
labeledBlockCount s offset r j a y :=
baseline_block_histogram s offset r hr (j,a,y)
theorem baseline_auxiliary_block_histogram
Under the stated inputs and conditions, The baseline's auxiliary block counter is the exact ordered-block histogram. This gives the stated result.
Formal statement
hypotheses
conclusion
= auxiliaryBlockCount s offset r j a
Proof (Lean source)
-- @node: baseline_auxiliary_block_histogram
lemma baseline_auxiliary_block_histogram {m d : Nat} (s : Fin m → AuxObs d)
(offset r : Nat) (hr : offset + r ≤ m) (j : Fin d) (a : Bool) :
finiteSampleHistogram (baselineBlockPrefix s offset r hr).points (j,a) =
auxiliaryBlockCount s offset r j a :=
baseline_block_histogram s offset r hr (j,a)
Under the stated inputs and conditions, A histogram can be evaluated as a sum of unit contributions over array positions. This gives the stated result.
Formal statement
hypotheses
conclusion
finiteSampleHistogram s x = ∑ i, if s i = x then 1 else 0
Proof (Lean source)
-- @node: baseline_histogram_sum
lemma baseline_histogram_sum {X : Type*} [DecidableEq X] {r : Nat}
(s : Fin r → X) (x : X) :
finiteSampleHistogram s x = ∑ i, if s i = x then 1 else 0 := by
classical
simp [finiteSampleHistogram, Fintype.card_subtype, Finset.sum_boole]
Under the stated inputs and conditions, Concatenating two arrays adds their histogram counts. This gives the stated result.
Formal statement
hypotheses
conclusion
finiteSampleHistogram (append s w) x
= finiteSampleHistogram s x + finiteSampleHistogram w x
Proof (Lean source)
-- @node: baseline_histogram_append
lemma baseline_histogram_append {X : Type*} [DecidableEq X] {r v : Nat}
(s : Fin r → X) (w : Fin v → X) (x : X) :
finiteSampleHistogram (append s w) x =
finiteSampleHistogram s x + finiteSampleHistogram w x := by
classical
simp only [baseline_histogram_sum]
rw [Fin.sum_univ_add]
simp only [Fin.append_left, Fin.append_right]
Under the stated inputs and conditions, Projecting away the binary outcome adds the two outcome-marked counts. This gives the stated result.
Formal statement
hypotheses
conclusion
finiteSampleHistogram (fun i => ((s i).1, (s i).2.1)) (j,a)
Proof (Lean source)
-- @node: baseline_projected_histogram
lemma baseline_projected_histogram {r d : Nat} (s : Fin r → Obs d)
(j : Fin d) (a : Bool) :
finiteSampleHistogram (fun i => ((s i).1, (s i).2.1)) (j,a) =
finiteSampleHistogram s (j,a,false) + finiteSampleHistogram s (j,a,true) := by
classical
simp only [baseline_histogram_sum, ← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro i _
obtain ⟨j', a', y⟩ := s i
cases y <;> simp
The baseline's auxiliary prefix concatenates projected complete records and auxiliary records in precisely the order used by the array implementation.
Definition (Lean source)
parameters
def
given by
l :
baselineBlockPrefix s.1 lo (min r size) hl
v :
baselineBlockPrefix s.2 ao (r - size) ha
⟨min r size
+ (r - size), append (fun i => ((l.points i).1, (l.points i).2.1)) v.points⟩
theorem baseline_auxiliary_prefix_count
Under the stated inputs and conditions, The concatenated auxiliary prefix has the requested length, including either empty part. This gives the stated result.
Formal statement
hypotheses
conclusion
(baselineAuxiliaryPrefix s lo size ao r hl ha).count = r
Proof (Lean source)
-- @node: baseline_auxiliary_prefix_count
lemma baseline_auxiliary_prefix_count {n m d : Nat} (s : Sample n m d)
(lo size ao r : Nat) (hl : lo + min r size ≤ n) (ha : ao + (r - size) ≤ m) :
(baselineAuxiliaryPrefix s lo size ao r hl ha).count = r := by
change min r size + (r - size) = r
omega
theorem baseline_pooled_arm_histogram
Under the stated inputs and conditions, The baseline's pooled arm counter is exactly the histogram of its concatenated prefix. This gives the stated result.
Formal statement
hypotheses
n m d :
s :
Sample n m d
lo size ao r :
hl :
lo + min r size ≤ n
ha :
ao + (r - size) ≤ m
j :
Fin d
a :
conclusion
= pooledArmCount s lo size ao r j a
Proof (Lean source)
-- @node: baseline_pooled_arm_histogram
lemma baseline_pooled_arm_histogram {n m d : Nat} (s : Sample n m d)
(lo size ao r : Nat) (hl : lo + min r size ≤ n) (ha : ao + (r - size) ≤ m)
(j : Fin d) (a : Bool) :
finiteSampleHistogram (baselineAuxiliaryPrefix s lo size ao r hl ha).points (j,a) =
pooledArmCount s lo size ao r j a := by
have hfalse := baseline_labeled_block_histogram s.1 lo (min r size) hl j a false
have htrue := baseline_labeled_block_histogram s.1 lo (min r size) hl j a true
have haux := baseline_auxiliary_block_histogram s.2 ao (r - size) ha j a
simp only [baselineBlockPrefix, FiniteSample.points, FiniteSample.count] at hfalse htrue haux
simp only [baselineAuxiliaryPrefix, baselineBlockPrefix, FiniteSample.points, FiniteSample.count]
rw [baseline_histogram_append, baseline_projected_histogram, hfalse, htrue, haux]
rfl
theorem baseline_prefix_statistic_eq_ordered
Under the stated inputs and conditions, The array baseline prefix statistic is exactly the clipped ordered-prefix statistic. The complete and auxiliary histogram identities retain every arm contribution. This gives the stated result.
Formal statement
hypotheses
n m d :
s :
Sample n m d
r v :
hr :
0 + r ≤ n
hl :
n / 2 + min v (n - n / 2) ≤ n
ha :
0 + (v - (n - n / 2)) ≤ m
conclusion
baselinePrefixStatistic r v s
= baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal) (baselineBlockPrefix s.1 0 r hr, baselineAuxiliaryPrefix s (n / 2) (n - n / 2) 0 v hl ha)
Proof (Lean source)
-- @node: baseline_prefix_statistic_eq_ordered
lemma baseline_prefix_statistic_eq_ordered {n m d : Nat} (s : Sample n m d)
(r v : Nat) (hr : 0 + r ≤ n)
(hl : n / 2 + min v (n - n / 2) ≤ n)
(ha : 0 + (v - (n - n / 2)) ≤ m) :
baselinePrefixStatistic r v s =
baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)
(baselineBlockPrefix s.1 0 r hr,
baselineAuxiliaryPrefix s (n / 2) (n - n / 2) 0 v hl ha) := by
have hZ (a : Bool) (j : Fin d) := baseline_labeled_block_histogram s.1 0 r hr j a true
have hK (a : Bool) (j : Fin d) :=
baseline_pooled_arm_histogram s (n / 2) (n - n / 2) 0 v hl ha j a
unfold baselineOrderedPrefixStatistic baselinePrefixCounts
simp only [Sum.elim_inl, Sum.elim_inr, hZ, hK,
NNReal.coe_div, NNReal.coe_natCast, NNReal.coe_ofNat]
simp only [baselinePrefixStatistic, Finset.sum_sub_distrib]
theorem baseline_truncated_poisson_integral Lemma baseline_truncated_poisson_integral in the paper ↗
Under the stated inputs and conditions, Integrating a zero-overflow two-prefix statistic gives exactly the finite product of Poisson weights. No continuity or nonzero cell mass is needed. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: baseline_truncated_poisson_integral
lemma baseline_truncated_poisson_integral (u t : NNReal) (h g : Nat)
(F : Nat → Nat → Real) :
(∫ k : Nat × Nat, if k.1 ≤ h ∧ k.2 ≤ g then F k.1 k.2 else 0
∂((poissonMeasure u).prod (poissonMeasure t))) =
∑ r ∈ range (h + 1), ∑ v ∈ range (g + 1),
exp (-(u : Real) - (t : Real)) *
((u : Real) ^ r / (r.factorial : Real)) *
((t : Real) ^ v / (v.factorial : Real)) * F r v := by
classical
let s := (range (h + 1)).product (range (g + 1))
have hind : (fun k : Nat × Nat => if k.1 ≤ h ∧ k.2 ≤ g then F k.1 k.2 else 0) =
(s : Set (Nat × Nat)).indicator (fun k => F k.1 k.2) := by
funext k
simp [s, indicator, Nat.lt_succ_iff]
rw [hind, integral_indicator s.measurableSet, setIntegral_finset s IntegrableOn.finset]
dsimp only [s]
simp only [Finset.product_eq_sprod, Finset.sum_product]
apply Finset.sum_congr rfl
intro r _
apply Finset.sum_congr rfl
intro v _
have hmass : ((poissonMeasure u).prod (poissonMeasure t)).real {(r,v)} =
(poissonMeasure u).real {r} * (poissonMeasure t).real {v} := by
simp only [Measure.real_def, ← Set.singleton_prod_singleton,
Measure.prod_prod, ENNReal.toReal_mul]
rw [smul_eq_mul, hmass, poissonMeasure_real_singleton, poissonMeasure_real_singleton]
rw [sub_eq_add_neg, Real.exp_add]
ring
theorem baseline_estimator_eq_poisson_integral
Under the stated inputs and conditions, The baseline's finite average is the expectation of its array prefix statistic with exactly zero output when either Poisson request exceeds its pool. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
conclusion 2
k.2
≤ n
- n / 2
+ m then baselinePrefixStatistic k.1 k.2 s else 0 ∂((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod (poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal)))
Proof (Lean source)
-- @node: baseline_estimator_eq_poisson_integral
lemma baseline_estimator_eq_poisson_integral {n m d : Nat} (s : Sample n m d)
(hn : 6 ≤ n) :
baselineEstimator n m d s =
∫ k : Nat × Nat,
if k.1 ≤ n / 2 ∧ k.2 ≤ n - n / 2 + m then
baselinePrefixStatistic k.1 k.2 s else 0
∂((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod
(poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal))) := by
rw [baseline_truncated_poisson_integral _ _ _ _
(fun r v => baselinePrefixStatistic r v s)]
simp only [baselineEstimator, if_neg (show ¬ n < 6 by omega),
NNReal.coe_div, NNReal.coe_natCast, NNReal.coe_ofNat]
On the original arrays, either requested prefix overflow resets the baseline to zero.
Definition (Lean source)
parameters
def
given by
if k.1 ≤ n / 2 ∧ k.2 ≤ n - n / 2 + m then baselinePrefixStatistic k.1 k.2 s else 0
theorem baseline_capped_array_mem_Icc
Under the stated inputs and conditions, Every original-array capped statistic lies in the clipping interval. This gives the stated result.
Formal statement
hypotheses
conclusion
baselineCappedArrayStatistic s k ∈ Icc (-1) 1
Proof (Lean source)
-- @node: baseline_capped_array_mem_Icc
lemma baseline_capped_array_mem_Icc {n m d : Nat} (s : Sample n m d) (k : Nat × Nat) :
baselineCappedArrayStatistic s k ∈ Icc (-1) 1 := by
unfold baselineCappedArrayStatistic
split_ifs
· exact ⟨le_max_left _ _, max_le (by norm_num) (min_le_left _ _)⟩
· norm_num
theorem baseline_averaging_sq_loss_le
Under the stated inputs and conditions, Conditional Poisson averaging contracts the squared loss at every original array. This gives the stated result.
Formal statement
hypotheses
conclusion
(baselineEstimator n m d s - theta) ^ 2
≤ ∫ k : Nat × Nat, (baselineCappedArrayStatistic s k - theta) ^ 2 ∂((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod (poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal)))
Proof (Lean source)
-- @node: baseline_averaging_sq_loss_le
lemma baseline_averaging_sq_loss_le {n m d : Nat} (s : Sample n m d)
(hn : 6 ≤ n) (theta : Real) :
(baselineEstimator n m d s - theta) ^ 2 ≤
∫ k : Nat × Nat, (baselineCappedArrayStatistic s k - theta) ^ 2
∂((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod
(poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal))) := by
let nu := (poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod
(poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal))
have hm : Measurable (baselineCappedArrayStatistic s) := by fun_prop
have hb : UniformlyBounded (baselineCappedArrayStatistic s) :=
⟨1, by norm_num, fun k => abs_le.mpr (baseline_capped_array_mem_Icc s k)⟩
have hj := sqLoss_kernelMean_le (Kernel.const Unit nu) hm hb theta ()
rw [baseline_estimator_eq_poisson_integral s hn]
simpa only [kernelMean, Kernel.const_apply, baselineCappedArrayStatistic,
nu] using hj
theorem baseline_averaging_sq_risk_le
Under the stated inputs and conditions, Conditional averaging also contracts squared risk under the original experiment's finite-data distribution. This step does not require a pool-law identification. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ sqRisk ((annotationLaw P n m).prod ((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod (poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal)))) (fun z => baselineCappedArrayStatistic z.1 z.2) (ateFunctional P)
Proof (Lean source)
-- @node: baseline_averaging_sq_risk_le
lemma baseline_averaging_sq_risk_le {n m d : Nat} (P : DiscreteLaw d) (hn : 6 ≤ n) :
sqRisk (annotationLaw P n m) (baselineEstimator n m d) (ateFunctional P) ≤
sqRisk ((annotationLaw P n m).prod
((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod
(poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal))))
(fun z => baselineCappedArrayStatistic z.1 z.2) (ateFunctional P) := by
let nu := (poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod
(poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal))
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have htheta := ateFunctional_mem_Icc P
have hm : Measurable (fun z : Sample n m d × (Nat × Nat) =>
(baselineCappedArrayStatistic z.1 z.2 - ateFunctional P) ^ 2) := by fun_prop
have hi : Integrable (fun z : Sample n m d × (Nat × Nat) =>
(baselineCappedArrayStatistic z.1 z.2 - ateFunctional P) ^ 2)
((annotationLaw P n m).prod nu) := by
apply Integrable.of_bound hm.aestronglyMeasurable 4
filter_upwards with z
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hc := baseline_capped_array_mem_Icc z.1 z.2
nlinarith [hc.1, hc.2, htheta.1, htheta.2]
unfold sqRisk
rw [integral_prod _ hi]
exact integral_mono_of_nonneg
(Filter.Eventually.of_forall (fun _ => sq_nonneg _)) hi.integral_prod_left
(Filter.Eventually.of_forall (fun s => baseline_averaging_sq_loss_le s hn _))
theorem baseline_ruleRisk_eq_sqRisk
Under the stated inputs and conditions, The independent uniform seed has no effect on the risk of the data-only baseline. This gives the stated result.
Formal statement
hypotheses
n m d :
P :
conclusion
Proof (Lean source)
-- @node: baseline_ruleRisk_eq_sqRisk
lemma baseline_ruleRisk_eq_sqRisk (n m d : Nat) (P : DiscreteLaw d) :
ruleRisk (liftRule (baselineEstimator n m d)) P =
sqRisk (annotationLaw P n m) (baselineEstimator n m d) (ateFunctional P) := by
let : IsProbabilityMeasure seedLaw := ⟨by
norm_num [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have hm : Measurable (fun z : Sample n m d × Real =>
(baselineEstimator n m d z.1 - ateFunctional P) ^ 2) := by fun_prop
have hi : Integrable (fun z : Sample n m d × Real =>
(baselineEstimator n m d z.1 - ateFunctional P) ^ 2)
((annotationLaw P n m).prod seedLaw) := by
apply Integrable.of_bound hm.aestronglyMeasurable 4
filter_upwards with z
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hc := baselineEstimator_mem_Icc n m d z.1
have ht := ateFunctional_mem_Icc P
nlinarith [hc.1, hc.2, ht.1, ht.2]
unfold ruleRisk liftRule sqRisk
rw [integral_prod _ hi]
simp
theorem baseline_ruleRisk_le_capped_array_risk
Under the stated inputs and conditions, The original randomized-rule risk is bounded by the loss of the zero-overflow prefix statistic before averaging over its independent Poisson requests. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ sqRisk ((annotationLaw P n m).prod ((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod (poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal)))) (fun z => baselineCappedArrayStatistic z.1 z.2) (ateFunctional P)
Proof (Lean source)
-- @node: baseline_ruleRisk_le_capped_array_risk
lemma baseline_ruleRisk_le_capped_array_risk {n m d : Nat} (P : DiscreteLaw d)
(hn : 6 ≤ n) :
ruleRisk (liftRule (baselineEstimator n m d)) P ≤
sqRisk ((annotationLaw P n m).prod
((poissonMeasure ((n / 2 : Nat) / 8 : NNReal)).prod
(poissonMeasure ((n - n / 2 + m : Nat) / 8 : NNReal))))
(fun z => baselineCappedArrayStatistic z.1 z.2) (ateFunctional P) := by
rw [baseline_ruleRisk_eq_sqRisk]
exact baseline_averaging_sq_risk_le P hn
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.BaselinePoolLaw 8 declarations Exact iid laws for splitting the complete-record array and concatenating its remaining treatment--covariate projections with the auxiliary array.
Exact iid laws for splitting the complete-record array and concatenating its remaining treatment--covariate projections with the auxiliary array.
Under the stated inputs and conditions, Splitting an iid array into its first and last blocks gives independent iid pools. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: baseline_split_iid_law
lemma baseline_split_iid_law {X : Type*} [MeasurableSpace X]
(P : Measure X) [IsProbabilityMeasure P] (h k : Nat) :
MeasurePreserving
(fun s : Fin (h + k) → X =>
(fun i : Fin h => s (castAdd k i), fun i : Fin k => s (natAdd h i)))
(Measure.pi (fun _ : Fin (h + k) => P))
((Measure.pi (fun _ : Fin h => P)).prod (Measure.pi (fun _ : Fin k => P))) := by
have hr := (measurePreserving_piCongrLeft
(fun _ : Fin (h + k) => P) (finSumFinEquiv : Fin h ⊕ Fin k ≃ Fin (h + k))).symm
have hs := measurePreserving_sumPiEquivProdPi (fun _ : Fin h ⊕ Fin k => P)
convert hs.comp hr using 1
funext s
apply Prod.ext <;> funext i <;> rfl
Under the stated hypotheses, The split law also holds when the total capacity is expressed by an equal index. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: baseline_split_iid_law_of_eq
lemma baseline_split_iid_law_of_eq {X : Type*} [MeasurableSpace X]
(P : Measure X) [IsProbabilityMeasure P] (h k n : Nat) (hn : n = h + k) :
MeasurePreserving
(fun s : Fin n → X =>
(fun i : Fin h => s ⟨i.val, by omega⟩,
fun i : Fin k => s ⟨h + i.val, by omega⟩))
(Measure.pi (fun _ : Fin n => P))
((Measure.pi (fun _ : Fin h => P)).prod (Measure.pi (fun _ : Fin k => P))) := by
subst n
exact baseline_split_iid_law P h k
Under the stated inputs and conditions, Concatenating independent iid arrays of the same law gives one iid array. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: baseline_append_iid_law
lemma baseline_append_iid_law {X : Type*} [MeasurableSpace X]
(P : Measure X) [IsProbabilityMeasure P] (h k : Nat) :
MeasurePreserving (fun s : (Fin h → X) × (Fin k → X) => append s.1 s.2)
((Measure.pi (fun _ : Fin h => P)).prod (Measure.pi (fun _ : Fin k => P)))
(Measure.pi (fun _ : Fin (h + k) => P)) := by
have hr := measurePreserving_piCongrLeft
(fun _ : Fin (h + k) => P) (finSumFinEquiv : Fin h ⊕ Fin k ≃ Fin (h + k))
have hs := measurePreserving_sumPiEquivProdPi_symm (fun _ : Fin h ⊕ Fin k => P)
convert hr.comp hs using 1
funext s i
refine Fin.addCases ?_ ?_ i
· intro j
simp [MeasurableEquiv.piCongrLeft, Equiv.piCongrLeft, Equiv.piCongrLeft']
rfl
· intro j
simp [MeasurableEquiv.piCongrLeft, Equiv.piCongrLeft, Equiv.piCongrLeft']
rfl
theorem baseline_project_iid_law
Under the stated inputs and conditions, Dropping the outcome from an iid complete-record block gives the iid marginal block. This gives the stated result.
Formal statement
hypotheses
conclusion
MeasurePreserving (fun s : Fin k → Obs d => fun i => ((s i).1, (s i).2.1)) (labeledProductLaw P k) (auxProductLaw P k)
Proof (Lean source)
-- @node: baseline_project_iid_law
lemma baseline_project_iid_law {d : Nat} (P : DiscreteLaw d) (k : Nat) :
MeasurePreserving (fun s : Fin k → Obs d => fun i => ((s i).1, (s i).2.1))
(labeledProductLaw P k) (auxProductLaw P k) := by
unfold labeledProductLaw auxProductLaw
apply measurePreserving_pi (fun _ : Fin k => obsLaw P)
(fun _ => (auxMarginal P).toMeasure)
(f := fun _ z => (z.1, z.2.1))
intro i
refine ⟨by fun_prop, ?_⟩
exact PMF.toMeasure_map (fun z : Obs d => (z.1, z.2.1)) P.pmf (by fun_prop)
The baseline pools preserve the original order: the first block is complete, and the second block concatenates the remaining projections and auxiliary records.
theorem baselineFixedPools_measurable
Extracting the two disjoint baseline pools is measurable. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: baselineFixedPools_measurable
lemma baselineFixedPools_measurable (n m d : Nat) :
Measurable (baselineFixedPools (n := n) (m := m) (d := d)) := by
unfold baselineFixedPools
apply prodMk
· fun_prop
· apply measurable_pi_lambda
intro i
refine Fin.addCases ?_ ?_ i
· intro j
simp only [Fin.append_left]
fun_prop
· intro j
simp only [Fin.append_right]
fun_prop
theorem baseline_fixed_pools_law
Under the stated inputs and conditions, Under the original two-channel experiment, the extracted pools are independent with exactly the complete-record and treatment--covariate iid laws. This gives the stated result.
Formal statement
hypotheses
n m d :
P :
conclusion
MeasurePreserving (baselineFixedPools (n := n) (m := m) (d := d)) (annotationLaw P n m) ((labeledProductLaw P (n / 2)).prod (auxProductLaw P (n - n / 2 + m)))
Proof (Lean source)
-- @node: baseline_fixed_pools_law
lemma baseline_fixed_pools_law {n m d : Nat} (P : DiscreteLaw d) :
MeasurePreserving (baselineFixedPools (n := n) (m := m) (d := d))
(annotationLaw P n m)
((labeledProductLaw P (n / 2)).prod (auxProductLaw P (n - n / 2 + m))) := by
have hsplit : MeasurePreserving
(fun s : Fin n → Obs d =>
(fun i : Fin (n / 2) => s ⟨i.val, by omega⟩,
fun i : Fin (n - n / 2) => s ⟨n / 2 + i.val, by omega⟩))
(labeledProductLaw P n)
((labeledProductLaw P (n / 2)).prod (labeledProductLaw P (n - n / 2))) := by
exact baseline_split_iid_law_of_eq (obsLaw P) (n / 2) (n - n / 2) n (by omega)
have hproj := baseline_project_iid_law P (n - n / 2)
have happ := baseline_append_iid_law (auxMarginal P).toMeasure (n - n / 2) m
have hfirst := hsplit.prod (MeasurePreserving.id (auxProductLaw P m))
have hassoc := measurePreserving_prodAssoc
(labeledProductLaw P (n / 2)) (labeledProductLaw P (n - n / 2)) (auxProductLaw P m)
have hlast := (MeasurePreserving.id (labeledProductLaw P (n / 2))).prod
(happ.comp (hproj.prod (MeasurePreserving.id (auxProductLaw P m))))
convert hlast.comp (hassoc.comp hfirst) using 1
· funext s
rfl
· rfl
· rfl
theorem baseline_fixed_pools_sqRisk
Under the stated inputs and conditions, Squared risk of a measurable statistic of the extracted baseline pools is exactly its risk in the independent fixed-pool experiment. This gives the stated result.
Formal statement
hypotheses
n m d :
P :
hT :
theta :
conclusion
Proof (Lean source)
-- @node: baseline_fixed_pools_sqRisk
lemma baseline_fixed_pools_sqRisk {n m d : Nat} (P : DiscreteLaw d)
(T : ((Fin (n / 2) → Obs d) × (Fin (n - n / 2 + m) → AuxObs d)) → Real)
(hT : Measurable T) (theta : Real) :
sqRisk (annotationLaw P n m) (T ∘ baselineFixedPools) theta =
sqRisk
((labeledProductLaw P (n / 2)).prod (auxProductLaw P (n - n / 2 + m)))
T theta := by
unfold sqRisk
rw [← (baseline_fixed_pools_law P).map_eq]
exact (integral_map (baselineFixedPools_measurable n m d).aemeasurable
((hT.sub measurable_const).pow_const 2).aestronglyMeasurable).symm
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.BaselinePoolStatistic 7 declarations Deterministic identification of the array baseline with the fixed-pool prefix rule.
Deterministic identification of the array baseline with the fixed-pool prefix rule.
Under the stated hypotheses, Prefixing an appended array retains the first block followed by the required part of the second block, also when the prefix stops inside the first block. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: baseline_prefix_append_eq
lemma baseline_prefix_append_eq {X : Type*} [MeasurableSpace X] {h m : Nat}
(s : Fin h → X) (v : Fin m → X) (r : Nat) (hr : r ≤ h + m) :
prefixOfLE (append s v) r hr =
⟨min r h + (r - h), append
(fun i : Fin (min r h) => s ⟨i.val, by omega⟩)
(fun i : Fin (r - h) => v ⟨i.val, by omega⟩)⟩ := by
have hc : r = min r h + (r - h) := by omega
apply Sigma.ext hc
apply (Fin.heq_fun_iff hc).2
intro i
simp only [prefixOfLE]
by_cases hi : i.val < min r h
· have hi' : i.val < h := by omega
simp [append, Fin.addCases, hi, hi']
· have hi' : ¬ i.val < h := by omega
have hmin : min r h = h := by omega
simp [append, Fin.addCases, hi', hmin]
theorem baseline_fixed_complete_prefix_eq
Under the stated inputs and conditions, The complete prefix of the extracted pool is the first original-array block. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: baseline_fixed_complete_prefix_eq
lemma baseline_fixed_complete_prefix_eq {n m d : Nat} (s : Sample n m d)
(r : Nat) (hr : r ≤ n / 2) :
prefixOfLE (baselineFixedPools s).1 r hr =
baselineBlockPrefix s.1 0 r (by omega) := by
simp [prefixOfLE, baselineFixedPools, baselineBlockPrefix]
theorem baseline_fixed_auxiliary_prefix_eq
Under the stated inputs and conditions, The marginal prefix of the extracted pool is the prescribed projected block concatenated with the prescribed auxiliary block. This gives the stated result.
Formal statement
hypotheses
conclusion
prefixOfLE (baselineFixedPools s).2 v hv
= baselineAuxiliaryPrefix s (n / 2) (n - n / 2) 0 v (by omega) (by omega)
Proof (Lean source)
-- @node: baseline_fixed_auxiliary_prefix_eq
lemma baseline_fixed_auxiliary_prefix_eq {n m d : Nat} (s : Sample n m d)
(v : Nat) (hv : v ≤ n - n / 2 + m) :
prefixOfLE (baselineFixedPools s).2 v hv =
baselineAuxiliaryPrefix s (n / 2) (n - n / 2) 0 v (by omega) (by omega) := by
simpa [baselineFixedPools, baselineAuxiliaryPrefix, baselineBlockPrefix,
FiniteSample.points, FiniteSample.count] using
baseline_prefix_append_eq
(fun i : Fin (n - n / 2) =>
((s.1 ⟨n / 2 + i.val, by omega⟩).1,
(s.1 ⟨n / 2 + i.val, by omega⟩).2.1)) s.2 v hv
theorem baseline_fixed_prefix_statistic_eq
Under the stated inputs and conditions, Every valid original-array prefix computes the fixed-pool statistic exactly. This gives the stated result.
Formal statement
hypotheses
conclusion
baselinePrefixStatistic r v s
= baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal) (prefixOfLE (baselineFixedPools s).1 r hr, prefixOfLE (baselineFixedPools s).2 v hv)
Proof (Lean source)
-- @node: baseline_fixed_prefix_statistic_eq
lemma baseline_fixed_prefix_statistic_eq {n m d : Nat} (s : Sample n m d)
(r v : Nat) (hr : r ≤ n / 2) (hv : v ≤ n - n / 2 + m) :
baselinePrefixStatistic r v s =
baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)
(prefixOfLE (baselineFixedPools s).1 r hr,
prefixOfLE (baselineFixedPools s).2 v hv) := by
rw [baseline_fixed_complete_prefix_eq, baseline_fixed_auxiliary_prefix_eq]
exact baseline_prefix_statistic_eq_ordered s r v (by omega) (by omega) (by omega)
theorem baseline_capped_array_eq_fixed
Under the stated inputs and conditions, Overflow is zero in both implementations, and valid prefixes agree exactly. This gives the stated result.
Formal statement
hypotheses
conclusion
= cappedPrefixStatistic (baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)) 0 (((baselineFixedPools s).1, k.1), ((baselineFixedPools s).2, k.2))
Proof (Lean source)
-- @node: baseline_capped_array_eq_fixed
lemma baseline_capped_array_eq_fixed {n m d : Nat} (s : Sample n m d) (k : Nat × Nat) :
baselineCappedArrayStatistic s k =
cappedPrefixStatistic (baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal))
0 (((baselineFixedPools s).1, k.1), ((baselineFixedPools s).2, k.2)) := by
by_cases hr : k.1 ≤ n / 2
· by_cases hv : k.2 ≤ n - n / 2 + m
· simp only [baselineCappedArrayStatistic, hr, hv, and_self, ite_true,
cappedPrefixStatistic, dif_pos]
exact baseline_fixed_prefix_statistic_eq s k.1 k.2 hr hv
· simp [baselineCappedArrayStatistic, cappedPrefixStatistic, hr, hv]
· simp [baselineCappedArrayStatistic, cappedPrefixStatistic, hr]
theorem baseline_estimator_eq_fixed_average
Under the stated inputs and conditions, The deterministic baseline is exactly the prefix average of its extracted pools. This gives the stated result.
Formal statement
hypotheses
conclusion
baselineEstimator n m d s
= independentPrefixRaoBlackwellStatistic ((n / 2 : Nat) / 8 : NNReal) ((n - n / 2 + m : Nat) / 8 : NNReal) (baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)) 0 (baselineFixedPools s)
Proof (Lean source)
-- @node: baseline_estimator_eq_fixed_average
lemma baseline_estimator_eq_fixed_average {n m d : Nat} (s : Sample n m d)
(hn : 6 ≤ n) :
baselineEstimator n m d s =
independentPrefixRaoBlackwellStatistic ((n / 2 : Nat) / 8 : NNReal)
((n - n / 2 + m : Nat) / 8 : NNReal)
(baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)) 0
(baselineFixedPools s) := by
rw [independentPrefixRaoBlackwellStatistic_eq_integral _ _ _
(baselineOrderedPrefixStatistic_measurable d _) 0,
baseline_estimator_eq_poisson_integral s hn]
apply integral_congr_ae
exact Filter.Eventually.of_forall (fun k => baseline_capped_array_eq_fixed s k)
theorem baseline_ruleRisk_eq_fixed_average
Under the stated inputs and conditions, The original rule risk equals the independent fixed-pool prefix-average risk. This gives the stated result.
Formal statement
hypotheses
conclusion
= sqRisk (fixedPoolsLaw (obsLaw P) (auxMarginal P).toMeasure (n / 2) (n - n / 2 + m)) (independentPrefixRaoBlackwellStatistic ((n / 2 : Nat) / 8 : NNReal) ((n - n / 2 + m : Nat) / 8 : NNReal) (baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)) 0) (ateFunctional P)
Proof (Lean source)
-- @node: baseline_ruleRisk_eq_fixed_average
lemma baseline_ruleRisk_eq_fixed_average {n m d : Nat} (P : DiscreteLaw d)
(hn : 6 ≤ n) :
ruleRisk (liftRule (baselineEstimator n m d)) P =
sqRisk
(fixedPoolsLaw (obsLaw P) (auxMarginal P).toMeasure
(n / 2) (n - n / 2 + m))
(independentPrefixRaoBlackwellStatistic ((n / 2 : Nat) / 8 : NNReal)
((n - n / 2 + m : Nat) / 8 : NNReal)
(baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)) 0)
(ateFunctional P) := by
rw [baseline_ruleRisk_eq_sqRisk]
have heq : baselineEstimator n m d =
(independentPrefixRaoBlackwellStatistic ((n / 2 : Nat) / 8 : NNReal)
((n - n / 2 + m : Nat) / 8 : NNReal)
(baselineOrderedPrefixStatistic ((n / 2 : Nat) / 8 : NNReal)) 0) ∘
baselineFixedPools := by
funext s
exact baseline_estimator_eq_fixed_average s hn
rw [heq, baseline_fixed_pools_sqRisk P _ (by fun_prop)]
rfl
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.BaselinePrefixCounts 14 declarations Exact thinning laws for the two independent ordered Poisson pools of the baseline.
Exact thinning laws for the two independent ordered Poisson pools of the baseline. The complete-record and auxiliary histograms are joined before selecting arm counts.
instance baseline_obsLaw_probability
The complete-record law is a probability measure.
Definition (Lean source)
-- @node: baseline_obsLaw_probability
instance baseline_obsLaw_probability {d : Nat} (P : DiscreteLaw d) :
IsProbabilityMeasure (obsLaw P) := by
unfold obsLaw
infer_instance
theorem baseline_finiteSampleHistogram_measurable Lemma baseline_finiteSampleHistogram_measurable in the paper ↗
The histogram of an ordered finite sample on a finite alphabet is measurable. This gives the stated conclusion.
Formal statement
hypotheses
X :
conclusion
Proof (Lean source)
@[fun_prop]
-- @node: baseline_finiteSampleHistogram_measurable
lemma baseline_finiteSampleHistogram_measurable {X : Type*} [Finite X]
[MeasurableSpace X] [MeasurableSingletonClass X] [DecidableEq X] :
Measurable (fun s : FiniteSample X => finiteSampleHistogram s.points) := by
apply measurable_to_countable'
intro c
rw [MeasurableSpace.measurableSet_iInf]
intro n
change MeasurableSet {x : Fin n → X | finiteSampleHistogram x = c}
exact Set.Finite.measurableSet (Set.toFinite _)
The full histogram of both pools, with disjoint indices for the two channels.
Definition (Lean source)
parameters
d :
s :
given by
theorem baselinePrefixCounts_measurable
Both ordered finite alphabets make the joint histogram measurable. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: baselinePrefixCounts_measurable
lemma baselinePrefixCounts_measurable (d : Nat) :
Measurable (baselinePrefixCounts (d := d)) := by
change Measurable ((MeasurableEquiv.sumPiEquivProdPi (fun _ : Obs d ⊕ AuxObs d => Nat)).symm ∘
(fun s : FiniteSample (Obs d) × FiniteSample (AuxObs d) =>
(finiteSampleHistogram s.1.points, finiteSampleHistogram s.2.points)))
exact (MeasurableEquiv.sumPiEquivProdPi _).symm.measurable.comp
((baseline_finiteSampleHistogram_measurable.comp measurable_fst).prodMk
(baseline_finiteSampleHistogram_measurable.comp measurable_snd))
theorem baseline_prefix_counts_law
Under the stated inputs and conditions, Thinning the independent ordered prefixes gives independent Poisson counts, including zero-mass atoms, jointly across the two channels. This gives the stated result.
Formal statement
hypotheses
conclusion
= Measure.pi (fun i : Obs d ⊕ AuxObs d => poissonMeasure (elim (fun z => u * ((obsLaw P) {z}).toNNReal) (fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal) i))
Proof (Lean source)
-- @node: baseline_prefix_counts_law
lemma baseline_prefix_counts_law {d : Nat} (P : DiscreteLaw d) (u t : NNReal) :
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map
baselinePrefixCounts =
Measure.pi (fun i : Obs d ⊕ AuxObs d => poissonMeasure
(elim (fun z => u * ((obsLaw P) {z}).toNNReal)
(fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal) i)) := by
have hleft : Measurable (fun s : FiniteSample (Obs d) => finiteSampleHistogram s.points) :=
baseline_finiteSampleHistogram_measurable
have hright : Measurable (fun s : FiniteSample (AuxObs d) => finiteSampleHistogram s.points) :=
baseline_finiteSampleHistogram_measurable
let rates : Obs d ⊕ AuxObs d → NNReal :=
elim (fun z => u * ((obsLaw P) {z}).toNNReal)
(fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal)
have hjoint := Measure.map_prod_map
(finitePoissonSampleLaw (obsLaw P) u)
(finitePoissonSampleLaw (auxMarginal P).toMeasure t) hleft hright
rw [finitePoissonSampleLaw_map_histogram, finitePoissonSampleLaw_map_histogram] at hjoint
have hsplit := (measurePreserving_sumPiEquivProdPi_symm
(fun i => poissonMeasure (rates i))).map_eq
rw [← hsplit]
change _ = Measure.map (MeasurableEquiv.sumPiEquivProdPi
(fun _ : Obs d ⊕ AuxObs d => Nat)).symm
((independentPoissonCountLaw (obsLaw P) u).prod
(independentPoissonCountLaw (auxMarginal P).toMeasure t))
rw [hjoint, Measure.map_map (by fun_prop) (hleft.prodMap hright)]
rfl
theorem baseline_prefix_count_marginal
Under the stated inputs and conditions, Each coordinate has its exact Poisson marginal under the joint prefix law. This gives the stated result.
Formal statement
hypotheses
conclusion
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map (fun s => baselinePrefixCounts s i)
= poissonMeasure (elim (fun z => u * ((obsLaw P) {z}).toNNReal) (fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal) i)
Proof (Lean source)
-- @node: baseline_prefix_count_marginal
lemma baseline_prefix_count_marginal {d : Nat} (P : DiscreteLaw d) (u t : NNReal)
(i : Obs d ⊕ AuxObs d) :
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map
(fun s => baselinePrefixCounts s i) = poissonMeasure
(elim (fun z => u * ((obsLaw P) {z}).toNNReal)
(fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal) i) := by
rw [show (fun s => baselinePrefixCounts s i) =
(fun c : Obs d ⊕ AuxObs d → Nat => c i) ∘ baselinePrefixCounts by rfl,
← Measure.map_map (by fun_prop) (baselinePrefixCounts_measurable d),
baseline_prefix_counts_law]
exact (measurePreserving_eval _ i).map_eq
theorem baseline_prefix_counts_independent
Under the stated inputs and conditions, The full collection of complete and auxiliary atom counts is independent. This gives the stated result.
Formal statement
hypotheses
conclusion
iIndepFun (fun i s => baselinePrefixCounts s i) (independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t)
Proof (Lean source)
-- @node: baseline_prefix_counts_independent
lemma baseline_prefix_counts_independent {d : Nat} (P : DiscreteLaw d) (u t : NNReal) :
iIndepFun (fun i s => baselinePrefixCounts s i)
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t) := by
let : IsProbabilityMeasure (independentPoissonPrefixLaw
(obsLaw P) (auxMarginal P).toMeasure u t) := by
unfold independentPoissonPrefixLaw
infer_instance
apply (iIndepFun_iff_map_fun_eq_pi_map
(fun i => ((measurable_pi_apply i).comp (baselinePrefixCounts_measurable d)).aemeasurable)).2
simp only [Function.comp_def]
simp_rw [baseline_prefix_count_marginal]
exact baseline_prefix_counts_law P u t
theorem baseline_prefix_arm_counts_independent
Under the stated inputs and conditions, Selecting one arm's success counts and both marginal arm counts preserves independence. This gives the stated result.
Formal statement
hypotheses
conclusion
iIndepFun (fun i : Fin 3 × Fin d => fun s => baselinePrefixCounts s (if i.1 = 0 then inl (i.2, a, true) else if i.1 = 1 then inr (i.2, a) else inr (i.2, !a))) (independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t)
Proof (Lean source)
-- @node: baseline_prefix_arm_counts_independent
lemma baseline_prefix_arm_counts_independent {d : Nat} (P : DiscreteLaw d)
(u t : NNReal) (a : Bool) :
iIndepFun (fun i : Fin 3 × Fin d => fun s => baselinePrefixCounts s
(if i.1 = 0 then inl (i.2, a, true)
else if i.1 = 1 then inr (i.2, a) else inr (i.2, !a)))
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t) := by
apply (baseline_prefix_counts_independent P u t).precomp
rintro ⟨i,j⟩ ⟨i',j'⟩ h
fin_cases i <;> fin_cases i' <;> cases a <;> simp_all
theorem baseline_prefix_success_count_law
Under the stated inputs and conditions, A complete-prefix success count has the exact marked-mass Poisson mean. This gives the stated result.
Formal statement
hypotheses
conclusion
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map (fun s => baselinePrefixCounts s (inl (j,a,true)))
Proof (Lean source)
-- @node: baseline_prefix_success_count_law
lemma baseline_prefix_success_count_law {d : Nat} (P : DiscreteLaw d)
(u t : NNReal) (j : Fin d) (a : Bool) :
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map
(fun s => baselinePrefixCounts s (inl (j,a,true))) =
poissonMeasure (u * toNNReal (markedMass P j a)) := by
rw [baseline_prefix_count_marginal]
simp [obsLaw, markedMass, jointMass]
theorem baseline_prefix_arm_count_law
Under the stated inputs and conditions, An auxiliary-prefix count has the exact arm-mass Poisson mean. This gives the stated result.
Formal statement
hypotheses
conclusion
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map (fun s => baselinePrefixCounts s (inr (j,a)))
Proof (Lean source)
-- @node: baseline_prefix_arm_count_law
lemma baseline_prefix_arm_count_law {d : Nat} (P : DiscreteLaw d)
(u t : NNReal) (j : Fin d) (a : Bool) :
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t).map
(fun s => baselinePrefixCounts s (inr (j,a))) =
poissonMeasure (t * toNNReal (armMass P j a)) := by
rw [baseline_prefix_count_marginal]
simp [← auxMarginal_toReal_armMass]
The clipped inverse-count contrast on the two untruncated ordered prefixes.
Definition (Lean source)
parameters
def
theorem baselineOrderedPrefixStatistic_measurable
The ordered-prefix statistic is measurable. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: baselineOrderedPrefixStatistic_measurable
lemma baselineOrderedPrefixStatistic_measurable (d : Nat) (u : NNReal) :
Measurable (baselineOrderedPrefixStatistic (d := d) u) := by
unfold baselineOrderedPrefixStatistic
fun_prop
theorem baselineOrderedPrefixStatistic_mem_Icc
Under the stated inputs and conditions, The ordered-prefix statistic takes values in the prescribed interval. This gives the stated result.
Formal statement
hypotheses
conclusion
baselineOrderedPrefixStatistic u s ∈ Icc (-1) 1
Proof (Lean source)
-- @node: baselineOrderedPrefixStatistic_mem_Icc
lemma baselineOrderedPrefixStatistic_mem_Icc {d : Nat} (u : NNReal)
(s : FiniteSample (Obs d) × FiniteSample (AuxObs d)) :
baselineOrderedPrefixStatistic u s ∈ Icc (-1) 1 := by
exact ⟨le_max_left _ _, max_le (by norm_num) (min_le_left _ _)⟩
theorem baseline_ordered_prefix_risk
Under the stated inputs and conditions, The ordered Poisson prefixes satisfy the baseline contrast risk bound by thinning, the independent-cell moment bounds, and clipping. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
sqRisk (independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t) (baselineOrderedPrefixStatistic u) (ateFunctional P)
Proof (Lean source)
-- @node: baseline_ordered_prefix_risk
lemma baseline_ordered_prefix_risk :
∃ C : Real, 0 < C ∧ ∀ (d : Nat) (eps : Real) (P : DiscreteLaw d) (u t : NNReal),
2 ≤ d → 0 < eps → eps ≤ 1 / 4 → ModelClass d eps P → 0 < u → 0 < t →
sqRisk
(independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t)
(baselineOrderedPrefixStatistic u) (ateFunctional P) ≤
C * (1 / ((u : Real) * eps) + 1 / ((t : Real) * eps)) +
4 * ((d : Real) / (exp 1 * (t : Real) * eps)) ^ 2 := by
obtain ⟨C,hC,hbound⟩ := baseline_poisson_contrast_risk
refine ⟨C,hC,?_⟩
intro d eps P u t hd heps heps' hP hu ht
let mu := independentPoissonPrefixLaw (obsLaw P) (auxMarginal P).toMeasure u t
let : IsProbabilityMeasure mu := by
unfold mu independentPoissonPrefixLaw
infer_instance
let Z := fun a j s => baselinePrefixCounts (d := d) s (inl (j,a,true))
let K := fun a j s => baselinePrefixCounts (d := d) s (inr (j,a))
have hm (a j) : Measurable (Z a j) ∧ Measurable (K a j) := by
constructor <;> fun_prop
have hZ (a j) : mu.map (Z a j) = poissonMeasure
(u * toNNReal (markedMass P j a)) := baseline_prefix_success_count_law P u t j a
have hK (a j) : mu.map (K a j) = poissonMeasure
(t * toNNReal (armMass P j a)) := baseline_prefix_arm_count_law P u t j a
have hi (a) : iIndepFun
(fun i : Fin 3 × Fin d => if i.1 = 0 then Z a i.2
else if i.1 = 1 then K a i.2 else K (!a) i.2) mu := by
convert baseline_prefix_arm_counts_independent P u t a using 1
funext i s
dsimp [Z,K]
split_ifs <;> rfl
exact hbound d eps P u t _ mu Z K hd heps heps' hP hu ht hm hZ hK hi
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.CausalRealization 25 declarations Existence and universal ATE identification for compatible finite potential laws.
Existence and universal ATE identification for compatible finite potential laws.
def independentExtension
Independent conditional Bernoulli completion followed by the factual outcome.
Definition (Lean source)
parameters
d :
P :
def
given by
mass :
fun z
=> cellMass P z.1 * bernoulliMass (propensity P z.1) z.2.1 * bernoulliMass (outcomeMean P false z.1) z.2.2.2.1 * bernoulliMass (outcomeMean P true z.1) z.2.2.2.2 * (if z.2.2.1 = (if z.2.1 then z.2.2.2.2 else z.2.2.2.1) then 1 else 0) (: FullObs d → Real)
theorem propensity_mem_Icc
Under the stated inputs and conditions, The propensity is a probability even on the stipulated null cells. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: propensity_mem_Icc
lemma propensity_mem_Icc {d : Nat} (P : DiscreteLaw d) (j : Fin d) :
propensity P j ∈ Icc (0 : Real) 1 := by
by_cases hz : cellMass P j = 0
· norm_num [propensity, hz]
· have hp : 0 < cellMass P j := lt_of_le_of_ne (cellMass_nonneg P j) (Ne.symm hz)
have hle : armMass P j true ≤ cellMass P j := by
have hsum : cellMass P j = armMass P j true + armMass P j false := by
simp [cellMass, armMass]
linarith [armMass_nonneg P j false]
simp only [propensity, if_neg hz, Set.mem_Icc]
exact ⟨div_nonneg (armMass_nonneg P j true) hp.le, (div_le_one hp).mpr hle⟩
Under the stated inputs and conditions, Each Bernoulli factor is nonnegative, including boundary probabilities. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: bernoulliMass_nonneg
lemma bernoulliMass_nonneg (r : Real) (hr : r ∈ Icc (0 : Real) 1) (b : Bool) :
0 ≤ bernoulliMass r b := by
cases b <;> simp [bernoulliMass, hr.1, sub_nonneg.mpr hr.2]
The factual-outcome indicator leaves a unit mass over all binary coordinates.
Formal statement
hypotheses
e r0 r1 :
conclusion
(∑ a : Bool, ∑ y : Bool, ∑ y0 : Bool, ∑ y1 : Bool, bernoulliMass e a * bernoulliMass r0 y0 * bernoulliMass r1 y1 * (if y = (if a then y1 else y0) then 1 else 0))
= 1
Proof (Lean source)
-- @node: completion_binary_mass_sum
lemma completion_binary_mass_sum (e r0 r1 : Real) :
(∑ a : Bool, ∑ y : Bool, ∑ y0 : Bool, ∑ y1 : Bool,
bernoulliMass e a * bernoulliMass r0 y0 * bernoulliMass r1 y1 *
(if y = (if a then y1 else y0) then 1 else 0)) = 1 := by
simp [bernoulliMass]
ring
theorem independentExtension_mass_sum
Under the stated inputs and conditions, The independent completion table is normalized before applying the PMF constructor. This gives the stated result.
Formal statement
hypotheses
d :
P :
conclusion
(∑ z : FullObs d, cellMass P z.1 * bernoulliMass (propensity P z.1) z.2.1 * bernoulliMass (outcomeMean P false z.1) z.2.2.2.1 * bernoulliMass (outcomeMean P true z.1) z.2.2.2.2 * (if z.2.2.1 = (if z.2.1 then z.2.2.2.2 else z.2.2.2.1) then 1 else 0))
= 1
Proof (Lean source)
-- @node: independentExtension_mass_sum
lemma independentExtension_mass_sum {d : Nat} (P : DiscreteLaw d) :
(∑ z : FullObs d,
cellMass P z.1 * bernoulliMass (propensity P z.1) z.2.1 *
bernoulliMass (outcomeMean P false z.1) z.2.2.2.1 *
bernoulliMass (outcomeMean P true z.1) z.2.2.2.2 *
(if z.2.2.1 = (if z.2.1 then z.2.2.2.2 else z.2.2.2.1) then 1 else 0)) = 1 := by
simp only [Fintype.sum_prod_type]
have hcell (j : Fin d) :
(∑ a : Bool, ∑ y : Bool, ∑ y0 : Bool, ∑ y1 : Bool,
cellMass P j * bernoulliMass (propensity P j) a *
bernoulliMass (outcomeMean P false j) y0 *
bernoulliMass (outcomeMean P true j) y1 *
(if y = (if a then y1 else y0) then 1 else 0)) = cellMass P j := by
calc
_ = cellMass P j * (∑ a : Bool, ∑ y : Bool, ∑ y0 : Bool, ∑ y1 : Bool,
bernoulliMass (propensity P j) a * bernoulliMass (outcomeMean P false j) y0 *
bernoulliMass (outcomeMean P true j) y1 *
(if y = (if a then y1 else y0) then 1 else 0)) := by
simp only [Finset.mul_sum, mul_assoc]
_ = cellMass P j := by rw [completion_binary_mass_sum, mul_one]
simp_rw [hcell]
exact sum_cellMass P
theorem independentExtension_fullMass
Under the stated inputs and conditions, Normalization preserves the explicit completion table on every full atom. This gives the stated result.
Formal statement
hypotheses
conclusion
fullMass (independentExtension P) z
= cellMass P z.1 * bernoulliMass (propensity P z.1) z.2.1 * bernoulliMass (outcomeMean P false z.1) z.2.2.2.1 * bernoulliMass (outcomeMean P true z.1) z.2.2.2.2 * (if z.2.2.1 = (if z.2.1 then z.2.2.2.2 else z.2.2.2.1) then 1 else 0)
Proof (Lean source)
-- @node: independentExtension_fullMass
lemma independentExtension_fullMass {d : Nat} (P : DiscreteLaw d) (z : FullObs d) :
fullMass (independentExtension P) z =
cellMass P z.1 * bernoulliMass (propensity P z.1) z.2.1 *
bernoulliMass (outcomeMean P false z.1) z.2.2.2.1 *
bernoulliMass (outcomeMean P true z.1) z.2.2.2.2 *
(if z.2.2.1 = (if z.2.1 then z.2.2.2.2 else z.2.2.2.1) then 1 else 0) := by
apply normalizedPMF_toReal
· intro w
apply mul_nonneg
· exact mul_nonneg (mul_nonneg (mul_nonneg (cellMass_nonneg P w.1)
(bernoulliMass_nonneg _ (propensity_mem_Icc P w.1) _))
(bernoulliMass_nonneg _ (outcomeMean_mem_Icc P false w.1) _))
(bernoulliMass_nonneg _ (outcomeMean_mem_Icc P true w.1) _)
· positivity
· exact independentExtension_mass_sum P
theorem independentExtension_consistency
Under the stated inputs and conditions, The completion assigns zero probability to every inconsistent factual outcome. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: independentExtension_consistency
lemma independentExtension_consistency {d : Nat} (P : DiscreteLaw d) :
Consistency (independentExtension P) := by
intro z hz
rw [independentExtension_fullMass, if_neg hz, mul_zero]
theorem independentExtension_poAtom
Under the stated inputs and conditions, Summing the factual outcome gives the product conditional Bernoulli table. This gives the stated result.
Formal statement
hypotheses
conclusion
poAtom (independentExtension P) j a y0 y1
= cellMass P j * bernoulliMass (propensity P j) a * bernoulliMass (outcomeMean P false j) y0 * bernoulliMass (outcomeMean P true j) y1
Proof (Lean source)
-- @node: independentExtension_poAtom
lemma independentExtension_poAtom {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a y0 y1 : Bool) :
poAtom (independentExtension P) j a y0 y1 =
cellMass P j * bernoulliMass (propensity P j) a *
bernoulliMass (outcomeMean P false j) y0 *
bernoulliMass (outcomeMean P true j) y1 := by
simp only [poAtom, independentExtension_fullMass, Fintype.sum_bool]
cases a <;> cases y0 <;> cases y1 <;> simp
theorem independentExtension_exchangeability
Under the stated inputs and conditions, The potential-outcome pair and treatment factor conditionally on each covariate cell. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: independentExtension_exchangeability
lemma independentExtension_exchangeability {d : Nat} (P : DiscreteLaw d) :
ConditionalExchangeability (independentExtension P) := by
intro j a y0 y1
simp only [independentExtension_poAtom, Fintype.sum_bool]
cases a <;> cases y0 <;> cases y1 <;> simp [bernoulliMass] <;> ring
theorem cellMass_mul_bernoulli_propensity
Under the stated inputs and conditions, The propensity weights reconstruct both arm masses, including null cells. This gives the stated result.
Formal statement
hypotheses
conclusion
cellMass P j * bernoulliMass (propensity P j) a = armMass P j a
Proof (Lean source)
-- @node: cellMass_mul_bernoulli_propensity
lemma cellMass_mul_bernoulli_propensity {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a : Bool) :
cellMass P j * bernoulliMass (propensity P j) a = armMass P j a := by
have hsum : cellMass P j = armMass P j true + armMass P j false := by
simp [cellMass, armMass]
by_cases hz : cellMass P j = 0
· have ha : armMass P j a = 0 := by
cases a <;> linarith [armMass_nonneg P j true, armMass_nonneg P j false]
simp [hz, ha]
· cases a <;> simp only [bernoulliMass, Bool.false_eq_true, ↓reduceIte, propensity,
if_neg hz]
· field_simp
linarith
· field_simp
theorem armMass_mul_bernoulli_outcomeMean
Under the stated inputs and conditions, The binary regression reconstructs the two outcome atom masses, including null arms. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: armMass_mul_bernoulli_outcomeMean
lemma armMass_mul_bernoulli_outcomeMean {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a y : Bool) :
armMass P j a * bernoulliMass (outcomeMean P a j) y = jointMass P j a y := by
have hsum : armMass P j a = jointMass P j a true + jointMass P j a false := by
simp [armMass]
by_cases hz : armMass P j a = 0
· have hy : jointMass P j a y = 0 := by
cases y <;> linarith [jointMass_nonneg P j a true, jointMass_nonneg P j a false]
simp [hz, hy]
· cases y <;> simp only [bernoulliMass, Bool.false_eq_true, ↓reduceIte, outcomeMean,
markedMass]
· field_simp
linarith
· field_simp
theorem observedMarginal_jointMass
Under the stated inputs and conditions, Mapping a full law to its observed coordinates sums out both potential outcomes. This gives the stated result.
Formal statement
hypotheses
conclusion
jointMass (observedMarginal H) j a y
Proof (Lean source)
-- @node: observedMarginal_jointMass
lemma observedMarginal_jointMass {d : Nat} (H : PotentialLaw d)
(j : Fin d) (a y : Bool) :
jointMass (observedMarginal H) j a y =
∑ y0 : Bool, ∑ y1 : Bool, fullMass H (j, a, y, y0, y1) := by
have hm : (H.pmf.map (fun z => (z.1, z.2.1, z.2.2.1))) (j, a, y) =
∑ y0 : Bool, ∑ y1 : Bool, H.pmf (j, a, y, y0, y1) := by
rw [PMF.map_apply, tsum_fintype]
cases a <;> cases y <;>
simp [Fintype.sum_prod_type, Prod.mk.injEq, Finset.sum_add_distrib]
simp only [jointMass, observedMarginal, hm, fullMass]
rw [ENNReal.toReal_sum (fun y0 _ => by
exact ENNReal.sum_ne_top.mpr (fun y1 _ => H.pmf.apply_ne_top _))]
congr 1
funext y0
exact ENNReal.toReal_sum (fun y1 _ => H.pmf.apply_ne_top _)
theorem independentExtension_jointMass
Under the stated inputs and conditions, The completion reproduces every original complete-record mass exactly. This gives the stated result.
Formal statement
hypotheses
conclusion
jointMass (observedMarginal (independentExtension P)) j a y = jointMass P j a y
Proof (Lean source)
-- @node: independentExtension_jointMass
lemma independentExtension_jointMass {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a y : Bool) :
jointMass (observedMarginal (independentExtension P)) j a y = jointMass P j a y := by
rw [observedMarginal_jointMass]
simp only [independentExtension_fullMass, Fintype.sum_bool]
have hbinary :
(∑ y0 : Bool, ∑ y1 : Bool,
cellMass P j * bernoulliMass (propensity P j) a *
bernoulliMass (outcomeMean P false j) y0 *
bernoulliMass (outcomeMean P true j) y1 *
(if y = (if a then y1 else y0) then 1 else 0)) =
cellMass P j * bernoulliMass (propensity P j) a *
bernoulliMass (outcomeMean P a j) y := by
cases a <;> cases y <;> simp [bernoulliMass] <;> ring
simp only [Fintype.sum_bool] at hbinary
rw [hbinary, cellMass_mul_bernoulli_propensity, armMass_mul_bernoulli_outcomeMean]
theorem independentExtension_observedMarginal
Under the stated inputs and conditions, The independent potential-outcome completion extends exactly the supplied observed law. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: independentExtension_observedMarginal
lemma independentExtension_observedMarginal {d : Nat} (P : DiscreteLaw d) :
observedMarginal (independentExtension P) = P := by
have hpmf : (observedMarginal (independentExtension P)).pmf = P.pmf := by
apply DFunLike.ext
intro z
rcases z with ⟨j, a, y⟩
apply (ENNReal.toReal_eq_toReal_iff'
((observedMarginal (independentExtension P)).pmf.apply_ne_top _)
(P.pmf.apply_ne_top _)).mp
exact independentExtension_jointMass P j a y
exact congrArg DiscreteLaw.mk hpmf
def potentialSuccessMass
Success mass of one potential outcome in a fixed covariate cell.
theorem poAtom_nonneg
Under the stated inputs and conditions, Potential-outcome marginal atom masses are nonnegative. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: poAtom_nonneg
lemma poAtom_nonneg {d : Nat} (H : PotentialLaw d) (j : Fin d) (a y0 y1 : Bool) :
0 ≤ poAtom H j a y0 y1 :=
Finset.sum_nonneg (fun _ _ => ENNReal.toReal_nonneg)
theorem observedMarginal_cellMass
Under the stated inputs and conditions, The full potential-outcome table and observed law have identical covariate masses. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: observedMarginal_cellMass
lemma observedMarginal_cellMass {d : Nat} (H : PotentialLaw d) (j : Fin d) :
cellMass (observedMarginal H) j =
∑ a : Bool, ∑ y0 : Bool, ∑ y1 : Bool, poAtom H j a y0 y1 := by
simp only [cellMass, observedMarginal_jointMass, poAtom, Fintype.sum_bool]
ring
theorem observedMarginal_armMass
Under the stated inputs and conditions, The full potential-outcome table and observed law have identical arm masses. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: observedMarginal_armMass
lemma observedMarginal_armMass {d : Nat} (H : PotentialLaw d) (j : Fin d) (a : Bool) :
armMass (observedMarginal H) j a =
∑ y0 : Bool, ∑ y1 : Bool, poAtom H j a y0 y1 := by
simp only [armMass, observedMarginal_jointMass, poAtom, Fintype.sum_bool]
ring
theorem consistent_markedMass
Under the stated inputs and conditions, Consistency identifies factual success mass with selected potential success mass in each arm. This gives the stated result.
Formal statement
hypotheses
conclusion
markedMass (observedMarginal H) j a
Proof (Lean source)
-- @node: consistent_markedMass
lemma consistent_markedMass {d : Nat} (H : PotentialLaw d) (hC : Consistency H)
(j : Fin d) (a : Bool) :
markedMass (observedMarginal H) j a =
∑ y0 : Bool, ∑ y1 : Bool, poAtom H j a y0 y1 *
(if (if a then y1 else y0) then 1 else 0) := by
have hzero (a y0 y1 : Bool) :
fullMass H (j, a, !(if a then y1 else y0), y0, y1) = 0 := by
apply hC
cases a <;> cases y0 <;> cases y1 <;> simp
have hzfff := hzero false false false
have hzfft := hzero false false true
have hzftf := hzero false true false
have hzftt := hzero false true true
have hztff := hzero true false false
have hztft := hzero true false true
have hzttf := hzero true true false
have hzttt := hzero true true true
simp only [Bool.false_eq_true, ↓reduceIte, Bool.not_false, Bool.not_true]
at hzfff hzfft hzftf hzftt hztff hztft hzttf hzttt
cases a <;> simp [markedMass, observedMarginal_jointMass, poAtom,
hzfff, hzfft, hzftf, hzftt, hztff, hztft, hzttf, hzttt]
theorem potentialSuccessMass_bounds
Under the stated inputs and conditions, Each potential success mass is bounded by its covariate mass. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
0 ≤ potentialSuccessMass H j a
conclusion 2
potentialSuccessMass H j a ≤ cellMass (observedMarginal H) j
Proof (Lean source)
-- @node: potentialSuccessMass_bounds
lemma potentialSuccessMass_bounds {d : Nat} (H : PotentialLaw d) (j : Fin d) (a : Bool) :
0 ≤ potentialSuccessMass H j a ∧
potentialSuccessMass H j a ≤ cellMass (observedMarginal H) j := by
constructor
· exact Finset.sum_nonneg (fun a' _ => Finset.sum_nonneg (fun y0 _ =>
Finset.sum_nonneg (fun y1 _ => mul_nonneg (poAtom_nonneg H j a' y0 y1)
(by positivity))))
· rw [observedMarginal_cellMass]
apply Finset.sum_le_sum
intro a' _
apply Finset.sum_le_sum
intro y0 _
apply Finset.sum_le_sum
intro y1 _
apply mul_le_of_le_one_right (poAtom_nonneg H j a' y0 y1)
cases a <;> cases y0 <;> cases y1 <;> norm_num
theorem compatible_success_identity
Under the stated inputs and conditions, Summing conditional exchangeability over a potential success event gives its identifying equation. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: compatible_success_identity
lemma compatible_success_identity {d : Nat} (H : PotentialLaw d)
(hC : Consistency H) (hE : ConditionalExchangeability H) (j : Fin d) (a : Bool) :
markedMass (observedMarginal H) j a * cellMass (observedMarginal H) j =
armMass (observedMarginal H) j a * potentialSuccessMass H j a := by
rw [consistent_markedMass H hC, observedMarginal_cellMass, observedMarginal_armMass]
simp only [potentialSuccessMass, Finset.sum_mul, Finset.mul_sum]
calc
_ = ∑ y0 : Bool, ∑ y1 : Bool,
(poAtom H j a y0 y1 *
(∑ a' : Bool, ∑ y0' : Bool, ∑ y1' : Bool, poAtom H j a' y0' y1')) *
(if (if a then y1 else y0) then 1 else 0) := by
simp only [Fintype.sum_bool]
ring
_ = ∑ y0 : Bool, ∑ y1 : Bool,
((∑ y0' : Bool, ∑ y1' : Bool, poAtom H j a y0' y1') *
(∑ a' : Bool, poAtom H j a' y0 y1)) *
(if (if a then y1 else y0) then 1 else 0) := by
simp_rw [hE j a]
_ = _ := by
simp only [Fintype.sum_bool]
ring
theorem compatible_cell_mean
Under the stated inputs and conditions, Overlap identifies each potential success mass without restricting the unseen coupling. This gives the stated result.
Formal statement
hypotheses
d :
P :
eps :
heps :
0 < eps
hP :
ModelClass d eps P
H :
hm :
observedMarginal H = P
hC :
Consistency H
hE :
j :
Fin d
a :
conclusion
potentialSuccessMass H j a = cellMass P j * outcomeMean P a j
Proof (Lean source)
-- @node: compatible_cell_mean
lemma compatible_cell_mean {d : Nat} (P : DiscreteLaw d) (eps : Real)
(heps : 0 < eps) (hP : ModelClass d eps P) (H : PotentialLaw d)
(hm : observedMarginal H = P) (hC : Consistency H)
(hE : ConditionalExchangeability H) (j : Fin d) (a : Bool) :
potentialSuccessMass H j a = cellMass P j * outcomeMean P a j := by
have hid := compatible_success_identity H hC hE j a
rw [hm] at hid
by_cases hz : cellMass P j = 0
· have hb := potentialSuccessMass_bounds H j a
rw [hm, hz] at hb
have hzero : potentialSuccessMass H j a = 0 := le_antisymm hb.2 hb.1
simp [hz, hzero]
· have hp : 0 < cellMass P j := lt_of_le_of_ne (cellMass_nonneg P j) (Ne.symm hz)
have he := hP.overlap j hp
have hs : 0 < armMass P j a := by
rw [← cellMass_mul_bernoulli_propensity]
apply mul_pos hp
cases a
· simp only [bernoulliMass, Bool.false_eq_true, ↓reduceIte]
linarith [he.2]
· simp only [bernoulliMass, ↓reduceIte]
exact lt_of_lt_of_le heps he.1
dsimp only [outcomeMean]
field_simp
nlinarith [hid]
theorem poContrast_eq_sum_potentialSuccessMass
Under the stated inputs and conditions, Finite expectation of the contrast equals the difference of the two potential success masses. This gives the stated result.
Formal statement
hypotheses
d :
H :
conclusion
Proof (Lean source)
-- @node: poContrast_eq_sum_potentialSuccessMass
lemma poContrast_eq_sum_potentialSuccessMass {d : Nat} (H : PotentialLaw d) :
poContrast H = ∑ j : Fin d,
(potentialSuccessMass H j true - potentialSuccessMass H j false) := by
simp only [poContrast, Fintype.sum_prod_type]
apply Finset.sum_congr rfl
intro j _
simp [potentialSuccessMass, poAtom]
ring
theorem compatible_poContrast
Under the stated inputs and conditions, Every compatible extension identifies the same observable ATE under occupied-cell overlap. This gives the stated result.
Formal statement
hypotheses
d :
P :
eps :
heps :
0 < eps
hP :
ModelClass d eps P
H :
hm :
observedMarginal H = P
hC :
Consistency H
hE :
conclusion
poContrast H = ateFunctional P
Proof (Lean source)
-- @node: compatible_poContrast
lemma compatible_poContrast {d : Nat} (P : DiscreteLaw d) (eps : Real)
(heps : 0 < eps) (hP : ModelClass d eps P) (H : PotentialLaw d)
(hm : observedMarginal H = P) (hC : Consistency H)
(hE : ConditionalExchangeability H) : poContrast H = ateFunctional P := by
rw [poContrast_eq_sum_potentialSuccessMass]
apply Finset.sum_congr rfl
intro j _
rw [compatible_cell_mean P eps heps hP H hm hC hE j true,
compatible_cell_mean P eps heps hP H hm hC hE j false]
ring
theorem causal_realization
For an observed law with at least two covariate cells, positive overlap, overlap at most one quarter, and membership in the model class, the independent potential-outcome extension realizes the observed law, and every compatible exchangeable extension has the stated causal contrast.
Formal statement
hypotheses
conclusion
conclusion 1
observedMarginal (independentExtension P) = P
Consistency (independentExtension P)
conclusion 2
poContrast H = ateFunctional P
Proof (Lean source)
lemma causal_realization (d : Nat) (eps : Real) (P : DiscreteLaw d)
(hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) (hP : ModelClass d eps P) :
(observedMarginal (independentExtension P) = P ∧ Consistency (independentExtension P) ∧
ConditionalExchangeability (independentExtension P)) ∧
∀ H : PotentialLaw d, observedMarginal H = P → Consistency H →
-- @realizes H(compatible extension of the specified observed law P)
ConditionalExchangeability H → poContrast H = ateFunctional P := by
refine ⟨⟨independentExtension_observedMarginal P, independentExtension_consistency P,
independentExtension_exchangeability P⟩, ?_⟩
intro H hm hC hE
exact compatible_poContrast P eps heps hP H hm hC hE
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.ChebyshevAlgebra 7 declarations Algebraic degree bounds and removable-value identities for the Chebyshev continuations.
Algebraic degree bounds and removable-value identities for the Chebyshev continuations.
Under the stated inputs and conditions, Division by the variable removes the zero constant coefficient of the first numerator. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: chebE_mul_X
lemma chebE_mul_X (L : Nat) :
X * chebE L = (2 * (L : Real) ^ 2)⁻¹ •
(1 - (T Real L).comp (1 - 2 * X)) := by
have h := Polynomial.modByMonic_add_div
((2 * (L : Real) ^ 2)⁻¹ •
(1 - (T Real L).comp (1 - 2 * X))) X
simpa [Polynomial.modByMonic_X, Polynomial.eval_comp,
Polynomial.Chebyshev.T_eval_one, chebE] using h
Under the stated inputs and conditions, The removable value of the first continuation is the normalized endpoint derivative. This gives the stated result.
Proof (Lean source)
-- @node: chebE_eval_zero
lemma chebE_eval_zero (L : Nat) (hL : 0 < L) : (chebE L).eval 0 = 1 := by
have h := congrArg (fun p : Polynomial Real => p.derivative.eval 0) (chebE_mul_X L)
simp [Polynomial.derivative_mul,
Polynomial.derivative_comp, Polynomial.eval_comp,
Polynomial.Chebyshev.derivative_T_eval_one] at h
have hLn : (L : Real) ≠ 0 := by exact_mod_cast hL.ne'
field_simp at h
nlinarith
Under the stated inputs and conditions, The second numerator also has zero constant coefficient, so division is exact. This gives the stated result.
Proof (Lean source)
-- @node: chebG_mul_X
lemma chebG_mul_X (L : Nat) (hL : 0 < L) :
X * chebG L = 1 - chebE L := by
have h := Polynomial.modByMonic_add_div (1 - chebE L) X
simpa [Polynomial.modByMonic_X, chebE_eval_zero L hL, chebG] using h
Under the stated inputs and conditions, Both polynomial divisions realize the displayed removable rational expressions. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: cheb_continuation_eval
lemma cheb_continuation_eval (L : Nat) (hL : 0 < L) (z : Real) (hz : z ≠ 0) :
(chebE L).eval z = (1 - (T Real L).eval (1 - 2 * z)) /
(2 * (L : Real) ^ 2 * z) ∧
(chebG L).eval z = (1 - (chebE L).eval z) / z := by
have hE := congrArg (eval z) (chebE_mul_X L)
have hG := congrArg (eval z) (chebG_mul_X L hL)
simp [Polynomial.eval_comp] at hE hG
constructor
· apply (eq_div_iff (mul_ne_zero (mul_ne_zero (by norm_num) (pow_ne_zero 2
(show (L : Real) ≠ 0 by exact_mod_cast hL.ne'))) hz)).2
have hh := hE
field_simp at hh
simpa only [mul_comm, mul_left_comm, mul_assoc] using hh
· apply (eq_div_iff hz).2
nlinarith only [hG]
Under the stated inputs and conditions, Each exact division lowers the degree by one, giving the factorial truncation order. This gives the stated result.
Proof (Lean source)
-- @node: chebG_natDegree_le
lemma chebG_natDegree_le (L : Nat) : (chebG L).natDegree ≤ L - 2 := by
have hlin : (1 - 2 * (Polynomial.X : Polynomial Real)).natDegree ≤ 1 := by
apply (Polynomial.natDegree_sub_le _ _).trans
simp
have hcomp : ((T Real L).comp
(1 - 2 * Polynomial.X)).natDegree ≤ L := by
apply Polynomial.natDegree_comp_le.trans
simpa [Polynomial.Chebyshev.natDegree_T] using Nat.mul_le_mul_left L hlin
have hnum : (1 - (T Real L).comp
(1 - 2 * Polynomial.X)).natDegree ≤ L := by
exact (Polynomial.natDegree_sub_le _ _).trans (by simpa using hcomp)
have hE : (chebE L).natDegree ≤ L - 1 := by
unfold chebE
rw [Polynomial.natDegree_divByMonic _ Polynomial.monic_X, Polynomial.natDegree_X]
exact Nat.sub_le_sub_right ((Polynomial.natDegree_smul_le _ _).trans hnum) 1
unfold chebG
rw [Polynomial.natDegree_divByMonic _ Polynomial.monic_X, Polynomial.natDegree_X]
have hsub : (1 - chebE L).natDegree ≤ L - 1 :=
(Polynomial.natDegree_sub_le _ _).trans (by simpa using hE)
have hh := Nat.sub_le_sub_right hsub 1
omega
Under the stated inputs and conditions, The Chebyshev numerator is nonnegative and bounded by both its range and endpoint slope. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: cheb_numerator_bounds
lemma cheb_numerator_bounds (L : Nat) (z : Real) (hz : 0 ≤ z) (hz' : z ≤ 1) :
0 ≤ 1 - (Polynomial.Chebyshev.T Real L).eval (1 - 2 * z) ∧
1 - (Polynomial.Chebyshev.T Real L).eval (1 - 2 * z) ≤ 2 ∧
1 - (Polynomial.Chebyshev.T Real L).eval (1 - 2 * z) ≤
2 * (L : Real) ^ 2 * z := by
let Q := Polynomial.Chebyshev.T Real L
have hmem : 1 - 2 * z ∈ Icc (-1 : Real) 1 := by
constructor <;> linarith
have hbound (x : Real) (hx : x ∈ Icc (-1 : Real) 1) : |Q.eval x| ≤ 1 :=
Polynomial.Chebyshev.abs_eval_T_real_le_one L (abs_le.mpr hx)
have hrange := abs_le.mp (hbound _ hmem)
have hderiv (x : Real) (hx : x ∈ Icc (-1 : Real) 1) :
‖Q.derivative.eval x‖ ≤ (L : Real) ^ 2 := by
simpa only [Real.norm_eq_abs, mul_one] using
MarkovPolynomial L Q 1 (by simp [Q, Polynomial.Chebyshev.natDegree_T]) hbound x hx
have hdiff := (convex_Icc (-1 : Real) 1).norm_image_sub_le_of_norm_hasDerivWithin_le
(fun x _ => Q.hasDerivWithinAt x _) hderiv hmem
(show (1 : Real) ∈ Icc (-1 : Real) 1 by norm_num)
have hone : Q.eval 1 = 1 := by simp [Q]
rw [hone, Real.norm_eq_abs, Real.norm_eq_abs,
abs_of_nonneg (by linarith : 0 ≤ 1 - (1 - 2 * z))] at hdiff
refine ⟨by linarith, by linarith, ?_⟩
have := (le_abs_self (1 - Q.eval (1 - 2 * z))).trans hdiff
dsimp only [Q] at this
nlinarith only [this]
Under the stated inputs and conditions, The first continuation lies between zero and the smaller of one and the inverse-scale bound. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: chebE_approximation_bounds
lemma chebE_approximation_bounds (L : Nat) (hL : 0 < L) (z : Real)
(hz : 0 < z) (hz' : z ≤ 1) :
0 ≤ (chebE L).eval z ∧
(chebE L).eval z ≤ min 1 ((L : Real) ^ 2 * z)⁻¹ := by
obtain ⟨hzero, htwo, hslope⟩ := cheb_numerator_bounds L z hz.le hz'
rw [(cheb_continuation_eval L hL z hz.ne').1]
have hLp : 0 < (L : Real) := by exact_mod_cast hL
have hden : 0 < 2 * (L : Real) ^ 2 * z := by positivity
refine ⟨div_nonneg hzero hden.le, le_min ?_ ?_⟩
· apply (div_le_iff₀ hden).2
simpa only [one_mul] using hslope
· apply (div_le_iff₀ hden).2
have hid : ((L : Real) ^ 2 * z)⁻¹ * (2 * (L : Real) ^ 2 * z) = 2 := by
field_simp
rw [hid]
exact htwo
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.CitedGates 4 declarations Finite Pinsker, Markov, and finite polynomial duality proved from Causalean primitives.
Finite Pinsker, Markov, and finite polynomial duality proved from Causalean primitives.
def extendedFiniteKL
Natural-log KL on a finite alphabet, with infinite divergence when a positive mass has zero reference mass. On probability vectors its finite branch is the usual KL sum.
Wu and Yang (2016), Remark 4, displayed Pinsker inequality, arXiv:1407.0381v3. Natural-log finite divergence, including infinite KL for singular probability pairs.
Formal statement
hypotheses
alpha :
Type
p :
alpha → ℝ
q :
alpha → ℝ
alpha :
Type
Fintype alpha
p q :
alpha → Real
(∀ z, 0 ≤ p z)
(∀ z, 0 ≤ q z)
(∑ z, p z = 1)
(∑ z, q z = 1)
conclusion
ofReal ((1 / 2) * (∑ z, |p z - q z|))
≤ (extendedFiniteKL p q / 2) ^ (1 / 2 : Real)
Proof (Lean source)
lemma PinskerFinite :
∀ (alpha : Type) [Fintype alpha] (p q : alpha → Real),
(∀ z, 0 ≤ p z) → (∀ z, 0 ≤ q z) → (∑ z, p z = 1) → (∑ z, q z = 1) →
ofReal ((1 / 2) * (∑ z, |p z - q z|)) ≤
(extendedFiniteKL p q / 2) ^ (1 / 2 : Real)
:= by
intro alpha _ p q hp hq hpsum hqsum
classical
by_cases hac : ∀ z, q z = 0 → p z = 0
swap
· simp [extendedFiniteKL, hac, ENNReal.top_div_of_ne_top]
let : MeasurableSpace alpha := ⊤
let : MeasurableSingletonClass alpha := ⟨fun _ => trivial⟩
let pp : PMF alpha := PMF.ofFintype (fun z => ofReal (p z)) (by
rw [← ENNReal.ofReal_sum_of_nonneg (fun z _ => hp z), hpsum]
simp)
let qq : PMF alpha := PMF.ofFintype (fun z => ofReal (q z)) (by
rw [← ENNReal.ofReal_sum_of_nonneg (fun z _ => hq z), hqsum]
simp)
have hpread (z : alpha) : pp.toMeasure.real {z} = p z := by
simp [Measure.real_def, pp, ENNReal.toReal_ofReal (hp z)]
have hqread (z : alpha) : qq.toMeasure.real {z} = q z := by
simp [Measure.real_def, qq, ENNReal.toReal_ofReal (hq z)]
have hpmf : pp.toMeasure ≪ qq.toMeasure := by
intro s hs
rw [pp.toMeasure_apply_eq_toOuterMeasure_apply (by trivial), PMF.toOuterMeasure_apply]
apply ENNReal.tsum_eq_zero.mpr
intro z
by_cases hz : z ∈ s
· have hqz : qq.toMeasure {z} = 0 := measure_mono_null (Set.singleton_subset_iff.mpr hz) hs
have hqzero : q z = 0 := by
have := congrArg toReal hqz
simpa [Measure.real_def, ← hqread z] using this
simp [Set.indicator_of_mem hz, pp, hac z hqzero]
· simp [Set.indicator_of_notMem hz]
have hfin : klDiv pp.toMeasure qq.toMeasure ≠ ⊤ :=
InformationTheory.klDiv_ne_top hpmf (by exact Integrable.of_finite)
have hKL := klDiv_toReal_eq_sum_measureReal
pp.toMeasure qq.toMeasure hpmf
simp only [hpread, hqread] at hKL
have hTV : (1 / 2 : Real) * ∑ z, |p z - q z| ≤
tvDist pp.toMeasure qq.toMeasure := by
let A : Finset alpha := Finset.univ.filter (fun z => 0 ≤ p z - q z)
have hsum : ∑ z, (p z - q z) = 0 := by rw [Finset.sum_sub_distrib, hpsum, hqsum]; ring
have habs : ∑ z, |p z - q z| = 2 * ∑ z ∈ A, (p z - q z) := by
have hsplit : ∑ z, (p z - q z) =
(∑ z ∈ A, (p z - q z)) + ∑ z ∈ Aᶜ, (p z - q z) := by
rw [Finset.sum_add_sum_compl A]
have hsplitabs : ∑ z, |p z - q z| =
(∑ z ∈ A, |p z - q z|) + ∑ z ∈ Aᶜ, |p z - q z| := by
rw [Finset.sum_add_sum_compl A]
have hpos : ∑ z ∈ A, |p z - q z| = ∑ z ∈ A, (p z - q z) := by
apply Finset.sum_congr rfl
intro z hz
exact abs_of_nonneg (Finset.mem_filter.mp hz).2
have hneg : ∑ z ∈ Aᶜ, |p z - q z| =
-(∑ z ∈ Aᶜ, (p z - q z)) := by
rw [← Finset.sum_neg_distrib]
apply Finset.sum_congr rfl
intro z hz
apply abs_of_neg
exact lt_of_not_ge (fun h => (Finset.mem_compl.mp hz)
(Finset.mem_filter.mpr ⟨Finset.mem_univ z, h⟩))
rw [hpos, hneg] at hsplitabs
linarith
have hevent (rr : PMF alpha) : rr.toMeasure.real (A : Set alpha) =
∑ z ∈ A, rr.toMeasure.real {z} := by
rw [probability_measureReal_eq_tsum_singletons rr.toMeasure
(A : Set alpha) (by trivial)]
simp [indicator]
have hgap : pp.toMeasure.real (A : Set alpha) - qq.toMeasure.real (A : Set alpha) =
∑ z ∈ A, (p z - q z) := by
rw [hevent pp, hevent qq, ← Finset.sum_sub_distrib]
simp only [hpread, hqread]
have hle := abs_measureReal_sub_le_tvDist
(μ := pp.toMeasure) (ν := qq.toMeasure) (show MeasurableSet (A : Set alpha) from by trivial)
rw [hgap] at hle
rw [habs]
nlinarith [le_abs_self (∑ z ∈ A, (p z - q z))]
have hPinsker := pinskerBound_of_ac_of_ne_top
pp.toMeasure qq.toMeasure hpmf hfin
change tvDist pp.toMeasure qq.toMeasure ≤ _ at hPinsker
rw [hKL] at hPinsker
have hnonneg : 0 ≤ ∑ z, p z * log (p z / q z) := by
rw [← hKL]
exact ENNReal.toReal_nonneg
have hbound := ENNReal.ofReal_le_ofReal (hTV.trans hPinsker)
rw [Real.sqrt_eq_rpow,
← ENNReal.ofReal_rpow_of_nonneg (div_nonneg hnonneg (by norm_num)) (by norm_num),
ENNReal.ofReal_div_of_pos (by norm_num : (0 : Real) < 2)] at hbound
simpa only [extendedFiniteKL, if_pos hac, ENNReal.ofReal_ofNat] using hbound
Pierzchała (2016), Theorem 1.1, DOI:10.1007/s00208-015-1294-9. The degree-bounded pointwise form of Markov's inequality.
Formal statement
hypotheses
L :
ℕ
Mb :
ℝ
x :
ℝ
L :
Q :
Mb :
Q.natDegree ≤ L
x :
conclusion
|Q.derivative.eval x| ≤ (L : Real) ^ 2 * Mb
Proof (Lean source)
lemma MarkovPolynomial :
∀ (L : Nat) (Q : Polynomial Real) (Mb : Real), Q.natDegree ≤ L →
(∀ x, x ∈ Icc (-1 : Real) 1 → |Q.eval x| ≤ Mb) →
∀ x, x ∈ Icc (-1 : Real) 1 → |Q.derivative.eval x| ≤ (L : Real) ^ 2 * Mb
:= by
intro L Q Mb hdeg hMb x hx
open Causalean.Mathlib.Analysis.FinitePolynomialAlternationDuality in
exact ((intervalSupNorm_le_iff Q.derivative.continuous.continuousOn (by norm_num)).mp
((markov_derivative_unitInterval Q L hdeg).trans
(mul_le_mul_of_nonneg_left
((intervalSupNorm_le_iff Q.continuous.continuousOn (by norm_num)).mpr hMb)
(sq_nonneg (L : Real))))) x hx
Wu and Yang (2016), Appendix E, identity (34) and alternating finite weights, arXiv:1407.0381v3. Weights have variation one and positive orientation.
Formal statement
hypotheses
g :
ℝ → ℝ
vlo :
ℝ
vhi :
ℝ
L :
ℕ
vlo vhi :
L :
vlo < vhi
ContinuousOn g (Icc vlo vhi)
conclusion
StrictMono nodes
conclusion 2
∑ i, |w i| = 1
conclusion 3
h :
h ≤ L
∑ i, w i * nodes i ^ h = 0
conclusion 4
∑ i, w i * g (nodes i)
= bestUniformApproxError g vlo vhi L
Proof (Lean source)
lemma FinitePolynomialDuality :
∀ (g : Real → Real) (vlo vhi : Real) (L : Nat), vlo < vhi →
ContinuousOn g (Icc vlo vhi) →
∃ (nodes : Fin (L + 2) → Real) (w : Fin (L + 2) → Real),
StrictMono nodes ∧ (∀ i, nodes i ∈ Icc vlo vhi) ∧
(∑ i, |w i| = 1) ∧ (∀ h, h ≤ L → ∑ i, w i * nodes i ^ h = 0) ∧
∑ i, w i * g (nodes i) =
bestUniformApproxError g
vlo vhi L
:= by
intro g vlo vhi L hv hg
obtain ⟨D⟩ :=
Causalean.Mathlib.Analysis.FinitePolynomialAlternationDuality.exists_finiteMomentDual hv hg L
by_cases h : 0 ≤ ∑ i, D.weights i * g (D.nodes i)
· exact ⟨D.nodes, D.weights, D.nodes_strictMono, D.nodes_mem, D.weights_normalized,
D.moments_zero, (abs_of_nonneg h).symm.trans D.target_abs_eq⟩
· refine ⟨D.nodes, fun i => -D.weights i, D.nodes_strictMono, D.nodes_mem, ?_, ?_, ?_⟩
· simpa only [abs_neg] using D.weights_normalized
· intro k hk
simpa only [neg_mul, Finset.sum_neg_distrib, neg_zero] using
congrArg neg (D.moments_zero k hk)
· simpa only [neg_mul, Finset.sum_neg_distrib, abs_of_neg (lt_of_not_ge h)] using
D.target_abs_eq
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.CoefficientBounds 4 declarations Coefficient bounds for the two Chebyshev continuation polynomials, using the shifted Chebyshev recurrence and invariance of the coefficient norm under multiplication by X.
Coefficient bounds for the two Chebyshev continuation polynomials, using the shifted Chebyshev recurrence and invariance of the coefficient norm under multiplication by X.
Under the stated inputs and conditions, Scalar multiplication scales the sum of absolute polynomial coefficients. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: continuation_coeffNorm_smul
lemma continuation_coeffNorm_smul (c : Real) (p : Polynomial Real) :
polynomialCoeffOneNorm (c • p) = |c| * polynomialCoeffOneNorm p := by
rw [polynomialCoeffOneNorm_range (c • p)
(lt_of_le_of_lt (natDegree_smul_le _ _) (lt_succ_self _)),
polynomialCoeffOneNorm_range p (lt_succ_self _)]
simp only [coeff_smul, smul_eq_mul, abs_mul, Finset.mul_sum]
Under the stated inputs and conditions, Multiplication by the variable shifts coefficients and preserves their absolute sum. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: continuation_coeffNorm_X_mul
lemma continuation_coeffNorm_X_mul (p : Polynomial Real) :
polynomialCoeffOneNorm (X * p) = polynomialCoeffOneNorm p := by
by_cases hp : p = 0
· simp [hp, polynomialCoeffOneNorm,
polyCoeffL1]
· rw [polynomialCoeffOneNorm_range (X * p) (m := p.natDegree + 2)
(by rw [natDegree_X_mul hp]; omega),
polynomialCoeffOneNorm_range p (lt_succ_self _)]
rw [show p.natDegree + 2 = (p.natDegree + 1) + 1 by omega,
Finset.sum_range_succ']
simp only [coeff_X_mul_zero, abs_zero, add_zero, Nat.succ_eq_add_one, coeff_X_mul]
Under the stated inputs and conditions, The first continuation inherits the shifted Chebyshev coefficient bound after division. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: chebE_coeffNorm_le
lemma chebE_coeffNorm_le (L : Nat) (hL : 0 < L) :
polynomialCoeffOneNorm (chebE L) ≤ (7 : Real) ^ L := by
have hLp : (1 : Real) ≤ L := by exact_mod_cast hL
have hden : 0 < 2 * (L : Real) ^ 2 := by positivity
have hnorm := congrArg polynomialCoeffOneNorm (chebE_mul_X L)
rw [continuation_coeffNorm_X_mul, continuation_coeffNorm_smul,
abs_of_pos (inv_pos.mpr hden)] at hnorm
have hone : polynomialCoeffOneNorm (1 : Polynomial Real) = 1 := by
rw [polynomialCoeffOneNorm_range _ (m := 1) (by simp)]
simp
have hshift : polynomialCoeffOneNorm
((T Real L).comp (1 - 2 * X)) ≤ (7 : Real) ^ L := by
simpa [shiftedCheb, Polynomial.C_ofNat] using shiftedCheb_coeffL1_le L
have hnum := (polynomialCoeffOneNorm_sub_le 1
((T Real L).comp (1 - 2 * X))).trans
(add_le_add (le_of_eq hone) hshift)
have hseven : (1 : Real) ≤ 7 ^ L := one_le_pow₀ (by norm_num)
rw [hnorm]
apply (inv_mul_le_iff₀ hden).2
nlinarith [sq_nonneg ((L : Real) - 1)]
Under the stated inputs and conditions, The second removable division gives the coefficient sum required by the certificate. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: chebG_coeff_sum_le
lemma chebG_coeff_sum_le (L : Nat) (hL : 2 ≤ L) :
(∑ h ∈ range (L - 1), |(chebG L).coeff h|) ≤ (14 : Real) ^ L := by
have hLp : 0 < L := by omega
rw [← polynomialCoeffOneNorm_range (chebG L)
(lt_of_le_of_lt (chebG_natDegree_le L) (by omega : L - 2 < L - 1))]
have hnorm := congrArg polynomialCoeffOneNorm (chebG_mul_X L hLp)
rw [continuation_coeffNorm_X_mul] at hnorm
have hone : polynomialCoeffOneNorm (1 : Polynomial Real) = 1 := by
rw [polynomialCoeffOneNorm_range _ (m := 1) (by simp)]
simp
have hseven : (1 : Real) ≤ 7 ^ L := one_le_pow₀ (by norm_num)
have htwo : (2 : Real) ≤ 2 ^ L := by
simpa using pow_le_pow_right₀ (by norm_num : (1 : Real) ≤ 2)
(show 1 ≤ L by omega)
calc
polynomialCoeffOneNorm (chebG L) = polynomialCoeffOneNorm (1 - chebE L) := hnorm
_ ≤ polynomialCoeffOneNorm (1 : Polynomial Real) +
polynomialCoeffOneNorm (chebE L) := polynomialCoeffOneNorm_sub_le _ _
_ ≤ 1 + (7 : Real) ^ L := by rw [hone]; exact add_le_add (le_refl 1) (chebE_coeffNorm_le L hLp)
_ ≤ 2 * (7 : Real) ^ L := by linarith
_ ≤ (2 : Real) ^ L * 7 ^ L := mul_le_mul_of_nonneg_right htwo (by positivity)
_ = (14 : Real) ^ L := by rw [← mul_pow]; norm_num
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.ComparisonPluginRisk 12 declarations Bounded-loss transport from a supplied marginal table to its proof-only replacement population, and the auxiliary plug-in rule used for experiment comparison.
Bounded-loss transport from a supplied marginal table to its proof-only replacement population, and the auxiliary plug-in rule used for experiment comparison.
theorem comparisonReplacementLaw_tv
Under the stated inputs and conditions, The replacement complete-record law is at most half the marginal L¹ error away in TV. This gives the stated result.
Formal statement
hypotheses
conclusion
tvDist (obsLaw P) (obsLaw (comparisonReplacementLaw P v))
Proof (Lean source)
-- @node: comparisonReplacementLaw_tv
lemma comparisonReplacementLaw_tv {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1) :
tvDist (obsLaw P) (obsLaw (comparisonReplacementLaw P v)) ≤
(1 / 2 : Real) * ∑ z : AuxObs d, |auxTable P z - v z| := by
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
let : IsProbabilityMeasure (obsLaw (comparisonReplacementLaw P v)) := by
unfold obsLaw; infer_instance
have h := tvDist_le_half_tsum_singleton_abs
(obsLaw P) (obsLaw (comparisonReplacementLaw P v))
simp only [tsum_fintype, measureReal_def, obsLaw] at h
simp_rw [PMF.toMeasure_apply_singleton _ _ (measurableSet_singleton _)] at h
exact h.trans_eq (congrArg ((1 / 2 : Real) * ·)
(comparisonReplacementLaw_l1 P v hv hs))
theorem comparison_squared_loss_target_error Lemma comparison_squared_loss_target_error in the paper ↗
Under the stated inputs and conditions, Squared loss on the action and target intervals is four-Lipschitz in the target. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparison_squared_loss_target_error
lemma comparison_squared_loss_target_error (h t t' : Real)
(hh : h ∈ Icc (-1) 1) (ht : t ∈ Icc (-1) 1)
(ht' : t' ∈ Icc (-1) 1) :
|(h - t) ^ 2 - (h - t') ^ 2| ≤ 4 * |t - t'| := by
have hb : |2 * h - t - t'| ≤ 4 := abs_le.mpr
⟨by linarith [hh.1, ht.2, ht'.2], by linarith [hh.2, ht.1, ht'.1]⟩
calc
_ = |t - t'| * |2 * h - t - t'| := by
rw [← abs_mul, ← abs_neg]
congr 1
ring
_ ≤ |t - t'| * 4 := mul_le_mul_of_nonneg_left hb (abs_nonneg _)
_ = _ := mul_comm _ _
def comparisonFixedTableRisk
Risk of a supplied-table rule when the supplied table need not equal the true one.
Definition (Lean source)
parameters
def
given by
∫ z : (Fin n → Obs d) × Real, (K.1 (z.1, v, z.2) - ateFunctional P) ^ 2 ∂((labeledProductLaw P n).prod seedLaw)
theorem comparisonFixedTableLoss_integrable
Under the stated inputs and conditions, Bounded measurable supplied-table losses are integrable at any population and target. This gives the stated result.
Formal statement
hypotheses
conclusion
Integrable (fun z : (Fin n → Obs d) × Real => (K.1 (z.1, v, z.2) - t) ^ 2) ((labeledProductLaw P n).prod seedLaw)
Proof (Lean source)
-- @node: comparisonFixedTableLoss_integrable
lemma comparisonFixedTableLoss_integrable {n d : Nat} (K : KnownRule n d)
(P : DiscreteLaw d) (v : AuxTable d) (t : Real) :
Integrable (fun z : (Fin n → Obs d) × Real => (K.1 (z.1, v, z.2) - t) ^ 2)
((labeledProductLaw P n).prod seedLaw) := by
let : IsProbabilityMeasure (labeledProductLaw P n) := by
unfold labeledProductLaw obsLaw; infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have hK := K.2.1
apply mse_integrable_of_estimator_bound (M := 1)
((labeledProductLaw P n).prod seedLaw)
(fun z : (Fin n → Obs d) × Real => K.1 (z.1, v, z.2)) (by fun_prop) (by norm_num)
exact fun z => K.2.2 _
theorem comparisonFixedTableRisk_transport
Under the stated inputs and conditions, For a fixed legal table, product-TV and target transport cost at most (2n + 4) times its marginal L¹ error. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonFixedTableRisk_transport
lemma comparisonFixedTableRisk_transport {n d : Nat} (K : KnownRule n d)
(P : DiscreteLaw d) (v : AuxTable d) (eps : Real) (heps : 0 < eps)
(hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(hlegal : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true) ∧
v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
comparisonFixedTableRisk K P v ≤
knownRuleRisk K.1 (comparisonReplacementLaw P v) +
(2 * (n : Real) + 4) * ∑ z : AuxObs d, |auxTable P z - v z| := by
let Q := comparisonReplacementLaw P v
let D := ∑ z : AuxObs d, |auxTable P z - v z|
let mu := (labeledProductLaw P n).prod seedLaw
let nu := (labeledProductLaw Q n).prod seedLaw
let f := fun z : (Fin n → Obs d) × Real => (K.1 (z.1, v, z.2) - ateFunctional P) ^ 2
let g := fun z : (Fin n → Obs d) × Real => (K.1 (z.1, v, z.2) - ateFunctional Q) ^ 2
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
let : IsProbabilityMeasure (obsLaw Q) := by unfold obsLaw; infer_instance
let : IsProbabilityMeasure (labeledProductLaw P n) := by
unfold labeledProductLaw; infer_instance
let : IsProbabilityMeasure (labeledProductLaw Q n) := by
unfold labeledProductLaw; infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have htv : tvDist mu nu ≤ (n : Real) * ((1 / 2) * D) := by
rw [labelFloor_tv_common_product]
exact (tvDist_pi_iid_le n _ _).trans
(mul_le_mul_of_nonneg_left (comparisonReplacementLaw_tv P v hv hs)
(Nat.cast_nonneg n))
have hK := K.2.1
have hf : Measurable f := by fun_prop
have hrange : ∀ z, f z ∈ Icc (0 : Real) (0 + 4) := by
intro z
have hb := K.2.2 (z.1, v, z.2)
have ht := ateFunctional_mem_Icc P
refine ⟨sq_nonneg _, ?_⟩
dsimp [f]
nlinarith [hb.1, hb.2, ht.1, ht.2]
have hdist := tvDist_integral_range mu nu f hf 0 4 (by norm_num) hrange
have hfirst : (∫ z, f z ∂mu) ≤ (∫ z, f z ∂nu) + 2 * n * D := by
have h := (le_abs_self ((∫ z, f z ∂mu) - ∫ z, f z ∂nu)).trans hdist
have hb := mul_le_mul_of_nonneg_right htv (by norm_num : (0 : Real) ≤ 4)
nlinarith
have htarget := comparisonReplacementLaw_target_error P v eps heps hv hs hlegal
have hsecond : (∫ z, f z ∂nu) ≤ (∫ z, g z ∂nu) + 4 * D := by
calc
_ ≤ ∫ z, (g z + 4 * D) ∂nu := by
apply integral_mono (comparisonFixedTableLoss_integrable K Q v _)
((comparisonFixedTableLoss_integrable K Q v _).add (integrable_const _))
intro z
have h := (le_abs_self ((K.1 (z.1, v, z.2) - ateFunctional P) ^ 2 -
(K.1 (z.1, v, z.2) - ateFunctional Q) ^ 2)).trans
(comparison_squared_loss_target_error _ _ _ (K.2.2 _)
(ateFunctional_mem_Icc P) (ateFunctional_mem_Icc Q))
dsimp [f, g]
linarith
_ = _ := by rw [integral_add (comparisonFixedTableLoss_integrable K Q v _)
(integrable_const _), integral_const]; simp [nu, g]
have hg : (∫ z, g z ∂nu) = knownRuleRisk K.1 Q := by
simp only [knownRuleRisk, Q, comparisonReplacementLaw_auxTable P v hv hs]
rfl
change (∫ z, f z ∂mu) ≤ _
rw [← hg]
dsimp only [D] at hfirst hsecond ⊢
linarith
Plug a legal auxiliary empirical table into any supplied-table rule.
theorem comparisonPluginRule_measurable
Under the stated hypotheses, the plug-in is jointly Borel in both channels and the seed. This gives the stated conclusion.
Formal statement
hypotheses
conclusion
Measurable (comparisonPluginRule K eps hm)
Proof (Lean source)
@[fun_prop]
-- @node: comparisonPluginRule_measurable
lemma comparisonPluginRule_measurable {n m d : Nat} (K : KnownRule n d)
(eps : Real) (hm : 0 < m) : Measurable (comparisonPluginRule K eps hm) := by
have hK := K.2.1
unfold comparisonPluginRule
fun_prop
theorem comparisonPluginRule_mem_Icc
Under the stated inputs and conditions, The supplied-table rule's range bound also bounds its empirical plug-in. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonPluginRule_mem_Icc
lemma comparisonPluginRule_mem_Icc {n m d : Nat} (K : KnownRule n d)
(eps : Real) (hm : 0 < m) (z : Sample n m d × Real) :
comparisonPluginRule K eps hm z ∈ Icc (-1) 1 := K.2.2 _
theorem comparisonPluginRule_risk_eq
Under the stated inputs and conditions, Product sampling expresses plug-in risk as the auxiliary average of fixed-table risks. This gives the stated result.
Formal statement
hypotheses
conclusion
ruleRisk (comparisonPluginRule K eps hm) P
= ∫ V, comparisonFixedTableRisk K P (comparisonEmpiricalTable eps hm V) ∂auxProductLaw P m
Proof (Lean source)
-- @node: comparisonPluginRule_risk_eq
lemma comparisonPluginRule_risk_eq {n m d : Nat} (K : KnownRule n d)
(eps : Real) (hm : 0 < m) (P : DiscreteLaw d) :
ruleRisk (comparisonPluginRule K eps hm) P =
∫ V, comparisonFixedTableRisk K P (comparisonEmpiricalTable eps hm V)
∂auxProductLaw P m := by
let : IsProbabilityMeasure (labeledProductLaw P n) := by
unfold labeledProductLaw obsLaw; infer_instance
let : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw; infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have hfull : Integrable (fun z =>
(comparisonPluginRule K eps hm z - ateFunctional P) ^ 2)
((annotationLaw P n m).prod seedLaw) :=
mse_integrable_of_estimator_bound (M := 1) _ _
(comparisonPluginRule_measurable K eps hm) (by norm_num)
(comparisonPluginRule_mem_Icc K eps hm)
unfold ruleRisk comparisonFixedTableRisk
rw [integral_prod _ hfull]
change (∫ LV, ∫ u, (comparisonPluginRule K eps hm (LV, u) - ateFunctional P) ^ 2
∂seedLaw ∂((labeledProductLaw P n).prod (auxProductLaw P m))) = _
rw [integral_prod_symm _ hfull.integral_prod_left]
apply integral_congr_ae
exact Filter.Eventually.of_forall fun V => by
dsimp only
rw [integral_prod _ (comparisonFixedTableLoss_integrable K P _ _)]
rfl
theorem comparisonKnownRuleRisk_bddAbove
Under the stated inputs and conditions, Every supplied-table rule has uniformly bounded squared risk, so its supremum is finite. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonKnownRuleRisk_bddAbove
lemma comparisonKnownRuleRisk_bddAbove {n d : Nat} (K : KnownRule n d) (eps : Real) :
BddAbove (range (fun P : ClassLaw d eps => knownRuleRisk K.1 P.1)) := by
refine ⟨4, ?_⟩
rintro _ ⟨P, rfl⟩
let : IsProbabilityMeasure (labeledProductLaw P.1 n) := by
unfold labeledProductLaw obsLaw; infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
exact labelFloor_bounded_mse ((labeledProductLaw P.1 n).prod seedLaw)
(fun z : (Fin n → Obs d) × Real => K.1 (z.1, auxTable P.1, z.2))
(fun z => K.2.2 _) _ (ateFunctional_mem_Icc P.1)
theorem comparisonPluginRule_risk_le
Under the stated inputs and conditions, Conditional transport and the expected empirical L¹ bound give the plug-in risk guarantee. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonPluginRule_risk_le
lemma comparisonPluginRule_risk_le {n m d : Nat} (K : KnownRule n d)
(eps : Real) (hm : 0 < m) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(P : ClassLaw d eps) :
ruleRisk (comparisonPluginRule K eps hm) P.1 ≤
(⨆ Q : ClassLaw d eps, knownRuleRisk K.1 Q.1) +
6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m) := by
let : IsProbabilityMeasure (auxProductLaw P.1 m) := by
unfold auxProductLaw; infer_instance
let W := ⨆ Q : ClassLaw d eps, knownRuleRisk K.1 Q.1
let D := fun V : Fin m → AuxObs d => ∑ z : AuxObs d,
|auxTable P.1 z - comparisonEmpiricalTable eps hm V z|
have hcond (V : Fin m → AuxObs d) :
comparisonFixedTableRisk K P.1 (comparisonEmpiricalTable eps hm V) ≤
W + (2 * (n : Real) + 4) * D V := by
obtain ⟨hv, hs⟩ := comparisonEmpiricalTable_probability eps hm hd heps.le (by linarith) V
have hlegal := comparisonEmpiricalTable_overlap eps hm heps.le (by linarith) V
let Q : ClassLaw d eps := ⟨comparisonReplacementLaw P.1
(comparisonEmpiricalTable eps hm V),
comparisonReplacementLaw_model P.1 _ eps hv hs hlegal⟩
exact (comparisonFixedTableRisk_transport K P.1 _ eps heps hv hs hlegal).trans
(add_le_add (le_ciSup (comparisonKnownRuleRisk_bddAbove K eps) Q) le_rfl)
have hmean : (∫ V, D V ∂auxProductLaw P.1 m) ≤
3 * sqrt (2 * (d : Real) / m) := by
simpa only [D, abs_sub_comm] using
comparisonEmpiricalTable_expected_l1 eps hm heps.le (by linarith) P.1 P.2
rw [comparisonPluginRule_risk_eq]
calc
_ ≤ ∫ V, (W + (2 * (n : Real) + 4) * D V) ∂auxProductLaw P.1 m :=
integral_mono Integrable.of_finite Integrable.of_finite hcond
_ = W + (2 * (n : Real) + 4) * (∫ V, D V ∂auxProductLaw P.1 m) := by
rw [integral_add (integrable_const _) (Integrable.of_finite.const_mul _),
integral_const_mul, integral_const]
simp
_ ≤ W + (2 * (n : Real) + 4) * (3 * sqrt (2 * (d : Real) / m)) :=
add_le_add le_rfl (mul_le_mul_of_nonneg_left hmean
(by positivity : 0 ≤ 2 * (n : Real) + 4))
_ = _ := by dsimp only [W]; ring
theorem comparison_minimax_le_known_add
Under the stated inputs and conditions, Taking rule infima after the plug-in construction gives the numerical experiment bound. This gives the stated result.
Formal statement
hypotheses
conclusion
minimaxRisk n m d eps
Proof (Lean source)
-- @node: comparison_minimax_le_known_add
lemma comparison_minimax_le_known_add (n m d : Nat) (eps : Real)
(hm : 1 ≤ m) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
minimaxRisk n m d eps ≤ knownMarginalRisk n d eps +
6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m) := by
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps'⟩
let : Nonempty (ClassLaw d eps) := ⟨P0⟩
let K0 : KnownRule n d := ⟨fun _ => 0, measurable_const, fun _ => by norm_num⟩
let : Nonempty (KnownRule n d) := ⟨K0⟩
let err := 6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m)
have hinf : minimaxRisk n m d eps - err ≤ knownMarginalRisk n d eps := by
unfold knownMarginalRisk
apply le_ciInf
intro K
let T : Rule n m d := ⟨comparisonPluginRule K eps (by omega),
comparisonPluginRule_measurable K eps _, comparisonPluginRule_mem_Icc K eps _⟩
have hmin : minimaxRisk n m d eps ≤ worstRisk T.1 eps :=
minimaxValue_le_worstCaseRisk_of_nonneg
(risk := fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
(fun T P => integral_nonneg (fun z => sq_nonneg _)) T
have hworst : worstRisk T.1 eps ≤
(⨆ Q : ClassLaw d eps, knownRuleRisk K.1 Q.1) + err := by
apply ciSup_le
intro P
exact comparisonPluginRule_risk_le K eps _ hd heps heps' P
exact sub_le_iff_le_add.mpr (hmin.trans hworst)
exact sub_le_iff_le_add.mp hinf
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.ComparisonReplacement 14 declarations Proof-only replacement of a population marginal while retaining its binary outcome regressions.
Proof-only replacement of a population marginal while retaining its binary outcome regressions. This realizes the comparison law in the benchmark-recovery roadmap.
theorem comparisonOutcomeWeight_nonneg
Under the stated inputs and conditions, Binary conditional outcome weights are nonnegative, including null original arms. This gives the stated result.
Formal statement
hypotheses
conclusion
0 ≤ bernoulliMass (outcomeMean P a j) y
Proof (Lean source)
-- @node: comparisonOutcomeWeight_nonneg
lemma comparisonOutcomeWeight_nonneg {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a y : Bool) : 0 ≤ bernoulliMass (outcomeMean P a j) y := by
have h := outcomeMean_mem_Icc P a j
cases y <;> simp only [bernoulliMass, Bool.false_eq_true, if_false, if_true] <;>
linarith [h.1, h.2]
theorem comparisonOutcomeWeight_sum
Under the stated inputs and conditions, The two conditional outcome weights sum to one. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonOutcomeWeight_sum
lemma comparisonOutcomeWeight_sum {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a : Bool) : ∑ y : Bool, bernoulliMass (outcomeMean P a j) y = 1 := by
simp [bernoulliMass]
theorem comparison_jointMass_factor
Under the stated inputs and conditions, Every original atom factors into its arm mass and its conditional outcome weight. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparison_jointMass_factor
lemma comparison_jointMass_factor {d : Nat} (P : DiscreteLaw d)
(j : Fin d) (a y : Bool) :
jointMass P j a y = armMass P j a * bernoulliMass (outcomeMean P a j) y := by
have hs := armMass_nonneg P j a
have hq := jointMass_nonneg P j a true
have hf := jointMass_nonneg P j a false
have hsum : armMass P j a = jointMass P j a true + jointMass P j a false := by
simp [armMass]
by_cases hz : armMass P j a = 0
· have ht : jointMass P j a true = 0 := by linarith
have hc : jointMass P j a false = 0 := by linarith
cases y <;> simp [hz, ht, hc]
· have hprod : armMass P j a * outcomeMean P a j = jointMass P j a true := by
unfold outcomeMean markedMass
field_simp
cases y <;> simp only [bernoulliMass, Bool.false_eq_true, if_false, if_true]
· nlinarith
· exact hprod.symm
def comparisonReplacementLaw
Attach the original outcome regressions to an arbitrary supplied table, with a population fallback on ambient inputs that cannot be normalized.
Definition (Lean source)
parameters
def
given by
⟨normalizedPMF (fun z => v (z.1, z.2.1) * bernoulliMass (outcomeMean P z.2.1 z.1) z.2.2) P.pmf⟩
theorem comparisonReplacementLaw_jointMass
Under the stated inputs and conditions, A nonnegative probability table gives exactly the specified replacement atoms. This gives the stated result.
Formal statement
hypotheses
conclusion
jointMass (comparisonReplacementLaw P v) j a y
= v (j, a) * bernoulliMass (outcomeMean P a j) y
Proof (Lean source)
-- @node: comparisonReplacementLaw_jointMass
lemma comparisonReplacementLaw_jointMass {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(j : Fin d) (a y : Bool) :
jointMass (comparisonReplacementLaw P v) j a y =
v (j, a) * bernoulliMass (outcomeMean P a j) y := by
apply normalizedPMF_toReal
· intro z
exact mul_nonneg (hv _) (comparisonOutcomeWeight_nonneg P _ _ _)
· simpa only [Fintype.sum_prod_type, ← Finset.mul_sum,
comparisonOutcomeWeight_sum, mul_one] using hs
theorem comparisonReplacementLaw_armMass
Under the stated inputs and conditions, The arm masses of the replacement law are the supplied table entries. This gives the stated result.
Formal statement
hypotheses
conclusion
armMass (comparisonReplacementLaw P v) j a = v (j, a)
Proof (Lean source)
-- @node: comparisonReplacementLaw_armMass
lemma comparisonReplacementLaw_armMass {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(j : Fin d) (a : Bool) : armMass (comparisonReplacementLaw P v) j a = v (j, a) := by
simp only [armMass, comparisonReplacementLaw_jointMass P v hv hs,
← Finset.mul_sum, comparisonOutcomeWeight_sum, mul_one]
theorem comparisonReplacementLaw_auxTable
Under the stated inputs and conditions, The auxiliary table is exactly the requested replacement marginal. This gives the stated result.
Formal statement
hypotheses
conclusion
auxTable (comparisonReplacementLaw P v) = v
Proof (Lean source)
-- @node: comparisonReplacementLaw_auxTable
lemma comparisonReplacementLaw_auxTable {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1) :
auxTable (comparisonReplacementLaw P v) = v := by
funext z
exact (auxMarginal_toReal_armMass _ z.1 z.2).trans
(comparisonReplacementLaw_armMass P v hv hs z.1 z.2)
theorem comparisonReplacementLaw_cellMass
Under the stated inputs and conditions, Covariate masses sum the supplied arm entries. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonReplacementLaw_cellMass
lemma comparisonReplacementLaw_cellMass {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(j : Fin d) : cellMass (comparisonReplacementLaw P v) j = v (j, true) + v (j, false) := by
change (∑ a : Bool, armMass (comparisonReplacementLaw P v) j a) = _
simp only [comparisonReplacementLaw_armMass P v hv hs, Fintype.sum_bool]
theorem comparisonReplacementLaw_model
Under the stated inputs and conditions, A legal replacement marginal gives a law in the same overlap model. This gives the stated result.
Formal statement
hypotheses
conclusion
ModelClass d eps (comparisonReplacementLaw P v)
Proof (Lean source)
-- @node: comparisonReplacementLaw_model
lemma comparisonReplacementLaw_model {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (eps : Real) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(hlegal : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true) ∧
v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
ModelClass d eps (comparisonReplacementLaw P v) := by
constructor
intro j hp
have hz : cellMass (comparisonReplacementLaw P v) j ≠ 0 := ne_of_gt hp
unfold propensity
rw [if_neg hz]
simp only [comparisonReplacementLaw_armMass P v hv hs,
comparisonReplacementLaw_cellMass P v hv hs] at hp ⊢
constructor
· exact (le_div_iff₀ hp).mpr (by simpa only [add_comm] using (hlegal j).1)
· exact (div_le_iff₀ hp).mpr (by simpa only [add_comm] using (hlegal j).2)
theorem comparisonReplacementLaw_outcomeMean
Under the stated inputs and conditions, In every positive replacement arm the conditional mean is inherited, even if the original arm was null (its inherited mean is then zero). This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonReplacementLaw_outcomeMean
lemma comparisonReplacementLaw_outcomeMean {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(j : Fin d) (a : Bool) (ha : v (j, a) ≠ 0) :
outcomeMean (comparisonReplacementLaw P v) a j = outcomeMean P a j := by
simp only [outcomeMean, markedMass, comparisonReplacementLaw_jointMass P v hv hs,
comparisonReplacementLaw_armMass P v hv hs, bernoulliMass, if_true]
exact mul_div_cancel_left₀ _ ha
theorem comparisonReplacementLaw_l1
Under the stated inputs and conditions, Retaining the binary conditional weights preserves the complete-table L¹ distance. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ z : Obs d, |jointMass P z.1 z.2.1 z.2.2 - jointMass (comparisonReplacementLaw P v) z.1 z.2.1 z.2.2|)
Proof (Lean source)
-- @node: comparisonReplacementLaw_l1
lemma comparisonReplacementLaw_l1 {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1) :
(∑ z : Obs d, |jointMass P z.1 z.2.1 z.2.2 -
jointMass (comparisonReplacementLaw P v) z.1 z.2.1 z.2.2|) =
∑ z : AuxObs d, |auxTable P z - v z| := by
simp only [Fintype.sum_prod_type]
apply Finset.sum_congr rfl
intro j _
apply Finset.sum_congr rfl
intro a _
simp only [comparisonReplacementLaw_jointMass P v hv hs, comparison_jointMass_factor P,
← sub_mul, abs_mul, abs_of_nonneg (comparisonOutcomeWeight_nonneg P j a _),
← Finset.mul_sum, comparisonOutcomeWeight_sum, mul_one, auxTable,
auxMarginal_toReal_armMass]
theorem comparisonReplacementLaw_weighted_mean
Under the stated inputs and conditions, In a legal occupied replacement cell, both inherited arm means are unchanged. This gives the stated result.
Formal statement
hypotheses
d :
P :
v :
AuxTable d
eps :
heps :
0 < eps
hv :
∀ z, 0 ≤ v z
hs :
∑ z, v z = 1
hlegal :
j :
Fin d
a :
conclusion
= cellMass (comparisonReplacementLaw P v) j * outcomeMean P a j
Proof (Lean source)
-- @node: comparisonReplacementLaw_weighted_mean
lemma comparisonReplacementLaw_weighted_mean {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (eps : Real) (heps : 0 < eps)
(hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(hlegal : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true) ∧
v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true)))
(j : Fin d) (a : Bool) :
cellMass (comparisonReplacementLaw P v) j *
outcomeMean (comparisonReplacementLaw P v) a j =
cellMass (comparisonReplacementLaw P v) j * outcomeMean P a j := by
by_cases hz : cellMass (comparisonReplacementLaw P v) j = 0
· simp only [hz, zero_mul]
· have hp : 0 < cellMass (comparisonReplacementLaw P v) j :=
lt_of_le_of_ne (cellMass_nonneg _ _) (Ne.symm hz)
have hmodel := comparisonReplacementLaw_model P v eps hv hs hlegal
have ha := (mul_pos heps hp).trans_le
(armMass_ge_overlap_cellMass _ eps hmodel j a)
rw [comparisonReplacementLaw_armMass P v hv hs] at ha
rw [comparisonReplacementLaw_outcomeMean P v hv hs j a (ne_of_gt ha)]
theorem comparisonReplacementLaw_target
Under the stated inputs and conditions, The replacement target is the original regression contrast under the new cell weights. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonReplacementLaw_target
lemma comparisonReplacementLaw_target {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (eps : Real) (heps : 0 < eps)
(hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(hlegal : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true) ∧
v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
ateFunctional (comparisonReplacementLaw P v) =
∑ j, (v (j, true) + v (j, false)) *
(outcomeMean P true j - outcomeMean P false j) := by
unfold ateFunctional
apply Finset.sum_congr rfl
intro j _
rw [mul_sub, comparisonReplacementLaw_weighted_mean P v eps heps hv hs hlegal,
comparisonReplacementLaw_weighted_mean P v eps heps hv hs hlegal,
← mul_sub, comparisonReplacementLaw_cellMass P v hv hs]
theorem comparisonReplacementLaw_target_error
Under the stated inputs and conditions, Moving a legal marginal changes the target by at most the marginal L¹ distance. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonReplacementLaw_target_error
lemma comparisonReplacementLaw_target_error {d : Nat} (P : DiscreteLaw d)
(v : AuxTable d) (eps : Real) (heps : 0 < eps)
(hv : ∀ z, 0 ≤ v z) (hs : ∑ z, v z = 1)
(hlegal : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true) ∧
v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
|ateFunctional P - ateFunctional (comparisonReplacementLaw P v)| ≤
∑ z : AuxObs d, |auxTable P z - v z| := by
have hcell (j : Fin d) : cellMass P j =
auxTable P (j, true) + auxTable P (j, false) := by
simp only [auxTable, auxMarginal_toReal_armMass, cellMass, armMass, Fintype.sum_bool]
have hcontrast (j : Fin d) : |outcomeMean P true j - outcomeMean P false j| ≤ 1 := by
have ht := outcomeMean_mem_Icc P true j
have hc := outcomeMean_mem_Icc P false j
exact abs_le.mpr ⟨by linarith [ht.1, hc.2], by linarith [ht.2, hc.1]⟩
rw [comparisonReplacementLaw_target P v eps heps hv hs hlegal]
unfold ateFunctional
rw [← Finset.sum_sub_distrib]
calc
_ ≤ ∑ j : Fin d, |cellMass P j * (outcomeMean P true j - outcomeMean P false j) -
(v (j, true) + v (j, false)) *
(outcomeMean P true j - outcomeMean P false j)| := Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ j : Fin d, (|auxTable P (j, true) - v (j, true)| +
|auxTable P (j, false) - v (j, false)|) := by
apply Finset.sum_le_sum
intro j _
rw [← sub_mul, abs_mul]
calc
_ ≤ |cellMass P j - (v (j, true) + v (j, false))| * 1 :=
mul_le_mul_of_nonneg_left (hcontrast j) (abs_nonneg _)
_ = |(auxTable P (j, true) - v (j, true)) +
(auxTable P (j, false) - v (j, false))| := by
rw [mul_one, hcell]; congr 1; ring
_ ≤ _ := abs_add_le _ _
_ = _ := by simp only [Fintype.sum_prod_type, Fintype.sum_bool]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.EmpiricalTableAdjustment 24 declarations Legal overlap adjustment of empirical marginal tables, its factor-three L¹ bound, and the expected empirical approximation error.
Legal overlap adjustment of empirical marginal tables, its factor-three L¹ bound, and the expected empirical approximation error.
Clamp the treated mass while keeping the empirical covariate mass fixed.
The legal adjustment is total on the ambient real table space.
theorem comparisonAdjustedTable_measurable
The adjustment is Borel, including at empty cells. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: comparisonAdjustedTable_measurable
lemma comparisonAdjustedTable_measurable (d : Nat) (eps : Real) :
Measurable (comparisonAdjustedTable (d := d) eps) := by
apply measurable_pi_lambda
intro z
unfold comparisonAdjustedTable comparisonAdjustedTreated
split_ifs <;> fun_prop
theorem comparisonAdjustedTable_cell_sum
Under the stated inputs and conditions, Adjustment preserves the covariate mass cell by cell. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonAdjustedTable_cell_sum
lemma comparisonAdjustedTable_cell_sum {d : Nat} (eps : Real) (v : AuxTable d)
(j : Fin d) :
comparisonAdjustedTable eps v (j, false) + comparisonAdjustedTable eps v (j, true) =
v (j, false) + v (j, true) := by
simp only [comparisonAdjustedTable, Bool.false_eq_true, if_false, if_true]
ring
theorem comparisonAdjustedTreated_bounds
Under the stated inputs and conditions, A nonnegative cell is adjusted into the overlap interval. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
eps * (x0 + x1) ≤ comparisonAdjustedTreated eps x0 x1
conclusion 2
comparisonAdjustedTreated eps x0 x1 ≤ (1 - eps) * (x0 + x1)
Proof (Lean source)
-- @node: comparisonAdjustedTreated_bounds
lemma comparisonAdjustedTreated_bounds (eps x0 x1 : Real)
(_heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (hx0 : 0 ≤ x0) (hx1 : 0 ≤ x1) :
eps * (x0 + x1) ≤ comparisonAdjustedTreated eps x0 x1 ∧
comparisonAdjustedTreated eps x0 x1 ≤ (1 - eps) * (x0 + x1) := by
have hends : eps * (x0 + x1) ≤ (1 - eps) * (x0 + x1) := by
nlinarith
exact ⟨le_min hends (le_max_left _ _), min_le_left _ _⟩
theorem comparisonAdjustedTreated_movement
Under the stated inputs and conditions, Relative to a legal population cell, moving the treated entry costs at most its L¹ error. This gives the stated result.
Formal statement
hypotheses
eps x0 x1 v0 v1 :
heps :
0 ≤ eps
heps' :
eps ≤ 1
hlo :
eps * (v0 + v1) ≤ v1
hhi :
v1 ≤ (1 - eps) * (v0 + v1)
hends :
eps * (x0 + x1) ≤ (1 - eps) * (x0 + x1)
conclusion
|comparisonAdjustedTreated eps x0 x1 - x1| ≤ |x0 - v0| + |x1 - v1|
Proof (Lean source)
-- @node: comparisonAdjustedTreated_movement
lemma comparisonAdjustedTreated_movement (eps x0 x1 v0 v1 : Real)
(heps : 0 ≤ eps) (heps' : eps ≤ 1)
(hlo : eps * (v0 + v1) ≤ v1) (hhi : v1 ≤ (1 - eps) * (v0 + v1))
(hends : eps * (x0 + x1) ≤ (1 - eps) * (x0 + x1)) :
|comparisonAdjustedTreated eps x0 x1 - x1| ≤ |x0 - v0| + |x1 - v1| := by
have h0 := abs_le.mp (le_refl |x0 - v0|)
have h1 := abs_le.mp (le_refl |x1 - v1|)
have he1 : 0 ≤ 1 - eps := by linarith
have h0e := mul_le_mul_of_nonneg_left h0.2 heps
have h1e := mul_le_mul_of_nonneg_left h1.2 heps
have h0c := mul_le_mul_of_nonneg_left h0.1 he1
have h1c := mul_le_mul_of_nonneg_left h1.1 he1
have hb0 := mul_le_mul_of_nonneg_right heps' (abs_nonneg (x0 - v0))
have hb1 := mul_le_mul_of_nonneg_right heps' (abs_nonneg (x1 - v1))
have hc0 := mul_le_mul_of_nonneg_right (show 1 - eps ≤ 1 by linarith)
(abs_nonneg (x0 - v0))
have hc1 := mul_le_mul_of_nonneg_right (show 1 - eps ≤ 1 by linarith)
(abs_nonneg (x1 - v1))
unfold comparisonAdjustedTreated
by_cases hl : x1 ≤ eps * (x0 + x1)
· rw [max_eq_left hl, min_eq_right hends, abs_of_nonneg (by linarith)]
nlinarith
· rw [max_eq_right (le_of_not_ge hl)]
by_cases hu : (1 - eps) * (x0 + x1) ≤ x1
· rw [min_eq_left hu, abs_of_nonpos (by linarith)]
nlinarith
· rw [min_eq_right (le_of_not_ge hu), sub_self, abs_zero]
positivity
theorem comparisonAdjustedTable_cell_error
Under the stated inputs and conditions, Both entries together incur at most three times their original L¹ error. This gives the stated result.
Formal statement
hypotheses
d :
eps :
x v :
AuxTable d
heps :
0 ≤ eps
heps' :
eps ≤ 1 / 2
j :
Fin d
hx0 :
0 ≤ x (j, false)
hx1 :
0 ≤ x (j, true)
conclusion
|comparisonAdjustedTable eps x (j, false)
- v (j, false)|
+ |comparisonAdjustedTable eps x (j, true)
- v (j, true)|
Proof (Lean source)
-- @node: comparisonAdjustedTable_cell_error
lemma comparisonAdjustedTable_cell_error {d : Nat} (eps : Real) (x v : AuxTable d)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (j : Fin d)
(hx0 : 0 ≤ x (j, false)) (hx1 : 0 ≤ x (j, true))
(hlo : eps * (v (j, false) + v (j, true)) ≤ v (j, true))
(hhi : v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
|comparisonAdjustedTable eps x (j, false) - v (j, false)| +
|comparisonAdjustedTable eps x (j, true) - v (j, true)| ≤
3 * (|x (j, false) - v (j, false)| + |x (j, true) - v (j, true)|) := by
let h := comparisonAdjustedTreated eps (x (j, false)) (x (j, true))
have hmove : |h - x (j, true)| ≤
|x (j, false) - v (j, false)| + |x (j, true) - v (j, true)| :=
comparisonAdjustedTreated_movement eps _ _ _ _ heps (by linarith) hlo hhi (by
nlinarith)
have hcontrol : |x (j, false) + x (j, true) - h - v (j, false)| ≤
|x (j, false) - v (j, false)| + |h - x (j, true)| := by
calc
_ = |(x (j, false) - v (j, false)) + -(h - x (j, true))| := by congr 1; ring
_ ≤ |x (j, false) - v (j, false)| + |-(h - x (j, true))| := abs_add_le _ _
_ = _ := by rw [abs_neg]
have htreated : |h - v (j, true)| ≤ |h - x (j, true)| + |x (j, true) - v (j, true)| :=
abs_sub_le h (x (j, true)) (v (j, true))
change |x (j, false) + x (j, true) - h - v (j, false)| + |h - v (j, true)| ≤ _
linarith
theorem comparisonAdjustedTable_l1_error
Under the stated inputs and conditions, Summing the cell bounds gives the factor-three table adjustment estimate in the roadmap. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ z : AuxObs d, |comparisonAdjustedTable eps x z - v z|)
≤ 3 * ∑ z : AuxObs d, |x z - v z|
Proof (Lean source)
-- @node: comparisonAdjustedTable_l1_error
lemma comparisonAdjustedTable_l1_error {d : Nat} (eps : Real) (x v : AuxTable d)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (hx : ∀ z, 0 ≤ x z)
(hlo : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true))
(hhi : ∀ j, v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
(∑ z : AuxObs d, |comparisonAdjustedTable eps x z - v z|) ≤
3 * ∑ z : AuxObs d, |x z - v z| := by
simp only [Fintype.sum_prod_type, Fintype.sum_bool]
rw [Finset.mul_sum]
apply Finset.sum_le_sum
intro j _
simpa only [add_comm] using comparisonAdjustedTable_cell_error eps x v heps heps' j
(hx _) (hx _) (hlo j) (hhi j)
theorem comparisonAdjustedTable_nonneg
Under the stated inputs and conditions, Nonnegative input coordinates remain nonnegative after adjustment. This gives the stated result.
Formal statement
hypotheses
conclusion
0 ≤ comparisonAdjustedTable eps x z
Proof (Lean source)
-- @node: comparisonAdjustedTable_nonneg
lemma comparisonAdjustedTable_nonneg {d : Nat} (eps : Real) (x : AuxTable d)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (hx : ∀ z, 0 ≤ x z) :
∀ z, 0 ≤ comparisonAdjustedTable eps x z := by
intro ⟨j, a⟩
obtain ⟨hlo, hhi⟩ := comparisonAdjustedTreated_bounds eps (x (j, false)) (x (j, true))
heps heps' (hx _) (hx _)
have hp : 0 ≤ x (j, false) + x (j, true) := add_nonneg (hx _) (hx _)
cases a <;> simp only [comparisonAdjustedTable, Bool.false_eq_true, if_false, if_true]
· nlinarith
· exact (mul_nonneg heps hp).trans hlo
theorem comparisonAdjustedTable_sum
Under the stated inputs and conditions, A normalized empirical table stays normalized. This gives the stated result.
Formal statement
hypotheses
conclusion
∑ z, comparisonAdjustedTable eps x z = 1
Proof (Lean source)
-- @node: comparisonAdjustedTable_sum
lemma comparisonAdjustedTable_sum {d : Nat} (eps : Real) (x : AuxTable d)
(hsum : ∑ z, x z = 1) : ∑ z, comparisonAdjustedTable eps x z = 1 := by
simp only [Fintype.sum_prod_type, Fintype.sum_bool] at hsum ⊢
simpa only [add_comm, comparisonAdjustedTable_cell_sum] using hsum
theorem comparisonAdjustedTable_overlap
Under the stated inputs and conditions, The adjusted table satisfies both arm overlap inequalities, including at null cells. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
≤ comparisonAdjustedTable eps x (j, true)
conclusion 2
comparisonAdjustedTable eps x (j, true)
Proof (Lean source)
-- @node: comparisonAdjustedTable_overlap
lemma comparisonAdjustedTable_overlap {d : Nat} (eps : Real) (x : AuxTable d)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (hx : ∀ z, 0 ≤ x z) (j : Fin d) :
eps * (comparisonAdjustedTable eps x (j, false) + comparisonAdjustedTable eps x (j, true)) ≤
comparisonAdjustedTable eps x (j, true) ∧
comparisonAdjustedTable eps x (j, true) ≤ (1 - eps) *
(comparisonAdjustedTable eps x (j, false) + comparisonAdjustedTable eps x (j, true)) := by
rw [comparisonAdjustedTable_cell_sum]
exact comparisonAdjustedTreated_bounds eps (x (j, false)) (x (j, true))
heps heps' (hx _) (hx _)
theorem comparisonAdjustedTable_eq_self
Under the stated inputs and conditions, A legal input table is fixed by the adjustment. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonAdjustedTable_eq_self
lemma comparisonAdjustedTable_eq_self {d : Nat} (eps : Real) (v : AuxTable d)
(hlo : ∀ j, eps * (v (j, false) + v (j, true)) ≤ v (j, true))
(hhi : ∀ j, v (j, true) ≤ (1 - eps) * (v (j, false) + v (j, true))) :
comparisonAdjustedTable eps v = v := by
funext ⟨j, a⟩
have ht : comparisonAdjustedTreated eps (v (j, false)) (v (j, true)) = v (j, true) := by
rw [comparisonAdjustedTreated, max_eq_right (hlo j), min_eq_right (hhi j)]
cases a <;> simp [comparisonAdjustedTable, ht]
theorem comparison_population_table_overlap
Under the stated inputs and conditions, The population marginal has exactly the legal overlap bounds used in the L¹ estimate. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparison_population_table_overlap
lemma comparison_population_table_overlap {d : Nat} (eps : Real) (P : DiscreteLaw d)
(hP : ModelClass d eps P) (j : Fin d) :
eps * (auxTable P (j, false) + auxTable P (j, true)) ≤ auxTable P (j, true) ∧
auxTable P (j, true) ≤ (1 - eps) * (auxTable P (j, false) + auxTable P (j, true)) := by
have ht := armMass_ge_overlap_cellMass P eps hP j true
have hc := armMass_ge_overlap_cellMass P eps hP j false
have hp : armMass P j false + armMass P j true = cellMass P j := by
simp [cellMass, armMass, add_comm]
simp only [auxTable, auxMarginal_toReal_armMass, hp]
exact ⟨ht, by linarith⟩
Auxiliary records determine the adjusted empirical table without using labeled data.
Definition (Lean source)
parameters
def
given by
theorem comparisonEmpiricalTable_measurable
Under the stated hypotheses, the adjusted empirical table is a total Borel function of the auxiliary array. This gives the stated conclusion.
Formal statement
hypotheses
conclusion
Measurable (comparisonEmpiricalTable (d := d) eps hm)
Proof (Lean source)
@[fun_prop]
-- @node: comparisonEmpiricalTable_measurable
lemma comparisonEmpiricalTable_measurable (m d : Nat) (eps : Real) (hm : 0 < m) :
Measurable (comparisonEmpiricalTable (d := d) eps hm) := by
exact measurable_of_countable _
theorem comparisonEmpiricalTable_probability
Under the stated inputs and conditions, Empirical adjustment preserves positivity and the normalization to a probability table. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 2
(∑ z, comparisonEmpiricalTable eps hm V z) = 1
Proof (Lean source)
-- @node: comparisonEmpiricalTable_probability
lemma comparisonEmpiricalTable_probability {m d : Nat} (eps : Real) (hm : 0 < m)
(hd : 2 ≤ d) (heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (V : Fin m → AuxObs d) :
(∀ z, 0 ≤ comparisonEmpiricalTable eps hm V z) ∧
(∑ z, comparisonEmpiricalTable eps hm V z) = 1 := by
let : Nonempty (Fin d) := ⟨⟨0, by omega⟩⟩
exact ⟨comparisonAdjustedTable_nonneg eps _ heps heps'
(empiricalFrequency_nonneg _ hm V),
comparisonAdjustedTable_sum eps _
(sum_empiricalFrequency _ hm V)⟩
theorem comparisonEmpiricalTable_overlap
Under the stated inputs and conditions, The adjusted empirical table belongs to the legal marginal set. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
≤ comparisonEmpiricalTable eps hm V (j, true)
conclusion 2
comparisonEmpiricalTable eps hm V (j, true)
≤ (1 - eps) * (comparisonEmpiricalTable eps hm V (j, false) + comparisonEmpiricalTable eps hm V (j, true))
Proof (Lean source)
-- @node: comparisonEmpiricalTable_overlap
lemma comparisonEmpiricalTable_overlap {m d : Nat} (eps : Real) (hm : 0 < m)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (V : Fin m → AuxObs d) (j : Fin d) :
eps * (comparisonEmpiricalTable eps hm V (j, false) +
comparisonEmpiricalTable eps hm V (j, true)) ≤
comparisonEmpiricalTable eps hm V (j, true) ∧
comparisonEmpiricalTable eps hm V (j, true) ≤ (1 - eps) *
(comparisonEmpiricalTable eps hm V (j, false) +
comparisonEmpiricalTable eps hm V (j, true)) := by
exact comparisonAdjustedTable_overlap eps _ heps heps'
(empiricalFrequency_nonneg _ hm V) j
theorem comparisonEmpiricalTable_l1_error
Under the stated inputs and conditions, The data-only legal adjustment costs at most three times the raw empirical L¹ error. This gives the stated result.
Formal statement
hypotheses
conclusion
- auxTable P z|
Proof (Lean source)
-- @node: comparisonEmpiricalTable_l1_error
lemma comparisonEmpiricalTable_l1_error {m d : Nat} (eps : Real) (hm : 0 < m)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (P : DiscreteLaw d)
(hP : ModelClass d eps P) (V : Fin m → AuxObs d) :
(∑ z : AuxObs d, |comparisonEmpiricalTable eps hm V z - auxTable P z|) ≤
3 * ∑ z : AuxObs d,
|empiricalFrequency (AuxObs d) hm V z -
auxTable P z| := by
exact comparisonAdjustedTable_l1_error eps _ _ heps heps'
(empiricalFrequency_nonneg _ hm V)
(fun j => (comparison_population_table_overlap eps P hP j).1)
(fun j => (comparison_population_table_overlap eps P hP j).2)
theorem comparisonEmpiricalFrequency_eq_mass Lemma comparisonEmpiricalFrequency_eq_mass in the paper ↗
Under the stated inputs and conditions, The library's empirical mass and the empirical-frequency table agree. This gives the stated result.
Formal statement
hypotheses
conclusion
empiricalFrequency (AuxObs d) hm V z
= empiricalMass V z
Proof (Lean source)
-- @node: comparisonEmpiricalFrequency_eq_mass
lemma comparisonEmpiricalFrequency_eq_mass {m d : Nat} (hm : 0 < m)
(V : Fin m → AuxObs d) (z : AuxObs d) :
empiricalFrequency (AuxObs d) hm V z =
empiricalMass V z := by
classical
simp only [empiricalFrequency,
empiricalMass, div_eq_mul_inv, mul_comm]
congr 1
apply Finset.sum_congr rfl
intro i _
split_ifs <;> rfl
theorem comparisonEmpiricalFrequency_mse
Under the stated inputs and conditions, Each empirical marginal coordinate has the exact binomial mean squared error. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonEmpiricalFrequency_mse
lemma comparisonEmpiricalFrequency_mse {m d : Nat} (hm : 0 < m)
(P : DiscreteLaw d) (z : AuxObs d) :
(∫ V, (empiricalFrequency
(AuxObs d) hm V z - auxTable P z) ^ 2 ∂auxProductLaw P m) =
(auxTable P z - (auxTable P z) ^ 2) / m := by
classical
let μ := (auxMarginal P).toMeasure
let : IsProbabilityMeasure μ := by dsimp [μ]; infer_instance
let : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
have hatom : μ.real {z} = auxTable P z := by
simp [μ, auxTable, measureReal_def]
have hmean := integral_empiricalMass μ hm z (measurableSet_singleton z)
have hsecond := integral_empiricalMass_sq μ hm z
(measurableSet_singleton z)
rw [hatom] at hmean hsecond
change (∫ V, _ ∂Measure.pi (fun _ : Fin m => μ)) = _ at hmean hsecond
simp_rw [comparisonEmpiricalFrequency_eq_mass]
have hexpand : (fun V : Fin m → AuxObs d =>
(empiricalMass V z - auxTable P z) ^ 2) =
fun V => empiricalMass V z ^ 2 -
2 * auxTable P z * empiricalMass V z + (auxTable P z) ^ 2 := by
funext V; ring
rw [hexpand]
have hi : Integrable (fun V : Fin m → AuxObs d =>
empiricalMass V z ^ 2 -
2 * auxTable P z * empiricalMass V z) (auxProductLaw P m) :=
Integrable.of_finite
have hi2 : Integrable (fun V : Fin m → AuxObs d =>
empiricalMass V z ^ 2) (auxProductLaw P m) := Integrable.of_finite
have hi1 : Integrable (fun V : Fin m → AuxObs d =>
2 * auxTable P z * empiricalMass V z) (auxProductLaw P m) :=
Integrable.of_finite
integral_linearity
change (∫ V, empiricalMass V z ^ 2 ∂Measure.pi (fun _ : Fin m => μ)) -
2 * auxTable P z * (∫ V, empiricalMass V z
∂Measure.pi (fun _ : Fin m => μ)) + _ = _
rw [hmean, hsecond]
simp only [integral_const, measure_univ, ENNReal.toReal_one, smul_eq_mul, one_mul,
measureReal_def]
ring
Under the stated inputs and conditions, The squared L¹ error is controlled by the sum of squared coordinate errors. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonTable_l1_sq_le
lemma comparisonTable_l1_sq_le {d : Nat} (x v : AuxTable d) :
(∑ z : AuxObs d, |x z - v z|) ^ 2 ≤
(2 * (d : Real)) * ∑ z : AuxObs d, (x z - v z) ^ 2 := by
simpa [AuxObs, Fintype.card_prod, sq_abs, mul_comm] using
(sq_sum_le_card_mul_sum_sq (s := univ)
(f := fun z : AuxObs d => |x z - v z|))
theorem comparisonEmpiricalFrequency_l1_sq
Under the stated inputs and conditions, The empirical marginal's expected squared L¹ error is at most alphabet size over budget. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ V, (∑ z : AuxObs d, |empiricalFrequency (AuxObs d) hm V z - auxTable P z|) ^ 2 ∂auxProductLaw P m)
≤ 2 * (d : Real) / m
Proof (Lean source)
-- @node: comparisonEmpiricalFrequency_l1_sq
lemma comparisonEmpiricalFrequency_l1_sq {m d : Nat} (hm : 0 < m)
(P : DiscreteLaw d) :
(∫ V, (∑ z : AuxObs d,
|empiricalFrequency (AuxObs d) hm V z -
auxTable P z|) ^ 2 ∂auxProductLaw P m) ≤ 2 * (d : Real) / m := by
let : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
have hsum : ∑ z : AuxObs d, auxTable P z = 1 := by
simpa [auxTable] using
(PMF.integral_eq_sum (auxMarginal P) (fun _ => (1 : Real))).symm
calc
_ ≤ ∫ V, (2 * (d : Real)) * ∑ z : AuxObs d,
(empiricalFrequency (AuxObs d) hm V z -
auxTable P z) ^ 2 ∂auxProductLaw P m :=
integral_mono Integrable.of_finite Integrable.of_finite
(fun V => comparisonTable_l1_sq_le _ _)
_ = (2 * (d : Real)) * ∑ z : AuxObs d,
(auxTable P z - (auxTable P z) ^ 2) / m := by
rw [integral_const_mul, integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
simp_rw [comparisonEmpiricalFrequency_mse hm]
_ ≤ (2 * (d : Real)) * ∑ z : AuxObs d, auxTable P z / m := by
gcongr with z
exact sub_le_self _ (sq_nonneg _)
_ = 2 * (d : Real) / m := by rw [← Finset.sum_div, hsum]; ring
theorem comparisonEmpiricalFrequency_expected_l1
Under the stated inputs and conditions, Nonnegative variance converts the squared L¹ bound into its expected-error bound. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: comparisonEmpiricalFrequency_expected_l1
lemma comparisonEmpiricalFrequency_expected_l1 {m d : Nat} (hm : 0 < m)
(P : DiscreteLaw d) :
(∫ V, ∑ z : AuxObs d,
|empiricalFrequency (AuxObs d) hm V z -
auxTable P z| ∂auxProductLaw P m) ≤ sqrt (2 * (d : Real) / m) := by
let : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
let W := fun V : Fin m → AuxObs d => ∑ z : AuxObs d,
|empiricalFrequency (AuxObs d) hm V z -
auxTable P z|
have hW : MemLp W 2 (auxProductLaw P m) :=
(memLp_two_iff_integrable_sq (by fun_prop)).2 Integrable.of_finite
have hv := variance_nonneg W (auxProductLaw P m)
rw [variance_eq_sub hW] at hv
have hsecond := comparisonEmpiricalFrequency_l1_sq hm P
change (∫ V, W V ^ 2 ∂auxProductLaw P m) ≤ _ at hsecond
have hmean : (∫ V, W V ∂auxProductLaw P m) ^ 2 ≤ 2 * (d : Real) / m := by
calc
_ ≤ ∫ V, W V ^ 2 ∂auxProductLaw P m := by
simpa only [Pi.pow_apply] using (sub_nonneg.mp hv)
_ ≤ _ := hsecond
exact Real.le_sqrt_of_sq_le hmean
theorem comparisonEmpiricalTable_expected_l1
Under the stated inputs and conditions, The legal empirical table has expected L¹ error at most three times the iid square-root rate. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonEmpiricalTable_expected_l1
lemma comparisonEmpiricalTable_expected_l1 {m d : Nat} (eps : Real) (hm : 0 < m)
(heps : 0 ≤ eps) (heps' : eps ≤ 1 / 2) (P : DiscreteLaw d)
(hP : ModelClass d eps P) :
(∫ V, ∑ z : AuxObs d,
|comparisonEmpiricalTable eps hm V z - auxTable P z| ∂auxProductLaw P m) ≤
3 * sqrt (2 * (d : Real) / m) := by
let : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
calc
_ ≤ ∫ V, 3 * ∑ z : AuxObs d,
|empiricalFrequency (AuxObs d) hm V z -
auxTable P z| ∂auxProductLaw P m :=
integral_mono Integrable.of_finite Integrable.of_finite
(comparisonEmpiricalTable_l1_error eps hm heps heps' P hP)
_ = 3 * (∫ V, ∑ z : AuxObs d,
|empiricalFrequency (AuxObs d) hm V z -
auxTable P z| ∂auxProductLaw P m) := by rw [integral_const_mul]
_ ≤ _ := mul_le_mul_of_nonneg_left (comparisonEmpiricalFrequency_expected_l1 hm P)
(by norm_num)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Estimator 13 declarations Ordered three-pool estimator with degree calibrated to rare-label information.
Ordered three-pool estimator with degree calibrated to rare-label information.
The hybrid schedule records the stated pool sizes, Poisson intensities, polynomial degree, and bandwidth.
def hybridTuning
The public tuning implements the ordered splits and logarithmic degree based on rare-label information.
Definition (Lean source)
parameters
def
def chebE
The first Chebyshev continuation divides its zero-constant numerator by the polynomial variable.
def chebG
The second continuation divides one minus the first by the polynomial variable.
Definition (Lean source)
def labeledBlockCount
Complete records in an ordered block with a fixed mark.
def auxiliaryBlockCount
The count uses an ordered auxiliary block and the specified arm-cell mark.
def pooledArmCount
Concatenation order is complete-record projections followed by auxiliary records.
Definition (Lean source)
def hybridCellValue
A pilot count selects the polynomial or inverse-count arm contribution.
Definition (Lean source)
parameters
def
def hybridPrefixStatistic
A valid prefix triple contributes the clipped sum of treated-minus-control hybrid cell values.
Definition (Lean source)
parameters
def
given by
tun :
hybridTuning n m eps
Z :
fun a j => labeledBlockCount s.1 0 r0 j a true
J :
fun a j => pooledArmCount s tun.h0 tun.hp 0 rp j a
K :
fun a j => pooledArmCount s (tun.h0 + tun.hp) tun.hf (m / 2) rf j a
V :
fun a j
=> hybridCellValue tun.L tun.B tun.k0 tun.u tun.t (Z a j) (J a j) (K a j) (K (!a) j)
def hybridEstimator
The attaining data-only rule averages all valid prefix triples with their Poisson weights; overflow outputs zero.
Definition (Lean source)
parameters
def
given by
fun s
=> if (n : Real) * eps < exp 4096 then 0 else let tun := hybridTuning n m eps ∑ r0 ∈ range (tun.h0 + 1), ∑ rp ∈ range (tun.Mp + 1), ∑ rf ∈ range (tun.Mf + 1), exp (-tun.u - tun.tp - tun.t) * (tun.u ^ r0 / (r0.factorial : Real)) * (tun.tp ^ rp / (rp.factorial : Real)) * (tun.t ^ rf / (rf.factorial : Real)) * hybridPrefixStatistic eps r0 rp rf s
theorem hybridEstimator_measurable
The finite prefix-average rule is measurable on the original finite data.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: hybridEstimator_measurable
lemma hybridEstimator_measurable (n m d : Nat) (eps : Real) :
Measurable (hybridEstimator n m d eps) := by
fun_prop
Under the stated inputs and conditions, A finite Poisson average with zero output on omitted counts preserves [-1,1]. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: finitePoissonAverage_mem_Icc
lemma finitePoissonAverage_mem_Icc (lambda : Real) (hlambda : 0 ≤ lambda)
(N : Nat) (f : Nat → Real) (hf : ∀ k, f k ∈ Icc (-1) 1) :
(∑ k ∈ range (N + 1), exp (-lambda) *
(lambda ^ k / (k.factorial : Real)) * f k) ∈ Icc (-1) 1 := by
let rate : NNReal := ⟨lambda, hlambda⟩
let w := poissonWeight rate
have hw (k : Nat) : 0 ≤ w k :=
Causalean.Mathlib.Probability.PoissonAddOnePoincare.poissonWeight_nonneg rate k
have hsum : ∑ k ∈ range (N + 1), w k ≤ 1 := by
have hp := poissonWeight_hasSum_one rate
exact (hp.summable.sum_le_tsum _ (fun k _ => hw k)).trans_eq hp.tsum_eq
have habs : |∑ k ∈ range (N + 1), w k * f k| ≤ 1 := by
calc
_ ≤ ∑ k ∈ range (N + 1), |w k * f k| := Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ k ∈ range (N + 1), w k := Finset.sum_le_sum (fun k _ => by
rw [abs_mul, abs_of_nonneg (hw k)]
exact mul_le_of_le_one_right (hw k) (abs_le.mpr (hf k)))
_ ≤ 1 := hsum
have hwdef (k : Nat) : w k = exp (-lambda) *
(lambda ^ k / (k.factorial : Real)) := by
change exp (-lambda) * lambda ^ k / (k.factorial : Real) = _
rw [mul_div_assoc]
simp_rw [hwdef] at habs
exact abs_le.mp habs
theorem hybridEstimator_mem_Icc
Under the stated inputs and conditions, The prefix-average rule takes values between minus one and one.
Formal statement
hypotheses
conclusion
hybridEstimator n m d eps s ∈ Icc (-1) 1
Proof (Lean source)
-- @node: hybridEstimator_mem_Icc
lemma hybridEstimator_mem_Icc (n m d : Nat) (eps : Real) (s : Sample n m d) :
hybridEstimator n m d eps s ∈ Icc (-1) 1 := by
classical
by_cases hsmall : (n : Real) * eps < exp 4096
· simp [hybridEstimator, hsmall]
· rw [hybridEstimator, if_neg hsmall]
let tun := hybridTuning n m eps
have hu : 0 ≤ tun.u := by dsimp [tun, hybridTuning]; positivity
have hp : 0 ≤ tun.tp := by dsimp [tun, hybridTuning]; positivity
have ht : 0 ≤ tun.t := by dsimp [tun, hybridTuning]; positivity
have hclip (r0 rp rf : Nat) : hybridPrefixStatistic eps r0 rp rf s ∈ Icc (-1) 1 := by
constructor
· exact le_max_left _ _
· exact max_le (by norm_num) (min_le_left _ _)
have havg := finitePoissonAverage_mem_Icc tun.u hu tun.h0 _ (fun r0 =>
finitePoissonAverage_mem_Icc tun.tp hp tun.Mp _ (fun rp =>
finitePoissonAverage_mem_Icc tun.t ht tun.Mf _ (hclip r0 rp)))
convert havg using 1
simp only [tun, Finset.mul_sum]
apply Finset.sum_congr rfl
intro r0 _
apply Finset.sum_congr rfl
intro rp _
apply Finset.sum_congr rfl
intro rf _
rw [show -(hybridTuning n m eps).u - (hybridTuning n m eps).tp -
(hybridTuning n m eps).t = -(hybridTuning n m eps).u +
-(hybridTuning n m eps).tp + -(hybridTuning n m eps).t by ring,
Real.exp_add, Real.exp_add]
ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.ExperimentComparison 13 declarations Conditional averaging of auxiliary samples for the exact-marginal experiment, and the zero-rule cap on the original minimax risk.
Conditional averaging of auxiliary samples for the exact-marginal experiment, and the zero-rule cap on the original minimax risk.
The iid auxiliary-array weight is a polynomial in the supplied table.
theorem comparisonArrayWeight_true
Under the stated inputs and conditions, At the true table the polynomial weight is the probability of the auxiliary array. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonArrayWeight_true
lemma comparisonArrayWeight_true {d : Nat} (P : DiscreteLaw d) (m : Nat)
(V : Fin m → AuxObs d) :
comparisonArrayWeight (auxTable P) V = (auxProductLaw P m).real {V} := by
simp [comparisonArrayWeight, auxTable, auxProductLaw, measureReal_def,
Measure.pi_singleton, ENNReal.toReal_prod, PMF.toMeasure_apply_singleton]
theorem comparisonArrayWeight_sum
Under the stated inputs and conditions, Auxiliary-array weights at a population table sum to one, including the empty array. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonArrayWeight_sum
lemma comparisonArrayWeight_sum {d : Nat} (P : DiscreteLaw d) (m : Nat) :
∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V = 1 := by
letI : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
simp_rw [comparisonArrayWeight_true]
simpa using (integral_fintype (μ := auxProductLaw P m)
(integrable_const (1 : Real))).symm
Averaging over auxiliary arrays retains the independent seed; clipping makes the rule legal on every ambient table.
theorem comparisonAverageRule_measurable
The averaged rule is Borel on the entire ambient table space. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: comparisonAverageRule_measurable
lemma comparisonAverageRule_measurable {n m d : Nat} (T : Rule n m d) :
Measurable (comparisonAverageRule T) := by
unfold comparisonAverageRule comparisonArrayWeight
have hT := T.2.1
fun_prop
theorem comparisonAverageRule_mem_Icc
Under the stated inputs and conditions, Clipping bounds the average even on tables that are not probability vectors. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparisonAverageRule_mem_Icc
lemma comparisonAverageRule_mem_Icc {n m d : Nat} (T : Rule n m d)
(z : (Fin n → Obs d) × AuxTable d × Real) :
comparisonAverageRule T z ∈ Icc (-1) 1 := by
exact ⟨le_max_left _ _, max_le (by norm_num) (min_le_left _ _)⟩
theorem comparisonAverageRule_true
Under the stated inputs and conditions, On a true marginal table the convex average already lies in the action interval. This gives the stated result.
Formal statement
hypotheses
conclusion
comparisonAverageRule T (L, auxTable P, u)
Proof (Lean source)
-- @node: comparisonAverageRule_true
lemma comparisonAverageRule_true {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d)
(L : Fin n → Obs d) (u : Real) :
comparisonAverageRule T (L, auxTable P, u) =
∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V * T.1 ((L, V), u) := by
have hw (V : Fin m → AuxObs d) : 0 ≤ comparisonArrayWeight (auxTable P) V := by
rw [comparisonArrayWeight_true]; exact ENNReal.toReal_nonneg
have hlo : -1 ≤ ∑ V : Fin m → AuxObs d,
comparisonArrayWeight (auxTable P) V * T.1 ((L, V), u) := by
calc
-1 = ∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V * (-1) := by
rw [← Finset.sum_mul, comparisonArrayWeight_sum, one_mul]
_ ≤ _ := Finset.sum_le_sum fun V _ => mul_le_mul_of_nonneg_left (T.2.2 _).1 (hw V)
have hhi : (∑ V : Fin m → AuxObs d,
comparisonArrayWeight (auxTable P) V * T.1 ((L, V), u)) ≤ 1 := by
calc
_ ≤ ∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V * 1 :=
Finset.sum_le_sum fun V _ => mul_le_mul_of_nonneg_left (T.2.2 _).2 (hw V)
_ = 1 := by rw [← Finset.sum_mul, comparisonArrayWeight_sum, one_mul]
unfold comparisonAverageRule
rw [min_eq_right hhi, max_eq_right hlo]
theorem comparisonAverageRule_loss_le
Under the stated inputs and conditions, Jensen's inequality for the finite conditional average of a randomized rule. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ ∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V * (T.1 ((L, V), u) - ateFunctional P) ^ 2
Proof (Lean source)
-- @node: comparisonAverageRule_loss_le
lemma comparisonAverageRule_loss_le {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d)
(L : Fin n → Obs d) (u : Real) :
(comparisonAverageRule T (L, auxTable P, u) - ateFunctional P) ^ 2 ≤
∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V *
(T.1 ((L, V), u) - ateFunctional P) ^ 2 := by
have hconvex : ConvexOn Real univ (fun x : Real => (x - ateFunctional P) ^ 2) := by
refine ⟨convex_univ, ?_⟩
intro x _ y _ a b ha hb hab
dsimp
have hid : a * (x - ateFunctional P) ^ 2 + b * (y - ateFunctional P) ^ 2 -
(a * x + b * y - ateFunctional P) ^ 2 = a * b * (x - y) ^ 2 := by
rw [show b = 1 - a by linarith]
ring
have hpos := mul_nonneg (mul_nonneg ha hb) (sq_nonneg (x - y))
linarith
rw [comparisonAverageRule_true]
simpa only [smul_eq_mul] using hconvex.map_sum_le
(t := univ) (w := fun V : Fin m → AuxObs d => comparisonArrayWeight (auxTable P) V)
(p := fun V => T.1 ((L, V), u))
(fun V _ => by rw [comparisonArrayWeight_true]; exact ENNReal.toReal_nonneg)
(comparisonArrayWeight_sum P m) (fun _ _ => Set.mem_univ _)
theorem comparisonAverageRule_risk_le
Under the stated inputs and conditions, Auxiliary conditional averaging cannot increase squared-error risk at any population. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: comparisonAverageRule_risk_le
lemma comparisonAverageRule_risk_le {n m d : Nat} (T : Rule n m d) (P : DiscreteLaw d) :
knownRuleRisk (comparisonAverageRule T) P ≤ ruleRisk T.1 P := by
letI : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
letI : IsProbabilityMeasure (labeledProductLaw P n) := by
unfold labeledProductLaw; infer_instance
letI : IsProbabilityMeasure (auxProductLaw P m) := by
unfold auxProductLaw; infer_instance
letI : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw; infer_instance
letI : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have hT := T.2.1
have hsection (L : Fin n → Obs d) (V : Fin m → AuxObs d) :
Integrable (fun u => (T.1 ((L, V), u) - ateFunctional P) ^ 2) seedLaw := by
apply mse_integrable_of_estimator_bound (M := 1)
seedLaw (fun u => T.1 ((L, V), u)) (by fun_prop) (by norm_num)
exact fun u => T.2.2 _
have hfull : Integrable (fun z => (T.1 z - ateFunctional P) ^ 2)
((annotationLaw P n m).prod seedLaw) :=
mse_integrable_of_estimator_bound (M := 1) _ T.1 hT (by norm_num) T.2.2
have havg : Integrable (fun z : (Fin n → Obs d) × Real =>
(comparisonAverageRule T (z.1, auxTable P, z.2) - ateFunctional P) ^ 2)
((labeledProductLaw P n).prod seedLaw) := by
have hm := comparisonAverageRule_measurable T
exact mse_integrable_of_estimator_bound (M := 1)
((labeledProductLaw P n).prod seedLaw)
(fun z => comparisonAverageRule T (z.1, auxTable P, z.2))
(hm.comp (measurable_fst.prodMk (measurable_const.prodMk measurable_snd)))
(by norm_num) (fun z => comparisonAverageRule_mem_Icc T _)
unfold knownRuleRisk ruleRisk
rw [integral_prod _ havg, integral_prod _ hfull]
change (∫ L, ∫ u, (comparisonAverageRule T (L, auxTable P, u) - ateFunctional P) ^ 2
∂seedLaw ∂labeledProductLaw P n) ≤
∫ LV, ∫ u, (T.1 (LV, u) - ateFunctional P) ^ 2 ∂seedLaw
∂((labeledProductLaw P n).prod (auxProductLaw P m))
rw [integral_prod _ hfull.integral_prod_left]
apply integral_mono havg.integral_prod_left
hfull.integral_prod_left.integral_prod_left
intro L
calc
(∫ u, (comparisonAverageRule T (L, auxTable P, u) - ateFunctional P) ^ 2 ∂seedLaw) ≤
∫ u, ∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V *
(T.1 ((L, V), u) - ateFunctional P) ^ 2 ∂seedLaw := by
apply integral_mono
· apply mse_integrable_of_estimator_bound (M := 1) _
(fun u => comparisonAverageRule T (L, auxTable P, u))
((comparisonAverageRule_measurable T).comp
(measurable_const.prodMk (measurable_const.prodMk measurable_id))) (by norm_num)
exact fun u => comparisonAverageRule_mem_Icc T _
· exact integrable_finsetSum _ fun V _ => (hsection L V).const_mul _
· exact fun u => comparisonAverageRule_loss_le T P L u
_ = ∑ V : Fin m → AuxObs d, comparisonArrayWeight (auxTable P) V *
(∫ u, (T.1 ((L, V), u) - ateFunctional P) ^ 2 ∂seedLaw) := by
rw [integral_finsetSum _ (fun V _ => (hsection L V).const_mul _)]
simp only [integral_const_mul]
_ = ∫ V, ∫ u, (T.1 ((L, V), u) - ateFunctional P) ^ 2
∂seedLaw ∂auxProductLaw P m := by
rw [integral_fintype (by
simpa using (IntegrableOn.of_finite (μ := auxProductLaw P m)
(f := fun V => ∫ u, (T.1 ((L, V), u) - ateFunctional P) ^ 2 ∂seedLaw)
(Set.finite_univ : (Set.univ : Set (Fin m → AuxObs d)).Finite)))]
simp only [comparisonArrayWeight_true, smul_eq_mul]
theorem comparison_known_le_minimax
Under the stated inputs and conditions, Conditional averaging proves that exact marginal information can only lower minimax risk. This gives the stated result.
Formal statement
hypotheses
conclusion
knownMarginalRisk n d eps ≤ minimaxRisk n m d eps
Proof (Lean source)
-- @node: comparison_known_le_minimax
lemma comparison_known_le_minimax (n m d : Nat) (eps : Real)
(hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
knownMarginalRisk n d eps ≤ minimaxRisk n m d eps := by
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps'⟩
letI : Nonempty (ClassLaw d eps) := ⟨P0⟩
let T0 : Rule n m d := ⟨fun _ => 0, measurable_const, fun _ => by norm_num⟩
letI : Nonempty (Rule n m d) := ⟨T0⟩
change minimaxValueReal
(fun (T : KnownRule n d) (P : ClassLaw d eps) => knownRuleRisk T.1 P.1) ≤
minimaxValueReal
(fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
apply minimaxValue_le_minimaxValue
(bddBelow_range_worstCaseRisk
(fun T P => integral_nonneg (fun z => sq_nonneg _)))
intro T
let K : KnownRule n d := ⟨comparisonAverageRule T,
comparisonAverageRule_measurable T, comparisonAverageRule_mem_Icc T⟩
refine ⟨K, ?_⟩
apply worstCaseRisk_le
intro P
apply (comparisonAverageRule_risk_le T P.1).trans
refine le_worstCaseRisk
(risk := fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
(e := T) ?_ P
refine ⟨4, ?_⟩
rintro _ ⟨Q, rfl⟩
letI : IsProbabilityMeasure (obsLaw Q.1) := by unfold obsLaw; infer_instance
letI : IsProbabilityMeasure (annotationLaw Q.1 n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw; infer_instance
letI : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
exact labelFloor_bounded_mse _ T.1 T.2.2 _ (ateFunctional_mem_Icc Q.1)
theorem comparison_known_nonneg
Under the stated inputs and conditions, Squared loss makes the exact-marginal minimax value nonnegative. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: comparison_known_nonneg
lemma comparison_known_nonneg (n d : Nat) (eps : Real) :
0 ≤ knownMarginalRisk n d eps := by
exact minimaxValue_nonneg
(fun (T : KnownRule n d) (P : ClassLaw d eps) => integral_nonneg (fun z => sq_nonneg _))
theorem comparison_minimax_le_one
Under the stated inputs and conditions, The original zero rule gives the universal minimax cap of one. This gives the stated result.
Formal statement
hypotheses
conclusion
minimaxRisk n m d eps ≤ 1
Proof (Lean source)
-- @node: comparison_minimax_le_one
lemma comparison_minimax_le_one (n m d : Nat) (eps : Real)
(hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
minimaxRisk n m d eps ≤ 1 := by
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps'⟩
letI : Nonempty (ClassLaw d eps) := ⟨P0⟩
let T0 : Rule n m d := ⟨fun _ => 0, measurable_const, fun _ => by norm_num⟩
apply (minimaxValue_le_worstCaseRisk_of_nonneg
(risk := fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
(fun T P => integral_nonneg (fun z => sq_nonneg _)) T0).trans
apply ciSup_le
intro P
letI : IsProbabilityMeasure (obsLaw P.1) := by unfold obsLaw; infer_instance
letI : IsProbabilityMeasure (annotationLaw P.1 n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw; infer_instance
letI : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have ht := ateFunctional_mem_Icc P.1
have hzero : ruleRisk T0.1 P.1 = (ateFunctional P.1) ^ 2 := by
simp [ruleRisk, T0]
change ruleRisk T0.1 P.1 ≤ 1
rw [hzero]
nlinarith [ht.1, ht.2]
theorem comparison_difference_bounds
Under the stated inputs and conditions, Both elementary endpoints of the experiment comparison hold for every auxiliary budget. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps
conclusion 2
minimaxRisk n m d eps - knownMarginalRisk n d eps ≤ 1
Proof (Lean source)
-- @node: comparison_difference_bounds
lemma comparison_difference_bounds (n m d : Nat) (eps : Real)
(hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps ∧
minimaxRisk n m d eps - knownMarginalRisk n d eps ≤ 1 := by
constructor
· exact sub_nonneg.mpr (comparison_known_le_minimax n m d eps hd heps heps')
· have hcap := comparison_minimax_le_one n m d eps hd heps heps'
have hnonneg := comparison_known_nonneg n d eps
linarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.FactorialCertificate 8 declarations Chebyshev continuation and factorial-lift mean and second-moment certificate.
Chebyshev continuation and factorial-lift mean and second-moment certificate.
Under the stated inputs and conditions, Products of falling factorials have finite Poisson expectation. This gives the stated result.
Formal statement
hypotheses
conclusion
Integrable (fun K : Nat => (K.descFactorial h : Real) * (K.descFactorial v : Real)) (poissonMeasure rate)
Proof (Lean source)
-- @node: factorial_product_integrable
lemma factorial_product_integrable (rate : NNReal) (h v : Nat) :
Integrable (fun K : Nat => (K.descFactorial h : Real) *
(K.descFactorial v : Real)) (poissonMeasure rate) := by
have hi := integrable_finsetSum (range (min h v + 1))
(fun r _ => (poisson_descFactorial_integrable
rate (h + v - r)).const_mul
((h.choose r : Real) * (v.choose r : Real) * (r.factorial : Real)))
exact hi.congr (Filter.Eventually.of_forall (fun K =>
(descFactorial_mul K h v).symm))
Under the stated inputs and conditions, Normalize the exact overlap formula by the factorial-pool intensity. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: normalized_factorial_mixed_moment
lemma normalized_factorial_mixed_moment (D z : Real) (h v : Nat)
(hD : 0 < D) (hz : 0 ≤ z) :
(∫ K : Nat, ((K.descFactorial h : Real) / D ^ h) *
((K.descFactorial v : Real) / D ^ v)
∂poissonMeasure (toNNReal (D * z))) =
∑ r ∈ range (min h v + 1),
(h.choose r : Real) * ((v.choose r : Real) * (r.factorial : Real) / D ^ r) *
z ^ (h + v - r) := by
simp_rw [div_mul_div_comm]
rw [integral_div, poisson_descFactorial_mixed,
Real.coe_toNNReal _ (mul_nonneg hD.le hz), Finset.sum_div]
apply Finset.sum_congr rfl
intro r hr
have hrhv : r ≤ h + v := by
have := Finset.mem_range.mp hr
have := Nat.min_le_left h v
omega
rw [mul_pow, show D ^ h * D ^ v = D ^ (h + v) by rw [pow_add],
show D ^ (h + v) = D ^ r * D ^ (h + v - r) by
rw [← pow_add, Nat.add_sub_of_le hrhv]]
field_simp
Under the stated inputs and conditions, Each overlap factor is at most one when the order is below the intensity. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: factorial_overlap_factor_le_one
lemma factorial_overlap_factor_le_one (D : Real) (v r : Nat)
(hD : 0 < D) (hv : (v : Real) ≤ D) :
(v.choose r : Real) * (r.factorial : Real) / D ^ r ≤ 1 := by
have hc : (v.choose r : Real) * (r.factorial : Real) =
(v.descFactorial r : Real) := by
exact_mod_cast (Nat.mul_comm (v.choose r) r.factorial).trans
(Nat.descFactorial_eq_factorial_mul_choose v r).symm
rw [hc, div_le_one (pow_pos hD r)]
have hb : (v.descFactorial r : Real) ≤ (v : Real) ^ r := by
exact_mod_cast Nat.descFactorial_le_pow v r
exact hb.trans (pow_le_pow_left₀ (Nat.cast_nonneg v) hv r)
Under the stated inputs and conditions, The binomial overlap sum gives the uniform mixed-moment envelope. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: normalized_factorial_mixed_bound
lemma normalized_factorial_mixed_bound (L h v : Nat) (D z : Real)
(hh : h ≤ L) (hv : v ≤ L) (hD : 0 < D) (hLD : (L : Real) ≤ D)
(hz : 0 ≤ z) :
(∫ K : Nat, ((K.descFactorial h : Real) / D ^ h) *
((K.descFactorial v : Real) / D ^ v)
∂poissonMeasure (toNNReal (D * z))) ≤
2 ^ L * (1 + z) ^ (2 * L) := by
rw [normalized_factorial_mixed_moment D z h v hD hz]
have hbase : 1 ≤ 1 + z := by linarith
have hpower (r : Nat) : z ^ (h + v - r) ≤ (1 + z) ^ (2 * L) :=
(pow_le_pow_left₀ hz (by linarith) _).trans
(pow_le_pow_right₀ hbase (by omega))
calc
_ ≤ ∑ r ∈ range (min h v + 1),
(h.choose r : Real) * (1 + z) ^ (2 * L) := by
apply Finset.sum_le_sum
intro r _
have hf := factorial_overlap_factor_le_one D v r hD
((by exact_mod_cast hv : (v : Real) ≤ (L : Real)).trans hLD)
have hn : 0 ≤ (v.choose r : Real) * (r.factorial : Real) / D ^ r := by positivity
apply mul_le_mul
· simpa using mul_le_mul_of_nonneg_left hf (Nat.cast_nonneg (h.choose r))
· exact hpower r
· positivity
· positivity
_ ≤ ∑ r ∈ range (h + 1),
(h.choose r : Real) * (1 + z) ^ (2 * L) := by
apply Finset.sum_le_sum_of_subset_of_nonneg
· exact Finset.range_mono (by omega : min h v + 1 ≤ h + 1)
· intros; positivity
_ = 2 ^ h * (1 + z) ^ (2 * L) := by
rw [← Finset.sum_mul]
congr 1
exact_mod_cast Nat.sum_range_choose h
_ ≤ _ := mul_le_mul_of_nonneg_right
(pow_le_pow_right₀ (by norm_num : (1 : Real) ≤ 2) hh) (by positivity)
theorem factorial_polynomial_second_moment_bound Lemma factorial_polynomial_second_moment_bound in the paper ↗
Under the stated inputs and conditions, Finite assembly of the mixed-moment envelope against absolute coefficients. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: factorial_polynomial_second_moment_bound
lemma factorial_polynomial_second_moment_bound (L J : Nat) (c : Nat → Real)
(D z : Real) (hJ : J ≤ L + 1) (hD : 0 < D) (hLD : (L : Real) ≤ D)
(hz : 0 ≤ z) :
(∫ K : Nat, (∑ h ∈ range J,
c h * (K.descFactorial h : Real) / D ^ h) ^ 2
∂poissonMeasure (toNNReal (D * z))) ≤
(∑ h ∈ range J, |c h|) ^ 2 *
(2 ^ L * (1 + z) ^ (2 * L)) := by
let mu := poissonMeasure (toNNReal (D * z))
let A := fun (h K : Nat) => (K.descFactorial h : Real) / D ^ h
let M := fun h v => ∫ K, A h K * A v K ∂mu
have hi (h v : Nat) : Integrable (fun K => (c h * A h K) * (c v * A v K)) mu := by
have hh := ((factorial_product_integrable (toNNReal (D * z)) h v).div_const
(D ^ h * D ^ v)).const_mul (c h * c v)
convert hh using 1
funext K
dsimp [A]
ring
have heq : (∫ K : Nat, (∑ h ∈ range J,
c h * (K.descFactorial h : Real) / D ^ h) ^ 2 ∂mu) =
∑ h ∈ range J, ∑ v ∈ range J, c h * c v * M h v := by
have halg (K : Nat) : (∑ h ∈ range J,
c h * (K.descFactorial h : Real) / D ^ h) ^ 2 =
∑ h ∈ range J, ∑ v ∈ range J,
(c h * A h K) * (c v * A v K) := by
simp only [A, pow_two, Finset.sum_mul, Finset.mul_sum, mul_div_assoc]
rw [Finset.sum_comm]
simp_rw [halg]
rw [integral_finsetSum]
· apply Finset.sum_congr rfl
intro h _
rw [integral_finsetSum (range J) (fun v _ => hi h v)]
apply Finset.sum_congr rfl
intro v _
have ht (K : Nat) : (c h * A h K) * (c v * A v K) =
(c h * c v) * (A h K * A v K) := by ring
simp_rw [ht]
exact integral_const_mul _ _
· intro h _
exact integrable_finsetSum _ (fun v _ => hi h v)
rw [heq]
calc
_ ≤ ∑ h ∈ range J, ∑ v ∈ range J,
|c h| * |c v| * (2 ^ L * (1 + z) ^ (2 * L)) := by
apply Finset.sum_le_sum
intro h hh
apply Finset.sum_le_sum
intro v hv
have hm0 : 0 ≤ M h v := by
apply integral_nonneg
intro K
dsimp [A]
positivity
have hm := normalized_factorial_mixed_bound L h v D z
(by have := Finset.mem_range.mp hh; omega)
(by have := Finset.mem_range.mp hv; omega) hD hLD hz
have hc : c h * c v ≤ |c h| * |c v| := by
rw [← abs_mul]
exact le_abs_self _
exact mul_le_mul hc hm hm0 (by positivity)
_ = _ := by
simp only [pow_two, Finset.sum_mul, Finset.mul_sum]
apply Finset.sum_congr rfl
intro h _
apply Finset.sum_congr rfl
intro v _
ring
Under the stated inputs and conditions, The coefficient certificate and the mixed-moment envelope give the stated constant. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: chebG_factorial_second_moment
lemma chebG_factorial_second_moment (L : Nat) (hL : 2 ≤ L) (t Bb z : Real)
(ht : 0 < t) (hBb : 0 < Bb) (hz : 0 ≤ z) (hD : (L : Real) ≤ t * Bb) :
(∫ K : Nat, (∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * Bb) ^ h) ^ 2
∂poissonMeasure (toNNReal (t * z * Bb))) ≤
392 ^ L * (1 + z) ^ (2 * L) := by
have hrate : t * z * Bb = (t * Bb) * z := by ring
rw [hrate]
have hc := chebG_coeff_sum_le L hL
have hs0 : 0 ≤ ∑ h ∈ range (L - 1), |(chebG L).coeff h| :=
Finset.sum_nonneg (fun _ _ => abs_nonneg _)
calc
_ ≤ (∑ h ∈ range (L - 1), |(chebG L).coeff h|) ^ 2 *
(2 ^ L * (1 + z) ^ (2 * L)) :=
factorial_polynomial_second_moment_bound L (L - 1) _ (t * Bb) z
(by omega) (mul_pos ht hBb) hD hz
_ ≤ (14 ^ L) ^ 2 * (2 ^ L * (1 + z) ^ (2 * L)) :=
mul_le_mul_of_nonneg_right (pow_le_pow_left₀ hs0 hc 2) (by positivity)
_ = _ := by
rw [← pow_mul, Nat.mul_comm L 2, pow_mul, ← mul_assoc, ← mul_pow]
norm_num
Under the stated inputs and conditions, The factorial lift of the second continuation is unbiased under its Poisson law. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: chebG_factorial_unbiased
lemma chebG_factorial_unbiased (L : Nat) (hL : 2 ≤ L) (t Bb z : Real)
(ht : 0 < t) (hBb : 0 < Bb) (hz : 0 ≤ z) :
(∫ K, (∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * Bb) ^ h)
∂poissonMeasure (toNNReal (t * z * Bb))) = (chebG L).eval z := by
have hrate : 0 ≤ t * z * Bb := by positivity
have hD : t * Bb ≠ 0 := ne_of_gt (mul_pos ht hBb)
have hterm (h : Nat) :
(∫ K, ((chebG L).coeff h / (t * Bb) ^ h) * (K.descFactorial h : Real)
∂poissonMeasure (toNNReal (t * z * Bb))) =
(chebG L).coeff h * z ^ h := by
rw [integral_const_mul, poisson_descFactorial_moment,
Real.coe_toNNReal _ hrate]
have heq : t * z * Bb = (t * Bb) * z := by ring
rw [heq, mul_pow]
field_simp
simp only [mul_pow, mul_assoc]
calc
_ = ∫ K, ∑ h ∈ range (L - 1),
((chebG L).coeff h / (t * Bb) ^ h) * (K.descFactorial h : Real)
∂poissonMeasure (toNNReal (t * z * Bb)) := by
congr 1
funext K
apply Finset.sum_congr rfl
intro h _
ring
_ = ∑ h ∈ range (L - 1), (chebG L).coeff h * z ^ h := by
rw [integral_finsetSum]
· exact Finset.sum_congr rfl (fun h _ => hterm h)
· intro h _
exact (poisson_descFactorial_integrable
(toNNReal (t * z * Bb)) h).const_mul _
_ = (chebG L).eval z := by
exact (Polynomial.eval_eq_sum_range' (lt_of_le_of_lt (chebG_natDegree_le L)
(by omega : L - 2 < L - 1)) z).symm
Under the stated hypotheses, The Chebyshev continuations obey the stated approximation, coefficient, unbiasedness, and factorial second-moment bounds. This gives the stated conclusion.
Formal statement
Proof (Lean source)
lemma chebyshev_factorial_certificate (L : Nat) (hL : 2 ≤ L) :
(∀ z : Real, 0 < z → z ≤ 1 → 0 ≤ (chebE L).eval z ∧
(chebE L).eval z ≤ min 1 ((L : Real) ^ 2 * z)⁻¹) ∧
(chebE L).eval 0 = 1 ∧
(∀ z : Real, z ≠ 0 →
(chebE L).eval z = (1 - (T Real L).eval (1 - 2 * z)) /
(2 * (L : Real) ^ 2 * z) ∧ (chebG L).eval z = (1 - (chebE L).eval z) / z) ∧
(chebG L).natDegree ≤ L - 2 ∧
(∑ h ∈ range (L - 1), |(chebG L).coeff h|) ≤ 14 ^ L ∧
∀ (t Bb z : Real), 0 < t → 0 < Bb → 0 ≤ z → (L : Real) ≤ t * Bb →
let Ghat : Nat → Real := fun K => ∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * Bb) ^ h
(∫ K, Ghat K ∂poissonMeasure (toNNReal (t * z * Bb))) = (chebG L).eval z ∧
(∫ K, Ghat K ^ 2 ∂poissonMeasure (toNNReal (t * z * Bb))) ≤
392 ^ L * (1 + z) ^ (2 * L) := by
have hLp : 0 < L := by omega
refine ⟨fun z hz hz' => chebE_approximation_bounds L hLp z hz hz',
chebE_eval_zero L hLp, cheb_continuation_eval L hLp, chebG_natDegree_le L, ?_, ?_⟩
· exact chebG_coeff_sum_le L hL
· intro t Bb z ht hBb hz hD
exact ⟨chebG_factorial_unbiased L hL t Bb z ht hBb hz,
chebG_factorial_second_moment L hL t Bb z ht hBb hz hD⟩
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.FiniteLaw 3 declarations Finite normalization with a total fallback outside the public parameter domain.
Finite normalization with a total fallback outside the public parameter domain.
def normalizedPMF
Normalize finite nonnegative masses; a supplied PMF handles illegal total inputs.
def bernoulliMass
For a success probability and a binary outcome, the Bernoulli atom mass is the success probability for a success and its complement for a failure.
Under the stated inputs and conditions, Normalizing an already normalized nonnegative table preserves every atom. This gives the stated result.
Formal statement
hypotheses
alpha :
Type
Fintype alpha
mass :
alpha → Real
fallback :
PMF alpha
hnonneg :
∀ z, 0 ≤ mass z
hsum :
∑ z, mass z = 1
z :
alpha
conclusion
(normalizedPMF mass fallback z).toReal = mass z
Proof (Lean source)
-- @node: normalizedPMF_toReal
lemma normalizedPMF_toReal {alpha : Type} [Fintype alpha]
(mass : alpha → Real) (fallback : PMF alpha)
(hnonneg : ∀ z, 0 ≤ mass z) (hsum : ∑ z, mass z = 1) (z : alpha) :
(normalizedPMF mass fallback z).toReal = mass z := by
have htotal : (∑' z, ofReal (mass z)) = 1 := by
rw [tsum_fintype, ← ENNReal.ofReal_sum_of_nonneg (fun z _ => hnonneg z),
hsum, ENNReal.ofReal_one]
simp only [normalizedPMF, htotal, ne_eq, one_ne_zero, ENNReal.one_ne_top,
not_false_eq_true, and_self, dite_true, PMF.normalize_apply, inv_one, mul_one]
exact ENNReal.toReal_ofReal (hnonneg z)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.FinitePrefixTransfer 3 declarations Run-facing aliases for the generic finite-family Poisson-prefix transfer theorems.
Run-facing aliases for the generic finite-family Poisson-prefix transfer theorems.
Under the stated inputs and conditions, zero overflow output costs at most the overflow probability for a target in [-1,1]. This gives the stated result.
Formal statement
hypotheses
I :
∀ i, MeasurableSpace (X i)
P :
∀ i, Measure (X i)
∀ i, IsProbabilityMeasure (P i)
lambda :
I → NNReal
h :
I → Nat
T :
PrefixFamily X → Real
hT :
hTmem :
∀ s, T s ∈ Icc (-1) 1
theta :
htheta :
theta ∈ Icc (-1) 1
conclusion
sqRisk (fixedPoolsLaw P h) (prefixRaoBlackwellStatistic lambda T 0) theta
≤ sqRisk (independentPoissonPrefixLaw P lambda) T theta
Proof (Lean source)
lemma zero_overflow_prefix_risk {I : Type*} [Fintype I]
{X : I → Type*} [∀ i, MeasurableSpace (X i)]
(P : ∀ i, Measure (X i)) [∀ i, IsProbabilityMeasure (P i)]
(lambda : I → NNReal) (h : I → Nat) {T : PrefixFamily X → Real}
(hT : Measurable T) (hTmem : ∀ s, T s ∈ Icc (-1) 1)
{theta : Real} (htheta : theta ∈ Icc (-1) 1) :
sqRisk (fixedPoolsLaw P h) (prefixRaoBlackwellStatistic lambda T 0) theta ≤
sqRisk (independentPoissonPrefixLaw P lambda) T theta +
∑ i, (poissonMeasure (lambda i)).real (Ioi (h i)) :=
zero_overflow_prefix_risk
P lambda h hT hTmem htheta
Under the stated inputs and conditions, a Poisson prefix with one eighth of the pool's mean overflows with probability at most exp(-h). This gives the stated result.
Formal statement
Proof (Lean source)
lemma eighth_pool_poisson_overflow (h : Nat) :
(poissonMeasure ((h : NNReal) / 8)).real (Ioi h) ≤ exp (-(h : Real)) :=
eighth_pool_poisson_overflow h
Under the stated hypotheses, For finitely many independent iid pools on arbitrary measurable observation spaces (including infinite Borel alphabets), a measurable statistic with values in [-1,1] and a target in [-1,1], finite independent prefix averaging with zero overflow output transfers risk with the stated exponential tail penalty and yields a total deterministic measurable rule. For Borel observation spaces the resulting rule is Borel. No alphabet finiteness is required. This gives the stated conclusion.
Formal statement
hypotheses
I :
∀ i, MeasurableSpace (X i)
P :
∀ i, Measure (X i)
∀ i, IsProbabilityMeasure (P i)
h :
I → Nat
hh :
∀ i, 1 ≤ h i
T :
PrefixFamily X → Real
hT :
hTmem :
∀ s, T s ∈ Icc (-1) 1
theta Ab :
htheta :
theta ∈ Icc (-1) 1
conclusion
conclusion 1
conclusion 2
s :
conclusion 3
sqRisk (fixedPoolsLaw P h) (prefixRaoBlackwellStatistic (N := h) (fun i => (h i : NNReal) / 8) T 0) theta
Proof (Lean source)
lemma finite_prefix_transfer {I : Type*} [Fintype I]
{X : I → Type*} [∀ i, MeasurableSpace (X i)]
(P : ∀ i, Measure (X i)) [∀ i, IsProbabilityMeasure (P i)]
(h : I → Nat) (hh : ∀ i, 1 ≤ h i) {T : PrefixFamily X → Real}
(hT : Measurable T) (hTmem : ∀ s, T s ∈ Icc (-1) 1)
{theta Ab : Real} (htheta : theta ∈ Icc (-1) 1)
(hA : sqRisk (independentPoissonPrefixLaw P (fun i => (h i : NNReal) / 8)) T theta ≤ Ab) :
Measurable (prefixRaoBlackwellStatistic (N := h) (fun i => (h i : NNReal) / 8) T 0) ∧
(∀ s, prefixRaoBlackwellStatistic (N := h) (fun i => (h i : NNReal) / 8) T 0 s ∈ Icc
(-1) 1) ∧
sqRisk (fixedPoolsLaw P h)
(prefixRaoBlackwellStatistic (N := h) (fun i => (h i : NNReal) / 8) T 0) theta ≤
Ab + ∑ i, exp (-(h i : Real)) :=
finite_prefix_transfer
P h hT hTmem htheta hA
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.AggregateBias 10 declarations Aggregate pilot bias and heavy-cell separation bounds for equations (10)--(13) of the hybrid arm risk proof.
Aggregate pilot bias and heavy-cell separation bounds for equations (10)--(13) of the hybrid arm risk proof.
Under the stated inputs and conditions, The incorrectly selected inverse branch has total bias at most S to the negative twentieth power, using only the total cell mass bound. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_false_heavy_bias_sum
lemma hybrid_false_heavy_bias_sum (S tp t : Real)
(hS : exp 4096 ≤ S) (htp : 0 < tp) (ht : 0 < t)
(hratio : 1 / 3 ≤ t / tp) {alpha : Type} (J : Finset alpha)
(s v mu : alpha → Real)
(hcell : ∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1)
(hmass : (J.sum (fun j => s j + v j)) ≤ 1) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
(∑ j ∈ J, v j * mu j *
((1 - (poissonMeasure (toNNReal (tp * s j))).real (Iic k0)) *
exp (-t * s j))) ≤ (S ^ 20)⁻¹ := by
dsimp only
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
calc
_ ≤ ∑ j ∈ J, (s j + v j) * (S ^ 20)⁻¹ := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hs, hv, hmu⟩ := hcell j hj
have hw : 0 ≤ v j * mu j := mul_nonneg hv hmu.1
have hwp : v j * mu j ≤ s j + v j :=
(mul_le_of_le_one_right hv hmu.2).trans (by linarith)
have hb := hybrid_false_heavy_bias_bound S tp t (s j) hS htp ht hs hratio
exact (mul_le_mul_of_nonneg_left hb hw).trans
(mul_le_mul_of_nonneg_right hwp (by positivity))
_ = (J.sum (fun j => s j + v j)) * (S ^ 20)⁻¹ := by rw [Finset.sum_mul]
_ ≤ (S ^ 20)⁻¹ := by
simpa using mul_le_mul_of_nonneg_right hmass (show 0 ≤ (S ^ 20)⁻¹ by positivity)
Under the stated inputs and conditions, A light cell's actual pilot mixture retains the polynomial approximation bias and an inverse twentieth power remainder weighted by cell mass. This gives the stated result.
Formal statement
hypotheses
S u tp t s v mu eps :
hS :
exp 4096 ≤ S
hu :
0 < u
htp :
0 < tp
ht :
0 < t
hs :
0 < s
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
heps :
0 < eps
hov :
eps * (s + v) ≤ s
hratio :
1 / 3 ≤ t / tp
conclusion
|(∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t (s * mu) s v))
- (s + v) * mu|
≤ B / (eps * (L : Real) ^ 2) + (s + v) * (S ^ 20)⁻¹
Proof (Lean source)
-- @node: hybrid_light_selected_bias
lemma hybrid_light_selected_bias (S u tp t s v mu eps : Real)
(hS : exp 4096 ≤ S) (hu : 0 < u) (htp : 0 < tp) (ht : 0 < t)
(hs : 0 < s) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1)
(heps : 0 < eps) (hov : eps * (s + v) ≤ s) (hratio : 1 / 3 ≤ t / tp) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
s ≤ B →
|(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s))).prod
(cellPoissonLaw u t (s * mu) s v)) - (s + v) * mu| ≤
B / (eps * (L : Real) ^ 2) + (s + v) * (S ^ 20)⁻¹ := by
dsimp only
intro hsB
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic k0)
have hL : 2 ≤ L := by have := (hybrid_degree_calibration S hS).1; omega
have hB : 0 < B := by
dsimp [B]
have hLp : 0 < (L : Real) := by exact_mod_cast (show 0 < L by omega)
positivity
have hpi : pi ∈ Icc 0 1 := ⟨measureReal_nonneg, measureReal_le_one⟩
have hE := (chebyshev_factorial_certificate L hL).1
(s / B) (div_pos hs hB) ((div_le_one hB).2 hsB)
have hb := hybrid_light_polynomial_bias L B u t s v mu eps
hL hB hu ht hs hsB hv hmu heps hov
rw [(hybrid_branch_means L B u t s v mu hL hB hu ht hs hv hmu).1] at hb
rw [sub_sub_cancel_left, abs_neg,
abs_of_nonneg (mul_nonneg (mul_nonneg hv hmu.1) hE.1)] at hb
have htail := hybrid_false_heavy_bias_bound S tp t s hS htp ht hs.le hratio
have hw : 0 ≤ v * mu := mul_nonneg hv hmu.1
have hwp : v * mu ≤ s + v :=
(mul_le_of_le_one_right hv hmu.2).trans (by linarith)
rw [hybrid_pilot_cell_mean L k0 B u tp t s v mu hL hB hu ht hs hv hmu]
have hnon : 0 ≤ v * mu * (pi * (chebE L).eval (s / B) +
(1 - pi) * exp (-t * s)) :=
mul_nonneg hw (add_nonneg (mul_nonneg hpi.1 hE.1)
(mul_nonneg (sub_nonneg.mpr hpi.2) (Real.exp_nonneg _)))
rw [sub_sub_cancel_left, abs_neg, abs_of_nonneg hnon]
change v * mu * (pi * (chebE L).eval (s / B) +
(1 - pi) * exp (-t * s)) ≤ _
calc
_ = pi * (v * mu * (chebE L).eval (s / B)) +
v * mu * ((1 - pi) * exp (-t * s)) := by ring
_ ≤ B / (eps * (L : Real) ^ 2) + (s + v) * (S ^ 20)⁻¹ := by
apply add_le_add
· exact (mul_le_of_le_one_left (mul_nonneg hw hE.1) hpi.2).trans hb
· exact (mul_le_mul_of_nonneg_left htail hw).trans
(mul_le_mul_of_nonneg_right hwp (by positivity))
Under the stated inputs and conditions, Equation (12) on the light cells: summing the actual cell biases gives one bandwidth term per cell and a single public-scale pilot remainder. This gives the stated result.
Formal statement
hypotheses
S u tp t eps :
hS :
exp 4096 ≤ S
hu :
0 < u
htp :
0 < tp
ht :
0 < t
heps :
0 < eps
hratio :
1 / 3 ≤ t / tp
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
conclusion
k0 :
floor (tp * B / 4)
(∀ j ∈ J, 0 < s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
|∑ j ∈ J, ((∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s j))).prod (cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j)|
≤ J.card * (B / (eps * (L : Real) ^ 2)) + (S ^ 20)⁻¹
Proof (Lean source)
-- @node: hybrid_light_selected_bias_sum
lemma hybrid_light_selected_bias_sum (S u tp t eps : Real)
(hS : exp 4096 ≤ S) (hu : 0 < u) (htp : 0 < tp) (ht : 0 < t)
(heps : 0 < eps) (hratio : 1 / 3 ≤ t / tp)
{alpha : Type} (J : Finset alpha) (s v mu : alpha → Real) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
(∀ j ∈ J, 0 < s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
|∑ j ∈ J, ((∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j)| ≤
J.card * (B / (eps * (L : Real) ^ 2)) + (S ^ 20)⁻¹ := by
dsimp only
intro hcell hmass
calc
_ ≤ ∑ j ∈ J, |(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue _ _ _ u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ j ∈ J, ((2 : Real) ^ 20 * floor (log S / 1024) / min tp t /
(eps * (floor (log S / 1024) : Real) ^ 2) +
(s j + v j) * (S ^ 20)⁻¹) := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hs, hsB, hv, hmu, hov⟩ := hcell j hj
exact hybrid_light_selected_bias S u tp t (s j) (v j) (mu j) eps
hS hu htp ht hs hv hmu heps hov hratio hsB
_ ≤ _ := by
rw [Finset.sum_add_distrib, Finset.sum_const, ← Finset.sum_mul, nsmul_eq_mul]
exact add_le_add_right
(by
simpa using mul_le_mul_of_nonneg_right hmass
(show 0 ≤ (S ^ 20)⁻¹ by positivity)) _
Under the stated inputs and conditions, The coefficient certificate controls the continuation polynomial on the entire nonnegative half-line, including cells outside the approximation interval. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_chebG_eval_growth
lemma hybrid_chebG_eval_growth (L : Nat) (hL : 2 ≤ L) (z : Real) (hz : 0 ≤ z) :
|(chebG L).eval z| ≤ (14 : Real) ^ L * (1 + z) ^ L := by
rw [Polynomial.eval_eq_sum_range' (lt_of_le_of_lt (chebG_natDegree_le L)
(by omega : L - 2 < L - 1))]
calc
_ ≤ ∑ h ∈ range (L - 1), |(chebG L).coeff h * z ^ h| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ h ∈ range (L - 1), |(chebG L).coeff h| * (1 + z) ^ L := by
apply Finset.sum_le_sum
intro h hh
rw [abs_mul, abs_of_nonneg (pow_nonneg hz _)]
apply mul_le_mul_of_nonneg_left _ (abs_nonneg _)
exact (pow_le_pow_left₀ hz (by linarith) h).trans
(pow_le_pow_right₀ (by linarith : 1 ≤ 1 + z) (by
have := Finset.mem_range.mp hh
omega))
_ = (∑ h ∈ range (L - 1), |(chebG L).coeff h|) * (1 + z) ^ L := by
rw [Finset.sum_mul]
_ ≤ _ := mul_le_mul_of_nonneg_right (chebG_coeff_sum_le L hL) (by positivity)
Under the stated inputs and conditions, The polynomial bias outside the bandwidth grows more slowly than the full factorial second-moment envelope used by the pilot-tail absorption. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_chebE_growth
lemma hybrid_chebE_growth (L : Nat) (hL : 2 ≤ L) (z : Real) (hz : 0 < z) :
|(chebE L).eval z| ≤
2 * (((2 : Real) ^ 24) ^ L * (1 + z) ^ (2 * L + 2)) := by
have hG := hybrid_chebG_eval_growth L hL z hz.le
have hid : (chebE L).eval z = 1 - z * (chebG L).eval z := by
rw [(cheb_continuation_eval L (by omega) z hz.ne').2]
field_simp
ring
have hgrowth : z * ((14 : Real) ^ L * (1 + z) ^ L) ≤
((2 : Real) ^ 24) ^ L * (1 + z) ^ (2 * L + 2) := by
calc
_ ≤ (1 + z) * (((2 : Real) ^ 24) ^ L * (1 + z) ^ L) := by
apply mul_le_mul (by linarith)
(mul_le_mul_of_nonneg_right
(pow_le_pow_left₀ (by norm_num : (0 : Real) ≤ 14) (by norm_num) L)
(by positivity)) (by positivity) (by positivity)
_ = ((2 : Real) ^ 24) ^ L * (1 + z) ^ (L + 1) := by ring
_ ≤ _ := mul_le_mul_of_nonneg_left
(pow_le_pow_right₀ (by linarith : 1 ≤ 1 + z) (by omega)) (by positivity)
have hone : 1 ≤ ((2 : Real) ^ 24) ^ L * (1 + z) ^ (2 * L + 2) :=
one_le_mul_of_one_le_of_one_le (one_le_pow₀ (by norm_num))
(one_le_pow₀ (by linarith))
calc
_ = |1 - z * (chebG L).eval z| := by rw [hid]
_ ≤ 1 + z * |(chebG L).eval z| := by
simpa [abs_mul, abs_of_pos hz] using abs_sub (1 : Real) (z * (chebG L).eval z)
_ ≤ 1 + z * ((14 : Real) ^ L * (1 + z) ^ L) :=
add_le_add_right (mul_le_mul_of_nonneg_left hG hz.le) 1
_ ≤ _ := by linarith
Under the stated inputs and conditions, A falsely light cell's polynomial bias is absorbed by its pilot probability, with no inverse-overlap factor in the remainder. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_false_light_bias_bound
lemma hybrid_false_light_bias_bound (S tp B s v mu : Real)
(hS : exp 4096 ≤ S) (htp : 0 < tp) (hB : 0 < B)
(hsB : B < s) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1)
(hscale : (2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B) :
let L := floor (log S / 1024)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4)))
pi * |v * mu * (chebE L).eval (s / B)| ≤ 2 * (s + v) * (S ^ 20)⁻¹ := by
dsimp only
let L := floor (log S / 1024)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4)))
let F := ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2)
have hs : 0 < s := hB.trans hsB
have hcal := hybrid_degree_calibration S hS
have hg := hybrid_chebE_growth L (by have := hcal.1; omega) (s / B) (div_pos hs hB)
have hp : 0 ≤ pi := measureReal_nonneg
have hvm : 0 ≤ v * mu := mul_nonneg hv hmu.1
have hvmp : v * mu ≤ s + v :=
(mul_le_of_le_one_right hv hmu.2).trans (by linarith)
have htail : pi * F ≤ (S ^ 20)⁻¹ := by
calc
_ ≤ exp (-tp * s / 4) * F :=
mul_le_mul_of_nonneg_right (pilot_false_light_tail tp B s htp hB hsB) (by positivity)
_ ≤ exp (-200000 * (L : Real)) := by
simpa only [F, mul_assoc] using
false_light_absorption L tp B s hcal.1 htp hB hsB hscale
_ ≤ _ := hcal.2.2.2
rw [abs_mul, abs_of_nonneg hvm]
calc
_ ≤ pi * ((s + v) * (2 * F)) :=
mul_le_mul_of_nonneg_left (mul_le_mul hvmp hg (abs_nonneg _) (by positivity)) hp
_ = 2 * (s + v) * (pi * F) := by ring
_ ≤ _ := mul_le_mul_of_nonneg_left htail (by positivity)
Under the stated inputs and conditions, The actual heavy-cell mixture bias includes both pilot errors and is bounded by three times cell mass times the public inverse twentieth power. This gives the stated result.
Formal statement
hypotheses
S u tp t s v mu :
hS :
exp 4096 ≤ S
hu :
0 < u
htp :
0 < tp
ht :
0 < t
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
hratio :
1 / 3 ≤ t / tp
conclusion
|(∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t (s * mu) s v))
- (s + v) * mu|
≤ 3 * (s + v) * (S ^ 20)⁻¹
Proof (Lean source)
-- @node: hybrid_heavy_selected_bias
lemma hybrid_heavy_selected_bias (S u tp t s v mu : Real)
(hS : exp 4096 ≤ S) (hu : 0 < u) (htp : 0 < tp) (ht : 0 < t)
(hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1) (hratio : 1 / 3 ≤ t / tp) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
B < s →
|(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s))).prod
(cellPoissonLaw u t (s * mu) s v)) - (s + v) * mu| ≤
3 * (s + v) * (S ^ 20)⁻¹ := by
dsimp only
intro hsB
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic k0)
have hL : 2 ≤ L := by have := (hybrid_degree_calibration S hS).1; omega
have hB : 0 < B := by
have hLp : 0 < (L : Real) := by exact_mod_cast (show 0 < L by omega)
dsimp [B]; positivity
have hs : 0 < s := hB.trans hsB
have hpi : 0 ≤ pi := measureReal_nonneg
have hvm : 0 ≤ v * mu := mul_nonneg hv hmu.1
have hvmp : v * mu ≤ s + v :=
(mul_le_of_le_one_right hv hmu.2).trans (by linarith)
have hpol := hybrid_false_light_bias_bound S tp B s v mu hS htp hB hsB hv hmu
(hybrid_bandwidth_pilot_scale L tp t htp ht)
have hinv := hybrid_false_heavy_bias_bound S tp t s hS htp ht hs.le hratio
rw [hybrid_pilot_cell_mean L k0 B u tp t s v mu hL hB hu ht hs hv hmu,
sub_sub_cancel_left, abs_neg]
change |v * mu * (pi * (chebE L).eval (s / B) + (1 - pi) * exp (-t * s))| ≤ _
calc
_ = |pi * (v * mu * (chebE L).eval (s / B)) +
v * mu * ((1 - pi) * exp (-t * s))| := by congr 1; ring
_ ≤ pi * |v * mu * (chebE L).eval (s / B)| +
v * mu * ((1 - pi) * exp (-t * s)) := by
have hnon : 0 ≤ v * mu * ((1 - pi) * exp (-t * s)) :=
mul_nonneg hvm (mul_nonneg
(sub_nonneg.mpr (show pi ≤ 1 from measureReal_le_one)) (Real.exp_nonneg _))
simpa only [abs_mul, abs_of_nonneg hpi, abs_of_nonneg hnon] using
abs_add_le (pi * (v * mu * (chebE L).eval (s / B)))
(v * mu * ((1 - pi) * exp (-t * s)))
_ ≤ 2 * (s + v) * (S ^ 20)⁻¹ + (s + v) * (S ^ 20)⁻¹ :=
add_le_add hpol ((mul_le_mul_of_nonneg_left hinv hvm).trans
(mul_le_mul_of_nonneg_right hvmp (by positivity)))
_ = _ := by ring
Under the stated inputs and conditions, A zero-success Poisson outcome count makes both branch expectations zero, including at a null own-arm cell. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
(∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t 0 s v) = 0
conclusion 2
(∫ z, inverseCellBranch u z ∂cellPoissonLaw u t 0 s v) = 0
Proof (Lean source)
-- @node: hybrid_zero_outcome_branch_means
lemma hybrid_zero_outcome_branch_means (L : Nat) (B u t s v : Real)
(hu : 0 < u) (ht : 0 < t) (hv : 0 ≤ v) :
(∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t 0 s v) = 0 ∧
(∫ z, inverseCellBranch u z ∂cellPoissonLaw u t 0 s v) = 0 := by
constructor
· let G : Nat → Real := fun K => ∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * B) ^ h
have hG : Integrable G (poissonMeasure (toNNReal (t * s))) := by
apply integrable_finsetSum
intro h _
exact ((poisson_descFactorial_integrable
_ h).const_mul _).div_const _
have heq : polynomialCellBranch L B u t =
(fun z : Nat × Nat × Nat => (z.1 : Real) / u *
(1 + (z.2.2 : Real) * (G z.2.1 / (t * B)))) := by
funext z; dsimp [polynomialCellBranch, G]; ring
rw [heq, poisson_cell_multiplier_mean u t 0 s v hu (by norm_num) hv ht
(fun K => G K / (t * B)) (hG.div_const _), zero_mul]
· let F : Nat → Real := fun K => ((K : Real) + 1)⁻¹
have hF : Integrable F (poissonMeasure (toNNReal (t * s))) := by
simpa only [F, Nat.cast_add, Nat.cast_one] using
(shifted_reciprocal_memLp_two
(toNNReal (t * s))).integrable (by norm_num)
have heq : inverseCellBranch u =
(fun z : Nat × Nat × Nat => (z.1 : Real) / u *
(1 + (z.2.2 : Real) * F z.2.1)) := by
funext z; simp [inverseCellBranch, F, div_eq_mul_inv]
rw [heq, poisson_cell_multiplier_mean u t 0 s v hu (by norm_num) hv ht F hF,
zero_mul]
Under the stated inputs and conditions, Null outcome counts contribute zero also after the independent pilot selection. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t 0 s v))
= 0
Proof (Lean source)
-- @node: hybrid_zero_outcome_selected_mean
lemma hybrid_zero_outcome_selected_mean (L k0 : Nat) (B u tp t s v : Real)
(hu : 0 < u) (ht : 0 < t) (hv : 0 ≤ v) :
(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t 0 s v)) = 0 := by
let : IsProbabilityMeasure (cellPoissonLaw u t 0 s v) := by
unfold cellPoissonLaw
infer_instance
rw [hybrid_cell_event_mixture]
have hLp := hybrid_cell_branches_memLp L B u t 0 s v
rw [integral_eventSelectedMixture _ _ _
measurableSet_Iic _ _ (hLp.1.integrable (by norm_num)) (hLp.2.integrable (by norm_num))]
obtain ⟨hp, hh⟩ := hybrid_zero_outcome_branch_means L B u t s v hu ht hv
rw [hp, hh]
ring
Under the stated inputs and conditions, Equation (12): the sum of canonical hybrid cell biases is bounded by one approximation term per cell and a universal pilot remainder. Null cells are included. This gives the stated result.
Formal statement
hypotheses
S u tp t eps :
hS :
exp 4096 ≤ S
hu :
0 < u
htp :
0 < tp
ht :
0 < t
heps :
0 < eps
hratio :
1 / 3 ≤ t / tp
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
conclusion
k0 :
floor (tp * B / 4)
(∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
|∑ j ∈ J, ((∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s j))).prod (cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j)|
≤ J.card * (B / (eps * (L : Real) ^ 2)) + 3 * (S ^ 20)⁻¹
Proof (Lean source)
-- @node: hybrid_selected_bias_sum
lemma hybrid_selected_bias_sum (S u tp t eps : Real)
(hS : exp 4096 ≤ S) (hu : 0 < u) (htp : 0 < tp) (ht : 0 < t)
(heps : 0 < eps) (hratio : 1 / 3 ≤ t / tp)
{alpha : Type} (J : Finset alpha) (s v mu : alpha → Real) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
(∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
|∑ j ∈ J, ((∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j)| ≤
J.card * (B / (eps * (L : Real) ^ 2)) + 3 * (S ^ 20)⁻¹ := by
dsimp only
intro hcell hmass
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
have hL : 2 ≤ L := by have := (hybrid_degree_calibration S hS).1; omega
have hB : 0 < B := by
have hLp : 0 < (L : Real) := by exact_mod_cast (show 0 < L by omega)
dsimp [B]; positivity
have happrox : 0 ≤ B / (eps * (L : Real) ^ 2) := by positivity
calc
_ ≤ ∑ j ∈ J, |(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ j ∈ J, (B / (eps * (L : Real) ^ 2) + 3 * (s j + v j) * (S ^ 20)⁻¹) := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hs, hv, hmu, hov⟩ := hcell j hj
by_cases hs0 : s j = 0
· have hv0 : v j = 0 := by rw [hs0] at hov; nlinarith
have hz := hybrid_zero_outcome_selected_mean L k0 B u tp t 0 0 hu ht (by norm_num)
simpa only [hs0, hv0, zero_mul, mul_zero, add_zero, sub_zero, abs_zero] using
(show |(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * 0))).prod (cellPoissonLaw u t 0 0 0))| ≤
B / (eps * (L : Real) ^ 2) from by rw [hz, abs_zero]; exact happrox)
· have hsp : 0 < s j := lt_of_le_of_ne hs (Ne.symm hs0)
by_cases hsB : s j ≤ B
· have hb := hybrid_light_selected_bias S u tp t (s j) (v j) (mu j) eps
hS hu htp ht hsp hv hmu heps hov hratio hsB
exact hb.trans (by
have hn : 0 ≤ (s j + v j) * (S ^ 20)⁻¹ := by positivity
change B / (eps * (L : Real) ^ 2) + (s j + v j) * (S ^ 20)⁻¹ ≤ _
nlinarith)
· have hb := hybrid_heavy_selected_bias S u tp t (s j) (v j) (mu j)
hS hu htp ht hv hmu hratio (lt_of_not_ge hsB)
exact hb.trans (le_add_of_nonneg_left happrox)
_ ≤ _ := by
rw [Finset.sum_add_distrib, Finset.sum_const, nsmul_eq_mul]
have heq : (∑ j ∈ J, 3 * (s j + v j) * (S ^ 20)⁻¹) =
3 * (∑ j ∈ J, (s j + v j)) * (S ^ 20)⁻¹ := by
simp only [← Finset.sum_mul, ← Finset.mul_sum]
rw [heq]
apply add_le_add_right
have hh := mul_le_mul_of_nonneg_right hmass
(show 0 ≤ 3 * (S ^ 20)⁻¹ by positivity)
nlinarith only [hh]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.AggregateMSE 2 declarations Mean squared error assembly for independent canonical hybrid cells, retaining every variance and squared-bias term from roadmap equations (12)--(14).
Mean squared error assembly for independent canonical hybrid cells, retaining every variance and squared-bias term from roadmap equations (12)--(14).
Under the stated inputs and conditions, Independence across canonical cells turns the MSE of their sum into the sum of the cell variances plus the squared sum of their biases. This gives the stated result.
Formal statement
hypotheses
conclusion
nu :
fun j
V :
Proof (Lean source)
-- @node: hybrid_canonical_mse_decomposition
lemma hybrid_canonical_mse_decomposition {d : Nat} (L k0 : Nat)
(B u tp t : Real) (s v mu : Fin d → Real) :
let nu := fun j => (poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))
let V := fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2
(∫ z : Fin d → Nat × (Nat × Nat × Nat),
((∑ j, V (z j)) - ∑ j, (s j + v j) * mu j) ^ 2 ∂Measure.pi nu) =
(∑ j, variance V (nu j)) +
(∑ j, ((∫ z, V z ∂nu j) - (s j + v j) * mu j)) ^ 2 := by
dsimp only
let nu := fun j => (poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))
let V := fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2
let (j : Fin d) : IsProbabilityMeasure (nu j) := by
dsimp [nu, cellPoissonLaw]
infer_instance
have hcell (j : Fin d) : MemLp V 2 (nu j) :=
hybrid_pilot_cell_memLp L k0 B u tp t (s j * mu j) (s j) (v j)
have heval (j : Fin d) : MemLp (fun z => V (z j)) 2 (Measure.pi nu) :=
(hcell j).comp_measurePreserving (measurePreserving_eval nu j)
have hsum : MemLp (fun z => ∑ j, V (z j)) 2 (Measure.pi nu) := by
apply memLp_finsetSum
intro j _
exact heval j
rw [baseline_mse_eq_variance_add_bias hsum]
have hvar : variance (fun z => ∑ j, V (z j)) (Measure.pi nu) =
∑ j, variance V (nu j) := by
have hfun : (fun z : Fin d → Nat × (Nat × Nat × Nat) => ∑ j, V (z j)) =
∑ j, (fun z => V (z j)) := by funext z; simp
rw [hfun]
exact variance_sum_pi hcell
have hmean : (∫ z, ∑ j, V (z j) ∂Measure.pi nu) =
∑ j, ∫ z, V z ∂nu j := by
rw [integral_finsetSum]
· apply Finset.sum_congr rfl
intro j _
exact integral_comp_eval (hcell j).aestronglyMeasurable
· intro j _
exact (heval j).integrable (by norm_num)
change variance (fun z => ∑ j, V (z j)) (Measure.pi nu) +
((∫ z, ∑ j, V (z j) ∂Measure.pi nu) - ∑ j, (s j + v j) * mu j) ^ 2 = _
rw [hvar, hmean, ← Finset.sum_sub_distrib]
Under the stated inputs and conditions, The proved light/heavy variance assembly and squared total bias bound give the canonical arm MSE envelope, without dropping either incorrect pilot branch. The inverse-count sum remains explicit for its separate overlap bound. This gives the stated result.
Formal statement
conclusion
∃ CP CH : Real,
0 < CP
0 < CH
conclusion 1
S u tp t eps :
exp 4096 ≤ S
0 < u
0 < tp
0 < t
0 < eps
eps ≤ 1
1 / 3 ≤ t / tp
d :
(∀ j, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(∑ j, (s j + v j)) ≤ 1
k0 :
floor (tp * B / 4)
light :
Finset.univ.filter (fun j => s j ≤ B)
M :
light.sum (fun j => s j + v j)
nu :
fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∫ z : Fin d → Nat × (Nat × Nat × Nat), ((∑ j, hybridCellValue L B k0 u t (z j).2.1 (z j).1 (z j).2.2.1 (z j).2.2.2) - ∑ j, (s j + v j) * mu j) ^ 2 ∂ Measure.pi (fun j => (poissonMeasure (toNNReal (tp * s j))).prod (nu j)))
≤ (∑ j, variance (inverseCellBranch u) (nu j))
+ CP * ((2 : Real) ^ 24) ^ L * (M / (u * eps) + M / t + light.card * B ^ 2 / eps ^ 2)
+ 2 * light.card * (B / (eps * (L : Real) ^ 2)) ^ 2
+ 1 / (exp 1 * t * eps)
+ CH * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t)
+ 18 * (S ^ 20)⁻¹
Proof (Lean source)
-- @node: hybrid_canonical_mse_envelope
lemma hybrid_canonical_mse_envelope :
∃ CP CH : Real, 0 < CP ∧ 0 < CH ∧ ∀ (S u tp t eps : Real),
exp 4096 ≤ S → 0 < u → 0 < tp → 0 < t → 0 < eps → eps ≤ 1 →
1 / 3 ≤ t / tp → ∀ (d : Nat) (s v mu : Fin d → Real),
(∀ j, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(∑ j, (s j + v j)) ≤ 1 →
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
let light := Finset.univ.filter (fun j => s j ≤ B)
let M := light.sum (fun j => s j + v j)
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∫ z : Fin d → Nat × (Nat × Nat × Nat),
((∑ j, hybridCellValue L B k0 u t (z j).2.1 (z j).1
(z j).2.2.1 (z j).2.2.2) - ∑ j, (s j + v j) * mu j) ^ 2 ∂
Measure.pi (fun j => (poissonMeasure (toNNReal (tp * s j))).prod (nu j))) ≤
(∑ j, variance (inverseCellBranch u) (nu j)) +
CP * ((2 : Real) ^ 24) ^ L * (M / (u * eps) + M / t + light.card * B ^ 2 / eps ^ 2) +
2 * light.card * (B / (eps * (L : Real) ^ 2)) ^ 2 +
1 / (exp 1 * t * eps) +
CH * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) +
2 * (d : Real) ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) +
18 * (S ^ 20)⁻¹ := by
obtain ⟨CP, CH, hCP, hCH, hvariance⟩ := hybrid_canonical_variance_sum
refine ⟨CP, CH, hCP, hCH, ?_⟩
intro S u tp t eps hS hu htp ht heps heps1 hratio d s v mu hcell hmass
dsimp only
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
have hL4 : 4 ≤ L := (hybrid_degree_calibration S hS).1
have hLp : 0 < (L : Real) := by exact_mod_cast (show 0 < L by omega)
have hB : 0 < B := by dsimp [B]; positivity
have hpilot := hybrid_bandwidth_pilot_scale L tp t htp ht
have hfactorial := hybrid_bandwidth_pilot_scale L t tp ht htp
rw [min_comm t tp] at hfactorial
have hD : (L : Real) ≤ t * B := by
change (2 : Real) ^ 20 * L ≤ t * B at hfactorial
have hpower : (1 : Real) ≤ 2 ^ 20 := by norm_num
nlinarith only [hfactorial, hpower, hLp]
have hv := hvariance S B u tp t eps hS hB hu htp ht heps heps1 hD hpilot
univ s v mu (fun j _ => hcell j) hmass
have hb := hybrid_selected_squared_bias_sum S u tp t eps hS hu htp ht heps hratio
univ s v mu (fun j _ => hcell j) hmass
rw [hybrid_canonical_mse_decomposition]
have hh := add_le_add hv hb
simpa only [Finset.card_univ, Fintype.card_fin, add_assoc] using hh
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.AggregateVariance 3 declarations Light-cell assembly of the pilot-mixture variance in roadmap equation (13).
Light-cell assembly of the pilot-mixture variance in roadmap equation (13). All three mixture terms are retained, including at null cells.
Under the stated inputs and conditions, Summing equation (13) over light cells retains the inverse-count variance, the polynomial mass envelope, and both parts of the branch separation. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
L k0 :
B u tp t eps :
4 ≤ L
0 < B
0 < u
0 < t
0 < eps
eps ≤ 1
(L : Real) ≤ t * B
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
(∀ j ∈ J, 0 ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
nu :
fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, variance (fun z : Nat × (Nat × Nat × Nat) => hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) ((poissonMeasure (toNNReal (tp * s j))).prod (nu j)))
≤ (∑ j ∈ J, variance (inverseCellBranch u) (nu j))
+ C * ((2 : Real) ^ 24) ^ L * ((J.sum (fun j => s j + v j)) / (u * eps) + (J.sum (fun j => s j + v j)) / t + J.card * B ^ 2 / eps ^ 2)
+ 2 * J.card * (B / (eps * (L : Real) ^ 2)) ^ 2
+ 1 / (exp 1 * t * eps)
Proof (Lean source)
-- @node: light_hybrid_variance_sum
lemma light_hybrid_variance_sum :
∃ C : Real, 0 < C ∧ ∀ (L k0 : Nat) (B u tp t eps : Real),
4 ≤ L → 0 < B → 0 < u → 0 < t → 0 < eps → eps ≤ 1 →
(L : Real) ≤ t * B → ∀ {alpha : Type} (J : Finset alpha)
(s v mu : alpha → Real),
(∀ j ∈ J, 0 ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, variance (fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2)
((poissonMeasure (toNNReal (tp * s j))).prod (nu j))) ≤
(∑ j ∈ J, variance (inverseCellBranch u) (nu j)) +
C * ((2 : Real) ^ 24) ^ L *
((J.sum (fun j => s j + v j)) / (u * eps) +
(J.sum (fun j => s j + v j)) / t + J.card * B ^ 2 / eps ^ 2) +
2 * J.card * (B / (eps * (L : Real) ^ 2)) ^ 2 +
1 / (exp 1 * t * eps) := by
obtain ⟨C, hC, hpol⟩ := light_polynomial_variance_sum
refine ⟨C, hC, ?_⟩
intro L k0 B u tp t eps hL hB hu ht heps heps1 hD alpha J s v mu hcell hmass
dsimp only
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
let pi := fun j => (poissonMeasure (toNNReal (tp * s j))).real (Iic k0)
have hq (j) (hj : j ∈ J) :
0 ≤ s j * mu j ∧ s j * mu j ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧
eps * (s j + v j) ≤ s j := by
obtain ⟨hs, hsB, hv, hmu, hov⟩ := hcell j hj
exact ⟨mul_nonneg hs hmu.1, mul_le_of_le_one_right hs hmu.2, hsB, hv, hov⟩
have hp := hpol L B u t eps hL hB hu ht heps heps1 hD
J (fun j => s j * mu j) s v hq
have hb := hybrid_light_between_branch_sum J s v mu pi L B u t eps
(by omega) hB hu ht heps hcell hmass
calc
_ ≤ ∑ j ∈ J, (variance (inverseCellBranch u) (nu j) +
variance (polynomialCellBranch L B u t) (nu j) +
pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂nu j) -
∫ z, inverseCellBranch u z ∂nu j) ^ 2) := by
apply Finset.sum_le_sum
intro j hj
rw [hybrid_pilot_cell_variance]
have hheavy : (1 - pi j) * variance (inverseCellBranch u) (nu j) ≤
variance (inverseCellBranch u) (nu j) :=
mul_le_of_le_one_left (variance_nonneg _ _) (by
have hh : 0 ≤ pi j := measureReal_nonneg
linarith)
have hpoly : pi j * variance (polynomialCellBranch L B u t) (nu j) ≤
variance (polynomialCellBranch L B u t) (nu j) :=
mul_le_of_le_one_left (variance_nonneg _ _) (show pi j ≤ 1 from measureReal_le_one)
have hh := add_le_add_right (add_le_add hpoly hheavy)
(pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂nu j) -
∫ z, inverseCellBranch u z ∂nu j) ^ 2)
simpa only [nu, pi, add_comm] using hh
_ = (∑ j ∈ J, variance (inverseCellBranch u) (nu j)) +
(∑ j ∈ J, variance (polynomialCellBranch L B u t) (nu j)) +
(∑ j ∈ J, pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂nu j) -
∫ z, inverseCellBranch u z ∂nu j) ^ 2) := by
rw [Finset.sum_add_distrib, Finset.sum_add_distrib]
_ ≤ _ := by
have hh := add_le_add (add_le_add_left hp
(∑ j ∈ J, variance (inverseCellBranch u) (nu j))) hb
simpa only [nu, add_assoc, add_comm, add_left_comm] using hh
Under the stated inputs and conditions, Adding the light and heavy cell bounds retains every term of equation (13). The only global mass requirement is total mass at most one. This gives the stated result.
Formal statement
conclusion
∃ CP CH : Real,
0 < CP
0 < CH
conclusion 1
S B u tp t eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
0 < eps
eps ≤ 1
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
(∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
k0 :
floor (tp * B / 4)
light :
J.filter (fun j => s j ≤ B)
M :
light.sum (fun j => s j + v j)
nu :
fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, variance (fun z : Nat × (Nat × Nat × Nat) => hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) ((poissonMeasure (toNNReal (tp * s j))).prod (nu j)))
≤ (∑ j ∈ J, variance (inverseCellBranch u) (nu j))
+ CP * ((2 : Real) ^ 24) ^ L * (M / (u * eps) + M / t + light.card * B ^ 2 / eps ^ 2)
+ 2 * light.card * (B / (eps * (L : Real) ^ 2)) ^ 2
+ 1 / (exp 1 * t * eps)
+ CH * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t)
Proof (Lean source)
-- @node: hybrid_canonical_variance_sum
lemma hybrid_canonical_variance_sum :
∃ CP CH : Real, 0 < CP ∧ 0 < CH ∧ ∀ (S B u tp t eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t → 0 < eps → eps ≤ 1 →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
∀ {alpha : Type} (J : Finset alpha) (s v mu : alpha → Real),
(∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
let L := floor (log S / 1024)
let k0 := floor (tp * B / 4)
let light := J.filter (fun j => s j ≤ B)
let M := light.sum (fun j => s j + v j)
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, variance (fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2)
((poissonMeasure (toNNReal (tp * s j))).prod (nu j))) ≤
(∑ j ∈ J, variance (inverseCellBranch u) (nu j)) +
CP * ((2 : Real) ^ 24) ^ L * (M / (u * eps) + M / t + light.card * B ^ 2 / eps ^ 2) +
2 * light.card * (B / (eps * (L : Real) ^ 2)) ^ 2 +
1 / (exp 1 * t * eps) +
CH * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) := by
obtain ⟨CP, hCP, hlight⟩ := light_hybrid_variance_sum
obtain ⟨CH, hCH, hheavy⟩ := heavy_hybrid_variance_sum
refine ⟨CP, CH, hCP, hCH, ?_⟩
intro S B u tp t eps hS hB hu htp ht heps heps1 hD hscale alpha J s v mu hcell hmass
dsimp only
let L := floor (log S / 1024)
let k0 := floor (tp * B / 4)
let light := J.filter (fun j => s j ≤ B)
let heavy := J.filter (fun j => ¬s j ≤ B)
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
let V := fun j => variance (fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2)
((poissonMeasure (toNNReal (tp * s j))).prod (nu j))
have hmass_sub (K : Finset alpha) (hK : K ⊆ J) :
(K.sum (fun j => s j + v j)) ≤ 1 := by
apply le_trans (Finset.sum_le_sum_of_subset_of_nonneg hK ?_) hmass
intro j hj _
exact add_nonneg (hcell j hj).1 (hcell j hj).2.1
have hlcell (j) (hj : j ∈ light) :
0 ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j := by
obtain ⟨hjJ, hsB⟩ := Finset.mem_filter.mp hj
obtain ⟨hs, hv, hm, hov⟩ := hcell j hjJ
exact ⟨hs, hsB, hv, hm, hov⟩
have hhcell (j) (hj : j ∈ heavy) :
B < s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j := by
obtain ⟨hjJ, hsB⟩ := Finset.mem_filter.mp hj
obtain ⟨hs, hv, hm, hov⟩ := hcell j hjJ
exact ⟨lt_of_not_ge hsB, hv, hm, hov⟩
have hl := hlight L k0 B u tp t eps (hybrid_degree_calibration S hS).1
hB hu ht heps heps1 hD light s v mu hlcell
(hmass_sub light (filter_subset _ _))
have hh := hheavy S B u tp t eps hS hB hu htp ht heps heps1 hD hscale
heavy s v mu hhcell (hmass_sub heavy (filter_subset _ _))
have hsplit (f : alpha → Real) : light.sum f + heavy.sum f = J.sum f :=
Finset.sum_filter_add_sum_filter_not J (fun j => s j ≤ B) f
have hadd := add_le_add hl hh
change light.sum V + heavy.sum V ≤ _ at hadd
rw [hsplit V] at hadd
have hinverse := hsplit (fun j => variance (inverseCellBranch u) (nu j))
dsimp only [nu] at hinverse
dsimp only [V, nu] at hadd
linarith only [hadd, hinverse]
Under the stated inputs and conditions, Squaring equation (12) gives the approximation term and a pilot remainder of the same order retained in equation (14), including null cells. This gives the stated result.
Formal statement
hypotheses
S u tp t eps :
hS :
exp 4096 ≤ S
hu :
0 < u
htp :
0 < tp
ht :
0 < t
heps :
0 < eps
hratio :
1 / 3 ≤ t / tp
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
conclusion
k0 :
floor (tp * B / 4)
(∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
(∑ j ∈ J, ((∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s j))).prod (cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j)) ^ 2
Proof (Lean source)
-- @node: hybrid_selected_squared_bias_sum
lemma hybrid_selected_squared_bias_sum (S u tp t eps : Real)
(hS : exp 4096 ≤ S) (hu : 0 < u) (htp : 0 < tp) (ht : 0 < t)
(heps : 0 < eps) (hratio : 1 / 3 ≤ t / tp)
{alpha : Type} (J : Finset alpha) (s v mu : alpha → Real) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
(∀ j ∈ J, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
(∑ j ∈ J, ((∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j))) - (s j + v j) * mu j)) ^ 2 ≤
2 * (J.card : Real) ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) +
18 * (S ^ 20)⁻¹ := by
dsimp only
intro hcell hmass
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let b : Real := J.card * (B / (eps * (L : Real) ^ 2))
let r : Real := (S ^ 20)⁻¹
have hb := hybrid_selected_bias_sum S u tp t eps hS hu htp ht heps hratio
J s v mu hcell hmass
have hS1 : 1 ≤ S := (Real.one_le_exp (by norm_num : (0 : Real) ≤ 4096)).trans hS
have hr0 : 0 ≤ r := by dsimp [r]; positivity
have hr1 : r ≤ 1 := by
dsimp [r]
exact inv_le_one_of_one_le₀ (one_le_pow₀ hS1)
have hr2 : r ^ 2 ≤ r := by nlinarith only [hr0, hr1]
have hsq := pow_le_pow_left₀ (abs_nonneg _) hb 2
rw [sq_abs] at hsq
change _ ≤ (b + 3 * r) ^ 2 at hsq
calc
_ ≤ (b + 3 * r) ^ 2 := hsq
_ ≤ 2 * b ^ 2 + 18 * r ^ 2 := by
nlinarith only [sq_nonneg (b - 3 * r)]
_ ≤ 2 * b ^ 2 + 18 * r := by linarith only [hr2]
_ = _ := by dsimp [b, r]; ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.BetweenBranch 4 declarations The light-cell between-branch contribution in equation (13), including null cells.
The light-cell between-branch contribution in equation (13), including null cells.
Under the stated inputs and conditions, Overlap controls squared missing-cell bias by cell mass over twice the intensity. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_squared_missing_cell_bound
lemma hybrid_squared_missing_cell_bound (p s t eps : Real)
(hp : 0 ≤ p) (ht : 0 < t) (heps : 0 < eps) (hov : eps * p ≤ s) :
(p * exp (-t * s)) ^ 2 ≤ p / (2 * exp 1 * t * eps) := by
have hexp : exp (-2 * t * s) ≤ exp (-(2 * t * eps * p)) := by
apply Real.exp_le_exp.mpr
nlinarith
have henv := inverse_count_exp_envelope p (2 * t) eps (by positivity) heps
calc
_ = p * (p * exp (-2 * t * s)) := by
rw [mul_pow, ← Real.exp_nat_mul]
ring_nf
_ ≤ p * (p * exp (-(2 * t * eps * p))) :=
mul_le_mul_of_nonneg_left (mul_le_mul_of_nonneg_left hexp hp) hp
_ ≤ p * (1 / (exp 1 * (2 * t) * eps)) :=
mul_le_mul_of_nonneg_left henv hp
_ = _ := by ring
Under the stated inputs and conditions, Summing the squared inverse-branch biases costs no factor for the cell count. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_squared_missing_mass_sum
lemma hybrid_squared_missing_mass_sum {alpha : Type} (J : Finset alpha)
(p s : alpha → Real) (t eps : Real) (ht : 0 < t) (heps : 0 < eps)
(hcell : ∀ j ∈ J, 0 ≤ p j ∧ eps * p j ≤ s j)
(hmass : (∑ j ∈ J, p j) ≤ 1) :
(∑ j ∈ J, (p j * exp (-t * s j)) ^ 2) ≤
1 / (2 * exp 1 * t * eps) := by
calc
_ ≤ ∑ j ∈ J, p j / (2 * exp 1 * t * eps) := by
apply Finset.sum_le_sum
intro j hj
exact hybrid_squared_missing_cell_bound (p j) (s j) t eps
(hcell j hj).1 ht heps (hcell j hj).2
_ = (∑ j ∈ J, p j) / (2 * exp 1 * t * eps) := by rw [Finset.sum_div]
_ ≤ _ := div_le_div_of_nonneg_right hmass (by positivity)
theorem hybrid_light_between_branch_bound_nonneg Lemma hybrid_light_between_branch_bound_nonneg in the paper ↗
Under the stated inputs and conditions, The light-cell branch separation bound extends to zero own-arm masses by overlap. This gives the stated result.
Formal statement
hypotheses
L :
B u t s v mu eps pi :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
hs :
0 ≤ s
hsB :
s ≤ B
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
heps :
0 < eps
hov :
eps * (s + v) ≤ s
conclusion
pi * (1 - pi) * ((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) - ∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) ^ 2
Proof (Lean source)
-- @node: hybrid_light_between_branch_bound_nonneg
lemma hybrid_light_between_branch_bound_nonneg (L : Nat) (B u t s v mu eps pi : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t)
(hs : 0 ≤ s) (hsB : s ≤ B) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1)
(heps : 0 < eps) (hov : eps * (s + v) ≤ s) :
pi * (1 - pi) *
((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) -
∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) ^ 2 ≤
2 * (B / (eps * (L : Real) ^ 2)) ^ 2 +
2 * ((s + v) * exp (-t * s)) ^ 2 := by
by_cases hs0 : s = 0
· have hv0 : v = 0 := by rw [hs0] at hov; nlinarith
subst s
subst v
obtain ⟨hp, hh⟩ := hybrid_zero_outcome_branch_means L B u t 0 0 hu ht (by norm_num)
simpa only [zero_mul, hp, hh, sub_self, zero_pow, mul_zero, zero_add, add_zero, ne_eq, OfNat.ofNat_ne_zero, not_false_eq_true] using
(show (0 : Real) ≤ 2 * (B / (eps * (L : Real) ^ 2)) ^ 2 from by positivity)
· exact hybrid_light_between_branch_bound L B u t s v mu eps pi hL hB hu ht
(lt_of_le_of_ne hs (Ne.symm hs0)) hsB hv hmu heps hov
Under the stated inputs and conditions, The light-cell part of equation (13) retains its squared approximation term and absorbs all inverse-branch separation terms using total mass at most one. This gives the stated result.
Formal statement
hypotheses
alpha :
Type
J :
Finset alpha
s v mu pi :
alpha → Real
L :
B u t eps :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
heps :
0 < eps
hcell :
∀ j ∈ J, 0 ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j
hmass :
(J.sum (fun j => s j + v j)) ≤ 1
conclusion
(∑ j ∈ J, pi j * (1 - pi j) * ((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s j * mu j) (s j) (v j)) - ∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s j * mu j) (s j) (v j)) ^ 2)
Proof (Lean source)
-- @node: hybrid_light_between_branch_sum
lemma hybrid_light_between_branch_sum {alpha : Type} (J : Finset alpha)
(s v mu pi : alpha → Real) (L : Nat) (B u t eps : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t) (heps : 0 < eps)
(hcell : ∀ j ∈ J, 0 ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j)
(hmass : (J.sum (fun j => s j + v j)) ≤ 1) :
(∑ j ∈ J, pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s j * mu j) (s j) (v j)) -
∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s j * mu j) (s j) (v j)) ^ 2) ≤
2 * J.card * (B / (eps * (L : Real) ^ 2)) ^ 2 +
1 / (exp 1 * t * eps) := by
have hmissing := hybrid_squared_missing_mass_sum J (fun j => s j + v j) s t eps ht heps
(fun j hj => ⟨add_nonneg (hcell j hj).1 (hcell j hj).2.2.1, (hcell j hj).2.2.2.2⟩)
hmass
calc
_ ≤ ∑ j ∈ J, (2 * (B / (eps * (L : Real) ^ 2)) ^ 2 +
2 * ((s j + v j) * exp (-t * s j)) ^ 2) := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hs, hsB, hv, hmu, hov⟩ := hcell j hj
exact hybrid_light_between_branch_bound_nonneg L B u t (s j) (v j) (mu j)
eps (pi j) hL hB hu ht hs hsB hv hmu heps hov
_ = 2 * J.card * (B / (eps * (L : Real) ^ 2)) ^ 2 +
2 * (∑ j ∈ J, ((s j + v j) * exp (-t * s j)) ^ 2) := by
rw [Finset.sum_add_distrib, Finset.sum_const, nsmul_eq_mul, ← Finset.mul_sum]
ring
_ ≤ _ := by
have hh := mul_le_mul_of_nonneg_left hmissing (by norm_num : (0 : Real) ≤ 2)
have heq : 2 * (1 / (2 * exp 1 * t * eps)) = 1 / (exp 1 * t * eps) := by ring
rw [heq] at hh
exact add_le_add_right hh _
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.BranchMeans 6 declarations Canonical Poisson branch statistics and their exact polynomial and inverse-count means.
Canonical Poisson branch statistics and their exact polynomial and inverse-count means.
def cellPoissonLaw
Canonical independent outcome-success, own-arm, and opposite-arm counts.
Definition (Lean source)
parameters
u t q s v :
given by
(poissonMeasure (toNNReal (u * q))).prod ((poissonMeasure (toNNReal (t * s))).prod (poissonMeasure (toNNReal (t * v))))
def polynomialCellBranch
For the polynomial degree, the scale bound, the marked-count intensity, the arm-count intensity, and a triple of observed counts, the polynomial cell branch is the factorial-polynomial estimator used for the light-cell branch.
def inverseCellBranch
For the marked-count intensity and a triple of observed counts, the inverse-count cell branch is the reciprocal-count estimator used for the heavy-cell branch.
Under the stated inputs and conditions, A finite factorial lift has the same mean as its ordinary polynomial. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: poisson_factorial_polynomial_mean
lemma poisson_factorial_polynomial_mean (p : Polynomial Real) (J : Nat)
(hJ : p.natDegree < J) (D z : Real) (hD : 0 < D) (hz : 0 ≤ z) :
(∫ K : Nat, (∑ h ∈ range J,
p.coeff h * (K.descFactorial h : Real) / D ^ h)
∂poissonMeasure (toNNReal (D * z))) = p.eval z := by
rw [integral_finsetSum]
· rw [p.eval_eq_sum_range' hJ z]
apply Finset.sum_congr rfl
intro h _
rw [integral_div, integral_const_mul,
poisson_descFactorial_moment]
simp only [Real.coe_toNNReal _ (mul_nonneg hD.le hz), mul_pow]
field_simp
· intro h _
exact ((poisson_descFactorial_integrable
_ h).const_mul _).div_const _
Under the stated inputs and conditions, Independent Poisson counts factor the outcome and opposite-arm multiplier means. This gives the stated result.
Formal statement
hypotheses
u t q s v :
hu :
0 < u
hq :
0 ≤ q
hv :
0 ≤ v
ht :
0 < t
hF :
Proof (Lean source)
-- @node: poisson_cell_multiplier_mean
lemma poisson_cell_multiplier_mean (u t q s v : Real) (hu : 0 < u)
(hq : 0 ≤ q) (hv : 0 ≤ v) (ht : 0 < t) (F : Nat → Real)
(hF : Integrable F (poissonMeasure (toNNReal (t * s)))) :
(∫ z : Nat × Nat × Nat, (z.1 : Real) / u *
(1 + (z.2.2 : Real) * F z.2.1) ∂cellPoissonLaw u t q s v) =
q * (1 + t * v * ∫ K, F K ∂poissonMeasure (toNNReal (t * s))) := by
have hW := (poisson_natCast_memLp_two
(toNNReal (t * v))).integrable (by norm_num)
have hm (r : Real) (hr : 0 ≤ r) :
(∫ K : Nat, (K : Real) ∂poissonMeasure (toNNReal r)) = r := by
have hh := poisson_descFactorial_moment
(toNNReal r) 1
simpa [Real.coe_toNNReal _ hr] using hh
have hprod := hF.mul_prod hW
simp only [cellPoissonLaw]
rw [integral_prod_mul (fun Z : Nat => (Z : Real) / u)
(fun z : Nat × Nat => 1 + (z.2 : Real) * F z.1),
integral_div, hm (u * q) (mul_nonneg hu.le hq)]
have heq : (fun z : Nat × Nat => 1 + (z.2 : Real) * F z.1) =
(fun z : Nat × Nat => 1 + F z.1 * (z.2 : Real)) := by
funext z; ring
rw [heq, integral_add (integrable_const 1) hprod, integral_prod_mul F (fun W : Nat => (W : Real)),
hm (t * v) (mul_nonneg ht.le hv)]
simp only [integral_const, smul_eq_mul, probReal_univ]
field_simp
Under the stated inputs and conditions, Both branch means equal the arm target minus their explicit bias terms.
Formal statement
hypotheses
L :
B u t s v mu :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
hs :
0 < s
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
conclusion
conclusion 1
(∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v)
= (s + v) * mu - v * mu * (chebE L).eval (s / B)
conclusion 2
(∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v)
= (s + v) * mu - v * mu * exp (-t * s)
Proof (Lean source)
-- @node: hybrid_branch_means
lemma hybrid_branch_means (L : Nat) (B u t s v mu : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t)
(hs : 0 < s) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1) :
(∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) =
(s + v) * mu - v * mu * (chebE L).eval (s / B) ∧
(∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) =
(s + v) * mu - v * mu * exp (-t * s) := by
let G : Nat → Real := fun K => ∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * B) ^ h
have hG : Integrable G (poissonMeasure (toNNReal (t * s))) := by
apply integrable_finsetSum
intro h _
exact ((poisson_descFactorial_integrable
_ h).const_mul _).div_const _
have hdeg := chebG_natDegree_le L
have hmean : (∫ K, G K ∂poissonMeasure (toNNReal (t * s))) =
(chebG L).eval (s / B) := by
have hr : t * B * (s / B) = t * s := by field_simp
simpa only [hr] using poisson_factorial_polynomial_mean (chebG L) (L - 1)
(by omega) (t * B) (s / B) (mul_pos ht hB) (by positivity)
have hq : 0 ≤ s * mu := mul_nonneg hs.le hmu.1
constructor
· have hh := poisson_cell_multiplier_mean u t (s * mu) s v hu hq hv ht
(fun K => G K / (t * B)) (hG.div_const _)
have heq : polynomialCellBranch L B u t =
(fun z : Nat × Nat × Nat => (z.1 : Real) / u *
(1 + (z.2.2 : Real) * (G z.2.1 / (t * B)))) := by
funext z; dsimp [polynomialCellBranch, G]; ring
rw [heq, hh, integral_div, hmean]
rw [(cheb_continuation_eval L (by omega) (s / B) (by positivity)).2]
field_simp
ring
· let F : Nat → Real := fun K => ((K : Real) + 1)⁻¹
have hF : Integrable F (poissonMeasure (toNNReal (t * s))) := by
simpa only [F, Nat.cast_add, Nat.cast_one] using
(shifted_reciprocal_memLp_two
(toNNReal (t * s))).integrable (by norm_num)
have hh := poisson_cell_multiplier_mean u t (s * mu) s v hu hq hv ht F hF
have heq : inverseCellBranch u =
(fun z : Nat × Nat × Nat => (z.1 : Real) / u *
(1 + (z.2.2 : Real) * F z.2.1)) := by
funext z; simp [inverseCellBranch, F, div_eq_mul_inv]
rw [heq, hh]
have hrec := (poisson_inverse_moments (toNNReal (t * s))).1
simp only [Real.coe_toNNReal _ (mul_nonneg ht.le hs.le),
if_neg (ne_of_gt (mul_pos ht hs))] at hrec
rw [show (∫ K, F K ∂poissonMeasure (toNNReal (t * s))) =
(1 - exp (-(t * s))) / (t * s) from hrec]
field_simp
ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.Calibration 1 declarations Numerical degree and exponential absorption calibrated in S, uniformly in overlap.
Numerical degree and exponential absorption calibrated in S, uniformly in overlap.
Under the stated inputs and conditions, The rare-label degree yields the numerical absorption needed in the hybrid proof.
Formal statement
Proof (Lean source)
-- @node: hybrid_degree_calibration
lemma hybrid_degree_calibration (S : Real) (hS : exp 4096 ≤ S) :
let L := floor (log S / 1024)
4 ≤ L ∧ log S < 1280 * (L : Real) ∧
((2 : Real) ^ 24) ^ L * (L : Real) ^ 3 ≤ sqrt S ∧
exp (-200000 * (L : Real)) ≤ (S ^ 20)⁻¹ := by
let L := floor (log S / 1024)
have hSpos : 0 < S := lt_of_lt_of_le (Real.exp_pos _) hS
have hy : 4096 ≤ log S := (Real.le_log_iff_exp_le hSpos).2 hS
have hL : 4 ≤ L := Nat.le_floor (by dsimp; linarith)
have hLreal : (4 : Real) ≤ L := by exact_mod_cast hL
have hLpos : 0 < (L : Real) := by linarith
have hfloor : (L : Real) ≤ log S / 1024 :=
Nat.floor_le (by linarith)
have hceil : log S / 1024 < (L : Real) + 1 :=
Nat.lt_floor_add_one _
have hyupper : log S < 1280 * (L : Real) := by linarith
refine ⟨hL, hyupper, ?_, ?_⟩
· apply (Real.log_le_log_iff (by positivity) (Real.sqrt_pos.2 hSpos)).1
rw [Real.log_mul (by positivity) (by positivity), Real.log_pow,
Real.log_pow, Real.log_pow, Real.log_sqrt hSpos.le]
have hlogtwo : log 2 ≤ 1 := by
linarith [Real.log_le_sub_one_of_pos (by norm_num : (0 : Real) < 2)]
have hlogL : log (L : Real) ≤ (L : Real) :=
(Real.log_le_sub_one_of_pos hLpos).trans (by linarith)
norm_num
nlinarith
· apply (Real.log_le_log_iff (Real.exp_pos _) (by positivity)).1
rw [Real.log_exp, Real.log_inv, Real.log_pow]
norm_num
linarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.CanonicalRate 3 declarations Overlap bounds for canonical inverse-count variances and the completed canonical hybrid arm rate.
Overlap bounds for canonical inverse-count variances and the completed canonical hybrid arm rate. The transfer to an arbitrary independent count family is separate.
Under the stated inputs and conditions, The canonical product counts satisfy the inverse-count cell variance envelope, including zero success intensity and null cells. This gives the stated result.
Formal statement
hypotheses
u t q s v eps :
hu :
0 < u
ht :
0 < t
hq :
0 ≤ q
hqs :
q ≤ s
hv :
0 ≤ v
heps :
0 < eps
hov :
eps * (s + v) ≤ s
conclusion
≤ (2 + 4 * (2 ^ 16 : Real)) * (s + v) * (1 / (u * eps) + 1 / (t * eps))
Proof (Lean source)
-- @node: hybrid_canonical_inverse_variance
lemma hybrid_canonical_inverse_variance (u t q s v eps : Real)
(hu : 0 < u) (ht : 0 < t) (hq : 0 ≤ q) (hqs : q ≤ s) (hv : 0 ≤ v)
(heps : 0 < eps) (hov : eps * (s + v) ≤ s) :
variance (inverseCellBranch u) (cellPoissonLaw u t q s v) ≤
(2 + 4 * (2 ^ 16 : Real)) * (s + v) * (1 / (u * eps) + 1 / (t * eps)) := by
let nu := (poissonMeasure (toNNReal (t * s))).prod
(poissonMeasure (toNNReal (t * v)))
let W : Nat × Nat → Real := fun z => 1 + (z.2 : Real) / ((z.1 : Real) + 1)
let : IsProbabilityMeasure (cellPoissonLaw u t q s v) := by
dsimp [cellPoissonLaw]; infer_instance
have hs : 0 ≤ s := hq.trans hqs
obtain ⟨hw, hws, hwv⟩ := inverse_count_weight_moments nu fst snd
(toNNReal (t * s)) (toNNReal (t * v)) (by fun_prop) (by fun_prop)
(by simp [nu]) (by simp [nu])
((indepFun_prod (μ := poissonMeasure (toNNReal (t * s)))
(ν := poissonMeasure (toNNReal (t * v))) measurable_id measurable_id).symm)
have hz := poisson_natCast_memLp_two
(toNNReal (u * q))
have hzm := poisson_natCast_first_moment
(toNNReal (u * q))
have hzs := poisson_count_second_moment
(toNNReal (u * q))
simp only [Real.coe_toNNReal _ (mul_nonneg hu.le hq)] at hzm hzs
have hm : (∫ z, inverseCellBranch u z ∂cellPoissonLaw u t q s v) =
q * (∫ z, W z ∂nu) := by
change (∫ z : Nat × (Nat × Nat), (z.1 : Real) / u * W z.2 ∂
(poissonMeasure (toNNReal (u * q))).prod nu) = _
rw [integral_prod_mul (fun k : Nat => (k : Real) / u) W, integral_div, hzm, mul_div_cancel_left₀ q hu.ne']
have hsq : (∫ z, inverseCellBranch u z ^ 2 ∂cellPoissonLaw u t q s v) =
(q / u + q ^ 2) * (∫ z, W z ^ 2 ∂nu) := by
change (∫ z : Nat × (Nat × Nat), ((z.1 : Real) / u * W z.2) ^ 2 ∂
(poissonMeasure (toNNReal (u * q))).prod nu) = _
simp only [mul_pow, div_pow]
rw [integral_prod_mul (fun k : Nat => (k : Real) ^ 2 / u ^ 2)
(fun z : Nat × Nat => W z ^ 2), integral_div, hzs]
congr 1
field_simp
ring
have hid : variance (inverseCellBranch u) (cellPoissonLaw u t q s v) =
q / u * (∫ z, W z ^ 2 ∂nu) + q ^ 2 * variance W nu := by
rw [variance_eq_sub (hybrid_cell_branches_memLp 0 1 u t q s v).2]
simp only [Pi.pow_apply]
rw [hm, hsq, variance_eq_sub hw]
simp only [Pi.pow_apply]
ring
have henv := inverse_count_cell_variance_envelope s v q u t eps (2 ^ 16)
hs hv hq hqs hu ht heps (by positivity) hov
rw [hid]
apply le_trans _ henv
have h1 := mul_le_mul_of_nonneg_left hws (show 0 ≤ q / u by positivity)
have h2 := mul_le_mul_of_nonneg_left hwv (sq_nonneg q)
simp only [Real.coe_toNNReal _ (mul_nonneg ht.le hs),
Real.coe_toNNReal _ (mul_nonneg ht.le hv)] at h1 h2
simpa only [W, mul_assoc] using add_le_add h1 h2
theorem hybrid_canonical_inverse_variance_sum Lemma hybrid_canonical_inverse_variance_sum in the paper ↗
Under the stated inputs and conditions, Summing canonical inverse-count variances uses total mass at most one and therefore has no cell-count factor. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_canonical_inverse_variance_sum
lemma hybrid_canonical_inverse_variance_sum {d : Nat} (u t eps : Real)
(s v mu : Fin d → Real) (hu : 0 < u) (ht : 0 < t) (heps : 0 < eps)
(hcell : ∀ j, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(hmass : (∑ j : Fin d, (s j + v j)) ≤ 1) :
(∑ j, variance (inverseCellBranch u) (cellPoissonLaw u t (s j * mu j) (s j) (v j))) ≤
(2 + 4 * (2 ^ 16 : Real)) * (1 / (u * eps) + 1 / (t * eps)) := by
calc
_ ≤ ∑ j, (2 + 4 * (2 ^ 16 : Real)) * (s j + v j) *
(1 / (u * eps) + 1 / (t * eps)) := by
apply Finset.sum_le_sum
intro j _
obtain ⟨hs, hv, hm, hov⟩ := hcell j
exact hybrid_canonical_inverse_variance u t (s j * mu j) (s j) (v j) eps hu ht
(mul_nonneg hs hm.1) (mul_le_of_le_one_right hs hm.2) hv heps hov
_ = (2 + 4 * (2 ^ 16 : Real)) * (∑ j : Fin d, (s j + v j)) *
(1 / (u * eps) + 1 / (t * eps)) := by
rw [← Finset.sum_mul, ← Finset.mul_sum]
_ ≤ _ := by
have hh := mul_le_mul_of_nonneg_left hmass
(show 0 ≤ (2 + 4 * (2 ^ 16 : Real)) * (1 / (u * eps) + 1 / (t * eps)) by positivity)
convert hh using 1 <;> first | rfl | ring
Under the stated inputs and conditions, The canonical independent three-pool statistic has the full universal arm rate, assembled from its actual MSE envelope and inverse-count variance bound. Null cells and boundary outcome means are included. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S N eps u tp t :
exp 4096 ≤ S
0 < eps
eps ≤ 1
S ≤ N * eps
S / 32 ≤ u * eps
N / 64 ≤ tp
N / 64 ≤ t
1 / 3 ≤ t / tp
d :
1 ≤ d
(∀ j, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(∑ j : Fin d, (s j + v j)) ≤ 1
k0 :
floor (tp * B / 4)
(∫ z : Fin d → Nat × (Nat × Nat × Nat), ((∑ j, hybridCellValue L B k0 u t (z j).2.1 (z j).1 (z j).2.2.1 (z j).2.2.2) - ∑ j, (s j + v j) * mu j) ^ 2 ∂ Measure.pi (fun j => (poissonMeasure (toNNReal (tp * s j))).prod (cellPoissonLaw u t (s j * mu j) (s j) (v j))))
≤ C * (1 / S + ((d : Real) / (N * eps * L)) ^ 2)
Proof (Lean source)
-- @node: hybrid_canonical_arm_rate
lemma hybrid_canonical_arm_rate :
∃ C : Real, 0 < C ∧ ∀ (S N eps u tp t : Real),
exp 4096 ≤ S → 0 < eps → eps ≤ 1 → S ≤ N * eps →
S / 32 ≤ u * eps → N / 64 ≤ tp → N / 64 ≤ t → 1 / 3 ≤ t / tp →
∀ (d : Nat) (s v mu : Fin d → Real), 1 ≤ d →
(∀ j, 0 ≤ s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j) →
(∑ j : Fin d, (s j + v j)) ≤ 1 →
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
(∫ z : Fin d → Nat × (Nat × Nat × Nat),
((∑ j, hybridCellValue L B k0 u t (z j).2.1 (z j).1
(z j).2.2.1 (z j).2.2.2) - ∑ j, (s j + v j) * mu j) ^ 2 ∂
Measure.pi (fun j => (poissonMeasure (toNNReal (tp * s j))).prod
(cellPoissonLaw u t (s j * mu j) (s j) (v j)))) ≤
C * (1 / S + ((d : Real) / (N * eps * L)) ^ 2) := by
obtain ⟨CP, CH, hCP, hCH, hmse⟩ := hybrid_canonical_mse_envelope
obtain ⟨C, hC, hrate⟩ := hybrid_canonical_envelope_rate
(2 + 4 * (2 ^ 16 : Real)) CP CH (by positivity) hCP hCH
refine ⟨C, hC, ?_⟩
intro S N eps u tp t hS heps heps1 hNS hu htp ht hratio d s v mu hd hcell hmass
dsimp only
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let light := Finset.univ.filter (fun j => s j ≤ B)
let M := light.sum (fun j => s j + v j)
have hSp : 0 < S := (Real.exp_pos _).trans_le hS
have hN : 0 < N := by nlinarith
have huP : 0 < u := by nlinarith
have htP : 0 < t := (show 0 < N / 64 by positivity).trans_le ht
have htpP : 0 < tp := (show 0 < N / 64 by positivity).trans_le htp
have hLp : 0 < (L : Real) := by
exact_mod_cast (show 0 < L from by have := (hybrid_degree_calibration S hS).1; omega)
have hB : 0 ≤ B := by dsimp [B]; positivity
have hm : 0 ≤ M := Finset.sum_nonneg (fun j _ => add_nonneg (hcell j).1 (hcell j).2.1)
have hmassLight : M ≤ 1 := by
apply le_trans (Finset.sum_le_sum_of_subset_of_nonneg (filter_subset _ _) ?_) hmass
intro j _ _
exact add_nonneg (hcell j).1 (hcell j).2.1
have hmassB : M ≤ (d : Real) * B / eps := by
have hh := (light_cell_mass_bound light s v B eps heps
(fun j hj => ⟨(hcell j).2.2.2, (Finset.mem_filter.mp hj).2⟩) hmassLight).trans
(min_le_right _ _)
apply hh.trans
apply div_le_div_of_nonneg_right _ heps.le
apply mul_le_mul_of_nonneg_right _ hB
exact_mod_cast (show light.card ≤ d from
(Finset.card_le_card (filter_subset _ _)).trans_eq (by simp))
have hc : (light.card : Real) ≤ d := by
exact_mod_cast (show light.card ≤ d from
(Finset.card_le_card (filter_subset _ _)).trans_eq (by simp))
have hi := hybrid_canonical_inverse_variance_sum u t eps s v mu huP htP heps hcell hmass
have henv := hmse S u tp t eps hS huP htpP htP heps heps1 hratio d s v mu hcell hmass
have hr := hrate S N d eps u tp t M light.card
(∑ j, variance (inverseCellBranch u) (cellPoissonLaw u t (s j * mu j) (s j) (v j)))
hS (by exact_mod_cast hd) heps heps1 hNS hu htp ht hm hmassB (by positivity) hc hi
exact henv.trans hr
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.CountTransfer 1 declarations Transfer from independent count families to the canonical four-count product law.
Transfer from independent count families to the canonical four-count product law.
Under the stated inputs and conditions, Grouping four independent counts in each cell gives the product of their four marginal laws, jointly over all cells. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: hybrid_four_count_product_law
lemma hybrid_four_count_product_law {d : Nat} {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (X : Fin 4 × Fin d → Om → Nat)
(hX : ∀ i, Measurable (X i)) (hind : iIndepFun X mu) :
mu.map (fun om j => (X (0,j) om, X (1,j) om, X (2,j) om, X (3,j) om)) =
Measure.pi (fun j => (mu.map (X (0,j))).prod
((mu.map (X (1,j))).prod ((mu.map (X (2,j))).prod (mu.map (X (3,j)))))) := by
classical
apply Measure.ext_of_singleton
intro z
rw [Measure.map_apply (by fun_prop) (measurableSet_singleton z)]
have hset : (fun om j => (X (0,j) om, X (1,j) om, X (2,j) om, X (3,j) om)) ⁻¹' {z} =
⋂ i ∈ (Finset.univ : Finset (Fin 4 × Fin d)), X i ⁻¹'
{if i.1 = 0 then (z i.2).1 else if i.1 = 1 then (z i.2).2.1
else if i.1 = 2 then (z i.2).2.2.1 else (z i.2).2.2.2} := by
ext om
simp only [Set.mem_preimage, Set.mem_singleton_iff, Set.mem_iInter,
Finset.mem_univ, forall_const]
constructor
· intro h i
rw [← h]
rcases i with ⟨i,j⟩
fin_cases i <;> simp
· intro h
funext j
exact Prod.ext (by simpa using h (0,j))
(Prod.ext (by simpa using h (1,j))
(Prod.ext (by simpa using h (2,j)) (by simpa using h (3,j))))
rw [hset, hind.measure_inter_preimage_eq_mul Finset.univ (fun _ _ => measurableSet_singleton _)]
have hpre (i : Fin 4 × Fin d) (k : Nat) :
mu (X i ⁻¹' {k}) = (mu.map (X i)) {k} :=
(Measure.map_apply (hX i) (measurableSet_singleton k)).symm
simp_rw [hpre]
have hz : ({z} : Set (Fin d → Nat × (Nat × Nat × Nat))) =
Set.univ.pi (fun j => {z j}) := by ext y; simp [funext_iff]
rw [hz, Measure.pi_pi]
simp only [Fintype.prod_prod_type]
rw [Finset.prod_comm]
apply Finset.prod_congr rfl
intro j _
rw [show ({z j} : Set (Nat × (Nat × Nat × Nat))) =
{(z j).1} ×ˢ ({(z j).2.1} ×ˢ ({(z j).2.2.1} ×ˢ {(z j).2.2.2})) by ext y; simp]
simp only [Measure.prod_prod, Fin.prod_univ_four, Fin.reduceEq, if_true, if_false]
ac_rfl
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.FalseLight 1 declarations Exponential absorption of false-light polynomial moments outside the bandwidth.
Exponential absorption of false-light polynomial moments outside the bandwidth.
Under the stated inputs and conditions, The heavy-cell pilot tail absorbs the full factorial second-moment growth.
Formal statement
Proof (Lean source)
-- @node: false_light_absorption
lemma false_light_absorption (L : Nat) (tp B s : Real) (hL : 4 ≤ L)
(htp : 0 < tp) (hB : 0 < B) (hs : B < s) (hscale : (2 : Real) ^ 20 * L ≤ tp * B) :
exp (-tp * s / 4) * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2) ≤
exp (-200000 * (L : Real)) := by
have hLreal : (4 : Real) ≤ L := by exact_mod_cast hL
have hLpos : 0 < (L : Real) := by linarith
have hz : 1 < s / B := (one_lt_div hB).2 hs
have hlogz : log (1 + s / B) ≤ s / B := by
have h := Real.log_le_sub_one_of_pos (by linarith : 0 < 1 + s / B)
linarith
have hlogtwo : log 2 ≤ 1 := by
linarith [Real.log_le_sub_one_of_pos (by norm_num : (0 : Real) < 2)]
have hscaled : (2 : Real) ^ 20 * L * (s / B) ≤ tp * s := by
have h := mul_le_mul_of_nonneg_right hscale (by linarith : 0 ≤ s / B)
have heq : tp * B * (s / B) = tp * s := by field_simp
rwa [heq] at h
apply (Real.log_le_log_iff (by positivity) (Real.exp_pos _)).1
rw [Real.log_mul (by positivity) (by positivity),
Real.log_mul (by positivity) (by positivity), Real.log_exp, Real.log_exp,
Real.log_pow, Real.log_pow, Real.log_pow]
norm_num at hscaled ⊢
have hpoly : ((2 : Real) * L + 2) * log (1 + s / B) ≤
((2 : Real) * L + 2) * (s / B) :=
mul_le_mul_of_nonneg_left hlogz (by positivity)
have hpower : (L : Real) * (24 * log 2) ≤ 24 * L := by
nlinarith
have hdegree : ((2 : Real) * L + 2) * (s / B) ≤
(5 / 2 : Real) * L * (s / B) := by
nlinarith
have hratio : (L : Real) ≤ L * (s / B) := by nlinarith
nlinarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.FalseLightRisk 11 declarations Global polynomial second moments and false-light pilot-weighted risk, implementing roadmap equations (7), (9), and (10) without a minimum cell mass.
Global polynomial second moments and false-light pilot-weighted risk, implementing roadmap equations (7), (9), and (10) without a minimum cell mass.
theorem polynomial_multiplier_second_moment Lemma polynomial_multiplier_second_moment in the paper ↗
Under the stated inputs and conditions, The factorial multiplier has a global squared-moment envelope. This gives the stated conclusion.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
(∫ z : Nat × Nat, (1 + (z.2 : Real) / (t * B) * (∑ h ∈ range (L - 1), (chebG L).coeff h * (z.1.descFactorial h : Real) / (t * B) ^ h)) ^ 2 ∂ (poissonMeasure (toNNReal (t * s))).prod (poissonMeasure (toNNReal (t * v))))
≤ C * (((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2))
Proof (Lean source)
-- @node: polynomial_multiplier_second_moment
lemma polynomial_multiplier_second_moment :
∃ C : Real, 0 < C ∧ ∀ (L : Nat) (B t s v : Real),
4 ≤ L → 0 < B → 0 < t → 0 ≤ s → 0 ≤ v → (L : Real) ≤ t * B →
(∫ z : Nat × Nat,
(1 + (z.2 : Real) / (t * B) * (∑ h ∈ range (L - 1),
(chebG L).coeff h * (z.1.descFactorial h : Real) / (t * B) ^ h)) ^ 2 ∂
(poissonMeasure (toNNReal (t * s))).prod (poissonMeasure (toNNReal (t * v)))) ≤
C * (((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) *
(1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) := by
refine ⟨2, by norm_num, ?_⟩
intro L B t s v hL hB ht hs hv hD
let G : Nat → Real := fun K => ∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * B) ^ h
let mu := poissonMeasure (toNNReal (t * s))
let nu := poissonMeasure (toNNReal (t * v))
have hG : MemLp G 2 mu := poisson_factorial_polynomial_memLp _ _ _ _
have hW := poisson_natCast_memLp_two
(toNNReal (t * v))
have hprod : Integrable (fun z : Nat × Nat => G z.1 ^ 2 *
((z.2 : Real) / (t * B)) ^ 2) (mu.prod nu) :=
hG.integrable_sq.mul_prod (by
simpa only [div_eq_mul_inv] using (hW.mul_const ((t * B)⁻¹)).integrable_sq)
have hbound : (∫ z : Nat × Nat, (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2
∂mu.prod nu) ≤ 2 + 2 * ((∫ K, G K ^ 2 ∂mu) *
(∫ W : Nat, ((W : Real) / (t * B)) ^ 2 ∂nu)) := by
calc
_ ≤ ∫ z : Nat × Nat, 2 + 2 * (G z.1 ^ 2 * ((z.2 : Real) / (t * B)) ^ 2)
∂mu.prod nu := by
apply integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
((integrable_const 2).add (hprod.const_mul 2))
exact Filter.Eventually.of_forall (fun z => by
have hh : (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2 ≤
2 + 2 * ((z.2 : Real) / (t * B) * G z.1) ^ 2 := by
nlinarith only [sq_nonneg (1 - (z.2 : Real) / (t * B) * G z.1)]
change (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2 ≤
2 + 2 * (G z.1 ^ 2 * ((z.2 : Real) / (t * B)) ^ 2)
calc
_ ≤ 2 + 2 * ((z.2 : Real) / (t * B) * G z.1) ^ 2 := hh
_ = _ := by ring)
_ = _ := by
rw [integral_add (integrable_const 2) (hprod.const_mul 2), integral_const_mul,
integral_prod_mul (fun K => G K ^ 2) (fun W : Nat => ((W : Real) / (t * B)) ^ 2)]
simp
have hsratio : 0 ≤ s / B := div_nonneg hs hB.le
have hcert := (chebyshev_factorial_certificate L (by omega)).2.2.2.2.2
t B (s / B) ht hB hsratio hD
have hrate : t * (s / B) * B = t * s := by field_simp
have hGs : (∫ K, G K ^ 2 ∂mu) ≤
((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L) := by
have hg := hcert.2
rw [hrate] at hg
exact hg.trans (mul_le_mul_of_nonneg_right
(pow_le_pow_left₀ (by norm_num : (0 : Real) ≤ 392) (by norm_num) L)
(by positivity))
have hWmoment : (∫ W : Nat, ((W : Real) / (t * B)) ^ 2 ∂nu) =
v ^ 2 / B ^ 2 + v / (t * B ^ 2) := by
have hm := poisson_descFactorial_mixed
(toNNReal (t * v)) 1 1
norm_num [Finset.sum_range_succ,
Real.toNNReal_of_nonneg (mul_nonneg ht.le hv)] at hm
simp_rw [div_pow]
rw [integral_div]
simp only [nu]
simp_rw [pow_two]
simp only [Real.toNNReal_of_nonneg (mul_nonneg ht.le hv)]
rw [hm]
field_simp
have hF : 1 ≤ (((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) := one_le_mul_of_one_le_of_one_le
(one_le_pow₀ (by norm_num)) (one_le_pow₀ (by linarith))
have hV : 0 ≤ v ^ 2 / B ^ 2 + v / (t * B ^ 2) := by positivity
change (∫ z : Nat × Nat, (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2
∂mu.prod nu) ≤ _
rw [hWmoment] at hbound
calc
_ ≤ 2 + 2 * ((∫ K, G K ^ 2 ∂mu) * (v ^ 2 / B ^ 2 + v / (t * B ^ 2))) := hbound
_ ≤ 2 * (((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) := by
have hh := mul_le_mul_of_nonneg_right hGs hV
calc
_ ≤ 2 * (((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) +
2 * ((((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) * (v ^ 2 / B ^ 2 + v / (t * B ^ 2))) :=
add_le_add (by linarith only [hF]) (mul_le_mul_of_nonneg_left hh (by norm_num))
_ = _ := by ring
Under the stated inputs and conditions, Outside the bandwidth, rescaling the light-cell overlap monomials retains only cell mass, squared cell mass, and the quadratic continuation factor. This gives the stated result.
Formal statement
hypotheses
B u t q s v eps :
hB :
0 < B
hu :
0 < u
ht :
0 < t
hq :
0 ≤ q
hqs :
q ≤ s
hsB :
B < s
hv :
0 ≤ v
heps :
0 < eps
heps1 :
eps ≤ 1
hov :
eps * (s + v) ≤ s
hD :
1 ≤ t * B
conclusion
(q / u + q ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2))
≤ 3 * (1 + s / B) ^ 2 * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t)
Proof (Lean source)
-- @node: false_light_second_moment_algebra
lemma false_light_second_moment_algebra (B u t q s v eps : Real)
(hB : 0 < B) (hu : 0 < u) (ht : 0 < t) (hq : 0 ≤ q) (hqs : q ≤ s)
(hsB : B < s) (hv : 0 ≤ v) (heps : 0 < eps) (heps1 : eps ≤ 1)
(hov : eps * (s + v) ≤ s) (hD : 1 ≤ t * B) :
(q / u + q ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) ≤
3 * (1 + s / B) ^ 2 * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t) := by
have hs : 0 < s := hB.trans hsB
have hDs : 1 ≤ t * s := hD.trans (mul_le_mul_of_nonneg_left hsB.le ht.le)
obtain ⟨h1, h2, h3, h4, h5, _⟩ :=
light_cell_overlap_monomials s t s v eps hs ht hs.le le_rfl hv heps hov hDs
have hp : 0 ≤ s + v := by positivity
have hse : s ≤ (s + v) / eps := by
apply (le_div_iff₀ heps).2
nlinarith
have hvse : v ≤ (s + v) / eps := by
apply (le_div_iff₀ heps).2
nlinarith
have hlocal : (q / u + q ^ 2) * (1 + v ^ 2 / s ^ 2 + v / (t * s ^ 2)) ≤
3 * (s + v) / (u * eps) + (s + v) / t + 2 * (s + v) ^ 2 := by
calc
_ ≤ (s / u + s ^ 2) * (1 + v ^ 2 / s ^ 2 + v / (t * s ^ 2)) :=
mul_le_mul_of_nonneg_right
(add_le_add (div_le_div_of_nonneg_right hqs hu.le)
(pow_le_pow_left₀ hq hqs 2)) (by positivity)
_ = s / u + (s * v ^ 2 / s ^ 2) / u +
(s * v / (t * s ^ 2)) / u + s ^ 2 +
s ^ 2 * v ^ 2 / s ^ 2 + s ^ 2 * v / (t * s ^ 2) := by ring
_ ≤ (s + v) / eps / u + (s + v) / eps / u +
(s + v) / eps / u + (s + v) ^ 2 + (s + v) ^ 2 + (s + v) / t :=
add_le_add (add_le_add (add_le_add (add_le_add (add_le_add
(div_le_div_of_nonneg_right hse hu.le)
(div_le_div_of_nonneg_right h1 hu.le))
(div_le_div_of_nonneg_right (h2.trans hvse) hu.le)) h5) h4) h3
_ = _ := by ring
have hz : 0 ≤ s / B := by positivity
have hz2 : (s / B) ^ 2 ≤ (1 + s / B) ^ 2 := by nlinarith
have hbase : 1 ≤ (1 + s / B) ^ 2 := by nlinarith
have hscale : 1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2) ≤
(1 + s / B) ^ 2 * (1 + v ^ 2 / s ^ 2 + v / (t * s ^ 2)) := by
have halg : v ^ 2 / B ^ 2 + v / (t * B ^ 2) =
(s / B) ^ 2 * (v ^ 2 / s ^ 2 + v / (t * s ^ 2)) := by field_simp
rw [show 1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2) =
1 + (v ^ 2 / B ^ 2 + v / (t * B ^ 2)) by ring, halg]
have hh := mul_le_mul_of_nonneg_right hz2
(show 0 ≤ v ^ 2 / s ^ 2 + v / (t * s ^ 2) by positivity)
nlinarith only [hbase, hh]
calc
_ ≤ (q / u + q ^ 2) * ((1 + s / B) ^ 2 *
(1 + v ^ 2 / s ^ 2 + v / (t * s ^ 2))) :=
mul_le_mul_of_nonneg_left hscale (by positivity)
_ = (1 + s / B) ^ 2 * ((q / u + q ^ 2) *
(1 + v ^ 2 / s ^ 2 + v / (t * s ^ 2))) := by ring
_ ≤ (1 + s / B) ^ 2 *
(3 * (s + v) / (u * eps) + (s + v) / t + 2 * (s + v) ^ 2) :=
mul_le_mul_of_nonneg_left hlocal (sq_nonneg _)
_ ≤ _ := by
have hh : 3 * (s + v) / (u * eps) + (s + v) / t + 2 * (s + v) ^ 2 ≤
3 * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t) := by
have hpT : 0 ≤ (s + v) / t := by positivity
calc
_ = 3 * ((s + v) / (u * eps)) + (s + v) / t + 2 * (s + v) ^ 2 := by ring
_ ≤ _ := by nlinarith only [hpT, sq_nonneg (s + v)]
calc
_ ≤ (1 + s / B) ^ 2 *
(3 * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t)) :=
mul_le_mul_of_nonneg_left hh (sq_nonneg _)
_ = _ := by ring
theorem false_light_polynomial_second_moment Lemma false_light_polynomial_second_moment in the paper ↗
Under the stated inputs and conditions, Equation (7): the heavy-cell polynomial second moment has the full factorial continuation factor and no extra inverse-overlap factor on its squared mass. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
L :
B u t q s v eps :
4 ≤ L
0 < B
0 < u
0 < t
0 ≤ q
q ≤ s
B < s
0 ≤ v
0 < eps
eps ≤ 1
eps * (s + v) ≤ s
(L : Real) ≤ t * B
(∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v)
≤ C * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2) * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t)
Proof (Lean source)
-- @node: false_light_polynomial_second_moment
lemma false_light_polynomial_second_moment :
∃ C : Real, 0 < C ∧ ∀ (L : Nat) (B u t q s v eps : Real),
4 ≤ L → 0 < B → 0 < u → 0 < t → 0 ≤ q → q ≤ s → B < s →
0 ≤ v → 0 < eps → eps ≤ 1 → eps * (s + v) ≤ s → (L : Real) ≤ t * B →
(∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v) ≤
C * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2) *
((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t) := by
obtain ⟨C, hC, hb⟩ := polynomial_multiplier_second_moment
refine ⟨3 * C, by positivity, ?_⟩
intro L B u t q s v eps hL hB hu ht hq hqs hsB hv heps heps1 hov hD
have hs := hB.trans hsB
have hD1 : 1 ≤ t * B := by
have hLreal : (4 : Real) ≤ L := by exact_mod_cast hL
linarith
have ha := false_light_second_moment_algebra B u t q s v eps
hB hu ht hq hqs hsB hv heps heps1 hov hD1
rw [polynomial_cell_second_moment L B u t q s v hu hq]
calc
_ ≤ (q / u + q ^ 2) * (C * (((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) *
(1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2))) :=
mul_le_mul_of_nonneg_left (hb L B t s v hL hB ht hs.le hv hD) (by positivity)
_ = (C * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) *
((q / u + q ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2))) := by ring
_ ≤ (C * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L)) *
(3 * (1 + s / B) ^ 2 * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t)) :=
mul_le_mul_of_nonneg_left ha (by positivity)
_ = _ := by rw [pow_add]; ring
theorem false_light_weighted_polynomial_moment Lemma false_light_weighted_polynomial_moment in the paper ↗
Equations (7) and (9): the actual false-light selection probability absorbs the polynomial branch's second moment into the inverse twentieth power of S.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S B u tp t q s v eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
0 ≤ q
q ≤ s
B < s
0 ≤ v
0 < eps
eps ≤ 1
eps * (s + v) ≤ s
pi :
pi * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v)
≤ C * (S ^ 20)⁻¹ * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t)
Proof (Lean source)
-- @node: false_light_weighted_polynomial_moment
lemma false_light_weighted_polynomial_moment :
∃ C : Real, 0 < C ∧ ∀ (S B u tp t q s v eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t →
0 ≤ q → q ≤ s → B < s → 0 ≤ v → 0 < eps → eps ≤ 1 →
eps * (s + v) ≤ s →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
let L := floor (log S / 1024)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4)))
pi * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v) ≤
C * (S ^ 20)⁻¹ * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t) := by
obtain ⟨C, hC, hb⟩ := false_light_polynomial_second_moment
refine ⟨C, hC, ?_⟩
intro S B u tp t q s v eps hS hB hu htp ht hq hqs hsB hv heps heps1 hov hD hscale
dsimp only
let L := floor (log S / 1024)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4)))
let M := (s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t
have hcal := hybrid_degree_calibration S hS
have hs : 0 < s := hB.trans hsB
have hM : 0 ≤ M := by dsimp [M]; positivity
have hpi : 0 ≤ pi := measureReal_nonneg
have htail := pilot_false_light_tail tp B s htp hB hsB
have hm := hb L B u t q s v eps hcal.1 hB hu ht hq hqs hsB hv heps heps1 hov hD
have habs := (false_light_absorption L tp B s hcal.1 htp hB hsB hscale).trans hcal.2.2.2
calc
_ ≤ pi * (C * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2) * M) :=
mul_le_mul_of_nonneg_left hm hpi
_ ≤ exp (-tp * s / 4) *
(C * ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2) * M) :=
mul_le_mul_of_nonneg_right htail (by positivity)
_ = (C * M) * (exp (-tp * s / 4) *
((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2)) := by ring
_ ≤ (C * M) * (S ^ 20)⁻¹ :=
mul_le_mul_of_nonneg_left habs (mul_nonneg hC.le hM)
_ = _ := by ring
Under the stated inputs and conditions, Equation (10): summing false-light polynomial second moments uses only the total mass bound, retaining the outcome and factorial intensity terms. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S B u tp t eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
0 < eps
eps ≤ 1
alpha :
Type
J :
Finset alpha
q s v :
alpha → Real
(∀ j ∈ J, 0 ≤ q j ∧ q j ≤ s j ∧ B < s j ∧ 0 ≤ v j ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
(∑ j ∈ J, (poissonMeasure (toNNReal (tp * s j))).real (Iic (floor (tp * B / 4))) * (∫ z, polynomialCellBranch (floor (log S / 1024)) B u t z ^ 2 ∂cellPoissonLaw u t (q j) (s j) (v j)))
≤ C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t)
Proof (Lean source)
-- @node: false_light_weighted_moment_sum
lemma false_light_weighted_moment_sum :
∃ C : Real, 0 < C ∧ ∀ (S B u tp t eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t → 0 < eps → eps ≤ 1 →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
∀ {alpha : Type} (J : Finset alpha) (q s v : alpha → Real),
(∀ j ∈ J, 0 ≤ q j ∧ q j ≤ s j ∧ B < s j ∧ 0 ≤ v j ∧ eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
(∑ j ∈ J, (poissonMeasure (toNNReal (tp * s j))).real
(Iic (floor (tp * B / 4))) *
(∫ z, polynomialCellBranch (floor (log S / 1024)) B u t z ^ 2
∂cellPoissonLaw u t (q j) (s j) (v j))) ≤
C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) := by
obtain ⟨C, hC, hb⟩ := false_light_weighted_polynomial_moment
refine ⟨C, hC, ?_⟩
intro S B u tp t eps hS hB hu htp ht heps heps1 hD hscale alpha J q s v hcell hmass
have hp (j) (hj : j ∈ J) : 0 ≤ s j + v j := by
obtain ⟨hq, hqs, _, hv, _⟩ := hcell j hj
linarith
have hp1 (j) (hj : j ∈ J) : s j + v j ≤ 1 := by
exact (Finset.single_le_sum (fun j hj => hp j hj) hj).trans hmass
have hsq : (J.sum (fun j => (s j + v j) ^ 2)) ≤ 1 := by
calc
_ ≤ J.sum (fun j => s j + v j) := by
apply Finset.sum_le_sum
intro j hj
nlinarith only [hp j hj, hp1 j hj]
_ ≤ 1 := hmass
calc
_ ≤ ∑ j ∈ J, C * (S ^ 20)⁻¹ *
((s j + v j) / (u * eps) + (s j + v j) ^ 2 + (s j + v j) / t) := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hq, hqs, hsB, hv, hov⟩ := hcell j hj
exact hb S B u tp t (q j) (s j) (v j) eps
hS hB hu htp ht hq hqs hsB hv heps heps1 hov hD hscale
_ = C * (S ^ 20)⁻¹ *
((J.sum (fun j => s j + v j)) / (u * eps) +
(J.sum (fun j => (s j + v j) ^ 2)) + (J.sum (fun j => s j + v j)) / t) := by
simp only [← Finset.mul_sum, Finset.sum_add_distrib, ← Finset.sum_div]
_ ≤ _ := by
apply mul_le_mul_of_nonneg_left _ (by positivity)
exact add_le_add (add_le_add
(div_le_div_of_nonneg_right hmass (by positivity)) hsq)
(div_le_div_of_nonneg_right hmass ht.le)
Under the stated inputs and conditions, The unweighted false-light pilot probability obeys the same public-scale tail bound, since the absorbed factorial continuation factor is at least one. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: false_light_pilot_public_scale
lemma false_light_pilot_public_scale (S tp B s : Real)
(hS : exp 4096 ≤ S) (htp : 0 < tp) (hB : 0 < B) (hsB : B < s)
(hscale : (2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B) :
(poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4))) ≤
(S ^ 20)⁻¹ := by
let L := floor (log S / 1024)
have hcal := hybrid_degree_calibration S hS
have hs : 0 < s := hB.trans hsB
have hF : 1 ≤ ((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2) :=
one_le_mul_of_one_le_of_one_le (one_le_pow₀ (by norm_num))
(one_le_pow₀ (le_add_of_nonneg_right (div_nonneg hs.le hB.le)))
calc
_ ≤ exp (-tp * s / 4) := pilot_false_light_tail tp B s htp hB hsB
_ ≤ exp (-tp * s / 4) *
(((2 : Real) ^ 24) ^ L * (1 + s / B) ^ (2 * L + 2)) :=
le_mul_of_one_le_right (Real.exp_nonneg _) hF
_ = exp (-tp * s / 4) * ((2 : Real) ^ 24) ^ L *
(1 + s / B) ^ (2 * L + 2) := by ring
_ ≤ exp (-200000 * (L : Real)) :=
false_light_absorption L tp B s hcal.1 htp hB hsB hscale
_ ≤ (S ^ 20)⁻¹ := hcal.2.2.2
Under the stated inputs and conditions, On a heavy cell the mixture separation is bounded by the pilot-weighted polynomial second moment and squared mass, using square integrability rather than branch independence. This gives the stated result.
Formal statement
hypotheses
L :
B u t s v mu pi :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
hs :
0 < s
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
hpi :
pi ∈ Icc 0 1
conclusion
pi * (1 - pi) * ((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) - ∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) ^ 2
≤ 2 * pi * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t (s * mu) s v)
+ 2 * pi * (s + v) ^ 2
Proof (Lean source)
-- @node: false_light_between_branch_moment
lemma false_light_between_branch_moment (L : Nat) (B u t s v mu pi : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t)
(hs : 0 < s) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1) (hpi : pi ∈ Icc 0 1) :
pi * (1 - pi) *
((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) -
∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) ^ 2 ≤
2 * pi * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t (s * mu) s v) +
2 * pi * (s + v) ^ 2 := by
let nu := cellPoissonLaw u t (s * mu) s v
let : IsProbabilityMeasure nu := by unfold nu cellPoissonLaw; infer_instance
let mp := ∫ z, polynomialCellBranch L B u t z ∂nu
let mh := ∫ z, inverseCellBranch u z ∂nu
have hp := (hybrid_cell_branches_memLp L B u t (s * mu) s v).1
have hvar := variance_nonneg (polynomialCellBranch L B u t) nu
rw [variance_eq_sub hp] at hvar
have hp2 : mp ^ 2 ≤ ∫ z, polynomialCellBranch L B u t z ^ 2 ∂nu := by
change 0 ≤ (∫ z, polynomialCellBranch L B u t z ^ 2 ∂nu) - mp ^ 2 at hvar
exact sub_nonneg.mp hvar
have hh := (hybrid_branch_means L B u t s v mu hL hB hu ht hs hv hmu).2
have hexp : exp (-t * s) ≤ 1 := Real.exp_le_one_iff.mpr (by nlinarith)
have hmh : mh ∈ Icc 0 (s + v) := by
dsimp [mh, nu]
rw [hh]
have hvm : 0 ≤ v * mu := mul_nonneg hv hmu.1
have hsm : 0 ≤ s * mu := mul_nonneg hs.le hmu.1
have hprod := mul_le_of_le_one_right hvm hexp
constructor
· nlinarith only [hprod, hsm]
· have hupper := mul_le_of_le_one_right (show 0 ≤ s + v by positivity) hmu.2
nlinarith only [hupper, mul_nonneg hvm (Real.exp_nonneg (-t * s))]
have hmh2 : mh ^ 2 ≤ (s + v) ^ 2 := pow_le_pow_left₀ hmh.1 hmh.2 2
calc
_ ≤ pi * (mp - mh) ^ 2 := by
apply mul_le_mul_of_nonneg_right _ (sq_nonneg _)
exact mul_le_of_le_one_right hpi.1 (by linarith [hpi.1])
_ ≤ pi * (2 * mp ^ 2 + 2 * mh ^ 2) :=
mul_le_mul_of_nonneg_left
(by nlinarith only [sq_nonneg (mp + mh)]) hpi.1
_ ≤ pi * (2 * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂nu) + 2 * (s + v) ^ 2) :=
mul_le_mul_of_nonneg_left (by linarith only [hp2, hmh2]) hpi.1
_ = _ := by ring
theorem false_light_between_branch_public_scale Lemma false_light_between_branch_public_scale in the paper ↗
The heavy-cell contribution to the between-branch mixture variance is absorbed by the same public-scale bound as its falsely selected polynomial second moment.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S B u tp t s v mu eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
B < s
0 ≤ v
mu ∈ Icc 0 1
0 < eps
eps ≤ 1
eps * (s + v) ≤ s
pi :
nu :
cellPoissonLaw u t (s * mu) s v
pi * (1 - pi) * ((∫ z, polynomialCellBranch L B u t z ∂nu) - ∫ z, inverseCellBranch u z ∂nu) ^ 2
≤ C * (S ^ 20)⁻¹ * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t)
Proof (Lean source)
-- @node: false_light_between_branch_public_scale
lemma false_light_between_branch_public_scale :
∃ C : Real, 0 < C ∧ ∀ (S B u tp t s v mu eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t → B < s →
0 ≤ v → mu ∈ Icc 0 1 → 0 < eps → eps ≤ 1 → eps * (s + v) ≤ s →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
let L := floor (log S / 1024)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4)))
let nu := cellPoissonLaw u t (s * mu) s v
pi * (1 - pi) *
((∫ z, polynomialCellBranch L B u t z ∂nu) - ∫ z, inverseCellBranch u z ∂nu) ^ 2 ≤
C * (S ^ 20)⁻¹ * ((s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t) := by
obtain ⟨C, hC, hb⟩ := false_light_weighted_polynomial_moment
refine ⟨2 * C + 2, by positivity, ?_⟩
intro S B u tp t s v mu eps hS hB hu htp ht hsB hv hmu heps heps1 hov hD hscale
dsimp only
let L := floor (log S / 1024)
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4)))
let nu := cellPoissonLaw u t (s * mu) s v
let M := (s + v) / (u * eps) + (s + v) ^ 2 + (s + v) / t
have hs : 0 < s := hB.trans hsB
have hp : 0 ≤ s + v := by positivity
have hq : 0 ≤ s * mu := mul_nonneg hs.le hmu.1
have hqs : s * mu ≤ s := mul_le_of_le_one_right hs.le hmu.2
have hpi : pi ∈ Icc 0 1 := ⟨measureReal_nonneg, measureReal_le_one⟩
have hm := hb S B u tp t (s * mu) s v eps
hS hB hu htp ht hq hqs hsB hv heps heps1 hov hD hscale
have htail := false_light_pilot_public_scale S tp B s hS htp hB hsB hscale
have hp2 : (s + v) ^ 2 ≤ M := by
have hU : 0 ≤ (s + v) / (u * eps) := by positivity
have hT : 0 ≤ (s + v) / t := by positivity
dsimp [M]
linarith
have hcal := hybrid_degree_calibration S hS
calc
_ ≤ 2 * pi * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂nu) + 2 * pi * (s + v) ^ 2 :=
false_light_between_branch_moment L B u t s v mu pi
(by have := hcal.1; omega) hB hu ht hs hv hmu hpi
_ = 2 * (pi * (∫ z, polynomialCellBranch L B u t z ^ 2 ∂nu)) +
2 * (pi * (s + v) ^ 2) := by ring
_ ≤ 2 * (C * (S ^ 20)⁻¹ * M) + 2 * ((S ^ 20)⁻¹ * (s + v) ^ 2) :=
add_le_add (mul_le_mul_of_nonneg_left hm (by norm_num))
(mul_le_mul_of_nonneg_left
(mul_le_mul_of_nonneg_right htail (sq_nonneg _)) (by norm_num))
_ ≤ 2 * (C * (S ^ 20)⁻¹ * M) + 2 * ((S ^ 20)⁻¹ * M) := by
apply add_le_add le_rfl
exact mul_le_mul_of_nonneg_left
(mul_le_mul_of_nonneg_left hp2 (by positivity)) (by norm_num)
_ = _ := by ring
Under the stated inputs and conditions, The falsely selected polynomial variances are bounded by the already summed second moments in equation (10). This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S B u tp t eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
0 < eps
eps ≤ 1
alpha :
Type
J :
Finset alpha
q s v :
alpha → Real
(∀ j ∈ J, 0 ≤ q j ∧ q j ≤ s j ∧ B < s j ∧ 0 ≤ v j ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
(∑ j ∈ J, (poissonMeasure (toNNReal (tp * s j))).real (Iic (floor (tp * B / 4))) * variance (polynomialCellBranch (floor (log S / 1024)) B u t) (cellPoissonLaw u t (q j) (s j) (v j)))
≤ C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t)
Proof (Lean source)
-- @node: false_light_weighted_variance_sum
lemma false_light_weighted_variance_sum :
∃ C : Real, 0 < C ∧ ∀ (S B u tp t eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t → 0 < eps → eps ≤ 1 →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
∀ {alpha : Type} (J : Finset alpha) (q s v : alpha → Real),
(∀ j ∈ J, 0 ≤ q j ∧ q j ≤ s j ∧ B < s j ∧ 0 ≤ v j ∧ eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
(∑ j ∈ J, (poissonMeasure (toNNReal (tp * s j))).real
(Iic (floor (tp * B / 4))) *
variance (polynomialCellBranch (floor (log S / 1024)) B u t)
(cellPoissonLaw u t (q j) (s j) (v j))) ≤
C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) := by
obtain ⟨C, hC, hb⟩ := false_light_weighted_moment_sum
refine ⟨C, hC, ?_⟩
intro S B u tp t eps hS hB hu htp ht heps heps1 hD hscale alpha J q s v hcell hmass
calc
_ ≤ ∑ j ∈ J, (poissonMeasure (toNNReal (tp * s j))).real
(Iic (floor (tp * B / 4))) *
(∫ z, polynomialCellBranch (floor (log S / 1024)) B u t z ^ 2
∂cellPoissonLaw u t (q j) (s j) (v j)) := by
apply Finset.sum_le_sum
intro j hj
let : IsProbabilityMeasure (cellPoissonLaw u t (q j) (s j) (v j)) := by
unfold cellPoissonLaw
infer_instance
exact mul_le_mul_of_nonneg_left
(variance_le_expectation_sq
(hybrid_cell_branches_memLp _ B u t (q j) (s j) (v j)).1.aestronglyMeasurable)
measureReal_nonneg
_ ≤ _ := hb S B u tp t eps hS hB hu htp ht heps heps1 hD hscale
J q s v hcell hmass
Under the stated inputs and conditions, The heavy-cell part of equation (13) sums to a universal pilot-tail remainder, without any independence assumption between the polynomial and inverse branches. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S B u tp t eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
0 < eps
eps ≤ 1
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
(∀ j ∈ J, B < s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
pi :
fun j
nu :
fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, pi j * (1 - pi j) * ((∫ z, polynomialCellBranch L B u t z ∂nu j) - ∫ z, inverseCellBranch u z ∂nu j) ^ 2)
≤ C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t)
Proof (Lean source)
-- @node: false_light_between_branch_sum
lemma false_light_between_branch_sum :
∃ C : Real, 0 < C ∧ ∀ (S B u tp t eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t → 0 < eps → eps ≤ 1 →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
∀ {alpha : Type} (J : Finset alpha) (s v mu : alpha → Real),
(∀ j ∈ J, B < s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
let L := floor (log S / 1024)
let pi := fun j => (poissonMeasure (toNNReal (tp * s j))).real
(Iic (floor (tp * B / 4)))
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂nu j) -
∫ z, inverseCellBranch u z ∂nu j) ^ 2) ≤
C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) := by
obtain ⟨C, hC, hb⟩ := false_light_between_branch_public_scale
refine ⟨C, hC, ?_⟩
intro S B u tp t eps hS hB hu htp ht heps heps1 hD hscale alpha J s v mu hcell hmass
dsimp only
have hp (j) (hj : j ∈ J) : 0 ≤ s j + v j := by
obtain ⟨hsB, hv, _, _⟩ := hcell j hj
linarith
have hp1 (j) (hj : j ∈ J) : s j + v j ≤ 1 :=
(Finset.single_le_sum (fun j hj => hp j hj) hj).trans hmass
have hsq : (J.sum (fun j => (s j + v j) ^ 2)) ≤ 1 := by
calc
_ ≤ J.sum (fun j => s j + v j) := by
apply Finset.sum_le_sum
intro j hj
nlinarith only [hp j hj, hp1 j hj]
_ ≤ 1 := hmass
calc
_ ≤ ∑ j ∈ J, C * (S ^ 20)⁻¹ *
((s j + v j) / (u * eps) + (s j + v j) ^ 2 + (s j + v j) / t) := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hsB, hv, hmu, hov⟩ := hcell j hj
exact hb S B u tp t (s j) (v j) (mu j) eps
hS hB hu htp ht hsB hv hmu heps heps1 hov hD hscale
_ = C * (S ^ 20)⁻¹ *
((J.sum (fun j => s j + v j)) / (u * eps) +
(J.sum (fun j => (s j + v j) ^ 2)) + (J.sum (fun j => s j + v j)) / t) := by
simp only [← Finset.mul_sum, Finset.sum_add_distrib, ← Finset.sum_div]
_ ≤ _ := by
apply mul_le_mul_of_nonneg_left _ (by positivity)
exact add_le_add (add_le_add
(div_le_div_of_nonneg_right hmass (by positivity)) hsq)
(div_le_div_of_nonneg_right hmass ht.le)
Under the stated inputs and conditions, Equation (13), summed over heavy cells: the actual pilot-selected variances retain the inverse-count variance sum and absorb both false-light contributions. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
S B u tp t eps :
exp 4096 ≤ S
0 < B
0 < u
0 < tp
0 < t
0 < eps
eps ≤ 1
alpha :
Type
J :
Finset alpha
s v mu :
alpha → Real
(∀ j ∈ J, B < s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧ eps * (s j + v j) ≤ s j)
(J.sum (fun j => s j + v j)) ≤ 1
k0 :
floor (tp * B / 4)
nu :
fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, variance (fun z : Nat × (Nat × Nat × Nat) => hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) ((poissonMeasure (toNNReal (tp * s j))).prod (nu j)))
≤ (∑ j ∈ J, variance (inverseCellBranch u) (nu j))
+ C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t)
Proof (Lean source)
-- @node: heavy_hybrid_variance_sum
lemma heavy_hybrid_variance_sum :
∃ C : Real, 0 < C ∧ ∀ (S B u tp t eps : Real),
exp 4096 ≤ S → 0 < B → 0 < u → 0 < tp → 0 < t → 0 < eps → eps ≤ 1 →
(floor (log S / 1024) : Real) ≤ t * B →
(2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B →
∀ {alpha : Type} (J : Finset alpha) (s v mu : alpha → Real),
(∀ j ∈ J, B < s j ∧ 0 ≤ v j ∧ mu j ∈ Icc 0 1 ∧
eps * (s j + v j) ≤ s j) →
(J.sum (fun j => s j + v j)) ≤ 1 →
let L := floor (log S / 1024)
let k0 := floor (tp * B / 4)
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
(∑ j ∈ J, variance (fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2)
((poissonMeasure (toNNReal (tp * s j))).prod (nu j))) ≤
(∑ j ∈ J, variance (inverseCellBranch u) (nu j)) +
C * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) := by
obtain ⟨CP, hCP, hpol⟩ := false_light_weighted_variance_sum
obtain ⟨CB, hCB, hbetween⟩ := false_light_between_branch_sum
refine ⟨CP + CB, by positivity, ?_⟩
intro S B u tp t eps hS hB hu htp ht heps heps1 hD hscale alpha J s v mu hcell hmass
dsimp only
let L := floor (log S / 1024)
let k0 := floor (tp * B / 4)
let nu := fun j => cellPoissonLaw u t (s j * mu j) (s j) (v j)
let pi := fun j => (poissonMeasure (toNNReal (tp * s j))).real (Iic k0)
have hq (j) (hj : j ∈ J) :
0 ≤ s j * mu j ∧ s j * mu j ≤ s j ∧ B < s j ∧ 0 ≤ v j ∧
eps * (s j + v j) ≤ s j := by
obtain ⟨hsB, hv, hmu, hov⟩ := hcell j hj
have hs := (hB.trans hsB).le
exact ⟨mul_nonneg hs hmu.1, mul_le_of_le_one_right hs hmu.2, hsB, hv, hov⟩
have hp := hpol S B u tp t eps hS hB hu htp ht heps heps1 hD hscale
J (fun j => s j * mu j) s v hq hmass
have hb := hbetween S B u tp t eps hS hB hu htp ht heps heps1 hD hscale
J s v mu hcell hmass
calc
_ ≤ ∑ j ∈ J, (variance (inverseCellBranch u) (nu j) +
pi j * variance (polynomialCellBranch L B u t) (nu j) +
pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂nu j) -
∫ z, inverseCellBranch u z ∂nu j) ^ 2) := by
apply Finset.sum_le_sum
intro j hj
rw [hybrid_pilot_cell_variance]
have hheavy : (1 - pi j) * variance (inverseCellBranch u) (nu j) ≤
variance (inverseCellBranch u) (nu j) :=
mul_le_of_le_one_left (variance_nonneg _ _) (by
have hh : 0 ≤ pi j := measureReal_nonneg
linarith)
change pi j * _ + (1 - pi j) * _ + _ ≤ _
linarith only [hheavy]
_ = (∑ j ∈ J, variance (inverseCellBranch u) (nu j)) +
(∑ j ∈ J, pi j * variance (polynomialCellBranch L B u t) (nu j)) +
(∑ j ∈ J, pi j * (1 - pi j) *
((∫ z, polynomialCellBranch L B u t z ∂nu j) -
∫ z, inverseCellBranch u z ∂nu j) ^ 2) := by
rw [Finset.sum_add_distrib, Finset.sum_add_distrib]
_ ≤ (∑ j ∈ J, variance (inverseCellBranch u) (nu j)) +
CP * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) +
CB * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) :=
add_le_add (add_le_add le_rfl hp) hb
_ = _ := by ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.LightCellRisk 7 declarations Light-cell polynomial second moments and the overlap algebra for equation (5).+
Light-cell polynomial second moments and the overlap algebra for equation (5).+
Under the stated inputs and conditions, The independent success count factors the polynomial branch's second moment into its Poisson second moment and the factorial multiplier's second moment. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v)
Proof (Lean source)
-- @node: polynomial_cell_second_moment
lemma polynomial_cell_second_moment (L : Nat) (B u t q s v : Real)
(hu : 0 < u) (hq : 0 ≤ q) :
(∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v) =
(q / u + q ^ 2) * (∫ z : Nat × Nat,
(1 + (z.2 : Real) / (t * B) * (∑ h ∈ range (L - 1),
(chebG L).coeff h * (z.1.descFactorial h : Real) / (t * B) ^ h)) ^ 2 ∂
(poissonMeasure (toNNReal (t * s))).prod
(poissonMeasure (toNNReal (t * v)))) := by
let W : Nat × Nat → Real := fun z =>
1 + (z.2 : Real) / (t * B) * (∑ h ∈ range (L - 1),
(chebG L).coeff h * (z.1.descFactorial h : Real) / (t * B) ^ h)
change (∫ z : Nat × (Nat × Nat), ((z.1 : Real) / u * W z.2) ^ 2 ∂
(poissonMeasure (toNNReal (u * q))).prod
((poissonMeasure (toNNReal (t * s))).prod
(poissonMeasure (toNNReal (t * v))))) =
(q / u + q ^ 2) * _
simp only [mul_pow]
rw [integral_prod_mul (fun Z : Nat => ((Z : Real) / u) ^ 2)
(fun z : Nat × Nat => W z ^ 2)]
have hm := poisson_count_second_moment
(toNNReal (u * q))
simp only [Real.coe_toNNReal _ (mul_nonneg hu.le hq)] at hm
have hz : (∫ Z : Nat, ((Z : Real) / u) ^ 2 ∂
poissonMeasure (toNNReal (u * q))) = q / u + q ^ 2 := by
simp_rw [div_pow]
rw [integral_div, hm]
field_simp
ring
rw [hz]
simp only [W, mul_pow]
Under the stated inputs and conditions, The light-cell multiplier certificate controls the actual polynomial branch variance, retaining both the outcome-count fluctuation and its squared mean. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: light_polynomial_variance_envelope
lemma light_polynomial_variance_envelope :
∃ C : Real, 0 < C ∧ ∀ (L : Nat) (B u t q s v : Real),
4 ≤ L → 0 < B → 0 < u → 0 < t → 0 ≤ q → 0 ≤ s → s ≤ B →
0 ≤ v → (L : Real) ≤ t * B →
variance (polynomialCellBranch L B u t) (cellPoissonLaw u t q s v) ≤
C * ((2 : Real) ^ 24) ^ L * (q / u + q ^ 2) *
(1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) := by
obtain ⟨C, hC, hbound⟩ := light_multiplier_second_moment
refine ⟨C, hC, ?_⟩
intro L B u t q s v hL hB hu ht hq hs hsB hv hD
let : IsProbabilityMeasure (cellPoissonLaw u t q s v) := by
unfold cellPoissonLaw
infer_instance
calc
_ ≤ ∫ z, polynomialCellBranch L B u t z ^ 2 ∂cellPoissonLaw u t q s v :=
variance_le_expectation_sq (hybrid_cell_branches_memLp L B u t q s v).1.aestronglyMeasurable
_ = _ := polynomial_cell_second_moment L B u t q s v hu hq
_ ≤ _ := by
have hh := mul_le_mul_of_nonneg_left (hbound L B t s v hL hB ht hs hsB hv hD)
(show 0 ≤ q / u + q ^ 2 by positivity)
convert hh using 1 <;> first | rfl | ring
Under the stated inputs and conditions, Equation (4): light-cell bandwidth and overlap control every mixed monomial in the polynomial second-moment envelope, including null own-arm cells. This gives the stated result.
Formal statement
hypotheses
B t s v eps :
hB :
0 < B
ht :
0 < t
hs :
0 ≤ s
hsB :
s ≤ B
hv :
0 ≤ v
heps :
0 < eps
hov :
eps * (s + v) ≤ s
hD :
1 ≤ t * B
conclusion
conclusion 1
s * v ^ 2 / B ^ 2 ≤ (s + v) / eps
conclusion 2
s * v / (t * B ^ 2) ≤ v
conclusion 3
s ^ 2 * v / (t * B ^ 2) ≤ (s + v) / t
conclusion 4
s ^ 2 * v ^ 2 / B ^ 2 ≤ (s + v) ^ 2
conclusion 5
s ^ 2 ≤ (s + v) ^ 2
conclusion 6
s + v ≤ B / eps
Proof (Lean source)
-- @node: light_cell_overlap_monomials
lemma light_cell_overlap_monomials (B t s v eps : Real)
(hB : 0 < B) (ht : 0 < t) (hs : 0 ≤ s) (hsB : s ≤ B) (hv : 0 ≤ v)
(heps : 0 < eps) (hov : eps * (s + v) ≤ s) (hD : 1 ≤ t * B) :
s * v ^ 2 / B ^ 2 ≤ (s + v) / eps ∧
s * v / (t * B ^ 2) ≤ v ∧
s ^ 2 * v / (t * B ^ 2) ≤ (s + v) / t ∧
s ^ 2 * v ^ 2 / B ^ 2 ≤ (s + v) ^ 2 ∧
s ^ 2 ≤ (s + v) ^ 2 ∧ s + v ≤ B / eps := by
have hs2 : s ^ 2 ≤ B ^ 2 := pow_le_pow_left₀ hs hsB 2
have hvp : v ≤ s + v := by linarith
have hp : 0 ≤ s + v := by positivity
have hvs : eps * v ≤ s := by nlinarith
have hv2 : eps * v ^ 2 ≤ s * (s + v) := by
have hh := mul_le_mul hvs hvp hv (by positivity : 0 ≤ s)
nlinarith
have htB : B ≤ t * B ^ 2 := by
have hh := mul_le_mul_of_nonneg_right hD hB.le
nlinarith
refine ⟨?_, ?_, ?_, ?_, ?_, ?_⟩
· apply (div_le_div_iff₀ (sq_pos_of_pos hB) heps).2
have hh := mul_le_mul_of_nonneg_left hv2 hs
have hh' := mul_le_mul_of_nonneg_right hs2 hp
nlinarith
· apply (div_le_iff₀ (mul_pos ht (sq_pos_of_pos hB))).2
exact (mul_le_mul_of_nonneg_right hsB hv).trans
(by nlinarith [mul_le_mul_of_nonneg_right htB hv])
· apply (div_le_div_iff₀ (mul_pos ht (sq_pos_of_pos hB)) ht).2
have hh := mul_le_mul hs2 hvp hv (sq_nonneg B)
have hh' := mul_le_mul_of_nonneg_left hh ht.le
nlinarith
· apply (div_le_iff₀ (sq_pos_of_pos hB)).2
have hvp2 := pow_le_pow_left₀ hv hvp 2
have hh := mul_le_mul hs2 hvp2 (sq_nonneg v) (sq_nonneg B)
nlinarith only [hh]
· exact pow_le_pow_left₀ hs (by linarith) 2
· exact (le_div_iff₀ heps).2 (by nlinarith [hov.trans hsB])
Under the stated inputs and conditions, Expanding the light-cell envelope gives the mass-over-intensity and squared-bandwidth terms used in equation (5), without a minimum positive cell mass. This gives the stated result.
Formal statement
hypotheses
B u t q s v eps :
hB :
0 < B
hu :
0 < u
ht :
0 < t
hq :
0 ≤ q
hqs :
q ≤ s
hsB :
s ≤ B
hv :
0 ≤ v
heps :
0 < eps
heps1 :
eps ≤ 1
hov :
eps * (s + v) ≤ s
hD :
1 ≤ t * B
conclusion
(q / u + q ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2))
≤ 3 * (s + v) / (u * eps) + (s + v) / t + 2 * B ^ 2 / eps ^ 2
Proof (Lean source)
-- @node: light_cell_second_moment_algebra
lemma light_cell_second_moment_algebra (B u t q s v eps : Real)
(hB : 0 < B) (hu : 0 < u) (ht : 0 < t) (hq : 0 ≤ q) (hqs : q ≤ s)
(hsB : s ≤ B) (hv : 0 ≤ v) (heps : 0 < eps) (heps1 : eps ≤ 1)
(hov : eps * (s + v) ≤ s) (hD : 1 ≤ t * B) :
(q / u + q ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) ≤
3 * (s + v) / (u * eps) + (s + v) / t + 2 * B ^ 2 / eps ^ 2 := by
have hs : 0 ≤ s := hq.trans hqs
obtain ⟨h1, h2, h3, h4, h5, h6⟩ :=
light_cell_overlap_monomials B t s v eps hB ht hs hsB hv heps hov hD
have hp : 0 ≤ s + v := by positivity
have hse : s ≤ (s + v) / eps := by
apply (le_div_iff₀ heps).2
nlinarith
have hsq : (s + v) ^ 2 ≤ B ^ 2 / eps ^ 2 := by
have hh := pow_le_pow_left₀ hp h6 2
simpa only [div_pow] using hh
have hqs2 := pow_le_pow_left₀ hq hqs 2
calc
_ ≤ (s / u + s ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) :=
mul_le_mul_of_nonneg_right
(add_le_add (div_le_div_of_nonneg_right hqs hu.le) hqs2) (by positivity)
_ = s / u + (s * v ^ 2 / B ^ 2) / u +
(s * v / (t * B ^ 2)) / u + s ^ 2 +
s ^ 2 * v ^ 2 / B ^ 2 + s ^ 2 * v / (t * B ^ 2) := by ring
_ ≤ (s + v) / eps / u + (s + v) / eps / u +
(s + v) / eps / u + B ^ 2 / eps ^ 2 +
B ^ 2 / eps ^ 2 + (s + v) / t := by
have hvse : v ≤ (s + v) / eps := by
apply (le_div_iff₀ heps).2
nlinarith
exact add_le_add (add_le_add (add_le_add (add_le_add (add_le_add
(div_le_div_of_nonneg_right hse hu.le)
(div_le_div_of_nonneg_right h1 hu.le))
(div_le_div_of_nonneg_right (h2.trans hvse) hu.le))
(h5.trans hsq)) (h4.trans hsq)) h3
_ = _ := by ring
Under the stated inputs and conditions, A light polynomial cell has variance controlled by its mass over the outcome and factorial intensities, plus squared bandwidth over squared overlap. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: light_polynomial_cell_variance
lemma light_polynomial_cell_variance :
∃ C : Real, 0 < C ∧ ∀ (L : Nat) (B u t q s v eps : Real),
4 ≤ L → 0 < B → 0 < u → 0 < t → 0 ≤ q → q ≤ s → s ≤ B →
0 ≤ v → 0 < eps → eps ≤ 1 → eps * (s + v) ≤ s → (L : Real) ≤ t * B →
variance (polynomialCellBranch L B u t) (cellPoissonLaw u t q s v) ≤
C * ((2 : Real) ^ 24) ^ L *
((s + v) / (u * eps) + (s + v) / t + B ^ 2 / eps ^ 2) := by
obtain ⟨C, hC, hb⟩ := light_polynomial_variance_envelope
refine ⟨3 * C, by positivity, ?_⟩
intro L B u t q s v eps hL hB hu ht hq hqs hsB hv heps heps1 hov hD
have hs := hq.trans hqs
have hD1 : 1 ≤ t * B := by
have hLreal : (4 : Real) ≤ L := by exact_mod_cast hL
linarith
have ha := light_cell_second_moment_algebra B u t q s v eps
hB hu ht hq hqs hsB hv heps heps1 hov hD1
have hF : 0 ≤ C * ((2 : Real) ^ 24) ^ L := by positivity
calc
_ ≤ C * ((2 : Real) ^ 24) ^ L * (q / u + q ^ 2) *
(1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) :=
hb L B u t q s v hL hB hu ht hq hs hsB hv hD
_ = C * ((2 : Real) ^ 24) ^ L *
((q / u + q ^ 2) * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2))) := by ring
_ ≤ C * ((2 : Real) ^ 24) ^ L *
(3 * (s + v) / (u * eps) + (s + v) / t + 2 * B ^ 2 / eps ^ 2) :=
mul_le_mul_of_nonneg_left ha hF
_ ≤ _ := by
have ht0 : 0 ≤ (s + v) / t := by positivity
have hB0 : 0 ≤ B ^ 2 / eps ^ 2 := by positivity
have hh : 3 * (s + v) / (u * eps) + (s + v) / t + 2 * B ^ 2 / eps ^ 2 ≤
3 * ((s + v) / (u * eps) + (s + v) / t + B ^ 2 / eps ^ 2) := by
calc
_ = 3 * ((s + v) / (u * eps)) + (s + v) / t +
2 * (B ^ 2 / eps ^ 2) := by ring
_ ≤ _ := by linarith
have hh' := mul_le_mul_of_nonneg_left hh hF
convert hh' using 1 <;> first | rfl | ring
Under the stated inputs and conditions, Equation (5): summing the actual light-cell polynomial variances keeps the total light mass in both intensity terms and the cell count in the bandwidth term. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: light_polynomial_variance_sum
lemma light_polynomial_variance_sum :
∃ C : Real, 0 < C ∧ ∀ (L : Nat) (B u t eps : Real),
4 ≤ L → 0 < B → 0 < u → 0 < t → 0 < eps → eps ≤ 1 →
(L : Real) ≤ t * B → ∀ {alpha : Type} (J : Finset alpha) (q s v : alpha → Real),
(∀ j ∈ J, 0 ≤ q j ∧ q j ≤ s j ∧ s j ≤ B ∧ 0 ≤ v j ∧
eps * (s j + v j) ≤ s j) →
(∑ j ∈ J, variance (polynomialCellBranch L B u t)
(cellPoissonLaw u t (q j) (s j) (v j))) ≤
C * ((2 : Real) ^ 24) ^ L *
((J.sum (fun j : alpha => s j + v j)) / (u * eps) +
(J.sum (fun j : alpha => s j + v j)) / t + (J.card : Real) * B ^ 2 / eps ^ 2) := by
obtain ⟨C, hC, hb⟩ := light_polynomial_cell_variance
refine ⟨C, hC, ?_⟩
intro L B u t eps hL hB hu ht heps heps1 hD alpha J q s v hcell
calc
_ ≤ ∑ j ∈ J, C * ((2 : Real) ^ 24) ^ L *
((s j + v j) / (u * eps) + (s j + v j) / t + B ^ 2 / eps ^ 2) := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hq, hqs, hsB, hv, hov⟩ := hcell j hj
exact hb L B u t (q j) (s j) (v j) eps
hL hB hu ht hq hqs hsB hv heps heps1 hov hD
_ = _ := by
simp only [← Finset.mul_sum, Finset.sum_add_distrib, ← Finset.sum_div,
Finset.sum_const, nsmul_eq_mul]
Under the stated hypotheses, The light-cell mass is at most one and at most the number of light cells times bandwidth over overlap, as used in equation (5). This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: light_cell_mass_bound
lemma light_cell_mass_bound {alpha : Type} (J : Finset alpha) (s v : alpha → Real)
(B eps : Real) (heps : 0 < eps)
(hcell : ∀ j ∈ J, eps * (s j + v j) ≤ s j ∧ s j ≤ B)
(hmass : (J.sum (fun j : alpha => s j + v j)) ≤ 1) :
(J.sum (fun j : alpha => s j + v j)) ≤ min 1 ((J.card : Real) * B / eps) := by
apply le_min hmass
calc
_ ≤ ∑ j ∈ J, B / eps := by
apply Finset.sum_le_sum
intro j hj
obtain ⟨hov, hsB⟩ := hcell j hj
apply (le_div_iff₀ heps).2
nlinarith
_ = _ := by simp [mul_div_assoc]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.LightVariance 2 declarations Factorial branch light-cell moments, upstream of the aggregate variance bound.
Factorial branch light-cell moments, upstream of the aggregate variance bound.
Under the stated inputs and conditions, Every finite normalized falling-factorial polynomial is square-integrable under a Poisson law, so its second-moment estimates can be integrated. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: poisson_factorial_polynomial_memLp
lemma poisson_factorial_polynomial_memLp (rate : NNReal) (J : Nat)
(c : Nat → Real) (D : Real) :
MemLp (fun K : Nat => ∑ h ∈ range J,
c h * (K.descFactorial h : Real) / D ^ h) 2 (poissonMeasure rate) := by
apply memLp_finsetSum
intro h _
have hh : MemLp (fun K : Nat => (K.descFactorial h : Real)) 2
(poissonMeasure rate) := by
apply (memLp_two_iff_integrable_sq (by fun_prop)).2
have hi := integrable_finsetSum (range (min h h + 1))
(fun r _ => (poisson_descFactorial_integrable
rate (h + h - r)).const_mul
((h.choose r : Real) * (h.choose r : Real) * (r.factorial : Real)))
exact hi.congr (Filter.Eventually.of_forall (fun K => by
simpa only [pow_two] using
(descFactorial_mul K h h).symm))
simpa only [div_eq_mul_inv] using (hh.const_mul (c h)).mul_const ((D ^ h)⁻¹)
Under the stated inputs and conditions, The factorial multiplier has a universal light-cell squared-moment bound. This gives the stated conclusion.
Formal statement
Proof (Lean source)
-- @node: light_multiplier_second_moment
lemma light_multiplier_second_moment :
∃ C : Real, 0 < C ∧ ∀ (L : Nat) (B t s v : Real),
4 ≤ L → 0 < B → 0 < t → 0 ≤ s → s ≤ B → 0 ≤ v → (L : Real) ≤ t * B →
(∫ z : Nat × Nat,
(1 + (z.2 : Real) / (t * B) * (∑ h ∈ range (L - 1),
(chebG L).coeff h * (z.1.descFactorial h : Real) / (t * B) ^ h)) ^ 2 ∂
(poissonMeasure (toNNReal (t * s))).prod (poissonMeasure (toNNReal (t * v)))) ≤
C * ((2 : Real) ^ 24) ^ L * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) := by
refine ⟨2, by norm_num, ?_⟩
intro L B t s v hL hB ht hs hsB hv hD
let G : Nat → Real := fun K => ∑ h ∈ range (L - 1),
(chebG L).coeff h * (K.descFactorial h : Real) / (t * B) ^ h
let mu := poissonMeasure (toNNReal (t * s))
let nu := poissonMeasure (toNNReal (t * v))
have hG : MemLp G 2 mu := poisson_factorial_polynomial_memLp _ _ _ _
have hW := poisson_natCast_memLp_two
(toNNReal (t * v))
have hprod : Integrable (fun z : Nat × Nat => G z.1 ^ 2 *
((z.2 : Real) / (t * B)) ^ 2) (mu.prod nu) :=
hG.integrable_sq.mul_prod (by
simpa only [div_eq_mul_inv] using (hW.mul_const ((t * B)⁻¹)).integrable_sq)
have hbound : (∫ z : Nat × Nat, (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2
∂mu.prod nu) ≤ 2 + 2 * ((∫ K, G K ^ 2 ∂mu) *
(∫ W : Nat, ((W : Real) / (t * B)) ^ 2 ∂nu)) := by
calc
_ ≤ ∫ z : Nat × Nat, 2 + 2 * (G z.1 ^ 2 * ((z.2 : Real) / (t * B)) ^ 2)
∂mu.prod nu := by
apply integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
((integrable_const 2).add (hprod.const_mul 2))
exact Filter.Eventually.of_forall (fun z => by
have hh : (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2 ≤
2 + 2 * ((z.2 : Real) / (t * B) * G z.1) ^ 2 := by
nlinarith only [sq_nonneg (1 - (z.2 : Real) / (t * B) * G z.1)]
change (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2 ≤
2 + 2 * (G z.1 ^ 2 * ((z.2 : Real) / (t * B)) ^ 2)
calc
_ ≤ 2 + 2 * ((z.2 : Real) / (t * B) * G z.1) ^ 2 := hh
_ = _ := by ring)
_ = _ := by
rw [integral_add (integrable_const 2) (hprod.const_mul 2), integral_const_mul,
integral_prod_mul (fun K => G K ^ 2) (fun W : Nat => ((W : Real) / (t * B)) ^ 2)]
simp
have hsratio : 0 ≤ s / B := div_nonneg hs hB.le
have hratio : s / B ≤ 1 := (div_le_one hB).2 hsB
have hcert := (chebyshev_factorial_certificate L (by omega)).2.2.2.2.2
t B (s / B) ht hB hsratio hD
have hrate : t * (s / B) * B = t * s := by field_simp
have hGs : (∫ K, G K ^ 2 ∂mu) ≤ ((2 : Real) ^ 24) ^ L := by
have hg := hcert.2
rw [hrate] at hg
have hp : (1 + s / B) ^ (2 * L) ≤ (2 : Real) ^ (2 * L) :=
pow_le_pow_left₀ (by positivity) (by linarith) _
calc
_ ≤ 392 ^ L * (1 + s / B) ^ (2 * L) := hg
_ ≤ 392 ^ L * (2 : Real) ^ (2 * L) := mul_le_mul_of_nonneg_left hp (by positivity)
_ = (1568 : Real) ^ L := by rw [pow_mul, ← mul_pow]; norm_num
_ ≤ ((2 : Real) ^ 24) ^ L := pow_le_pow_left₀ (by norm_num) (by norm_num) _
have hWmoment : (∫ W : Nat, ((W : Real) / (t * B)) ^ 2 ∂nu) =
v ^ 2 / B ^ 2 + v / (t * B ^ 2) := by
have hm := poisson_descFactorial_mixed
(toNNReal (t * v)) 1 1
norm_num [Finset.sum_range_succ,
Real.toNNReal_of_nonneg (mul_nonneg ht.le hv)] at hm
simp_rw [div_pow]
rw [integral_div]
simp only [nu]
simp_rw [pow_two]
simp only [Real.toNNReal_of_nonneg (mul_nonneg ht.le hv)]
rw [hm]
field_simp
have hF : 1 ≤ ((2 : Real) ^ 24) ^ L := one_le_pow₀ (by norm_num)
have hV : 0 ≤ v ^ 2 / B ^ 2 + v / (t * B ^ 2) := by positivity
change (∫ z : Nat × Nat, (1 + (z.2 : Real) / (t * B) * G z.1) ^ 2
∂mu.prod nu) ≤ _
rw [hWmoment] at hbound
calc
_ ≤ 2 + 2 * ((∫ K, G K ^ 2 ∂mu) * (v ^ 2 / B ^ 2 + v / (t * B ^ 2))) := hbound
_ ≤ 2 * ((2 : Real) ^ 24) ^ L * (1 + v ^ 2 / B ^ 2 + v / (t * B ^ 2)) := by
have hh := mul_le_mul_of_nonneg_right hGs hV
calc
_ ≤ 2 * ((2 : Real) ^ 24) ^ L +
2 * (((2 : Real) ^ 24) ^ L * (v ^ 2 / B ^ 2 + v / (t * B ^ 2))) :=
add_le_add (by linarith only [hF]) (mul_le_mul_of_nonneg_left hh (by norm_num))
_ = _ := by ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.MixtureVariance 8 declarations Variance algebra for pilot-controlled branch mixtures, without branch independence.
Variance algebra for pilot-controlled branch mixtures, without branch independence.
Under the stated inputs and conditions, The Bernoulli branch-mixture variance expression admits a deterministic upper bound.
Formal statement
Proof (Lean source)
-- @node: branch_mixture_variance_le
lemma branch_mixture_variance_le (pi VP VH mp mh : Real)
(hpi : pi ∈ Icc 0 1) (hVP : 0 ≤ VP) (hVH : 0 ≤ VH) :
pi * VP + (1 - pi) * VH + pi * (1 - pi) * (mp - mh) ^ 2 ≤ VP + VH + (mp - mh) ^ 2 := by
have hmix : pi * (1 - pi) ≤ 1 := by
nlinarith [sq_nonneg pi]
have hpol : pi * VP ≤ VP :=
mul_le_of_le_one_left hVP hpi.2
have hheavy : (1 - pi) * VH ≤ VH :=
mul_le_of_le_one_left hVH (by linarith [hpi.1])
have hbetween : pi * (1 - pi) * (mp - mh) ^ 2 ≤ (mp - mh) ^ 2 :=
mul_le_of_le_one_left (sq_nonneg _) hmix
exact add_le_add (add_le_add hpol hheavy) hbetween
Under the stated inputs and conditions, Both canonical branches are square-integrable, including null cells and zero intensities. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
conclusion 2
Proof (Lean source)
-- @node: hybrid_cell_branches_memLp
lemma hybrid_cell_branches_memLp (L : Nat) (B u t q s v : Real) :
MemLp (polynomialCellBranch L B u t) 2 (cellPoissonLaw u t q s v) ∧
MemLp (inverseCellBranch u) 2 (cellPoissonLaw u t q s v) := by
let muZ := poissonMeasure (toNNReal (u * q))
let muK := poissonMeasure (toNNReal (t * s))
let muW := poissonMeasure (toNNReal (t * v))
have hZ : MemLp (fun Z : Nat => (Z : Real) / u) 2 muZ := by
simpa only [div_eq_mul_inv] using
(poisson_natCast_memLp_two _).mul_const u⁻¹
have hW : MemLp (fun W : Nat => (W : Real)) 2 muW :=
poisson_natCast_memLp_two _
have hG := poisson_factorial_polynomial_memLp (toNNReal (t * s))
(L - 1) (fun h => (chebG L).coeff h) (t * B)
have hrec : MemLp (fun K : Nat => ((K : Real) + 1)⁻¹) 2 muK := by
simpa only [Nat.cast_add, Nat.cast_one] using
shifted_reciprocal_memLp_two
(toNNReal (t * s))
have hpol := memLp_mul_prod_two
(hG.mul_const ((t * B)⁻¹)) hW
have hheavy := memLp_mul_prod_two hrec hW
have hpol' := (memLp_const (μ := muK.prod muW) (1 : Real)).add hpol
have hheavy' := (memLp_const (μ := muK.prod muW) (1 : Real)).add hheavy
constructor
· have hh := memLp_mul_prod_two hZ hpol'
convert hh using 1 <;> try rfl
funext z
simp only [polynomialCellBranch, Pi.add_apply, div_eq_mul_inv]
ring
· have hh := memLp_mul_prod_two hZ hheavy'
convert hh using 1 <;> try rfl
funext z
simp only [inverseCellBranch, Pi.add_apply, div_eq_mul_inv]
ring
The implemented hybrid cell is exactly the pilot-event mixture of the two branches.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_cell_event_mixture
lemma hybrid_cell_event_mixture (L k0 : Nat) (B u t : Real) :
(fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) =
eventSelectedMixture (Iic k0)
(polynomialCellBranch L B u t) (inverseCellBranch u) := by
funext z
by_cases hz : z.1 ≤ k0
· simp [eventSelectedMixture, hybridCellValue,
polynomialCellBranch, hz, Set.mem_Iic, div_eq_mul_inv, mul_assoc, mul_comm]
· simp [eventSelectedMixture, hybridCellValue,
inverseCellBranch, hz, Set.mem_Iic]
The actual pilot-selected cell has finite second moment under the independent pool law.
Formal statement
hypotheses
conclusion
MemLp (fun z : Nat × (Nat × Nat × Nat) => hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) 2 ((poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t q s v))
Proof (Lean source)
-- @node: hybrid_pilot_cell_memLp
lemma hybrid_pilot_cell_memLp (L k0 : Nat) (B u tp t q s v : Real) :
MemLp (fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) 2
((poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t q s v)) := by
let : IsProbabilityMeasure (cellPoissonLaw u t q s v) := by
unfold cellPoissonLaw
infer_instance
rw [hybrid_cell_event_mixture]
exact memLp_eventSelectedMixture _ _ _
measurableSet_Iic _ _ (hybrid_cell_branches_memLp L B u t q s v).1
(hybrid_cell_branches_memLp L B u t q s v).2
Equation (13): the independent pilot mixes branch variances and their mean separation; the polynomial and heavy branches may share all their evaluation counts.
Formal statement
hypotheses
conclusion
pi :
nu :
cellPoissonLaw u t q s v
variance (fun z : Nat × (Nat × Nat × Nat) => hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2) ((poissonMeasure (toNNReal (tp * s))).prod nu)
= pi * variance (polynomialCellBranch L B u t) nu
+ (1 - pi) * variance (inverseCellBranch u) nu
+ pi * (1 - pi) * ((∫ z, polynomialCellBranch L B u t z ∂nu) - ∫ z, inverseCellBranch u z ∂nu) ^ 2
Proof (Lean source)
-- @node: hybrid_pilot_cell_variance
lemma hybrid_pilot_cell_variance (L k0 : Nat) (B u tp t q s v : Real) :
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic k0)
let nu := cellPoissonLaw u t q s v
variance (fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2)
((poissonMeasure (toNNReal (tp * s))).prod nu) =
pi * variance (polynomialCellBranch L B u t) nu +
(1 - pi) * variance (inverseCellBranch u) nu + pi * (1 - pi) *
((∫ z, polynomialCellBranch L B u t z ∂nu) -
∫ z, inverseCellBranch u z ∂nu) ^ 2 := by
let : IsProbabilityMeasure (cellPoissonLaw u t q s v) := by
unfold cellPoissonLaw
infer_instance
dsimp only
rw [hybrid_cell_event_mixture]
exact variance_eventSelectedMixture _ _ _
measurableSet_Iic _ _ (hybrid_cell_branches_memLp L B u t q s v).1
(hybrid_cell_branches_memLp L B u t q s v).2
Under the stated inputs and conditions, The hybrid's bias is the pilot-weighted sum of the two explicit branch biases. This gives the stated result.
Formal statement
hypotheses
L k0 :
B u tp t s v mu :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
hs :
0 < s
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
conclusion
pi :
(∫ z : Nat × (Nat × Nat × Nat), hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂ (poissonMeasure (toNNReal (tp * s))).prod (cellPoissonLaw u t (s * mu) s v))
Proof (Lean source)
-- @node: hybrid_pilot_cell_mean
lemma hybrid_pilot_cell_mean (L k0 : Nat) (B u tp t s v mu : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t)
(hs : 0 < s) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1) :
let pi := (poissonMeasure (toNNReal (tp * s))).real (Iic k0)
(∫ z : Nat × (Nat × Nat × Nat),
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2 ∂
(poissonMeasure (toNNReal (tp * s))).prod
(cellPoissonLaw u t (s * mu) s v)) =
(s + v) * mu - v * mu *
(pi * (chebE L).eval (s / B) + (1 - pi) * exp (-t * s)) := by
let : IsProbabilityMeasure (cellPoissonLaw u t (s * mu) s v) := by
unfold cellPoissonLaw
infer_instance
dsimp only
rw [hybrid_cell_event_mixture]
have hLp := hybrid_cell_branches_memLp L B u t (s * mu) s v
rw [integral_eventSelectedMixture _ _ _
measurableSet_Iic _ _ (hLp.1.integrable (by norm_num)) (hLp.2.integrable (by norm_num))]
obtain ⟨hp, hh⟩ := hybrid_branch_means L B u t s v mu hL hB hu ht hs hv hmu
rw [hp, hh]
ring
Under the stated inputs and conditions, Equation (2): overlap turns the light-cell approximation error into the public bandwidth divided by epsilon and squared degree. This gives the stated result.
Formal statement
hypotheses
L :
B u t s v mu eps :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
hs :
0 < s
hsB :
s ≤ B
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
heps :
0 < eps
hov :
eps * (s + v) ≤ s
conclusion
|(∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) - (s + v) * mu|
≤ B / (eps * (L : Real) ^ 2)
Proof (Lean source)
-- @node: hybrid_light_polynomial_bias
lemma hybrid_light_polynomial_bias (L : Nat) (B u t s v mu eps : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t)
(hs : 0 < s) (hsB : s ≤ B) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1)
(heps : 0 < eps) (hov : eps * (s + v) ≤ s) :
|(∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) -
(s + v) * mu| ≤ B / (eps * (L : Real) ^ 2) := by
have hLpos : 0 < (L : Real) := by exact_mod_cast (show 0 < L by omega)
obtain ⟨hE0, hE⟩ := (chebyshev_factorial_certificate L hL).1
(s / B) (div_pos hs hB) ((div_le_one hB).2 hsB)
have hEB : (chebE L).eval (s / B) ≤ B / ((L : Real) ^ 2 * s) := by
calc
_ ≤ ((L : Real) ^ 2 * (s / B))⁻¹ := hE.trans (min_le_right _ _)
_ = _ := by field_simp
rw [(hybrid_branch_means L B u t s v mu hL hB hu ht hs hv hmu).1]
rw [sub_sub_cancel_left, abs_neg, abs_of_nonneg (mul_nonneg (mul_nonneg hv hmu.1) hE0)]
have hvmu : v * mu ≤ v := mul_le_of_le_one_right hv hmu.2
calc
_ ≤ v * (B / ((L : Real) ^ 2 * s)) := mul_le_mul hvmu hEB hE0 hv
_ ≤ B / (eps * (L : Real) ^ 2) := by
apply (le_div_iff₀ (mul_pos heps (sq_pos_of_pos hLpos))).2
have hvs : eps * v ≤ s := by nlinarith
have hh := mul_le_mul_of_nonneg_right hvs (div_nonneg hB.le hs.le)
have heq : v * (B / ((L : Real) ^ 2 * s)) * (eps * (L : Real) ^ 2) =
eps * v * (B / s) := by field_simp
rw [heq]
simpa only [mul_div_cancel₀ B hs.ne'] using hh
Under the stated inputs and conditions, On a light cell the between-branch term of equation (13) is bounded by twice the squared polynomial bias plus twice the squared inverse-count bias. This gives the stated result.
Formal statement
hypotheses
L :
B u t s v mu eps pi :
hL :
2 ≤ L
hB :
0 < B
hu :
0 < u
ht :
0 < t
hs :
0 < s
hsB :
s ≤ B
hv :
0 ≤ v
hmu :
mu ∈ Icc 0 1
heps :
0 < eps
hov :
eps * (s + v) ≤ s
conclusion
pi * (1 - pi) * ((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) - ∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) ^ 2
Proof (Lean source)
-- @node: hybrid_light_between_branch_bound
lemma hybrid_light_between_branch_bound (L : Nat) (B u t s v mu eps pi : Real)
(hL : 2 ≤ L) (hB : 0 < B) (hu : 0 < u) (ht : 0 < t)
(hs : 0 < s) (hsB : s ≤ B) (hv : 0 ≤ v) (hmu : mu ∈ Icc 0 1)
(heps : 0 < eps) (hov : eps * (s + v) ≤ s) :
pi * (1 - pi) *
((∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v) -
∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v) ^ 2 ≤
2 * (B / (eps * (L : Real) ^ 2)) ^ 2 +
2 * ((s + v) * exp (-t * s)) ^ 2 := by
let mp := ∫ z, polynomialCellBranch L B u t z ∂cellPoissonLaw u t (s * mu) s v
let mh := ∫ z, inverseCellBranch u z ∂cellPoissonLaw u t (s * mu) s v
let theta := (s + v) * mu
have hp := hybrid_light_polynomial_bias L B u t s v mu eps
hL hB hu ht hs hsB hv hmu heps hov
have hp2 : (mp - theta) ^ 2 ≤ (B / (eps * (L : Real) ^ 2)) ^ 2 := by
simpa only [sq_abs] using pow_le_pow_left₀ (abs_nonneg _) hp 2
have hh := (hybrid_branch_means L B u t s v mu hL hB hu ht hs hv hmu).2
have hheavy : |mh - theta| ≤ (s + v) * exp (-t * s) := by
dsimp [mh, theta]
rw [hh, sub_sub_cancel_left, abs_neg,
abs_of_nonneg (mul_nonneg (mul_nonneg hv hmu.1) (Real.exp_nonneg _))]
exact mul_le_mul_of_nonneg_right
((mul_le_of_le_one_right hv hmu.2).trans (by linarith)) (Real.exp_nonneg _)
have hh2 : (mh - theta) ^ 2 ≤ ((s + v) * exp (-t * s)) ^ 2 := by
simpa only [sq_abs] using pow_le_pow_left₀ (abs_nonneg _) hheavy 2
have hmix : pi * (1 - pi) ≤ 1 := by nlinarith [sq_nonneg pi]
calc
_ ≤ (mp - mh) ^ 2 := mul_le_of_le_one_left (sq_nonneg _) hmix
_ ≤ 2 * (mp - theta) ^ 2 + 2 * (mh - theta) ^ 2 := by
nlinarith only [sq_nonneg ((mp - theta) + (mh - theta))]
_ ≤ _ := by linarith
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.PilotTails 6 declarations Poisson pilot misclassification probabilities at the public bandwidth cutoff.
Poisson pilot misclassification probabilities at the public bandwidth cutoff.
Under the stated inputs and conditions, A heavy cell is selected as light with exponentially small probability.
Formal statement
Proof (Lean source)
-- @node: pilot_false_light_tail
lemma pilot_false_light_tail (tp B s : Real) (htp : 0 < tp) (hB : 0 < B) (hs : B < s) :
(poissonMeasure (toNNReal (tp * s))).real (Iic (floor (tp * B / 4))) ≤
exp (-tp * s / 4) := by
have hspos : 0 < s := hB.trans hs
have hmean : 0 ≤ tp * s := (mul_pos htp hspos).le
have hcut : (floor (tp * B / 4) : Real) <
(toNNReal (tp * s) : Real) / 4 := by
rw [Real.coe_toNNReal _ hmean]
have hfloor := Nat.floor_le (show 0 ≤ tp * B / 4 by positivity)
have hmul := mul_lt_mul_of_pos_left hs htp
linarith
have htail := poisson_le_cutoff_of_cutoff_lt_quarter
(toNNReal (tp * s)) (floor (tp * B / 4)) hcut
have hreal := ENNReal.toReal_mono ENNReal.ofReal_ne_top htail
simpa only [Measure.real, Iic, Real.coe_toNNReal _ hmean,
ENNReal.toReal_ofReal (Real.exp_nonneg _), neg_mul] using hreal
Under the stated inputs and conditions, The heavy-branch probability times its inverse-count bias has a cutoff-only bound, including at zero cell mass. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: pilot_false_heavy_weighted_tail
lemma pilot_false_heavy_weighted_tail (tp t s : Real) (k : Nat)
(htp : 0 < tp) (ht : 0 ≤ t) (hs : 0 ≤ s) :
(poissonMeasure (toNNReal (tp * s))).real (Ioi k) *
exp (-t * s) ≤ exp (-log (1 + t / tp) * (k : Real)) := by
have h := poisson_upper_tail_mul_exp_le
(toNNReal (tp * s)) k (r := t / tp) (by positivity)
have heq : -(t / tp) * (toNNReal (tp * s) : Real) = -t * s := by
rw [Real.coe_toNNReal _ (mul_nonneg htp.le hs)]
field_simp
simpa only [heq, Ioi] using h
Under the stated inputs and conditions, The public bandwidth makes the weighted heavy-branch error exponentially small in the degree, uniformly over every nonnegative cell mass. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: pilot_false_heavy_absorption
lemma pilot_false_heavy_absorption (L : Nat) (tp t B s : Real)
(hL : 4 ≤ L) (htp : 0 < tp) (ht : 0 ≤ t) (hs : 0 ≤ s)
(hratio : 1 / 3 ≤ t / tp) (hscale : (2 : Real) ^ 20 * L ≤ tp * B) :
(poissonMeasure (toNNReal (tp * s))).real
(Ioi (floor (tp * B / 4))) * exp (-t * s) ≤
exp (-30000 * (L : Real)) := by
have hLreal : (4 : Real) ≤ L := by exact_mod_cast hL
have hfloor := Nat.lt_floor_add_one (tp * B / 4)
have hk : (2 : Real) ^ 20 * L / 8 ≤ (floor (tp * B / 4) : Real) := by
norm_num at hscale ⊢
linarith
have hlog : (2 / 7 : Real) ≤ log (1 + t / tp) := by
have hbase := Real.le_log_one_add_of_nonneg (by norm_num : (0 : Real) ≤ 1 / 3)
norm_num at hbase
exact hbase.trans (Real.log_le_log (by norm_num) (by linarith))
calc
_ ≤ exp (-log (1 + t / tp) * (floor (tp * B / 4) : Real)) :=
pilot_false_heavy_weighted_tail tp t s _ htp ht hs
_ ≤ exp (-30000 * (L : Real)) := by
apply Real.exp_le_exp.mpr
have hmul := mul_le_mul_of_nonneg_right hlog
(Nat.cast_nonneg (floor (tp * B / 4)) : (0 : Real) ≤ _)
norm_num at hk
nlinarith
Under the stated inputs and conditions, Degree calibration turns the opposite pilot error into the required inverse twentieth power of the label scale. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: pilot_false_heavy_public_scale
lemma pilot_false_heavy_public_scale (S tp t B s : Real)
(hS : exp 4096 ≤ S) (htp : 0 < tp) (ht : 0 ≤ t) (hs : 0 ≤ s)
(hratio : 1 / 3 ≤ t / tp)
(hscale : (2 : Real) ^ 20 * floor (log S / 1024) ≤ tp * B) :
(poissonMeasure (toNNReal (tp * s))).real
(Ioi (floor (tp * B / 4))) * exp (-t * s) ≤ (S ^ 20)⁻¹ := by
let L := floor (log S / 1024)
have hcal := hybrid_degree_calibration S hS
have hSpos : 0 < S := lt_of_lt_of_le (Real.exp_pos _) hS
have hL : 4 ≤ L := hcal.1
have hy : log S < 1280 * (L : Real) := hcal.2.1
calc
_ ≤ exp (-30000 * (L : Real)) :=
pilot_false_heavy_absorption L tp t B s hL htp ht hs hratio hscale
_ ≤ (S ^ 20)⁻¹ := by
apply (Real.log_le_log_iff (Real.exp_pos _) (by positivity)).1
rw [Real.log_exp, Real.log_inv, Real.log_pow]
have hLnonneg : (0 : Real) ≤ L := Nat.cast_nonneg L
norm_num
linarith
Under the stated inputs and conditions, The estimator's actual minimum-pool bandwidth satisfies the pilot scale condition. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_bandwidth_pilot_scale
lemma hybrid_bandwidth_pilot_scale (L : Nat) (tp t : Real)
(htp : 0 < tp) (ht : 0 < t) :
(2 : Real) ^ 20 * L ≤ tp * ((2 : Real) ^ 20 * L / min tp t) := by
have hmin : 0 < min tp t := lt_min htp ht
have hratio : 1 ≤ tp / min tp t := (one_le_div hmin).2 (min_le_left _ _)
have hmul := mul_le_mul_of_nonneg_left hratio
(by positivity : 0 ≤ (2 : Real) ^ 20 * L)
calc
_ ≤ (2 : Real) ^ 20 * L * (tp / min tp t) := by simpa using hmul
_ = _ := by ring
Under the stated inputs and conditions, With the prescribed bandwidth and pilot cutoff, the heavy selection probability times the inverse-count bias is at most the inverse twentieth power of S. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_false_heavy_bias_bound
lemma hybrid_false_heavy_bias_bound (S tp t s : Real)
(hS : exp 4096 ≤ S) (htp : 0 < tp) (ht : 0 < t)
(hs : 0 ≤ s) (hratio : 1 / 3 ≤ t / tp) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
(1 - (poissonMeasure (toNNReal (tp * s))).real (Iic k0)) *
exp (-t * s) ≤ (S ^ 20)⁻¹ := by
dsimp only
have hcompl (k : Nat) :
1 - (poissonMeasure (toNNReal (tp * s))).real (Iic k) =
(poissonMeasure (toNNReal (tp * s))).real (Ioi k) := by
rw [← Set.compl_Iic, measureReal_compl (measurableSet_Iic)]
simp
rw [hcompl]
exact pilot_false_heavy_public_scale S tp t _ s hS htp ht.le hs hratio
(hybrid_bandwidth_pilot_scale _ tp t htp ht)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.Hybrid.RateAbsorption 8 declarations Bandwidth rescaling and absorption of the intermediate hybrid variance terms in roadmap equations (14)--(15).
Bandwidth rescaling and absorption of the intermediate hybrid variance terms in roadmap equations (14)--(15). All degree calibration uses S = n epsilon.
Under the stated inputs and conditions, The lower bounds on both factorial intensities give the public bandwidth bound. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_bandwidth_public_bound
lemma hybrid_bandwidth_public_bound (N tp t : Real) (L : Nat)
(hN : 0 < N) (htp : N / 64 ≤ tp) (ht : N / 64 ≤ t) :
(2 : Real) ^ 20 * L / min tp t ≤ (64 * (2 : Real) ^ 20) * L / N := by
have hm : N / 64 ≤ min tp t := le_min htp ht
have hmpos : 0 < min tp t := lt_of_lt_of_le (by positivity) hm
calc
_ ≤ (2 : Real) ^ 20 * L / (N / 64) :=
div_le_div_of_nonneg_left (by positivity) (by positivity) hm
_ = _ := by ring
Under the stated inputs and conditions, Equation (15): a bandwidth bound controls the light mass, the variance bandwidth, and the squared total bias at the common public scale d / (N epsilon L). This gives the stated result.
Formal statement
hypotheses
A B d N eps L M :
_hA :
0 ≤ A
hB :
0 ≤ B
hd :
0 ≤ d
hN :
0 < N
heps :
0 < eps
hL :
0 < L
hband :
B ≤ A * L / N
hmass :
M ≤ d * B / eps
conclusion
conclusion 1
M ≤ A * (d / (N * eps * L)) * L ^ 2
conclusion 2
d * B ^ 2 / eps ^ 2 ≤ A ^ 2 * (d / (N * eps * L)) * L ^ 3 / (N * eps)
conclusion 3
d ^ 2 * B ^ 2 / (eps ^ 2 * L ^ 4) ≤ A ^ 2 * (d / (N * eps * L)) ^ 2
Proof (Lean source)
-- @node: hybrid_bandwidth_rate_monomials
lemma hybrid_bandwidth_rate_monomials (A B d N eps L M : Real)
(_hA : 0 ≤ A) (hB : 0 ≤ B) (hd : 0 ≤ d) (hN : 0 < N)
(heps : 0 < eps) (hL : 0 < L) (hband : B ≤ A * L / N)
(hmass : M ≤ d * B / eps) :
M ≤ A * (d / (N * eps * L)) * L ^ 2 ∧
d * B ^ 2 / eps ^ 2 ≤
A ^ 2 * (d / (N * eps * L)) * L ^ 3 / (N * eps) ∧
d ^ 2 * B ^ 2 / (eps ^ 2 * L ^ 4) ≤
A ^ 2 * (d / (N * eps * L)) ^ 2 := by
have hB2 := pow_le_pow_left₀ hB hband 2
refine ⟨?_, ?_, ?_⟩
· calc
M ≤ d * B / eps := hmass
_ ≤ d * (A * L / N) / eps :=
div_le_div_of_nonneg_right (mul_le_mul_of_nonneg_left hband hd) heps.le
_ = _ := by field_simp
· calc
_ ≤ d * (A * L / N) ^ 2 / eps ^ 2 :=
div_le_div_of_nonneg_right (mul_le_mul_of_nonneg_left hB2 hd) (sq_nonneg eps)
_ = _ := by field_simp
· calc
_ ≤ d ^ 2 * (A * L / N) ^ 2 / (eps ^ 2 * L ^ 4) :=
div_le_div_of_nonneg_right (mul_le_mul_of_nonneg_left hB2 (sq_nonneg d))
(by positivity)
_ = _ := by field_simp
theorem hybrid_light_variance_rate_absorption Lemma hybrid_light_variance_rate_absorption in the paper ↗
Under the stated inputs and conditions, The calibrated factorial growth absorbs both light-mass terms and the bandwidth variance term into z / sqrt S, retaining the factorial-intensity constant. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_light_variance_rate_absorption
lemma hybrid_light_variance_rate_absorption (A F S L M H z t : Real)
(hA : 0 ≤ A) (hF : 0 ≤ F) (hS : 0 < S) (hL : 1 ≤ L)
(hM : 0 ≤ M) (hz : 0 ≤ z) (_ht : 0 < t)
(hcal : F * L ^ 3 ≤ sqrt S)
(hmass : M ≤ A * z * L ^ 2)
(hband : H ≤ A ^ 2 * z * L ^ 3 / S)
(htinv : 1 / t ≤ 64 / S) :
F * (M / S + M / t + H) ≤ (65 * A + A ^ 2) * z / sqrt S := by
have hroot : 0 < sqrt S := Real.sqrt_pos.mpr hS
have hroot2 : (sqrt S) ^ 2 = S := Real.sq_sqrt hS.le
have hLpos : 0 ≤ L := by linarith
have hL23 : L ^ 2 ≤ L ^ 3 := by nlinarith [sq_nonneg L]
have hcal2 : F * L ^ 2 ≤ sqrt S :=
(mul_le_mul_of_nonneg_left hL23 hF).trans hcal
have hFM : F * M ≤ A * z * sqrt S := by
calc
_ ≤ F * (A * z * L ^ 2) := mul_le_mul_of_nonneg_left hmass hF
_ = (A * z) * (F * L ^ 2) := by ring
_ ≤ _ := mul_le_mul_of_nonneg_left hcal2 (mul_nonneg hA hz)
have hFH : F * H ≤ A ^ 2 * z * sqrt S / S := by
calc
_ ≤ F * (A ^ 2 * z * L ^ 3 / S) := mul_le_mul_of_nonneg_left hband hF
_ = (A ^ 2 * z) * (F * L ^ 3) / S := by ring
_ ≤ _ := div_le_div_of_nonneg_right
(mul_le_mul_of_nonneg_left hcal (mul_nonneg (sq_nonneg A) hz)) hS.le
have hT : F * M / t ≤ 64 * (A * z * sqrt S / S) := by
calc
_ = (F * M) * (1 / t) := by ring
_ ≤ (F * M) * (64 / S) := mul_le_mul_of_nonneg_left htinv (mul_nonneg hF hM)
_ ≤ (A * z * sqrt S) * (64 / S) :=
mul_le_mul_of_nonneg_right hFM (by positivity)
_ = _ := by ring
calc
_ = F * M / S + F * M / t + F * H := by ring
_ ≤ A * z * sqrt S / S +
64 * (A * z * sqrt S / S) + A ^ 2 * z * sqrt S / S :=
add_le_add (add_le_add (div_le_div_of_nonneg_right hFM hS.le) hT) hFH
_ = (65 * A + A ^ 2) * z / sqrt S := by
field_simp
rw [hroot2]
ring
Under the stated inputs and conditions, Young's inequality at the public scale converts the mixed rate to the two terms of the arm risk, with no restriction on the cell-budget scale z. This gives the stated result.
Proof (Lean source)
-- @node: hybrid_mixed_rate_le
lemma hybrid_mixed_rate_le (S z : Real) (hS : 0 < S) :
2 * z / sqrt S ≤ z ^ 2 + 1 / S := by
have hroot : 0 < sqrt S := Real.sqrt_pos.mpr hS
have hroot2 : (sqrt S) ^ 2 = S := Real.sq_sqrt hS.le
have hinv : (1 / sqrt S) ^ 2 = 1 / S := by
rw [div_pow, hroot2]
norm_num
have hh := sq_nonneg (z - 1 / sqrt S)
rw [show 2 * z / sqrt S = 2 * z * (1 / sqrt S) by ring]
nlinarith only [hh, hinv]
Under the stated inputs and conditions, The pilot's inverse twentieth power remainder is smaller than the leading rate. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_pilot_remainder_le
lemma hybrid_pilot_remainder_le (S : Real) (hS : 1 ≤ S) :
(S ^ 20)⁻¹ ≤ 1 / S := by
have hpos : 0 < S := by linarith
have hp : S ≤ S ^ 20 := by
calc
S = S ^ 1 := by ring
_ ≤ S ^ 20 := pow_le_pow_right₀ hS (by norm_num)
simpa only [one_div] using one_div_le_one_div_of_le hpos hp
theorem hybrid_calibrated_light_variance_rate Lemma hybrid_calibrated_light_variance_rate in the paper ↗
Under the stated inputs and conditions, Applying the actual degree calibration and pool-size assumptions absorbs the entire equation (5) light-variance envelope into the target arm rate. This gives the stated result.
Formal statement
hypotheses
S N d eps tp t M :
hS :
exp 4096 ≤ S
hd :
0 ≤ d
heps :
0 < eps
heps1 :
eps ≤ 1
hNS :
S ≤ N * eps
htp :
N / 64 ≤ tp
ht :
N / 64 ≤ t
Proof (Lean source)
-- @node: hybrid_calibrated_light_variance_rate
lemma hybrid_calibrated_light_variance_rate (S N d eps tp t M : Real)
(hS : exp 4096 ≤ S) (hd : 0 ≤ d) (heps : 0 < eps) (heps1 : eps ≤ 1)
(hNS : S ≤ N * eps) (htp : N / 64 ≤ tp) (ht : N / 64 ≤ t) :
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let A : Real := 64 * (2 : Real) ^ 20
0 ≤ M → M ≤ d * B / eps →
((2 : Real) ^ 24) ^ L * (M / S + M / t + d * B ^ 2 / eps ^ 2) ≤
((65 * A + A ^ 2) / 2) * (1 / S + (d / (N * eps * L)) ^ 2) := by
dsimp only
intro hM hmass
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let A : Real := 64 * (2 : Real) ^ 20
let z := d / (N * eps * L)
have hSpos : 0 < S := (Real.exp_pos _).trans_le hS
have hN : 0 < N := by nlinarith
have htpos : 0 < t := (show 0 < N / 64 by positivity).trans_le ht
have htpPos : 0 < tp := (show 0 < N / 64 by positivity).trans_le htp
have hcal := hybrid_degree_calibration S hS
have hL4 : (4 : Real) ≤ L := by exact_mod_cast hcal.1
have hL : 0 < (L : Real) := by linarith
have hB : 0 ≤ B := by dsimp [B]; positivity
have hA : 0 ≤ A := by norm_num [A]
have hband : B ≤ A * L / N := hybrid_bandwidth_public_bound N tp t L hN htp ht
obtain ⟨hm, hb, _⟩ := hybrid_bandwidth_rate_monomials A B d N eps L M
hA hB hd hN heps hL hband hmass
have hbS : d * B ^ 2 / eps ^ 2 ≤ A ^ 2 * z * (L : Real) ^ 3 / S := by
exact hb.trans (div_le_div_of_nonneg_left (by positivity) hSpos hNS)
have hNle : N * eps ≤ N := mul_le_of_le_one_right hN.le heps1
have htinv : 1 / t ≤ 64 / S := by
apply (div_le_div_iff₀ htpos hSpos).2
nlinarith only [hNS, hNle, ht]
have hz : 0 ≤ z := by dsimp [z]; positivity
have habs := hybrid_light_variance_rate_absorption A (((2 : Real) ^ 24) ^ L)
S L M (d * B ^ 2 / eps ^ 2) z t hA (by positivity) hSpos (by linarith)
hM hz htpos hcal.2.2.1 hm hbS htinv
calc
_ ≤ (65 * A + A ^ 2) * z / sqrt S := habs
_ = ((65 * A + A ^ 2) / 2) * (2 * z / sqrt S) := by ring
_ ≤ ((65 * A + A ^ 2) / 2) * (z ^ 2 + 1 / S) :=
mul_le_mul_of_nonneg_left (hybrid_mixed_rate_le S z hSpos) (by positivity)
_ = _ := by ring
Every retained term of equation (14), including the single-cell squared-bias sum and the false-pilot remainder, is bounded by a universal multiple of the arm rate. This is deterministic absorption; the probabilistic MSE assembly is separate.
Formal statement
Proof (Lean source)
-- @node: hybrid_intermediate_envelope_rate
lemma hybrid_intermediate_envelope_rate :
∃ C : Real, 0 < C ∧ ∀ (S N d eps tp t M : Real),
exp 4096 ≤ S → 1 ≤ d → 0 < eps → eps ≤ 1 →
S ≤ N * eps → N / 64 ≤ tp → N / 64 ≤ t →
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
0 ≤ M → M ≤ d * B / eps →
1 / S + d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) +
((2 : Real) ^ 24) ^ L * (M / S + M / t + d * B ^ 2 / eps ^ 2) +
d * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) + (S ^ 20)⁻¹ ≤
C * (1 / S + (d / (N * eps * L)) ^ 2) := by
let A : Real := 64 * (2 : Real) ^ 20
let K : Real := (65 * A + A ^ 2) / 2
refine ⟨2 + 2 * A ^ 2 + K, by dsimp [K, A]; positivity, ?_⟩
intro S N d eps tp t M hS hd heps heps1 hNS htp ht
dsimp only
intro hM hmass
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let z := d / (N * eps * L)
have hSpos : 0 < S := (Real.exp_pos _).trans_le hS
have hS1 : 1 ≤ S := (Real.one_le_exp (by norm_num : (0 : Real) ≤ 4096)).trans hS
have hN : 0 < N := by nlinarith
have hL4 : (4 : Real) ≤ L := by exact_mod_cast (hybrid_degree_calibration S hS).1
have hL : 0 < (L : Real) := by linarith
have htpos : 0 < t := (show 0 < N / 64 by positivity).trans_le ht
have htpPos : 0 < tp := (show 0 < N / 64 by positivity).trans_le htp
have hB : 0 ≤ B := by dsimp [B]; positivity
have hA : 0 ≤ A := by norm_num [A]
have hband : B ≤ A * L / N := hybrid_bandwidth_public_bound N tp t L hN htp ht
have hbig := (hybrid_bandwidth_rate_monomials A B d N eps L M
hA hB (by linarith) hN heps hL hband hmass).2.2
have hsmall : d * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) ≤ A ^ 2 * z ^ 2 := by
apply le_trans _ hbig
apply div_le_div_of_nonneg_right _ (by positivity)
apply mul_le_mul_of_nonneg_right _ (sq_nonneg B)
nlinarith only [hd]
have hlight := hybrid_calibrated_light_variance_rate S N d eps tp t M
hS (by linarith) heps heps1 hNS htp ht hM hmass
have htail := hybrid_pilot_remainder_le S hS1
have hR : 0 ≤ 1 / S := by positivity
have hK : 0 ≤ K := by dsimp [K]; positivity
change 1 / S + d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) +
((2 : Real) ^ 24) ^ L * (M / S + M / t + d * B ^ 2 / eps ^ 2) +
d * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) + (S ^ 20)⁻¹ ≤
(2 + 2 * A ^ 2 + K) * (1 / S + z ^ 2)
change ((2 : Real) ^ 24) ^ L * (M / S + M / t + d * B ^ 2 / eps ^ 2) ≤
K * (1 / S + z ^ 2) at hlight
change d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) ≤ A ^ 2 * z ^ 2 at hbig
nlinarith only [hbig, hsmall, hlight, htail, hR, sq_nonneg z,
mul_nonneg (sq_nonneg A) hR]
Under the stated hypotheses, The actual canonical MSE envelope, with its branch constants and pool intensities, absorbs into the public arm rate. The inverse-count variance is supplied by its separate proved overlap bound; no retained pilot term is discarded. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_canonical_envelope_rate
lemma hybrid_canonical_envelope_rate (C0 CP CH : Real)
(hC0 : 0 < C0) (hCP : 0 < CP) (hCH : 0 < CH) :
∃ C : Real, 0 < C ∧ ∀ (S N d eps u tp t M c H : Real),
exp 4096 ≤ S → 1 ≤ d → 0 < eps → eps ≤ 1 →
S ≤ N * eps → S / 32 ≤ u * eps → N / 64 ≤ tp → N / 64 ≤ t →
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
0 ≤ M → M ≤ d * B / eps → 0 ≤ c → c ≤ d →
H ≤ C0 * (1 / (u * eps) + 1 / (t * eps)) →
H + CP * ((2 : Real) ^ 24) ^ L * (M / (u * eps) + M / t + c * B ^ 2 / eps ^ 2) +
2 * c * (B / (eps * (L : Real) ^ 2)) ^ 2 +
1 / (exp 1 * t * eps) +
CH * (S ^ 20)⁻¹ * (1 / (u * eps) + 1 + 1 / t) +
2 * d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) + 18 * (S ^ 20)⁻¹ ≤
C * (1 / S + (d / (N * eps * L)) ^ 2) := by
obtain ⟨CE, hCE, habs⟩ := hybrid_intermediate_envelope_rate
let K := 96 * C0 + 32 * CP + 97 * CH + 86
refine ⟨K * CE, mul_pos (by dsimp [K]; positivity) hCE, ?_⟩
intro S N d eps u tp t M c H hS hd heps heps1 hNS hu htp ht
dsimp only
intro hM hmass hc hcd hH
let L := floor (log S / 1024)
let B : Real := (2 : Real) ^ 20 * L / min tp t
let F := ((2 : Real) ^ 24) ^ L
let Q := d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4)
let W := F * (M / S + M / t + d * B ^ 2 / eps ^ 2)
let R := d * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4)
let r := (S ^ 20)⁻¹
let E := 1 / S + Q + W + R + r
have hS1 : 1 ≤ S := (Real.one_le_exp (by norm_num : (0 : Real) ≤ 4096)).trans hS
have hSp : 0 < S := by linarith
have hN : 0 < N := by nlinarith
have htpP : 0 < tp := (show 0 < N / 64 by positivity).trans_le htp
have htP : 0 < t := (show 0 < N / 64 by positivity).trans_le ht
have hue : 0 < u * eps := (show 0 < S / 32 by positivity).trans_le hu
have hL : 0 < (L : Real) := by
exact_mod_cast (show 0 < L from by have := (hybrid_degree_calibration S hS).1; omega)
have hB : 0 ≤ B := by dsimp [B]; positivity
have hF : 0 ≤ F := by positivity
have hQ : 0 ≤ Q := by dsimp [Q]; positivity
have hW : 0 ≤ W := by dsimp [W]; positivity
have hR : 0 ≤ R := by dsimp [R]; positivity
have hr : 0 ≤ r := by dsimp [r]; positivity
have hEinv : 1 / S ≤ E := by dsimp [E]; linarith
have hEQ : Q ≤ E := by dsimp [E]; linarith [show 0 ≤ 1 / S by positivity]
have hEW : W ≤ E := by dsimp [E]; linarith [show 0 ≤ 1 / S by positivity]
have hER : R ≤ E := by dsimp [E]; linarith [show 0 ≤ 1 / S by positivity]
have hEr : r ≤ E := by dsimp [E]; linarith [show 0 ≤ 1 / S by positivity]
have hui : 1 / (u * eps) ≤ 32 / S := by
apply (div_le_div_iff₀ hue hSp).2
linarith
have hti : 1 / (t * eps) ≤ 64 / S := by
apply (div_le_div_iff₀ (mul_pos htP heps) hSp).2
have hh := mul_le_mul_of_nonneg_right ht heps.le
nlinarith only [hh, hNS]
have htplain : 1 / t ≤ 64 / S := by
apply (div_le_div_iff₀ htP hSp).2
have hne : N * eps ≤ N := mul_le_of_le_one_right hN.le heps1
linarith
have hi1 : 1 / S ≤ 1 := (div_le_one hSp).2 hS1
have hheavy : H ≤ 96 * C0 * E := by
calc
H ≤ C0 * (1 / (u * eps) + 1 / (t * eps)) := hH
_ ≤ C0 * (32 / S + 64 / S) := mul_le_mul_of_nonneg_left (add_le_add hui hti) hC0.le
_ = 96 * C0 * (1 / S) := by ring
_ ≤ _ := mul_le_mul_of_nonneg_left hEinv (by positivity)
have hpoly : CP * F * (M / (u * eps) + M / t + c * B ^ 2 / eps ^ 2) ≤ 32 * CP * E := by
have hm : M / (u * eps) ≤ 32 * (M / S) := by
convert mul_le_mul_of_nonneg_left hui hM using 1 <;> first | rfl | ring
have hcB : c * B ^ 2 / eps ^ 2 ≤ d * B ^ 2 / eps ^ 2 :=
div_le_div_of_nonneg_right (mul_le_mul_of_nonneg_right hcd (sq_nonneg B)) (sq_nonneg eps)
have hsum : M / (u * eps) + M / t + c * B ^ 2 / eps ^ 2 ≤
32 * (M / S + M / t + d * B ^ 2 / eps ^ 2) := by
have hmt : 0 ≤ M / t := by positivity
have hdb : 0 ≤ d * B ^ 2 / eps ^ 2 := by positivity
linarith
calc
_ ≤ CP * F * (32 * (M / S + M / t + d * B ^ 2 / eps ^ 2)) :=
mul_le_mul_of_nonneg_left hsum (by positivity)
_ = 32 * CP * W := by dsimp [W]; ring
_ ≤ _ := mul_le_mul_of_nonneg_left hEW (by positivity)
have hsep : 2 * c * (B / (eps * (L : Real) ^ 2)) ^ 2 ≤ 2 * E := by
calc
_ ≤ 2 * d * (B / (eps * (L : Real) ^ 2)) ^ 2 :=
mul_le_mul_of_nonneg_right (by linarith : 2 * c ≤ 2 * d) (sq_nonneg _)
_ = 2 * R := by dsimp [R]; ring
_ ≤ _ := by linarith
have hexp : 1 / (exp 1 * t * eps) ≤ 64 * E := by
have he : 1 ≤ exp 1 := Real.one_le_exp (by norm_num)
calc
_ ≤ 1 / (t * eps) := one_div_le_one_div_of_le (by positivity) (by
have hh := mul_le_mul_of_nonneg_right he (mul_nonneg htP.le heps.le)
nlinarith only [hh])
_ ≤ 64 / S := hti
_ = 64 * (1 / S) := by ring
_ ≤ _ := mul_le_mul_of_nonneg_left hEinv (by norm_num)
have htail : CH * r * (1 / (u * eps) + 1 + 1 / t) ≤ 97 * CH * E := by
have hsum : 1 / (u * eps) + 1 + 1 / t ≤ 97 := by
have hu1 : 32 / S ≤ 32 := by
convert mul_le_mul_of_nonneg_left hi1 (by norm_num : (0 : Real) ≤ 32) using 1 <;>
first | rfl | ring
have ht1 : 64 / S ≤ 64 := by
convert mul_le_mul_of_nonneg_left hi1 (by norm_num : (0 : Real) ≤ 64) using 1 <;>
first | rfl | ring
linarith
calc
_ ≤ CH * r * 97 := mul_le_mul_of_nonneg_left hsum (by positivity)
_ = 97 * CH * r := by ring
_ ≤ _ := mul_le_mul_of_nonneg_left hEr (by positivity)
have henv : E ≤ CE * (1 / S + (d / (N * eps * L)) ^ 2) :=
habs S N d eps tp t M hS hd heps heps1 hNS htp ht hM hmass
have hbias : 2 * d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) ≤ 2 * E := by
calc
_ = 2 * Q := by dsimp [Q]; ring
_ ≤ _ := by linarith only [hEQ]
calc
_ ≤ K * E := by
change H + CP * F * (M / (u * eps) + M / t + c * B ^ 2 / eps ^ 2) +
2 * c * (B / (eps * (L : Real) ^ 2)) ^ 2 +
1 / (exp 1 * t * eps) + CH * r * (1 / (u * eps) + 1 + 1 / t) +
2 * d ^ 2 * B ^ 2 / (eps ^ 2 * (L : Real) ^ 4) + 18 * r ≤ K * E
dsimp only [K]
linarith only [hheavy, hpoly, hsep, hexp, htail, hbias, hEr]
_ ≤ K * (CE * (1 / S + (d / (N * eps * L)) ^ 2)) :=
mul_le_mul_of_nonneg_left henv (by dsimp [K]; positivity)
_ = _ := by ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridArmRisk 1 declarations Uniform arm MSE bound for the independent three-pool hybrid statistic.
Uniform arm MSE bound for the independent three-pool hybrid statistic.
theorem uniform_hybrid_arm_risk
The three-pool Poisson hybrid arm statistic has the universal rare-overlap mean squared error bound.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
n m d :
eps :
P :
a :
u tp t :
1 ≤ n
2 ≤ d
0 < eps
eps ≤ 1 / 4
ModelClass d eps P
(n : Real) / 32 ≤ u
((n : Real) + m) / 64 ≤ tp
((n : Real) + m) / 64 ≤ t
1 / 3 ≤ t / tp
t / tp ≤ 3
k0 :
floor (tp * B / 4)
Om :
Type
mu :
Measure Om
(∫ om, ((∑ j : Fin d, hybridCellValue L B k0 u t (Z a j om) (J a j om) (K a j om) (K (!a) j om)) - ∑ j : Fin d, cellMass P j * outcomeMean P a j) ^ 2 ∂mu)
Proof (Lean source)
lemma uniform_hybrid_arm_risk :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real) (P : DiscreteLaw d)
(a : Bool) (u tp t : Real),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 → exp 4096 ≤ (n : Real) * eps →
ModelClass d eps P → (n : Real) / 32 ≤ u → ((n : Real) + m) / 64 ≤ tp →
((n : Real) + m) / 64 ≤ t → 1 / 3 ≤ t / tp → t / tp ≤ 3 →
let L := floor (log ((n : Real) * eps) / 1024)
let B : Real := 2 ^ 20 * L / min tp t
let k0 := floor (tp * B / 4)
∀ (Om : Type) [MeasurableSpace Om] (mu : Measure Om) [IsProbabilityMeasure mu]
(Z J K : Bool → Fin d → Om → Nat),
(∀ b j, Measurable (Z b j) ∧ Measurable (J b j) ∧ Measurable (K b j)) →
(∀ b j, mu.map (Z b j) = poissonMeasure (toNNReal (u * markedMass P j b))) →
(∀ b j, mu.map (J b j) = poissonMeasure (toNNReal (tp * armMass P j b))) →
(∀ b j, mu.map (K b j) = poissonMeasure (toNNReal (t * armMass P j b))) →
iIndepFun (fun i : Fin 3 × Bool × Fin d =>
if i.1 = 0 then Z i.2.1 i.2.2 else if i.1 = 1 then J i.2.1 i.2.2 else K i.2.1 i.2.2) mu →
(∫ om, ((∑ j : Fin d, hybridCellValue L B k0 u t
(Z a j om) (J a j om) (K a j om) (K (!a) j om)) -
∑ j : Fin d, cellMass P j * outcomeMean P a j) ^ 2 ∂mu) ≤
C * (1 / ((n : Real) * eps) + ((d : Real) / (((n : Real) + m) * eps * L)) ^ 2) :=
by
classical
obtain ⟨C, hC, hcanonical⟩ := hybrid_canonical_arm_rate
refine ⟨C, hC, ?_⟩
intro n m d eps P a u tp t hn hd heps heps4 hS hP hu htp ht hratio _
dsimp only
intro Om _ mu _ Z J K hmeas hZ hJ hK hind
let X : Fin 4 × Fin d → Om → Nat := fun i =>
if i.1 = 0 then J a i.2 else if i.1 = 1 then Z a i.2
else if i.1 = 2 then K a i.2 else K (!a) i.2
have hX (i) : Measurable (X i) := by
dsimp [X]
split_ifs <;> first
| exact (hmeas a i.2).1
| exact (hmeas a i.2).2.1
| exact (hmeas a i.2).2.2
| exact (hmeas (!a) i.2).2.2
let idx : Fin 4 × Fin d → Fin 3 × Bool × Fin d := fun i =>
if i.1 = 0 then (1,a,i.2) else if i.1 = 1 then (0,a,i.2)
else if i.1 = 2 then (2,a,i.2) else (2,!a,i.2)
have hinj : Injective idx := by
rintro ⟨i,j⟩ ⟨i',j'⟩ h
fin_cases i <;> fin_cases i' <;> simp [idx] at h ⊢ <;>
exact h
have hi : iIndepFun X mu := by
have hh := hind.precomp hinj
have hfun : (fun i : Fin 4 × Fin d =>
if (idx i).1 = 0 then Z (idx i).2.1 (idx i).2.2
else if (idx i).1 = 1 then J (idx i).2.1 (idx i).2.2
else K (idx i).2.1 (idx i).2.2) = X := by
funext ⟨i,j⟩
fin_cases i <;> simp [idx, X]
rw [hfun] at hh
exact hh
have hlaw := hybrid_four_count_product_law mu X hX hi
simp only [X, Fin.reduceEq, if_true, if_false] at hlaw
simp_rw [hJ, hZ, hK] at hlaw
have hsum (j : Fin d) : armMass P j a + armMass P j (!a) = cellMass P j := by
cases a <;> simp [cellMass, armMass, add_comm]
have hq (j : Fin d) : armMass P j a * outcomeMean P a j = markedMass P j a := by
by_cases hz : armMass P j a = 0
· have hbound : markedMass P j a ≤ armMass P j a := by
simp only [markedMass, armMass, Fintype.sum_bool]
linarith [jointMass_nonneg P j a false]
have hzero : markedMass P j a = 0 :=
le_antisymm (hz ▸ hbound) (jointMass_nonneg P j a true)
simp [hz, hzero]
· dsimp only [outcomeMean]; field_simp
have hc := hcanonical ((n : Real) * eps) ((n : Real) + m) eps u tp t hS heps
(by linarith) (mul_le_mul_of_nonneg_right
(le_add_of_nonneg_right (Nat.cast_nonneg m)) heps.le)
(by nlinarith) htp ht hratio d
(fun j => armMass P j a) (fun j => armMass P j (!a)) (outcomeMean P a)
(by omega)
(fun j => ⟨armMass_nonneg P j a, armMass_nonneg P j (!a),
outcomeMean_mem_Icc P a j, by rw [hsum]; exact armMass_ge_overlap_cellMass P eps hP j a⟩)
(by simpa only [hsum, sum_cellMass] using (le_refl (1 : Real)))
dsimp only at hc
simp_rw [hsum, hq] at hc
let F : (Fin d → Nat × (Nat × Nat × Nat)) → Real := fun z =>
((∑ j, hybridCellValue (floor (log ((n : Real) * eps) / 1024))
(2 ^ 20 * floor (log ((n : Real) * eps) / 1024) / min tp t)
(floor (tp * (2 ^ 20 * floor (log ((n : Real) * eps) / 1024) /
min tp t) / 4)) u t (z j).2.1 (z j).1 (z j).2.2.1 (z j).2.2.2) -
∑ j, cellMass P j * outcomeMean P a j) ^ 2
have hmap : Measurable (fun om j => (J a j om, Z a j om, K a j om, K (!a) j om)) := by
apply measurable_pi_lambda
intro j
exact (hmeas a j).2.1.prodMk ((hmeas a j).1.prodMk
((hmeas a j).2.2.prodMk (hmeas (!a) j).2.2))
have hF : Measurable F := measurable_of_countable F
have heq := integral_map (μ := mu) hmap.aemeasurable hF.aestronglyMeasurable
rw [hlaw] at heq
change (∫ om, F (fun j => (J a j om, Z a j om, K a j om, K (!a) j om)) ∂mu) ≤ _
rw [← heq]
exact hc
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridFiniteEncoding 13 declarations Ordered three-pool count readback and conditional averaging for the finite hybrid rule.
Ordered three-pool count readback and conditional averaging for the finite hybrid rule.
def hybridOrderedPrefixStatistic
The ideal ordered statistic retains the outcome, pilot, and factorial pools separately.
Definition (Lean source)
parameters
d :
tun :
s :
def
given by
Z :
fun a j => finiteSampleHistogram s.1.points (j,a,true)
J :
fun a j => finiteSampleHistogram s.2.1.points (j,a)
K :
fun a j => finiteSampleHistogram s.2.2.points (j,a)
theorem hybrid_prefix_statistic_eq_ordered
Under the stated inputs and conditions, Ordered histograms recover exactly the three original array counters. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
s :
Sample n m d
r0 rp rf :
h0 :
0 + r0 ≤ n
hp :
hap :
0 + (rp - (hybridTuning n m eps).hp) ≤ m
hf :
≤ n
haf :
m / 2 + (rf - (hybridTuning n m eps).hf) ≤ m
conclusion
hybridPrefixStatistic eps r0 rp rf s
= hybridOrderedPrefixStatistic (hybridTuning n m eps) (baselineBlockPrefix s.1 0 r0 h0, baselineAuxiliaryPrefix s (hybridTuning n m eps).h0 (hybridTuning n m eps).hp 0 rp hp hap, baselineAuxiliaryPrefix s ((hybridTuning n m eps).h0 + (hybridTuning n m eps).hp) (hybridTuning n m eps).hf (m / 2) rf hf haf)
Proof (Lean source)
-- @node: hybrid_prefix_statistic_eq_ordered
lemma hybrid_prefix_statistic_eq_ordered {n m d : Nat} (eps : Real) (s : Sample n m d)
(r0 rp rf : Nat)
(h0 : 0 + r0 ≤ n)
(hp : (hybridTuning n m eps).h0 + min rp (hybridTuning n m eps).hp ≤ n)
(hap : 0 + (rp - (hybridTuning n m eps).hp) ≤ m)
(hf : (hybridTuning n m eps).h0 + (hybridTuning n m eps).hp +
min rf (hybridTuning n m eps).hf ≤ n)
(haf : m / 2 + (rf - (hybridTuning n m eps).hf) ≤ m) :
hybridPrefixStatistic eps r0 rp rf s =
hybridOrderedPrefixStatistic (hybridTuning n m eps)
(baselineBlockPrefix s.1 0 r0 h0,
baselineAuxiliaryPrefix s (hybridTuning n m eps).h0
(hybridTuning n m eps).hp 0 rp hp hap,
baselineAuxiliaryPrefix s ((hybridTuning n m eps).h0 +
(hybridTuning n m eps).hp) (hybridTuning n m eps).hf (m / 2) rf hf haf) := by
unfold hybridOrderedPrefixStatistic
simp only [baseline_labeled_block_histogram, baseline_pooled_arm_histogram]
rfl
theorem hybrid_valid_prefix_statistic_eq_ordered
Under the stated hypotheses, The count bridge applies to every retained triple, with no extra array-bound premises. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
s :
Sample n m d
r0 rp rf :
hr0 :
r0 ≤ (hybridTuning n m eps).h0
hrp :
rp ≤ (hybridTuning n m eps).Mp
hrf :
rf ≤ (hybridTuning n m eps).Mf
conclusion
∃ (h0 : 0 + r0 ≤ n) (hp : (hybridTuning n m eps).h0 + min rp (hybridTuning n m eps).hp ≤ n) (hap : 0 + (rp - (hybridTuning n m eps).hp) ≤ m) (hf : (hybridTuning n m eps).h0 + (hybridTuning n m eps).hp + min rf (hybridTuning n m eps).hf ≤ n) (haf : m / 2 + (rf - (hybridTuning n m eps).hf) ≤ m),
hybridPrefixStatistic eps r0 rp rf s
= hybridOrderedPrefixStatistic (hybridTuning n m eps) (baselineBlockPrefix s.1 0 r0 h0, baselineAuxiliaryPrefix s (hybridTuning n m eps).h0 (hybridTuning n m eps).hp 0 rp hp hap, baselineAuxiliaryPrefix s ((hybridTuning n m eps).h0 + (hybridTuning n m eps).hp) (hybridTuning n m eps).hf (m / 2) rf hf haf)
Proof (Lean source)
-- @node: hybrid_valid_prefix_statistic_eq_ordered
lemma hybrid_valid_prefix_statistic_eq_ordered {n m d : Nat} (eps : Real)
(s : Sample n m d) (r0 rp rf : Nat)
(hr0 : r0 ≤ (hybridTuning n m eps).h0)
(hrp : rp ≤ (hybridTuning n m eps).Mp)
(hrf : rf ≤ (hybridTuning n m eps).Mf) :
∃ (h0 : 0 + r0 ≤ n)
(hp : (hybridTuning n m eps).h0 + min rp (hybridTuning n m eps).hp ≤ n)
(hap : 0 + (rp - (hybridTuning n m eps).hp) ≤ m)
(hf : (hybridTuning n m eps).h0 + (hybridTuning n m eps).hp +
min rf (hybridTuning n m eps).hf ≤ n)
(haf : m / 2 + (rf - (hybridTuning n m eps).hf) ≤ m),
hybridPrefixStatistic eps r0 rp rf s =
hybridOrderedPrefixStatistic (hybridTuning n m eps)
(baselineBlockPrefix s.1 0 r0 h0,
baselineAuxiliaryPrefix s (hybridTuning n m eps).h0
(hybridTuning n m eps).hp 0 rp hp hap,
baselineAuxiliaryPrefix s ((hybridTuning n m eps).h0 +
(hybridTuning n m eps).hp) (hybridTuning n m eps).hf (m / 2) rf hf haf) := by
have h0 : 0 + r0 ≤ n := by dsimp [hybridTuning] at hr0; omega
have hp : (hybridTuning n m eps).h0 + min rp (hybridTuning n m eps).hp ≤ n := by
dsimp [hybridTuning]; omega
have hap : 0 + (rp - (hybridTuning n m eps).hp) ≤ m := by
dsimp [hybridTuning] at hrp ⊢; omega
have hf : (hybridTuning n m eps).h0 + (hybridTuning n m eps).hp +
min rf (hybridTuning n m eps).hf ≤ n := by dsimp [hybridTuning]; omega
have haf : m / 2 + (rf - (hybridTuning n m eps).hf) ≤ m := by
dsimp [hybridTuning] at hrf ⊢; omega
exact ⟨h0, hp, hap, hf, haf, hybrid_prefix_statistic_eq_ordered eps s r0 rp rf
h0 hp hap hf haf⟩
Under the stated inputs and conditions, Integrating a capped triple gives the exact product of the three Poisson weights. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ k : Nat × Nat × Nat, if k.1 ≤ h ∧ k.2.1 ≤ p ∧ k.2.2 ≤ f then F k.1 k.2.1 k.2.2 else 0 ∂((poissonMeasure u).prod ((poissonMeasure tp).prod (poissonMeasure t))))
Proof (Lean source)
-- @node: hybrid_truncated_poisson_integral
lemma hybrid_truncated_poisson_integral (u tp t : NNReal) (h p f : Nat)
(F : Nat → Nat → Nat → Real) :
(∫ k : Nat × Nat × Nat, if k.1 ≤ h ∧ k.2.1 ≤ p ∧ k.2.2 ≤ f then
F k.1 k.2.1 k.2.2 else 0
∂((poissonMeasure u).prod ((poissonMeasure tp).prod (poissonMeasure t)))) =
∑ r0 ∈ range (h + 1), ∑ rp ∈ range (p + 1),
∑ rf ∈ range (f + 1),
exp (-(u : Real) - (tp : Real) - (t : Real)) *
((u : Real) ^ r0 / (r0.factorial : Real)) *
((tp : Real) ^ rp / (rp.factorial : Real)) *
((t : Real) ^ rf / (rf.factorial : Real)) * F r0 rp rf := by
classical
let s := (range (h + 1)).product
((range (p + 1)).product (range (f + 1)))
have hind : (fun k : Nat × Nat × Nat =>
if k.1 ≤ h ∧ k.2.1 ≤ p ∧ k.2.2 ≤ f then F k.1 k.2.1 k.2.2 else 0) =
(s : Set (Nat × Nat × Nat)).indicator (fun k => F k.1 k.2.1 k.2.2) := by
funext k
simp [s, indicator]
rw [hind, integral_indicator s.measurableSet, setIntegral_finset s IntegrableOn.finset]
dsimp only [s]
simp only [Finset.product_eq_sprod, Finset.sum_product]
apply Finset.sum_congr rfl
intro r0 _
apply Finset.sum_congr rfl
intro rp _
apply Finset.sum_congr rfl
intro rf _
have hmass : ((poissonMeasure u).prod
((poissonMeasure tp).prod (poissonMeasure t))).real {(r0,rp,rf)} =
(poissonMeasure u).real {r0} *
((poissonMeasure tp).real {rp} * (poissonMeasure t).real {rf}) := by
simp only [Measure.real_def, ← Set.singleton_prod_singleton,
Measure.prod_prod, ENNReal.toReal_mul]
rw [smul_eq_mul, hmass, poissonMeasure_real_singleton,
poissonMeasure_real_singleton, poissonMeasure_real_singleton]
rw [show -(u : Real) - (tp : Real) - (t : Real) =
-(u : Real) + -(tp : Real) + -(t : Real) by ring, Real.exp_add, Real.exp_add]
ring
Each Poisson request is independent, with the exact public pool intensity.
Definition (Lean source)
parameters
given by
(poissonMeasure ((hybridTuning n m eps).h0 / 8 : NNReal)).prod ((poissonMeasure ((hybridTuning n m eps).Mp / 8 : NNReal)).prod (poissonMeasure ((hybridTuning n m eps).Mf / 8 : NNReal)))
instance hybridRequestLaw_probability
The independent request law has total mass one, also when a pool is empty.
Definition (Lean source)
-- @node: hybridRequestLaw_probability
instance hybridRequestLaw_probability (n m : Nat) (eps : Real) :
IsProbabilityMeasure (hybridRequestLaw n m eps) := by
unfold hybridRequestLaw
infer_instance
Overflow in any of the three pools returns zero.
Definition (Lean source)
parameters
def
given by
tun :
hybridTuning n m eps
if k.1 ≤ tun.h0 ∧
k.2.1 ≤ tun.Mp ∧
k.2.2 ≤ tun.Mf then hybridPrefixStatistic eps k.1 k.2.1 k.2.2 s else 0
theorem hybrid_capped_array_mem_Icc
Under the stated inputs and conditions, Both valid-prefix clipping and zero overflow lie in the required interval. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_capped_array_mem_Icc
lemma hybrid_capped_array_mem_Icc {n m d : Nat} (eps : Real) (s : Sample n m d)
(k : Nat × Nat × Nat) : hybridCappedArrayStatistic eps s k ∈ Icc (-1) 1 := by
unfold hybridCappedArrayStatistic
dsimp only
split_ifs
· exact ⟨le_max_left _ _, max_le (by norm_num) (min_le_left _ _)⟩
· norm_num
theorem hybrid_estimator_eq_poisson_integral
Under the stated inputs and conditions, The estimator's finite sum is exactly conditional averaging of the capped statistic. This gives the stated result.
Formal statement
hypotheses
conclusion
hybridEstimator n m d eps s
= ∫ k, hybridCappedArrayStatistic eps s k ∂hybridRequestLaw n m eps
Proof (Lean source)
-- @node: hybrid_estimator_eq_poisson_integral
lemma hybrid_estimator_eq_poisson_integral {n m d : Nat} (eps : Real) (s : Sample n m d)
(hS : exp 4096 ≤ (n : Real) * eps) :
hybridEstimator n m d eps s =
∫ k, hybridCappedArrayStatistic eps s k ∂hybridRequestLaw n m eps := by
unfold hybridRequestLaw hybridCappedArrayStatistic
dsimp only
rw [hybrid_truncated_poisson_integral _ _ _ _ _ _
(fun r0 rp rf => hybridPrefixStatistic eps r0 rp rf s)]
simp only [hybridEstimator, if_neg (not_lt.mpr hS), hybridTuning,
NNReal.coe_div, NNReal.coe_natCast, NNReal.coe_ofNat]
theorem hybrid_averaging_sq_loss_le
Under the stated inputs and conditions, Jensen's inequality contracts the squared loss at each fixed original sample. This gives the stated result.
Formal statement
hypotheses
conclusion
(hybridEstimator n m d eps s - theta) ^ 2
≤ ∫ k, (hybridCappedArrayStatistic eps s k - theta) ^ 2 ∂hybridRequestLaw n m eps
Proof (Lean source)
-- @node: hybrid_averaging_sq_loss_le
lemma hybrid_averaging_sq_loss_le {n m d : Nat} (eps : Real) (s : Sample n m d)
(hS : exp 4096 ≤ (n : Real) * eps) (theta : Real) :
(hybridEstimator n m d eps s - theta) ^ 2 ≤
∫ k, (hybridCappedArrayStatistic eps s k - theta) ^ 2
∂hybridRequestLaw n m eps := by
have hm : Measurable (hybridCappedArrayStatistic eps s) := by fun_prop
have hb : UniformlyBounded (hybridCappedArrayStatistic eps s) :=
⟨1, by norm_num, fun k => abs_le.mpr (hybrid_capped_array_mem_Icc eps s k)⟩
have hj := sqLoss_kernelMean_le
(Kernel.const Unit (hybridRequestLaw n m eps)) hm hb theta ()
rw [hybrid_estimator_eq_poisson_integral eps s hS]
simpa only [kernelMean, Kernel.const_apply] using hj
theorem hybrid_averaging_sq_risk_le
Under the stated inputs and conditions, Conditional averaging contracts risk under the original product data law. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ sqRisk ((annotationLaw P n m).prod (hybridRequestLaw n m eps)) (fun z => hybridCappedArrayStatistic eps z.1 z.2) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_averaging_sq_risk_le
lemma hybrid_averaging_sq_risk_le {n m d : Nat} (eps : Real) (P : DiscreteLaw d)
(hS : exp 4096 ≤ (n : Real) * eps) :
sqRisk (annotationLaw P n m) (hybridEstimator n m d eps)
(ateFunctional P) ≤
sqRisk ((annotationLaw P n m).prod (hybridRequestLaw n m eps))
(fun z => hybridCappedArrayStatistic eps z.1 z.2) (ateFunctional P) := by
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have htheta := ateFunctional_mem_Icc P
have hm : Measurable (fun z : Sample n m d × (Nat × Nat × Nat) =>
(hybridCappedArrayStatistic eps z.1 z.2 - ateFunctional P) ^ 2) := by fun_prop
have hi : Integrable (fun z : Sample n m d × (Nat × Nat × Nat) =>
(hybridCappedArrayStatistic eps z.1 z.2 - ateFunctional P) ^ 2)
((annotationLaw P n m).prod (hybridRequestLaw n m eps)) := by
apply Integrable.of_bound hm.aestronglyMeasurable 4
filter_upwards with z
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hc := hybrid_capped_array_mem_Icc eps z.1 z.2
nlinarith [hc.1, hc.2, htheta.1, htheta.2]
unfold sqRisk
rw [integral_prod _ hi]
exact integral_mono_of_nonneg
(Filter.Eventually.of_forall (fun _ => sq_nonneg _)) hi.integral_prod_left
(Filter.Eventually.of_forall (fun s => hybrid_averaging_sq_loss_le eps s hS _))
theorem hybrid_ruleRisk_eq_sqRisk
Under the stated inputs and conditions, The independent seed leaves the data-only hybrid rule's risk unchanged. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_ruleRisk_eq_sqRisk
lemma hybrid_ruleRisk_eq_sqRisk (n m d : Nat) (eps : Real) (P : DiscreteLaw d) :
ruleRisk (liftRule (hybridEstimator n m d eps)) P =
sqRisk (annotationLaw P n m) (hybridEstimator n m d eps)
(ateFunctional P) := by
let : IsProbabilityMeasure seedLaw := ⟨by
norm_num [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have hm : Measurable (fun z : Sample n m d × Real =>
(hybridEstimator n m d eps z.1 - ateFunctional P) ^ 2) := by fun_prop
have hi : Integrable (fun z : Sample n m d × Real =>
(hybridEstimator n m d eps z.1 - ateFunctional P) ^ 2)
((annotationLaw P n m).prod seedLaw) := by
apply Integrable.of_bound hm.aestronglyMeasurable 4
filter_upwards with z
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hc := hybridEstimator_mem_Icc n m d eps z.1
have ht := ateFunctional_mem_Icc P
nlinarith [hc.1, hc.2, ht.1, ht.2]
unfold ruleRisk liftRule sqRisk
rw [integral_prod _ hi]
simp
theorem hybrid_ruleRisk_le_capped_array_risk
Under the stated inputs and conditions, The original rule risk is bounded by its unaveraged, zero-overflow prefix risk. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ sqRisk ((annotationLaw P n m).prod (hybridRequestLaw n m eps)) (fun z => hybridCappedArrayStatistic eps z.1 z.2) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_ruleRisk_le_capped_array_risk
lemma hybrid_ruleRisk_le_capped_array_risk {n m d : Nat} (eps : Real) (P : DiscreteLaw d)
(hS : exp 4096 ≤ (n : Real) * eps) :
ruleRisk (liftRule (hybridEstimator n m d eps)) P ≤
sqRisk ((annotationLaw P n m).prod (hybridRequestLaw n m eps))
(fun z => hybridCappedArrayStatistic eps z.1 z.2) (ateFunctional P) := by
rw [hybrid_ruleRisk_eq_sqRisk]
exact hybrid_averaging_sq_risk_le eps P hS
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridFiniteRate 9 declarations Finite-pool size, intensity, overflow, and logarithmic rate calibration for the hybrid upper bound.
Finite-pool size, intensity, overflow, and logarithmic rate calibration for the hybrid upper bound.
Under the stated inputs and conditions, The large-information branch has at least twelve complete records. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_large_sample_size
lemma hybrid_large_sample_size (n : Nat) (eps : Real)
(heps : eps ≤ 1 / 4) (hS : exp 4096 ≤ (n : Real) * eps) :
12 ≤ n := by
have hexp := Real.add_one_le_exp (4096 : Real)
have hn : (0 : Real) ≤ n := Nat.cast_nonneg n
have hlarge : (12 : Real) ≤ n := by nlinarith
exact_mod_cast hlarge
Under the stated inputs and conditions, The prescribed splits retain positive pools and comparable marginal pool lengths. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
n ≤ 4 * (hybridTuning n m eps).h0
conclusion 2
n + m ≤ 8 * (hybridTuning n m eps).Mp
conclusion 3
n + m ≤ 8 * (hybridTuning n m eps).Mf
conclusion 4
1 ≤ (hybridTuning n m eps).h0
conclusion 5
1 ≤ (hybridTuning n m eps).Mp
conclusion 6
1 ≤ (hybridTuning n m eps).Mf
conclusion 7
(hybridTuning n m eps).Mp ≤ (hybridTuning n m eps).Mf
conclusion 8
(hybridTuning n m eps).Mf ≤ 3 * (hybridTuning n m eps).Mp
Proof (Lean source)
-- @node: hybrid_finite_pool_sizes
lemma hybrid_finite_pool_sizes (n m : Nat) (eps : Real) (hn : 12 ≤ n) :
n ≤ 4 * (hybridTuning n m eps).h0 ∧
n + m ≤ 8 * (hybridTuning n m eps).Mp ∧
n + m ≤ 8 * (hybridTuning n m eps).Mf ∧
1 ≤ (hybridTuning n m eps).h0 ∧
1 ≤ (hybridTuning n m eps).Mp ∧
1 ≤ (hybridTuning n m eps).Mf ∧
(hybridTuning n m eps).Mp ≤ (hybridTuning n m eps).Mf ∧
(hybridTuning n m eps).Mf ≤ 3 * (hybridTuning n m eps).Mp := by
dsimp [hybridTuning]
omega
Under the stated inputs and conditions, Pool intensities satisfy exactly the hypotheses of the uniform hybrid arm theorem. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
(n : Real) / 32 ≤ (hybridTuning n m eps).u
conclusion 2
((n : Real) + m) / 64 ≤ (hybridTuning n m eps).tp
conclusion 3
((n : Real) + m) / 64 ≤ (hybridTuning n m eps).t
conclusion 4
0 < (hybridTuning n m eps).tp
conclusion 5
1 / 3 ≤ (hybridTuning n m eps).t / (hybridTuning n m eps).tp
conclusion 6
(hybridTuning n m eps).t / (hybridTuning n m eps).tp ≤ 3
Proof (Lean source)
-- @node: hybrid_finite_pool_intensities
lemma hybrid_finite_pool_intensities (n m : Nat) (eps : Real) (hn : 12 ≤ n) :
(n : Real) / 32 ≤ (hybridTuning n m eps).u ∧
((n : Real) + m) / 64 ≤ (hybridTuning n m eps).tp ∧
((n : Real) + m) / 64 ≤ (hybridTuning n m eps).t ∧
0 < (hybridTuning n m eps).tp ∧
1 / 3 ≤ (hybridTuning n m eps).t / (hybridTuning n m eps).tp ∧
(hybridTuning n m eps).t / (hybridTuning n m eps).tp ≤ 3 := by
obtain ⟨h0, hp, hf, _, hppos, _, hpf, hfp⟩ := hybrid_finite_pool_sizes n m eps hn
have h0' : (n : Real) ≤ 4 * (hybridTuning n m eps).h0 := by exact_mod_cast h0
have hp' : (n : Real) + m ≤ 8 * (hybridTuning n m eps).Mp := by exact_mod_cast hp
have hf' : (n : Real) + m ≤ 8 * (hybridTuning n m eps).Mf := by exact_mod_cast hf
have hpos : (0 : Real) < (hybridTuning n m eps).Mp := by exact_mod_cast hppos
have hpf' : ((hybridTuning n m eps).Mp : Real) ≤ (hybridTuning n m eps).Mf := by
exact_mod_cast hpf
have hfp' : ((hybridTuning n m eps).Mf : Real) ≤ 3 * (hybridTuning n m eps).Mp := by
exact_mod_cast hfp
have htp : 0 < (hybridTuning n m eps).tp := by dsimp [hybridTuning] at hpos ⊢; positivity
refine ⟨?_, ?_, ?_, htp, (le_div_iff₀ htp).2 ?_, (div_le_iff₀ htp).2 ?_⟩ <;>
dsimp [hybridTuning] at * <;> linarith
Under the stated inputs and conditions, Each positive capacity has an exponential overflow cost bounded by its reciprocal. This gives the stated result.
Proof (Lean source)
-- @node: hybrid_capacity_exp_bound
lemma hybrid_capacity_exp_bound (h : Nat) (hh : 0 < h) :
exp (-(h : Real)) ≤ 1 / (h : Real) := by
have hp : (0 : Real) < h := by exact_mod_cast hh
rw [Real.exp_neg, one_div]
exact (inv_le_inv₀ (Real.exp_pos _) hp).2
(by linarith [Real.add_one_le_exp (h : Real)])
Under the stated inputs and conditions, The three overflow costs are absorbed by the rare-label inverse-information rate. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ 5 / ((n : Real) * eps)
Proof (Lean source)
-- @node: hybrid_finite_overflow_rate
lemma hybrid_finite_overflow_rate (n m : Nat) (eps : Real) (hn : 12 ≤ n)
(heps : 0 < eps) (heps4 : eps ≤ 1 / 4) :
exp (-((hybridTuning n m eps).h0 : Real)) +
exp (-((hybridTuning n m eps).Mp : Real)) +
exp (-((hybridTuning n m eps).Mf : Real)) ≤
5 / ((n : Real) * eps) := by
obtain ⟨h0, hp, hf, h0pos, hppos, hfpos, _, _⟩ := hybrid_finite_pool_sizes n m eps hn
have hnpos : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hcap (h : Nat) (hh : 1 ≤ h) (hb : n ≤ 8 * h) :
exp (-(h : Real)) ≤ 8 / (n : Real) := by
have hhpos : (0 : Real) < h := by exact_mod_cast hh
have hb' : (n : Real) ≤ 8 * h := by exact_mod_cast hb
exact (hybrid_capacity_exp_bound h (by omega)).trans
((div_le_div_iff₀ hhpos hnpos).2 (by nlinarith))
have hzero : exp (-((hybridTuning n m eps).h0 : Real)) ≤ 4 / (n : Real) := by
have hhpos : (0 : Real) < (hybridTuning n m eps).h0 := by exact_mod_cast h0pos
have hb : (n : Real) ≤ 4 * (hybridTuning n m eps).h0 := by exact_mod_cast h0
exact (hybrid_capacity_exp_bound _ (by omega)).trans
((div_le_div_iff₀ hhpos hnpos).2 (by nlinarith))
have hpilot := hcap _ hppos (by omega)
have hfactorial := hcap _ hfpos (by omega)
have hrate : 20 / (n : Real) ≤ 5 / ((n : Real) * eps) := by
apply (div_le_div_iff₀ hnpos (mul_pos hnpos heps)).2
nlinarith
calc
_ ≤ 4 / (n : Real) + 8 / (n : Real) + 8 / (n : Real) :=
add_le_add (add_le_add hzero hpilot) hfactorial
_ = 20 / (n : Real) := by ring
_ ≤ _ := hrate
Under the stated inputs and conditions, The degree calibrated to log S also controls the frontier logarithm. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_frontier_log_degree
lemma hybrid_frontier_log_degree (S : Real) (hS : exp 4096 ≤ S) :
log (exp 1 + S) ≤
1281 * (floor (log S / 1024) : Real) := by
obtain ⟨hL, hy, _, _⟩ := hybrid_degree_calibration S hS
have hSp : 0 < S := (Real.exp_pos _).trans_le hS
have hStwo : 2 ≤ S := by linarith [Real.add_one_le_exp (4096 : Real)]
have hetwo : 2 ≤ exp 1 := by linarith [Real.add_one_le_exp (1 : Real)]
have hlog : log (exp 1 + S) ≤ log S + 1 := by
apply (Real.log_le_iff_le_exp (by positivity)).2
rw [Real.exp_add, Real.exp_log hSp]
nlinarith
have hL' : (4 : Real) ≤ floor (log S / 1024) := by exact_mod_cast hL
linarith
theorem hybrid_finite_degree_rate
Under the stated inputs and conditions, Converting from the polynomial degree to the frontier logarithm costs a numerical factor. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_finite_degree_rate
lemma hybrid_finite_degree_rate (n m d : Nat) (eps : Real)
(heps : 0 < eps) (hS : exp 4096 ≤ (n : Real) * eps) :
1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * (hybridTuning n m eps).L)) ^ 2 ≤
(1281 : Real) ^ 2 * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) := by
have hSp : 0 < (n : Real) * eps := (Real.exp_pos _).trans_le hS
have hn : (0 : Real) < n := by
by_contra hh
have hz : (n : Real) = 0 := le_antisymm (le_of_not_gt hh) (Nat.cast_nonneg n)
simp [hz] at hSp
have hN : 0 < ((n : Real) + m) * eps :=
mul_pos (by linarith [(Nat.cast_nonneg m : (0 : Real) ≤ m)]) heps
have hL := (hybrid_degree_calibration _ hS).1
have hLp : (0 : Real) < (hybridTuning n m eps).L := by
dsimp [hybridTuning]
exact_mod_cast (show 0 < floor (log ((n : Real) * eps) / 1024) by omega)
have hell : 0 < logScale n eps := by
dsimp [logScale, labelScale]
apply Real.log_pos
linarith [Real.add_one_le_exp (1 : Real)]
have hlog : logScale n eps ≤ 1281 * (hybridTuning n m eps).L :=
hybrid_frontier_log_degree _ hS
have hratio : (d : Real) / (((n : Real) + m) * eps * (hybridTuning n m eps).L) ≤
1281 * ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) := by
rw [← mul_div_assoc]
apply (div_le_div_iff₀ (mul_pos hN hLp) (mul_pos hN hell)).2
convert mul_le_mul_of_nonneg_left hlog
(mul_nonneg (Nat.cast_nonneg d) hN.le) using 1 <;> ring
have hsquare := mul_self_le_mul_self (by positivity :
0 ≤ (d : Real) / (((n : Real) + m) * eps * (hybridTuning n m eps).L)) hratio
nlinarith [one_div_pos.mpr hSp]
theorem hybrid_small_information_risk
Under the stated inputs and conditions, The exact zero branch has risk at most one. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_small_information_risk
lemma hybrid_small_information_risk (n m d : Nat) (eps : Real) (P : DiscreteLaw d)
(hsmall : (n : Real) * eps < exp 4096) :
ruleRisk (liftRule (hybridEstimator n m d eps)) P ≤ 1 := by
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by
norm_num [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have ht := ateFunctional_mem_Icc P
have hrisk : ruleRisk (liftRule (hybridEstimator n m d eps)) P =
(ateFunctional P) ^ 2 := by
simp [ruleRisk, liftRule, hybridEstimator, hsmall, integral_const,
Measure.real_def]
rw [hrisk]
nlinarith [ht.1, ht.2]
theorem hybrid_small_information_rate
Under the stated inputs and conditions, The frontier rate absorbs the zero-branch risk with a universal numerical constant. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_small_information_rate
lemma hybrid_small_information_rate (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps) (hsmall : (n : Real) * eps < exp 4096) :
1 ≤ exp 4096 * frontierRate n m d eps := by
have hnpos : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hSp := mul_pos hnpos heps
have hinv : 1 / exp 4096 ≤ 1 / ((n : Real) * eps) :=
one_div_le_one_div_of_le hSp hsmall.le
have hunit : 1 / exp 4096 ≤ 1 := by
apply (div_le_iff₀ (Real.exp_pos _)).2
linarith [Real.add_one_le_exp (4096 : Real)]
have hrate : 1 / exp 4096 ≤ frontierRate n m d eps :=
le_min hunit (hinv.trans (le_add_of_nonneg_right (sq_nonneg _)))
exact (div_le_iff₀ (Real.exp_pos _)).mp hrate |>.trans_eq (mul_comm _ _)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridFiniteTransfer 17 declarations The three independent hybrid pools as a finite family, and exact finite-prefix risk transfer.
The three independent hybrid pools as a finite family, and exact finite-prefix risk transfer.
Three singleton coordinates distinguish the complete, pilot, and factorial pools.
def hybridTripleEquiv
Evaluation identifies a three-coordinate dependent family with its ordered triple.
Definition (Lean source)
parameters
X :
HybridPoolIndex → Type
∀ i, MeasurableSpace (X i)
def
hybridTripleEquiv X :
(∀ i, X i) ≃ᵐ (X (.inl ()) × X (.inr (.inl ())) × X (.inr (.inr ())))
given by
clause 1
toFun s := (s (.inl ()), s (.inr (.inl ())), s (.inr (.inr ())))
clause 2
invFun s := fun i
=> match i with | .inl () => s.1 | .inr (.inl ()) => s.2.1 | .inr (.inr ()) => s.2.2
clause 3
left_inv s := by funext i; rcases i with ⟨⟨⟩⟩ | ⟨⟨⟨⟩⟩ | ⟨⟨⟩⟩⟩ <;> rfl
clause 4
right_inv s := rfl
clause 5
measurable_toFun := by change Measurable (fun s : (∀ i, X i) => (s (.inl ()), s (.inr (.inl ())), s (.inr (.inr ())))) fun_prop
clause 6
measurable_invFun := by apply measurable_pi_lambda intro i rcases i with ⟨⟨⟩⟩ | ⟨⟨⟨⟩⟩ | ⟨⟨⟩⟩⟩ <;> dsimp <;> fun_prop
theorem hybrid_triple_pi_law
Under the stated inputs and conditions, The triple equivalence preserves the independent product law. This gives the stated result.
Formal statement
hypotheses
X :
HybridPoolIndex → Type
∀ i, MeasurableSpace (X i)
mu :
∀ i, Measure (X i)
∀ i, SigmaFinite (mu i)
conclusion
MeasurePreserving (hybridTripleEquiv X) (Measure.pi mu) ((mu (.inl ())).prod ((mu (.inr (.inl ()))).prod (mu (.inr (.inr ())))))
Proof (Lean source)
-- @node: hybrid_triple_pi_law
lemma hybrid_triple_pi_law (X : HybridPoolIndex → Type) [∀ i, MeasurableSpace (X i)]
(mu : ∀ i, Measure (X i)) [∀ i, SigmaFinite (mu i)] :
MeasurePreserving (hybridTripleEquiv X) (Measure.pi mu)
((mu (.inl ())).prod ((mu (.inr (.inl ()))).prod (mu (.inr (.inr ()))))) := by
have hsplit := measurePreserving_sumPiEquivProdPi mu
have hinner := measurePreserving_sumPiEquivProdPi (fun i => mu (.inr i))
have hu0 := measurePreserving_piUnique (fun i : Unit => mu (.inl i))
have hu1 := measurePreserving_piUnique (fun i : Unit => mu (.inr (.inl i)))
have hu2 := measurePreserving_piUnique (fun i : Unit => mu (.inr (.inr i)))
convert (hu0.prod ((hu1.prod hu2).comp hinner)).comp hsplit using 1
funext s
rfl
def HybridPoolAlphabet
The alphabet family retains the two marginal pools as different coordinates.
Definition (Lean source)
parameters
d :
def
instance hybridPoolAlphabet_measurableSpace
Each coordinate inherits its finite observation alphabet's measurable structure.
Definition (Lean source)
-- @node: hybridPoolAlphabet_measurableSpace
instance hybridPoolAlphabet_measurableSpace (d : Nat) :
∀ i, MeasurableSpace (HybridPoolAlphabet d i)
| .inl () => (inferInstance : MeasurableSpace (Obs d))
| .inr (.inl ()) => (inferInstance : MeasurableSpace (AuxObs d))
| .inr (.inr ()) => (inferInstance : MeasurableSpace (AuxObs d))
def hybridPoolObservationLaw
The observation-law family assigns the complete law and its two marginal copies.
Definition (Lean source)
parameters
def
given by
clause 1
| .inl () => obsLaw P
clause 2
| .inr (.inl ()) => (auxMarginal P).toMeasure
clause 3
| .inr (.inr ()) => (auxMarginal P).toMeasure
instance hybridPoolObservationLaw_probability
Each pool's observation law is a probability measure.
Definition (Lean source)
-- @node: hybridPoolObservationLaw_probability
instance hybridPoolObservationLaw_probability {d : Nat} (P : DiscreteLaw d) :
∀ i, IsProbabilityMeasure (hybridPoolObservationLaw P i)
| .inl () => by dsimp [hybridPoolObservationLaw, obsLaw]; infer_instance
| .inr (.inl ()) => by dsimp [hybridPoolObservationLaw]; infer_instance
| .inr (.inr ()) => by dsimp [hybridPoolObservationLaw]; infer_instance
def hybridPoolCapacity
The family capacities are exactly the three public hybrid pool lengths.
Definition (Lean source)
parameters
def
given by
fun i
=> match i with | .inl () => (hybridTuning n m eps).h0 | .inr (.inl ()) => (hybridTuning n m eps).Mp | .inr (.inr ()) => (hybridTuning n m eps).Mf
def hybridFamilyStatistic
Prefix-family readback computes the same ordered clipped hybrid contrast.
Definition (Lean source)
parameters
def
given by
hybridOrderedPrefixStatistic (hybridTuning n m eps) ∘ hybridTripleEquiv (fun i => FiniteSample (HybridPoolAlphabet d i))
theorem hybridFamilyStatistic_measurable
Prefix-family readback is measurable. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: hybridFamilyStatistic_measurable
lemma hybridFamilyStatistic_measurable (n m d : Nat) (eps : Real) :
Measurable (hybridFamilyStatistic n m d eps) :=
(hybridOrderedPrefixStatistic_measurable d _).comp (hybridTripleEquiv _).measurable
theorem hybridFamilyStatistic_mem_Icc
Under the stated inputs and conditions, The family statistic retains the clipped interval at every prefix. This gives the stated result.
Formal statement
hypotheses
conclusion
hybridFamilyStatistic n m d eps s ∈ Icc (-1) 1
Proof (Lean source)
-- @node: hybridFamilyStatistic_mem_Icc
lemma hybridFamilyStatistic_mem_Icc (n m d : Nat) (eps : Real)
(s : PrefixFamily (HybridPoolAlphabet d)) :
hybridFamilyStatistic n m d eps s ∈ Icc (-1) 1 :=
hybridOrderedPrefixStatistic_mem_Icc _ _
theorem hybrid_family_prefix_risk_eq
Under the stated inputs and conditions, The ideal prefix-family risk is exactly the already established triple-prefix risk. This gives the stated result.
Formal statement
hypotheses
conclusion
sqRisk (independentPoissonPrefixLaw (hybridPoolObservationLaw P) (fun i => (hybridPoolCapacity n m eps i : NNReal) / 8)) (hybridFamilyStatistic n m d eps) (ateFunctional P)
= sqRisk (hybridPoissonPrefixLaw P ((hybridTuning n m eps).h0 / 8 : NNReal) ((hybridTuning n m eps).Mp / 8 : NNReal) ((hybridTuning n m eps).Mf / 8 : NNReal)) (hybridOrderedPrefixStatistic (hybridTuning n m eps)) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_family_prefix_risk_eq
lemma hybrid_family_prefix_risk_eq (n m d : Nat) (eps : Real) (P : DiscreteLaw d) :
sqRisk
(independentPoissonPrefixLaw (hybridPoolObservationLaw P)
(fun i => (hybridPoolCapacity n m eps i : NNReal) / 8))
(hybridFamilyStatistic n m d eps) (ateFunctional P) =
sqRisk
(hybridPoissonPrefixLaw P ((hybridTuning n m eps).h0 / 8 : NNReal)
((hybridTuning n m eps).Mp / 8 : NNReal)
((hybridTuning n m eps).Mf / 8 : NNReal))
(hybridOrderedPrefixStatistic (hybridTuning n m eps)) (ateFunctional P) := by
have hmap := hybrid_triple_pi_law (fun i => FiniteSample (HybridPoolAlphabet d i))
(fun i => finitePoissonSampleLaw (hybridPoolObservationLaw P i)
((hybridPoolCapacity n m eps i : NNReal) / 8))
unfold sqRisk hybridFamilyStatistic
exact hmap.integral_comp' (fun s =>
(hybridOrderedPrefixStatistic (hybridTuning n m eps) s - ateFunctional P) ^ 2)
theorem hybrid_family_fixed_pools_law
Under the stated inputs and conditions, Extracting and packing the original three pools gives the finite-family iid law. This gives the stated result.
Formal statement
hypotheses
conclusion
MeasurePreserving ((hybridTripleEquiv (fun i => Fin (hybridPoolCapacity n m eps i) → HybridPoolAlphabet d i)).symm ∘ hybridFixedPools eps) (annotationLaw P n m) (fixedPoolsLaw (hybridPoolObservationLaw P) (hybridPoolCapacity n m eps))
Proof (Lean source)
-- @node: hybrid_family_fixed_pools_law
lemma hybrid_family_fixed_pools_law (n m d : Nat) (eps : Real) (P : DiscreteLaw d) :
MeasurePreserving
((hybridTripleEquiv (fun i => Fin (hybridPoolCapacity n m eps i) →
HybridPoolAlphabet d i)).symm ∘ hybridFixedPools eps)
(annotationLaw P n m)
(fixedPoolsLaw (hybridPoolObservationLaw P) (hybridPoolCapacity n m eps)) := by
have hpi := hybrid_triple_pi_law
(fun i => Fin (hybridPoolCapacity n m eps i) → HybridPoolAlphabet d i)
(fun i => Measure.pi (fun _ : Fin (hybridPoolCapacity n m eps i) =>
hybridPoolObservationLaw P i))
exact hpi.symm.comp (hybrid_fixed_pools_law eps P)
theorem hybrid_family_capped_statistic_eq
Under the stated inputs and conditions, Valid requests have identical family and triple readback; overflow resets both to zero. This gives the stated result.
Formal statement
hypotheses
conclusion
cappedPrefixStatistic (hybridFamilyStatistic n m d eps) 0 (fun i => (s i, k i))
Proof (Lean source)
-- @node: hybrid_family_capped_statistic_eq
lemma hybrid_family_capped_statistic_eq (n m d : Nat) (eps : Real)
(s : FixedPools (HybridPoolAlphabet d) (hybridPoolCapacity n m eps))
(k : HybridPoolIndex → Nat) :
cappedPrefixStatistic (hybridFamilyStatistic n m d eps) 0 (fun i => (s i, k i)) =
hybridCappedFixedStatistic eps (hybridTripleEquiv _ s) (hybridTripleEquiv _ k) := by
have hv : (∀ i, k i ≤ hybridPoolCapacity n m eps i) ↔
k (.inl ()) ≤ (hybridTuning n m eps).h0 ∧
k (.inr (.inl ())) ≤ (hybridTuning n m eps).Mp ∧
k (.inr (.inr ())) ≤ (hybridTuning n m eps).Mf := by
constructor
· intro h; exact ⟨h (.inl ()), h (.inr (.inl ())), h (.inr (.inr ()))⟩
· rintro ⟨h0, hp, hf⟩ i
rcases i with ⟨⟨⟩⟩ | ⟨⟨⟨⟩⟩ | ⟨⟨⟩⟩⟩
· exact h0
· exact hp
· exact hf
by_cases hk : ∀ i, k i ≤ hybridPoolCapacity n m eps i
· rw [cappedPrefixStatistic, dif_pos hk]
change _ = if h : k (.inl ()) ≤ (hybridTuning n m eps).h0 ∧
k (.inr (.inl ())) ≤ (hybridTuning n m eps).Mp ∧
k (.inr (.inr ())) ≤ (hybridTuning n m eps).Mf then _ else 0
rw [dif_pos (hv.mp hk)]
rfl
· have hn := mt hv.mpr hk
rw [cappedPrefixStatistic, dif_neg hk]
change 0 = if h : k (.inl ()) ≤ (hybridTuning n m eps).h0 ∧
k (.inr (.inl ())) ≤ (hybridTuning n m eps).Mp ∧
k (.inr (.inr ())) ≤ (hybridTuning n m eps).Mf then _ else 0
rw [dif_neg hn]
theorem hybrid_estimator_eq_family_average
Under the stated hypotheses, The estimator is precisely the finite-family conditional prefix average. This gives the stated result.
Formal statement
hypotheses
conclusion
hybridEstimator n m d eps s
= prefixRaoBlackwellStatistic (fun i => (hybridPoolCapacity n m eps i : NNReal) / 8) (hybridFamilyStatistic n m d eps) 0 ((hybridTripleEquiv (fun i => Fin (hybridPoolCapacity n m eps i) → HybridPoolAlphabet d i)).symm (hybridFixedPools eps s))
Proof (Lean source)
-- @node: hybrid_estimator_eq_family_average
lemma hybrid_estimator_eq_family_average (n m d : Nat) (eps : Real) (s : Sample n m d)
(hS : exp 4096 ≤ (n : Real) * eps) :
hybridEstimator n m d eps s =
prefixRaoBlackwellStatistic (fun i => (hybridPoolCapacity n m eps i : NNReal) / 8)
(hybridFamilyStatistic n m d eps) 0
((hybridTripleEquiv (fun i => Fin (hybridPoolCapacity n m eps i) →
HybridPoolAlphabet d i)).symm (hybridFixedPools eps s)) := by
rw [hybrid_estimator_eq_poisson_integral eps s hS,
prefixRaoBlackwellStatistic_eq_integral _ _
(hybridFamilyStatistic_measurable n m d eps) 0]
have hcount := hybrid_triple_pi_law (fun _ => Nat)
(fun i => poissonMeasure ((hybridPoolCapacity n m eps i : NNReal) / 8))
have hm : Measurable (fun k : Nat × Nat × Nat =>
hybridCappedFixedStatistic eps (hybridFixedPools eps s) k) := by fun_prop
have hi := hcount.integral_comp' (fun k : Nat × Nat × Nat =>
hybridCappedFixedStatistic eps (hybridFixedPools eps s) k)
change (∫ k, hybridCappedFixedStatistic eps (hybridFixedPools eps s)
(hybridTripleEquiv (fun _ => Nat) k) ∂Measure.pi
(fun i => poissonMeasure ((hybridPoolCapacity n m eps i : NNReal) / 8))) =
(∫ k, hybridCappedFixedStatistic eps (hybridFixedPools eps s) k
∂hybridRequestLaw n m eps) at hi
simp_rw [hybrid_capped_array_eq_fixed]
rw [← hi]
apply integral_congr_ae
filter_upwards with k
rw [hybrid_family_capped_statistic_eq]
simp only [MeasurableEquiv.apply_symm_apply]
theorem hybrid_ruleRisk_eq_family_average_risk
Under the stated hypotheses, Original-record risk equals the finite-family Rao--Blackwell risk. This gives the stated result.
Formal statement
hypotheses
conclusion
= sqRisk (fixedPoolsLaw (hybridPoolObservationLaw P) (hybridPoolCapacity n m eps)) (prefixRaoBlackwellStatistic (fun i => (hybridPoolCapacity n m eps i : NNReal) / 8) (hybridFamilyStatistic n m d eps) 0) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_ruleRisk_eq_family_average_risk
lemma hybrid_ruleRisk_eq_family_average_risk (n m d : Nat) (eps : Real)
(P : DiscreteLaw d) (hS : exp 4096 ≤ (n : Real) * eps) :
ruleRisk (liftRule (hybridEstimator n m d eps)) P =
sqRisk
(fixedPoolsLaw (hybridPoolObservationLaw P) (hybridPoolCapacity n m eps))
(prefixRaoBlackwellStatistic
(fun i => (hybridPoolCapacity n m eps i : NNReal) / 8)
(hybridFamilyStatistic n m d eps) 0) (ateFunctional P) := by
rw [hybrid_ruleRisk_eq_sqRisk]
unfold sqRisk
simp_rw [hybrid_estimator_eq_family_average n m d eps _ hS]
have hmap := hybrid_family_fixed_pools_law n m d eps P
rw [← hmap.map_eq]
exact (integral_map hmap.measurable.aemeasurable
(show AEStronglyMeasurable (fun x =>
(prefixRaoBlackwellStatistic
(fun i => (hybridPoolCapacity n m eps i : NNReal) / 8)
(hybridFamilyStatistic n m d eps) 0 x - ateFunctional P) ^ 2) _ from
(by fun_prop : Measurable _).aestronglyMeasurable)).symm
theorem hybrid_large_information_risk
Under the stated inputs and conditions, Finite-prefix averaging transfers the exact-tuning bound to the original records. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
Proof (Lean source)
-- @node: hybrid_large_information_risk
lemma hybrid_large_information_risk :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real) (P : DiscreteLaw d),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
exp 4096 ≤ (n : Real) * eps → ModelClass d eps P →
ruleRisk (liftRule (hybridEstimator n m d eps)) P ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) := by
obtain ⟨C, hC, hbound⟩ := hybrid_tuned_prefix_and_overflow_rate
refine ⟨C, hC, ?_⟩
intro n m d eps P hn hd heps heps4 hS hP
obtain ⟨_, _, _, h0, hp, hf, _, _⟩ := hybrid_finite_pool_sizes n m eps
(hybrid_large_sample_size n eps heps4 hS)
have hcap : ∀ i, 1 ≤ hybridPoolCapacity n m eps i := by
intro i
rcases i with ⟨⟨⟩⟩ | ⟨⟨⟨⟩⟩ | ⟨⟨⟩⟩⟩
· exact h0
· exact hp
· exact hf
have ht := (finite_prefix_transfer (hybridPoolObservationLaw P)
(hybridPoolCapacity n m eps) hcap
(hybridFamilyStatistic_measurable n m d eps)
(hybridFamilyStatistic_mem_Icc n m d eps)
(ateFunctional_mem_Icc P) le_rfl).2.2
rw [← hybrid_ruleRisk_eq_family_average_risk n m d eps P hS,
hybrid_family_prefix_risk_eq] at ht
have hsum : (∑ i : HybridPoolIndex, exp (-(hybridPoolCapacity n m eps i : Real))) =
exp (-((hybridTuning n m eps).h0 : Real)) +
exp (-((hybridTuning n m eps).Mp : Real)) +
exp (-((hybridTuning n m eps).Mf : Real)) := by
simp [Fintype.sum_sum_type, hybridPoolCapacity, add_assoc]
rw [hsum] at ht
exact ht.trans (hbound n m d eps P hn hd heps heps4 hS hP)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridPoolLaw 17 declarations Exact independent laws of the three disjoint original-record hybrid pools.
Exact independent laws of the three disjoint original-record hybrid pools.
Under the stated hypotheses, Three consecutive blocks of an iid array retain their independent iid laws. This gives the stated result.
Formal statement
hypotheses
conclusion
MeasurePreserving (fun s : Fin n → X => (fun i : Fin h => s ⟨i.val, by omega⟩, fun i : Fin p => s ⟨h + i.val, by omega⟩, fun i : Fin f => s ⟨h + p + i.val, by omega⟩)) (Measure.pi (fun _ : Fin n => P)) ((Measure.pi (fun _ : Fin h => P)).prod ((Measure.pi (fun _ : Fin p => P)).prod (Measure.pi (fun _ : Fin f => P))))
Proof (Lean source)
-- @node: hybrid_split_three_iid_law
lemma hybrid_split_three_iid_law {X : Type*} [MeasurableSpace X]
(P : Measure X) [IsProbabilityMeasure P] (h p f n : Nat) (hn : n = h + (p + f)) :
MeasurePreserving
(fun s : Fin n → X =>
(fun i : Fin h => s ⟨i.val, by omega⟩,
fun i : Fin p => s ⟨h + i.val, by omega⟩,
fun i : Fin f => s ⟨h + p + i.val, by omega⟩))
(Measure.pi (fun _ : Fin n => P))
((Measure.pi (fun _ : Fin h => P)).prod
((Measure.pi (fun _ : Fin p => P)).prod (Measure.pi (fun _ : Fin f => P)))) := by
have hfirst := baseline_split_iid_law_of_eq P h (p + f) n hn
have hlast := (MeasurePreserving.id (Measure.pi (fun _ : Fin h => P))).prod
(baseline_split_iid_law P p f)
convert hlast.comp hfirst using 1
funext s
apply Prod.ext
· rfl
· apply Prod.ext
· rfl
· funext i
change s ⟨h + p + i.val, _⟩ = s ⟨h + (p + i.val), _⟩
exact congrArg s (Fin.ext (Nat.add_assoc h p i.val))
Under the stated inputs and conditions, Regroup four independent pools to pair the pilot blocks and the factorial blocks. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybrid_regroup_four_pools_law
lemma hybrid_regroup_four_pools_law {A B C D : Type*}
[MeasurableSpace A] [MeasurableSpace B] [MeasurableSpace C] [MeasurableSpace D]
(pa : Measure A) (pb : Measure B) (pc : Measure C) (pd : Measure D)
[IsProbabilityMeasure pa] [IsProbabilityMeasure pb]
[IsProbabilityMeasure pc] [IsProbabilityMeasure pd] :
MeasurePreserving (fun z : (A × B) × (C × D) => ((z.1.1,z.2.1),(z.1.2,z.2.2)))
((pa.prod pb).prod (pc.prod pd)) ((pa.prod pc).prod (pb.prod pd)) := by
have h1 := measurePreserving_prodAssoc pa pb (pc.prod pd)
have h2 := (MeasurePreserving.id pa).prod (measurePreserving_prodAssoc pb pc pd).symm
have h3 := (MeasurePreserving.id pa).prod
((Measure.measurePreserving_swap (μ := pb) (ν := pc)).prod (MeasurePreserving.id pd))
have h4 := (MeasurePreserving.id pa).prod (measurePreserving_prodAssoc pc pb pd)
have h5 := (measurePreserving_prodAssoc pa pc (pb.prod pd)).symm
exact h5.comp (h4.comp (h3.comp (h2.comp h1)))
Under the stated hypotheses, Rewriting a pool capacity preserves its iid law and ordered coordinates. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: hybrid_cast_iid_law
lemma hybrid_cast_iid_law {X : Type*} [MeasurableSpace X]
(P : Measure X) (h k : Nat) (hk : k = h) :
MeasurePreserving (fun s : Fin h → X => fun i : Fin k => s (cast hk i))
(Measure.pi (fun _ : Fin h => P)) (Measure.pi (fun _ : Fin k => P)) := by
subst k
exact MeasurePreserving.id _
Complete outcomes, pilot marginals, and factorial marginals use disjoint ordered blocks.
Definition (Lean source)
parameters
def
hybridFixedPools eps s :
(Fin (hybridTuning n m eps).h0 → Obs d) × (Fin (hybridTuning n m eps).Mp → AuxObs d) × (Fin (hybridTuning n m eps).Mf → AuxObs d)
given by
tun :
hybridTuning n m eps
(fun i => s.1 ⟨i.val, by dsimp [tun, hybridTuning] at i ⊢; omega⟩, append (fun i : Fin tun.hp => let z := s.1 ⟨tun.h0 + i.val, by dsimp [tun, hybridTuning] at i ⊢; omega⟩ (z.1,z.2.1)) (fun i : Fin (m / 2) => s.2 ⟨i.val, by omega⟩), fun i : Fin tun.Mf => (append (fun i : Fin tun.hf => let z := s.1 ⟨tun.h0 + tun.hp + i.val, by dsimp [tun, hybridTuning] at i ⊢; omega⟩ (z.1,z.2.1)) (fun i : Fin (m - m / 2) => s.2 ⟨m / 2 + i.val, by omega⟩)) (cast (by dsimp [tun, hybridTuning]; omega) i))
theorem hybridFixedPools_measurable
Extracting the three finite pools is measurable. This gives the stated conclusion.
Formal statement
hypotheses
conclusion
Measurable (hybridFixedPools (n := n) (m := m) (d := d) eps)
Proof (Lean source)
@[fun_prop]
-- @node: hybridFixedPools_measurable
lemma hybridFixedPools_measurable (n m d : Nat) (eps : Real) :
Measurable (hybridFixedPools (n := n) (m := m) (d := d) eps) := by
unfold hybridFixedPools
apply prodMk
· fun_prop
· apply prodMk
all_goals
apply measurable_pi_lambda
intro i
first
| refine Fin.addCases ?_ ?_ i
| refine Fin.addCases ?_ ?_ (cast _ i)
· intro j
simp only [Fin.append_left]
fun_prop
· intro j
simp only [Fin.append_right]
fun_prop
theorem hybrid_fixed_pools_law
Under the stated inputs and conditions, Under the original data law, the extracted hybrid pools are mutually independent. The two marginal pools concatenate projected complete records with auxiliary records. This gives the stated result.
Formal statement
hypotheses
conclusion
MeasurePreserving (hybridFixedPools (n := n) (m := m) eps) (annotationLaw P n m) ((labeledProductLaw P (hybridTuning n m eps).h0).prod ((auxProductLaw P (hybridTuning n m eps).Mp).prod (auxProductLaw P (hybridTuning n m eps).Mf)))
Proof (Lean source)
-- @node: hybrid_fixed_pools_law
lemma hybrid_fixed_pools_law {n m d : Nat} (eps : Real) (P : DiscreteLaw d) :
MeasurePreserving (hybridFixedPools (n := n) (m := m) eps)
(annotationLaw P n m)
((labeledProductLaw P (hybridTuning n m eps).h0).prod
((auxProductLaw P (hybridTuning n m eps).Mp).prod
(auxProductLaw P (hybridTuning n m eps).Mf))) := by
let tun := hybridTuning n m eps
let auxprob (k : Nat) : IsProbabilityMeasure (auxProductLaw P k) := by
unfold auxProductLaw
infer_instance
have hsplit := hybrid_split_three_iid_law (obsLaw P) tun.h0 tun.hp tun.hf n
(by dsimp [tun, hybridTuning]; omega)
have haux := baseline_split_iid_law_of_eq (auxMarginal P).toMeasure
(m / 2) (m - m / 2) m (by omega)
have hproj := (MeasurePreserving.id (labeledProductLaw P tun.h0)).prod
((baseline_project_iid_law P tun.hp).prod (baseline_project_iid_law P tun.hf))
have hfirst := (hproj.comp hsplit).prod haux
have hassoc := measurePreserving_prodAssoc (labeledProductLaw P tun.h0)
((auxProductLaw P tun.hp).prod (auxProductLaw P tun.hf))
((auxProductLaw P (m / 2)).prod (auxProductLaw P (m - m / 2)))
have hshuffle := (MeasurePreserving.id (labeledProductLaw P tun.h0)).prod
(hybrid_regroup_four_pools_law (auxProductLaw P tun.hp) (auxProductLaw P tun.hf)
(auxProductLaw P (m / 2)) (auxProductLaw P (m - m / 2)))
have happ := (MeasurePreserving.id (labeledProductLaw P tun.h0)).prod
((baseline_append_iid_law (auxMarginal P).toMeasure tun.hp (m / 2)).prod
(baseline_append_iid_law (auxMarginal P).toMeasure tun.hf (m - m / 2)))
have hcast : MeasurePreserving
(fun s : Fin (tun.hf + (m - m / 2)) → AuxObs d =>
fun i : Fin tun.Mf => s (cast (by dsimp [tun, hybridTuning]; omega) i))
(auxProductLaw P (tun.hf + (m - m / 2))) (auxProductLaw P tun.Mf) := by
exact hybrid_cast_iid_law (auxMarginal P).toMeasure _ _
(by dsimp [tun, hybridTuning]; omega)
have hlast := (MeasurePreserving.id (labeledProductLaw P tun.h0)).prod
((MeasurePreserving.id (auxProductLaw P tun.Mp)).prod hcast)
convert hlast.comp (happ.comp (hshuffle.comp (hassoc.comp hfirst))) using 1
· funext s
rfl
· rfl
theorem hybrid_fixed_pools_sqRisk
Under the stated inputs and conditions, Any measurable statistic of the three extracted pools has the independent-pool risk. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
P :
T :
((Fin (hybridTuning n m eps).h0 → Obs d) × (Fin (hybridTuning n m eps).Mp → AuxObs d) × (Fin (hybridTuning n m eps).Mf → AuxObs d)) → Real
hT :
theta :
conclusion
= sqRisk ((labeledProductLaw P (hybridTuning n m eps).h0).prod ((auxProductLaw P (hybridTuning n m eps).Mp).prod (auxProductLaw P (hybridTuning n m eps).Mf))) T theta
Proof (Lean source)
-- @node: hybrid_fixed_pools_sqRisk
lemma hybrid_fixed_pools_sqRisk {n m d : Nat} (eps : Real) (P : DiscreteLaw d)
(T : ((Fin (hybridTuning n m eps).h0 → Obs d) ×
(Fin (hybridTuning n m eps).Mp → AuxObs d) ×
(Fin (hybridTuning n m eps).Mf → AuxObs d)) → Real)
(hT : Measurable T) (theta : Real) :
sqRisk (annotationLaw P n m) (T ∘ hybridFixedPools eps) theta =
sqRisk
((labeledProductLaw P (hybridTuning n m eps).h0).prod
((auxProductLaw P (hybridTuning n m eps).Mp).prod
(auxProductLaw P (hybridTuning n m eps).Mf))) T theta := by
unfold sqRisk
rw [← (hybrid_fixed_pools_law eps P).map_eq]
exact (integral_map (hybridFixedPools_measurable n m d eps).aemeasurable
((hT.sub measurable_const).pow_const 2).aestronglyMeasurable).symm
theorem hybrid_fixed_complete_prefix_eq
Under the stated inputs and conditions, The outcome prefix keeps exactly the first requested complete records. This gives the stated result.
Formal statement
hypotheses
conclusion
prefixOfLE (hybridFixedPools eps s).1 r hr
= baselineBlockPrefix s.1 0 r (by dsimp [hybridTuning] at hr; omega)
Proof (Lean source)
-- @node: hybrid_fixed_complete_prefix_eq
lemma hybrid_fixed_complete_prefix_eq {n m d : Nat} (eps : Real) (s : Sample n m d)
(r : Nat) (hr : r ≤ (hybridTuning n m eps).h0) :
prefixOfLE (hybridFixedPools eps s).1 r hr =
baselineBlockPrefix s.1 0 r (by dsimp [hybridTuning] at hr; omega) := by
simp [prefixOfLE, hybridFixedPools, baselineBlockPrefix]
theorem hybrid_fixed_pilot_prefix_eq
Under the stated inputs and conditions, The pilot prefix uses the pilot complete block before its auxiliary block. This gives the stated result.
Formal statement
hypotheses
conclusion
prefixOfLE (hybridFixedPools eps s).2.1 r hr
= baselineAuxiliaryPrefix s (hybridTuning n m eps).h0 (hybridTuning n m eps).hp 0 r (by dsimp [hybridTuning]; omega) (by dsimp [hybridTuning] at hr ⊢; omega)
Proof (Lean source)
-- @node: hybrid_fixed_pilot_prefix_eq
lemma hybrid_fixed_pilot_prefix_eq {n m d : Nat} (eps : Real) (s : Sample n m d)
(r : Nat) (hr : r ≤ (hybridTuning n m eps).Mp) :
prefixOfLE (hybridFixedPools eps s).2.1 r hr =
baselineAuxiliaryPrefix s (hybridTuning n m eps).h0
(hybridTuning n m eps).hp 0 r
(by dsimp [hybridTuning]; omega)
(by dsimp [hybridTuning] at hr ⊢; omega) := by
dsimp [hybridFixedPools, hybridTuning] at hr ⊢
rw [baseline_prefix_append_eq]
simp only [baselineAuxiliaryPrefix, baselineBlockPrefix, FiniteSample.points,
FiniteSample.count, Nat.zero_add]
Under the stated hypotheses, Casting an array capacity does not change any retained ordered prefix. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_prefix_cast_eq
lemma hybrid_prefix_cast_eq {X : Type*} [MeasurableSpace X] {h k : Nat}
(hk : k = h) (s : Fin h → X) (r : Nat) (hr : r ≤ k) :
prefixOfLE (fun i : Fin k => s (cast hk i)) r hr =
prefixOfLE s r (by omega) := by
subst k
rfl
theorem hybrid_fixed_factorial_prefix_eq
Under the stated inputs and conditions, The factorial prefix uses the final complete block before the final auxiliary block. This gives the stated result.
Formal statement
hypotheses
conclusion
prefixOfLE (hybridFixedPools eps s).2.2 r hr
= baselineAuxiliaryPrefix s ((hybridTuning n m eps).h0 + (hybridTuning n m eps).hp) (hybridTuning n m eps).hf (m / 2) r (by dsimp [hybridTuning]; omega) (by dsimp [hybridTuning] at hr ⊢; omega)
Proof (Lean source)
-- @node: hybrid_fixed_factorial_prefix_eq
lemma hybrid_fixed_factorial_prefix_eq {n m d : Nat} (eps : Real) (s : Sample n m d)
(r : Nat) (hr : r ≤ (hybridTuning n m eps).Mf) :
prefixOfLE (hybridFixedPools eps s).2.2 r hr =
baselineAuxiliaryPrefix s
((hybridTuning n m eps).h0 + (hybridTuning n m eps).hp)
(hybridTuning n m eps).hf (m / 2) r
(by dsimp [hybridTuning]; omega)
(by dsimp [hybridTuning] at hr ⊢; omega) := by
dsimp only [hybridFixedPools]
rw [hybrid_prefix_cast_eq]
rw [baseline_prefix_append_eq]
rfl
theorem hybrid_fixed_prefix_statistic_eq
Under the stated inputs and conditions, Every retained triple computes the same clipped statistic in the array and pool forms. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
s :
Sample n m d
r0 rp rf :
hr0 :
r0 ≤ (hybridTuning n m eps).h0
hrp :
rp ≤ (hybridTuning n m eps).Mp
hrf :
rf ≤ (hybridTuning n m eps).Mf
conclusion
hybridPrefixStatistic eps r0 rp rf s
= hybridOrderedPrefixStatistic (hybridTuning n m eps) (prefixOfLE (hybridFixedPools eps s).1 r0 hr0, prefixOfLE (hybridFixedPools eps s).2.1 rp hrp, prefixOfLE (hybridFixedPools eps s).2.2 rf hrf)
Proof (Lean source)
-- @node: hybrid_fixed_prefix_statistic_eq
lemma hybrid_fixed_prefix_statistic_eq {n m d : Nat} (eps : Real) (s : Sample n m d)
(r0 rp rf : Nat)
(hr0 : r0 ≤ (hybridTuning n m eps).h0)
(hrp : rp ≤ (hybridTuning n m eps).Mp)
(hrf : rf ≤ (hybridTuning n m eps).Mf) :
hybridPrefixStatistic eps r0 rp rf s =
hybridOrderedPrefixStatistic (hybridTuning n m eps)
(prefixOfLE (hybridFixedPools eps s).1 r0 hr0,
prefixOfLE (hybridFixedPools eps s).2.1 rp hrp,
prefixOfLE (hybridFixedPools eps s).2.2 rf hrf) := by
rw [hybrid_fixed_complete_prefix_eq, hybrid_fixed_pilot_prefix_eq,
hybrid_fixed_factorial_prefix_eq]
exact hybrid_prefix_statistic_eq_ordered eps s r0 rp rf _ _ _ _ _
The independent fixed-pool statistic returns zero whenever any request overflows.
Definition (Lean source)
parameters
n m d :
eps :
s :
(Fin (hybridTuning n m eps).h0 → Obs d) × (Fin (hybridTuning n m eps).Mp → AuxObs d) × (Fin (hybridTuning n m eps).Mf → AuxObs d)
def
given by
if hk : k.1 ≤ (hybridTuning n m eps).h0 ∧
k.2.1 ≤ (hybridTuning n m eps).Mp ∧
k.2.2
≤ (hybridTuning n m eps).Mf then hybridOrderedPrefixStatistic (hybridTuning n m eps) (prefixOfLE s.1 k.1 hk.1, prefixOfLE s.2.1 k.2.1 hk.2.1, prefixOfLE s.2.2 k.2.2 hk.2.2) else 0
theorem hybrid_capped_array_eq_fixed
Under the stated inputs and conditions, Valid requests agree by count readback, and both forms reset overflow to zero. This gives the stated result.
Formal statement
hypotheses
conclusion
hybridCappedArrayStatistic eps s k
= hybridCappedFixedStatistic eps (hybridFixedPools eps s) k
Proof (Lean source)
-- @node: hybrid_capped_array_eq_fixed
lemma hybrid_capped_array_eq_fixed {n m d : Nat} (eps : Real) (s : Sample n m d)
(k : Nat × Nat × Nat) :
hybridCappedArrayStatistic eps s k =
hybridCappedFixedStatistic eps (hybridFixedPools eps s) k := by
by_cases hk : k.1 ≤ (hybridTuning n m eps).h0 ∧
k.2.1 ≤ (hybridTuning n m eps).Mp ∧ k.2.2 ≤ (hybridTuning n m eps).Mf
· simp only [hybridCappedArrayStatistic, hybridCappedFixedStatistic, hk, ite_true, dif_pos]
exact hybrid_fixed_prefix_statistic_eq eps s k.1 k.2.1 k.2.2 hk.1 hk.2.1 hk.2.2
· simp [hybridCappedArrayStatistic, hybridCappedFixedStatistic, hk]
theorem hybrid_fixed_pools_request_law
Under the stated inputs and conditions, Extraction of disjoint pools preserves the independent request law as well. This gives the stated result.
Formal statement
hypotheses
conclusion
MeasurePreserving (map (hybridFixedPools (n := n) (m := m) eps) id) ((annotationLaw P n m).prod (hybridRequestLaw n m eps)) (((labeledProductLaw P (hybridTuning n m eps).h0).prod ((auxProductLaw P (hybridTuning n m eps).Mp).prod (auxProductLaw P (hybridTuning n m eps).Mf))).prod (hybridRequestLaw n m eps))
Proof (Lean source)
-- @node: hybrid_fixed_pools_request_law
lemma hybrid_fixed_pools_request_law {n m d : Nat} (eps : Real) (P : DiscreteLaw d) :
MeasurePreserving (map (hybridFixedPools (n := n) (m := m) eps) id)
((annotationLaw P n m).prod (hybridRequestLaw n m eps))
(((labeledProductLaw P (hybridTuning n m eps).h0).prod
((auxProductLaw P (hybridTuning n m eps).Mp).prod
(auxProductLaw P (hybridTuning n m eps).Mf))).prod (hybridRequestLaw n m eps)) :=
(hybrid_fixed_pools_law eps P).prod (MeasurePreserving.id _)
theorem hybrid_capped_array_risk_eq_fixed
Under the stated inputs and conditions, The capped array risk is exactly its independent three-pool risk. This gives the stated result.
Formal statement
hypotheses
conclusion
sqRisk ((annotationLaw P n m).prod (hybridRequestLaw n m eps)) (fun z => hybridCappedArrayStatistic eps z.1 z.2) (ateFunctional P)
= sqRisk (((labeledProductLaw P (hybridTuning n m eps).h0).prod ((auxProductLaw P (hybridTuning n m eps).Mp).prod (auxProductLaw P (hybridTuning n m eps).Mf))).prod (hybridRequestLaw n m eps)) (fun z => hybridCappedFixedStatistic eps z.1 z.2) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_capped_array_risk_eq_fixed
lemma hybrid_capped_array_risk_eq_fixed {n m d : Nat} (eps : Real) (P : DiscreteLaw d) :
sqRisk ((annotationLaw P n m).prod (hybridRequestLaw n m eps))
(fun z => hybridCappedArrayStatistic eps z.1 z.2) (ateFunctional P) =
sqRisk
(((labeledProductLaw P (hybridTuning n m eps).h0).prod
((auxProductLaw P (hybridTuning n m eps).Mp).prod
(auxProductLaw P (hybridTuning n m eps).Mf))).prod (hybridRequestLaw n m eps))
(fun z => hybridCappedFixedStatistic eps z.1 z.2) (ateFunctional P) := by
have hmap := hybrid_fixed_pools_request_law (n := n) (m := m) eps P
have hm : Measurable (fun z : ((Fin (hybridTuning n m eps).h0 → Obs d) ×
(Fin (hybridTuning n m eps).Mp → AuxObs d) ×
(Fin (hybridTuning n m eps).Mf → AuxObs d)) × (Nat × Nat × Nat) =>
(hybridCappedFixedStatistic (n := n) (m := m) (d := d) eps z.1 z.2 -
ateFunctional P) ^ 2) := by fun_prop
unfold sqRisk
rw [← hmap.map_eq, integral_map hmap.measurable.aemeasurable hm.aestronglyMeasurable]
apply integral_congr_ae
filter_upwards with z
simp only [hybrid_capped_array_eq_fixed]
rfl
theorem hybrid_ruleRisk_le_capped_fixed_risk
Under the stated inputs and conditions, Conditional averaging bounds original-record risk by the independent capped-pool risk. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ sqRisk (((labeledProductLaw P (hybridTuning n m eps).h0).prod ((auxProductLaw P (hybridTuning n m eps).Mp).prod (auxProductLaw P (hybridTuning n m eps).Mf))).prod (hybridRequestLaw n m eps)) (fun z => hybridCappedFixedStatistic eps z.1 z.2) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_ruleRisk_le_capped_fixed_risk
lemma hybrid_ruleRisk_le_capped_fixed_risk {n m d : Nat} (eps : Real) (P : DiscreteLaw d)
(hS : exp 4096 ≤ (n : Real) * eps) :
ruleRisk (liftRule (hybridEstimator n m d eps)) P ≤
sqRisk
(((labeledProductLaw P (hybridTuning n m eps).h0).prod
((auxProductLaw P (hybridTuning n m eps).Mp).prod
(auxProductLaw P (hybridTuning n m eps).Mf))).prod (hybridRequestLaw n m eps))
(fun z => hybridCappedFixedStatistic eps z.1 z.2) (ateFunctional P) := by
rw [← hybrid_capped_array_risk_eq_fixed]
exact hybrid_ruleRisk_le_capped_array_risk eps P hS
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridPrefixContrast 6 declarations Square integrability and clipped contrast risk for the independent three-pool hybrid prefixes.
Square integrability and clipped contrast risk for the independent three-pool hybrid prefixes.
theorem hybrid_count_arm_memLp
Under the stated hypotheses, Independent Poisson counts give a square-integrable hybrid arm sum at every tuning. This gives the stated result.
Formal statement
hypotheses
d :
L k0 :
B u tp t :
P :
a :
Om :
Type
mu :
hmeas :
hZ :
hJ :
hK :
conclusion
MemLp (fun om => ∑ j, hybridCellValue L B k0 u t (Z a j om) (J a j om) (K a j om) (K (!a) j om)) 2 mu
Proof (Lean source)
-- @node: hybrid_count_arm_memLp
lemma hybrid_count_arm_memLp {d : Nat} (L k0 : Nat) (B u tp t : Real)
(P : DiscreteLaw d) (a : Bool) {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (Z J K : Bool → Fin d → Om → Nat)
(hmeas : ∀ b j, Measurable (Z b j) ∧ Measurable (J b j) ∧ Measurable (K b j))
(hZ : ∀ b j, mu.map (Z b j) = poissonMeasure (toNNReal (u * markedMass P j b)))
(hJ : ∀ b j, mu.map (J b j) = poissonMeasure (toNNReal (tp * armMass P j b)))
(hK : ∀ b j, mu.map (K b j) = poissonMeasure (toNNReal (t * armMass P j b)))
(hind : iIndepFun (fun i : Fin 3 × Bool × Fin d =>
if i.1 = 0 then Z i.2.1 i.2.2 else if i.1 = 1 then J i.2.1 i.2.2
else K i.2.1 i.2.2) mu) :
MemLp (fun om => ∑ j, hybridCellValue L B k0 u t
(Z a j om) (J a j om) (K a j om) (K (!a) j om)) 2 mu := by
classical
let X : Fin 4 × Fin d → Om → Nat := fun i =>
if i.1 = 0 then J a i.2 else if i.1 = 1 then Z a i.2
else if i.1 = 2 then K a i.2 else K (!a) i.2
have hX (i) : Measurable (X i) := by
dsimp [X]
split_ifs <;> first
| exact (hmeas a i.2).1
| exact (hmeas a i.2).2.1
| exact (hmeas a i.2).2.2
| exact (hmeas (!a) i.2).2.2
let idx : Fin 4 × Fin d → Fin 3 × Bool × Fin d := fun i =>
if i.1 = 0 then (1,a,i.2) else if i.1 = 1 then (0,a,i.2)
else if i.1 = 2 then (2,a,i.2) else (2,!a,i.2)
have hinj : Injective idx := by
rintro ⟨i,j⟩ ⟨i',j'⟩ h
fin_cases i <;> fin_cases i' <;> simp [idx] at h ⊢ <;>
exact h
have hi : iIndepFun X mu := by
have hh := hind.precomp hinj
have hfun : (fun i : Fin 4 × Fin d =>
if (idx i).1 = 0 then Z (idx i).2.1 (idx i).2.2
else if (idx i).1 = 1 then J (idx i).2.1 (idx i).2.2
else K (idx i).2.1 (idx i).2.2) = X := by
funext ⟨i,j⟩
fin_cases i <;> simp [idx, X]
rw [hfun] at hh
exact hh
have hlaw := hybrid_four_count_product_law mu X hX hi
simp only [X, Fin.reduceEq, if_true, if_false] at hlaw
simp_rw [hJ, hZ, hK] at hlaw
let nu := fun j => (poissonMeasure (toNNReal (tp * armMass P j a))).prod
(cellPoissonLaw u t (markedMass P j a) (armMass P j a) (armMass P j (!a)))
let V := fun z : Nat × (Nat × Nat × Nat) =>
hybridCellValue L B k0 u t z.2.1 z.1 z.2.2.1 z.2.2.2
let (j : Fin d) : IsProbabilityMeasure (nu j) := by
dsimp [nu, cellPoissonLaw]; infer_instance
have hcell (j : Fin d) : MemLp V 2 (nu j) :=
hybrid_pilot_cell_memLp L k0 B u tp t (markedMass P j a)
(armMass P j a) (armMass P j (!a))
have hsum : MemLp (fun z => ∑ j, V (z j)) 2 (Measure.pi nu) := by
apply memLp_finsetSum
intro j _
exact (hcell j).comp_measurePreserving (measurePreserving_eval nu j)
have hmap : Measurable (fun om j => (J a j om, Z a j om, K a j om, K (!a) j om)) := by
apply measurable_pi_lambda
intro j
exact (hmeas a j).2.1.prodMk ((hmeas a j).1.prodMk
((hmeas a j).2.2.prodMk (hmeas (!a) j).2.2))
have hlaw' : mu.map (fun om j => (J a j om, Z a j om, K a j om, K (!a) j om)) =
Measure.pi nu := hlaw
exact hsum.comp_measurePreserving ⟨hmap, hlaw'⟩
theorem hybrid_ordered_arm_memLp
Under the stated inputs and conditions, Each ordered-prefix hybrid arm has finite second moment, including zero intensities. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_ordered_arm_memLp
lemma hybrid_ordered_arm_memLp {d : Nat} (P : DiscreteLaw d) (L k0 : Nat)
(B : Real) (u tp t : NNReal) (a : Bool) :
MemLp (hybridOrderedArmStatistic L B k0 u t a) 2
(hybridPoissonPrefixLaw P u tp t) := by
let Z := fun b j s => hybridPrefixCounts (d := d) s (inl (j,b,true))
let J := fun b j s => hybridPrefixCounts (d := d) s (inr (inl (j,b)))
let K := fun b j s => hybridPrefixCounts (d := d) s (inr (inr (j,b)))
have hm (b j) : Measurable (Z b j) ∧ Measurable (J b j) ∧ Measurable (K b j) := by
constructor
· exact (measurable_pi_apply _).comp (hybridPrefixCounts_measurable d)
· constructor <;> exact (measurable_pi_apply _).comp (hybridPrefixCounts_measurable d)
have hZ (b j) : (hybridPoissonPrefixLaw P u tp t).map (Z b j) =
poissonMeasure (toNNReal ((u : Real) * markedMass P j b)) := by
simpa only [Real.toNNReal_mul u.coe_nonneg, Real.toNNReal_coe] using
hybrid_prefix_success_count_law P u tp t j b
have hJ (b j) : (hybridPoissonPrefixLaw P u tp t).map (J b j) =
poissonMeasure (toNNReal ((tp : Real) * armMass P j b)) := by
simpa only [Real.toNNReal_mul tp.coe_nonneg, Real.toNNReal_coe] using
hybrid_prefix_pilot_count_law P u tp t j b
have hK (b j) : (hybridPoissonPrefixLaw P u tp t).map (K b j) =
poissonMeasure (toNNReal ((t : Real) * armMass P j b)) := by
simpa only [Real.toNNReal_mul t.coe_nonneg, Real.toNNReal_coe] using
hybrid_prefix_factorial_count_law P u tp t j b
have hi : iIndepFun (fun i : Fin 3 × Bool × Fin d =>
if i.1 = 0 then Z i.2.1 i.2.2 else if i.1 = 1 then J i.2.1 i.2.2
else K i.2.1 i.2.2) (hybridPoissonPrefixLaw P u tp t) := by
convert hybrid_prefix_arm_counts_independent P u tp t using 1
funext i s
dsimp [Z,J,K]
split_ifs <;> rfl
exact hybrid_count_arm_memLp L k0 B u tp t P a
(hybridPoissonPrefixLaw P u tp t) Z J K hm hZ hJ hK hi
theorem hybrid_ordered_statistic_eq_contrast
Under the stated inputs and conditions, The ordered clipped statistic is exactly the difference of the two hybrid arm sums. This gives the stated result.
Formal statement
hypotheses
d :
tun :
s :
conclusion
= max (-1) (min 1 (hybridOrderedArmStatistic tun.L tun.B tun.k0 tun.u tun.t true s - hybridOrderedArmStatistic tun.L tun.B tun.k0 tun.u tun.t false s))
Proof (Lean source)
-- @node: hybrid_ordered_statistic_eq_contrast
lemma hybrid_ordered_statistic_eq_contrast {d : Nat} (tun : HybridTuning)
(s : FiniteSample (Obs d) × FiniteSample (AuxObs d) × FiniteSample (AuxObs d)) :
hybridOrderedPrefixStatistic tun s = max (-1) (min 1
(hybridOrderedArmStatistic tun.L tun.B tun.k0 tun.u tun.t true s -
hybridOrderedArmStatistic tun.L tun.B tun.k0 tun.u tun.t false s)) := by
simp only [hybridOrderedPrefixStatistic, hybridOrderedArmStatistic,
hybridPrefixCounts, Sum.elim_inl, Sum.elim_inr, Bool.not_true, Bool.not_false,
Finset.sum_sub_distrib]
theorem hybrid_ordered_prefix_contrast_risk
Under the stated inputs and conditions, Squared-difference and clipping bounds aggregate the two arm risks without arm independence. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
n m d :
eps :
P :
u tp t :
1 ≤ n
2 ≤ d
0 < eps
eps ≤ 1 / 4
ModelClass d eps P
1 / 3 ≤ (t : Real) / tp
(t : Real) / tp ≤ 3
H :
fun a => hybridOrderedArmStatistic (d := d) L B k0 u t a
sqRisk (hybridPoissonPrefixLaw P u tp t) (fun s => max (-1) (min 1 (H true s - H false s))) (ateFunctional P)
Proof (Lean source)
-- @node: hybrid_ordered_prefix_contrast_risk
lemma hybrid_ordered_prefix_contrast_risk :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real) (P : DiscreteLaw d)
(u tp t : NNReal),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 → exp 4096 ≤ (n : Real) * eps →
ModelClass d eps P → (n : Real) / 32 ≤ (u : Real) →
((n : Real) + m) / 64 ≤ (tp : Real) →
((n : Real) + m) / 64 ≤ (t : Real) →
1 / 3 ≤ (t : Real) / tp → (t : Real) / tp ≤ 3 →
let L := floor (log ((n : Real) * eps) / 1024)
let B : Real := 2 ^ 20 * L / min (tp : Real) t
let k0 := floor ((tp : Real) * B / 4)
let H := fun a => hybridOrderedArmStatistic (d := d) L B k0 u t a
sqRisk (hybridPoissonPrefixLaw P u tp t)
(fun s => max (-1) (min 1 (H true s - H false s))) (ateFunctional P) ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * L)) ^ 2) := by
obtain ⟨C, hC, hbound⟩ := hybrid_ordered_prefix_arm_risk
refine ⟨4 * C, by positivity, ?_⟩
intro n m d eps P u tp t hn hd heps heps4 hS hP hu htp ht hlo hhi
dsimp only
let L := floor (log ((n : Real) * eps) / 1024)
let B : Real := 2 ^ 20 * L / min (tp : Real) t
let k0 := floor ((tp : Real) * B / 4)
let H := fun a => hybridOrderedArmStatistic (d := d) L B k0 u t a
let psi := fun a => ∑ j : Fin d, cellMass P j * outcomeMean P a j
let mu := hybridPoissonPrefixLaw P u tp t
have hLp (a) : MemLp (H a) 2 mu := hybrid_ordered_arm_memLp P L k0 B u tp t a
have hint (a) : Integrable (fun s => (H a s - psi a) ^ 2) mu :=
((hLp a).sub (memLp_const (psi a))).integrable_sq
have hcontrast : Integrable (fun s => (H true s - H false s - ateFunctional P) ^ 2) mu :=
(((hLp true).sub (hLp false)).sub (memLp_const (ateFunctional P))).integrable_sq
have htau : ateFunctional P = psi true - psi false := by
simp only [ateFunctional, psi, Finset.sum_sub_distrib, mul_sub]
have hMSE (a) : (∫ s, (H a s - psi a) ^ 2 ∂mu) ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * L)) ^ 2) :=
hbound n m d eps P a u tp t hn hd heps heps4 hS hP hu htp ht hlo hhi
change (∫ s, (max (-1) (min 1 (H true s - H false s)) - ateFunctional P) ^ 2 ∂mu) ≤ _
calc
_ ≤ ∫ s, (H true s - H false s - ateFunctional P) ^ 2 ∂mu :=
integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
hcontrast (Filter.Eventually.of_forall (fun s =>
knownMarginal_clip_loss _ _ (ateFunctional_mem_Icc P)))
_ ≤ ∫ s, (2 * (H true s - psi true) ^ 2 + 2 * (H false s - psi false) ^ 2) ∂mu := by
apply integral_mono hcontrast ((hint true).const_mul 2 |>.add ((hint false).const_mul 2))
intro s
change (H true s - H false s - ateFunctional P) ^ 2 ≤
2 * (H true s - psi true) ^ 2 + 2 * (H false s - psi false) ^ 2
rw [htau]
nlinarith [sq_nonneg ((H true s - psi true) + (H false s - psi false))]
_ = 2 * (∫ s, (H true s - psi true) ^ 2 ∂mu) +
2 * (∫ s, (H false s - psi false) ^ 2 ∂mu) := by
rw [integral_add ((hint true).const_mul 2) ((hint false).const_mul 2),
integral_const_mul, integral_const_mul]
_ ≤ _ := by nlinarith [hMSE true, hMSE false]
theorem hybridOrderedPrefixStatistic_measurable
The clipped ordered hybrid statistic is measurable. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: hybridOrderedPrefixStatistic_measurable
lemma hybridOrderedPrefixStatistic_measurable (d : Nat) (tun : HybridTuning) :
Measurable (hybridOrderedPrefixStatistic (d := d) tun) := by
let F : (Obs d ⊕ (AuxObs d ⊕ AuxObs d) → Nat) → Real := fun z =>
max (-1) (min 1 (∑ j : Fin d,
(hybridCellValue tun.L tun.B tun.k0 tun.u tun.t
(z (inl (j,true,true))) (z (inr (inl (j,true))))
(z (inr (inr (j,true)))) (z (inr (inr (j,false)))) -
hybridCellValue tun.L tun.B tun.k0 tun.u tun.t
(z (inl (j,false,true))) (z (inr (inl (j,false))))
(z (inr (inr (j,false)))) (z (inr (inr (j,true)))))))
have h := (measurable_of_countable F).comp (hybridPrefixCounts_measurable d)
change Measurable (fun s => F (hybridPrefixCounts s)) at h
unfold hybridOrderedPrefixStatistic
simpa only [F, hybridPrefixCounts,
Sum.elim_inl, Sum.elim_inr] using h
theorem hybridOrderedPrefixStatistic_mem_Icc
Under the stated inputs and conditions, Clipping keeps every ordered hybrid output in the target interval. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: hybridOrderedPrefixStatistic_mem_Icc
lemma hybridOrderedPrefixStatistic_mem_Icc {d : Nat} (tun : HybridTuning)
(s : FiniteSample (Obs d) × FiniteSample (AuxObs d) × FiniteSample (AuxObs d)) :
hybridOrderedPrefixStatistic tun s ∈ Icc (-1) 1 := by
rw [hybrid_ordered_statistic_eq_contrast]
exact ⟨le_max_left _ _, max_le (by norm_num) (min_le_left _ _)⟩
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridPrefixCounts 11 declarations Joint independent Poisson histogram laws for the outcome, pilot, and factorial prefixes.
Joint independent Poisson histogram laws for the outcome, pilot, and factorial prefixes.
def hybridPoissonPrefixLaw
The three ideal ordered pools have independent Poisson sample laws.
Definition (Lean source)
parameters
def
hybridPoissonPrefixLaw P u tp t :
given by
(finitePoissonSampleLaw (obsLaw P) u).prod ((finitePoissonSampleLaw (auxMarginal P).toMeasure tp).prod (finitePoissonSampleLaw (auxMarginal P).toMeasure t))
instance hybridPoissonPrefixLaw_probability
The ideal three-pool experiment is a probability law even at zero intensities.
Definition (Lean source)
-- @node: hybridPoissonPrefixLaw_probability
instance hybridPoissonPrefixLaw_probability {d : Nat} (P : DiscreteLaw d) (u tp t : NNReal) :
IsProbabilityMeasure (hybridPoissonPrefixLaw P u tp t) := by
unfold hybridPoissonPrefixLaw
infer_instance
Disjoint sum indices distinguish all atoms of the three histograms.
Definition (Lean source)
parameters
d :
s :
given by
elim (finiteSampleHistogram s.1.points) (elim (finiteSampleHistogram s.2.1.points) (finiteSampleHistogram s.2.2.points))
theorem hybridPrefixCounts_measurable
Histograms of all three ordered finite samples are measurable. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: hybridPrefixCounts_measurable
lemma hybridPrefixCounts_measurable (d : Nat) :
Measurable (hybridPrefixCounts (d := d)) := by
change Measurable ((MeasurableEquiv.sumPiEquivProdPi
(fun _ : Obs d ⊕ (AuxObs d ⊕ AuxObs d) => Nat)).symm ∘
(fun s : FiniteSample (Obs d) × FiniteSample (AuxObs d) × FiniteSample (AuxObs d) =>
(finiteSampleHistogram s.1.points,
(MeasurableEquiv.sumPiEquivProdPi (fun _ : AuxObs d ⊕ AuxObs d => Nat)).symm
(finiteSampleHistogram s.2.1.points, finiteSampleHistogram s.2.2.points))))
apply (MeasurableEquiv.sumPiEquivProdPi _).symm.measurable.comp
apply prodMk
· exact baseline_finiteSampleHistogram_measurable.comp measurable_fst
· exact (MeasurableEquiv.sumPiEquivProdPi _).symm.measurable.comp
((baseline_finiteSampleHistogram_measurable.comp (measurable_fst.comp measurable_snd)).prodMk
(baseline_finiteSampleHistogram_measurable.comp (measurable_snd.comp measurable_snd)))
theorem hybrid_prefix_counts_law
Under the stated inputs and conditions, Thinning and the product law give independent Poisson counts for every atom. This gives the stated result.
Formal statement
hypotheses
conclusion
= Measure.pi (fun i : Obs d ⊕ (AuxObs d ⊕ AuxObs d) => poissonMeasure (elim (fun z => u * ((obsLaw P) {z}).toNNReal) (elim (fun z => tp * ((auxMarginal P).toMeasure {z}).toNNReal) (fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal)) i))
Proof (Lean source)
-- @node: hybrid_prefix_counts_law
lemma hybrid_prefix_counts_law {d : Nat} (P : DiscreteLaw d) (u tp t : NNReal) :
(hybridPoissonPrefixLaw P u tp t).map hybridPrefixCounts =
Measure.pi (fun i : Obs d ⊕ (AuxObs d ⊕ AuxObs d) => poissonMeasure
(elim (fun z => u * ((obsLaw P) {z}).toNNReal)
(elim (fun z => tp * ((auxMarginal P).toMeasure {z}).toNNReal)
(fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal)) i)) := by
let H : FiniteSample (AuxObs d) × FiniteSample (AuxObs d) →
(AuxObs d ⊕ AuxObs d → Nat) := fun s =>
elim (finiteSampleHistogram s.1.points) (finiteSampleHistogram s.2.points)
have hH : Measurable H := by
change Measurable ((MeasurableEquiv.sumPiEquivProdPi
(fun _ : AuxObs d ⊕ AuxObs d => Nat)).symm ∘
(fun s : FiniteSample (AuxObs d) × FiniteSample (AuxObs d) =>
(finiteSampleHistogram s.1.points, finiteSampleHistogram s.2.points)))
exact (MeasurableEquiv.sumPiEquivProdPi _).symm.measurable.comp
((baseline_finiteSampleHistogram_measurable.comp measurable_fst).prodMk
(baseline_finiteSampleHistogram_measurable.comp measurable_snd))
let rates : Obs d ⊕ (AuxObs d ⊕ AuxObs d) → NNReal :=
elim (fun z => u * ((obsLaw P) {z}).toNNReal)
(elim (fun z => tp * ((auxMarginal P).toMeasure {z}).toNNReal)
(fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal))
have haux := Measure.map_prod_map
(finitePoissonSampleLaw (auxMarginal P).toMeasure tp)
(finitePoissonSampleLaw (auxMarginal P).toMeasure t)
(baseline_finiteSampleHistogram_measurable (X := AuxObs d))
(baseline_finiteSampleHistogram_measurable (X := AuxObs d))
rw [finitePoissonSampleLaw_map_histogram, finitePoissonSampleLaw_map_histogram] at haux
have hauxlaw : ((finitePoissonSampleLaw (auxMarginal P).toMeasure tp).prod
(finitePoissonSampleLaw (auxMarginal P).toMeasure t)).map H =
Measure.pi (fun i : AuxObs d ⊕ AuxObs d => poissonMeasure (rates (.inr i))) := by
rw [← (measurePreserving_sumPiEquivProdPi_symm
(fun i : AuxObs d ⊕ AuxObs d => poissonMeasure (rates (.inr i)))).map_eq]
change _ = Measure.map (MeasurableEquiv.sumPiEquivProdPi
(fun _ : AuxObs d ⊕ AuxObs d => Nat)).symm
((independentPoissonCountLaw (auxMarginal P).toMeasure tp).prod
(independentPoissonCountLaw (auxMarginal P).toMeasure t))
rw [haux, Measure.map_map (MeasurableEquiv.sumPiEquivProdPi _).symm.measurable
(by fun_prop)]
rfl
have hjoint := Measure.map_prod_map (finitePoissonSampleLaw (obsLaw P) u)
((finitePoissonSampleLaw (auxMarginal P).toMeasure tp).prod
(finitePoissonSampleLaw (auxMarginal P).toMeasure t))
(baseline_finiteSampleHistogram_measurable (X := Obs d)) hH
rw [finitePoissonSampleLaw_map_histogram, hauxlaw] at hjoint
rw [← (measurePreserving_sumPiEquivProdPi_symm
(fun i => poissonMeasure (rates i))).map_eq]
change _ = Measure.map (MeasurableEquiv.sumPiEquivProdPi
(fun _ : Obs d ⊕ (AuxObs d ⊕ AuxObs d) => Nat)).symm
((independentPoissonCountLaw (obsLaw P) u).prod
(Measure.pi (fun i : AuxObs d ⊕ AuxObs d => poissonMeasure (rates (.inr i)))))
rw [hjoint, Measure.map_map (MeasurableEquiv.sumPiEquivProdPi _).symm.measurable
((baseline_finiteSampleHistogram_measurable).prodMap hH)]
rfl
theorem hybrid_prefix_count_marginal
Under the stated inputs and conditions, Every atom has its prescribed Poisson marginal in the three-pool law. This gives the stated result.
Formal statement
hypotheses
conclusion
= poissonMeasure (elim (fun z => u * ((obsLaw P) {z}).toNNReal) (elim (fun z => tp * ((auxMarginal P).toMeasure {z}).toNNReal) (fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal)) i)
Proof (Lean source)
-- @node: hybrid_prefix_count_marginal
lemma hybrid_prefix_count_marginal {d : Nat} (P : DiscreteLaw d) (u tp t : NNReal)
(i : Obs d ⊕ (AuxObs d ⊕ AuxObs d)) :
(hybridPoissonPrefixLaw P u tp t).map (fun s => hybridPrefixCounts s i) =
poissonMeasure (elim (fun z => u * ((obsLaw P) {z}).toNNReal)
(elim (fun z => tp * ((auxMarginal P).toMeasure {z}).toNNReal)
(fun z => t * ((auxMarginal P).toMeasure {z}).toNNReal)) i) := by
rw [show (fun s => hybridPrefixCounts s i) =
(fun c : Obs d ⊕ (AuxObs d ⊕ AuxObs d) → Nat => c i) ∘ hybridPrefixCounts by rfl,
← Measure.map_map (by fun_prop) (hybridPrefixCounts_measurable d),
hybrid_prefix_counts_law]
exact (measurePreserving_eval _ i).map_eq
theorem hybrid_prefix_counts_independent
Under the stated inputs and conditions, All atom counts are mutually independent across cells and across pools. This gives the stated result.
Formal statement
hypotheses
conclusion
iIndepFun (fun i s => hybridPrefixCounts s i) (hybridPoissonPrefixLaw P u tp t)
Proof (Lean source)
-- @node: hybrid_prefix_counts_independent
lemma hybrid_prefix_counts_independent {d : Nat} (P : DiscreteLaw d) (u tp t : NNReal) :
iIndepFun (fun i s => hybridPrefixCounts s i) (hybridPoissonPrefixLaw P u tp t) := by
apply (iIndepFun_iff_map_fun_eq_pi_map
(fun i => ((measurable_pi_apply i).comp (hybridPrefixCounts_measurable d)).aemeasurable)).2
simp only [Function.comp_def]
simp_rw [hybrid_prefix_count_marginal]
exact hybrid_prefix_counts_law P u tp t
theorem hybrid_prefix_arm_counts_independent
Under the stated inputs and conditions, The outcome success counts and both marginal pools form the independent arm experiment. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: hybrid_prefix_arm_counts_independent
lemma hybrid_prefix_arm_counts_independent {d : Nat} (P : DiscreteLaw d)
(u tp t : NNReal) :
iIndepFun (fun i : Fin 3 × Bool × Fin d => fun s => hybridPrefixCounts s
(if i.1 = 0 then inl (i.2.2,i.2.1,true)
else if i.1 = 1 then inr (inl (i.2.2,i.2.1))
else inr (inr (i.2.2,i.2.1)))) (hybridPoissonPrefixLaw P u tp t) := by
apply (hybrid_prefix_counts_independent P u tp t).precomp
rintro ⟨i,a,j⟩ ⟨i',a',j'⟩ h
fin_cases i <;> fin_cases i' <;> simp_all
theorem hybrid_prefix_success_count_law
Under the stated inputs and conditions, Outcome success counts have mean intensity times the marked mass. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_prefix_success_count_law
lemma hybrid_prefix_success_count_law {d : Nat} (P : DiscreteLaw d)
(u tp t : NNReal) (j : Fin d) (a : Bool) :
(hybridPoissonPrefixLaw P u tp t).map
(fun s => hybridPrefixCounts s (inl (j,a,true))) =
poissonMeasure (u * toNNReal (markedMass P j a)) := by
rw [hybrid_prefix_count_marginal]
simp [obsLaw, markedMass, jointMass]
theorem hybrid_prefix_pilot_count_law
Under the stated inputs and conditions, Pilot arm counts have mean pilot intensity times the arm mass. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_prefix_pilot_count_law
lemma hybrid_prefix_pilot_count_law {d : Nat} (P : DiscreteLaw d)
(u tp t : NNReal) (j : Fin d) (a : Bool) :
(hybridPoissonPrefixLaw P u tp t).map
(fun s => hybridPrefixCounts s (inr (inl (j,a)))) =
poissonMeasure (tp * toNNReal (armMass P j a)) := by
rw [hybrid_prefix_count_marginal]
simp [← auxMarginal_toReal_armMass]
theorem hybrid_prefix_factorial_count_law
Under the stated inputs and conditions, Factorial arm counts have mean factorial intensity times the arm mass. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_prefix_factorial_count_law
lemma hybrid_prefix_factorial_count_law {d : Nat} (P : DiscreteLaw d)
(u tp t : NNReal) (j : Fin d) (a : Bool) :
(hybridPoissonPrefixLaw P u tp t).map
(fun s => hybridPrefixCounts s (inr (inr (j,a)))) =
poissonMeasure (t * toNNReal (armMass P j a)) := by
rw [hybrid_prefix_count_marginal]
simp [← auxMarginal_toReal_armMass]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridPrefixRisk 2 declarations Arm risk readback from independent ordered Poisson prefixes.
Arm risk readback from independent ordered Poisson prefixes.
The unclipped arm sum computed from the three ordered histograms.
Definition (Lean source)
parameters
d :
L :
B :
k0 :
u t :
a :
s :
def
given by
∑ j : Fin d, hybridCellValue L B k0 u t (hybridPrefixCounts s (inl (j,a,true))) (hybridPrefixCounts s (inr (inl (j,a)))) (hybridPrefixCounts s (inr (inr (j,a)))) (hybridPrefixCounts s (inr (inr (j,!a))))
theorem hybrid_ordered_prefix_arm_risk
Under the stated inputs and conditions, Applying the count-experiment theorem bounds each ordered-prefix arm's risk. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
n m d :
eps :
P :
a :
u tp t :
1 ≤ n
2 ≤ d
0 < eps
eps ≤ 1 / 4
ModelClass d eps P
1 / 3 ≤ (t : Real) / tp
(t : Real) / tp ≤ 3
sqRisk (hybridPoissonPrefixLaw P u tp t) (hybridOrderedArmStatistic L B k0 u t a) (∑ j : Fin d, cellMass P j * outcomeMean P a j)
Proof (Lean source)
-- @node: hybrid_ordered_prefix_arm_risk
lemma hybrid_ordered_prefix_arm_risk :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real) (P : DiscreteLaw d)
(a : Bool) (u tp t : NNReal),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 → exp 4096 ≤ (n : Real) * eps →
ModelClass d eps P → (n : Real) / 32 ≤ (u : Real) →
((n : Real) + m) / 64 ≤ (tp : Real) →
((n : Real) + m) / 64 ≤ (t : Real) →
1 / 3 ≤ (t : Real) / tp → (t : Real) / tp ≤ 3 →
let L := floor (log ((n : Real) * eps) / 1024)
let B : Real := 2 ^ 20 * L / min (tp : Real) t
let k0 := floor ((tp : Real) * B / 4)
sqRisk (hybridPoissonPrefixLaw P u tp t)
(hybridOrderedArmStatistic L B k0 u t a)
(∑ j : Fin d, cellMass P j * outcomeMean P a j) ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * L)) ^ 2) := by
obtain ⟨C, hC, hbound⟩ := uniform_hybrid_arm_risk
refine ⟨C, hC, ?_⟩
intro n m d eps P a u tp t hn hd heps heps4 hS hP hu htp ht hlo hhi
dsimp only
let Z := fun b j s => hybridPrefixCounts (d := d) s (inl (j,b,true))
let J := fun b j s => hybridPrefixCounts (d := d) s (inr (inl (j,b)))
let K := fun b j s => hybridPrefixCounts (d := d) s (inr (inr (j,b)))
have hm (b j) : Measurable (Z b j) ∧ Measurable (J b j) ∧ Measurable (K b j) := by
constructor
· exact (measurable_pi_apply _).comp (hybridPrefixCounts_measurable d)
· constructor <;> exact (measurable_pi_apply _).comp (hybridPrefixCounts_measurable d)
have hZ (b j) : (hybridPoissonPrefixLaw P u tp t).map (Z b j) =
poissonMeasure (toNNReal ((u : Real) * markedMass P j b)) := by
simpa only [Real.toNNReal_mul u.coe_nonneg, Real.toNNReal_coe] using
hybrid_prefix_success_count_law P u tp t j b
have hJ (b j) : (hybridPoissonPrefixLaw P u tp t).map (J b j) =
poissonMeasure (toNNReal ((tp : Real) * armMass P j b)) := by
simpa only [Real.toNNReal_mul tp.coe_nonneg, Real.toNNReal_coe] using
hybrid_prefix_pilot_count_law P u tp t j b
have hK (b j) : (hybridPoissonPrefixLaw P u tp t).map (K b j) =
poissonMeasure (toNNReal ((t : Real) * armMass P j b)) := by
simpa only [Real.toNNReal_mul t.coe_nonneg, Real.toNNReal_coe] using
hybrid_prefix_factorial_count_law P u tp t j b
have hi : iIndepFun (fun i : Fin 3 × Bool × Fin d =>
if i.1 = 0 then Z i.2.1 i.2.2 else if i.1 = 1 then J i.2.1 i.2.2
else K i.2.1 i.2.2) (hybridPoissonPrefixLaw P u tp t) := by
convert hybrid_prefix_arm_counts_independent P u tp t using 1
funext i s
dsimp [Z,J,K]
split_ifs <;> rfl
exact hbound n m d eps P a u tp t hn hd heps heps4 hS hP hu htp ht hlo hhi
_ (hybridPoissonPrefixLaw P u tp t) Z J K hm hZ hJ hK hi
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridTunedRisk 3 declarations Risk bounds for the exact public hybrid tuning, including the logarithmic conversion and the three finite-pool overflow costs.
Risk bounds for the exact public hybrid tuning, including the logarithmic conversion and the three finite-pool overflow costs.
theorem hybrid_tuned_prefix_risk
Under the stated inputs and conditions, The prescribed intensities and bandwidth realize the ordered-prefix contrast bound. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
sqRisk (hybridPoissonPrefixLaw P ((hybridTuning n m eps).h0 / 8 : NNReal) ((hybridTuning n m eps).Mp / 8 : NNReal) ((hybridTuning n m eps).Mf / 8 : NNReal)) (hybridOrderedPrefixStatistic (hybridTuning n m eps)) (ateFunctional P)
≤ C * (1 / ((n : Real) * eps) + ((d : Real) / (((n : Real) + m) * eps * (hybridTuning n m eps).L)) ^ 2)
Proof (Lean source)
-- @node: hybrid_tuned_prefix_risk
lemma hybrid_tuned_prefix_risk :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real) (P : DiscreteLaw d),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
exp 4096 ≤ (n : Real) * eps → ModelClass d eps P →
sqRisk
(hybridPoissonPrefixLaw P
((hybridTuning n m eps).h0 / 8 : NNReal)
((hybridTuning n m eps).Mp / 8 : NNReal)
((hybridTuning n m eps).Mf / 8 : NNReal))
(hybridOrderedPrefixStatistic (hybridTuning n m eps)) (ateFunctional P) ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * (hybridTuning n m eps).L)) ^ 2) := by
obtain ⟨C, hC, hbound⟩ := hybrid_ordered_prefix_contrast_risk
refine ⟨C, hC, ?_⟩
intro n m d eps P hn hd heps heps4 hS hP
obtain ⟨hu, hp, ht, _, hlo, hhi⟩ := hybrid_finite_pool_intensities n m eps
(hybrid_large_sample_size n eps heps4 hS)
have hb := hbound n m d eps P
((hybridTuning n m eps).h0 / 8 : NNReal)
((hybridTuning n m eps).Mp / 8 : NNReal)
((hybridTuning n m eps).Mf / 8 : NNReal) hn hd heps heps4 hS hP
simp only [NNReal.coe_div, NNReal.coe_natCast, NNReal.coe_ofNat] at hb
have hb' := hb hu hp ht hlo hhi
change sqRisk _
(fun s => hybridOrderedPrefixStatistic (hybridTuning n m eps) s) _ ≤ _
simp_rw [hybrid_ordered_statistic_eq_contrast]
simpa only [hybridTuning,
NNReal.coe_div, NNReal.coe_natCast, NNReal.coe_ofNat] using hb'
theorem hybrid_tuned_prefix_and_overflow_rate
Under the stated inputs and conditions, The ideal contrast risk and all three overflow costs have the uncapped frontier order. This gives the stated result.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
sqRisk (hybridPoissonPrefixLaw P ((hybridTuning n m eps).h0 / 8 : NNReal) ((hybridTuning n m eps).Mp / 8 : NNReal) ((hybridTuning n m eps).Mf / 8 : NNReal)) (hybridOrderedPrefixStatistic (hybridTuning n m eps)) (ateFunctional P)
+ (exp (-((hybridTuning n m eps).h0 : Real)) + exp (-((hybridTuning n m eps).Mp : Real)) + exp (-((hybridTuning n m eps).Mf : Real)))
Proof (Lean source)
-- @node: hybrid_tuned_prefix_and_overflow_rate
lemma hybrid_tuned_prefix_and_overflow_rate :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real) (P : DiscreteLaw d),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
exp 4096 ≤ (n : Real) * eps → ModelClass d eps P →
sqRisk
(hybridPoissonPrefixLaw P
((hybridTuning n m eps).h0 / 8 : NNReal)
((hybridTuning n m eps).Mp / 8 : NNReal)
((hybridTuning n m eps).Mf / 8 : NNReal))
(hybridOrderedPrefixStatistic (hybridTuning n m eps)) (ateFunctional P) +
(exp (-((hybridTuning n m eps).h0 : Real)) +
exp (-((hybridTuning n m eps).Mp : Real)) +
exp (-((hybridTuning n m eps).Mf : Real))) ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) := by
obtain ⟨C, hC, hbound⟩ := hybrid_tuned_prefix_risk
refine ⟨C * (1281 : Real) ^ 2 + 5, by positivity, ?_⟩
intro n m d eps P hn hd heps heps4 hS hP
have hrisk := hbound n m d eps P hn hd heps heps4 hS hP
have hdegree := mul_le_mul_of_nonneg_left
(hybrid_finite_degree_rate n m d eps heps hS) hC.le
have hoverflow := hybrid_finite_overflow_rate n m eps
(hybrid_large_sample_size n eps heps4 hS) heps heps4
have hsquare := sq_nonneg
((d : Real) / (((n : Real) + m) * eps * logScale n eps))
calc
_ ≤ C * (1281 ^ 2 * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2)) +
5 / ((n : Real) * eps) := add_le_add (hrisk.trans hdegree) hoverflow
_ = (C * 1281 ^ 2 + 5) * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2) -
5 * ((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2 := by ring
_ ≤ _ := sub_le_self _ (mul_nonneg (by norm_num) hsquare)
theorem hybrid_risk_le_four
Under the stated inputs and conditions, The clipped finite hybrid estimator has squared risk at most four for every law. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: hybrid_risk_le_four
lemma hybrid_risk_le_four (n m d : Nat) (eps : Real) (P : DiscreteLaw d) :
ruleRisk (liftRule (hybridEstimator n m d eps)) P ≤ 4 := by
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have ht := ateFunctional_mem_Icc P
rw [hybrid_ruleRisk_eq_sqRisk]
unfold Causalean.Stat.sqRisk
calc
_ ≤ ∫ _s : Sample n m d, (4 : Real) ∂annotationLaw P n m := by
apply integral_mono_of_nonneg
(Filter.Eventually.of_forall (fun _ => sq_nonneg _)) (integrable_const 4)
apply Filter.Eventually.of_forall
intro s
have hs := hybridEstimator_mem_Icc n m d eps s
nlinarith [hs.1, hs.2, ht.1, ht.2]
_ = 4 := by simp
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.HybridUpper 1 declarations Gate-free original-record upper bound and range for the finite hybrid rule.
Gate-free original-record upper bound and range for the finite hybrid rule.
theorem hybrid_upper
The finite hybrid rule is measurable, clipped, and uniformly attains the frontier upper order.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
Proof (Lean source)
-- @node: hybrid_upper
lemma hybrid_upper :
∃ C : Real, 0 < C ∧ ∀ (n m d : Nat) (eps : Real),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
Measurable (hybridEstimator n m d eps) ∧
(∀ s, hybridEstimator n m d eps s ∈ Icc (-1) 1) ∧
worstRisk (liftRule (hybridEstimator n m d eps)) eps ≤ C * frontierRate n m d eps := by
obtain ⟨C, hC, hlarge⟩ := hybrid_large_information_risk
let D := max (max C 4) (exp 4096)
refine ⟨D, lt_of_lt_of_le hC
((le_max_left C 4).trans (le_max_left _ _)), ?_⟩
intro n m d eps hn hd heps heps4
refine ⟨hybridEstimator_measurable n m d eps,
hybridEstimator_mem_Icc n m d eps, ?_⟩
have hrate := (frontierRate_pos n m d eps hn heps).le
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps4⟩
letI : Nonempty (ClassLaw d eps) := ⟨P0⟩
unfold worstRisk
apply ciSup_le
intro P
by_cases hsmall : (n : Real) * eps < exp 4096
· calc
ruleRisk (liftRule (hybridEstimator n m d eps)) P.1 ≤ 1 :=
hybrid_small_information_risk n m d eps P.1 hsmall
_ ≤ exp 4096 * frontierRate n m d eps :=
hybrid_small_information_rate n m d eps hn heps hsmall
_ ≤ _ := mul_le_mul_of_nonneg_right (le_max_right _ _) hrate
· have hS := le_of_not_gt hsmall
by_cases hcap : 1 ≤ 1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * logScale n eps)) ^ 2
· rw [frontierRate, labelScale, min_eq_left hcap, mul_one]
exact (hybrid_risk_le_four n m d eps P.1).trans
((le_max_right C 4).trans (le_max_left _ _))
· rw [frontierRate, labelScale, min_eq_right (le_of_not_ge hcap)]
exact (hlarge n m d eps P.1 hn hd heps heps4 hS P.2).trans
(mul_le_mul_of_nonneg_right
((le_max_left C 4).trans (le_max_left _ _)) (by positivity))
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.InverseCountArmRisk 11 declarations Overlap-uniform inverse-count arm bias and variance under independent Poisson counts.
Overlap-uniform inverse-count arm bias and variance under independent Poisson counts.
theorem armMass_ge_overlap_cellMass
Under the stated inputs and conditions, Both arms inherit the occupied-cell overlap floor, including zero-mass cells. This gives the stated result.
Formal statement
hypotheses
conclusion
eps * cellMass P j ≤ armMass P j a
Proof (Lean source)
-- @node: armMass_ge_overlap_cellMass
lemma armMass_ge_overlap_cellMass {d : Nat} (P : DiscreteLaw d) (eps : Real)
(hP : ModelClass d eps P) (j : Fin d) (a : Bool) :
eps * cellMass P j ≤ armMass P j a := by
have hsum : cellMass P j = armMass P j true + armMass P j false := by
simp [cellMass, armMass]
by_cases hz : cellMass P j = 0
· simpa [hz] using armMass_nonneg P j a
· have hp : 0 < cellMass P j := lt_of_le_of_ne (cellMass_nonneg P j) (Ne.symm hz)
have he := hP.overlap j hp
simp only [propensity, if_neg hz] at he
have h1 := (le_div_iff₀ hp).mp he.1
have h0 := (div_le_iff₀ hp).mp he.2
cases a
· nlinarith
· exact h1
Under the stated inputs and conditions, The exponential missing-cell envelope has the sharp reciprocal-intensity bound. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: inverse_count_exp_envelope
lemma inverse_count_exp_envelope (p t eps : Real) (ht : 0 < t) (heps : 0 < eps) :
p * exp (-(t * eps * p)) ≤ 1 / (exp 1 * t * eps) := by
have h := Real.mul_exp_neg_le_exp_neg_one (t * eps * p)
apply (le_div_iff₀ (by positivity : 0 < exp 1 * t * eps)).2
have hscaled := mul_le_mul_of_nonneg_right h (Real.exp_pos 1).le
rw [Real.exp_neg 1, inv_mul_cancel₀ (ne_of_gt (Real.exp_pos 1))] at hscaled
nlinarith
theorem inverse_count_missing_mass_bound
Under the stated inputs and conditions, The sum of exponential missing-cell masses is capped by one and by the cell budget. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: inverse_count_missing_mass_bound
lemma inverse_count_missing_mass_bound {d : Nat} (P : DiscreteLaw d) (t eps : Real)
(ht : 0 < t) (heps : 0 < eps) :
(∑ j, cellMass P j * exp (-(t * eps * cellMass P j))) ≤
min 1 ((d : Real) / (exp 1 * t * eps)) := by
apply le_min
· calc
(∑ j, cellMass P j * exp (-(t * eps * cellMass P j))) ≤
∑ j, cellMass P j := by
apply Finset.sum_le_sum
intro j _
apply mul_le_of_le_one_right (cellMass_nonneg P j)
exact Real.exp_le_one_iff.mpr (neg_nonpos.mpr
(mul_nonneg (mul_nonneg ht.le heps.le) (cellMass_nonneg P j)))
_ = 1 := sum_cellMass P
· calc
(∑ j, cellMass P j * exp (-(t * eps * cellMass P j))) ≤
∑ _j : Fin d, 1 / (exp 1 * t * eps) := by
exact Finset.sum_le_sum (fun j _ => inverse_count_exp_envelope _ _ _ ht heps)
_ = (d : Real) / (exp 1 * t * eps) := by simp [div_eq_mul_inv]
theorem inverse_count_bias_bound
Under the stated inputs and conditions, The explicit inverse-count bias obeys the overlap-uniform bound without excluding null cells. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: inverse_count_bias_bound
lemma inverse_count_bias_bound {d : Nat} (P : DiscreteLaw d) (a : Bool) (t eps : Real)
(ht : 0 < t) (heps : 0 < eps) (hP : ModelClass d eps P) :
|∑ j, armMass P j (!a) * outcomeMean P a j * exp (-(t * armMass P j a))| ≤
min 1 ((d : Real) / (exp 1 * t * eps)) := by
have hnonneg : 0 ≤ ∑ j, armMass P j (!a) * outcomeMean P a j *
exp (-(t * armMass P j a)) := by
exact Finset.sum_nonneg (fun j _ => mul_nonneg
(mul_nonneg (armMass_nonneg P j (!a)) (outcomeMean_mem_Icc P a j).1)
(Real.exp_pos _).le)
rw [abs_of_nonneg hnonneg]
apply le_trans _ (inverse_count_missing_mass_bound P t eps ht heps)
apply Finset.sum_le_sum
intro j _
have hv : armMass P j (!a) ≤ cellMass P j := by
have hsum : cellMass P j = armMass P j true + armMass P j false := by
simp [cellMass, armMass]
cases a <;> simp only [Bool.not_false, Bool.not_true] <;>
linarith [armMass_nonneg P j false, armMass_nonneg P j true]
have hfactor : armMass P j (!a) * outcomeMean P a j ≤ cellMass P j :=
(mul_le_of_le_one_right (armMass_nonneg P j (!a)) (outcomeMean_mem_Icc P a j).2).trans hv
apply mul_le_mul hfactor _ (Real.exp_pos _).le (cellMass_nonneg P j)
apply Real.exp_le_exp.mpr
have hs := mul_le_mul_of_nonneg_left (armMass_ge_overlap_cellMass P eps hP j a) ht.le
nlinarith
theorem inverse_count_cell_mean_identity
Under the stated inputs and conditions, The Poisson reciprocal first-moment formula gives target minus missing-cell bias. This gives the stated result.
Formal statement
hypotheses
conclusion
markedMass P j a * (1 + t * armMass P j (!a) * (if t * armMass P j a = 0 then 1 else (1 - exp (-(t * armMass P j a))) / (t * armMass P j a)))
= cellMass P j * outcomeMean P a j
Proof (Lean source)
-- @node: inverse_count_cell_mean_identity
lemma inverse_count_cell_mean_identity {d : Nat} (P : DiscreteLaw d) (a : Bool)
(j : Fin d) (t : Real) (ht : 0 < t) :
markedMass P j a * (1 + t * armMass P j (!a) *
(if t * armMass P j a = 0 then 1 else
(1 - exp (-(t * armMass P j a))) / (t * armMass P j a))) =
cellMass P j * outcomeMean P a j -
armMass P j (!a) * outcomeMean P a j * exp (-(t * armMass P j a)) := by
have hsum : cellMass P j = armMass P j a + armMass P j (!a) := by
cases a <;> simp [cellMass, armMass, add_comm]
by_cases hs : armMass P j a = 0
· have hq : markedMass P j a = 0 := by
have hmass : armMass P j a = jointMass P j a true + jointMass P j a false := by
simp [armMass]
dsimp only [markedMass]
linarith [jointMass_nonneg P j a true, jointMass_nonneg P j a false]
simp [hs, hq, outcomeMean]
· rw [if_neg (mul_ne_zero ht.ne' hs), hsum]
dsimp only [outcomeMean]
field_simp
ring
theorem inverse_count_mean_bias_bound
Under the stated inputs and conditions, Summed cell means satisfy the claimed capped bias bound once the Poisson means are read back. This gives the stated result.
Formal statement
hypotheses
conclusion
|(∑ j, markedMass P j a * (1 + t * armMass P j (!a) * (if t * armMass P j a = 0 then 1 else (1 - exp (-(t * armMass P j a))) / (t * armMass P j a))))
- (∑ j, cellMass P j * outcomeMean P a j)|
Proof (Lean source)
-- @node: inverse_count_mean_bias_bound
lemma inverse_count_mean_bias_bound {d : Nat} (P : DiscreteLaw d) (a : Bool) (t eps : Real)
(ht : 0 < t) (heps : 0 < eps) (hP : ModelClass d eps P) :
|(∑ j, markedMass P j a * (1 + t * armMass P j (!a) *
(if t * armMass P j a = 0 then 1 else
(1 - exp (-(t * armMass P j a))) / (t * armMass P j a)))) -
(∑ j, cellMass P j * outcomeMean P a j)| ≤
min 1 ((d : Real) / (exp 1 * t * eps)) := by
simp_rw [inverse_count_cell_mean_identity P a _ t ht]
rw [Finset.sum_sub_distrib, sub_sub_cancel_left, abs_neg]
exact inverse_count_bias_bound P a t eps ht heps hP
Under the stated hypotheses, Pull back Poisson integrability and the exact count mean along a measurable count. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
Integrable (fun om => (X om : Real)) mu
Proof (Lean source)
-- @node: inverse_count_count_moment
lemma inverse_count_count_moment {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (X : Om → Nat) (lam : NNReal)
(hX : Measurable X) (hlaw : mu.map X = poissonMeasure lam) :
Integrable (fun om => (X om : Real)) mu ∧
(∫ om, (X om : Real) ∂mu) = (lam : Real) := by
have hi := (poisson_natCast_memLp_two lam).integrable
(by norm_num)
have hi' : Integrable (fun k : Nat => (k : Real)) (mu.map X) := hlaw.symm ▸ hi
refine ⟨hi'.comp_measurable hX, ?_⟩
rw [← integral_map hX.aemeasurable (by fun_prop), hlaw]
exact poisson_natCast_first_moment lam
Under the stated inputs and conditions, Independent Poisson factors give the exact inverse-count cell mean and integrability. This gives the stated result.
Formal statement
hypotheses
Om :
Type
mu :
u :
hX :
∀ i, Measurable (X i)
hlaw :
∀ i, mu.map (X i) = poissonMeasure (lam i)
hind :
iIndepFun X mu
conclusion
conclusion 1
Proof (Lean source)
-- @node: inverse_count_poisson_cell_mean
lemma inverse_count_poisson_cell_mean {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (X : Fin 3 → Om → Nat)
(lam : Fin 3 → NNReal) (u : Real)
(hX : ∀ i, Measurable (X i)) (hlaw : ∀ i, mu.map (X i) = poissonMeasure (lam i))
(hind : iIndepFun X mu) :
Integrable (fun om => (X 0 om : Real) / u *
(1 + (X 2 om : Real) / ((X 1 om : Real) + 1))) mu ∧
(∫ om, (X 0 om : Real) / u *
(1 + (X 2 om : Real) / ((X 1 om : Real) + 1)) ∂mu) =
(lam 0 : Real) / u * (1 + (lam 2 : Real) *
(if (lam 1 : Real) = 0 then 1 else
(1 - exp (-(lam 1 : Real))) / (lam 1 : Real))) := by
let f : Fin 3 → Nat → Real := fun i k =>
if i = 1 then ((k : Real) + 1)⁻¹ else (k : Real)
let V : Fin 3 → Om → Real := fun i om => f i (X i om)
have hm (i) : Measurable (V i) := by
dsimp [V, f]
split_ifs <;> fun_prop
have hi (i) : Integrable (V i) mu := by
fin_cases i
· simpa [V, f] using (inverse_count_count_moment mu (X 0) (lam 0) (hX 0) (hlaw 0)).1
· have hg := (shifted_reciprocal_memLp_two
(lam 1)).integrable (by norm_num)
have hg' : Integrable (fun k : Nat => ((k : Real) + 1)⁻¹) (mu.map (X 1)) := by
simpa only [hlaw 1, Nat.cast_add, Nat.cast_one] using hg
simpa [V, f, Function.comp_def] using hg'.comp_measurable (hX 1)
· simpa [V, f] using (inverse_count_count_moment mu (X 2) (lam 2) (hX 2) (hlaw 2)).1
have hv : iIndepFun V mu := hind.comp f (fun _ => measurable_of_countable _)
have h01 : IndepFun (V 0) (V 1) mu := hv.indepFun (by decide)
have hi01 := h01.integrable_mul (hi 0) (hi 1)
have hpair : IndepFun (V 0 * V 1) (V 2) mu := hv.indepFun_mul_left (fun i => hm i) 0 1 2
(show (0 : Fin 3) ≠ 2 by decide) (show (1 : Fin 3) ≠ 2 by decide)
have h012 := hpair.integrable_mul hi01 (hi 2)
have heq : (fun om => (X 0 om : Real) / u *
(1 + (X 2 om : Real) / ((X 1 om : Real) + 1))) =
(fun om => V 0 om / u + (V 0 om * V 1 om * V 2 om) / u) := by
funext om
simp [V, f, div_eq_mul_inv]
ring
rw [heq]
refine ⟨((hi 0).div_const u).add (h012.div_const u), ?_⟩
have hmean0 := (inverse_count_count_moment mu (X 0) (lam 0) (hX 0) (hlaw 0)).2
have hmean2 := (inverse_count_count_moment mu (X 2) (lam 2) (hX 2) (hlaw 2)).2
have hmean1 : (∫ om, V 1 om ∂mu) =
(if (lam 1 : Real) = 0 then 1 else
(1 - exp (-(lam 1 : Real))) / (lam 1 : Real)) := by
dsimp [V, f]
rw [← integral_map (f := fun k : Nat => ((k : Real) + 1)⁻¹)
(hX 1).aemeasurable (by fun_prop), hlaw 1]
exact (poisson_inverse_moments (lam 1)).1
have hp' : (∫ om, V 0 om * V 1 om * V 2 om ∂mu) =
(∫ om, V 0 om ∂mu) * (∫ om, V 1 om ∂mu) * (∫ om, V 2 om ∂mu) := by
change (∫ om, ((V 0 * V 1) * V 2) om ∂mu) = _
rw [hpair.integral_mul_eq_mul_integral hi01.aestronglyMeasurable (hi 2).aestronglyMeasurable,
h01.integral_mul_eq_mul_integral (hi 0).aestronglyMeasurable (hi 1).aestronglyMeasurable]
have hdiv0 := (hi 0).div_const u
have hdiv012 : Integrable (fun om => V 0 om * V 1 om * V 2 om / u) mu :=
h012.div_const u
integral_linearity
rw [hp', hmean1]
simp only [V, f, if_false, show (0 : Fin 3) ≠ 1 by decide,
show (2 : Fin 3) ≠ 1 by decide] at *
rw [hmean0, hmean2]
ring
theorem inverse_count_poisson_arm_mean
Under the stated hypotheses, Reading back all independent Poisson cell means gives the arm expectation, including null cells. This gives the stated result.
Formal statement
hypotheses
d :
P :
a :
u t :
hu :
0 < u
Om :
Type
mu :
hmeas :
hZ :
hK :
hW :
conclusion
conclusion 1
Integrable (fun om => ∑ j, (Z j om : Real) / (u : Real) * (1 + (W j om : Real) / ((K j om : Real) + 1))) mu
Proof (Lean source)
-- @node: inverse_count_poisson_arm_mean
lemma inverse_count_poisson_arm_mean {d : Nat} (P : DiscreteLaw d) (a : Bool)
(u t : NNReal) (hu : 0 < u) {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (Z K W : Fin d → Om → Nat)
(hmeas : ∀ j, Measurable (Z j) ∧ Measurable (K j) ∧ Measurable (W j))
(hZ : ∀ j, mu.map (Z j) = poissonMeasure (u * toNNReal (markedMass P j a)))
(hK : ∀ j, mu.map (K j) = poissonMeasure (t * toNNReal (armMass P j a)))
(hW : ∀ j, mu.map (W j) = poissonMeasure (t * toNNReal (armMass P j (!a))))
(hind : iIndepFun
(fun i : Fin 3 × Fin d => if i.1 = 0 then Z i.2 else if i.1 = 1 then K i.2 else W i.2) mu) :
Integrable (fun om => ∑ j, (Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1))) mu ∧
(∫ om, ∑ j, (Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1)) ∂mu) =
∑ j, markedMass P j a * (1 + (t : Real) * armMass P j (!a) *
(if (t : Real) * armMass P j a = 0 then 1 else
(1 - exp (-((t : Real) * armMass P j a))) / ((t : Real) * armMass P j a))) := by
have hcell (j : Fin d) :
Integrable (fun om => (Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1))) mu ∧
(∫ om, (Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1)) ∂mu) =
markedMass P j a * (1 + (t : Real) * armMass P j (!a) *
(if (t : Real) * armMass P j a = 0 then 1 else
(1 - exp (-((t : Real) * armMass P j a))) / ((t : Real) * armMass P j a))) := by
let X : Fin 3 → Om → Nat := fun i => if i = 0 then Z j else if i = 1 then K j else W j
let lam : Fin 3 → NNReal := fun i => if i = 0 then u * toNNReal (markedMass P j a)
else if i = 1 then t * toNNReal (armMass P j a)
else t * toNNReal (armMass P j (!a))
have hx (i) : Measurable (X i) := by
dsimp [X]
split_ifs <;> first | exact (hmeas j).1 | exact (hmeas j).2.1 | exact (hmeas j).2.2
have hl (i) : mu.map (X i) = poissonMeasure (lam i) := by
dsimp [X, lam]
split_ifs <;> first | exact hZ j | exact hK j | exact hW j
have hind' : iIndepFun X mu :=
hind.precomp (g := fun i : Fin 3 => (i, j))
(fun i i' h => (Prod.mk.inj h).1)
have h := inverse_count_poisson_cell_mean mu X lam u hx hl hind'
have hq : 0 ≤ markedMass P j a := jointMass_nonneg P j a true
have hu' : (u : Real) ≠ 0 := (NNReal.coe_pos.mpr hu).ne'
simpa [X, lam, NNReal.coe_mul, Real.coe_toNNReal _ hq,
Real.coe_toNNReal _ (armMass_nonneg P j a),
Real.coe_toNNReal _ (armMass_nonneg P j (!a)), mul_div_cancel_left₀ _ hu'] using h
refine ⟨integrable_finset_sum _ (fun j _ => (hcell j).1), ?_⟩
have hi (j : Fin d) : Integrable (fun om => (Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1))) mu := (hcell j).1
integral_linearity
exact Finset.sum_congr rfl (fun j _ => (hcell j).2)
theorem inverse_count_arm_variance_envelope
Under the stated inputs and conditions, Summing the numerical Poisson cell envelopes gives a universal arm envelope. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ j : Fin d, (markedMass P j a / u * (2 + 2 * (2 ^ 16 : Real) * ((t * armMass P j (!a)) ^ 2 + t * armMass P j (!a)) / (1 + t * armMass P j a) ^ 2) + markedMass P j a ^ 2 * (2 ^ 16 : Real) * (t * armMass P j (!a) / (1 + t * armMass P j a) ^ 2 + (t * armMass P j (!a)) ^ 2 * (t * armMass P j a) / (1 + t * armMass P j a) ^ 4)))
≤ 2 ^ 20 * (1 / (u * eps) + 1 / (t * eps))
Proof (Lean source)
-- @node: inverse_count_arm_variance_envelope
lemma inverse_count_arm_variance_envelope {d : Nat} (P : DiscreteLaw d) (a : Bool)
(u t eps : Real) (hu : 0 < u) (ht : 0 < t) (heps : 0 < eps)
(hP : ModelClass d eps P) :
(∑ j : Fin d,
(markedMass P j a / u * (2 + 2 * (2 ^ 16 : Real) *
((t * armMass P j (!a)) ^ 2 + t * armMass P j (!a)) /
(1 + t * armMass P j a) ^ 2) +
markedMass P j a ^ 2 * (2 ^ 16 : Real) *
(t * armMass P j (!a) / (1 + t * armMass P j a) ^ 2 +
(t * armMass P j (!a)) ^ 2 * (t * armMass P j a) /
(1 + t * armMass P j a) ^ 4))) ≤
2 ^ 20 * (1 / (u * eps) + 1 / (t * eps)) := by
have hcell (j : Fin d) := inverse_count_cell_variance_envelope
(armMass P j a) (armMass P j (!a)) (markedMass P j a) u t eps (2 ^ 16)
(armMass_nonneg P j a) (armMass_nonneg P j (!a)) (jointMass_nonneg P j a true)
(show markedMass P j a ≤ armMass P j a by
simp only [markedMass, armMass, Fintype.sum_bool]
linarith [jointMass_nonneg P j a false]) hu ht heps (by positivity)
(by
have hsum : armMass P j a + armMass P j (!a) = cellMass P j := by
cases a <;> simp [cellMass, armMass, add_comm]
rw [hsum]
exact armMass_ge_overlap_cellMass P eps hP j a)
calc
_ ≤ ∑ j : Fin d, (2 + 4 * (2 ^ 16 : Real)) *
(armMass P j a + armMass P j (!a)) * (1 / (u * eps) + 1 / (t * eps)) :=
Finset.sum_le_sum (fun j _ => hcell j)
_ = (2 + 4 * (2 ^ 16 : Real)) * (1 / (u * eps) + 1 / (t * eps)) := by
have hmass : ∑ j : Fin d, (armMass P j a + armMass P j (!a)) = 1 := by
convert sum_cellMass P using 1
apply Finset.sum_congr rfl
intro j _
cases a <;> simp [cellMass, armMass, add_comm]
rw [← Finset.sum_mul, ← Finset.mul_sum, hmass, mul_one]
_ ≤ _ := mul_le_mul_of_nonneg_right (by norm_num) (by positivity)
theorem inverse_count_arm_risk
Under the stated inputs and conditions, Independent Poisson count families yield the overlap-explicit arm bias and variance guarantee. This gives the stated conclusion.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
d :
eps :
P :
a :
u t :
Om :
Type
mu :
Measure Om
2 ≤ d
0 < eps
eps ≤ 1 / 4
ModelClass d eps P
0 < u
0 < t
Hs :
psi :
Proof (Lean source)
lemma inverse_count_arm_risk :
∃ C : Real, 0 < C ∧
∀ (d : Nat) (eps : Real) (P : DiscreteLaw d) (a : Bool) (u t : NNReal)
(Om : Type) [MeasurableSpace Om] (mu : Measure Om) [IsProbabilityMeasure mu]
(Z K W : Fin d → Om → Nat),
2 ≤ d → 0 < eps → eps ≤ 1 / 4 → ModelClass d eps P → 0 < u → 0 < t →
(∀ j, Measurable (Z j) ∧ Measurable (K j) ∧ Measurable (W j)) →
(∀ j, mu.map (Z j) = poissonMeasure (u * toNNReal (markedMass P j a))) →
(∀ j, mu.map (K j) = poissonMeasure (t * toNNReal (armMass P j a))) →
(∀ j, mu.map (W j) = poissonMeasure (t * toNNReal (armMass P j (!a)))) →
iIndepFun
(fun i : Fin 3 × Fin d => if i.1 = 0 then Z i.2 else if i.1 = 1 then K i.2 else W i.2) mu →
let Hs : Om → Real := fun om => ∑ j : Fin d,
(Z j om : Real) / (u : Real) * (1 + (W j om : Real) / ((K j om : Real) + 1))
let psi : Real := ∑ j : Fin d, cellMass P j * outcomeMean P a j
|(∫ om, Hs om ∂mu) - psi| ≤ min 1 ((d : Real) / (exp 1 * (t : Real) * eps)) ∧
variance Hs mu ≤ C * (1 / ((u : Real) * eps) + 1 / ((t : Real) * eps)) :=
by
refine ⟨2 ^ 20, by norm_num, ?_⟩
intro d eps P a u t Om _ mu _ Z K W hd heps heps' hP hu ht hmeas hZ hK hW hind
dsimp only
constructor
· rw [(inverse_count_poisson_arm_mean P a u t hu mu Z K W hmeas hZ hK hW hind).2]
exact inverse_count_mean_bias_bound P a t eps (NNReal.coe_pos.mpr ht) heps hP
· have hvariance :
variance (fun om => ∑ j : Fin d,
(Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1))) mu ≤
∑ j : Fin d,
(markedMass P j a / (u : Real) * (2 + 2 * (2 ^ 16 : Real) *
(((t : Real) * armMass P j (!a)) ^ 2 +
(t : Real) * armMass P j (!a)) /
(1 + (t : Real) * armMass P j a) ^ 2) +
markedMass P j a ^ 2 * (2 ^ 16 : Real) *
((t : Real) * armMass P j (!a) / (1 + (t : Real) * armMass P j a) ^ 2 +
((t : Real) * armMass P j (!a)) ^ 2 * ((t : Real) * armMass P j a) /
(1 + (t : Real) * armMass P j a) ^ 4)) := by
let H : Fin d → Om → Real := fun j om =>
(Z j om : Real) / (u : Real) *
(1 + (W j om : Real) / ((K j om : Real) + 1))
have hcell (j : Fin d) : MemLp (H j) 2 mu ∧ variance (H j) mu ≤
markedMass P j a / (u : Real) * (2 + 2 * (2 ^ 16 : Real) *
(((t : Real) * armMass P j (!a)) ^ 2 +
(t : Real) * armMass P j (!a)) / (1 + (t : Real) * armMass P j a) ^ 2) +
markedMass P j a ^ 2 * (2 ^ 16 : Real) *
((t : Real) * armMass P j (!a) / (1 + (t : Real) * armMass P j a) ^ 2 +
((t : Real) * armMass P j (!a)) ^ 2 * ((t : Real) * armMass P j a) /
(1 + (t : Real) * armMass P j a) ^ 4) := by
let X : Fin 3 → Om → Nat :=
fun i => if i = 0 then Z j else if i = 1 then K j else W j
let lam : Fin 3 → NNReal :=
fun i => if i = 0 then u * toNNReal (markedMass P j a)
else if i = 1 then t * toNNReal (armMass P j a)
else t * toNNReal (armMass P j (!a))
have hx (i) : Measurable (X i) := by
dsimp [X]
split_ifs <;> first
| exact (hmeas j).1
| exact (hmeas j).2.1
| exact (hmeas j).2.2
have hl (i) : mu.map (X i) = poissonMeasure (lam i) := by
dsimp [X, lam]
split_ifs <;> first | exact hZ j | exact hK j | exact hW j
have hind' : iIndepFun X mu :=
hind.precomp (g := fun i : Fin 3 => (i, j))
(fun i i' h => (Prod.mk.inj h).1)
have h := inverse_count_poisson_cell_variance mu X lam u hx hl hind'
have hq : 0 ≤ markedMass P j a := jointMass_nonneg P j a true
have hu' : (u : Real) ≠ 0 := (NNReal.coe_pos.mpr hu).ne'
have hscale : (u : Real) * markedMass P j a / (u : Real) ^ 2 =
markedMass P j a / (u : Real) := by
field_simp
simpa [X, lam, H, NNReal.coe_mul,
Real.coe_toNNReal _ hq,
Real.coe_toNNReal _ (armMass_nonneg P j a),
Real.coe_toNNReal _ (armMass_nonneg P j (!a)),
mul_div_cancel_left₀ _ hu', hscale] using h
have hpair : Pairwise (↑(Finset.univ : Finset (Fin d)))
(fun j k => IndepFun (H j) (H k) mu) := by
intro j _ k _ hjk
let X : Fin 3 × Fin d → Om → Nat := fun i =>
if i.1 = 0 then Z i.2 else if i.1 = 1 then K i.2 else W i.2
have hx (i) : Measurable (X i) := by
dsimp [X]
split_ifs <;> first
| exact (hmeas i.2).1
| exact (hmeas i.2).2.1
| exact (hmeas i.2).2.2
simpa [X, H] using inverse_count_cell_blocks_independent mu X hx hind u j k hjk
have hsum := IndepFun.variance_sum
(s := Finset.univ) (X := H) (fun j _ => (hcell j).1) hpair
change variance (fun om => ∑ j, H j om) mu ≤ _
have hsum' : variance (fun om => ∑ j, H j om) mu = ∑ j, variance (H j) mu := by
simpa only [← Finset.sum_fn] using hsum
rw [hsum']
exact Finset.sum_le_sum (fun j _ => (hcell j).2)
exact hvariance.trans (inverse_count_arm_variance_envelope P a u t eps
(NNReal.coe_pos.mpr hu) (NNReal.coe_pos.mpr ht) heps hP)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.InverseCountProductMoments 7 declarations Independent product moments for the inverse-count arm statistic.
Independent product moments for the inverse-count arm statistic.
theorem inverse_count_independent_product_memLp Lemma inverse_count_independent_product_memLp in the paper ↗
Under the stated inputs and conditions, Independent square-integrable factors have a square-integrable product. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: inverse_count_independent_product_memLp
lemma inverse_count_independent_product_memLp {Om : Type} [MeasurableSpace Om]
{mu : Measure Om} {X Y : Om → Real}
(hX : MemLp X 2 mu) (hY : MemLp Y 2 mu) (hind : IndepFun X Y mu) :
MemLp (fun om => X om * Y om) 2 mu := by
apply (memLp_two_iff_integrable_sq (hX.aestronglyMeasurable.mul
hY.aestronglyMeasurable)).2
have hi := (hind.comp (φ := fun x : Real => x ^ 2) (ψ := fun y : Real => y ^ 2)
(by fun_prop) (by fun_prop)).integrable_mul hX.integrable_sq hY.integrable_sq
change Integrable (fun om => X om ^ 2 * Y om ^ 2) mu at hi
change Integrable (fun om => (X om * Y om) ^ 2) mu
simpa only [mul_pow] using hi
theorem inverse_count_independent_product_variance Lemma inverse_count_independent_product_variance in the paper ↗
Under the stated inputs and conditions, The exact independent-product variance identity separates count and weight fluctuations. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: inverse_count_independent_product_variance
lemma inverse_count_independent_product_variance {Om : Type} [MeasurableSpace Om]
{mu : Measure Om} [IsProbabilityMeasure mu] {X Y : Om → Real}
(hX : MemLp X 2 mu) (hY : MemLp Y 2 mu) (hind : IndepFun X Y mu) :
variance (fun om => X om * Y om) mu =
variance X mu * (∫ om, Y om ^ 2 ∂mu) +
(∫ om, X om ∂mu) ^ 2 * variance Y mu := by
have hm := hind.integral_mul_eq_mul_integral hX.aestronglyMeasurable
hY.aestronglyMeasurable
have hs := (hind.comp (φ := fun x : Real => x ^ 2) (ψ := fun y : Real => y ^ 2)
(by fun_prop) (by fun_prop)).integral_mul_eq_mul_integral
hX.integrable_sq.aestronglyMeasurable hY.integrable_sq.aestronglyMeasurable
rw [variance_eq_sub (inverse_count_independent_product_memLp hX hY hind),
variance_eq_sub hX, variance_eq_sub hY]
simp only [Pi.pow_apply, mul_pow, Pi.mul_apply, Function.comp_def] at hm hs ⊢
rw [hm, hs]
ring
theorem inverse_count_poisson_count_moments Lemma inverse_count_poisson_count_moments in the paper ↗
Under the stated hypotheses, A measurable Poisson count inherits its first two moments and variance. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: inverse_count_poisson_count_moments
lemma inverse_count_poisson_count_moments {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (X : Om → Nat) (lam : NNReal)
(hX : Measurable X) (hlaw : mu.map X = poissonMeasure lam) :
MemLp (fun om => (X om : Real)) 2 mu ∧
(∫ om, (X om : Real) ∂mu) = (lam : Real) ∧
(∫ om, (X om : Real) ^ 2 ∂mu) = (lam : Real) ^ 2 + (lam : Real) ∧
variance (fun om => (X om : Real)) mu = (lam : Real) := by
have hLp : MemLp (fun om => (X om : Real)) 2 mu :=
MeasureTheory.MemLp.comp_measurePreserving
(poisson_natCast_memLp_two lam) ⟨hX, hlaw⟩
have hmean : (∫ om, (X om : Real) ∂mu) = (lam : Real) := by
rw [← integral_map hX.aemeasurable (by fun_prop), hlaw]
exact Causalean.Mathlib.Probability.Poisson.poisson_natCast_first_moment lam
have hsq : (∫ om, (X om : Real) ^ 2 ∂mu) = (lam : Real) ^ 2 + (lam : Real) := by
rw [← integral_map (f := fun k : Nat => (k : Real) ^ 2) hX.aemeasurable
(by fun_prop), hlaw]
exact poisson_count_second_moment lam
refine ⟨hLp, hmean, hsq, ?_⟩
rw [variance_eq_sub hLp]
change (∫ om, (X om : Real) ^ 2 ∂mu) - (∫ om, (X om : Real) ∂mu) ^ 2 = _
rw [hsq, hmean]
ring
theorem inverse_count_poisson_reciprocal_moments Lemma inverse_count_poisson_reciprocal_moments in the paper ↗
Under the stated hypotheses, A measurable Poisson count inherits reciprocal square-integrability and moment bounds. This gives the stated result.
Formal statement
hypotheses
Proof (Lean source)
-- @node: inverse_count_poisson_reciprocal_moments
lemma inverse_count_poisson_reciprocal_moments {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (K : Om → Nat) (lam : NNReal)
(hK : Measurable K) (hlaw : mu.map K = poissonMeasure lam) :
MemLp (fun om => ((K om : Real) + 1)⁻¹) 2 mu ∧
(∫ om, (((K om : Real) + 1)⁻¹) ^ 2 ∂mu) ≤
2 ^ 16 / (1 + (lam : Real)) ^ 2 ∧
variance (fun om => ((K om : Real) + 1)⁻¹) mu ≤
2 ^ 16 * (lam : Real) / (1 + (lam : Real)) ^ 4 := by
have hg : MemLp (fun k : Nat => ((k : Real) + 1)⁻¹) 2 (poissonMeasure lam) := by
simpa only [Nat.cast_add, Nat.cast_one] using
shifted_reciprocal_memLp_two lam
refine ⟨hg.comp_measurePreserving ⟨hK, hlaw⟩, ?_, ?_⟩
· rw [← integral_map (f := fun k : Nat => (((k : Real) + 1)⁻¹) ^ 2)
hK.aemeasurable (by fun_prop), hlaw]
exact (poisson_inverse_moments lam).2.1
· have hp : MeasurePreserving K mu (poissonMeasure lam) := ⟨hK, hlaw⟩
rw [hp.variance_fun_comp (f := fun k : Nat => ((k : Real) + 1)⁻¹) (by fun_prop)]
exact (poisson_inverse_moments lam).2.2
Under the stated hypotheses, Independence yields both moment bounds for the marginal inverse-count weight. This gives the stated result.
Formal statement
hypotheses
Om :
Type
mu :
K W :
Om → Nat
lam rho :
hK :
hW :
hKlaw :
mu.map K = poissonMeasure lam
hWlaw :
mu.map W = poissonMeasure rho
hind :
IndepFun W K mu
conclusion
conclusion 2
Proof (Lean source)
-- @node: inverse_count_weight_moments
lemma inverse_count_weight_moments {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (K W : Om → Nat) (lam rho : NNReal)
(hK : Measurable K) (hW : Measurable W)
(hKlaw : mu.map K = poissonMeasure lam) (hWlaw : mu.map W = poissonMeasure rho)
(hind : IndepFun W K mu) :
MemLp (fun om => 1 + (W om : Real) / ((K om : Real) + 1)) 2 mu ∧
(∫ om, (1 + (W om : Real) / ((K om : Real) + 1)) ^ 2 ∂mu) ≤
2 + 2 * (2 ^ 16 : Real) * ((rho : Real) ^ 2 + rho) / (1 + (lam : Real)) ^ 2 ∧
variance (fun om => 1 + (W om : Real) / ((K om : Real) + 1)) mu ≤
(2 ^ 16 : Real) * ((rho : Real) / (1 + (lam : Real)) ^ 2 +
(rho : Real) ^ 2 * (lam : Real) / (1 + (lam : Real)) ^ 4) := by
obtain ⟨hw, hwm, hws, hwv⟩ := inverse_count_poisson_count_moments mu W rho hW hWlaw
obtain ⟨hg, hgs, hgv⟩ := inverse_count_poisson_reciprocal_moments mu K lam hK hKlaw
let V : Om → Real := fun om => (W om : Real) * ((K om : Real) + 1)⁻¹
have hi : IndepFun (fun om => (W om : Real))
(fun om => ((K om : Real) + 1)⁻¹) mu :=
hind.comp (φ := fun k : Nat => (k : Real))
(ψ := fun k : Nat => ((k : Real) + 1)⁻¹) (by fun_prop) (by fun_prop)
have hv : MemLp V 2 mu := inverse_count_independent_product_memLp hw hg hi
have hs : (∫ om, V om ^ 2 ∂mu) =
((rho : Real) ^ 2 + rho) * (∫ om, (((K om : Real) + 1)⁻¹) ^ 2 ∂mu) := by
have hs := (hi.comp (φ := fun x : Real => x ^ 2)
(ψ := fun x : Real => x ^ 2) (by fun_prop) (by fun_prop)).integral_mul_eq_mul_integral
hw.integrable_sq.aestronglyMeasurable
hg.integrable_sq.aestronglyMeasurable
simpa only [V, mul_pow, Function.comp_def, Pi.mul_apply, hws] using hs
have hvv := inverse_count_independent_product_variance hw hg hi
change variance V mu = _ at hvv
rw [hwv, hwm] at hvv
have hLp : MemLp (fun om => 1 + V om) 2 mu := (memLp_const (μ := mu) (1 : Real)).add hv
change MemLp (fun om => 1 + V om) 2 mu ∧
(∫ om, (1 + V om) ^ 2 ∂mu) ≤ _ ∧ variance (fun om => 1 + V om) mu ≤ _
refine ⟨hLp, ?_, ?_⟩
· calc
(∫ om, (1 + V om) ^ 2 ∂mu) ≤ ∫ om, 2 + 2 * V om ^ 2 ∂mu := by
apply integral_mono hLp.integrable_sq
((integrable_const 2).add (hv.integrable_sq.const_mul 2))
intro om
change (1 + V om) ^ 2 ≤ 2 + 2 * V om ^ 2
nlinarith only [sq_nonneg (V om - 1)]
_ = 2 + 2 * (∫ om, V om ^ 2 ∂mu) := by
rw [integral_add (integrable_const 2) (hv.integrable_sq.const_mul 2),
integral_const, integral_const_mul]
simp
_ ≤ _ := by
rw [hs]
have hb := mul_le_mul_of_nonneg_left hgs
(show 0 ≤ (rho : Real) ^ 2 + rho by positivity)
convert add_le_add_left
(mul_le_mul_of_nonneg_left hb (by norm_num : (0 : Real) ≤ 2)) 2 using 1 <;>
first | rfl | ring
· rw [variance_const_add hv.aestronglyMeasurable 1, hvv]
have hb := mul_le_mul_of_nonneg_left hgs rho.coe_nonneg
have hc := mul_le_mul_of_nonneg_left hgv (sq_nonneg (rho : Real))
calc
_ ≤ (rho : Real) * (2 ^ 16 / (1 + (lam : Real)) ^ 2) +
(rho : Real) ^ 2 * (2 ^ 16 * (lam : Real) / (1 + (lam : Real)) ^ 4) :=
add_le_add hb hc
_ = _ := by ring
theorem inverse_count_poisson_cell_variance Lemma inverse_count_poisson_cell_variance in the paper ↗
Under the stated inputs and conditions, All three independent Poisson factors give the cell variance envelope. This gives the stated result.
Formal statement
hypotheses
Om :
Type
mu :
u :
hX :
∀ i, Measurable (X i)
hlaw :
∀ i, mu.map (X i) = poissonMeasure (lam i)
hind :
iIndepFun X mu
Proof (Lean source)
-- @node: inverse_count_poisson_cell_variance
lemma inverse_count_poisson_cell_variance {Om : Type} [MeasurableSpace Om]
(mu : Measure Om) [IsProbabilityMeasure mu] (X : Fin 3 → Om → Nat)
(lam : Fin 3 → NNReal) (u : Real)
(hX : ∀ i, Measurable (X i)) (hlaw : ∀ i, mu.map (X i) = poissonMeasure (lam i))
(hind : iIndepFun X mu) :
MemLp (fun om => (X 0 om : Real) / u *
(1 + (X 2 om : Real) / ((X 1 om : Real) + 1))) 2 mu ∧
variance (fun om => (X 0 om : Real) / u *
(1 + (X 2 om : Real) / ((X 1 om : Real) + 1))) mu ≤
(lam 0 : Real) / u ^ 2 * (2 + 2 * (2 ^ 16 : Real) *
((lam 2 : Real) ^ 2 + lam 2) / (1 + (lam 1 : Real)) ^ 2) +
((lam 0 : Real) / u) ^ 2 * (2 ^ 16 : Real) *
((lam 2 : Real) / (1 + (lam 1 : Real)) ^ 2 +
(lam 2 : Real) ^ 2 * (lam 1 : Real) / (1 + (lam 1 : Real)) ^ 4) := by
obtain ⟨hz, hzm, _, hzv⟩ :=
inverse_count_poisson_count_moments mu (X 0) (lam 0) (hX 0) (hlaw 0)
have hwk : IndepFun (X 2) (X 1) mu := hind.indepFun (by decide)
obtain ⟨hb, hbs, hbv⟩ := inverse_count_weight_moments mu (X 1) (X 2)
(lam 1) (lam 2) (hX 1) (hX 2) (hlaw 1) (hlaw 2) hwk
have hip := (hind.indepFun_prodMk hX 2 1 0 (by decide) (by decide)).symm
have hi : IndepFun (fun om => (X 0 om : Real) / u)
(fun om => 1 + (X 2 om : Real) / ((X 1 om : Real) + 1)) mu :=
hip.comp (φ := fun k : Nat => (k : Real) / u)
(ψ := fun kw : Nat × Nat => 1 + (kw.1 : Real) / ((kw.2 : Real) + 1))
(by fun_prop) (by fun_prop)
have hzu : MemLp (fun om => (X 0 om : Real) / u) 2 mu := by
simpa only [div_eq_mul_inv] using hz.mul_const u⁻¹
refine ⟨inverse_count_independent_product_memLp hzu hb hi, ?_⟩
rw [inverse_count_independent_product_variance hzu hb hi,
integral_div, hzm]
have hv : variance (fun om => (X 0 om : Real) / u) mu = (lam 0 : Real) / u ^ 2 := by
simp only [div_eq_mul_inv, variance_mul_const, hzv, inv_pow]
rw [hv]
calc
_ ≤ (lam 0 : Real) / u ^ 2 * (2 + 2 * (2 ^ 16 : Real) *
((lam 2 : Real) ^ 2 + lam 2) / (1 + (lam 1 : Real)) ^ 2) +
((lam 0 : Real) / u) ^ 2 * ((2 ^ 16 : Real) *
((lam 2 : Real) / (1 + (lam 1 : Real)) ^ 2 +
(lam 2 : Real) ^ 2 * (lam 1 : Real) / (1 + (lam 1 : Real)) ^ 4)) :=
add_le_add (mul_le_mul_of_nonneg_left hbs (by positivity))
(mul_le_mul_of_nonneg_left hbv (sq_nonneg _))
_ = _ := by ring
theorem inverse_count_cell_blocks_independent Lemma inverse_count_cell_blocks_independent in the paper ↗
Under the stated inputs and conditions, Statistics computed in distinct three-count cells are independent. This gives the stated result.
Formal statement
hypotheses
Om :
Type
J :
Type
mu :
Measure Om
hX :
∀ i, Measurable (X i)
hind :
iIndepFun X mu
u :
j k :
J
hjk :
j ≠ k
Proof (Lean source)
-- @node: inverse_count_cell_blocks_independent
lemma inverse_count_cell_blocks_independent {Om : Type} [MeasurableSpace Om]
{J : Type} (mu : Measure Om)
(X : Fin 3 × J → Om → Nat) (hX : ∀ i, Measurable (X i))
(hind : iIndepFun X mu) (u : Real) (j k : J) (hjk : j ≠ k) :
IndepFun (fun om => (X (0, j) om : Real) / u *
(1 + (X (2, j) om : Real) / ((X (1, j) om : Real) + 1)))
(fun om => (X (0, k) om : Real) / u *
(1 + (X (2, k) om : Real) / ((X (1, k) om : Real) + 1))) mu := by
classical
let S : Finset (Fin 3 × J) := univ ×ˢ {j}
let T : Finset (Fin 3 × J) := univ ×ˢ {k}
have hd : Disjoint S T := by
apply Finset.disjoint_left.mpr
intro x hx hy
have hxj : x.2 = j := by simpa only [S, Finset.mem_product, Finset.mem_univ,
Finset.mem_singleton, true_and] using hx
have hxk : x.2 = k := by simpa only [T, Finset.mem_product, Finset.mem_univ,
Finset.mem_singleton, true_and] using hy
exact hjk (hxj.symm.trans hxk)
have hi := hind.indepFun_finset S T hd hX
let f : (S → Nat) → Real := fun v =>
(v ⟨(0, j), by simp [S]⟩ : Real) / u *
(1 + (v ⟨(2, j), by simp [S]⟩ : Real) /
((v ⟨(1, j), by simp [S]⟩ : Real) + 1))
let g : (T → Nat) → Real := fun v =>
(v ⟨(0, k), by simp [T]⟩ : Real) / u *
(1 + (v ⟨(2, k), by simp [T]⟩ : Real) /
((v ⟨(1, k), by simp [T]⟩ : Real) + 1))
exact hi.comp (φ := f) (ψ := g) (by dsimp [f]; fun_prop) (by dsimp [g]; fun_prop)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.InverseCountVarianceAlgebra 3 declarations Numerical cell envelopes for the inverse-count Poisson variance calculation.
Numerical cell envelopes for the inverse-count Poisson variance calculation.
Under the stated inputs and conditions, Overlap controls the squared opposite-arm mass divided by the current-arm mass. This gives the stated result.
Formal statement
hypotheses
conclusion
v ^ 2 / s ≤ (s + v) / eps
Proof (Lean source)
-- @node: inverse_count_overlap_ratio_bound
lemma inverse_count_overlap_ratio_bound (s v eps : Real)
(hs : 0 < s) (hv : 0 ≤ v) (heps : 0 < eps)
(hoverlap : eps * (s + v) ≤ s) :
v ^ 2 / s ≤ (s + v) / eps := by
apply (div_le_div_iff₀ hs heps).2
have hvs : eps * v ≤ s := by nlinarith
have hprod := mul_le_mul_of_nonneg_right hvs hv
nlinarith [sq_nonneg s]
Under the stated inputs and conditions, The four rational factors in the Poisson second-moment and variance bounds. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
s * (t * v) ^ 2 / (1 + t * s) ^ 2 ≤ v ^ 2 / s
conclusion 2
s * (t * v) / (1 + t * s) ^ 2 ≤ v
conclusion 3
s ^ 2 * (t * v) / (1 + t * s) ^ 2 ≤ v / t
conclusion 4
s ^ 2 * (t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4 ≤ v ^ 2 / (t * s)
Proof (Lean source)
-- @node: inverse_count_rational_factors
lemma inverse_count_rational_factors (s v t : Real)
(hs : 0 < s) (hv : 0 ≤ v) (ht : 0 < t) :
s * (t * v) ^ 2 / (1 + t * s) ^ 2 ≤ v ^ 2 / s ∧
s * (t * v) / (1 + t * s) ^ 2 ≤ v ∧
s ^ 2 * (t * v) / (1 + t * s) ^ 2 ≤ v / t ∧
s ^ 2 * (t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4 ≤ v ^ 2 / (t * s) := by
have hd : 0 < 1 + t * s := by positivity
have hr0 : 0 ≤ t * s / (1 + t * s) := by positivity
have hr1 : t * s / (1 + t * s) ≤ 1 := (div_le_one hd).2 (by linarith)
have hr2 : (t * s / (1 + t * s)) ^ 2 ≤ 1 := by
simpa using pow_le_pow_left₀ hr0 hr1 2
have hr4 : (t * s / (1 + t * s)) ^ 4 ≤ 1 := by
simpa using pow_le_pow_left₀ hr0 hr1 4
have hsquare : t * s ≤ (1 + t * s) ^ 2 := by nlinarith [sq_nonneg (t * s)]
refine ⟨?_, ?_, ?_, ?_⟩
· calc
_ = (v ^ 2 / s) * (t * s / (1 + t * s)) ^ 2 := by field_simp
_ ≤ (v ^ 2 / s) * 1 := mul_le_mul_of_nonneg_left hr2 (by positivity)
_ = _ := mul_one _
· apply (div_le_iff₀ (sq_pos_of_pos hd)).2
nlinarith [mul_le_mul_of_nonneg_right hsquare hv]
· calc
_ = (v / t) * (t * s / (1 + t * s)) ^ 2 := by field_simp
_ ≤ (v / t) * 1 := mul_le_mul_of_nonneg_left hr2 (by positivity)
_ = _ := mul_one _
· calc
_ = (v ^ 2 / (t * s)) * (t * s / (1 + t * s)) ^ 4 := by field_simp
_ ≤ (v ^ 2 / (t * s)) * 1 := mul_le_mul_of_nonneg_left hr4 (by positivity)
_ = _ := mul_one _
theorem inverse_count_cell_variance_envelope Lemma inverse_count_cell_variance_envelope in the paper ↗
Under the stated inputs and conditions, The cell second-moment and variance envelopes are uniform even when the arm mass is zero. This gives the stated result.
Formal statement
hypotheses
s v q u t eps C0 :
hs :
0 ≤ s
hv :
0 ≤ v
hq :
0 ≤ q
hqs :
q ≤ s
hu :
0 < u
ht :
0 < t
heps :
0 < eps
hC :
0 ≤ C0
hoverlap :
eps * (s + v) ≤ s
conclusion
q / u * (2 + 2 * C0 * ((t * v) ^ 2 + t * v) / (1 + t * s) ^ 2)
+ q ^ 2 * C0 * (t * v / (1 + t * s) ^ 2 + (t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4)
≤ (2 + 4 * C0) * (s + v) * (1 / (u * eps) + 1 / (t * eps))
Proof (Lean source)
-- @node: inverse_count_cell_variance_envelope
lemma inverse_count_cell_variance_envelope (s v q u t eps C0 : Real)
(hs : 0 ≤ s) (hv : 0 ≤ v) (hq : 0 ≤ q) (hqs : q ≤ s)
(hu : 0 < u) (ht : 0 < t) (heps : 0 < eps)
(hC : 0 ≤ C0) (hoverlap : eps * (s + v) ≤ s) :
q / u * (2 + 2 * C0 * ((t * v) ^ 2 + t * v) / (1 + t * s) ^ 2) +
q ^ 2 * C0 * (t * v / (1 + t * s) ^ 2 +
(t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4) ≤
(2 + 4 * C0) * (s + v) * (1 / (u * eps) + 1 / (t * eps)) := by
by_cases hz : s = 0
· have hq0 : q = 0 := by linarith
have hv0 : v = 0 := by rw [hz] at hoverlap; nlinarith
simp [hz, hq0, hv0]
have hs0 : 0 < s := lt_of_le_of_ne hs (Ne.symm hz)
have hp : 0 ≤ s + v := by positivity
have hd : 0 < 1 + t * s := by positivity
obtain ⟨h1, h2, h3, h4⟩ := inverse_count_rational_factors s v t hs0 hv ht
have hr := inverse_count_overlap_ratio_bound s v eps hs0 hv heps hoverlap
have hpE : s + v ≤ (s + v) / eps := (le_div_iff₀ heps).2 (by nlinarith)
have hvE : v ≤ (s + v) / eps := (by linarith : v ≤ s + v).trans hpE
have hqE : q ≤ (s + v) / eps := hqs.trans ((by linarith : s ≤ s + v).trans hpE)
have hsecond : q * (((t * v) ^ 2 + t * v) / (1 + t * s) ^ 2) ≤
2 * ((s + v) / eps) := by
calc
_ ≤ s * (((t * v) ^ 2 + t * v) / (1 + t * s) ^ 2) :=
mul_le_mul_of_nonneg_right hqs (by positivity)
_ = s * (t * v) ^ 2 / (1 + t * s) ^ 2 +
s * (t * v) / (1 + t * s) ^ 2 := by ring
_ ≤ v ^ 2 / s + v := add_le_add h1 h2
_ ≤ _ := by linarith
have hvar : q ^ 2 * (t * v / (1 + t * s) ^ 2 +
(t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4) ≤
2 * ((s + v) / (t * eps)) := by
have hsq : q ^ 2 ≤ s ^ 2 := pow_le_pow_left₀ hq hqs 2
have hfirst : v / t ≤ (s + v) / (t * eps) := by
simpa only [div_div, mul_comm eps t] using div_le_div_of_nonneg_right hvE ht.le
have hlast : v ^ 2 / (t * s) ≤ (s + v) / (t * eps) := by
simpa only [div_div, mul_comm s t, mul_comm eps t] using
div_le_div_of_nonneg_right hr ht.le
calc
_ ≤ s ^ 2 * (t * v / (1 + t * s) ^ 2 +
(t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4) :=
mul_le_mul_of_nonneg_right hsq (by positivity)
_ = s ^ 2 * (t * v) / (1 + t * s) ^ 2 +
s ^ 2 * (t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4 := by ring
_ ≤ v / t + v ^ 2 / (t * s) := add_le_add h3 h4
_ ≤ _ := by linarith
have hfirst := mul_le_mul_of_nonneg_left hqE (show 0 ≤ 2 / u by positivity)
have hsecond' := mul_le_mul_of_nonneg_left hsecond (show 0 ≤ 2 * C0 / u by positivity)
have hvar' := mul_le_mul_of_nonneg_left hvar hC
have hslack : 0 ≤ (2 + 2 * C0) * ((s + v) / (t * eps)) := by positivity
have hid :
(2 + 4 * C0) * (s + v) * (1 / (u * eps) + 1 / (t * eps)) =
2 / u * ((s + v) / eps) + 2 * C0 / u * (2 * ((s + v) / eps)) +
C0 * (2 * ((s + v) / (t * eps))) +
(2 + 2 * C0) * ((s + v) / (t * eps)) := by ring
rw [hid]
calc
_ = (2 / u * q + 2 * C0 / u *
(q * (((t * v) ^ 2 + t * v) / (1 + t * s) ^ 2))) +
C0 * (q ^ 2 * (t * v / (1 + t * s) ^ 2 +
(t * v) ^ 2 * (t * s) / (1 + t * s) ^ 4)) := by ring
_ ≤ (2 / u * ((s + v) / eps) +
2 * C0 / u * (2 * ((s + v) / eps))) +
C0 * (2 * ((s + v) / (t * eps))) :=
add_le_add (add_le_add hfirst hsecond') hvar'
_ ≤ _ := by linarith only [hslack]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.KnownMarginalUpper 12 declarations A total supplied-marginal inverse-propensity rule and its finite-sample risk bound.
A total supplied-marginal inverse-propensity rule and its finite-sample risk bound.
The binary inverse-propensity score reads both cell masses from the supplied table. Division is total even for illegal tables and null cells.
theorem knownMarginalScore_measurable
The score is Borel jointly in its real table and finite record. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: knownMarginalScore_measurable
lemma knownMarginalScore_measurable (d : Nat) :
Measurable (fun z : AuxTable d × Obs d => knownMarginalScore z.1 z.2) := by
apply measurable_from_prod_countable_left
intro z
dsimp only [knownMarginalScore]
split_ifs <;> fun_prop
Clipping makes the supplied-table mean a total bounded rule on every table.
theorem knownMarginalEstimator_measurable
The supplied-table rule is jointly Borel. This gives the stated conclusion.
Formal statement
Proof (Lean source)
@[fun_prop]
-- @node: knownMarginalEstimator_measurable
lemma knownMarginalEstimator_measurable (n d : Nat) (eps : Real) :
Measurable (knownMarginalEstimator n d eps) := by
unfold knownMarginalEstimator
split_ifs
· fun_prop
· have hscore (i : Fin n) : Measurable (fun z : (Fin n → Obs d) × AuxTable d × Real =>
knownMarginalScore z.2.1 (z.1 i)) :=
(knownMarginalScore_measurable d).comp
(show Measurable (fun z : (Fin n → Obs d) × AuxTable d × Real =>
(z.2.1, z.1 i)) by fun_prop)
fun_prop
theorem knownMarginalEstimator_mem_Icc
Under the stated inputs and conditions, The total supplied-table rule is clipped to the prescribed action interval. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: knownMarginalEstimator_mem_Icc
lemma knownMarginalEstimator_mem_Icc (n d : Nat) (eps : Real)
(z : (Fin n → Obs d) × AuxTable d × Real) :
knownMarginalEstimator n d eps z ∈ Icc (-1) 1 := by
unfold knownMarginalEstimator
split
· norm_num
· exact ⟨le_max_left _ _, max_le (by norm_num) (min_le_left _ _)⟩
theorem knownMarginalScore_mean
Under the stated inputs and conditions, At the actual marginal table the score has exactly the observable ATE mean. This gives the stated result.
Formal statement
hypotheses
d :
P :
conclusion
Proof (Lean source)
-- @node: knownMarginalScore_mean
lemma knownMarginalScore_mean {d : Nat} (P : DiscreteLaw d) :
(∫ z, knownMarginalScore (auxTable P) z ∂obsLaw P) = ateFunctional P := by
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
rw [obsLaw, PMF.integral_eq_sum]
simp only [smul_eq_mul]
change (∑ z : Obs d, jointMass P z.1 z.2.1 z.2.2 *
knownMarginalScore (auxTable P) z) = _
simp only [Fintype.sum_prod_type, Fintype.sum_bool, knownMarginalScore,
auxTable, auxMarginal_toReal_armMass, Bool.false_eq_true, if_false, if_true]
unfold ateFunctional outcomeMean markedMass
apply Finset.sum_congr rfl
intro j _
have hp : armMass P j false + armMass P j true = cellMass P j := by
simp [cellMass, armMass, add_comm]
rw [hp]
ring
theorem knownMarginalScore_arm_second_moment
Under the stated inputs and conditions, Overlap bounds each arm's marked contribution to the score's second moment. This gives the stated result.
Formal statement
hypotheses
conclusion
markedMass P j a * (cellMass P j / armMass P j a) ^ 2 ≤ cellMass P j / eps
Proof (Lean source)
-- @node: knownMarginalScore_arm_second_moment
lemma knownMarginalScore_arm_second_moment {d : Nat} (P : DiscreteLaw d)
(eps : Real) (heps : 0 < eps) (hP : ModelClass d eps P) (j : Fin d) (a : Bool) :
markedMass P j a * (cellMass P j / armMass P j a) ^ 2 ≤ cellMass P j / eps := by
have hp := cellMass_nonneg P j
have hs := armMass_nonneg P j a
have hqs : markedMass P j a ≤ armMass P j a := by
simp only [armMass, markedMass, Fintype.sum_bool]
linarith [jointMass_nonneg P j a false]
have hover := armMass_ge_overlap_cellMass P eps hP j a
by_cases hz : armMass P j a = 0
· simp only [hz, div_zero, zero_pow (by decide : (2 : Nat) ≠ 0), mul_zero]
positivity
· have hspos : 0 < armMass P j a := lt_of_le_of_ne hs (Ne.symm hz)
calc
_ ≤ armMass P j a * (cellMass P j / armMass P j a) ^ 2 :=
mul_le_mul_of_nonneg_right hqs (sq_nonneg _)
_ = cellMass P j * (cellMass P j / armMass P j a) := by field_simp
_ ≤ cellMass P j * (1 / eps) := by
apply mul_le_mul_of_nonneg_left _ hp
apply (div_le_iff₀ hspos).2
have hh := (le_div_iff₀ heps).2 (by simpa only [mul_comm] using hover)
convert hh using 1 <;> ring
_ = _ := by ring
theorem knownMarginalScore_second_moment
Under the stated inputs and conditions, The score's second moment is at most twice the inverse overlap floor. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ z, knownMarginalScore (auxTable P) z ^ 2 ∂obsLaw P) ≤ 2 / eps
Proof (Lean source)
-- @node: knownMarginalScore_second_moment
lemma knownMarginalScore_second_moment {d : Nat} (P : DiscreteLaw d)
(eps : Real) (heps : 0 < eps) (hP : ModelClass d eps P) :
(∫ z, knownMarginalScore (auxTable P) z ^ 2 ∂obsLaw P) ≤ 2 / eps := by
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
rw [obsLaw, PMF.integral_eq_sum]
simp only [smul_eq_mul]
change (∑ z : Obs d, jointMass P z.1 z.2.1 z.2.2 *
knownMarginalScore (auxTable P) z ^ 2) ≤ _
simp only [Fintype.sum_prod_type, Fintype.sum_bool, knownMarginalScore,
auxTable, auxMarginal_toReal_armMass, Bool.false_eq_true, if_false, if_true]
have hcell (j : Fin d) : armMass P j false + armMass P j true = cellMass P j := by
simp [cellMass, armMass, add_comm]
simp_rw [hcell]
calc
_ = ∑ j : Fin d, (markedMass P j false * (cellMass P j / armMass P j false) ^ 2 +
markedMass P j true * (cellMass P j / armMass P j true) ^ 2) := by
apply Finset.sum_congr rfl
intro j _
simp [markedMass, neg_div, neg_sq, add_comm]
_ ≤ ∑ j : Fin d, (cellMass P j / eps + cellMass P j / eps) :=
Finset.sum_le_sum (fun j _ => add_le_add
(knownMarginalScore_arm_second_moment P eps heps hP j false)
(knownMarginalScore_arm_second_moment P eps heps hP j true))
_ = 2 / eps := by
rw [Finset.sum_add_distrib, ← Finset.sum_div, sum_cellMass]
ring
theorem knownMarginalMean_risk
Under the stated inputs and conditions, Independence of complete records divides the score variance by the labeled budget. This gives the stated result.
Formal statement
hypotheses
conclusion
(∫ x, ((∑ i : Fin n, knownMarginalScore (auxTable P) (x i)) / n - ateFunctional P) ^ 2 ∂labeledProductLaw P n)
≤ 2 / ((n : Real) * eps)
Proof (Lean source)
-- @node: knownMarginalMean_risk
lemma knownMarginalMean_risk {d : Nat} (P : DiscreteLaw d) (n : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps) (hP : ModelClass d eps P) :
(∫ x, ((∑ i : Fin n, knownMarginalScore (auxTable P) (x i)) / n -
ateFunctional P) ^ 2 ∂labeledProductLaw P n) ≤ 2 / ((n : Real) * eps) := by
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
let : IsProbabilityMeasure (labeledProductLaw P n) := by unfold labeledProductLaw; infer_instance
let F := knownMarginalScore (auxTable P)
let M : (Fin n → Obs d) → Real := fun x => (∑ i : Fin n, F (x i)) / n
have hn0 : (n : Real) ≠ 0 := by exact_mod_cast (show n ≠ 0 by omega)
have hF : MemLp F 2 (obsLaw P) :=
(memLp_two_iff_integrable_sq (by fun_prop)).2 Integrable.of_finite
have hM : Measurable M := by fun_prop
have hmean : (∫ x, M x ∂labeledProductLaw P n) = ateFunctional P := by
dsimp [M, labeledProductLaw]
rw [integral_div, integral_finsetSum]
· have heval (i : Fin n) : (∫ x : Fin n → Obs d, F (x i)
∂Measure.pi (fun _ : Fin n => obsLaw P)) = ∫ z, F z ∂obsLaw P :=
integral_comp_eval (μ := fun _ : Fin n => obsLaw P) (i := i) hF.aestronglyMeasurable
simp_rw [heval]
simp [F, knownMarginalScore_mean, hn0]
· intro i _
exact Integrable.of_finite
have hvar : Var[M; labeledProductLaw P n] = Var[F; obsLaw P] / n := by
dsimp [M, labeledProductLaw]
simp only [div_eq_mul_inv]
rw [variance_mul_const]
have hsum : (fun x : Fin n → Obs d => ∑ i : Fin n, F (x i)) =
∑ i : Fin n, (fun x : Fin n → Obs d => F (x i)) := by
funext x
simp
rw [hsum, variance_sum_pi (fun _ : Fin n => hF)]
simp only [Finset.sum_const, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul]
field_simp
have hv := (variance_le_expectation_sq hF.aestronglyMeasurable).trans
(knownMarginalScore_second_moment P eps heps hP)
calc
_ = Var[M; labeledProductLaw P n] := by
rw [variance_eq_integral hM.aemeasurable, hmean]
_ = Var[F; obsLaw P] / n := hvar
_ ≤ (2 / eps) / n := div_le_div_of_nonneg_right hv (Nat.cast_nonneg n)
_ = _ := by ring
Under the stated inputs and conditions, Projection onto the action interval cannot increase squared distance to a legal target. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: knownMarginal_clip_loss
lemma knownMarginal_clip_loss (x tau : Real) (htau : tau ∈ Icc (-1) 1) :
(max (-1) (min 1 x) - tau) ^ 2 ≤ (x - tau) ^ 2 := by
by_cases hlo : x < -1
· rw [min_eq_right (by linarith), max_eq_left (by linarith)]
nlinarith [htau.1]
· by_cases hhi : 1 < x
· rw [min_eq_left hhi.le, max_eq_right (by norm_num)]
nlinarith [htau.2]
· rw [min_eq_right (by linarith), max_eq_right (by linarith)]
theorem knownMarginalEstimator_risk
Under the stated inputs and conditions, The zero branch and the clipped iid mean attain the capped oracle label rate. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: knownMarginalEstimator_risk
lemma knownMarginalEstimator_risk {d : Nat} (P : DiscreteLaw d) (n : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps) (hP : ModelClass d eps P) :
knownRuleRisk (knownMarginalEstimator n d eps) P ≤ 2 * labelBenchmark n eps := by
let : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
let : IsProbabilityMeasure (labeledProductLaw P n) := by unfold labeledProductLaw; infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have hnpos : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hS := mul_pos hnpos heps
have htau := ateFunctional_mem_Icc P
by_cases hsmall : (n : Real) * eps < 1
· have hbench : labelBenchmark n eps = 1 := by
exact min_eq_left ((one_le_inv₀ hS).2 hsmall.le)
have hrisk : knownRuleRisk (knownMarginalEstimator n d eps) P = ateFunctional P ^ 2 := by
simp [knownRuleRisk, knownMarginalEstimator, hsmall]
rw [hrisk, hbench, mul_one]
nlinarith [htau.1, htau.2]
· have hbench : labelBenchmark n eps = 1 / ((n : Real) * eps) := by
simpa only [labelBenchmark, labelScale, one_div] using
min_eq_right (inv_le_one_of_one_le₀ (le_of_not_gt hsmall))
rw [hbench]
have hint : Integrable (fun x : Fin n → Obs d =>
((∑ i : Fin n, knownMarginalScore (auxTable P) (x i)) / n -
ateFunctional P) ^ 2) (labeledProductLaw P n) := Integrable.of_finite
have hprod := hint.comp_fst seedLaw
unfold knownRuleRisk
calc
_ ≤ ∫ z : (Fin n → Obs d) × Real,
((∑ i : Fin n, knownMarginalScore (auxTable P) (z.1 i)) / n -
ateFunctional P) ^ 2 ∂((labeledProductLaw P n).prod seedLaw) := by
apply integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
hprod (Filter.Eventually.of_forall (fun z => ?_))
simp only [knownMarginalEstimator, if_neg hsmall]
exact knownMarginal_clip_loss _ _ htau
_ = ∫ x, ((∑ i : Fin n, knownMarginalScore (auxTable P) (x i)) / n -
ateFunctional P) ^ 2 ∂labeledProductLaw P n := by
rw [integral_prod]
· simp
· exact hprod
_ ≤ 2 / ((n : Real) * eps) := knownMarginalMean_risk P n eps hn heps hP
_ = _ := by ring
theorem knownMarginalRisk_le_labelBenchmark
Under the stated inputs and conditions, Taking the infimum over Borel supplied-table rules preserves the oracle upper bound. This gives the stated result.
Formal statement
hypotheses
conclusion
knownMarginalRisk n d eps ≤ 2 * labelBenchmark n eps
Proof (Lean source)
-- @node: knownMarginalRisk_le_labelBenchmark
lemma knownMarginalRisk_le_labelBenchmark (n d : Nat) (eps : Real)
(hn : 1 ≤ n) (hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
knownMarginalRisk n d eps ≤ 2 * labelBenchmark n eps := by
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps'⟩
letI : Nonempty (ClassLaw d eps) := ⟨P0⟩
let T : KnownRule n d := ⟨knownMarginalEstimator n d eps,
knownMarginalEstimator_measurable n d eps, knownMarginalEstimator_mem_Icc n d eps⟩
apply (minimaxValue_le_worstCaseRisk_of_nonneg
(risk := fun (T : KnownRule n d) (P : ClassLaw d eps) => knownRuleRisk T.1 P.1)
(fun T P => integral_nonneg (fun z => sq_nonneg _)) T).trans
exact ciSup_le (fun P => knownMarginalEstimator_risk P.1 n eps hn heps P.2)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.LabelFloorPair 18 declarations Explicit diagnostic pair sharing its treatment-covariate marginal.
Explicit diagnostic pair sharing its treatment-covariate marginal.
The arbitrary real diagnostic amplitude is positive and at most one eighth. @realizes \delta(arbitrary Real amplitude in (0,1/8])
Definition (Lean source)
parameters
delta :
def
LabelFloorAmplitude delta :
Prop
given by
clause 1
0 < delta
clause 2
delta ≤ 1 / 8
The displayed two-cell law, with total fallback outside legal scalar ranges.
Definition (Lean source)
parameters
def
given by
mass :
fun z
=> if z.1.val < 2 then (1 / 2) * bernoulliMass eps z.2.1 * bernoulliMass (if z.2.1 then 1 / 2 + (if sign then delta else -delta) else 1 / 2) z.2.2 else 0 (: Obs d → Real)
theorem labelFloorAmplitude_eighth
The fixed endpoint one eighth is a legal diagnostic amplitude.
Formal statement
conclusion
LabelFloorAmplitude (1 / 8)
Proof (Lean source)
lemma labelFloorAmplitude_eighth : LabelFloorAmplitude (1 / 8) := by
norm_num [LabelFloorAmplitude]
theorem labelFloorAmplitude_scaled
Under the stated inputs and conditions, The displayed scale choice is a legal diagnostic amplitude for every scale.
Formal statement
Proof (Lean source)
lemma labelFloorAmplitude_scaled (S : Real) :
LabelFloorAmplitude (1 / (16 * sqrt (max 1 S))) := by
have hmax : (1 : Real) ≤ max 1 S := le_max_left _ _
have hsqrt : (1 : Real) ≤ sqrt (max 1 S) := by
have h := Real.sqrt_le_sqrt hmax
simpa only [Real.sqrt_one] using h
constructor
· positivity
· apply (div_le_iff₀ (by positivity : 0 < 16 * sqrt (max 1 S))).mpr
nlinarith
Under the stated inputs and conditions, Summing a constant over the two occupied covariates gives twice that constant. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: labelFloor_sum_two_cells
lemma labelFloor_sum_two_cells (d : Nat) (hd : 2 ≤ d) (r : Real) :
(∑ j : Fin d, if j.val < 2 then r else 0) = 2 * r := by
let j0 : Fin d := ⟨0, by omega⟩
let j1 : Fin d := ⟨1, by omega⟩
have hsplit (j : Fin d) :
(if j.val < 2 then r else 0) =
(if j = j0 then r else 0) + (if j = j1 then r else 0) := by
by_cases h0 : j = j0
· subst j
simp [j0, j1]
· by_cases h1 : j = j1
· subst j
simp [j0, j1]
· have hv0 : j.val ≠ 0 := fun h => h0 (Fin.ext h)
have hv1 : j.val ≠ 1 := fun h => h1 (Fin.ext h)
have hv : ¬j.val < 2 := by omega
simp [hv, h0, h1]
simp_rw [hsplit]
simp [Finset.sum_add_distrib, two_mul]
theorem labelFloor_mass_nonneg
Under the stated inputs and conditions, The diagnostic table has nonnegative entries throughout the legal parameter range. This gives the stated result.
Formal statement
hypotheses
conclusion
0
≤ (1 / 2 : Real) * bernoulliMass eps a * bernoulliMass (if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2) y
Proof (Lean source)
-- @node: labelFloor_mass_nonneg
lemma labelFloor_mass_nonneg (eps delta : Real) (sign a y : Bool)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) (hdelta : LabelFloorAmplitude delta) :
0 ≤ (1 / 2 : Real) * bernoulliMass eps a *
bernoulliMass (if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2) y := by
rcases hdelta with ⟨hd, hd'⟩
apply mul_nonneg
· apply mul_nonneg (by norm_num)
cases a <;> dsimp [bernoulliMass] <;> linarith
· cases sign <;> cases a <;> cases y <;> dsimp [bernoulliMass] <;> linarith
Under the stated inputs and conditions, The displayed diagnostic table already has total mass one. This gives the stated result.
Formal statement
hypotheses
conclusion
(∑ z : Obs d, if z.1.val < 2 then (1 / 2 : Real) * bernoulliMass eps z.2.1 * bernoulliMass (if z.2.1 then 1 / 2 + (if sign then delta else -delta) else 1 / 2) z.2.2 else 0)
= 1
Proof (Lean source)
-- @node: labelFloor_mass_sum
lemma labelFloor_mass_sum (d : Nat) (hd : 2 ≤ d) (eps delta : Real) (sign : Bool) :
(∑ z : Obs d, if z.1.val < 2 then (1 / 2 : Real) * bernoulliMass eps z.2.1 *
bernoulliMass (if z.2.1 then 1 / 2 + (if sign then delta else -delta) else 1 / 2) z.2.2
else 0) = 1 := by
simp only [Fintype.sum_prod_type]
have hcell (j : Fin d) :
(∑ a : Bool, ∑ y : Bool, if j.val < 2 then (1 / 2 : Real) * bernoulliMass eps a *
bernoulliMass (if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2) y
else 0) = if j.val < 2 then 1 / 2 else 0 := by
by_cases hj : j.val < 2
· simp [hj, bernoulliMass, Fintype.sum_bool]
ring
· simp [hj]
simp_rw [hcell]
rw [labelFloor_sum_two_cells d hd]
norm_num
theorem labelFloor_jointMass
Under the stated inputs and conditions, Reading back a diagnostic atom gives exactly its specified table entry. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
j :
Fin d
a y :
conclusion
jointMass (labelFloorLaw d eps delta sign hd hdelta) j a y
= if j.val < 2 then (1 / 2 : Real) * bernoulliMass eps a * bernoulliMass (if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2) y else 0
Proof (Lean source)
-- @node: labelFloor_jointMass
lemma labelFloor_jointMass (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(j : Fin d) (a y : Bool) :
jointMass (labelFloorLaw d eps delta sign hd hdelta) j a y =
if j.val < 2 then (1 / 2 : Real) * bernoulliMass eps a *
bernoulliMass (if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2) y
else 0 := by
apply normalizedPMF_toReal
· intro z
by_cases hz : z.1.val < 2
· simp only [if_pos hz]
exact labelFloor_mass_nonneg eps delta sign _ _ heps heps' hdelta
· simp only [if_neg hz, le_refl]
· exact labelFloor_mass_sum d hd eps delta sign
theorem labelFloor_armMass
Under the stated inputs and conditions, Summing out the outcome leaves a diagnostic arm mass independent of the sign. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
j :
Fin d
a :
conclusion
armMass (labelFloorLaw d eps delta sign hd hdelta) j a
= if j.val < 2 then (1 / 2 : Real) * bernoulliMass eps a else 0
Proof (Lean source)
-- @node: labelFloor_armMass
lemma labelFloor_armMass (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(j : Fin d) (a : Bool) :
armMass (labelFloorLaw d eps delta sign hd hdelta) j a =
if j.val < 2 then (1 / 2 : Real) * bernoulliMass eps a else 0 := by
simp only [armMass, labelFloor_jointMass d eps delta sign hd hdelta heps heps',
Fintype.sum_bool]
by_cases hj : j.val < 2
· cases a <;> simp [hj, bernoulliMass] <;> ring
· simp [hj]
theorem labelFloor_cellMass
Under the stated inputs and conditions, Each occupied diagnostic covariate has mass one half, and all other cells are null. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
j :
Fin d
conclusion
cellMass (labelFloorLaw d eps delta sign hd hdelta) j
= if j.val < 2 then (1 / 2 : Real) else 0
Proof (Lean source)
-- @node: labelFloor_cellMass
lemma labelFloor_cellMass (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(j : Fin d) :
cellMass (labelFloorLaw d eps delta sign hd hdelta) j =
if j.val < 2 then (1 / 2 : Real) else 0 := by
change (∑ a : Bool, armMass _ j a) = _
simp only [labelFloor_armMass d eps delta sign hd hdelta heps heps', Fintype.sum_bool]
by_cases hj : j.val < 2
· simp [hj, bernoulliMass]
ring
· simp [hj]
theorem labelFloor_propensity
Under the stated inputs and conditions, The occupied diagnostic propensities equal the public overlap floor. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
j :
Fin d
hj :
j.val < 2
conclusion
propensity (labelFloorLaw d eps delta sign hd hdelta) j = eps
Proof (Lean source)
-- @node: labelFloor_propensity
lemma labelFloor_propensity (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(j : Fin d) (hj : j.val < 2) :
propensity (labelFloorLaw d eps delta sign hd hdelta) j = eps := by
simp [propensity, labelFloor_cellMass d eps delta sign hd hdelta heps heps',
labelFloor_armMass d eps delta sign hd hdelta heps heps', hj, bernoulliMass]
ring
theorem labelFloor_model
Under the stated inputs and conditions, Both signs of the diagnostic pair belong to the public model class. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
conclusion
ModelClass d eps (labelFloorLaw d eps delta sign hd hdelta)
Proof (Lean source)
-- @node: labelFloor_model
lemma labelFloor_model (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
ModelClass d eps (labelFloorLaw d eps delta sign hd hdelta) := by
constructor
intro j hp
have hj : j.val < 2 := by
by_contra h
rw [labelFloor_cellMass d eps delta sign hd hdelta heps heps', if_neg h] at hp
exact (lt_irrefl 0) hp
rw [labelFloor_propensity d eps delta sign hd hdelta heps heps' j hj]
exact ⟨le_rfl, by linarith⟩
theorem labelFloor_outcomeMean
Under the stated inputs and conditions, Diagnostic conditional means match the specified control and treated Bernoulli means. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
j :
Fin d
hj :
j.val < 2
a :
conclusion
outcomeMean (labelFloorLaw d eps delta sign hd hdelta) a j
= if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2
Proof (Lean source)
-- @node: labelFloor_outcomeMean
lemma labelFloor_outcomeMean (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(j : Fin d) (hj : j.val < 2) (a : Bool) :
outcomeMean (labelFloorLaw d eps delta sign hd hdelta) a j =
if a then 1 / 2 + (if sign then delta else -delta) else 1 / 2 := by
simp only [outcomeMean, markedMass,
labelFloor_jointMass d eps delta sign hd hdelta heps heps',
labelFloor_armMass d eps delta sign hd hdelta heps heps', if_pos hj,
bernoulliMass, ite_true]
have heps0 : eps ≠ 0 := heps.ne'
have heps1 : 1 - eps ≠ 0 := by linarith
cases a <;> simp only [Bool.false_eq_true, Bool.true_eq, if_false, if_true] <;>
field_simp <;> ring
theorem labelFloor_target
Under the stated inputs and conditions, Summing the two occupied-cell contrasts yields the signed diagnostic amplitude. This gives the stated result.
Formal statement
hypotheses
d :
eps delta :
sign :
hd :
2 ≤ d
hdelta :
LabelFloorAmplitude delta
heps :
0 < eps
heps' :
eps ≤ 1 / 4
conclusion
ateFunctional (labelFloorLaw d eps delta sign hd hdelta) = if sign then delta else -delta
Proof (Lean source)
-- @node: labelFloor_target
lemma labelFloor_target (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
ateFunctional (labelFloorLaw d eps delta sign hd hdelta) =
if sign then delta else -delta := by
unfold ateFunctional
have hterm (j : Fin d) : cellMass (labelFloorLaw d eps delta sign hd hdelta) j *
(outcomeMean (labelFloorLaw d eps delta sign hd hdelta) true j -
outcomeMean (labelFloorLaw d eps delta sign hd hdelta) false j) =
if j.val < 2 then (1 / 2 : Real) * (if sign then delta else -delta) else 0 := by
by_cases hj : j.val < 2
· rw [labelFloor_cellMass d eps delta sign hd hdelta heps heps', if_pos hj,
labelFloor_outcomeMean d eps delta sign hd hdelta heps heps' j hj true,
labelFloor_outcomeMean d eps delta sign hd hdelta heps heps' j hj false, if_pos hj]
simp
· rw [labelFloor_cellMass d eps delta sign hd hdelta heps heps', if_neg hj]
simp [hj]
simp_rw [hterm]
rw [labelFloor_sum_two_cells d hd]
ring
theorem auxMarginal_toReal_armMass
Under the stated inputs and conditions, The real mass of a projected treatment-covariate atom is its arm-cell mass. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: auxMarginal_toReal_armMass
lemma auxMarginal_toReal_armMass {d : Nat} (P : DiscreteLaw d) (j : Fin d) (a : Bool) :
(auxMarginal P (j, a)).toReal = armMass P j a := by
have hm : auxMarginal P (j, a) = ∑ y : Bool, P.pmf (j, a, y) := by
rw [auxMarginal, PMF.map_apply, tsum_fintype]
cases a <;> simp [Fintype.sum_prod_type, Prod.mk.injEq, Finset.sum_add_distrib]
rw [hm, ENNReal.toReal_sum (fun y _ => P.pmf.apply_ne_top _)]
rfl
theorem labelFloor_auxMarginal
Under the stated inputs and conditions, The positive and negative diagnostic laws have exactly the same auxiliary PMF. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: labelFloor_auxMarginal
lemma labelFloor_auxMarginal (d : Nat) (eps delta : Real) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
auxMarginal (labelFloorLaw d eps delta true hd hdelta) =
auxMarginal (labelFloorLaw d eps delta false hd hdelta) := by
apply PMF.ext
intro z
apply (ENNReal.toReal_eq_toReal_iff'
((auxMarginal _).apply_ne_top z) ((auxMarginal _).apply_ne_top z)).mp
rcases z with ⟨j, a⟩
rw [auxMarginal_toReal_armMass, auxMarginal_toReal_armMass,
labelFloor_armMass d eps delta true hd hdelta heps heps',
labelFloor_armMass d eps delta false hd hdelta heps heps']
Finite natural-log divergence sums population mass times the log probability ratio.
theorem labelFloor_KL_eighth
Under the stated inputs and conditions, At amplitude one eighth the diagnostic one-record KL has its exact stated value. This gives the stated result.
Formal statement
hypotheses
conclusion
finiteKL (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth) (labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth)
= eps * log (5 / 3) / 4
Proof (Lean source)
-- @node: labelFloor_KL_eighth
lemma labelFloor_KL_eighth (d : Nat) (eps : Real) (hd : 2 ≤ d)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
finiteKL (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth)
(labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth) =
eps * log (5 / 3) / 4 := by
have heps0 : eps ≠ 0 := heps.ne'
have heps1 : 1 - eps ≠ 0 := by linarith
have hlog : log (3 / 5 : Real) = -log (5 / 3 : Real) := by
rw [show (3 / 5 : Real) = (5 / 3)⁻¹ by norm_num, Real.log_inv]
unfold finiteKL
change (∑ z : Obs d,
jointMass (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth)
z.1 z.2.1 z.2.2 * log
(jointMass (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth)
z.1 z.2.1 z.2.2 /
jointMass (labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth)
z.1 z.2.1 z.2.2)) = _
simp only [Fintype.sum_prod_type]
have hcell (j : Fin d) :
(∑ a : Bool, ∑ y : Bool,
jointMass (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth) j a y *
log (jointMass (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth)
j a y / jointMass (labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth)
j a y)) =
if j.val < 2 then eps * log (5 / 3) / 8 else 0 := by
simp only [labelFloor_jointMass d eps (1 / 8) true hd labelFloorAmplitude_eighth heps heps',
labelFloor_jointMass d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps',
Fintype.sum_bool]
by_cases hj : j.val < 2
· simp [hj, bernoulliMass]
rw [mul_div_mul_left _ _ (mul_ne_zero (by norm_num : (2 : Real)⁻¹ ≠ 0) heps0),
mul_div_mul_left _ _ (mul_ne_zero (by norm_num : (2 : Real)⁻¹ ≠ 0) heps0)]
norm_num
rw [hlog]
ring
· simp [hj]
simp_rw [hcell]
rw [labelFloor_sum_two_cells d hd]
ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.LabelFloorTesting 10 declarations Scaled diagnostic KL and testing bounds for the rare-label pair, including common randomness.
Scaled diagnostic KL and testing bounds for the rare-label pair, including common randomness.
theorem labelFloor_KL_formula
Under the stated inputs and conditions, The diagnostic pair has the stated KL formula at every legal amplitude. This gives the stated result.
Formal statement
hypotheses
conclusion
= 2 * eps * delta * log ((1 / 2 + delta) / (1 / 2 - delta))
Proof (Lean source)
-- @node: labelFloor_KL_formula
lemma labelFloor_KL_formula (d : Nat) (eps delta : Real) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
finiteKL (labelFloorLaw d eps delta true hd hdelta)
(labelFloorLaw d eps delta false hd hdelta) =
2 * eps * delta * log ((1 / 2 + delta) / (1 / 2 - delta)) := by
have heps0 : eps ≠ 0 := heps.ne'
have heps1 : 1 - eps ≠ 0 := by linarith
have hlog : log ((1 / 2 - delta) / (1 / 2 + delta)) =
-log ((1 / 2 + delta) / (1 / 2 - delta)) := by
rw [← inv_div, Real.log_inv]
unfold finiteKL
change (∑ z : Obs d,
jointMass (labelFloorLaw d eps delta true hd hdelta) z.1 z.2.1 z.2.2 *
log (jointMass (labelFloorLaw d eps delta true hd hdelta) z.1 z.2.1 z.2.2 /
jointMass (labelFloorLaw d eps delta false hd hdelta) z.1 z.2.1 z.2.2)) = _
simp only [Fintype.sum_prod_type]
have hcell (j : Fin d) :
(∑ a : Bool, ∑ y : Bool,
jointMass (labelFloorLaw d eps delta true hd hdelta) j a y *
log (jointMass (labelFloorLaw d eps delta true hd hdelta) j a y /
jointMass (labelFloorLaw d eps delta false hd hdelta) j a y)) =
if j.val < 2 then eps * delta * log ((1 / 2 + delta) / (1 / 2 - delta)) else 0 := by
simp only [labelFloor_jointMass d eps delta true hd hdelta heps heps',
labelFloor_jointMass d eps delta false hd hdelta heps heps', Fintype.sum_bool]
by_cases hj : j.val < 2
· simp [hj, bernoulliMass, ← sub_eq_add_neg]
rw [mul_div_mul_left _ _ (mul_ne_zero (by norm_num : (2 : Real)⁻¹ ≠ 0) heps0),
mul_div_mul_left _ _ (mul_ne_zero (by norm_num : (2 : Real)⁻¹ ≠ 0) heps0)]
rw [show (1 - (2⁻¹ + delta) : Real) = 1 / 2 - delta by ring,
show (1 - (2⁻¹ - delta) : Real) = 1 / 2 + delta by ring, hlog]
ring
· simp [hj]
simp_rw [hcell]
rw [labelFloor_sum_two_cells d hd]
ring
theorem labelFloor_KL_bound
Under the stated inputs and conditions, The diagnostic one-record KL is bounded by sixteen times overlap times amplitude squared. This gives the stated result.
Formal statement
hypotheses
conclusion
≤ 16 * eps * delta ^ 2
Proof (Lean source)
-- @node: labelFloor_KL_bound
lemma labelFloor_KL_bound (d : Nat) (eps delta : Real) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
finiteKL (labelFloorLaw d eps delta true hd hdelta)
(labelFloorLaw d eps delta false hd hdelta) ≤ 16 * eps * delta ^ 2 := by
have hd0 := hdelta.1
have hd1 := hdelta.2
have hden : 0 < (1 / 2 : Real) - delta := by linarith
have hlog := Real.log_le_sub_one_of_pos
(div_pos (show (0 : Real) < 1 / 2 + delta by linarith) hden)
have hratio : (1 / 2 + delta) / (1 / 2 - delta) - 1 ≤ 8 * delta := by
apply (sub_le_iff_le_add).2
apply (div_le_iff₀ hden).2
nlinarith
rw [labelFloor_KL_formula d eps delta hd hdelta heps heps']
calc
_ ≤ (2 * eps * delta) * (8 * delta) :=
mul_le_mul_of_nonneg_left (hlog.trans hratio) (by positivity)
_ = _ := by ring
theorem labelFloor_absolutelyContinuous
Under the stated inputs and conditions, The diagnostic laws have common support even when the remaining cells are null. This gives the stated result.
Formal statement
hypotheses
conclusion
obsLaw (labelFloorLaw d eps delta true hd hdelta) ≪ obsLaw (labelFloorLaw d eps delta false hd hdelta)
Proof (Lean source)
-- @node: labelFloor_absolutelyContinuous
lemma labelFloor_absolutelyContinuous (d : Nat) (eps delta : Real) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
obsLaw (labelFloorLaw d eps delta true hd hdelta) ≪
obsLaw (labelFloorLaw d eps delta false hd hdelta) := by
let P := labelFloorLaw d eps delta true hd hdelta
let Q := labelFloorLaw d eps delta false hd hdelta
have hsupport (z : Obs d) (hq : Q.pmf z = 0) : P.pmf z = 0 := by
have hqreal : jointMass Q z.1 z.2.1 z.2.2 = 0 := by
change (Q.pmf z).toReal = 0
rw [hq, ENNReal.toReal_zero]
have hj : ¬z.1.val < 2 := by
intro hj
rw [labelFloor_jointMass d eps delta false hd hdelta heps heps', if_pos hj] at hqreal
have hp : 0 < (1 / 2 : Real) * bernoulliMass eps z.2.1 *
bernoulliMass (if z.2.1 then 1 / 2 - delta else 1 / 2) z.2.2 := by
have hδ := hdelta.2
have he1 : 0 < 1 - eps := by linarith
have hd1 : 0 < 1 / 2 - delta := by linarith
have hd2 : 0 < 1 - (1 / 2 - delta) := by linarith [hdelta.1]
cases ha : z.2.1 <;> cases hy : z.2.2 <;> dsimp [bernoulliMass] <;> positivity
simpa only [Bool.false_eq_true, if_false, ← sub_eq_add_neg] using hp.ne' hqreal
apply (ENNReal.toReal_eq_toReal_iff'
(P.pmf.apply_ne_top z) ENNReal.zero_ne_top).mp
change jointMass P z.1 z.2.1 z.2.2 = 0
rw [labelFloor_jointMass d eps delta true hd hdelta heps heps', if_neg hj]
intro s hs
change P.pmf.toMeasure s = 0
rw [P.pmf.toMeasure_apply_eq_toOuterMeasure_apply (by exact Set.toFinite s |>.countable.measurableSet),
PMF.toOuterMeasure_apply]
apply ENNReal.tsum_eq_zero.mpr
intro z
by_cases hz : z ∈ s
· have hq : Q.pmf z = 0 := by
have h := measure_mono_null (Set.singleton_subset_iff.mpr hz) hs
simpa [obsLaw, Q] using h
simp [Set.indicator_of_mem hz, hsupport z hq]
· simp [Set.indicator_of_notMem hz]
theorem labelFloor_labeled_KL_bound
Under the stated inputs and conditions, The ordered labeled arrays have divergence at most sixteen times their rare-label scale. This gives the stated result.
Formal statement
hypotheses
conclusion
conclusion 1
klDiv (labeledProductLaw (labelFloorLaw d eps delta true hd hdelta) n) (labeledProductLaw (labelFloorLaw d eps delta false hd hdelta) n)
≠ ⊤
conclusion 2
(klDiv (labeledProductLaw (labelFloorLaw d eps delta true hd hdelta) n) (labeledProductLaw (labelFloorLaw d eps delta false hd hdelta) n)).toReal
≤ 16 * ((n : Real) * eps) * delta ^ 2
Proof (Lean source)
-- @node: labelFloor_labeled_KL_bound
lemma labelFloor_labeled_KL_bound (n d : Nat) (eps delta : Real) (hd : 2 ≤ d)
(hdelta : LabelFloorAmplitude delta) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
klDiv
(labeledProductLaw (labelFloorLaw d eps delta true hd hdelta) n)
(labeledProductLaw (labelFloorLaw d eps delta false hd hdelta) n) ≠ ⊤ ∧
(klDiv
(labeledProductLaw (labelFloorLaw d eps delta true hd hdelta) n)
(labeledProductLaw (labelFloorLaw d eps delta false hd hdelta) n)).toReal ≤
16 * ((n : Real) * eps) * delta ^ 2 := by
let P := labelFloorLaw d eps delta true hd hdelta
let Q := labelFloorLaw d eps delta false hd hdelta
letI : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
letI : IsProbabilityMeasure (obsLaw Q) := by unfold obsLaw; infer_instance
have hac := labelFloor_absolutelyContinuous d eps delta hd hdelta heps heps'
have ht := productKL_tensorization n
(obsLaw P) (obsLaw Q) hac (by exact Integrable.of_finite)
refine ⟨ht.1, ?_⟩
have hKL : (klDiv (obsLaw P) (obsLaw Q)).toReal = finiteKL P Q := by
rw [klDiv_toReal_eq_sum_measureReal _ _ hac]
simp [obsLaw, finiteKL, Measure.real_def, P, Q]
calc
_ ≤ (n : Real) * (klDiv (obsLaw P) (obsLaw Q)).toReal := ht.2.2
_ = (n : Real) * finiteKL P Q := by rw [hKL]
_ ≤ (n : Real) * (16 * eps * delta ^ 2) :=
mul_le_mul_of_nonneg_left (labelFloor_KL_bound d eps delta hd hdelta heps heps')
(Nat.cast_nonneg _)
_ = _ := by ring
Under the stated inputs and conditions, Squaring the chosen amplitude gives the inverse maximum-scale expression. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: labelFloor_scaled_square
lemma labelFloor_scaled_square (S : Real) :
(1 / (16 * sqrt (max 1 S))) ^ 2 = 1 / (256 * max 1 S) := by
rw [div_pow, mul_pow, Real.sq_sqrt (by positivity)]
norm_num
theorem labelFloor_scaled_tv
Under the stated inputs and conditions, The scaled labeled pair is uniformly close in total variation. This gives the stated result.
Formal statement
hypotheses
conclusion
tvDist (labeledProductLaw (labelFloorLaw d eps (1 / (16 * sqrt (max 1 ((n : Real) * eps)))) true hd (labelFloorAmplitude_scaled _)) n) (labeledProductLaw (labelFloorLaw d eps (1 / (16 * sqrt (max 1 ((n : Real) * eps)))) false hd (labelFloorAmplitude_scaled _)) n)
≤ 1 / 4
Proof (Lean source)
-- @node: labelFloor_scaled_tv
lemma labelFloor_scaled_tv (n d : Nat) (eps : Real) (hd : 2 ≤ d)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
tvDist
(labeledProductLaw (labelFloorLaw d eps
(1 / (16 * sqrt (max 1 ((n : Real) * eps)))) true hd
(labelFloorAmplitude_scaled _)) n)
(labeledProductLaw (labelFloorLaw d eps
(1 / (16 * sqrt (max 1 ((n : Real) * eps)))) false hd
(labelFloorAmplitude_scaled _)) n) ≤ 1 / 4 := by
let delta := 1 / (16 * sqrt (max 1 ((n : Real) * eps)))
let hdelta := labelFloorAmplitude_scaled ((n : Real) * eps)
let P := labelFloorLaw d eps delta true hd hdelta
let Q := labelFloorLaw d eps delta false hd hdelta
letI : IsProbabilityMeasure (obsLaw P) := by unfold obsLaw; infer_instance
letI : IsProbabilityMeasure (obsLaw Q) := by unfold obsLaw; infer_instance
letI : IsProbabilityMeasure (labeledProductLaw P n) := by unfold labeledProductLaw; infer_instance
letI : IsProbabilityMeasure (labeledProductLaw Q n) := by unfold labeledProductLaw; infer_instance
have hac := pi_iid_absolutelyContinuous
(obsLaw P) (obsLaw Q) (labelFloor_absolutelyContinuous d eps delta hd hdelta heps heps') n
obtain ⟨hfin, hKL⟩ := labelFloor_labeled_KL_bound n d eps delta hd hdelta heps heps'
have hbudget : 16 * ((n : Real) * eps) * delta ^ 2 ≤ 1 / 16 := by
rw [show delta ^ 2 = 1 / (256 * max 1 ((n : Real) * eps)) from labelFloor_scaled_square _]
have hmax : 0 < max 1 ((n : Real) * eps) := by positivity
apply (le_div_iff₀ (by norm_num : (0 : Real) < 16)).2
field_simp
nlinarith [le_max_right (1 : Real) ((n : Real) * eps)]
have htv := pinskerBound_of_ac_of_ne_top
(labeledProductLaw P n) (labeledProductLaw Q n) hac hfin
apply htv.trans
apply (Real.sqrt_le_iff).2
constructor
· norm_num
· nlinarith [hKL.trans hbudget]
theorem labelFloor_scaled_rate
Under the stated inputs and conditions, The amplitude squared divided by four is exactly the asserted label benchmark constant. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: labelFloor_scaled_rate
lemma labelFloor_scaled_rate (n : Nat) (eps : Real) (hn : 1 ≤ n) (heps : 0 < eps) :
(1 / (16 * sqrt (max 1 ((n : Real) * eps)))) ^ 2 / 4 =
(1 / 1024 : Real) * labelBenchmark n eps := by
have hS : 0 < (n : Real) * eps := mul_pos (by exact_mod_cast (show 0 < n by omega)) heps
rw [labelFloor_scaled_square]
dsimp [labelBenchmark, labelScale]
by_cases h : (n : Real) * eps ≤ 1
· rw [max_eq_left h, min_eq_left ((one_le_inv₀ hS).2 h)]
norm_num
· rw [max_eq_right (le_of_not_ge h), min_eq_right (inv_le_one_of_one_le₀ (le_of_not_ge h))]
ring
Under the stated inputs and conditions, A close pair with opposite targets forces squared loss for every clipped Borel rule. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: labelFloor_testing_mse
lemma labelFloor_testing_mse {Z : Type*} [MeasurableSpace Z]
(mu nu : Measure Z) [IsProbabilityMeasure mu] [IsProbabilityMeasure nu]
(delta : Real) (hd : 0 ≤ delta) (htv : tvDist mu nu ≤ 1 / 4)
(T : Z → Real) (hT : Measurable T) (hbound : ∀ z, T z ∈ Icc (-1) 1) :
delta ^ 2 / 4 ≤ max (∫ z, (T z - delta) ^ 2 ∂mu) (∫ z, (T z - -delta) ^ 2 ∂nu) := by
have hsep : 2 * delta ≤ dist delta (-delta) := by
rw [Real.dist_eq, show delta - -delta = 2 * delta by ring, abs_of_nonneg (by positivity)]
have htest := half_one_sub_tvDist_le_max_error
(P₀ := mu) (P₁ := nu) hT hsep
have hloss (rho : Measure Z) [IsProbabilityMeasure rho] (theta : Real) :
delta ^ 2 * rho.real {z | delta ≤ dist (T z) theta} ≤
∫ z, (T z - theta) ^ 2 ∂rho := by
have hset : {z | delta ≤ dist (T z) theta} =
{z | delta ^ 2 ≤ (T z - theta) ^ 2} := by
ext z
simp only [Set.mem_setOf_eq, Real.dist_eq]
constructor <;> intro h <;>
nlinarith [abs_nonneg (T z - theta), sq_abs (T z - theta)]
rw [hset]
exact mul_meas_ge_le_integral_of_nonneg (Filter.Eventually.of_forall (fun z => sq_nonneg _))
(mse_integrable_of_estimator_bound rho T hT (by norm_num) hbound) _
have hmax := max_le_max (hloss mu delta) (hloss nu (-delta))
rw [← mul_max_of_nonneg _ _ (sq_nonneg delta)] at hmax
have hfloor : (1 / 4 : Real) ≤
max (mu.real {z | delta ≤ dist (T z) delta})
(nu.real {z | delta ≤ dist (T z) (-delta)}) := by linarith
exact (show delta ^ 2 / 4 = delta ^ 2 * (1 / 4 : Real) by ring).le.trans
((mul_le_mul_of_nonneg_left hfloor (sq_nonneg delta)).trans hmax)
Under the stated inputs and conditions, Adjoining an independent common probability law preserves total variation. This gives the stated result.
Proof (Lean source)
-- @node: labelFloor_tv_common_product
lemma labelFloor_tv_common_product {A B : Type*} [MeasurableSpace A] [MeasurableSpace B]
(mu nu : Measure A) (rho : Measure B)
[IsProbabilityMeasure mu] [IsProbabilityMeasure nu] [IsProbabilityMeasure rho] :
tvDist (mu.prod rho) (nu.prod rho) = tvDist mu nu := by
simpa only [Measure.compProd_const] using
tvDist_compProd_eq mu nu (Kernel.const A rho)
Under the stated inputs and conditions, Clipping the rule and target bounds squared risk by four under any probability law. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: labelFloor_bounded_mse
lemma labelFloor_bounded_mse {Z : Type*} [MeasurableSpace Z]
(rho : Measure Z) [IsProbabilityMeasure rho] (T : Z → Real)
(hbound : ∀ z, T z ∈ Icc (-1) 1) (theta : Real) (htheta : theta ∈ Icc (-1) 1) :
(∫ z, (T z - theta) ^ 2 ∂rho) ≤ 4 := by
have hpoint (z : Z) : (T z - theta) ^ 2 ≤ 4 := by
have h := hbound z
have habs : |T z - theta| ≤ 2 := abs_le.mpr ⟨by linarith [h.1, htheta.2],
by linarith [h.2, htheta.1]⟩
nlinarith [sq_abs (T z - theta), abs_nonneg (T z - theta)]
have h := integral_mono_of_nonneg (μ := rho)
(Filter.Eventually.of_forall (fun z => sq_nonneg (T z - theta)))
(integrable_const (4 : Real)) (Filter.Eventually.of_forall hpoint)
simpa using h
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.OraclePhaseAlgebra 5 declarations The capped frontier has oracle order exactly when the dimension is bounded by the marginal information divided by the square root of the label information.
The capped frontier has oracle order exactly when the dimension is bounded by the marginal information divided by the square root of the label information.
Under the stated inputs and conditions, A positive dimension bound can be expressed as a bound on its normalized ratio. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_dimension_bound_iff
lemma phase_dimension_bound_iff (d D S K : Real) (hD : 0 < D) (hS : 0 < S) :
d ≤ K * (D / sqrt S) ↔ (d / D) * sqrt S ≤ K := by
have hs := Real.sqrt_pos.mpr hS
rw [← mul_div_assoc, le_div_iff₀ hs, div_mul_eq_mul_div, div_le_iff₀ hD]
theorem oracle_of_frontier_comparison
Under the stated inputs and conditions, Uniform frontier comparisons turn bounded oracle risk ratios into the dimension condition. This gives the stated result.
Formal statement
hypotheses
v :
c C :
hc :
0 < c
hC :
0 < C
hcmp :
∀ k,
c * frontierRate (v k).n (v k).m (v k).d (v k).eps ≤ Rr k ∧
Rr k ≤ C * frontierRate (v k).n (v k).m (v k).d (v k).eps
hdiverge :
conclusion
Proof (Lean source)
-- @node: oracle_of_frontier_comparison
lemma oracle_of_frontier_comparison (v : ExperimentSeq) (Rr : Nat → Real)
(c C : Real) (hc : 0 < c) (hC : 0 < C)
(hcmp : ∀ k, c * frontierRate (v k).n (v k).m (v k).d (v k).eps ≤ Rr k ∧
Rr k ≤ C * frontierRate (v k).n (v k).m (v k).d (v k).eps)
(hdiverge : Tendsto (fun k => labelScale (v k).n (v k).eps) atTop atTop) :
IsBoundedUnder (· ≤ ·) atTop
(fun k => Rr k / labelBenchmark (v k).n (v k).eps) ↔
(fun k => ((v k).d : Real)) =O[atTop]
(fun k => (((v k).n : Real) + (v k).m) * (v k).eps *
logScale (v k).n (v k).eps / sqrt (labelScale (v k).n (v k).eps)) := by
let S : Nat → Real := fun k => labelScale (v k).n (v k).eps
let D : Nat → Real := fun k =>
(((v k).n : Real) + (v k).m) * (v k).eps * logScale (v k).n (v k).eps
let x : Nat → Real := fun k => ((v k).d : Real) / D k
let r : Nat → Real := fun k => frontierRate (v k).n (v k).m (v k).d (v k).eps
have hS (k) : 0 < S k := mul_pos (by exact_mod_cast (v k).n_pos) (v k).eps_pos
have hD (k) : 0 < D k :=
mul_pos (mul_pos (by
have hn : (0 : Real) < (v k).n := by exact_mod_cast (v k).n_pos
positivity) (v k).eps_pos)
(phase_logScale_pos _ _ (v k).n_pos (v k).eps_pos)
have hx (k) : 0 ≤ x k := div_nonneg (Nat.cast_nonneg _) (hD k).le
have hbench : ∀ᶠ k in atTop, labelBenchmark (v k).n (v k).eps = 1 / S k := by
filter_upwards [hdiverge.eventually (eventually_ge_atTop 1)] with k hk
exact (min_eq_right (inv_le_one_of_one_le₀ hk)).trans (by rw [one_div])
constructor
· intro hbounded
obtain ⟨M, hM⟩ := hbounded.eventually_le
let K : Real := max M 1 / c
have hK : 0 < K := div_pos (lt_of_lt_of_le zero_lt_one (le_max_right _ _)) hc
have hrate : ∀ᶠ k in atTop, r k ≤ K / S k := by
filter_upwards [hbench, hM] with k hb hm
have hupper : Rr k ≤ max M 1 / S k := by
have ht := (div_le_iff₀ (one_div_pos.mpr (hS k))).mp (by simpa only [hb] using hm)
calc
Rr k ≤ M * (1 / S k) := ht
_ ≤ max M 1 * (1 / S k) := mul_le_mul_of_nonneg_right
(le_max_left _ _) (one_div_pos.mpr (hS k)).le
_ = max M 1 / S k := by ring
apply (mul_le_mul_iff_right₀ hc).mp
calc
c * r k ≤ Rr k := (hcmp k).1
_ ≤ max M 1 / S k := hupper
_ = c * (K / S k) := by dsimp [K]; field_simp
apply isBigO_iff.mpr
refine ⟨Real.sqrt K, ?_⟩
filter_upwards [hrate, hdiverge.eventually (eventually_gt_atTop K)] with k hr hk
have hcap : r k = 1 / S k + x k ^ 2 := by
have hlt : r k < 1 := hr.trans_lt ((div_lt_one (hS k)).mpr hk)
exact min_eq_right (le_of_lt (by
simpa only [r, frontierRate, S, x, D, min_lt_iff, lt_self_iff_false, false_or] using hlt))
have hsquare : (x k * sqrt (S k)) ^ 2 ≤ K := by
have ht := (le_div_iff₀ (hS k)).mp (by simpa only [hcap] using hr)
rw [mul_pow, Real.sq_sqrt (hS k).le]
have hid : (1 / S k + x k ^ 2) * S k = 1 + x k ^ 2 * S k := by
field_simp [(hS k).ne']
rw [hid] at ht
linarith
have hnorm : x k * sqrt (S k) ≤ sqrt K := by
have hsK := Real.sq_sqrt hK.le
have hn := Real.sqrt_nonneg K
nlinarith
have hdim := (phase_dimension_bound_iff ((v k).d : Real) (D k) (S k)
(sqrt K) (hD k) (hS k)).mpr hnorm
change ‖((v k).d : Real)‖ ≤ sqrt K * ‖D k / sqrt (S k)‖
simpa only [Real.norm_eq_abs, abs_of_nonneg (Nat.cast_nonneg (v k).d : (0 : Real) ≤ _),
abs_of_nonneg (div_nonneg (hD k).le (Real.sqrt_nonneg _))] using hdim
· intro hbig
obtain ⟨K, hK, hdim⟩ := isBigO_iff'.mp hbig
apply isBoundedUnder_of_eventually_le (a := C * (1 + K ^ 2))
filter_upwards [hbench, hdim] with k hb hd
have hdim' : ((v k).d : Real) ≤ K * (D k / sqrt (S k)) := by
change ‖((v k).d : Real)‖ ≤ K * ‖D k / sqrt (S k)‖ at hd
simpa only [Real.norm_eq_abs, abs_of_nonneg (Nat.cast_nonneg (v k).d : (0 : Real) ≤ _),
abs_of_nonneg (div_nonneg (hD k).le (Real.sqrt_nonneg _))] using hd
have hnorm := (phase_dimension_bound_iff ((v k).d : Real) (D k) (S k)
K (hD k) (hS k)).mp hdim'
have hsquare : x k ^ 2 * S k ≤ K ^ 2 := by
have hn : 0 ≤ x k * sqrt (S k) := mul_nonneg (hx k) (Real.sqrt_nonneg _)
have ht : (x k * sqrt (S k)) ^ 2 ≤ K ^ 2 := by nlinarith
simpa only [mul_pow, Real.sq_sqrt (hS k).le] using ht
have hrate : r k ≤ (1 + K ^ 2) / S k := by
apply (min_le_right _ _).trans
apply (le_div_iff₀ (hS k)).mpr
have hid : (1 / S k + x k ^ 2) * S k = 1 + x k ^ 2 * S k := by field_simp [(hS k).ne']
change (1 / S k + x k ^ 2) * S k ≤ 1 + K ^ 2
rw [hid]
linarith
rw [hb]
apply (div_le_iff₀ (one_div_pos.mpr (hS k))).mpr
calc
Rr k ≤ C * r k := (hcmp k).2
_ ≤ C * ((1 + K ^ 2) / S k) := mul_le_mul_of_nonneg_left hrate hC.le
_ = C * (1 + K ^ 2) * (1 / S k) := by ring
theorem phase_supervised_oracle_scale
Under the stated inputs and conditions, On the supervised slice, the oracle dimension scale is exactly square-root label information. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: phase_supervised_oracle_scale
lemma phase_supervised_oracle_scale (n : Nat) (eps : Real) (hn : 1 ≤ n) (heps : 0 < eps) :
(n : Real) * eps * logScale n eps / sqrt (labelScale n eps) =
sqrt (labelScale n eps) * logScale n eps := by
have hS : 0 < labelScale n eps := mul_pos (by exact_mod_cast hn) heps
apply (div_eq_iff (Real.sqrt_pos.mpr hS).ne').mpr
have hs := Real.sq_sqrt hS.le
change labelScale n eps * logScale n eps = _
calc
labelScale n eps * logScale n eps =
(sqrt (labelScale n eps)) ^ 2 * logScale n eps := by rw [hs]
_ = _ := by ring
Under the stated inputs and conditions, Positive factors convert the dimension condition into the marginal-budget condition. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_budget_bound_iff
lemma phase_budget_bound_iff (d N a S K : Real) (ha : 0 < a) (hS : 0 < S) :
d ≤ K * (N * a / sqrt S) ↔ d * sqrt S / a ≤ K * N := by
rw [← mul_div_assoc, le_div_iff₀ (Real.sqrt_pos.mpr hS), div_le_iff₀ ha]
rw [mul_assoc]
theorem phase_dimension_budget_iff
Under the stated inputs and conditions, The dimension and marginal-budget big-O conditions use exactly the same eventual constants. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_dimension_budget_iff
lemma phase_dimension_budget_iff (v : ExperimentSeq) :
(fun k => ((v k).d : Real)) =O[atTop]
(fun k => (((v k).n : Real) + (v k).m) * (v k).eps *
logScale (v k).n (v k).eps / sqrt (labelScale (v k).n (v k).eps)) ↔
(fun k => ((v k).d : Real) * sqrt (labelScale (v k).n (v k).eps) /
((v k).eps * logScale (v k).n (v k).eps)) =O[atTop]
(fun k => ((v k).n : Real) + (v k).m) := by
have hS (k) : 0 < labelScale (v k).n (v k).eps :=
mul_pos (by exact_mod_cast (v k).n_pos) (v k).eps_pos
have ha (k) : 0 < (v k).eps * logScale (v k).n (v k).eps :=
mul_pos (v k).eps_pos (phase_logScale_pos _ _ (v k).n_pos (v k).eps_pos)
have hN (k) : 0 ≤ ((v k).n : Real) + (v k).m := by positivity
have hequiv (k : Nat) (K : Real) :
‖((v k).d : Real)‖ ≤ K * ‖(((v k).n : Real) + (v k).m) * (v k).eps *
logScale (v k).n (v k).eps / sqrt (labelScale (v k).n (v k).eps)‖ ↔
‖((v k).d : Real) * sqrt (labelScale (v k).n (v k).eps) /
((v k).eps * logScale (v k).n (v k).eps)‖ ≤ K * ‖((v k).n : Real) + (v k).m‖ := by
rw [mul_assoc]
simp only [Real.norm_eq_abs,
abs_of_nonneg (Nat.cast_nonneg (v k).d : (0 : Real) ≤ _),
abs_of_nonneg (hN k),
abs_of_nonneg (div_nonneg (mul_nonneg (hN k) (ha k).le) (Real.sqrt_nonneg _)),
abs_of_nonneg (div_nonneg (mul_nonneg (Nat.cast_nonneg _) (Real.sqrt_nonneg _)) (ha k).le)]
exact phase_budget_bound_iff _ _ _ _ _ (ha k) (hS k)
constructor
· intro h
obtain ⟨K, hK⟩ := isBigO_iff.mp h
exact isBigO_iff.mpr ⟨K, hK.mono (fun k hk => (hequiv k K).mp hk)⟩
· intro h
obtain ⟨K, hK⟩ := isBigO_iff.mp h
exact isBigO_iff.mpr ⟨K, hK.mono (fun k hk => (hequiv k K).mpr hk)⟩
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.PhaseAlgebra 5 declarations Abstract sequence algebra for risks uniformly comparable to the frontier.
Abstract sequence algebra for risks uniformly comparable to the frontier.
theorem phase_logScale_pos
Under the stated inputs and conditions, The logarithmic diagnostic is positive for every positive label scale. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_logScale_pos
lemma phase_logScale_pos (n : Nat) (eps : Real) (hn : 1 ≤ n) (heps : 0 < eps) :
0 < logScale n eps := by
apply Real.log_pos
have hS : 0 < labelScale n eps := mul_pos (by exact_mod_cast hn) heps
have he : 1 < exp 1 := Real.one_lt_exp_iff.mpr zero_lt_one
change 1 < exp 1 + labelScale n eps
linarith
theorem phase_labelBenchmark_le_frontier
Under the stated inputs and conditions, The frontier always dominates the capped rare-label benchmark. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_labelBenchmark_le_frontier
lemma phase_labelBenchmark_le_frontier (n m d : Nat) (eps : Real) :
labelBenchmark n eps ≤ frontierRate n m d eps := by
apply min_le_min_left
simpa only [one_div] using le_add_of_nonneg_right
(sq_nonneg ((d : Real) / (((n : Real) + m) * eps * logScale n eps)))
theorem phase_frontier_saturation
Under the stated inputs and conditions, At the dimension saturation threshold the capped frontier equals one. This gives the stated result.
Formal statement
hypotheses
conclusion
frontierRate n m d eps = 1
Proof (Lean source)
-- @node: phase_frontier_saturation
lemma phase_frontier_saturation (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps)
(hd : ((n : Real) + m) * eps * logScale n eps ≤ d) :
frontierRate n m d eps = 1 := by
have hS : 0 < labelScale n eps := mul_pos (by exact_mod_cast hn) heps
have hD : 0 < ((n : Real) + m) * eps * logScale n eps :=
mul_pos (mul_pos (by positivity) heps) (phase_logScale_pos n eps hn heps)
have hx : 1 ≤ (d : Real) / (((n : Real) + m) * eps * logScale n eps) :=
(le_div_iff₀ hD).mpr (by simpa using hd)
apply min_eq_left
nlinarith [one_div_pos.mpr hS]
theorem phase_frontier_plateau
Under the stated inputs and conditions, The marginal-budget plateau bounds the frontier by twice the capped label benchmark. This gives the stated result.
Formal statement
hypotheses
n m d :
eps :
hn :
1 ≤ n
heps :
0 < eps
conclusion
frontierRate n m d eps ≤ 2 * labelBenchmark n eps
Proof (Lean source)
-- @node: phase_frontier_plateau
lemma phase_frontier_plateau (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps)
(hbudget : (d : Real) * sqrt ((n : Real) * eps) / (eps * logScale n eps) ≤
(n : Real) + m) :
frontierRate n m d eps ≤ 2 * labelBenchmark n eps := by
let S : Real := labelScale n eps
let D : Real := ((n : Real) + m) * eps * logScale n eps
let x : Real := (d : Real) / D
have hS : 0 < S := mul_pos (by exact_mod_cast hn) heps
have hlog : 0 < eps * logScale n eps := mul_pos heps (phase_logScale_pos n eps hn heps)
have hD : 0 < D := by
dsimp [D]
rw [mul_assoc]
exact mul_pos (by positivity) hlog
have hsqrt : 0 < sqrt S := Real.sqrt_pos.mpr hS
have hsquare : (sqrt S) ^ 2 = S := Real.sq_sqrt hS.le
have hdim : (d : Real) * sqrt S ≤ D := by
simpa [D, S, labelScale, mul_assoc] using (div_le_iff₀ hlog).mp hbudget
have hx : x * sqrt S ≤ 1 := by
dsimp [x]
rw [div_mul_eq_mul_div]
exact (div_le_one hD).mpr hdim
have hxpos : 0 ≤ x := div_nonneg (Nat.cast_nonneg _) hD.le
have hsq : x ^ 2 ≤ 1 / S := by
apply (le_div_iff₀ hS).mpr
have hprod : (x * sqrt S) ^ 2 = x ^ 2 * S := by rw [mul_pow, hsquare]
nlinarith [mul_nonneg (mul_nonneg hxpos hsqrt.le) (sub_nonneg.mpr hx)]
change min 1 (1 / S + x ^ 2) ≤ 2 * min 1 S⁻¹
by_cases hinv : S⁻¹ ≤ 1
· rw [min_eq_right hinv]
exact (min_le_right _ _).trans (by simpa only [one_div] using (show
1 / S + x ^ 2 ≤ 2 * (1 / S) by linarith))
· rw [min_eq_left (le_of_not_ge hinv)]
exact (min_le_left _ _).trans (by norm_num)
theorem consistency_of_frontier_comparison
Under the stated inputs and conditions, Any sequence risk uniformly comparable to the displayed frontier has the consistency phase.
Formal statement
hypotheses
v :
c C :
hc :
0 < c
_hC :
0 < C
hcmp :
∀ k,
c * frontierRate (v k).n (v k).m (v k).d (v k).eps ≤ Rr k ∧
Rr k ≤ C * frontierRate (v k).n (v k).m (v k).d (v k).eps
Proof (Lean source)
-- @node: consistency_of_frontier_comparison
lemma consistency_of_frontier_comparison (v : ExperimentSeq) (Rr : Nat → Real)
(c C : Real) (hc : 0 < c) (_hC : 0 < C)
(hcmp : ∀ k, c * frontierRate (v k).n (v k).m (v k).d (v k).eps ≤ Rr k ∧
Rr k ≤ C * frontierRate (v k).n (v k).m (v k).d (v k).eps) :
Tendsto Rr atTop (nhds 0) ↔
Tendsto (fun k => labelScale (v k).n (v k).eps) atTop atTop ∧
(fun k => ((v k).d : Real)) =o[atTop]
(fun k => (((v k).n : Real) + (v k).m) * (v k).eps * logScale (v k).n (v k).eps) := by
let S : Nat → Real := fun k => labelScale (v k).n (v k).eps
let D : Nat → Real := fun k =>
(((v k).n : Real) + (v k).m) * (v k).eps * logScale (v k).n (v k).eps
let z : Nat → Real := fun k => ((v k).d : Real) / D k
let r : Nat → Real := fun k => frontierRate (v k).n (v k).m (v k).d (v k).eps
have hS : ∀ k, 0 < S k := by
intro k
have hn : (0 : Real) < (v k).n := by exact_mod_cast (v k).n_pos
exact mul_pos hn (v k).eps_pos
have hD : ∀ k, 0 < D k := by
intro k
have hn : (0 : Real) < (v k).n := by exact_mod_cast (v k).n_pos
have hell : 0 < logScale (v k).n (v k).eps := by
apply Real.log_pos
have hexp : (1 : Real) < exp 1 := by
exact Real.one_lt_exp_iff.mpr zero_lt_one
change 1 < exp 1 + S k
linarith [hS k]
exact mul_pos (mul_pos (by positivity) (v k).eps_pos) hell
have hz : ∀ k, 0 ≤ z k := fun k => div_nonneg (Nat.cast_nonneg _) (hD k).le
have hr : ∀ k, 0 ≤ r k := fun k =>
(frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos).le
have hlittle :
(fun k => ((v k).d : Real)) =o[atTop] D ↔ Tendsto z atTop (nhds 0) :=
isLittleO_iff_tendsto (fun k hk => False.elim ((hD k).ne' hk))
have hrate : Tendsto Rr atTop (nhds 0) ↔ Tendsto r atTop (nhds 0) := by
constructor
· intro hR
apply squeeze_zero hr (fun k => (le_div_iff₀ hc).2 ?_)
(by simpa using hR.div_const c)
simpa only [mul_comm] using (hcmp k).1
· intro hR
apply squeeze_zero (fun k => le_trans (mul_nonneg hc.le (hr k)) (hcmp k).1)
(fun k => (hcmp k).2)
simpa using hR.const_mul C
rw [hrate, hlittle]
constructor
· intro hR
-- A vanishing capped rate eventually lies strictly below its cap.
have hcap : ∀ᶠ k in atTop, r k = 1 / S k + z k ^ 2 := by
filter_upwards [hR.eventually_lt_const (by norm_num : (0 : Real) < 1)] with k hk
dsimp [r, frontierRate, S, z, D] at hk ⊢
exact min_eq_right (le_of_lt (by simpa only [min_lt_iff, lt_self_iff_false,
false_or] using hk))
have hsum : Tendsto (fun k => 1 / S k + z k ^ 2) atTop (nhds 0) :=
hR.congr' hcap
have hinv : Tendsto (fun k => (S k)⁻¹) atTop (nhds 0) := by
apply squeeze_zero (fun k => inv_nonneg.mpr (hS k).le)
(fun k => ?_) hsum
simpa only [one_div] using le_add_of_nonneg_right (sq_nonneg (z k))
have hsq : Tendsto (fun k => z k ^ 2) atTop (nhds 0) :=
squeeze_zero (fun k => sq_nonneg _) (fun k =>
le_add_of_nonneg_left (div_nonneg (by norm_num) (hS k).le)) hsum
have hzlim : Tendsto z atTop (nhds 0) := by
have ht := Real.continuous_sqrt.continuousAt.tendsto.comp hsq
simpa only [Function.comp_def, Real.sqrt_zero, Real.sqrt_sq_eq_abs,
abs_of_nonneg (hz _)] using ht
have hinvpos : Tendsto (fun k => (S k)⁻¹) atTop (nhdsWithin 0 (Ioi 0)) :=
tendsto_nhdsWithin_iff.mpr ⟨hinv,
Eventually.of_forall (fun k => inv_pos.mpr (hS k))⟩
have hdiverge := tendsto_inv_nhdsGT_zero.comp hinvpos
exact ⟨by simpa only [Function.comp_def, inv_inv] using hdiverge, hzlim⟩
· rintro ⟨hdiverge, hzlim⟩
have hinv : Tendsto (fun k => 1 / S k) atTop (nhds 0) := by
simpa only [one_div, Function.comp_def] using
(tendsto_inv_atTop_zero.comp hdiverge)
have hsum : Tendsto (fun k => 1 / S k + z k ^ 2) atTop (nhds 0) := by
simpa using hinv.add (hzlim.pow 2)
apply squeeze_zero hr (fun k => min_le_right _ _) hsum
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.PoissonInverseMoments 1 declarations First moment, squared moment, and variance of a shifted reciprocal Poisson count.
First moment, squared moment, and variance of a shifted reciprocal Poisson count.
The shifted reciprocal Poisson count has its exact first moment and the displayed universal squared-moment and variance bounds.
Formal statement
hypotheses
lam :
conclusion
Proof (Lean source)
lemma poisson_inverse_moments (lam : NNReal) :
let g : Nat → Real := fun k => ((k : Real) + 1)⁻¹
(∫ k, g k ∂poissonMeasure lam) =
(if (lam : Real) = 0 then 1 else (1 - exp (-(lam : Real))) / (lam : Real)) ∧
(∫ k, g k ^ 2 ∂poissonMeasure lam) ≤ 2 ^ 16 / (1 + (lam : Real)) ^ 2 ∧
variance g (poissonMeasure lam) ≤ 2 ^ 16 * (lam : Real) / (1 + (lam :
Real)) ^ 4 :=
by
dsimp only
let g : Nat → Real := fun k => ((k : Real) + 1)⁻¹
let r : Nat → Real := fun k =>
((((k + 1 : Nat) : Real) * ((k + 2 : Nat) : Real)))⁻¹
have hg : MemLp g 2 (poissonMeasure lam) := by
simpa only [g, Nat.cast_add, Nat.cast_one] using
shifted_reciprocal_memLp_two lam
have hr := Causalean.Mathlib.Probability.Poisson.shifted_reciprocal_pair_memLp_two lam
have hdiff : addOne g =
fun k => -r k := by
funext k
dsimp [addOne, g, r]
push_cast
field_simp
ring
have hd : MemLp
(addOne g) 2
(poissonMeasure lam) := by
rw [hdiff]
exact hr.neg
have hvar := poisson_addOne_poincare
lam g hg hd
rw [hdiff] at hvar
simp only [neg_sq] at hvar
have hsecond := shifted_reciprocal_sq_bound lam
have hfourth := shifted_reciprocal_pair_sq_bound lam
simp only [Nat.cast_add, Nat.cast_one] at hsecond
have hden : 0 < 1 + (lam : Real) := by positivity
refine ⟨?_, ?_, ?_⟩
· by_cases hz : (lam : Real) = 0
· have hl : lam = 0 := NNReal.coe_eq_zero.mp hz
subst lam
rw [integral_poissonMeasure]
simp only [NNReal.coe_zero, neg_zero, Real.exp_zero, one_mul, smul_eq_mul]
rw [tsum_eq_single 0]
· norm_num
· intro k hk
simp [zero_pow hk]
· rw [if_neg hz]
apply (eq_div_iff hz).2
simpa only [Nat.cast_add, Nat.cast_one, mul_comm] using
shifted_reciprocal_first_moment lam
· apply (le_div_iff₀ (pow_pos hden 2)).2
nlinarith [hsecond]
· apply (le_div_iff₀ (pow_pos hden 4)).2
have hscaled := mul_le_mul_of_nonneg_right hvar (le_of_lt (pow_pos hden 4))
have hmoment := mul_le_mul_of_nonneg_left hfourth lam.coe_nonneg
change variance g (poissonMeasure lam) * (1 + (lam : Real)) ^ 4 ≤ _
dsimp only [r] at hscaled
nlinarith [hscaled, hmoment, lam.coe_nonneg]
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.PriorHandle 13 declarations Finite core/filler affine priors with a reservoir and shared normalizer.
Finite core/filler affine priors with a reservoir and shared normalizer.
structure AffineTuning
The affine-prior schedule records its intensities, degree, bandwidth, cone scale, and number of rare cells.
def affineTuning
The affine schedule uses all marginal records and a degree calibrated to rare-label information.
Definition (Lean source)
parameters
def
structure ConeDual
A finite signed reciprocal-gap certificate, with all its defining properties.
Definition (Lean source)
parameters
fields
strictMono :
StrictMono nodes
mem :
∀ i, nodes i ∈ Icc 0 B
abs_sum :
∑ i, |weights i| = 1
moments :
∀ h
if
h ≤ L
then
∑ i, weights i * nodes i ^ h = 0
def Latent
The latent type retains a separate filler branch even when a core atom is at zero.
instance instMeasurableSpaceLatent
For the cone degree, the measurable-space structure on latent states is the discrete measurable space.
Definition (Lean source)
def latentNode
The core branch reads its certificate node and the filler branch uses zero.
def latentPolar
The core sign is the polar sign of its signed weight; the filler carries no signed contribution.
def latentCoreMass
Core latent probabilities tilt the absolute certificate weights by the reciprocal treated mass.
def latentPMF
The finite latent PMF uses reciprocal-tilted core weights and places the remaining probability on the filler.
Definition (Lean source)
parameters
given by
normalizedPMF (fun z => match z with | none => 1 - ∑ i, latentCoreMass sigma i | some i => latentCoreMass sigma i) (PMF.pure none)
def latentMeanMass
The common expected raw cell mass averages b0 plus the latent node.
def rawTable
Raw complete-record masses implement the affine cone, signed outcome marking, reservoir, and null remaining cells.
Definition (Lean source)
parameters
hyp :
n m d :
eps :
sigma :
lat :
def
given by
tun :
affineTuning n m d eps
pstar :
1 - (tun.Kstar : Real) * latentMeanMass sigma
fun z
=> if hj : z.1.val < tun.Kstar then let v := latentNode sigma (lat ⟨z.1.val, hj⟩) let arm := if z.2.1 then tun.alpha
+ (1 - eps) * v else (1 - eps) * tun.b0
+ eps * v let mean := if z.2.1 then match lat ⟨z.1.val, hj⟩ with | none
=> 0 | some i => (1 + (if hyp then 1 else -1) * latentPolar sigma (some i)) / 2 else 0 arm * bernoulliMass mean z.2.2 else if z.1.val = tun.Kstar then (pstar / 2) * bernoulliMass 0 z.2.2 else 0
def normalizedLaw
The raw complete-record table is divided by its total mass, with a total fallback outside legal inputs.
Definition (Lean source)
parameters
hyp :
n m d :
eps :
hd :
2 ≤ d
sigma :
lat :
def
given by
⟨normalizedPMF (rawTable hyp n m d eps sigma lat) (PMF.pure (⟨0, lt_of_lt_of_le (zero_lt_succ 1 : 0 < 2) hd⟩, false, false))⟩
def affineProductPrior
The finite product of common latent draws pushes forward to normalized complete-record tables under each hypothesis.
Definition (Lean source)
parameters
hyp :
n m d :
eps :
hd :
2 ≤ d
sigma :
def
given by
(Measure.pi (fun _ : Fin (affineTuning n m d eps).Kstar => (latentPMF sigma).toMeasure)).map (normalizedLaw hyp n m d eps hd sigma)
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.Helpers.ShrinkingConeDual 5 declarations Reciprocal approximation gap on the shrinking overlap cone, using proved polynomial interfaces.
Reciprocal approximation gap on the shrinking overlap cone, using proved polynomial interfaces.
Under the stated inputs and conditions, The shrinking-cone reciprocal is continuous even at the zero endpoint. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: cone_reciprocal_continuous
lemma cone_reciprocal_continuous (L : Nat) (B eps : Real)
(hL : 2 ≤ L) (hB : 0 < B) (heps' : eps ≤ 1 / 4) :
ContinuousOn (fun v => (B / (100 * (L : Real) ^ 2)) /
((B / (100 * (L : Real) ^ 2)) + (1 - eps) * v)) (Icc 0 B) := by
have hLR : (0 : Real) < L := by exact_mod_cast (by omega : 0 < L)
apply ContinuousOn.div continuousOn_const (by fun_prop)
intro v hv
have : 0 < B / (100 * (L : Real) ^ 2) + (1 - eps) * v := by
have : 0 ≤ (1 - eps) * v := mul_nonneg (by linarith) hv.1
positivity
exact this.ne'
Under the stated inputs and conditions, Markov's derivative bound rules out error below one eighth on the shrinking cone. This gives the stated result.
Formal statement
hypotheses
L :
B eps :
hL :
2 ≤ L
hB :
0 < B
heps' :
eps ≤ 1 / 4
Q :
hQ :
Q.natDegree ≤ L
conclusion
1 / 8
≤ uniformApproxError (fun v => (B / (100 * (L : Real) ^ 2)) / ((B / (100 * (L : Real) ^ 2)) + (1 - eps) * v)) 0 B Q
Proof (Lean source)
-- @node: cone_reciprocal_polynomial_gap
lemma cone_reciprocal_polynomial_gap (L : Nat) (B eps : Real)
(hL : 2 ≤ L) (hB : 0 < B) (heps' : eps ≤ 1 / 4)
(Q : Polynomial Real) (hQ : Q.natDegree ≤ L) :
1 / 8 ≤ uniformApproxError (fun v => (B / (100 * (L : Real) ^ 2)) /
((B / (100 * (L : Real) ^ 2)) + (1 - eps) * v)) 0 B Q := by
let alpha := B / (100 * (L : Real) ^ 2)
let c := 1 - eps
let g : Real → Real := fun v => alpha / (alpha + c * v)
have hLR : (2 : Real) ≤ L := by exact_mod_cast hL
have hsq : 0 < (L : Real) ^ 2 := by positivity
have ha : 0 < alpha := by dsimp [alpha]; positivity
have hc : 3 / 4 ≤ c := by dsimp [c]; linarith
have hcpos : 0 < c := by linarith
have hg : ContinuousOn g (Icc 0 B) :=
cone_reciprocal_continuous L B eps hL hB heps'
by_contra h
have herr : uniformApproxError g 0 B Q < 1 / 8 := lt_of_not_ge h
have hpoint (v : Real) (hv : v ∈ Icc 0 B) : |g v - Q.eval v| < 1 / 8 :=
lt_of_le_of_lt ((intervalSupNorm_le_iff (hg.sub Q.continuous.continuousOn)
hB.le).mp (le_refl (uniformApproxError g 0 B Q)) v hv) herr
have hgBounds (v : Real) (hv : v ∈ Icc 0 B) : 0 ≤ g v ∧ g v ≤ 1 := by
have hv' : 0 ≤ c * v := mul_nonneg hcpos.le hv.1
have hden : 0 < alpha + c * v := by positivity
dsimp [g]
constructor
· positivity
· apply (div_le_iff₀ hden).2
linarith
have hnorm : intervalSupNorm (fun v => Q.eval v) 0 B ≤ 9 / 8 := by
apply (intervalSupNorm_le_iff Q.continuous.continuousOn hB.le).mpr
intro v hv
have hp := hpoint v hv
have hb := hgBounds v hv
have htri := abs_add_le (Q.eval v - g v) (g v)
rw [sub_add_cancel, abs_sub_comm, abs_of_nonneg hb.1] at htri
linarith
let z := alpha / c
have hzpos : 0 < z := div_pos ha hcpos
have hzB : z ≤ B := by
apply (div_le_iff₀ hcpos).2
have haeq : alpha * (100 * (L : Real) ^ 2) = B := by
dsimp [alpha]
field_simp
have hs : (4 : Real) ≤ (L : Real) ^ 2 := by nlinarith
nlinarith
have hg0 : g 0 = 1 := by simp [g, ha.ne']
have hgz : g z = 1 / 2 := by
dsimp [g, z]
field_simp
ring
have hp0 := hpoint 0 ⟨le_rfl, hB.le⟩
have hpz := hpoint z ⟨hzpos.le, hzB⟩
rw [hg0] at hp0
rw [hgz] at hpz
have hdrop : 1 / 4 < Q.eval 0 - Q.eval z := by
have hp0' := (abs_lt.mp hp0).2
have hpz' := (abs_lt.mp hpz).1
linarith
obtain ⟨v, hv, hslope⟩ := exists_deriv_eq_slope (fun x => Q.eval x) hzpos
Q.continuous.continuousOn Q.differentiableOn
have hMarkov := markov_derivative_Icc Q hB L hQ
have hderiv : |Q.derivative.eval v| ≤ 9 * (L : Real) ^ 2 / (4 * B) := by
have hp := (intervalSupNorm_le_iff Q.derivative.continuous.continuousOn hB.le).mp
hMarkov v ⟨hv.1.le, hv.2.le.trans hzB⟩
have hs := mul_le_mul_of_nonneg_left hnorm
(by positivity : 0 ≤ 2 * (L : Real) ^ 2 / (B - 0))
calc
|Q.derivative.eval v| ≤ _ := hp
_ ≤ (2 * (L : Real) ^ 2 / (B - 0)) * (9 / 8) := hs
_ = _ := by ring
rw [Polynomial.deriv, sub_zero] at hslope
have hslope' : Q.derivative.eval v * z = Q.eval z - Q.eval 0 :=
(eq_div_iff hzpos.ne').mp hslope
have hprod := mul_le_mul_of_nonneg_right hderiv hzpos.le
have hsmall : (9 * (L : Real) ^ 2 / (4 * B)) * z < 1 / 4 := by
have hid : (9 * (L : Real) ^ 2 / (4 * B)) * z = 9 / (400 * c) := by
dsimp [z, alpha]
field_simp
ring
rw [hid]
apply (div_lt_iff₀ (by positivity : 0 < 400 * c)).2
linarith
have habs : |Q.eval z - Q.eval 0| = |Q.derivative.eval v| * z := by
rw [← hslope', abs_mul, abs_of_pos hzpos]
have hlarge := le_abs_self (Q.eval 0 - Q.eval z)
rw [abs_sub_comm] at hlarge
rw [habs] at hlarge
linarith
Under the stated inputs and conditions, The polynomial inequalities imply a variation-one finite signed moment certificate with reciprocal gap at least one eighth.
Formal statement
Proof (Lean source)
-- @node: lem:shrinking-cone-dual
lemma shrinking_cone_dual (L : Nat) (B eps : Real) (hL : 2 ≤ L) (hB : 0 < B)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) : Nonempty (ConeDual L B eps) := by
let g : Real → Real := fun v => (B / (100 * (L : Real) ^ 2)) /
((B / (100 * (L : Real) ^ 2)) + (1 - eps) * v)
have hg := cone_reciprocal_continuous L B eps hL hB heps'
obtain ⟨Q, hQ, hbest⟩ := exists_bestPolynomial hB hg L
have hgap : 1 / 8 ≤ bestUniformApproxError g 0 B L := by
rw [← hbest]
exact cone_reciprocal_polynomial_gap L B eps hL hB heps' Q hQ
obtain ⟨nodes, w, hmono, hmem, habs, hmom, hvalue⟩ :=
FinitePolynomialDuality g 0 B L hB hg
exact ⟨⟨nodes, w, hmono, hmem, habs, hmom, hvalue.symm ▸ hgap⟩⟩
theorem affine_schedule_dual
Under the stated inputs and conditions, The public affine schedule admits the named shrinking-cone dual witness.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma affine_schedule_dual (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
Nonempty (ConeDual (affineTuning n m d eps).L (affineTuning n m d eps).B eps) := by
have hnR : (1 : Real) ≤ n := by exact_mod_cast hn
have hS : 0 < (n : Real) * eps := mul_pos (by linarith) heps
have hell : 1 < logScale n eps := by
have h := Real.strictMonoOn_log (Real.exp_pos 1)
(by positivity : 0 < exp 1 + (n : Real) * eps) (by linarith)
simpa only [Real.log_exp, logScale, labelScale] using h
have hceil : 8 * logScale n eps ≤ ((affineTuning n m d eps).L : Real) :=
Nat.le_ceil _
have hL : 2 ≤ (affineTuning n m d eps).L := by
have h : (2 : Real) ≤ (affineTuning n m d eps).L := by linarith
exact_mod_cast h
have hLpos : (0 : Real) < (affineTuning n m d eps).L := by
exact_mod_cast (by omega : 0 < (affineTuning n m d eps).L)
have hB : 0 < (affineTuning n m d eps).B := by
change 0 < ((affineTuning n m d eps).L : Real) /
(1000 * (128 * (n : Real) + 128 * ((n : Real) + m)))
positivity
exact shrinking_cone_dual _ _ _ hL hB heps heps'
def affinePrior
The testing priors use the displayed Dirac pair in the rare-label regime and choose the named dual witness only in the affine branch.
Definition (Lean source)
parameters
def
given by
S :
labelScale n eps
if S < exp 4096 ∨ x ^ 2
≤ 1 / S then Measure.dirac (labelFloorLaw d eps (1 / (16 * sqrt (max 1 S))) hyp hd (labelFloorAmplitude_scaled S)) else let sigma := choice (affine_schedule_dual n m d eps hn heps heps') affineProductPrior hyp n m d eps hd sigma
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.T_AnnotationResourcePhases 11 declarations Exact resource phases for arbitrary sequences and universal saturation and oracle plateaus.
Exact resource phases for arbitrary sequences and universal saturation and oracle plateaus.
theorem phase_minimaxRisk_le_one
Under the stated inputs and conditions, The zero Borel rule gives a risk at most one on the full observable model class. This gives the stated result.
Formal statement
hypotheses
conclusion
minimaxRisk n m d eps ≤ 1
Proof (Lean source)
-- @node: phase_minimaxRisk_le_one
lemma phase_minimaxRisk_le_one (n m d : Nat) (eps : Real)
(hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
minimaxRisk n m d eps ≤ 1 := by
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps'⟩
letI : Nonempty (ClassLaw d eps) := ⟨P0⟩
let T : Rule n m d := ⟨fun _ => 0, measurable_const, by intro z; norm_num⟩
apply (minimaxValue_le_worstCaseRisk_of_nonneg
(risk := fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
(fun T P => integral_nonneg (fun z => sq_nonneg _)) T).trans
apply ciSup_le
intro P
have htarget := ateFunctional_mem_Icc P.1
have hsq : (ateFunctional P.1) ^ 2 ≤ 1 := by
nlinarith [mul_nonneg (sub_nonneg.mpr htarget.2) (by linarith [htarget.1] : 0 ≤ ateFunctional P.1 + 1)]
haveI : IsProbabilityMeasure (obsLaw P.1) := by
dsimp [obsLaw]
infer_instance
haveI : IsProbabilityMeasure (labeledProductLaw P.1 n) := by
dsimp [labeledProductLaw]
infer_instance
haveI : IsProbabilityMeasure (auxProductLaw P.1 m) := by
dsimp [auxProductLaw]
infer_instance
haveI : IsProbabilityMeasure (annotationLaw P.1 n m) := by
dsimp [annotationLaw]
infer_instance
haveI : IsProbabilityMeasure seedLaw := by
constructor
simp [seedLaw]
simpa [ruleRisk, T, integral_const] using hsq
Under the stated inputs and conditions, Uniform positive comparisons transfer vanishing ratios in both directions. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_ratio_comparison_tendsto
lemma phase_ratio_comparison_tendsto (A B X Y : Nat → Real) (c C : Real)
(hc : 0 < c) (hC : 0 < C) (hA : ∀ k, 0 < A k) (hB : ∀ k, 0 < B k)
(hX : ∀ k, c * A k ≤ X k ∧ X k ≤ C * A k)
(hY : ∀ k, c * B k ≤ Y k ∧ Y k ≤ C * B k) :
Tendsto (fun k => X k / Y k) atTop (nhds 0) ↔
Tendsto (fun k => A k / B k) atTop (nhds 0) := by
have hXp (k) : 0 < X k := (mul_pos hc (hA k)).trans_le (hX k).1
have hYp (k) : 0 < Y k := (mul_pos hc (hB k)).trans_le (hY k).1
have hupper (k) : X k / Y k ≤ (C / c) * (A k / B k) := by
have h := (div_le_div_iff₀ (hYp k) (mul_pos hc (hB k))).2
(show X k * (c * B k) ≤ (C * A k) * Y k by
nlinarith [mul_le_mul_of_nonneg_right (hX k).2 (mul_pos hc (hB k)).le,
mul_le_mul_of_nonneg_left (hY k).1 (mul_pos hC (hA k)).le])
calc
X k / Y k ≤ (C * A k) / (c * B k) := h
_ = (C / c) * (A k / B k) := by field_simp
have hlower (k) : A k / B k ≤ (C / c) * (X k / Y k) := by
have h := (div_le_div_iff₀ (mul_pos hC (hB k)) (hYp k)).2
(show (c * A k) * Y k ≤ X k * (C * B k) by
nlinarith [mul_le_mul_of_nonneg_left (hY k).2 (mul_pos hc (hA k)).le,
mul_le_mul_of_nonneg_right (hX k).1 (mul_pos hC (hB k)).le])
have h' := mul_le_mul_of_nonneg_left h (le_of_lt (div_pos hC hc))
calc
A k / B k = (C / c) * ((c * A k) / (C * B k)) := by field_simp
_ ≤ (C / c) * (X k / Y k) := h'
constructor
· intro h
exact squeeze_zero (fun k => (div_pos (hA k) (hB k)).le) hlower
(by simpa using h.const_mul (C / c))
· intro h
exact squeeze_zero (fun k => (div_pos (hXp k) (hYp k)).le) hupper
(by simpa using h.const_mul (C / c))
Under the stated inputs and conditions, The supervised cap differs from the dimension-only cap by at most the inverse label scale. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_supervised_cap_bounds
lemma phase_supervised_cap_bounds (S x : Real) (hS : 0 < S) :
min 1 (x ^ 2) ≤ min 1 (1 / S + x ^ 2) ∧
min 1 (1 / S + x ^ 2) ≤ min 1 (x ^ 2) + 1 / S := by
refine ⟨min_le_min_left _ (by linarith [one_div_pos.mpr hS]), ?_⟩
by_cases hx : x ^ 2 ≤ 1
· rw [min_eq_right hx]
exact (min_le_right _ _).trans_eq (by ring)
· rw [min_eq_left (le_of_not_ge hx)]
exact (min_le_left _ _).trans (by linarith [one_div_pos.mpr hS])
Under the stated inputs and conditions, Normalizing the auxiliary dimension term by its cap gives the exact budget ratio square. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_improvement_budget_square
lemma phase_improvement_budget_square (x h : Real) (hx : 0 < x) (hh : 0 < h) :
(x / h) ^ 2 / min 1 (x ^ 2) = (max 1 x / h) ^ 2 := by
by_cases hsmall : x ≤ 1
· have hsq : x ^ 2 ≤ 1 := by nlinarith
rw [min_eq_right hsq, max_eq_left hsmall]
field_simp
· have hlarge : 1 ≤ x := le_of_not_ge hsmall
have hsq : 1 ≤ x ^ 2 := by nlinarith
rw [min_eq_left hsq, max_eq_right hlarge, div_one]
theorem improvement_ratio_of_frontier_comparison
Under the stated hypotheses, Strict improvement is exactly vanishing frontier ratio under the uniform comparison. This gives the stated result.
Formal statement
hypotheses
v :
c C :
hc :
0 < c
hC :
0 < C
hcmp :
if
1 ≤ n
and
2 ≤ d
and
0 < eps
and
eps ≤ 1 / 4
then
c * frontierRate n m d eps ≤ minimaxRisk n m d eps ∧
minimaxRisk n m d eps ≤ C * frontierRate n m d eps
conclusion
v ∈ improvementPhase
↔ Tendsto (fun k => frontierRate (v k).n (v k).m (v k).d (v k).eps / frontierRate (v k).n 0 (v k).d (v k).eps) atTop (nhds 0)
Proof (Lean source)
-- @node: improvement_ratio_of_frontier_comparison
lemma improvement_ratio_of_frontier_comparison (v : ExperimentSeq) (c C : Real)
(hc : 0 < c) (hC : 0 < C)
(hcmp : ∀ (n m d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d →
0 < eps → eps ≤ 1 / 4 →
c * frontierRate n m d eps ≤ minimaxRisk n m d eps ∧
minimaxRisk n m d eps ≤ C * frontierRate n m d eps) :
v ∈ improvementPhase ↔ Tendsto (fun k =>
frontierRate (v k).n (v k).m (v k).d (v k).eps /
frontierRate (v k).n 0 (v k).d (v k).eps) atTop (nhds 0) := by
exact phase_ratio_comparison_tendsto _ _ _ _ c C hc hC
(fun k => frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos)
(fun k => frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos)
(fun k => hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le)
(fun k => hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le)
theorem phase_improvement_label_divergence
Under the stated inputs and conditions, A vanishing frontier ratio forces diverging label information, even on oscillating sequences. This gives the stated result.
Formal statement
hypotheses
v :
h :
Tendsto (fun k => frontierRate (v k).n (v k).m (v k).d (v k).eps / frontierRate (v k).n 0 (v k).d (v k).eps) atTop (nhds 0)
conclusion
Proof (Lean source)
-- @node: phase_improvement_label_divergence
lemma phase_improvement_label_divergence (v : ExperimentSeq)
(h : Tendsto (fun k =>
frontierRate (v k).n (v k).m (v k).d (v k).eps /
frontierRate (v k).n 0 (v k).d (v k).eps) atTop (nhds 0)) :
Tendsto (fun k => labelScale (v k).n (v k).eps) atTop atTop := by
have hrate : Tendsto (fun k => frontierRate (v k).n (v k).m (v k).d (v k).eps)
atTop (nhds 0) := by
apply squeeze_zero (fun k =>
(frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos).le) (fun k => ?_) h
apply (le_div_iff₀ (frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos)).mpr
exact mul_le_of_le_one_right
(frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos).le
(frontierRate_le_one _ _ _ _)
exact ((consistency_of_frontier_comparison v
(fun k => frontierRate (v k).n (v k).m (v k).d (v k).eps)
1 1 zero_lt_one zero_lt_one (fun k => by simp)).mp hrate).1
Under the stated inputs and conditions, The capped frontier ratio is bounded by the three inverse information scales. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: phase_improvement_ratio_bound
lemma phase_improvement_ratio_bound (S x h : Real)
(hS : 0 < S) (hx : 0 < x) (hh : 0 < h) :
min 1 (1 / S + (x / h) ^ 2) / min 1 (1 / S + x ^ 2) ≤
S⁻¹ + ((sqrt S * x)⁻¹) ^ 2 + ((h / max 1 x)⁻¹) ^ 2 := by
let t := min 1 (x ^ 2)
have ht : 0 < t := lt_min zero_lt_one (sq_pos_of_pos hx)
have hB : t ≤ min 1 (1 / S + x ^ 2) :=
(phase_supervised_cap_bounds S x hS).1
have hlabel : (1 / S) / t ≤ S⁻¹ + ((sqrt S * x)⁻¹) ^ 2 := by
have hsqrt := Real.sq_sqrt hS.le
have hsqrtpos : 0 < sqrt S := Real.sqrt_pos.mpr hS
have hid : (1 / S) / x ^ 2 = ((sqrt S * x)⁻¹) ^ 2 := by
rw [mul_inv, mul_pow, inv_pow, hsqrt]
simp only [one_div, div_eq_mul_inv, inv_pow, one_mul]
by_cases hsmall : x ^ 2 ≤ 1
· dsimp [t]
rw [min_eq_right hsmall, hid]
exact le_add_of_nonneg_left (inv_nonneg.mpr hS.le)
· dsimp [t]
rw [min_eq_left (le_of_not_ge hsmall), div_one, one_div]
exact le_add_of_nonneg_right (sq_nonneg _)
have hbudget : (x / h) ^ 2 / t = ((h / max 1 x)⁻¹) ^ 2 := by
rw [phase_improvement_budget_square x h hx hh, inv_div]
calc
_ ≤ (1 / S + (x / h) ^ 2) / t :=
div_le_div₀ (by positivity) (min_le_right _ _) ht hB
_ = (1 / S) / t + (x / h) ^ 2 / t := add_div _ _ _
_ ≤ _ := by rw [hbudget]; exact add_le_add hlabel le_rfl
Under the stated inputs and conditions, Diverging label, relative-dimension, and marginal-budget scales imply strict improvement. This gives the stated result.
Formal statement
hypotheses
hS :
∀ k, 0 < S k
hx :
∀ k, 0 < x k
hh :
∀ k, 0 < h k
Proof (Lean source)
-- @node: phase_improvement_ratio_sufficient
lemma phase_improvement_ratio_sufficient (S x h : Nat → Real)
(hS : ∀ k, 0 < S k) (hx : ∀ k, 0 < x k) (hh : ∀ k, 0 < h k)
(hSlim : Tendsto S atTop atTop)
(hxdim : Tendsto (fun k => sqrt (S k) * x k) atTop atTop)
(hbudget : Tendsto (fun k => h k / max 1 (x k)) atTop atTop) :
Tendsto (fun k => min 1 (1 / S k + (x k / h k) ^ 2) /
min 1 (1 / S k + x k ^ 2)) atTop (nhds 0) := by
have h1 := tendsto_inv_atTop_zero.comp hSlim
have h2 := (tendsto_inv_atTop_zero.comp hxdim).pow 2
have h3 := (tendsto_inv_atTop_zero.comp hbudget).pow 2
apply squeeze_zero (fun k => by
have hSk := hS k
positivity)
(fun k => phase_improvement_ratio_bound _ _ _ (hS k) (hx k) (hh k))
simpa using (h1.add h2).add h3
Under the stated inputs and conditions, Once the label cap is inactive, a small frontier ratio controls both inverse scales. This gives the stated result.
Formal statement
hypotheses
S x h :
hS :
1 ≤ S
hx :
0 < x
hh :
0 < h
Proof (Lean source)
-- @node: phase_improvement_necessary_bounds
lemma phase_improvement_necessary_bounds (S x h : Real)
(hS : 1 ≤ S) (hx : 0 < x) (hh : 0 < h)
(hr : min 1 (1 / S + (x / h) ^ 2) / min 1 (1 / S + x ^ 2) ≤ 1 / 2) :
((sqrt S * x)⁻¹) ^ 2 ≤
2 * (min 1 (1 / S + (x / h) ^ 2) / min 1 (1 / S + x ^ 2)) ∧
((h / max 1 x)⁻¹) ^ 2 ≤
2 * (min 1 (1 / S + (x / h) ^ 2) / min 1 (1 / S + x ^ 2)) := by
let a := 1 / S
let A := min 1 (a + (x / h) ^ 2)
let B := min 1 (a + x ^ 2)
let r := A / B
let t := min 1 (x ^ 2)
have hSp : 0 < S := lt_of_lt_of_le zero_lt_one hS
have ha : 0 < a := one_div_pos.mpr hSp
have ha1 : a ≤ 1 := by simpa [a] using one_div_le_one_div_of_le zero_lt_one hS
have hB : 0 < B := lt_min zero_lt_one (by positivity)
have ht : 0 < t := lt_min zero_lt_one (sq_pos_of_pos hx)
have hr0 : 0 ≤ r := div_nonneg (le_min (by norm_num) (by positivity)) hB.le
have hrhalf : r ≤ 1 / 2 := hr
have hAB : r * B = A := div_mul_cancel₀ A hB.ne'
have hAhalf : A ≤ 1 / 2 := by
have := mul_le_mul_of_nonneg_left (min_le_left 1 (a + x ^ 2)) hr0
simp only [mul_one] at this
change r * B ≤ r at this
linarith
have hcap : A = a + (x / h) ^ 2 := by
apply min_eq_right
have : min 1 (a + (x / h) ^ 2) < 1 := lt_of_le_of_lt hAhalf (by norm_num)
exact le_of_lt (by
simpa only [min_lt_iff, lt_self_iff_false, false_or] using this)
have haA : a ≤ A := by rw [hcap]; exact le_add_of_nonneg_right (sq_nonneg _)
have hBsum : B ≤ a + x ^ 2 := min_le_right _ _
have har : a ≤ r * (a + x ^ 2) :=
haA.trans (hAB ▸ mul_le_mul_of_nonneg_left hBsum hr0)
have hax : a ≤ x ^ 2 := by
have := mul_le_mul_of_nonneg_right hrhalf (by positivity : 0 ≤ a + x ^ 2)
nlinarith
have haz : a ≤ 2 * r * x ^ 2 := by
have := mul_le_mul_of_nonneg_right hrhalf ha.le
nlinarith
have hid : ((sqrt S * x)⁻¹) ^ 2 = a / x ^ 2 := by
rw [mul_inv, mul_pow, inv_pow, Real.sq_sqrt hSp.le]
simp only [a, div_eq_mul_inv, inv_pow, one_mul]
refine ⟨?_, ?_⟩
· rw [hid]
exact (div_le_iff₀ (sq_pos_of_pos hx)).mpr haz
· have hat : a ≤ t := le_min ha1 hax
have hBt : B ≤ 2 * t := by
have hb := (phase_supervised_cap_bounds S x hSp).2
change B ≤ t + a at hb
linarith
have hnum : (x / h) ^ 2 ≤ r * (2 * t) := by
have hb := mul_le_mul_of_nonneg_left hBt hr0
rw [hAB, hcap] at hb
linarith
rw [inv_div, ← phase_improvement_budget_square x h hx hh]
exact (div_le_iff₀ ht).mpr (by nlinarith [hnum])
Under the stated inputs and conditions, Vanishing capped frontier ratios force both dimension and marginal-budget divergence. This gives the stated result.
Formal statement
hypotheses
hS :
∀ k, 0 < S k
hx :
∀ k, 0 < x k
hh :
∀ k, 0 < h k
Proof (Lean source)
-- @node: phase_improvement_ratio_necessary
lemma phase_improvement_ratio_necessary (S x h : Nat → Real)
(hS : ∀ k, 0 < S k) (hx : ∀ k, 0 < x k) (hh : ∀ k, 0 < h k)
(hSlim : Tendsto S atTop atTop)
(hratio : Tendsto (fun k => min 1 (1 / S k + (x k / h k) ^ 2) /
min 1 (1 / S k + x k ^ 2)) atTop (nhds 0)) :
Tendsto (fun k => sqrt (S k) * x k) atTop atTop ∧
Tendsto (fun k => h k / max 1 (x k)) atTop atTop := by
have hbounds : ∀ᶠ k in atTop,
((sqrt (S k) * x k)⁻¹) ^ 2 ≤
2 * (min 1 (1 / S k + (x k / h k) ^ 2) / min 1 (1 / S k + x k ^ 2)) ∧
((h k / max 1 (x k))⁻¹) ^ 2 ≤
2 * (min 1 (1 / S k + (x k / h k) ^ 2) / min 1 (1 / S k + x k ^ 2)) := by
filter_upwards [hSlim.eventually (eventually_ge_atTop 1),
hratio.eventually_lt_const (by norm_num : (0 : Real) < 1 / 2)] with k hs hr
exact phase_improvement_necessary_bounds _ _ _ hs (hx k) (hh k) hr.le
have hzero : Tendsto (fun k =>
2 * (min 1 (1 / S k + (x k / h k) ^ 2) / min 1 (1 / S k + x k ^ 2)))
atTop (nhds 0) := by simpa using hratio.const_mul 2
have hinvert (f : Nat → Real) (hp : ∀ k, 0 < f k)
(hsq : Tendsto (fun k => (f k)⁻¹ ^ 2) atTop (nhds 0)) :
Tendsto f atTop atTop := by
have hi : Tendsto (fun k => (f k)⁻¹) atTop (nhds 0) := by
have ht := Real.continuous_sqrt.continuousAt.tendsto.comp hsq
simpa only [Function.comp_def, Real.sqrt_zero, Real.sqrt_sq_eq_abs,
abs_of_pos (inv_pos.mpr (hp _))] using ht
have hip : Tendsto (fun k => (f k)⁻¹) atTop (nhdsWithin 0 (Ioi 0)) :=
tendsto_nhdsWithin_iff.mpr ⟨hi, Eventually.of_forall (fun k => inv_pos.mpr (hp k))⟩
simpa only [Function.comp_def, inv_inv] using tendsto_inv_nhdsGT_zero.comp hip
constructor
· apply hinvert _ (fun k => mul_pos (Real.sqrt_pos.mpr (hS k)) (hx k))
exact squeeze_zero' (Eventually.of_forall (fun k => sq_nonneg _))
(hbounds.mono (fun k hk => hk.1)) hzero
· apply hinvert _ (fun k => div_pos (hh k) (lt_of_lt_of_le zero_lt_one (le_max_left _ _)))
exact squeeze_zero' (Eventually.of_forall (fun k => sq_nonneg _))
(hbounds.mono (fun k hk => hk.2)) hzero
theorem annotation_resource_phases
Exact consistency, oracle, and strict-improvement phases and the two finite-budget elbows.
Formal statement
conclusion
conclusion 1
v :
S :
fun k => labelScale (v k).n (v k).eps
Nn :
fun k => ((v k).n : Real) + (v k).m
ell :
fun k => logScale (v k).n (v k).eps
dd :
fun k => ((v k).d : Real)
dd =o[atTop] (fun k => Nn k * (v k).eps * ell k)
(∀ k, (v k).m = 0)
v ∉ improvementPhase
v ∈ oraclePhase
conclusion 3
∃ c C : Real,
0 < c
0 < C
n m d :
eps :
1 ≤ n
2 ≤ d
0 < eps
eps ≤ 1 / 4
c * labelBenchmark n eps ≤ minimaxRisk n m d eps
minimaxRisk n m d eps ≤ C * labelBenchmark n eps
Proof (Lean source)
theorem annotation_resource_phases :
(∀ v : ExperimentSeq,
let S := fun k => labelScale (v k).n (v k).eps
let Nn := fun k => ((v k).n : Real) + (v k).m
let ell := fun k => logScale (v k).n (v k).eps
let dd := fun k => ((v k).d : Real)
(v ∈ consistencyPhase ↔ Tendsto S atTop atTop ∧
dd =o[atTop] (fun k => Nn k * (v k).eps * ell k)) ∧
(v ∈ oraclePhase ↔ Tendsto S atTop atTop ∧
dd =O[atTop] (fun k => Nn k * (v k).eps * ell k / sqrt (S k))) ∧
(v ∈ improvementPhase ↔ Tendsto S atTop atTop ∧
Tendsto (fun k => dd k / (sqrt (S k) * ell k)) atTop atTop ∧
Tendsto (fun k => (Nn k / (v k).n) / max 1 (dd k / (S k * ell k))) atTop atTop) ∧
((∀ k, (v k).m = 0) →
(v ∈ consistencyPhase ↔ Tendsto S atTop atTop ∧ dd =o[atTop] (fun k => S k * ell k)) ∧
(v ∈ oraclePhase ↔ Tendsto S atTop atTop ∧ dd =O[atTop] (fun k => sqrt (S k) * ell
k)) ∧
v ∉ improvementPhase) ∧
(Tendsto S atTop atTop → (v ∈ oraclePhase ↔
(fun k => dd k * sqrt (S k) / ((v k).eps * ell k)) =O[atTop] Nn)) ∧
(v ∈ oraclePhase →
(∃ c C : Real, 0 < c ∧ 0 < C ∧ ∀ᶠ k in atTop,
c / S k ≤ (v k).risk ∧ (v k).risk ≤ C / S k) ∧
∃ c : Real, 0 < c ∧ ∀ᶠ k in atTop, ∀ m' : Nat,
c / S k ≤ minimaxRisk (v k).n m' (v k).d (v k).eps)) ∧
(∃ c : Real, 0 < c ∧ ∀ (n m d : Nat) (eps : Real),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
((n : Real) + m) * eps * logScale n eps ≤ d →
c ≤ minimaxRisk n m d eps ∧ minimaxRisk n m d eps ≤ 1) ∧
(∃ c C : Real, 0 < c ∧ 0 < C ∧ ∀ (n m d : Nat) (eps : Real),
1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
(d : Real) * sqrt ((n : Real) * eps) / (eps * logScale n eps) ≤ (n : Real) + m →
c * labelBenchmark n eps ≤ minimaxRisk n m d eps ∧
minimaxRisk n m d eps ≤ C * labelBenchmark n eps) := by
obtain ⟨c, C, hc, hcC, hfront⟩ := uniform_annotation_frontier.1
have hC : 0 < C := hc.trans_le hcC
have hcmp (n m d : Nat) (eps : Real) (hn : 1 ≤ n) (hd : 2 ≤ d)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
c * frontierRate n m d eps ≤ minimaxRisk n m d eps ∧
minimaxRisk n m d eps ≤ C * frontierRate n m d eps := by
obtain ⟨_, _, hlo, hmid, hhi⟩ := hfront n m d eps hn hd heps heps'
exact ⟨hlo, hmid.trans hhi⟩
refine ⟨?_, ?_, ?_⟩
· intro v
dsimp only
have hcons : v ∈ consistencyPhase ↔
Tendsto (fun k => labelScale (v k).n (v k).eps) atTop atTop ∧
(fun k => ((v k).d : Real)) =o[atTop]
(fun k => (((v k).n : Real) + (v k).m) * (v k).eps *
logScale (v k).n (v k).eps) := by
exact consistency_of_frontier_comparison v (fun k => (v k).risk) c C hc hC
(fun k => hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le)
have horacle : v ∈ oraclePhase ↔
Tendsto (fun k => labelScale (v k).n (v k).eps) atTop atTop ∧
(fun k => ((v k).d : Real)) =O[atTop]
(fun k => (((v k).n : Real) + (v k).m) * (v k).eps *
logScale (v k).n (v k).eps / sqrt (labelScale (v k).n (v k).eps)) := by
constructor
· rintro ⟨hS, hbounded⟩
exact ⟨hS, (oracle_of_frontier_comparison v (fun k => (v k).risk) c C hc hC
(fun k => hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le)
hS).mp hbounded⟩
· rintro ⟨hS, hdim⟩
exact ⟨hS, (oracle_of_frontier_comparison v (fun k => (v k).risk) c C hc hC
(fun k => hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le)
hS).mpr hdim⟩
refine ⟨hcons, horacle, ?_, ?_, ?_, ?_⟩
· rw [improvement_ratio_of_frontier_comparison v c C hc hC hcmp]
constructor
· intro hratio
refine ⟨phase_improvement_label_divergence v hratio, ?_⟩
let S := fun k => labelScale (v k).n (v k).eps
let ell := fun k => logScale (v k).n (v k).eps
let x := fun k => ((v k).d : Real) / (S k * ell k)
let h := fun k => (((v k).n : Real) + (v k).m) / (v k).n
have hS (k) : 0 < S k := mul_pos (by exact_mod_cast (v k).n_pos) (v k).eps_pos
have hell (k) : 0 < ell k := phase_logScale_pos _ _ (v k).n_pos (v k).eps_pos
have hx (k) : 0 < x k := div_pos
(by exact_mod_cast (show 0 < (v k).d by have := (v k).d_ge; omega))
(mul_pos (hS k) (hell k))
have hn (k) : (0 : Real) < (v k).n := by exact_mod_cast (v k).n_pos
have hh (k) : 0 < h k := div_pos
(add_pos_of_pos_of_nonneg (hn k) (Nat.cast_nonneg _)) (hn k)
have hrate (k) : frontierRate (v k).n (v k).m (v k).d (v k).eps =
min 1 (1 / S k + (x k / h k) ^ 2) := by
dsimp [frontierRate, x, h, S, labelScale, ell]
congr 2
field_simp [ne_of_gt (hn k)]
have hsuper (k) : frontierRate (v k).n 0 (v k).d (v k).eps =
min 1 (1 / S k + x k ^ 2) := by
simp only [frontierRate, Nat.cast_zero, add_zero, x, S, labelScale, ell]
have hnormalized : Tendsto (fun k => min 1 (1 / S k + (x k / h k) ^ 2) /
min 1 (1 / S k + x k ^ 2)) atTop (nhds 0) := by
simpa only [hrate, hsuper] using hratio
obtain ⟨hdim, hbudget⟩ := phase_improvement_ratio_necessary S x h hS hx hh
(phase_improvement_label_divergence v hratio) hnormalized
refine ⟨?_, hbudget⟩
convert hdim using 1
funext k
have hs := Real.sq_sqrt (hS k).le
have hsp := Real.sqrt_pos.mpr (hS k)
change ((v k).d : Real) / (sqrt (S k) * ell k) =
sqrt (S k) * (((v k).d : Real) / (S k * ell k))
field_simp [ne_of_gt (hS k), ne_of_gt (hell k), ne_of_gt hsp]
nlinarith [hs]
· intro hlimits
let S := fun k => labelScale (v k).n (v k).eps
let ell := fun k => logScale (v k).n (v k).eps
let x := fun k => ((v k).d : Real) / (S k * ell k)
let h := fun k => (((v k).n : Real) + (v k).m) / (v k).n
have hS (k) : 0 < S k := mul_pos (by exact_mod_cast (v k).n_pos) (v k).eps_pos
have hell (k) : 0 < ell k := phase_logScale_pos _ _ (v k).n_pos (v k).eps_pos
have hx (k) : 0 < x k := div_pos
(by exact_mod_cast (show 0 < (v k).d by have := (v k).d_ge; omega))
(mul_pos (hS k) (hell k))
have hn (k) : (0 : Real) < (v k).n := by exact_mod_cast (v k).n_pos
have hh (k) : 0 < h k := div_pos
(add_pos_of_pos_of_nonneg (hn k) (Nat.cast_nonneg _)) (hn k)
have hdim : Tendsto (fun k => sqrt (S k) * x k) atTop atTop := by
convert hlimits.2.1 using 1
funext k
have hs := Real.sq_sqrt (hS k).le
have hsp := Real.sqrt_pos.mpr (hS k)
change sqrt (S k) * (((v k).d : Real) / (S k * ell k)) =
((v k).d : Real) / (sqrt (S k) * ell k)
field_simp [ne_of_gt (hS k), ne_of_gt (hell k), ne_of_gt hsp]
nlinarith [hs]
have hrate (k) : frontierRate (v k).n (v k).m (v k).d (v k).eps =
min 1 (1 / S k + (x k / h k) ^ 2) := by
dsimp [frontierRate, x, h, S, labelScale, ell]
congr 2
field_simp [ne_of_gt (hn k)]
have hsuper (k) : frontierRate (v k).n 0 (v k).d (v k).eps =
min 1 (1 / S k + x k ^ 2) := by
simp only [frontierRate, Nat.cast_zero, add_zero, x, S, labelScale, ell]
simpa only [hrate, hsuper] using
phase_improvement_ratio_sufficient S x h hS hx hh hlimits.1 hdim hlimits.2.2
· intro hsupervised
refine ⟨?_, ?_, ?_⟩
· simpa only [hsupervised, Nat.cast_zero, add_zero, labelScale] using hcons
· have hscale :
(fun k => (((v k).n : Real) + (v k).m) * (v k).eps *
logScale (v k).n (v k).eps / sqrt (labelScale (v k).n (v k).eps)) =
(fun k => sqrt (labelScale (v k).n (v k).eps) *
logScale (v k).n (v k).eps) := by
funext k
simpa only [hsupervised, Nat.cast_zero, add_zero] using
phase_supervised_oracle_scale _ _ (v k).n_pos (v k).eps_pos
simpa only [hscale] using horacle
· intro himprove
have hpositive (k : Nat) : 0 < (v k).risk :=
(mul_pos hc (frontierRate_pos _ _ _ _ (v k).n_pos (v k).eps_pos)).trans_le
(hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le).1
have hone : (fun k => (v k).risk / minimaxRisk (v k).n 0 (v k).d (v k).eps) =
(fun _ : Nat => (1 : Real)) := by
funext k
have heq : minimaxRisk (v k).n 0 (v k).d (v k).eps = (v k).risk := by
simp only [PublicIndex.risk, hsupervised]
rw [heq, div_self (hpositive k).ne']
change Tendsto (fun k => (v k).risk /
minimaxRisk (v k).n 0 (v k).d (v k).eps) atTop (nhds 0) at himprove
rw [hone] at himprove
have hfalse : (1 : Real) = 0 := tendsto_nhds_unique tendsto_const_nhds himprove
exact one_ne_zero hfalse
· intro hS
rw [horacle, and_iff_right hS]
exact phase_dimension_budget_iff v
· intro horacle
obtain ⟨hS, hbounded⟩ := horacle
obtain ⟨M, hM⟩ := hbounded.eventually_le
have hlarge : ∀ᶠ k in atTop, 1 ≤ labelScale (v k).n (v k).eps :=
hS.eventually (eventually_ge_atTop 1)
have hbench : ∀ᶠ k in atTop,
labelBenchmark (v k).n (v k).eps = 1 / labelScale (v k).n (v k).eps := by
filter_upwards [hlarge] with k hk
dsimp only [labelBenchmark]
rw [min_eq_right (inv_le_one_of_one_le₀ hk), one_div]
have hlower (k : Nat) (m' : Nat) :
c * labelBenchmark (v k).n (v k).eps ≤
minimaxRisk (v k).n m' (v k).d (v k).eps :=
(mul_le_mul_of_nonneg_left
(phase_labelBenchmark_le_frontier _ _ _ _) hc.le).trans
(hcmp _ _ _ _ (v k).n_pos (v k).d_ge (v k).eps_pos (v k).eps_le).1
refine ⟨⟨c, max M 1, hc, lt_of_lt_of_le zero_lt_one (le_max_right _ _), ?_⟩,
c, hc, ?_⟩
· filter_upwards [hbench, hM] with k hk hMk
have hpos : 0 < labelScale (v k).n (v k).eps :=
mul_pos (by exact_mod_cast (v k).n_pos) (v k).eps_pos
have hupper := (div_le_iff₀ (one_div_pos.mpr hpos)).mp (by
simpa only [hk] using hMk)
constructor
· simpa only [hk, mul_one_div, PublicIndex.risk] using hlower k (v k).m
· calc
(v k).risk ≤ M * (1 / labelScale (v k).n (v k).eps) := hupper
_ ≤ max M 1 * (1 / labelScale (v k).n (v k).eps) :=
mul_le_mul_of_nonneg_right (le_max_left _ _) (one_div_pos.mpr hpos).le
_ = max M 1 / labelScale (v k).n (v k).eps := by rw [mul_one_div]
· filter_upwards [hbench] with k hk
intro m'
simpa only [hk, mul_one_div] using hlower k m'
· refine ⟨c, hc, ?_⟩
intro n m d eps hn hd heps heps' hdim
have hsat := phase_frontier_saturation n m d eps hn heps hdim
exact ⟨by simpa only [hsat, mul_one] using
(hcmp n m d eps hn hd heps heps').1,
phase_minimaxRisk_le_one n m d eps hd heps heps'⟩
· refine ⟨c, 2 * C, hc, by positivity, ?_⟩
intro n m d eps hn hd heps heps' hbudget
obtain ⟨hlo, hhi⟩ := hcmp n m d eps hn hd heps heps'
constructor
· exact (mul_le_mul_of_nonneg_left
(phase_labelBenchmark_le_frontier n m d eps) hc.le).trans hlo
· calc
minimaxRisk n m d eps ≤ C * frontierRate n m d eps := hhi
_ ≤ C * (2 * labelBenchmark n eps) := mul_le_mul_of_nonneg_left
(phase_frontier_plateau n m d eps hn heps hbudget) hC.le
_ = (2 * C) * labelBenchmark n eps := by ring
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.T_BenchmarkRecovery 8 declarations Same-class benchmark recovery and exact-marginal quantitative experiment comparison.
Same-class benchmark recovery and exact-marginal quantitative experiment comparison.
theorem benchmark_label_floor_on_bounded_scale
Under the stated inputs and conditions, The capped label benchmark has a positive uniform floor on bounded label scales. This gives the stated result.
Formal statement
hypotheses
conclusion
1 / max 1 M0 ≤ labelBenchmark n eps
Proof (Lean source)
-- @node: benchmark_label_floor_on_bounded_scale
lemma benchmark_label_floor_on_bounded_scale (n : Nat) (eps M0 : Real)
(hn : 1 ≤ n) (heps : 0 < eps)
(hscale : (n : Real) * eps ≤ M0) :
1 / max 1 M0 ≤ labelBenchmark n eps := by
have hn' : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hS : 0 < (n : Real) * eps := mul_pos hn' heps
have hmax : 0 < max (1 : Real) M0 := lt_of_lt_of_le zero_lt_one (le_max_left _ _)
change 1 / max 1 M0 ≤ min 1 (((n : Real) * eps)⁻¹)
refine le_min ?_ ?_
· exact (div_le_one hmax).2 (le_max_left _ _)
· simpa only [one_div] using
one_div_le_one_div_of_le hS (hscale.trans (le_max_right _ _))
Under the stated inputs and conditions, The fixed-parameter comparison error vanishes as the auxiliary sample grows. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: benchmark_comparison_error_tendsto
lemma benchmark_comparison_error_tendsto (n d : Nat) :
Tendsto (fun m : Nat =>
6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m)) atTop (nhds 0) := by
have hinv : Tendsto (fun m : Nat => (m : Real)⁻¹) atTop (nhds (0 : Real)) :=
tendsto_inv_atTop_zero.comp tendsto_natCast_atTop_atTop
have hdiv : Tendsto (fun m : Nat => 2 * (d : Real) / m) atTop (nhds (0 : Real)) := by
simpa only [div_eq_mul_inv, mul_zero] using hinv.const_mul (2 * (d : Real))
simpa only [Real.sqrt_zero, mul_zero] using
hdiv.sqrt.const_mul (6 * ((n : Real) + 2))
theorem benchmark_limit_of_comparison
Under the stated inputs and conditions, A quantitative comparison with the exact-marginal experiment implies its limit. This gives the stated result.
Formal statement
hypotheses
n d :
eps :
hcomparison :
∀ m : Nat
if
1 ≤ m
then
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps ∧
minimaxRisk n m d eps - knownMarginalRisk n d eps
conclusion
Proof (Lean source)
-- @node: benchmark_limit_of_comparison
lemma benchmark_limit_of_comparison (n d : Nat) (eps : Real)
(hcomparison : ∀ m : Nat, 1 ≤ m →
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps ∧
minimaxRisk n m d eps - knownMarginalRisk n d eps ≤
min 1 (6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m))) :
Tendsto (fun m => minimaxRisk n m d eps) atTop (nhds (knownMarginalRisk n d eps)) := by
have hnonneg : ∀ᶠ m : Nat in atTop,
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps := by
filter_upwards [eventually_ge_atTop 1] with m hm
exact (hcomparison m hm).1
have hupper : ∀ᶠ m : Nat in atTop,
minimaxRisk n m d eps - knownMarginalRisk n d eps ≤
6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m) := by
filter_upwards [eventually_ge_atTop 1] with m hm
exact (hcomparison m hm).2.trans (min_le_right _ _)
have hzero := squeeze_zero' hnonneg hupper (benchmark_comparison_error_tendsto n d)
simpa only [sub_add_cancel, zero_add] using hzero.add_const (knownMarginalRisk n d eps)
theorem benchmark_minimax_le_baseline
Under the stated inputs and conditions, The baseline guarantee bounds the infimum over all original-data Borel rules. This gives the stated result.
Formal statement
hypotheses
n m d :
eps C :
hd :
2 ≤ d
heps :
0 < eps
heps' :
eps ≤ 1 / 4
hbaseline :
conclusion
minimaxRisk n m d eps ≤ C
Proof (Lean source)
-- @node: benchmark_minimax_le_baseline
lemma benchmark_minimax_le_baseline (n m d : Nat) (eps C : Real)
(hd : 2 ≤ d) (heps : 0 < eps) (heps' : eps ≤ 1 / 4)
(hbaseline : ∀ P : ClassLaw d eps,
ruleRisk (liftRule (baselineEstimator n m d)) P.1 ≤ C) :
minimaxRisk n m d eps ≤ C := by
let P0 : ClassLaw d eps :=
⟨labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth,
labelFloor_model d eps (1 / 8) false hd labelFloorAmplitude_eighth heps heps'⟩
letI : Nonempty (ClassLaw d eps) := ⟨P0⟩
let T : Rule n m d := ⟨liftRule (baselineEstimator n m d),
(baselineEstimator_measurable n m d).comp measurable_fst,
fun z => baselineEstimator_mem_Icc n m d z.1⟩
apply (minimaxValue_le_worstCaseRisk_of_nonneg
(risk := fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
(fun T P => integral_nonneg (fun z => sq_nonneg _)) T).trans
exact ciSup_le hbaseline
theorem benchmark_binary_baseline_rate
Under the stated inputs and conditions, With two covariate cells the baseline comparison is at most five label benchmarks. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: benchmark_binary_baseline_rate
lemma benchmark_binary_baseline_rate (n m : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps) :
min 1 (1 / ((n : Real) * eps) + (2 / (((n : Real) + m) * eps)) ^ 2) ≤
5 * labelBenchmark n eps := by
have hn' : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hS : 0 < (n : Real) * eps := mul_pos hn' heps
have hN : (n : Real) * eps ≤ ((n : Real) + m) * eps := by
nlinarith [Nat.cast_nonneg (α := Real) m]
by_cases hlarge : 1 ≤ (n : Real) * eps
· have hratio : 2 / (((n : Real) + m) * eps) ≤ 2 / ((n : Real) * eps) :=
div_le_div_of_nonneg_left (by norm_num) hS hN
have hsq : (2 / (((n : Real) + m) * eps)) ^ 2 ≤
(2 / ((n : Real) * eps)) ^ 2 := by
exact pow_le_pow_left₀ (by positivity) hratio 2
have hinv : 0 ≤ 1 / ((n : Real) * eps) := by positivity
have hinv' : 1 / ((n : Real) * eps) ≤ 1 := (div_le_one hS).2 hlarge
have hbound : 1 / ((n : Real) * eps) +
(2 / (((n : Real) + m) * eps)) ^ 2 ≤ 5 / ((n : Real) * eps) := by
have htwo : 2 / ((n : Real) * eps) = 2 * (1 / ((n : Real) * eps)) := by ring
rw [htwo] at hsq
have hfive : 5 / ((n : Real) * eps) = 5 * (1 / ((n : Real) * eps)) := by ring
rw [hfive]
nlinarith [mul_nonneg hinv (sub_nonneg.mpr hinv')]
calc
min 1 _ ≤ 5 / ((n : Real) * eps) := (min_le_right _ _).trans hbound
_ = 5 * labelBenchmark n eps := by
simp only [labelBenchmark, labelScale, min_eq_right (inv_le_one_of_one_le₀ hlarge),
one_div, div_eq_mul_inv]
· have hbench : labelBenchmark n eps = 1 := by
dsimp only [labelBenchmark, labelScale]
exact min_eq_left ((one_le_inv₀ hS).2 (by linarith))
rw [hbench]
exact (min_le_left _ _).trans (by norm_num)
theorem benchmark_log_comparison
Under the stated inputs and conditions, Uniform logarithmic comparisons at a fixed positive overlap floor. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: benchmark_log_comparison
lemma benchmark_log_comparison (n : Nat) (eps : Real)
(hn : 1 ≤ n) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
1 ≤ logScale n eps ∧
logScale n eps ≤ 2 * log (exp 1 * n) ∧
log (exp 1 * n) ≤
(2 + log (1 / eps)) * logScale n eps := by
have hn1 : (1 : Real) ≤ n := by exact_mod_cast hn
have hn0 : (0 : Real) < n := by linarith
have he : 0 < exp 1 := Real.exp_pos _
have hS : 0 < (n : Real) * eps := mul_pos hn0 heps
have hlogn : 0 ≤ log (n : Real) := Real.log_nonneg hn1
have hlogeps : 0 ≤ log (1 / eps) :=
Real.log_nonneg ((one_le_div heps).2 (by linarith))
have hLambda : log (exp 1 * n) = 1 + log (n : Real) := by
rw [Real.log_mul he.ne' hn0.ne', Real.log_exp]
have hlow : 1 ≤ logScale n eps := by
simpa only [Real.log_exp, logScale, labelScale] using
Real.log_le_log he (show exp 1 ≤ exp 1 + (n : Real) * eps by linarith)
have he2 : exp 1 + 1 ≤ exp 2 := by
have hgt : 2 < exp 1 := by
convert Real.add_one_lt_exp (show (1 : Real) ≠ 0 by norm_num) using 1
norm_num
rw [show (2 : Real) = 1 + 1 by norm_num, Real.exp_add]
nlinarith
have hu := Real.log_le_log (add_pos he hS)
(show exp 1 + (n : Real) * eps ≤ exp 2 * n by
have hprod := mul_le_mul_of_nonneg_right he2 hn0.le
nlinarith)
rw [Real.log_mul (Real.exp_pos _).ne' hn0.ne', Real.log_exp] at hu
have hl := Real.log_le_log hn0
(show (n : Real) ≤ (exp 1 + (n : Real) * eps) * (1 / eps) by
field_simp
linarith)
rw [Real.log_mul (add_pos he hS).ne' (one_div_pos.mpr heps).ne'] at hl
change log (n : Real) ≤ logScale n eps + log (1 / eps) at hl
refine ⟨hlow, ?_, ?_⟩
· rw [hLambda]
change log (exp 1 + (n : Real) * eps) ≤ _
linarith
· rw [hLambda]
nlinarith [mul_nonneg hlogeps (sub_nonneg.mpr hlow)]
theorem benchmark_fixed_overlap_comparison
Under the stated hypotheses, Fixed-floor logarithmic comparisons give the uncapped and capped rate comparisons. This gives the stated result.
Formal statement
hypotheses
conclusion
∃ B : Real,
0 < B
conclusion 1
n m d :
1 ≤ n
fixedOverlapBenchmark n m d ≤ frontierRate n m d eps
frontierRate n m d eps ≤ B * fixedOverlapBenchmark n m d
Proof (Lean source)
-- @node: benchmark_fixed_overlap_comparison
lemma benchmark_fixed_overlap_comparison (eps : Real)
(heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
∃ B : Real, 0 < B ∧ ∀ (n m d : Nat), 1 ≤ n →
fixedOverlapBenchmark n m d ≤ frontierRate n m d eps ∧
frontierRate n m d eps ≤ B * fixedOverlapBenchmark n m d := by
let K : Real := 2 + log (1 / eps)
let B : Real := max 1 (max (1 / eps) (K / eps) ^ 2)
have hlogeps : 0 ≤ log (1 / eps) :=
Real.log_nonneg ((one_le_div heps).2 (by linarith))
have hK : 0 < K := by dsimp [K]; linarith
have hB1 : 1 ≤ B := le_max_left _ _
have hB : 0 < B := zero_lt_one.trans_le hB1
have hEinv : 1 ≤ 1 / eps := (one_le_div heps).2 (by linarith)
have hbase : 1 / eps ≤ B := by
have hz : 1 ≤ max (1 / eps) (K / eps) := hEinv.trans (le_max_left _ _)
have hs : max (1 / eps) (K / eps) ≤ max (1 / eps) (K / eps) ^ 2 := by nlinarith
exact (le_max_left _ _).trans (hs.trans (le_max_right _ _))
have hfactor : (K / eps) ^ 2 ≤ B :=
(pow_le_pow_left₀ (by positivity) (le_max_right _ _) 2).trans (le_max_right _ _)
refine ⟨B, hB, ?_⟩
intro n m d hn
have hn0 : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hN : 0 < (n : Real) + m := by positivity
obtain ⟨hl1, hlu, hll⟩ := benchmark_log_comparison n eps hn heps heps'
let ell := logScale n eps
let Lam := log (exp 1 * n)
have hl : 0 < ell := zero_lt_one.trans_le hl1
have hLam : 0 < Lam := by
dsimp [Lam]
rw [Real.log_mul (Real.exp_pos _).ne' hn0.ne', Real.log_exp]
linarith [Real.log_nonneg (show (1 : Real) ≤ n by exact_mod_cast hn)]
have hden : ((n : Real) + m) * eps * ell ≤ ((n : Real) + m) * Lam := by
have : eps * ell ≤ Lam := by nlinarith
nlinarith
have hratio : (d : Real) / (((n : Real) + m) * Lam) ≤
(d : Real) / (((n : Real) + m) * eps * ell) :=
div_le_div_of_nonneg_left (by positivity) (by positivity) hden
have hratio' : (d : Real) / (((n : Real) + m) * eps * ell) ≤
(K / eps) * ((d : Real) / (((n : Real) + m) * Lam)) := by
apply (div_le_iff₀ (by positivity : 0 < ((n : Real) + m) * eps * ell)).2
have hid : (K / eps) * ((d : Real) / (((n : Real) + m) * Lam)) *
(((n : Real) + m) * eps * ell) = (d : Real) * (K * ell / Lam) := by
field_simp
rw [hid]
exact le_mul_of_one_le_right (by positivity)
((le_div_iff₀ hLam).2 (by simpa only [one_mul] using hll))
have hterm : (d : Real) ^ 2 / (((n : Real) + m) ^ 2 * Lam ^ 2) =
((d : Real) / (((n : Real) + m) * Lam)) ^ 2 := by
rw [div_pow, mul_pow]
have hlo : 1 / (n : Real) + (d : Real) ^ 2 /
(((n : Real) + m) ^ 2 * Lam ^ 2) ≤
1 / ((n : Real) * eps) + ((d : Real) / (((n : Real) + m) * eps * ell)) ^ 2 := by
rw [hterm]
exact add_le_add (one_div_le_one_div_of_le (by positivity) (by nlinarith))
(pow_le_pow_left₀ (by positivity) hratio 2)
have hhi : 1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps * ell)) ^ 2 ≤
B * (1 / (n : Real) + (d : Real) ^ 2 / (((n : Real) + m) ^ 2 * Lam ^ 2)) := by
rw [hterm, mul_add]
apply add_le_add
· calc
1 / ((n : Real) * eps) = (1 / eps) * (1 / (n : Real)) := by ring
_ ≤ B * (1 / (n : Real)) := mul_le_mul_of_nonneg_right hbase (by positivity)
· calc
_ ≤ ((K / eps) * ((d : Real) / (((n : Real) + m) * Lam))) ^ 2 :=
pow_le_pow_left₀ (by positivity) hratio' 2
_ = (K / eps) ^ 2 * ((d : Real) / (((n : Real) + m) * Lam)) ^ 2 := mul_pow _ _ _
_ ≤ _ := mul_le_mul_of_nonneg_right hfactor (sq_nonneg _)
constructor
· exact min_le_min_left _ hlo
· change min 1 _ ≤ B * min 1 _
rw [mul_min_of_nonneg _ _ hB.le]
exact le_min ((min_le_left _ _).trans (by simpa using hB1))
((min_le_right _ _).trans hhi)
theorem benchmark_recovery
Under the stated inputs and conditions, Fixed-floor recovery, binary-alphabet and oracle orders, quantitative comparison, and limit. This gives the stated conclusion.
Formal statement
conclusion
conclusion 1
∃ a Bc : Real,
0 < a
0 < Bc
a * fixedOverlapBenchmark n m d ≤ frontierRate n m d eps
frontierRate n m d eps ≤ Bc * fixedOverlapBenchmark n m d
conclusion 2
∃ c C : Real,
0 < c
0 < C
c * labelBenchmark n eps ≤ minimaxRisk n m 2 eps
minimaxRisk n m 2 eps ≤ C * labelBenchmark n eps
c * labelBenchmark n eps ≤ knownMarginalRisk n d eps
knownMarginalRisk n d eps ≤ C * labelBenchmark n eps
conclusion 3
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps
minimaxRisk n m d eps - knownMarginalRisk n d eps
conclusion 4
Proof (Lean source)
theorem benchmark_recovery :
(∀ eps : Real, 0 < eps → eps ≤ 1 / 4 →
∃ a Bc : Real, 0 < a ∧ 0 < Bc ∧ ∀ (n m d : Nat), 1 ≤ n → 2 ≤ d →
a * fixedOverlapBenchmark n m d ≤ frontierRate n m d eps ∧
frontierRate n m d eps ≤ Bc * fixedOverlapBenchmark n m d) ∧
(∃ c C : Real, 0 < c ∧ 0 < C ∧
∀ (n m d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
(c * labelBenchmark n eps ≤ minimaxRisk n m 2 eps ∧
minimaxRisk n m 2 eps ≤ C * labelBenchmark n eps) ∧
(c * labelBenchmark n eps ≤ knownMarginalRisk n d eps ∧
knownMarginalRisk n d eps ≤ C * labelBenchmark n eps)) ∧
(∀ (n m d : Nat) (eps : Real), 1 ≤ n → 1 ≤ m → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps ∧
minimaxRisk n m d eps - knownMarginalRisk n d eps ≤
min 1 (6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m))) ∧
(∀ (n d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
Tendsto (fun m => minimaxRisk n m d eps) atTop (nhds (knownMarginalRisk n d eps))) ∧
(∀ M0 : Real, 0 < M0 → ∃ kappa : Real, 0 < kappa ∧
∀ (n m d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
(n : Real) * eps ≤ M0 → kappa ≤ minimaxRisk n m d eps) := by
have hcomparison : ∀ (n m d : Nat) (eps : Real),
1 ≤ n → 1 ≤ m → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
0 ≤ minimaxRisk n m d eps - knownMarginalRisk n d eps ∧
minimaxRisk n m d eps - knownMarginalRisk n d eps ≤
min 1 (6 * ((n : Real) + 2) * sqrt (2 * (d : Real) / m)) := by
intro n m d eps hn hm hd heps heps'
obtain ⟨hlower, hcap⟩ := comparison_difference_bounds n m d eps hd heps heps'
refine ⟨hlower, le_min hcap ?_⟩
have hupper := comparison_minimax_le_known_add n m d eps hm hd heps heps'
linarith
refine ⟨?_, ?_, hcomparison, ?_, ?_⟩
· intro eps heps heps'
obtain ⟨B, hB, hcmp⟩ := benchmark_fixed_overlap_comparison eps heps heps'
refine ⟨1, B, zero_lt_one, hB, ?_⟩
intro n m d hn hd
simpa only [one_mul] using hcmp n m d hn
· obtain ⟨c, hc, hfloor⟩ := rare_label_floor.1
obtain ⟨CB, hCB, hbaseline⟩ := uniform_baseline
refine ⟨c, max (5 * CB) 2, hc,
lt_of_lt_of_le (by norm_num : (0 : Real) < 2) (le_max_right _ _), ?_⟩
intro n m d eps hn hd heps heps'
obtain ⟨_, _, hbase⟩ := hbaseline n m 2 eps hn (by omega) heps heps'
have hbinary : minimaxRisk n m 2 eps ≤ 5 * CB * labelBenchmark n eps := by
calc
minimaxRisk n m 2 eps ≤ CB *
min 1 (1 / ((n : Real) * eps) + (2 / (((n : Real) + m) * eps)) ^ 2) :=
benchmark_minimax_le_baseline n m 2 eps _ (by omega) heps heps' (by
simpa only [Nat.cast_ofNat] using hbase)
_ ≤ CB * (5 * labelBenchmark n eps) := mul_le_mul_of_nonneg_left
(benchmark_binary_baseline_rate n m eps hn heps) hCB.le
_ = 5 * CB * labelBenchmark n eps := by ring
have hbench : 0 ≤ labelBenchmark n eps := by
dsimp only [labelBenchmark, labelScale]
positivity
refine ⟨⟨(hfloor n m 2 eps hn (by omega) heps heps').1,
hbinary.trans (mul_le_mul_of_nonneg_right (le_max_left _ _) hbench)⟩,
(hfloor n m d eps hn hd heps heps').2, ?_⟩
exact (knownMarginalRisk_le_labelBenchmark n d eps hn hd heps heps').trans
(mul_le_mul_of_nonneg_right (le_max_right _ _) hbench)
· intro n d eps hn hd heps heps'
exact benchmark_limit_of_comparison n d eps
(fun m hm => hcomparison n m d eps hn hm hd heps heps')
· intro M0 hM0
obtain ⟨c, hc, hfloor⟩ := rare_label_floor.1
have hmax : 0 < max (1 : Real) M0 :=
lt_of_lt_of_le zero_lt_one (le_max_left _ _)
refine ⟨c * (1 / max 1 M0), mul_pos hc (one_div_pos.mpr hmax), ?_⟩
intro n m d eps hn hd heps heps' hscale
exact (mul_le_mul_of_nonneg_left
(benchmark_label_floor_on_bounded_scale n eps M0 hn heps hscale) hc.le).trans
(hfloor n m d eps hn hd heps heps').1
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.T_RareLabelFloor 1 declarations Rare-label lower bound uniform in auxiliary budget and exact-marginal side information.
Rare-label lower bound uniform in auxiliary budget and exact-marginal side information.
theorem rare_label_floor
Uniform rare-label floors, explicit legal common-marginal pair, and its exact KL.
Formal statement
conclusion
conclusion 1
∃ c : Real,
0 < c
c * labelBenchmark n eps ≤ minimaxRisk n m d eps
c * labelBenchmark n eps ≤ knownMarginalRisk n d eps
conclusion 2
ModelClass d eps (labelFloorLaw d eps delta sign hd hdelta)
ateFunctional (labelFloorLaw d eps delta sign hd hdelta)
= (if sign then delta else -delta)
j :
Fin d
j.val < 2
cellMass (labelFloorLaw d eps delta sign hd hdelta) j = 1 / 2
propensity (labelFloorLaw d eps delta sign hd hdelta) j = eps
= 1 / 2 + (if sign then delta else -delta)
j :
Fin d
2 ≤ j.val
cellMass (labelFloorLaw d eps delta sign hd hdelta) j = 0
conclusion 3
finiteKL (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth) (labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth)
= eps * log (5 / 3) / 4
Proof (Lean source)
theorem rare_label_floor :
(∃ c : Real, 0 < c ∧ -- @realizes c(positive numerical constant before all public indices)
∀ (n m d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
c * labelBenchmark n eps ≤ minimaxRisk n m d eps ∧
c * labelBenchmark n eps ≤ knownMarginalRisk n d eps) ∧
(∀ (d : Nat) (eps delta : Real) (sign : Bool) (hd : 2 ≤ d),
0 < eps → eps ≤ 1 / 4 → ∀ hdelta : LabelFloorAmplitude delta,
-- @realizes \delta(separation in (0,1/8])
ModelClass d eps (labelFloorLaw d eps delta sign hd hdelta) ∧
ateFunctional (labelFloorLaw d eps delta sign hd hdelta) = (if sign then delta else -delta) ∧
auxMarginal (labelFloorLaw d eps delta true hd hdelta) =
auxMarginal (labelFloorLaw d eps delta
false hd hdelta) ∧
(∀ j : Fin d, j.val < 2 →
cellMass (labelFloorLaw d eps delta sign hd hdelta) j = 1 / 2 ∧
propensity (labelFloorLaw d eps delta sign hd hdelta) j = eps ∧
outcomeMean (labelFloorLaw d eps delta sign hd hdelta) false j = 1 / 2 ∧
outcomeMean (labelFloorLaw d eps delta sign hd hdelta) true j =
1 / 2 + (if sign then delta else -delta)) ∧
(∀ j : Fin d, 2 ≤ j.val → cellMass (labelFloorLaw d eps delta sign hd hdelta) j = 0)) ∧
(∀ (d : Nat) (eps : Real) (hd : 2 ≤ d), 0 < eps → eps ≤ 1 / 4 →
finiteKL (labelFloorLaw d eps (1 / 8) true hd labelFloorAmplitude_eighth)
(labelFloorLaw d eps (1 / 8) false hd labelFloorAmplitude_eighth) =
eps * log (5 / 3) / 4) := by
refine ⟨?_, ?_, ?_⟩
· refine ⟨1 / 1024, by norm_num, ?_⟩
intro n m d eps hn hd heps heps'
let delta := 1 / (16 * sqrt (max 1 ((n : Real) * eps)))
let hdelta := labelFloorAmplitude_scaled ((n : Real) * eps)
let P : ClassLaw d eps := ⟨labelFloorLaw d eps delta true hd hdelta,
labelFloor_model d eps delta true hd hdelta heps heps'⟩
let Q : ClassLaw d eps := ⟨labelFloorLaw d eps delta false hd hdelta,
labelFloor_model d eps delta false hd hdelta heps heps'⟩
have htargetP : ateFunctional P.1 = delta := by
exact labelFloor_target d eps delta true hd hdelta heps heps'
have htargetQ : ateFunctional Q.1 = -delta := by
exact labelFloor_target d eps delta false hd hdelta heps heps'
have haux : auxMarginal P.1 = auxMarginal Q.1 :=
labelFloor_auxMarginal d eps delta hd hdelta heps heps'
have htable : auxTable P.1 = auxTable Q.1 := congrArg (fun p => fun z => (p z).toReal) haux
letI : IsProbabilityMeasure seedLaw := ⟨by norm_num [seedLaw, Real.volume_Icc]⟩
have hobs (R : DiscreteLaw d) : IsProbabilityMeasure (obsLaw R) := by
unfold obsLaw
infer_instance
letI (R : DiscreteLaw d) : IsProbabilityMeasure (obsLaw R) := hobs R
letI (R : DiscreteLaw d) : IsProbabilityMeasure (labeledProductLaw R n) := by
unfold labeledProductLaw
infer_instance
letI (R : DiscreteLaw d) : IsProbabilityMeasure (auxProductLaw R m) := by
unfold auxProductLaw
infer_instance
letI (R : DiscreteLaw d) : IsProbabilityMeasure (annotationLaw R n m) := by
unfold annotationLaw
infer_instance
have htv : tvDist (labeledProductLaw P.1 n) (labeledProductLaw Q.1 n) ≤ 1 / 4 :=
labelFloor_scaled_tv n d eps hd heps heps'
have hrate : delta ^ 2 / 4 = (1 / 1024 : Real) * labelBenchmark n eps :=
labelFloor_scaled_rate n eps hn heps
letI : Nonempty (Rule n m d) := ⟨⟨fun _ => 0, measurable_const, by intro z; norm_num⟩⟩
letI : Nonempty (KnownRule n d) := ⟨⟨fun _ => 0, measurable_const, by intro z; norm_num⟩⟩
constructor
· have htvData : tvDist
((annotationLaw P.1 n m).prod seedLaw) ((annotationLaw Q.1 n m).prod seedLaw) ≤ 1 / 4 := by
rw [labelFloor_tv_common_product]
unfold annotationLaw auxProductLaw
rw [← haux, labelFloor_tv_common_product]
exact htv
have hbdd (T : Rule n m d) : BddAbove (range (fun R : ClassLaw d eps => ruleRisk T.1 R.1)) := by
refine ⟨4, ?_⟩
rintro _ ⟨R, rfl⟩
exact labelFloor_bounded_mse _ T.1 T.2.2 _ (ateFunctional_mem_Icc R.1)
apply le_minimaxValue_of_two_point
(risk := fun (T : Rule n m d) (R : ClassLaw d eps) => ruleRisk T.1 R.1) P Q hbdd
intro T
have h := labelFloor_testing_mse
((annotationLaw P.1 n m).prod seedLaw) ((annotationLaw Q.1 n m).prod seedLaw)
delta hdelta.1.le htvData T.1 T.2.1 T.2.2
simpa only [ruleRisk, htargetP, htargetQ, hrate] using h
· have htvSeed : tvDist
((labeledProductLaw P.1 n).prod seedLaw) ((labeledProductLaw Q.1 n).prod seedLaw) ≤ 1 / 4 := by
rw [labelFloor_tv_common_product]
exact htv
have hbdd (T : KnownRule n d) : BddAbove (range (fun R : ClassLaw d eps => knownRuleRisk T.1 R.1)) := by
refine ⟨4, ?_⟩
rintro _ ⟨R, rfl⟩
exact labelFloor_bounded_mse _ (fun z : (Fin n → Obs d) × Real => T.1 (z.1, auxTable R.1, z.2))
(fun z => T.2.2 _) _ (ateFunctional_mem_Icc R.1)
apply le_minimaxValue_of_two_point
(risk := fun (T : KnownRule n d) (R : ClassLaw d eps) => knownRuleRisk T.1 R.1) P Q hbdd
intro T
have hmeas : Measurable (fun z : (Fin n → Obs d) × Real => T.1 (z.1, auxTable P.1, z.2)) := by
exact T.2.1.comp (measurable_fst.prodMk (measurable_const.prodMk measurable_snd))
have h := labelFloor_testing_mse
((labeledProductLaw P.1 n).prod seedLaw) ((labeledProductLaw Q.1 n).prod seedLaw)
delta hdelta.1.le htvSeed (fun z => T.1 (z.1, auxTable P.1, z.2)) hmeas (fun z => T.2.2 _)
simpa only [knownRuleRisk, htargetP, htargetQ, hrate, ← htable] using h
· intro d eps delta sign hd heps heps' hdelta
refine ⟨labelFloor_model d eps delta sign hd hdelta heps heps',
labelFloor_target d eps delta sign hd hdelta heps heps',
labelFloor_auxMarginal d eps delta hd hdelta heps heps', ?_, ?_⟩
· intro j hj
refine ⟨?_, labelFloor_propensity d eps delta sign hd hdelta heps heps' j hj, ?_, ?_⟩
· rw [labelFloor_cellMass d eps delta sign hd hdelta heps heps', if_pos hj]
· exact labelFloor_outcomeMean d eps delta sign hd hdelta heps heps' j hj false
· exact labelFloor_outcomeMean d eps delta sign hd hdelta heps heps' j hj true
· intro j hj
rw [labelFloor_cellMass d eps delta sign hd hdelta heps heps', if_neg (by omega)]
· exact labelFloor_KL_eighth
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.T_UniformAnnotationFrontier 1 declarations Uniform rare-overlap annotation frontier and existence and identification of causal extensions.
Uniform rare-overlap annotation frontier and existence and identification of causal extensions.
theorem uniform_annotation_frontier
Universal four-index minimax comparison, attaining rule, and compatible causal interpretation.
Formal statement
conclusion
conclusion 1
∃ c C : Real,
0 < c
c ≤ C
Measurable (hybridEstimator n m d eps)
s :
hybridEstimator n m d eps s ∈ Icc (-1) 1
c * frontierRate n m d eps ≤ minimaxRisk n m d eps
conclusion 2
∃ H : PotentialLaw d,
poContrast H = ateFunctional P
Proof (Lean source)
-- @node: thm:uniform-annotation-frontier
theorem uniform_annotation_frontier :
(∃ c C : Real, 0 < c ∧ c ≤ C ∧
-- @realizes c(positive lower constant independent of all public indices)
-- @realizes C(finite upper constant at least c independent of all public indices)
∀ (n m d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
Measurable (hybridEstimator n m d eps) ∧
(∀ s, hybridEstimator n m d eps s ∈ Icc (-1) 1) ∧
c * frontierRate n m d eps ≤ minimaxRisk n m d eps ∧
minimaxRisk n m d eps ≤ worstRisk (liftRule (hybridEstimator n m d eps)) eps ∧
worstRisk (liftRule (hybridEstimator n m d eps)) eps ≤ C * frontierRate n m d eps) ∧
(∀ (d : Nat) (eps : Real) (P : DiscreteLaw d),
2 ≤ d → 0 < eps → eps ≤ 1 / 4 → ModelClass d eps P →
(∃ H : PotentialLaw d, observedMarginal H = P ∧ Consistency H ∧ ConditionalExchangeability
H) ∧
∀ H : PotentialLaw d, observedMarginal H = P → Consistency H →
-- @realizes H(universal extension has observed marginal P and satisfies consistency)
ConditionalExchangeability H → poContrast H = ateFunctional P) := by
obtain ⟨clab, hclab, hlab⟩ := rare_label_floor.1
obtain ⟨caff, hcaff, haff⟩ := normalized_affine_testing
obtain ⟨Cup, hCup, hup⟩ := hybrid_upper
let E : Real := exp 4096
have hE : 1 ≤ E := Real.one_le_exp (by norm_num)
have hEpos : 0 < E := Real.exp_pos _
let c : Real := min (clab / (2 * E)) (caff / 2)
have hc : 0 < c := lt_min (div_pos hclab (by positivity)) (by positivity)
have hcl : c ≤ clab / (2 * E) := min_le_left _ _
have hca : c ≤ caff / 2 := min_le_right _ _
have hcl2 : 2 * c ≤ clab := by
have := (le_div_iff₀ (show 0 < 2 * E by positivity)).mp hcl
nlinarith
constructor
· refine ⟨c, max Cup c, hc, le_max_right _ _, ?_⟩
intro n m d eps hn hd heps heps'
obtain ⟨hmeas, hrange, hupper⟩ := hup n m d eps hn hd heps heps'
have hrate : 0 ≤ frontierRate n m d eps := (frontierRate_pos n m d eps hn heps).le
have hminimax : minimaxRisk n m d eps ≤
worstRisk (liftRule (hybridEstimator n m d eps)) eps := by
have hmeasLift : Measurable (liftRule (hybridEstimator n m d eps)) := by
exact hmeas.comp measurable_fst
let T : Rule n m d := ⟨liftRule (hybridEstimator n m d eps), hmeasLift,
fun z => hrange z.1⟩
exact minimaxValue_le_worstCaseRisk_of_nonneg
(risk := fun (T : Rule n m d) (P : ClassLaw d eps) => ruleRisk T.1 P.1)
(fun T P => integral_nonneg (fun z => sq_nonneg _)) T
refine ⟨hmeas, hrange, ?_, hminimax,
hupper.trans (mul_le_mul_of_nonneg_right (le_max_left _ _) hrate)⟩
have hlabel := (hlab n m d eps hn hd heps heps').1
let S : Real := (n : Real) * eps
let x : Real := (d : Real) / (((n : Real) + m) * eps * logScale n eps)
have hS : 0 < S := mul_pos (by exact_mod_cast hn) heps
have hinv : 0 < 1 / S := one_div_pos.mpr hS
have hrdef : frontierRate n m d eps = min 1 (1 / S + x ^ 2) := rfl
have hbdef : labelBenchmark n eps = min 1 (1 / S) := by
simp only [labelBenchmark, labelScale, one_div, S]
by_cases hsmall : S < E
· have hbinv : 1 / E ≤ 1 / S := one_div_le_one_div_of_le hS hsmall.le
have hb : 1 / E ≤ labelBenchmark n eps := by
rw [hbdef]
exact le_min ((div_le_one hEpos).mpr hE) hbinv
have hcE : c ≤ clab * (1 / E) := by
have := (le_div_iff₀ (show 0 < 2 * E by positivity)).mp hcl
have hdiv : (clab * (1 / E)) * E = clab := by field_simp
nlinarith
calc
c * frontierRate n m d eps ≤ c * 1 :=
mul_le_mul_of_nonneg_left (frontierRate_le_one n m d eps) hc.le
_ ≤ clab * (1 / E) := by simpa using hcE
_ ≤ clab * labelBenchmark n eps := mul_le_mul_of_nonneg_left hb hclab.le
_ ≤ minimaxRisk n m d eps := hlabel
· have hlarge : E ≤ S := le_of_not_gt hsmall
have hSone : 1 ≤ S := hE.trans hlarge
have hb : labelBenchmark n eps = 1 / S := by
rw [hbdef, min_eq_right ((div_le_one hS).mpr hSone)]
by_cases hx : x ^ 2 ≤ 1 / S
· have hr : frontierRate n m d eps ≤ 2 * (1 / S) := by
rw [hrdef]
exact (min_le_right _ _).trans (by linarith)
calc
c * frontierRate n m d eps ≤ c * (2 * (1 / S)) :=
mul_le_mul_of_nonneg_left hr hc.le
_ ≤ clab * (1 / S) := by nlinarith [mul_le_mul_of_nonneg_right hcl2 hinv.le]
_ ≤ minimaxRisk n m d eps := by simpa only [hb] using hlabel
· have ha := (haff n m d eps hn hd heps heps' hlarge (lt_of_not_ge hx)).2.2
change caff * min 1 (x ^ 2) ≤ minimaxRisk n m d eps at ha
have hr : frontierRate n m d eps ≤ 2 * min 1 (x ^ 2) := by
rw [hrdef]
by_cases hxone : x ^ 2 ≤ 1
· rw [min_eq_right hxone]
exact (min_le_right _ _).trans (by linarith [lt_of_not_ge hx])
· rw [min_eq_left (le_of_not_ge hxone)]
exact (min_le_left _ _).trans (by norm_num)
have hxmin : 0 ≤ min 1 (x ^ 2) := le_min (by norm_num) (sq_nonneg _)
have hca2 : 2 * c ≤ caff := by linarith
calc
c * frontierRate n m d eps ≤ c * (2 * min 1 (x ^ 2)) :=
mul_le_mul_of_nonneg_left hr hc.le
_ ≤ caff * min 1 (x ^ 2) := by
nlinarith [mul_le_mul_of_nonneg_right hca2 hxmin]
_ ≤ minimaxRisk n m d eps := ha
· intro d eps P hd heps heps' hP
obtain ⟨hwitness, hidentify⟩ := causal_realization d eps P hd heps heps' hP
exact ⟨⟨independentExtension P, hwitness⟩, hidentify⟩
CausalSmith.Stat.STAT_AnnotationRarearmFrontier_Research.T_UniformBaseline 4 declarations Uniform original-record inverse-count baseline theorem.
Uniform original-record inverse-count baseline theorem.
theorem baseline_small_sample_risk
Under the stated inputs and conditions, The prescribed zero branch has squared risk at most one. This gives the stated result.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: baseline_small_sample_risk
lemma baseline_small_sample_risk (n m d : Nat) (P : DiscreteLaw d)
(hsmall : n < 6) :
ruleRisk (liftRule (baselineEstimator n m d)) P ≤ 1 := by
let : IsProbabilityMeasure (obsLaw P) := by
unfold obsLaw
infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by
norm_num [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have ht := ateFunctional_mem_Icc P
have hrisk : ruleRisk (liftRule (baselineEstimator n m d)) P =
(ateFunctional P) ^ 2 := by
simp [ruleRisk, liftRule, baselineEstimator, hsmall, integral_const,
Measure.real_def]
rw [hrisk]
nlinarith [ht.1, ht.2]
theorem baseline_risk_le_four
Under the stated inputs and conditions, Clipping both the baseline and the target bounds squared loss by four. This gives the stated result.
Formal statement
hypotheses
n m d :
P :
conclusion
Proof (Lean source)
-- @node: baseline_risk_le_four
lemma baseline_risk_le_four (n m d : Nat) (P : DiscreteLaw d) :
ruleRisk (liftRule (baselineEstimator n m d)) P ≤ 4 := by
let : IsProbabilityMeasure (obsLaw P) := by
unfold obsLaw
infer_instance
let : IsProbabilityMeasure seedLaw := ⟨by
norm_num [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure (annotationLaw P n m) := by
unfold annotationLaw labeledProductLaw auxProductLaw
infer_instance
have ht := ateFunctional_mem_Icc P
have hpoint (z : Sample n m d × Real) :
(liftRule (baselineEstimator n m d) z - ateFunctional P) ^ 2 ≤ 4 := by
have hb := baselineEstimator_mem_Icc n m d z.1
dsimp [liftRule]
nlinarith [hb.1, hb.2, ht.1, ht.2]
unfold ruleRisk
calc
_ ≤ ∫ _z : Sample n m d × Real, (4 : Real)
∂((annotationLaw P n m).prod seedLaw) :=
integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
(integrable_const 4) (Filter.Eventually.of_forall hpoint)
_ = 4 := by simp
Under the stated inputs and conditions, For fewer than six labeled records, the capped baseline rate is at least four fifths. This gives the stated result.
Formal statement
Proof (Lean source)
-- @node: baseline_small_sample_rate
lemma baseline_small_sample_rate (n m d : Nat) (eps : Real)
(hn : 1 ≤ n) (hsmall : n < 6) (heps : 0 < eps) (heps' : eps ≤ 1 / 4) :
(4 / 5 : Real) ≤
min 1 (1 / ((n : Real) * eps) + ((d : Real) / (((n : Real) + m) * eps)) ^ 2) := by
have hn0 : (0 : Real) < n := by exact_mod_cast (show 0 < n by omega)
have hn5 : (n : Real) ≤ 5 := by exact_mod_cast (show n ≤ 5 by omega)
have hS : 0 < (n : Real) * eps := mul_pos hn0 heps
have hSupper : (n : Real) * eps ≤ 5 / 4 := by nlinarith
have hinv : (4 / 5 : Real) ≤ 1 / ((n : Real) * eps) := by
apply (le_div_iff₀ hS).2
nlinarith
exact le_min (by norm_num) (hinv.trans (le_add_of_nonneg_right (sq_nonneg _)))
theorem uniform_baseline
Under the stated inputs and conditions, The total inverse-count baseline has a universal overlap-explicit risk guarantee. This gives the stated conclusion.
Formal statement
conclusion
∃ C : Real,
0 < C
conclusion 1
Measurable (baselineEstimator n m d)
s :
baselineEstimator n m d s ∈ Icc (-1) 1
Proof (Lean source)
theorem uniform_baseline :
∃ C : Real, 0 < C ∧ -- @realizes C(positive universal bound constant)
∀ (n m d : Nat) (eps : Real), 1 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
Measurable (baselineEstimator n m d) ∧
(∀ s, baselineEstimator n m d s ∈ Icc (-1) 1) ∧
∀ P : ClassLaw d eps, ruleRisk (liftRule (baselineEstimator n m d)) P.1 ≤
C * min 1 (1 / ((n : Real) * eps) + ((d : Real) / (((n : Real) + m) * eps)) ^ 2) :=
by
have hlarge : ∃ C : Real, 0 < C ∧
∀ (n m d : Nat) (eps : Real), 6 ≤ n → 2 ≤ d → 0 < eps → eps ≤ 1 / 4 →
∀ P : ClassLaw d eps, ruleRisk (liftRule (baselineEstimator n m d)) P.1 ≤
C * (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps)) ^ 2) := by
obtain ⟨C, hC, hpoisson⟩ := baseline_ordered_prefix_risk
refine ⟨4 * (40 * C + 1026), by positivity, ?_⟩
intro n m d eps hn hd heps heps' P
have hpool : ruleRisk (liftRule (baselineEstimator n m d)) P.1 ≤
C * (1 / (((n / 2 : Nat) : Real) / 8 * eps) +
1 / (((n - n / 2 + m : Nat) : Real) / 8 * eps)) +
4 * ((d : Real) /
(exp 1 * ((n - n / 2 + m : Nat) : Real) / 8 * eps)) ^ 2 +
4 * (exp (-(n / 2 : Nat)) + exp (-(n - n / 2 + m : Nat))) := by
rw [baseline_ruleRisk_eq_fixed_average P.1 hn]
have hu : (0 : NNReal) < (n / 2 : Nat) / 8 := by
have hh : 0 < n / 2 := by omega
positivity
have ht : (0 : NNReal) < (n - n / 2 + m : Nat) / 8 := by
have hg : 0 < n - n / 2 + m := by omega
positivity
have htransfer := sqRisk_independentPrefixRaoBlackwellStatistic_le
(obsLaw P.1) (auxMarginal P.1).toMeasure
((n / 2 : Nat) / 8 : NNReal) ((n - n / 2 + m : Nat) / 8 : NNReal)
(n / 2) (n - n / 2 + m)
(baselineOrderedPrefixStatistic_measurable d ((n / 2 : Nat) / 8 : NNReal))
(by norm_num : (-1 : Real) ≤ 1)
(baselineOrderedPrefixStatistic_mem_Icc ((n / 2 : Nat) / 8 : NNReal))
(ateFunctional_mem_Icc P.1)
(by norm_num : (0 : Real) ∈ Icc (-1) 1)
have hprefix := hpoisson d eps P.1 _ _ hd heps heps' P.2 hu ht
have htail := add_le_add (eighth_pool_poisson_overflow (n / 2))
(eighth_pool_poisson_overflow (n - n / 2 + m))
norm_num only at htransfer
simpa only [NNReal.coe_div, NNReal.coe_natCast, NNReal.coe_ofNat, mul_div_assoc] using
htransfer.trans (add_le_add hprefix
(mul_le_mul_of_nonneg_left htail (by norm_num)))
have hrate := baseline_pool_rate_bound n m d eps C hn heps heps' hC
have hnonneg : 0 ≤ C * (1 / (((n / 2 : Nat) : Real) / 8 * eps) +
1 / (((n - n / 2 + m : Nat) : Real) / 8 * eps)) +
4 * ((d : Real) /
(exp 1 * ((n - n / 2 + m : Nat) : Real) / 8 * eps)) ^ 2 := by
positivity
nlinarith
obtain ⟨C, hC, hbound⟩ := hlarge
refine ⟨max C 4, lt_of_lt_of_le hC (le_max_left _ _), ?_⟩
intro n m d eps hn hd heps heps'
refine ⟨baselineEstimator_measurable n m d, baselineEstimator_mem_Icc n m d, ?_⟩
intro P
have hrate : 0 ≤ min 1 (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps)) ^ 2) := by positivity
by_cases hsmall : n < 6
· have hlow := baseline_small_sample_rate n m d eps hn hsmall heps heps'
calc
ruleRisk (liftRule (baselineEstimator n m d)) P.1 ≤ 1 :=
baseline_small_sample_risk n m d P.1 hsmall
_ ≤ 2 * min 1 (1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps)) ^ 2) := by linarith
_ ≤ _ := mul_le_mul_of_nonneg_right
((by norm_num : (2 : Real) ≤ 4).trans (le_max_right _ _)) hrate
· by_cases hcap : 1 ≤ 1 / ((n : Real) * eps) +
((d : Real) / (((n : Real) + m) * eps)) ^ 2
· rw [min_eq_left hcap, mul_one]
exact (baseline_risk_le_four n m d P.1).trans (le_max_right _ _)
· rw [min_eq_right (le_of_not_ge hcap)]
exact (hbound n m d eps (by omega) hd heps heps' P).trans
(mul_le_mul_of_nonneg_right (le_max_left _ _) (by positivity))