Formalization: Minimax Precision for Total Treatment Effects with Partially Observed Interference Networks
The complete Lean development behind this paper — every definition, lemma, and theorem of its module, including helpers the paper text never cites. Identifiers link within this page, into the Causalean library, or out to the official Mathlib docs.
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Basic 59 declarations Raw directed coefficients are needed for the schedule class and the hidden-allocation prior.
Fixed additive schedules and the complete thinned record
Raw directed coefficients are needed for the schedule class and the hidden-allocation prior. The exposure-based potential-outcome substrate and superpopulation PO systems are therefore bypassed. The out-degree predicate and extended minimax carrier reuse Causalean directly. The design is Measure-valued to permit continuous baseline mixtures and an independent seed; the finite Bernoulli design is used through the expectation bridge in Helpers/DesignBridge.
def PopulationCarrier
The paper population has n units; the paper label of i is i.val + 1.
Definition (Lean source)
A fixed simple directed off-diagonal graph with additive coefficients.
Definition (Lean source)
parameters
V :
Type*
fields
edge :
V → V → Prop
decEdge :
DecidableRel edge
irrefl :
∀ i, ¬ edge i i
a :
V → ℝ
t :
V → ℝ
b :
V → V → ℝ
def inNbhd
The off-diagonal treatment sources of a recipient.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
i :
V
def
given by
Finset.univ.filter (fun j => θ.edge j i)
def treatment
The zero-one numeric value of a treatment coordinate.
Definition (Lean source)
def signOf
The centered sign, twice treatment minus one.
Definition (Lean source)
def potentialOutcome
The additive potential outcome.
def tte
The all-treated minus all-control population average.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
def
tte θ :
ℝ
given by
(Fintype.card V : ℝ)⁻¹ * ∑ i, (potentialOutcome θ i (fun _ => true) - potentialOutcome θ i (fun _ => false))
def InDegreeLE
Each recipient has at most d off-diagonal sources.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
d :
ℕ
i :
def
InDegreeLE θ d :
Prop
given by
(inNbhd θ i).card ≤ d
def CoefficientMassLE
The row coefficient mass includes the baseline and own coordinate.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
i :
def
CoefficientMassLE θ :
Prop
given by
|θ.a i| + |θ.t i| + ∑ j ∈ inNbhd θ i, |θ.b i j| ≤ 1
structure ScheduleClass
The additive fixed-schedule model class.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
d :
ℕ
assumes
inDegree :
InDegreeLE θ d
outDegree :
BlockDegreeLE (inNbhd θ) d
The imported predicate is the sole realization of the off-diagonal out-degree atom.
coeffMass :
theorem mono
Enlarging the degree bound preserves schedule membership. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
ScheduleClass θ d'
Proof (Lean source)
-- @node: ScheduleClass.mono
lemma ScheduleClass.mono {θ : Schedule V} {d d' : ℕ}
(h : ScheduleClass θ d) (hdd' : d ≤ d') : ScheduleClass θ d' := by
exact ⟨fun i => (h.inDegree i).trans hdd',
fun j => (h.outDegree j).trans hdd', h.coeffMass⟩
theorem abs_potentialOutcome_le_one
The coefficient-mass envelope bounds every potential outcome by one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
|potentialOutcome θ i z| ≤ 1
Proof (Lean source)
-- @node: abs_potentialOutcome_le_one
lemma abs_potentialOutcome_le_one {θ : Schedule V} {d : ℕ}
(h : ScheduleClass θ d) (i : V) (z : V → Bool) :
|potentialOutcome θ i z| ≤ 1 := by
have hterm (c : ℝ) (b : Bool) : |c * treatment b| ≤ |c| := by
cases b <;> simp [treatment]
calc
|potentialOutcome θ i z| ≤ |θ.a i| + |θ.t i * treatment (z i)| +
|∑ j ∈ inNbhd θ i, θ.b i j * treatment (z j)| :=
(abs_add_le _ _).trans (add_le_add (abs_add_le _ _) le_rfl)
_ ≤ |θ.a i| + |θ.t i| + ∑ j ∈ inNbhd θ i, |θ.b i j| := by
apply add_le_add
· exact add_le_add le_rfl (hterm _ _)
· exact (Finset.abs_sum_le_sum_abs _ _).trans
(Finset.sum_le_sum fun j _ => hterm _ _)
_ ≤ 1 := h.coeffMass i
theorem abs_tte_le_one
The coefficient-mass envelope bounds the total treatment effect by one. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: abs_tte_le_one
lemma abs_tte_le_one {θ : Schedule V} {d : ℕ}
(h : ScheduleClass θ d) : |tte θ| ≤ 1 := by
have hcontrast (i : V) :
|potentialOutcome θ i (fun _ => true) -
potentialOutcome θ i (fun _ => false)| ≤ 1 := by
have hmass := h.coeffMass i
have hsum := Finset.abs_sum_le_sum_abs (fun j => θ.b i j) (inNbhd θ i)
simp only [potentialOutcome, treatment, Bool.false_eq_true, ite_false, ite_true,
mul_one, mul_zero, Finset.sum_const_zero, add_zero] at ⊢
have heq : θ.a i + θ.t i + ∑ j ∈ inNbhd θ i, θ.b i j - θ.a i =
θ.t i + ∑ j ∈ inNbhd θ i, θ.b i j := by ring
rw [heq]
calc
|θ.t i + ∑ j ∈ inNbhd θ i, θ.b i j| ≤
|θ.t i| + |∑ j ∈ inNbhd θ i, θ.b i j| := abs_add_le _ _
_ ≤ |θ.t i| + ∑ j ∈ inNbhd θ i, |θ.b i j| := add_le_add le_rfl hsum
_ ≤ 1 := by linarith [abs_nonneg (θ.a i)]
calc
|tte θ| = (Fintype.card V : ℝ)⁻¹ *
|∑ i, (potentialOutcome θ i (fun _ => true) -
potentialOutcome θ i (fun _ => false))| := by
rw [tte, abs_mul, abs_of_nonneg (by positivity : 0 ≤ (Fintype.card V : ℝ)⁻¹)]
_ ≤ (Fintype.card V : ℝ)⁻¹ * ∑ _i : V, (1 : ℝ) := by
apply mul_le_mul_of_nonneg_left _ (by positivity)
exact (Finset.abs_sum_le_sum_abs _ _).trans
(Finset.sum_le_sum fun i _ => hcontrast i)
_ ≤ 1 := by
simp only [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, mul_one]
by_cases hn : (Fintype.card V : ℝ) = 0
· simp [hn]
· rw [inv_mul_cancel₀ hn]
def OffDiag
Ordered off-diagonal source-recipient labels.
Definition (Lean source)
Complete treatment assignment.
Definition (Lean source)
Latent independent audit marks, including marks on nonedges.
def Record
The full graph, treatment, and noiseless outcome record.
def bernoulliLaw
A two-atom Bernoulli measure, total also outside the probability regime.
def halfBernoulli
Independent half-Bernoulli assignments.
def auditLaw
Independent edge audit with retention q; endpoints are point masses.
The canonical independent assignment-audit product design.
def ProbabilityDesign
Probability-design scope for the extended minimax risk.
Definition (Lean source)
parameters
def
ProbabilityDesign D :
Prop
given by
The finite Bernoulli marginal is a probability measure at admissible retention. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: bernoulliLaw_probability
lemma bernoulliLaw_probability (p : ℝ) (hp : p ∈ Icc 0 1) :
IsProbabilityMeasure (bernoulliLaw p) := by
simpa only [bernoulliLaw, bernoulliBool, add_comm] using
bernoulliBool_isProbabilityMeasure hp.1 hp.2
theorem thinnedDesign_probabilityDesign
Admissible retention makes the canonical experiment a probability design. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: thinnedDesign_probabilityDesign
lemma thinnedDesign_probabilityDesign (q : ℝ) (hq : q ∈ Icc 0 1) :
ProbabilityDesign (thinnedDesign V q) := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (bernoulliLaw q) := bernoulliLaw_probability q hq
unfold ProbabilityDesign thinnedDesign halfBernoulli auditLaw
infer_instance
def AssignmentLaw
The assignment marginal is the product half-Bernoulli law.
Definition (Lean source)
def AuditLaw
The audit marginal is the independent product law with admissible retention.
def DesignIndependent
Assignment and audit coordinates are independent.
def recordOf
The original record retains precisely the audited true arrows.
Definition (Lean source)
parameters
def
given by
(fun e => decide (θ.edge e.1.1 e.1.2) && ω.2 e, ω.1, fun i => potentialOutcome θ i ω.1)
def seedLaw
An independent uniform seed on the unit interval.
def recordLaw
The fixed-schedule seeded original-record law.
All measurable real-valued randomized estimators of the complete record.
Definition (Lean source)
def sqLoss
Extended nonnegative squared loss avoids nonintegrable-real-integral junk values.
Worst-case risk over the fixed schedule class.
Definition (Lean source)
parameters
def
worstRisk D d T :
ℝ≥0∞
given by
worstCaseRiskENNReal (fun (T : Estimator V) (θ : {θ : Schedule V // ScheduleClass θ d}) => sqLoss D θ.1 T) T
def minimaxRisk
The unrestricted minimax risk, built on the substrate extended minimax value.
Definition (Lean source)
parameters
def
minimaxRisk D d :
ℝ≥0∞
given by
minimaxValueENNReal (fun (T : Estimator V) (θ : {θ : Schedule V // ScheduleClass θ d}) => sqLoss D θ.1 T)
def SuppliedRecord
The complete original record together with the true off-diagonal graph.
def SuppliedEstimator
All measurable randomized estimators when the true graph is supplied.
Definition (Lean source)
def suppliedGraphOf
The true off-diagonal graph as a labeled Boolean array.
def suppliedRecordOf
The original record and the supplied true graph under one design realization.
Definition (Lean source)
def suppliedRecordLaw
The fixed-schedule law of the supplied-graph record and an independent seed.
Definition (Lean source)
parameters
def
given by
def suppliedSqLoss
Extended squared risk of an estimator that observes the true graph.
Definition (Lean source)
parameters
def
suppliedSqLoss D θ T :
ℝ≥0∞
given by
def suppliedWorstRisk
Worst-case supplied-graph risk over the unchanged fixed schedule class.
Definition (Lean source)
parameters
def
suppliedWorstRisk D d T :
ℝ≥0∞
given by
worstCaseRiskENNReal (fun (T : SuppliedEstimator V) (θ : {θ : Schedule V // ScheduleClass θ d}) => suppliedSqLoss D θ.1 T) T
def suppliedMinimaxRisk
Minimax squared risk when the complete original record and true graph are supplied.
Definition (Lean source)
parameters
def
suppliedMinimaxRisk D d :
ℝ≥0∞
given by
minimaxValueENNReal (fun (T : SuppliedEstimator V) (θ : {θ : Schedule V // ScheduleClass θ d}) => suppliedSqLoss D θ.1 T)
theorem minimaxRisk_le_one
The zero estimator bounds minimax risk by one on any probability design. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
minimaxRisk D d ≤ 1
Proof (Lean source)
-- @node: minimaxRisk_le_one
lemma minimaxRisk_le_one (D : Measure (Assign V × Audit V)) (hD : ProbabilityDesign D)
(d : ℕ) : minimaxRisk D d ≤ 1 := by
let : IsProbabilityMeasure D := hD
let : IsProbabilityMeasure seedLaw := ⟨by simp [seedLaw, Real.volume_Icc]⟩
let T : Estimator V := ⟨fun _ => 0, measurable_const⟩
apply (minimaxValueENNReal_le_worstCaseRisk T).trans
apply Causalean.Stat.worstCaseRiskENNReal_le
intro θ
have hm : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(recordOf θ.1 ω.1, ω.2)) := by
have hr : Measurable (recordOf θ.1) := by
fun_prop (disch := exact Measurable.of_discrete)
exact (hr.comp measurable_fst).prodMk measurable_snd
let : IsProbabilityMeasure (recordLaw D θ.1) :=
(D.prod seedLaw).isProbabilityMeasure_map hm.aemeasurable
have ht : (tte θ.1) ^ 2 ≤ 1 :=
(sq_le_one_iff_abs_le_one _).mpr (abs_tte_le_one θ.2)
unfold sqLoss
calc
(∫⁻ x, ofReal ((T.1 x - tte θ.1) ^ 2) ∂recordLaw D θ.1) ≤
∫⁻ _x, (1 : ℝ≥0∞) ∂recordLaw D θ.1 := by
apply lintegral_mono
intro x
simpa [T] using (ENNReal.ofReal_le_ofReal ht)
_ = 1 := by simp
def RG
The paper's supplied-graph minimax risk at admissible population, degree and retention.
Definition (Lean source)
parameters
def
RG n d q _hn _hd _hdu _hq :
ℝ≥0∞
given by
Canonical-design minimax risk.
Definition (Lean source)
def recordedNbhd
The recorded sources of a recipient, read solely from the observed record.
The observable inverse-edge-inclusion additive score.
Definition (Lean source)
parameters
def
auditScore q o :
ℝ
given by
(2 / (Fintype.card V : ℝ)) * ∑ i, o.2.2 i * (signOf (o.2.1 i) + q⁻¹ * ∑ j ∈ recordedNbhd o.1 i, signOf (o.2.1 j))
theorem auditScore_measurable
The finite observed graph and assignments give measurable score weights, with the outcome coordinates entering by a finite linear combination.
Formal statement
Proof (Lean source)
-- @node: auditScore_measurable
@[fun_prop] lemma auditScore_measurable (q : ℝ) :
Measurable (auditScore (V := V) q) := by
have hw (i : V) : Measurable (fun o : Record V =>
signOf (o.2.1 i) + q⁻¹ * ∑ j ∈ recordedNbhd o.1 i, signOf (o.2.1 j)) := by
exact (Measurable.of_discrete (f := fun hz : (OffDiag V → Bool) × Assign V =>
signOf (hz.2 i) + q⁻¹ * ∑ j ∈ recordedNbhd hz.1 i, signOf (hz.2 j))).comp
(measurable_fst.prodMk (measurable_fst.comp measurable_snd))
unfold auditScore
exact measurable_const.mul (Finset.measurable_sum _ fun i _ =>
(measurable_snd.comp measurable_snd).eval.mul (hw i))
def oracleScore
The additive supplied-graph score.
Definition (Lean source)
parameters
def
oracleScore θ z :
ℝ
given by
(2 / (Fintype.card V : ℝ)) * ∑ i, potentialOutcome θ i z * (signOf (z i) + ∑ j ∈ inNbhd θ i, signOf (z j))
The risk envelope is infinite at zero retention.
Definition (Lean source)
parameters
n d :
ℕ
q :
ℝ
def
upperEnvelope n d q :
ℝ≥0∞
given by
if 0 < q then ofReal (5 * ((d : ℝ) + 1) ^ 2 / n + 4 * d * (1 - q) / (n * q)) else ⊤
The total clipped-or-zero observable rule.
Definition (Lean source)
parameters
def
upperRule n d q o :
ℝ
given by
theorem upperRule_measurable
The clipped-or-zero rule is measurable on the entire record space.
Formal statement
Proof (Lean source)
-- @node: upperRule_measurable
@[fun_prop]
lemma upperRule_measurable (n d : ℕ) (q : ℝ) :
Measurable (upperRule (V := V) n d q) := by
unfold upperRule
split_ifs <;> fun_prop
theorem upperRule_seeded_measurable
Ignoring the independent seed preserves measurability of the upper rule.
Formal statement
hypotheses
V :
Type u_1
n d :
ℕ
q :
ℝ
conclusion
Proof (Lean source)
-- @node: upperRule_seeded_measurable
@[fun_prop]
lemma upperRule_seeded_measurable (n d : ℕ) (q : ℝ) :
Measurable (fun x : Record V × ℝ => upperRule n d q x.1) := by
fun_prop
The measurable seeded representative of the total upper rule.
Definition (Lean source)
parameters
V :
Type u_1
n d :
ℕ
q :
ℝ
def
given by
⟨fun x => upperRule n d q x.1, upperRule_seeded_measurable n d q⟩
def frontierScale
The explicit positive finite precision scale on the admissible regime.
Definition (Lean source)
parameters
n d :
ℕ
q :
ℝ
def
frontierScale n d q :
ℝ
given by
if 0 < q then min 1 ((d : ℝ) ^ 2 / (n * (1 - (1 - q) ^ d))) else 1
The attaining observable rule is the clipped-or-zero rule.
The seeded measurable representative of the attaining frontier rule.
Definition (Lean source)
def frontierHandle
Names the scale-rule pair without inserting any proved risk properties.
Definition (Lean source)
Admissible degree and retention sequences, with only the n ≥ 4 tail constrained.
Definition (Lean source)
parameters
dseq :
ℕ → ℕ
qseq :
ℕ → ℝ
n :
4 ≤ n
def
AdmissibleSequences dseq qseq :
Prop
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ActiveCapacityBridge 3 declarations An exact reindexing removes the unused slots outside a response support before applying the active-slot contraction bound.
Masking inactive capacities in the actual conditional coefficient
An exact reindexing removes the unused slots outside a response support before applying the active-slot contraction bound.
Only rows belonging to the response support retain hidden slots.
theorem allocationResponseSupport_degree_zero
Inactive rows have zero hidden degree in every compatible allocation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
t ℓ = 0
Proof (Lean source)
-- @node: allocationResponseSupport_degree_zero
lemma allocationResponseSupport_degree_zero (B : ℕ) (j t : Fin B → ℕ)
(T : Finset (Fin B)) (ht : allocationResponseSupport B j t = T)
(ℓ : Fin B) (hℓ : ℓ ∉ T) : t ℓ = 0 := by
have hn : j ℓ + t ℓ = 0 := by
have : ℓ ∉ allocationResponseSupport B j t := ht ▸ hℓ
simpa [allocationResponseSupport] using this
omega
theorem allocationSupportCoeff_activeCapacity
Masking inactive capacities preserves the full labeled support coefficient: its allocations, total degrees, binomial weights and response factors agree. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
allocationSupportCoeff B d h u j k T y
= allocationSupportCoeff B d h (activeCapacity B u T) j k T y
Proof (Lean source)
-- @node: allocationSupportCoeff_activeCapacity
lemma allocationSupportCoeff_activeCapacity (B d : ℕ) (h : ℝ)
(u j : Fin B → ℕ) (k : ℕ) (T : Finset (Fin B)) (y : Fin B → ℝ) :
allocationSupportCoeff B d h u j k T y =
allocationSupportCoeff B d h (activeCapacity B u T) j k T y := by
unfold allocationSupportCoeff
let S := Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) =>
k = ∑ ℓ, (t ℓ).val ∧ allocationResponseSupport B j (fun ℓ => (t ℓ).val) = T)
have hz (t : ∀ ℓ, Fin (u ℓ + 1)) (ht : t ∈ S) (ℓ : Fin B) (hℓ : ℓ ∉ T) :
(t ℓ).val = 0 :=
allocationResponseSupport_degree_zero B j _ T (Finset.mem_filter.mp ht).2.2 ℓ hℓ
let shrink (t : ∀ ℓ, Fin (u ℓ + 1)) (ht : t ∈ S) :
∀ ℓ, Fin (activeCapacity B u T ℓ + 1) := fun ℓ =>
⟨(t ℓ).val, by
by_cases hℓ : ℓ ∈ T
· simpa [activeCapacity, hℓ] using (t ℓ).isLt
· simp [activeCapacity, hℓ, hz t ht ℓ hℓ]⟩
apply Finset.sum_bij shrink
· intro t ht
simpa only [mem_filter, Finset.mem_univ, true_and, shrink] using
(Finset.mem_filter.mp ht).2
· intro t ht v hv he
funext ℓ
apply Fin.ext
exact congrArg (fun f => (f ℓ).val) he
· intro v hv
let t : ∀ ℓ, Fin (u ℓ + 1) := fun ℓ => ⟨(v ℓ).val, by
have := (v ℓ).isLt
have hm : activeCapacity B u T ℓ ≤ u ℓ := by
unfold activeCapacity
split_ifs <;> omega
omega⟩
have ht : t ∈ S := by
simpa only [S, mem_filter, Finset.mem_univ, true_and, t] using
(Finset.mem_filter.mp hv).2
refine ⟨t, ht, ?_⟩
funext ℓ
apply Fin.ext
rfl
· intro t ht
congr 1
apply Finset.prod_congr rfl
intro ℓ _
by_cases hℓ : ℓ ∈ T
· simp [activeCapacity, hℓ, shrink]
· simp [activeCapacity, hℓ, shrink, hz t ht ℓ hℓ]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ActiveLabelEnergyBound 6 declarations Sum the masked allocation bound over disjoint hidden-only response supports, then retain the actual revealed-label indexing and remove its constant term.
Active-support bound for the original labeled coefficient energy
Sum the masked allocation bound over disjoint hidden-only response supports, then retain the actual revealed-label indexing and remove its constant term.
The rowwise energy envelope for one exact revealed-label degree vector.
Definition (Lean source)
parameters
def
activeSupportEnergyBound B d h u j :
ℝ
given by
A :
Finset.univ.filter (fun ℓ => j ℓ ≠ 0)
∑ E ∈ (univ \ A).powerset, (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ∏ ℓ, ∑ t : Fin (activeCapacity B u (A ∪ E) ℓ + 1), if (if j ℓ ≠ 0 then True else if ℓ ∈ A ∪ E then 0 < t.val else t.val = 0) then ((activeCapacity B u (A ∪ E) ℓ).choose t.val : ℝ) * gamma d h (j ℓ + t.val) else 0
theorem allocationSupportCoeff_total_energy_le
The full hidden-degree energy for a fixed revealed subset is bounded by its disjoint active-support envelope, using the original total hidden capacity. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d M :
ℕ
h :
ℝ
u j :
Fin B → ℕ
hB :
0 < B
hd :
0 < d
hu :
∀ ℓ, u ℓ ≤ d
hM :
(∑ ℓ, u ℓ) ≤ M
hgood :
(B * d : ℕ) / (4 : ℝ) ≤ M
conclusion
(∑ k ∈ range (M + 1), (∑ T : Finset (Fin B), ∫ y : Fin B → ℝ, allocationSupportCoeff B d h u j k T y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))) / (M.choose k : ℝ))
≤ activeSupportEnergyBound B d h u j
Proof (Lean source)
-- @node: allocationSupportCoeff_total_energy_le
lemma allocationSupportCoeff_total_energy_le (B d M : ℕ) (h : ℝ)
(u j : Fin B → ℕ) (hB : 0 < B) (hd : 0 < d)
(hu : ∀ ℓ, u ℓ ≤ d) (hM : (∑ ℓ, u ℓ) ≤ M)
(hgood : (B * d : ℕ) / (4 : ℝ) ≤ M) :
(∑ k ∈ range (M + 1),
(∑ T : Finset (Fin B), ∫ y : Fin B → ℝ,
allocationSupportCoeff B d h u j k T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) / (M.choose k : ℝ)) ≤
activeSupportEnergyBound B d h u j := by
classical
simp_rw [allocationSupportCoeff_disjoint_energy_sum, Finset.sum_div]
rw [Finset.sum_comm]
unfold activeSupportEnergyBound
apply Finset.sum_le_sum
intro E hE
let A := Finset.univ.filter (fun ℓ => j ℓ ≠ 0)
have hEA : Disjoint A E := by
apply Finset.disjoint_left.mpr
intro ℓ hA hℓ
exact (Finset.mem_sdiff.mp (Finset.mem_powerset.mp hE hℓ)).2 hA
have hs : (A ∪ E) \ A = E := Finset.union_sdiff_cancel_left hEA
have hb := allocationSupportCoeff_summed_energy_le B d M h u j
(A ∪ E) hB hd hu hM hgood
(fun ℓ hj => mem_union_left E (Finset.mem_filter.mpr ⟨Finset.mem_univ _, hj⟩))
dsimp only [A] at hs hb
rw [hs] at hb
exact hb
theorem actualLabelWalshCoeff_constant_integral
The single removed coefficient has unit reference energy. This identifies its contribution before the full labeled sum is bounded. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H (∅, 0) y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= 1
Proof (Lean source)
-- @node: actualLabelWalshCoeff_constant_integral
lemma actualLabelWalshCoeff_constant_integral (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) :
(∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H (∅, 0) y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) = 1 := by
let := referenceRowLaw_probability d h
calc
_ = ∫ _ : Fin B → ℝ, (1 : ℝ)
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
apply integral_congr_ae
filter_upwards [referenceRows_positive_ae B d h] with y hy
simp [actualLabelWalshCoeff, groupedHiddenWalshCoeff_empty_zero_pos n B d h H y hy]
_ = 1 := by simp
theorem actualLabelEnergy_add_one
Restoring exactly the constant coefficient gives the complete original-label energy sum, with all hidden-degree denominators unchanged. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
actualLabelEnergy n B d h H + 1
= ∑ i : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) × Fin (undiscovered n B d H + 1), (∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))) / (choose (undiscovered n B d H) i.2.val : ℝ)
Proof (Lean source)
-- @node: actualLabelEnergy_add_one
lemma actualLabelEnergy_add_one (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) :
actualLabelEnergy n B d h H + 1 =
∑ i : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1),
(∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) /
(choose (undiscovered n B d H) i.2.val : ℝ) := by
unfold actualLabelEnergy
have hc : (∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H (∅, 0) y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) /
(choose (undiscovered n B d H) 0 : ℝ) = 1 := by
simp [actualLabelWalshCoeff_constant_integral]
rw [← hc]
exact Finset.sum_erase_add _ _ (Finset.mem_univ _)
Sum the active-support envelope over the actual labeled revealed subsets.
Definition (Lean source)
parameters
def
actualLabelActiveEnvelope n B d h H :
ℝ
given by
∑ J : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card), activeSupportEnergyBound B d h (capacity n B d H) (fun ℓ => ((J.map (revealedLabelEmbedding n B d H)) ∩ revealedSources n B d H ℓ).card)
theorem actualLabelEnergy_le_activeEnvelope
On the good event the exact nonconstant labeled energy is bounded by the summed disjoint-support envelope after subtracting its one constant coefficient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
actualLabelEnergy n B d h H ≤ actualLabelActiveEnvelope n B d h H - 1
Proof (Lean source)
-- @node: actualLabelEnergy_le_activeEnvelope
lemma actualLabelEnergy_le_activeEnvelope (n B d : ℕ) (h : ℝ)
(hB : 0 < B) (hd : 0 < d) (H : OffDiag (Fin n) → Bool)
(hgood : (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H) :
actualLabelEnergy n B d h H ≤ actualLabelActiveEnvelope n B d h H - 1 := by
have hs : actualLabelEnergy n B d h H + 1 ≤ actualLabelActiveEnvelope n B d h H := by
rw [actualLabelEnergy_add_one, Fintype.sum_prod_type]
unfold actualLabelActiveEnvelope
apply Finset.sum_le_sum
intro J _
simp_rw [actualLabelWalshCoeff_support_energy n B d h hd H]
rw [Fin.sum_univ_eq_sum_range (fun k =>
(∑ T : Finset (Fin B), ∫ y : Fin B → ℝ,
allocationSupportCoeff B d h (capacity n B d H)
(fun ℓ => ((J.map (revealedLabelEmbedding n B d H)) ∩
revealedSources n B d H ℓ).card) k T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) /
((undiscovered n B d H).choose k : ℝ))]
apply allocationSupportCoeff_total_energy_le B d (undiscovered n B d H) h
(capacity n B d H) _ hB hd
· intro ℓ
exact sub_le _ _
· rfl
· exact hgood
linarith
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ActiveRowAverage 5 declarations The revealed and hidden active row sets are counted with their labels intact.
Averaging and counting disjoint active rows
The revealed and hidden active row sets are counted with their labels intact. Their independent reveal-energy averages feed the finite contraction series.
Counting subsets of the remaining labeled rows gives a binomial sum, extended by zero to the full row range. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: active_row_complement_sum
lemma active_row_complement_sum (B : ℕ) (A : Finset (Fin B)) (F : ℕ → ℝ) :
(∑ E ∈ (univ \ A).powerset, F E.card) =
∑ b ∈ range (B + 1), ((B - A.card).choose b : ℝ) * F b := by
classical
rw [Finset.sum_powerset_apply_card]
have hc : (univ \ A).card = B - A.card := by
rw [Finset.card_sdiff_of_subset (Finset.subset_univ A)]
simp
simp only [hc, nsmul_eq_mul]
apply Finset.sum_subset
· intro b hb
simp only [Finset.mem_range] at hb ⊢
omega
· intro b hb hn
have hlt : B - A.card < b := by
simp only [Finset.mem_range] at hn
omega
simp [Nat.choose_eq_zero_of_lt hlt]
Ordered disjoint active row sets of sizes a and b have exactly the two binomial factors used by the contraction series. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: active_row_pair_sum_by_card
lemma active_row_pair_sum_by_card (B : ℕ) (F : ℕ → ℕ → ℝ) :
(∑ A : Finset (Fin B), ∑ E ∈ (Finset.univ \ A).powerset,
F A.card E.card) =
∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
(B.choose a : ℝ) * ((B - a).choose b : ℝ) * F a b := by
classical
simp_rw [active_row_complement_sum]
have hc := Finset.sum_powerset_apply_card
(fun a => ∑ b ∈ range (B + 1), ((B - a).choose b : ℝ) * F a b)
(x := (Finset.univ : Finset (Fin B)))
simp only [Finset.powerset_univ, Finset.card_univ, Fintype.card_fin,
nsmul_eq_mul] at hc
rw [hc]
simp only [Finset.mul_sum, mul_assoc]
theorem active_row_indicator_mean_le
A good-event indicator can be dropped from each nonnegative disjoint active row contribution before averaging independent source reveals. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ rows : Fin B → Fin d → Bool, indicator G (fun rows => (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) * (∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))) rows ∂Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p)))
Proof (Lean source)
-- @node: active_row_indicator_mean_le
lemma active_row_indicator_mean_le (B d : ℕ) (h p : ℝ)
(hp : p ∈ Icc 0 1) (A E : Finset (Fin B)) (hAE : Disjoint A E)
(G : Set (Fin B → Fin d → Bool)) :
(∫ rows : Fin B → Fin d → Bool, indicator G
(fun rows => (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))) rows
∂Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p))) ≤
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((p * etaOne d h) ^ A.card * (eta d h) ^ E.card) := by
classical
let := bernoulliLaw_probability p hp
let μ := Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p))
let c : ℝ := (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card
let F (rows : Fin B → Fin d → Bool) : ℝ :=
(∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ))
have hc : 0 ≤ c := by dsimp [c]; positivity
have hF (rows) : 0 ≤ F rows := mul_nonneg
(Finset.prod_nonneg (fun ℓ _ => (rowRevealEnergy_bounds d h (rows ℓ)).1))
(Finset.prod_nonneg (fun ℓ _ => (rowRevealEnergy_bounds d h (rows ℓ)).2.1))
change (∫ rows, indicator G (fun rows => c * F rows) rows ∂μ) ≤ _
calc
_ ≤ ∫ rows, c * F rows ∂μ := by
apply integral_mono
· fun_prop
· fun_prop
· intro rows
by_cases hg : rows ∈ G
· simp [Set.indicator_of_mem hg]
· simpa [Set.indicator_of_notMem hg] using mul_nonneg hc (hF rows)
_ = c * ∫ rows, F rows ∂μ := integral_const_mul c F
_ ≤ _ := mul_le_mul_of_nonneg_left
(independent_rowRevealEnergy_product_bound B d h p hp A E hAE) hc
theorem active_row_nonconstant_average_le
Summing labeled disjoint row contributions, omitting the constant term, and then averaging gives exactly the finite binomial contraction bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ rows : Fin B → Fin d → Bool, ∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset, if A.card + E.card = 0 then 0 else indicator G (fun rows => (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) * (∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))) rows ∂Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p)))
+ b
Proof (Lean source)
-- @node: active_row_nonconstant_average_le
lemma active_row_nonconstant_average_le (B d : ℕ) (h p : ℝ)
(hp : p ∈ Icc 0 1) (G : Set (Fin B → Fin d → Bool)) :
(∫ rows : Fin B → Fin d → Bool,
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then 0 else indicator G
(fun rows => (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))) rows
∂Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p))) ≤
∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a + b = 0 then 0 else
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b *
(p * etaOne d h) ^ a * (eta d h) ^ b := by
classical
let := bernoulliLaw_probability p hp
rw [integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
simp_rw [integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
calc
_ ≤ ∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then 0 else
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((p * etaOne d h) ^ A.card * (eta d h) ^ E.card) := by
apply Finset.sum_le_sum
intro A _
apply Finset.sum_le_sum
intro E hE
by_cases hz : A.card + E.card = 0
· simp [hz]
· simp only [hz, if_false]
apply active_row_indicator_mean_le B d h p hp A E
· apply Finset.disjoint_left.mpr
intro ℓ hA hEℓ
have hs := Finset.mem_powerset.mp hE hEℓ
exact (Finset.mem_sdiff.mp hs).2 hA
_ = _ := by
rw [active_row_pair_sum_by_card B (fun a b =>
if a + b = 0 then 0 else
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b *
((p * etaOne d h) ^ a * (eta d h) ^ b))]
apply Finset.sum_congr rfl
intro a _
apply Finset.sum_congr rfl
intro b _
split_ifs <;> ring
theorem active_row_average_contraction_bound
The averaged nonconstant active-row energy, with any good-event indicator, is bounded by the exponential/geometric expression of the contraction roadmap. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ rows : Fin B → Fin d → Bool, ∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset, if A.card + E.card = 0 then 0 else indicator G (fun rows => (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) * (∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))) rows ∂Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p)))
Proof (Lean source)
-- @node: active_row_average_contraction_bound
lemma active_row_average_contraction_bound (B d : ℕ) (h p : ℝ)
(hB : 0 < B) (hp : p ∈ Icc 0 1)
(hsmall : 4 * exp 1 * eta d h < 1)
(G : Set (Fin B → Fin d → Bool)) :
(∫ rows : Fin B → Fin d → Bool,
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then 0 else indicator G
(fun rows => (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))) rows
∂Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p))) ≤
exp (exp 1 * B * p * etaOne d h) /
(1 - 4 * exp 1 * eta d h) - 1 := by
have henergy := eta_nonneg_le_etaOne d h
have hx : 0 ≤ p * etaOne d h := mul_nonneg hp.1 (henergy.1.trans henergy.2)
have hs := contraction_nonconstant_sum_bound B (p * etaOne d h)
(eta d h) hB hx henergy.1 hsmall
rw [show exp 1 * (B : ℝ) * (p * etaOne d h) =
exp 1 * B * p * etaOne d h by ring] at hs
exact (active_row_nonconstant_average_le B d h p hp G).trans hs
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ActiveSupportBridge 5 declarations Compatible response supports are reindexed by their hidden-only active rows.
Disjoint active rows of the actual conditional likelihood
Compatible response supports are reindexed by their hidden-only active rows. The masked allocation coefficient then satisfies the existing good-event energy bound with the exact number of hidden-only rows.
A response support containing the revealed-active rows is uniquely their union with a subset of the complementary labeled rows. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: active_support_sum_reindex
lemma active_support_sum_reindex (B : ℕ) (A : Finset (Fin B))
(F : Finset (Fin B) → ℝ) (hzero : ∀ T, ¬ A ⊆ T → F T = 0) :
(∑ T : Finset (Fin B), F T) =
∑ E ∈ (univ \ A).powerset, F (A ∪ E) := by
classical
rw [← Finset.sum_subset (filter_subset (fun T => A ⊆ T) univ)
(fun T _ hT => hzero T (by simpa using hT))]
apply Finset.sum_bij (fun T _ => T \ A)
· intro T hT
simp only [Finset.mem_powerset]
intro ℓ hℓ
exact Finset.mem_sdiff.mpr ⟨Finset.mem_univ _, (Finset.mem_sdiff.mp hℓ).2⟩
· intro T hT U hU he
have hAT := (Finset.mem_filter.mp hT).2
have hAU := (Finset.mem_filter.mp hU).2
ext ℓ
by_cases hℓ : ℓ ∈ A
· simp [hAT hℓ, hAU hℓ]
· have := congrArg (fun S => ℓ ∈ S) he
simpa [hℓ] using this
· intro E hE
refine ⟨A ∪ E, by simp, ?_⟩
ext ℓ
have hEA : ℓ ∈ E → ℓ ∉ A := fun hℓ =>
(Finset.mem_sdiff.mp (Finset.mem_powerset.mp hE hℓ)).2
simp only [mem_sdiff, mem_union]
tauto
· intro T hT
congr 1
have hAT := (Finset.mem_filter.mp hT).2
ext ℓ
simp only [mem_union, mem_sdiff]
constructor
· intro hℓ
by_cases hA : ℓ ∈ A
· exact inl hA
· exact inr ⟨hℓ, hA⟩
· rintro (hA | ⟨hℓ, _⟩)
· exact hAT hA
· exact hℓ
theorem allocationSupportCoeff_disjoint_energy_sum
The actual coefficient's squared norm splits over disjoint hidden-only active rows, with incompatible supports contributing zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ T : Finset (Fin B), ∫ y : Fin B → ℝ, allocationSupportCoeff B d h u j k T y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= ∑ E ∈ (univ \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).powerset, ∫ y : Fin B → ℝ, allocationSupportCoeff B d h u j k ((Finset.univ.filter (fun ℓ => j ℓ ≠ 0)) ∪ E) y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: allocationSupportCoeff_disjoint_energy_sum
lemma allocationSupportCoeff_disjoint_energy_sum (B d : ℕ) (h : ℝ)
(u j : Fin B → ℕ) (k : ℕ) :
(∑ T : Finset (Fin B), ∫ y : Fin B → ℝ,
allocationSupportCoeff B d h u j k T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) =
∑ E ∈ (univ \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).powerset,
∫ y : Fin B → ℝ,
allocationSupportCoeff B d h u j k
((Finset.univ.filter (fun ℓ => j ℓ ≠ 0)) ∪ E) y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
apply active_support_sum_reindex
intro T hT
have hn : ¬ ∀ ℓ, j ℓ ≠ 0 → ℓ ∈ T := by
simpa only [Finset.subset_iff, mem_filter, Finset.mem_univ, true_and] using hT
simp_rw [allocationSupportCoeff_eq_rowwise_or_zero, if_neg hn]
simp
theorem activeCapacity_sum_bounds
A masked support uses no more than all hidden slots and no more than degree times the number of its active rows. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
(∑ ℓ, activeCapacity B u T ℓ) ≤ ∑ ℓ, u ℓ
conclusion 2
(∑ ℓ, activeCapacity B u T ℓ) ≤ d * T.card
Proof (Lean source)
-- @node: activeCapacity_sum_bounds
lemma activeCapacity_sum_bounds (B d : ℕ) (u : Fin B → ℕ)
(T : Finset (Fin B)) (hu : ∀ ℓ, u ℓ ≤ d) :
(∑ ℓ, activeCapacity B u T ℓ) ≤ ∑ ℓ, u ℓ ∧
(∑ ℓ, activeCapacity B u T ℓ) ≤ d * T.card := by
constructor
· apply Finset.sum_le_sum
intro ℓ _
simp only [activeCapacity]
split_ifs <;> omega
· rw [show (∑ ℓ, activeCapacity B u T ℓ) = ∑ ℓ ∈ T, u ℓ by
simp [activeCapacity]]
calc
_ ≤ ∑ ℓ ∈ T, d := Finset.sum_le_sum (fun ℓ _ => hu ℓ)
_ = _ := by simp [Nat.mul_comm]
theorem allocationSupportCoeff_summed_energy_le
Each compatible actual support satisfies the all-hidden-degrees energy bound after masking inactive capacities, with no independence of hidden rows. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d M :
ℕ
h :
ℝ
u j :
Fin B → ℕ
hB :
0 < B
hd :
0 < d
hu :
∀ ℓ, u ℓ ≤ d
hM :
(∑ ℓ, u ℓ) ≤ M
hgood :
(B * d : ℕ) / (4 : ℝ) ≤ M
hT :
∀ ℓ
if
j ℓ ≠ 0
then
ℓ ∈ T
conclusion
(∑ k ∈ range (M + 1), (∫ y : Fin B → ℝ, allocationSupportCoeff B d h u j k T y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))) / (M.choose k : ℝ))
≤ (min 1 (4 * ((Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card + (T \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card : ℕ) / (B : ℝ))) ^ (T \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card * ∏ ℓ, ∑ t : Fin (activeCapacity B u T ℓ + 1), if (if j ℓ ≠ 0 then True else if ℓ ∈ T then 0 < t.val else t.val = 0) then ((activeCapacity B u T ℓ).choose t.val : ℝ) * gamma d h (j ℓ + t.val) else 0
Proof (Lean source)
-- @node: allocationSupportCoeff_summed_energy_le
lemma allocationSupportCoeff_summed_energy_le (B d M : ℕ) (h : ℝ)
(u j : Fin B → ℕ) (T : Finset (Fin B))
(hB : 0 < B) (hd : 0 < d) (hu : ∀ ℓ, u ℓ ≤ d)
(hM : (∑ ℓ, u ℓ) ≤ M)
(hgood : (B * d : ℕ) / (4 : ℝ) ≤ M)
(hT : ∀ ℓ, j ℓ ≠ 0 → ℓ ∈ T) :
(∑ k ∈ range (M + 1),
(∫ y : Fin B → ℝ, allocationSupportCoeff B d h u j k T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) / (M.choose k : ℝ)) ≤
(min 1 (4 * ((Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card +
(T \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card : ℕ) / (B : ℝ))) ^
(T \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card *
∏ ℓ, ∑ t : Fin (activeCapacity B u T ℓ + 1),
if (if j ℓ ≠ 0 then True else
if ℓ ∈ T then 0 < t.val else t.val = 0) then
((activeCapacity B u T ℓ).choose t.val : ℝ) *
gamma d h (j ℓ + t.val) else 0 := by
classical
have hAT : Finset.univ.filter (fun ℓ => j ℓ ≠ 0) ⊆ T := by
intro ℓ hℓ
exact hT ℓ (Finset.mem_filter.mp hℓ).2
have hcard : (Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card +
(T \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card = T.card := by
rw [Finset.card_sdiff_of_subset hAT]
have := Finset.card_le_card hAT
omega
have hb := activeCapacity_sum_bounds B d u T hu
have hmask (k : ℕ) (y : Fin B → ℝ) :=
allocationSupportCoeff_activeCapacity B d h u j k T y
have hrow (k : ℕ) (y : Fin B → ℝ) :=
allocationSupportCoeff_eq_rowwise B d h (activeCapacity B u T) j k T hT y
simp_rw [hmask, hrow]
have hbound := hidden_allocation_rowwise_summed_energy_le B d M
(Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card
(T \ Finset.univ.filter (fun ℓ => j ℓ ≠ 0)).card h
(activeCapacity B u T) j
(fun ℓ t => if j ℓ ≠ 0 then True else
if ℓ ∈ T then 0 < t.val else t.val = 0) hB hd (hb.1.trans hM)
(by simpa only [hcard] using hb.2) hgood
(fun t ht => active_support_min_hidden_degree B (activeCapacity B u T) j T t ht)
convert hbound using 1 <;> congr 2
· funext k
congr 1
apply integral_congr_ae
filter_upwards [] with y
congr 1
simp only [Finset.sum_filter]
apply Finset.sum_congr rfl
intro t _
split_ifs <;> rfl
· funext ℓ
apply Finset.sum_congr rfl
intro t _
split_ifs <;> rfl
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ActualLabelSupportEnergy 9 declarations The capacity-constrained coefficient is partitioned by its actual positive-degree response rows.
Active response supports of the actual labeled coefficients
The capacity-constrained coefficient is partitioned by its actual positive-degree response rows. Orthogonality gives an exact decomposition of its energy without assuming orthogonality between different positive degrees within a row.
The positive response rows of a revealed and hidden degree vector.
Restrict the exact bounded allocation coefficient to one response support.
Definition (Lean source)
parameters
def
allocationSupportCoeff B d h u j k T y :
ℝ
given by
∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) => k = ∑ ℓ, (t ℓ).val ∧ allocationResponseSupport B j (fun ℓ => (t ℓ).val) = T), (∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) * ∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)
theorem allocationSupportCoeff_sum
Every bounded allocation belongs to precisely one actual response support. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= ∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) => k = ∑ ℓ, (t ℓ).val), (∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) * ∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)
Proof (Lean source)
-- @node: allocationSupportCoeff_sum
lemma allocationSupportCoeff_sum (B d : ℕ) (h : ℝ) (u j : Fin B → ℕ)
(k : ℕ) (y : Fin B → ℝ) :
(∑ T : Finset (Fin B), allocationSupportCoeff B d h u j k T y) =
∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) =>
k = ∑ ℓ, (t ℓ).val),
(∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) *
∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ) := by
unfold allocationSupportCoeff
simpa only [Finset.filter_filter] using
(Finset.sum_fiberwise
(Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) => k = ∑ ℓ, (t ℓ).val))
(fun t => allocationResponseSupport B j (fun ℓ => (t ℓ).val))
(fun t => (∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) *
∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)))
theorem allocationSupportCoeff_energy
Distinct actual response supports contribute additive squared norms. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ y : Fin B → ℝ, (∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) => k = ∑ ℓ, (t ℓ).val), (∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) * ∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)) ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= ∑ T : Finset (Fin B), ∫ y : Fin B → ℝ, allocationSupportCoeff B d h u j k T y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: allocationSupportCoeff_energy
lemma allocationSupportCoeff_energy (B d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(u j : Fin B → ℕ) (k : ℕ) :
(∫ y : Fin B → ℝ,
(∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) =>
k = ∑ ℓ, (t ℓ).val),
(∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) *
∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)) ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) =
∑ T : Finset (Fin B), ∫ y : Fin B → ℝ,
allocationSupportCoeff B d h u j k T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
simp_rw [← allocationSupportCoeff_sum B d h u j k]
unfold allocationSupportCoeff
apply hidden_active_support_group_energy B d h hd
univ
(fun T => Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) =>
k = ∑ ℓ, (t ℓ).val ∧
allocationResponseSupport B j (fun ℓ => (t ℓ).val) = T))
(fun _ t => ∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ))
(fun _ t ℓ => j ℓ + (t ℓ).val) id
· intro T _ t ht ℓ
have he := (Finset.mem_filter.mp ht).2.2
rw [← he]
simp [allocationResponseSupport]
· intro T _ U _ hTU
exact hTU
theorem actualLabelWalshCoeff_support_energy
The actual enumerated-label coefficient has the same support decomposition, with every original row capacity and labeled revealed count retained. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h :
ℝ
hd :
1 ≤ d
i :
conclusion
(∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= ∑ T : Finset (Fin B), ∫ y : Fin B → ℝ, allocationSupportCoeff B d h (capacity n B d H) (fun ℓ => ((i.1.map (revealedLabelEmbedding n B d H)) ∩ revealedSources n B d H ℓ).card) i.2.val T y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: actualLabelWalshCoeff_support_energy
lemma actualLabelWalshCoeff_support_energy (n B d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(H : OffDiag (Fin n) → Bool)
(i : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1)) :
(∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) =
∑ T : Finset (Fin B), ∫ y : Fin B → ℝ,
allocationSupportCoeff B d h (capacity n B d H)
(fun ℓ => ((i.1.map (revealedLabelEmbedding n B d H)) ∩
revealedSources n B d H ℓ).card) i.2.val T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
simp_rw [actualLabelWalshCoeff, groupedHiddenWalshCoeff_eq_bounded]
exact allocationSupportCoeff_energy B d h hd _ _ _
theorem allocationResponseSupport_eq_iff_rowwise
A coefficient fiber's support condition is exactly a rowwise restriction: revealed-active rows allow every hidden degree, hidden-only active rows require a positive degree, and all remaining rows require degree zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= T ↔ ∀ ℓ, if j ℓ ≠ 0 then True else if ℓ ∈ T then 0 < t ℓ else t ℓ = 0
Proof (Lean source)
-- @node: allocationResponseSupport_eq_iff_rowwise
lemma allocationResponseSupport_eq_iff_rowwise (B : ℕ) (j t : Fin B → ℕ)
(T : Finset (Fin B)) (hT : ∀ ℓ, j ℓ ≠ 0 → ℓ ∈ T) :
allocationResponseSupport B j t = T ↔
∀ ℓ, if j ℓ ≠ 0 then True else if ℓ ∈ T then 0 < t ℓ else t ℓ = 0 := by
constructor
· intro he ℓ
have hm : j ℓ + t ℓ ≠ 0 ↔ ℓ ∈ T := by
rw [← he]
simp [allocationResponseSupport]
split_ifs with hj hmem
· have : j ℓ = 0 := by simpa using hj
have := hm.mpr hmem
omega
· have : j ℓ = 0 := by simpa using hj
have := mt hm.mp hmem
omega
· intro ht
ext ℓ
simp only [allocationResponseSupport, mem_filter, Finset.mem_univ, true_and]
by_cases hj : j ℓ ≠ 0
· have hmem := hT ℓ hj
simp only [hmem, iff_true]
omega
· have hj0 : j ℓ = 0 := by simpa using hj
have := ht ℓ
by_cases hmem : ℓ ∈ T
· simp only [hj, hmem, if_false, if_true] at this
simp only [hj0, zero_add, hmem, iff_true]
omega
· simp only [hj, hmem, if_false] at this
simp [hj0, this, hmem]
theorem allocationSupportCoeff_eq_rowwise
Each support-group coefficient is the exact rowwise-restricted allocation sum needed by the allocation-energy bound, with no new statistical premise. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
allocationSupportCoeff B d h u j k T y
= ∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) => (∀ ℓ, if j ℓ ≠ 0 then True else if ℓ ∈ T then 0 < (t ℓ).val else (t ℓ).val = 0) ∧ k = ∑ ℓ, (t ℓ).val), (∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) * ∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)
Proof (Lean source)
-- @node: allocationSupportCoeff_eq_rowwise
lemma allocationSupportCoeff_eq_rowwise (B d : ℕ) (h : ℝ) (u j : Fin B → ℕ)
(k : ℕ) (T : Finset (Fin B)) (hT : ∀ ℓ, j ℓ ≠ 0 → ℓ ∈ T)
(y : Fin B → ℝ) :
allocationSupportCoeff B d h u j k T y =
∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) =>
(∀ ℓ, if j ℓ ≠ 0 then True else
if ℓ ∈ T then 0 < (t ℓ).val else (t ℓ).val = 0) ∧
k = ∑ ℓ, (t ℓ).val),
(∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) *
∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ) := by
unfold allocationSupportCoeff
congr 1
ext t
simp only [mem_filter, Finset.mem_univ, true_and,
allocationResponseSupport_eq_iff_rowwise B j _ T hT, and_comm]
theorem allocationSupportCoeff_eq_rowwise_or_zero
A response support omitting any revealed-active row has no compatible allocation; every compatible support gives the exact rowwise coefficient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
allocationSupportCoeff B d h u j k T y
= if ∀ ℓ, j ℓ ≠ 0 → ℓ ∈ T then ∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) => (∀ ℓ, if j ℓ ≠ 0 then True else if ℓ ∈ T then 0 < (t ℓ).val else (t ℓ).val = 0) ∧ k = ∑ ℓ, (t ℓ).val), (∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) * ∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ) else 0
Proof (Lean source)
-- @node: allocationSupportCoeff_eq_rowwise_or_zero
lemma allocationSupportCoeff_eq_rowwise_or_zero (B d : ℕ) (h : ℝ)
(u j : Fin B → ℕ) (k : ℕ) (T : Finset (Fin B)) (y : Fin B → ℝ) :
allocationSupportCoeff B d h u j k T y =
if ∀ ℓ, j ℓ ≠ 0 → ℓ ∈ T then
∑ t ∈ Finset.univ.filter (fun t : ∀ ℓ, Fin (u ℓ + 1) =>
(∀ ℓ, if j ℓ ≠ 0 then True else
if ℓ ∈ T then 0 < (t ℓ).val else (t ℓ).val = 0) ∧
k = ∑ ℓ, (t ℓ).val),
(∏ ℓ, ((u ℓ).choose (t ℓ).val : ℝ)) *
∏ ℓ, walshCoeff d h (j ℓ + (t ℓ).val) (y ℓ)
else 0 := by
split_ifs with hT
· exact allocationSupportCoeff_eq_rowwise B d h u j k T hT y
· unfold allocationSupportCoeff
apply Finset.sum_eq_zero
intro t ht
exfalso
apply hT
intro ℓ hj
have he := (Finset.mem_filter.mp ht).2.2
rw [← he]
simp only [allocationResponseSupport, mem_filter, Finset.mem_univ, true_and]
omega
theorem actualLabelEnergy_support_sum
The exact nonconstant energy of the actual posterior splits by response support before any Cauchy–Schwarz bound is applied. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
actualLabelEnergy n B d h H
= ∑ i ∈ (Finset.univ : Finset (Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) × Fin (undiscovered n B d H + 1))).erase (∅, 0), (∑ T : Finset (Fin B), ∫ y : Fin B → ℝ, allocationSupportCoeff B d h (capacity n B d H) (fun ℓ => ((i.1.map (revealedLabelEmbedding n B d H)) ∩ revealedSources n B d H ℓ).card) i.2.val T y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))) / (choose (undiscovered n B d H) i.2.val : ℝ)
Proof (Lean source)
-- @node: actualLabelEnergy_support_sum
lemma actualLabelEnergy_support_sum (n B d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(H : OffDiag (Fin n) → Bool) :
actualLabelEnergy n B d h H =
∑ i ∈ (Finset.univ : Finset
(Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1))).erase (∅, 0),
(∑ T : Finset (Fin B), ∫ y : Fin B → ℝ,
allocationSupportCoeff B d h (capacity n B d H)
(fun ℓ => ((i.1.map (revealedLabelEmbedding n B d H)) ∩
revealedSources n B d H ℓ).card) i.2.val T y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) /
(choose (undiscovered n B d H) i.2.val : ℝ) := by
unfold actualLabelEnergy
apply Finset.sum_congr rfl
intro i _
rw [actualLabelWalshCoeff_support_energy n B d h hd H i]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ActualLabelWalshBridge 19 declarations The original assignment coordinates indexed by revealed and hidden source labels have their exact product law.
Transport of the actual labeled likelihood to independent signs
The original assignment coordinates indexed by revealed and hidden source labels have their exact product law. Ancillary coordinates are integrated out by a measure-preserving projection, rather than replacing hidden row assignments.
An injective selection of actual assignment labels retains independent fair signs. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: halfBernoulli_map_embedding
lemma halfBernoulli_map_embedding {ι κ : Type*} [Fintype ι] [Fintype κ]
(e : κ ↪ ι) :
(halfBernoulli ι).map (fun z i => z (e i)) = halfBernoulli κ := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
have hi := ProbabilityTheory.iIndepFun_pi
(μ := fun _ : ι => bernoulliLaw (1 / 2))
(X := fun _ (b : Bool) => b) (fun _ => measurable_id.aemeasurable)
have hs := hi.precomp e.injective
unfold halfBernoulli
rw [hs.map_fun_eq_pi_map (fun i => (measurable_pi_apply (e i)).aemeasurable)]
simp only [Measure.pi_map_eval, measure_univ, Finset.prod_const_one, one_smul]
theorem halfBernoulli_map_disjoint_embeddings
Two disjoint injective label selections have the exact independent sign law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(halfBernoulli ι).map (fun z => (fun i => z (r i), fun j => z (u j)))
Proof (Lean source)
-- @node: halfBernoulli_map_disjoint_embeddings
lemma halfBernoulli_map_disjoint_embeddings {ι : Type*} [Fintype ι]
(R M : ℕ) (r : Fin R ↪ ι) (u : Fin M ↪ ι)
(hru : ∀ i j, r i ≠ u j) :
(halfBernoulli ι).map (fun z => (fun i => z (r i), fun j => z (u j))) =
(halfBernoulli (Fin R)).prod (halfBernoulli (Fin M)) := by
let e : Fin R ⊕ Fin M ↪ ι :=
⟨Sum.elim r u, by
intro a b hab
cases a with
| inl a => cases b with
| inl b => exact congrArg inl (r.injective hab)
| inr b => exact False.elim (hru a b hab)
| inr a => cases b with
| inl b => exact False.elim (hru b a hab.symm)
| inr b => exact congrArg inr (u.injective hab)⟩
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
have he := halfBernoulli_map_embedding e
have hp := (measurePreserving_sumPiEquivProdPi
(fun _ : Fin R ⊕ Fin M => bernoulliLaw (1 / 2))).map_eq
change (halfBernoulli (Fin R ⊕ Fin M)).map
(MeasurableEquiv.sumPiEquivProdPi (fun _ => Bool)) =
(halfBernoulli (Fin R)).prod (halfBernoulli (Fin M)) at hp
rw [← he] at hp
rw [Measure.map_map (by fun_prop) (measurable_of_finite _)] at hp
exact hp
Enumerate every actually revealed source label once.
Definition (Lean source)
parameters
def
given by
(Finset.equivFin (Finset.univ.biUnion (revealedSources n B d H))).symm.toEmbedding.trans (Function.Embedding.subtype _)
theorem actualLabelSigns_map
The signs occurring in the actual posterior likelihood have the product law used in the hidden/revealed energy identity. The full assignment is the source. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(halfBernoulli (Fin n)).map (fun z => (fun i => z (revealedLabelEmbedding n B d H i), fun j => z (hiddenSourceLabelEmbedding n B d hfit H ((hiddenSourceEnumeration n B d hfit H s).symm j))))
= (halfBernoulli (Fin (Finset.univ.biUnion (revealedSources n B d H)).card)).prod (halfBernoulli (Fin (undiscovered n B d H)))
Proof (Lean source)
-- @node: actualLabelSigns_map
lemma actualLabelSigns_map (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(H : OffDiag (Fin n) → Bool) (s : CompatiblePartition n B d H) :
(halfBernoulli (Fin n)).map (fun z =>
(fun i => z (revealedLabelEmbedding n B d H i),
fun j => z (hiddenSourceLabelEmbedding n B d hfit H
((hiddenSourceEnumeration n B d hfit H s).symm j)))) =
(halfBernoulli (Fin (Finset.univ.biUnion (revealedSources n B d H)).card)).prod
(halfBernoulli (Fin (undiscovered n B d H))) := by
exact halfBernoulli_map_disjoint_embeddings _ _
(revealedLabelEmbedding n B d H)
((hiddenSourceEnumeration n B d hfit H s).symm.toEmbedding.trans
(hiddenSourceLabelEmbedding n B d hfit H))
(fun i j => revealed_hidden_label_ne n B d hfit H i _)
theorem actualLabel_response_nonconstant_energy
Transport the nonconstant energy identity back to the original assignment, including all unused recipient and padding coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Ω :
Type*
n R M :
ℕ
hmap :
hzero :
(∅, 0) ∈ G
hc0 :
∀ y, c (∅, 0) y = 1
hm :
∀ i, Measurable (c i)
hc :
∀ i ∈ G, ∀ j ∈ G, Integrable (fun y => c i y * c j y) ν
conclusion
(∫ p : Assign (Fin n) × Ω, ((∑ i ∈ G, ((∏ j ∈ i.1, signOf (p.1 (r j))) * symmetricSignPoly M i.2.val (fun j => p.1 (u j))) * c i p.2) - 1) ^ 2 ∂(halfBernoulli (Fin n)).prod ν)
= ∑ i ∈ G.erase (∅, 0), (∫ y, c i y ^ 2 ∂ν) / (M.choose i.2.val : ℝ)
Proof (Lean source)
-- @node: actualLabel_response_nonconstant_energy
lemma actualLabel_response_nonconstant_energy {Ω : Type*} [MeasurableSpace Ω]
(ν : Measure Ω) [SFinite ν] (n R M : ℕ)
(r : Fin R ↪ Fin n) (u : Fin M ↪ Fin n)
(hmap : (halfBernoulli (Fin n)).map
(fun z => (fun i => z (r i), fun j => z (u j))) =
(halfBernoulli (Fin R)).prod (halfBernoulli (Fin M)))
(G : Finset (Finset (Fin R) × Fin (M + 1)))
(c : (Finset (Fin R) × Fin (M + 1)) → Ω → ℝ)
(hzero : (∅, 0) ∈ G) (hc0 : ∀ y, c (∅, 0) y = 1)
(hm : ∀ i, Measurable (c i))
(hc : ∀ i ∈ G, ∀ j ∈ G, Integrable (fun y => c i y * c j y) ν) :
(∫ p : Assign (Fin n) × Ω,
((∑ i ∈ G, ((∏ j ∈ i.1, signOf (p.1 (r j))) *
symmetricSignPoly M i.2.val (fun j => p.1 (u j))) * c i p.2) - 1) ^ 2
∂(halfBernoulli (Fin n)).prod ν) =
∑ i ∈ G.erase (∅, 0), (∫ y, c i y ^ 2 ∂ν) / (M.choose i.2.val : ℝ) := by
let F : ((Fin R → Bool) × (Fin M → Bool)) × Ω → ℝ := fun p =>
((∑ i ∈ G, ((∏ j ∈ i.1, signOf (p.1.1 j)) *
symmetricSignPoly M i.2.val p.1.2) * c i p.2) - 1) ^ 2
have hF : Measurable F := by
apply Measurable.pow_const
apply Measurable.sub _ measurable_const
apply Finset.measurable_sum
intro i hi
exact ((measurable_of_finite (fun x : (Fin R → Bool) × (Fin M → Bool) =>
(∏ j ∈ i.1, signOf (x.1 j)) * symmetricSignPoly M i.2.val x.2)).comp
measurable_fst).mul ((hm i).comp measurable_snd)
have hp : ((halfBernoulli (Fin n)).prod ν).map
(fun p => ((fun i => p.1 (r i), fun j => p.1 (u j)), p.2)) =
((halfBernoulli (Fin R)).prod (halfBernoulli (Fin M))).prod ν := by
have hprod := Measure.map_prod_map (halfBernoulli (Fin n)) ν
(f := fun z => (fun i => z (r i), fun j => z (u j)))
(g := id) (measurable_of_finite _) measurable_id
simp only [Prod.map_def, id_eq] at hprod
rw [← hprod, hmap, Measure.map_id]
have he := integral_map (μ := (halfBernoulli (Fin n)).prod ν)
(φ := fun p => ((fun i => p.1 (r i), fun j => p.1 (u j)), p.2))
(by fun_prop) hF.aestronglyMeasurable
rw [hp] at he
exact he.symm.trans (hidden_revealed_response_nonconstant_energy ν R M G c hzero hc0 hc)
theorem revealedLabelEmbedding_univ_map
The revealed enumeration covers exactly the observed union of row labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Finset.univ.map (revealedLabelEmbedding n B d H)
= Finset.univ.biUnion (revealedSources n B d H)
Proof (Lean source)
-- @node: revealedLabelEmbedding_univ_map
lemma revealedLabelEmbedding_univ_map (n B d : ℕ) (H : OffDiag (Fin n) → Bool) :
Finset.univ.map (revealedLabelEmbedding n B d H) =
Finset.univ.biUnion (revealedSources n B d H) := by
ext j
constructor
· rintro hj
obtain ⟨i, _, rfl⟩ := Finset.mem_map.mp hj
exact ((Finset.equivFin (Finset.univ.biUnion (revealedSources n B d H))).symm i).2
· intro hj
refine Finset.mem_map.mpr ⟨
Finset.equivFin (Finset.univ.biUnion (revealedSources n B d H)) ⟨j, hj⟩,
Finset.mem_univ _, ?_⟩
simp [revealedLabelEmbedding]
theorem groupedHiddenWalshCoeff_empty_zero
The exact zero-hidden-degree coefficient contains only the zero row vector. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
groupedHiddenWalshCoeff n B d h H (fun _ => ∅) 0 y = ∏ ℓ, walshCoeff d h 0 (y ℓ)
Proof (Lean source)
-- @node: groupedHiddenWalshCoeff_empty_zero
lemma groupedHiddenWalshCoeff_empty_zero (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (y : Fin B → ℝ) :
groupedHiddenWalshCoeff n B d h H (fun _ => ∅) 0 y =
∏ ℓ, walshCoeff d h 0 (y ℓ) := by
rw [groupedHiddenWalshCoeff_eq_bounded]
have hf : Finset.univ.filter
(fun k : ∀ ℓ : Fin B, Fin (capacity n B d H ℓ + 1) =>
0 = ∑ ℓ, (k ℓ).val) = {fun _ => 0} := by
ext k
simp only [mem_filter, Finset.mem_univ, true_and, Finset.mem_singleton]
constructor
· intro hk
funext ℓ
apply Fin.ext
exact (Finset.sum_eq_zero_iff.mp hk.symm) ℓ (Finset.mem_univ ℓ)
· rintro rfl
simp
rw [hf]
simp
theorem groupedHiddenWalshCoeff_empty_zero_pos
On positive reference support the actual constant posterior coefficient is one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
groupedHiddenWalshCoeff n B d h H (fun _ => ∅) 0 y = 1
Proof (Lean source)
-- @node: groupedHiddenWalshCoeff_empty_zero_pos
lemma groupedHiddenWalshCoeff_empty_zero_pos (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
groupedHiddenWalshCoeff n B d h H (fun _ => ∅) 0 y = 1 := by
rw [groupedHiddenWalshCoeff_empty_zero]
simp_rw [walshCoeff_zero d h _ (hy _)]
simp
Actual response coefficients indexed by enumerated revealed subsets and total hidden degree. The coefficients themselves keep all row capacities.
Definition (Lean source)
parameters
def
actualLabelWalshCoeff n B d h H i y :
ℝ
given by
groupedHiddenWalshCoeff n B d h H (fun ℓ => (i.1.map (revealedLabelEmbedding n B d H)) ∩ revealedSources n B d H ℓ) i.2.val y
Injective label maps carry the full subset family to the image subset family. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(S.map e).powerset = S.powerset.map (mapEmbedding e).toEmbedding
Proof (Lean source)
-- @node: labeled_powerset_map
lemma labeled_powerset_map {α β : Type*}
(S : Finset α) (e : α ↪ β) :
(S.map e).powerset = S.powerset.map (mapEmbedding e).toEmbedding := by
classical
simp only [Finset.map_eq_image, RelEmbedding.coe_toEmbedding]
rw [Finset.powerset_image]
congr 1
funext E
exact (Finset.map_eq_image e E).symm
theorem blockDensity_actualLabel_walsh_expansion
The original-label likelihood is exactly the finite expansion indexed by the independent sign projection, wherever the reference density is positive. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s :
CompatiblePartition n B d H
y :
Fin B → ℝ
hy :
∀ ℓ, 0 < refDensity d h (y ℓ)
conclusion
= ∑ i : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) × Fin (undiscovered n B d H + 1), ((∏ j ∈ i.1, signOf (z (revealedLabelEmbedding n B d H j))) * symmetricSignPoly (undiscovered n B d H) i.2.val (fun j => z (hiddenSourceLabelEmbedding n B d hfit H ((hiddenSourceEnumeration n B d hfit H s).symm j)))) * actualLabelWalshCoeff n B d h H i y
Proof (Lean source)
-- @node: blockDensity_actualLabel_walsh_expansion
lemma blockDensity_actualLabel_walsh_expansion (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(s : CompatiblePartition n B d H) (z : Assign (Fin n)) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) =
∑ i : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1),
((∏ j ∈ i.1, signOf (z (revealedLabelEmbedding n B d H j))) *
symmetricSignPoly (undiscovered n B d H) i.2.val
(fun j => z (hiddenSourceLabelEmbedding n B d hfit H
((hiddenSourceEnumeration n B d hfit H s).symm j)))) *
actualLabelWalshCoeff n B d h H i y := by
rw [blockDensity_grouped_hidden_walsh_expansion n B d hd hfit h H z s y hy,
groupedHiddenWalsh_sum_labelSubsets n B d h H s]
conv_lhs =>
rw [← revealedLabelEmbedding_univ_map n B d H, labeled_powerset_map, Finset.sum_map]
rw [Fintype.sum_prod_type]
simp only [Finset.powerset_univ, RelEmbedding.coe_toEmbedding, Finset.mapEmbedding_apply]
apply Finset.sum_congr rfl
intro J hJ
rw [Finset.prod_map, Finset.mul_sum]
rw [← Fin.sum_univ_eq_sum_range]
apply Finset.sum_congr rfl
intro t ht
dsimp [actualLabelWalshCoeff]
ring
theorem blockDensity_actualLabel_nonconstant_energy
At each positive reference response the actual assignment norm of the centered likelihood is exactly its nonconstant coefficient energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s :
CompatiblePartition n B d H
y :
Fin B → ℝ
hy :
∀ ℓ, 0 < refDensity d h (y ℓ)
conclusion
(∫ z : Assign (Fin n), (blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) - 1) ^ 2 ∂halfBernoulli (Fin n))
= ∑ i ∈ (Finset.univ : Finset (Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) × Fin (undiscovered n B d H + 1))).erase (∅, 0), actualLabelWalshCoeff n B d h H i y ^ 2 / (choose (undiscovered n B d H) i.2.val : ℝ)
Proof (Lean source)
-- @node: blockDensity_actualLabel_nonconstant_energy
lemma blockDensity_actualLabel_nonconstant_energy (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(s : CompatiblePartition n B d H) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
(∫ z : Assign (Fin n),
(blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) - 1) ^ 2
∂halfBernoulli (Fin n)) =
∑ i ∈ (Finset.univ : Finset
(Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1))).erase (∅, 0),
actualLabelWalshCoeff n B d h H i y ^ 2 /
(choose (undiscovered n B d H) i.2.val : ℝ) := by
let r := revealedLabelEmbedding n B d H
let u := (hiddenSourceEnumeration n B d hfit H s).symm.toEmbedding.trans
(hiddenSourceLabelEmbedding n B d hfit H)
have hc0 : actualLabelWalshCoeff n B d h H (∅, 0) y = 1 := by
simpa [actualLabelWalshCoeff] using
groupedHiddenWalshCoeff_empty_zero_pos n B d h H y hy
have he := actualLabel_response_nonconstant_energy (Measure.dirac ()) n
(Finset.univ.biUnion (revealedSources n B d H)).card (undiscovered n B d H)
r u (actualLabelSigns_map n B d hfit H s) Finset.univ
(fun i (_ : Unit) => actualLabelWalshCoeff n B d h H i y)
(Finset.mem_univ _) (fun _ => hc0) (fun _ => measurable_const)
(fun _ _ _ _ => integrable_const _)
rw [integral_prod _ (by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (halfBernoulli (Fin n)) := by
unfold halfBernoulli; infer_instance
exact Integrable.of_finite)] at he
simp only [integral_const, Measure.real, measure_univ, ENNReal.toReal_one, one_smul] at he
simp_rw [blockDensity_actualLabel_walsh_expansion n B d hd hfit h H s _ y hy]
exact he
The independent reference responses almost surely lie in positive row support. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
h :
ℝ
conclusion
∀ᵐ y ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))), ∀ ℓ, 0 < refDensity d h (y ℓ)
Proof (Lean source)
-- @node: referenceRows_positive_ae
lemma referenceRows_positive_ae (B d : ℕ) (h : ℝ) :
∀ᵐ y ∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))),
∀ ℓ, 0 < refDensity d h (y ℓ) := by
let ν := volume.withDensity (fun w => ofReal (refDensity d h w))
let := referenceRowLaw_probability d h
have hw : ∀ᵐ w ∂ν, 0 < refDensity d h w := by
rw [ae_withDensity_iff (by fun_prop)]
filter_upwards [] with w hw
exact lt_of_le_of_ne (refDensity_nonneg d h w)
(by intro hz; apply hw; rw [← hz]; simp)
apply ae_all_iff.mpr
intro ℓ
have he : ∀ᵐ w ∂(Measure.pi (fun _ : Fin B => ν)).map
(fun y : Fin B → ℝ => y ℓ), 0 < refDensity d h w := by
rw [(measurePreserving_eval (fun _ : Fin B => ν) ℓ).map_eq]
exact hw
exact ae_of_ae_map (f := fun y : Fin B → ℝ => y ℓ)
(p := fun w => 0 < refDensity d h w) (measurable_pi_apply ℓ).aemeasurable he
theorem actualLabelWalshCoeff_cross_integrable
Exact posterior coefficient cross products are integrable under the actual row reference; no new regularity premise is needed. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h :
ℝ
i j :
conclusion
Integrable (fun y => actualLabelWalshCoeff n B d h H i y * actualLabelWalshCoeff n B d h H j y) (Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
Proof (Lean source)
-- @node: actualLabelWalshCoeff_cross_integrable
lemma actualLabelWalshCoeff_cross_integrable (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool)
(i j : Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1)) :
Integrable (fun y => actualLabelWalshCoeff n B d h H i y *
actualLabelWalshCoeff n B d h H j y)
(Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) := by
simp_rw [actualLabelWalshCoeff, groupedHiddenWalshCoeff_eq_bounded]
exact hidden_support_sum_cross_integrable B d h _ _ _ _ _ _
theorem blockDensity_actualLabel_integrated_nonconstant_energy
Integrating the actual original-label likelihood norm over the reference responses gives precisely the nonconstant coefficient norms used in contraction. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ y : Fin B → ℝ, (∫ z : Assign (Fin n), (blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) - 1) ^ 2 ∂halfBernoulli (Fin n)) ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= ∑ i ∈ (Finset.univ : Finset (Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) × Fin (undiscovered n B d H + 1))).erase (∅, 0), (∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))) / (choose (undiscovered n B d H) i.2.val : ℝ)
Proof (Lean source)
-- @node: blockDensity_actualLabel_integrated_nonconstant_energy
lemma blockDensity_actualLabel_integrated_nonconstant_energy
(n B d : ℕ) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (s : CompatiblePartition n B d H) :
(∫ y : Fin B → ℝ,
(∫ z : Assign (Fin n),
(blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) - 1) ^ 2
∂halfBernoulli (Fin n))
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) =
∑ i ∈ (Finset.univ : Finset
(Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1))).erase (∅, 0),
(∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) /
(choose (undiscovered n B d H) i.2.val : ℝ) := by
calc
_ = ∫ y : Fin B → ℝ,
∑ i ∈ (Finset.univ : Finset
(Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) ×
Fin (undiscovered n B d H + 1))).erase (∅, 0),
actualLabelWalshCoeff n B d h H i y ^ 2 /
(choose (undiscovered n B d H) i.2.val : ℝ)
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
apply integral_congr_ae
filter_upwards [referenceRows_positive_ae B d h] with y hy
exact blockDensity_actualLabel_nonconstant_energy n B d hd hfit h H s y hy
_ = _ := by
rw [integral_finsetSum _ (fun i _ => ?_)]
· apply Finset.sum_congr rfl
intro i hi
exact integral_div _ _
· simp only [pow_two]
exact (actualLabelWalshCoeff_cross_integrable n B d h H i i).div_const _
Original-assignment energy of the posterior density ratio under the response reference.
Definition (Lean source)
parameters
def
actualDensityEnergy n B d h H :
ℝ
given by
∫ y : Fin B → ℝ, (∫ z : Assign (Fin n), (blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) - 1) ^ 2 ∂halfBernoulli (Fin n)) ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))
Exact nonconstant posterior coefficient energy, retaining every hidden-degree weight.
Definition (Lean source)
parameters
def
actualLabelEnergy n B d h H :
ℝ
given by
∑ i ∈ (Finset.univ : Finset (Finset (Fin (Finset.univ.biUnion (revealedSources n B d H)).card) × Fin (undiscovered n B d H + 1))).erase (∅, 0), (∫ y : Fin B → ℝ, actualLabelWalshCoeff n B d h H i y ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))) / (choose (undiscovered n B d H) i.2.val : ℝ)
theorem actualDensityEnergy_eq_actualLabelEnergy
The actual density energy is exactly the label coefficient energy on every compatible detailed graph. This is the bridge consumed by contraction. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
actualDensityEnergy n B d h H = actualLabelEnergy n B d h H
Proof (Lean source)
-- @node: actualDensityEnergy_eq_actualLabelEnergy
lemma actualDensityEnergy_eq_actualLabelEnergy (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(s : CompatiblePartition n B d H) :
actualDensityEnergy n B d h H = actualLabelEnergy n B d h H :=
blockDensity_actualLabel_integrated_nonconstant_energy n B d hd hfit h H s
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.AuditVariance 9 declarations Independent audit noise and conditional second moments
Independent audit noise and conditional second moments
A numeric Bernoulli mark has variance q(1-q). For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: variance_bernoulli_treatment
lemma variance_bernoulli_treatment (q : ℝ) (hq : q ∈ Icc 0 1) :
variance treatment (bernoulliLaw q) = q * (1 - q) := by
let := bernoulliLaw_probability q hq
rw [variance_eq_sub (show MemLp treatment 2 (bernoulliLaw q) from MemLp.of_discrete)]
have he : bernoulliLaw q = bernoulliBool q := by
unfold bernoulliLaw bernoulliBool
exact add_comm _ _
rw [he]
rw [bernoulliBool_integral hq.1 hq.2,
bernoulliBool_integral hq.1 hq.2]
norm_num [treatment, Pi.pow_apply]
ring
theorem variance_weighted_audit_marks
Independent centered audit marks have a diagonal weighted variance. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: variance_weighted_audit_marks
lemma variance_weighted_audit_marks (q : ℝ) (hq : q ∈ Icc 0 1)
(c : OffDiag V → ℝ) :
variance (fun w : Audit V => ∑ e, c e * (treatment (w e) - q)) (auditLaw V q) =
q * (1 - q) * ∑ e, (c e) ^ 2 := by
let := bernoulliLaw_probability q hq
have hs := variance_sum_pi (μ := fun _ : OffDiag V => bernoulliLaw q)
(X := fun e b => c e * (treatment b - q))
(fun _ => (show MemLp _ 2 _ from MemLp.of_discrete))
have hs' : variance (fun w : Audit V => ∑ e, c e * (treatment (w e) - q))
(auditLaw V q) = ∑ e, variance (fun b => c e * (treatment b - q)) (bernoulliLaw q) := by
convert hs using 1
congr 1
funext w
simp
rw [hs']
simp_rw [variance_const_mul,
variance_sub_const (show AEStronglyMeasurable treatment (bernoulliLaw q) from by fun_prop),
variance_bernoulli_treatment q hq]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro e _
ring
An off-diagonal label sum can be organized by recipient and source. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
f :
OffDiag V → ℝ
conclusion
(∑ e, f e) = ∑ i : V, ∑ j : V, if hji : j ≠ i then f ⟨(j, i), hji⟩ else 0
Proof (Lean source)
-- @node: sum_offDiag_by_recipient
lemma sum_offDiag_by_recipient (f : OffDiag V → ℝ) :
(∑ e, f e) = ∑ i : V, ∑ j : V, if hji : j ≠ i then f ⟨(j, i), hji⟩ else 0 := by
let g : V × V → ℝ := fun e => if he : e.1 ≠ e.2 then f ⟨e, he⟩ else 0
have ht : (∑ e, f e) = ∑ e : V × V, g e := by
calc
(∑ e, f e) = ∑ e : OffDiag V, g e.val := by
apply Finset.sum_congr rfl
intro e _
simp only [g, dif_pos e.property]
_ = ∑ e ∈ Finset.univ.filter (fun e : V × V => e.1 ≠ e.2), g e :=
(Finset.sum_subtype _ (by simp) g).symm
_ = ∑ e : V × V, g e := by
rw [Finset.sum_filter]
apply Finset.sum_congr rfl
intro e _
by_cases he : e.1 ≠ e.2 <;> simp [g, he]
rw [ht, Fintype.sum_prod_type_right]
theorem recorded_sign_sum_eq_mark_sum
The recorded-neighbor sum is the true-edge sum multiplied by numeric marks. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j ∈ recordedNbhd (recordOf θ (z, w)).1 i, signOf (z j))
= ∑ j : V, if he : θ.edge j i then treatment (w ⟨(j, i), by intro hji; change j = i at hji; subst j; exact θ.irrefl i he⟩) * signOf (z j) else 0
Proof (Lean source)
-- @node: recorded_sign_sum_eq_mark_sum
lemma recorded_sign_sum_eq_mark_sum (θ : Schedule V) (z : Assign V) (w : Audit V) (i : V) :
(∑ j ∈ recordedNbhd (recordOf θ (z, w)).1 i, signOf (z j)) =
∑ j : V, if he : θ.edge j i then
treatment (w ⟨(j, i), by
intro hji; change j = i at hji; subst j; exact θ.irrefl i he⟩) *
signOf (z j) else 0 := by
simp only [recordedNbhd]
rw [Finset.sum_filter]
dsimp only [recordOf]
apply Finset.sum_congr rfl
intro j _
by_cases he : θ.edge j i
· have hji : j ≠ i := by intro hji; change j = i at hji; subst j; exact θ.irrefl i he
cases hw : w ⟨(j, i), hji⟩ <;> simp [he, hji, hw, treatment]
· simp [he]
theorem auditScore_sub_oracle_eq_noise
The audit error is a weighted sum of independent centered true-edge marks. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (2 / (Fintype.card V : ℝ) / q) * ∑ e : OffDiag V, (if θ.edge e.1.1 e.1.2 then potentialOutcome θ e.1.2 z * signOf (z e.1.1) else 0) * (treatment (w e) - q)
Proof (Lean source)
-- @node: auditScore_sub_oracle_eq_noise
lemma auditScore_sub_oracle_eq_noise (θ : Schedule V) (q : ℝ) (hq : q ≠ 0)
(z : Assign V) (w : Audit V) :
auditScore q (recordOf θ (z, w)) - oracleScore θ z =
(2 / (Fintype.card V : ℝ) / q) * ∑ e : OffDiag V,
(if θ.edge e.1.1 e.1.2 then potentialOutcome θ e.1.2 z * signOf (z e.1.1)
else 0) * (treatment (w e) - q) := by
rw [sum_offDiag_by_recipient]
simp only [auditScore, oracleScore]
change (2 / (Fintype.card V : ℝ)) *
(∑ i, potentialOutcome θ i z * (signOf (z i) + q⁻¹ *
∑ j ∈ recordedNbhd (recordOf θ (z, w)).1 i, signOf (z j))) - _ = _
simp_rw [recorded_sign_sum_eq_mark_sum]
rw [← mul_sub, ← Finset.sum_sub_distrib, Finset.mul_sum, Finset.mul_sum]
apply Finset.sum_congr rfl
intro i _
rw [show potentialOutcome θ i z * (signOf (z i) + q⁻¹ *
∑ j, if he : θ.edge j i then
treatment (w ⟨(j, i), by
intro hji; change j = i at hji; subst j; exact θ.irrefl i he⟩) * signOf (z j)
else 0) - potentialOutcome θ i z *
(signOf (z i) + ∑ j ∈ inNbhd θ i, signOf (z j)) =
potentialOutcome θ i z * (q⁻¹ *
(∑ j, if he : θ.edge j i then
treatment (w ⟨(j, i), by
intro hji; change j = i at hji; subst j; exact θ.irrefl i he⟩) * signOf (z j)
else 0) - ∑ j ∈ inNbhd θ i, signOf (z j)) by ring]
simp only [inNbhd, Finset.sum_filter]
rw [Finset.mul_sum, ← Finset.sum_sub_distrib, Finset.mul_sum, Finset.mul_sum,
Finset.mul_sum]
apply Finset.sum_congr rfl
intro j _
by_cases he : θ.edge j i
· have hji : j ≠ i := fun hji => θ.irrefl i (hji ▸ he)
simp only [dif_pos he, if_pos he, dif_pos hji]
field_simp
· by_cases hji : j ≠ i <;> simp [he, hji]
theorem audit_noise_weights_sq
Squared noise weights sum to the in-degree weighted squared outcomes. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ e : OffDiag V, (if θ.edge e.1.1 e.1.2 then potentialOutcome θ e.1.2 z * signOf (z e.1.1) else 0) ^ 2)
= ∑ i, (inNbhd θ i).card * (potentialOutcome θ i z) ^ 2
Proof (Lean source)
-- @node: audit_noise_weights_sq
lemma audit_noise_weights_sq (θ : Schedule V) (z : Assign V) :
(∑ e : OffDiag V,
(if θ.edge e.1.1 e.1.2 then potentialOutcome θ e.1.2 z * signOf (z e.1.1)
else 0) ^ 2) = ∑ i, (inNbhd θ i).card * (potentialOutcome θ i z) ^ 2 := by
rw [sum_offDiag_by_recipient]
apply Finset.sum_congr rfl
intro i _
have hp : (∑ j : V, if hji : j ≠ i then
(if θ.edge j i then potentialOutcome θ i z * signOf (z j) else 0) ^ 2 else 0) =
∑ j ∈ inNbhd θ i, (potentialOutcome θ i z) ^ 2 := by
rw [inNbhd, Finset.sum_filter]
apply Finset.sum_congr rfl
intro j _
by_cases he : θ.edge j i
· have hji : j ≠ i := fun hji => θ.irrefl i (hji ▸ he)
cases z j <;> simp [he, hji, signOf]
· by_cases hji : j ≠ i <;> simp [he, hji]
rw [hp, Finset.sum_const, nsmul_eq_mul]
theorem variance_auditScore_given_assignment
Conditional audit variance follows from the diagonal centered-mark calculation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
q :
ℝ
hq :
q ∈ Icc 0 1
hq0 :
0 < q
hn :
0 < Fintype.card V
z :
Assign V
conclusion
= 4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) * ∑ i, (inNbhd θ i).card * (potentialOutcome θ i z) ^ 2
Proof (Lean source)
-- @node: variance_auditScore_given_assignment
lemma variance_auditScore_given_assignment (θ : Schedule V) (q : ℝ)
(hq : q ∈ Icc 0 1) (hq0 : 0 < q) (hn : 0 < Fintype.card V) (z : Assign V) :
variance (fun w => auditScore q (recordOf θ (z, w))) (auditLaw V q) =
4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) *
∑ i, (inNbhd θ i).card * (potentialOutcome θ i z) ^ 2 := by
let := auditLaw_probability (V := V) q hq
have he : (fun w => auditScore q (recordOf θ (z, w)) - oracleScore θ z) =
fun w => (2 / (Fintype.card V : ℝ) / q) * ∑ e : OffDiag V,
(if θ.edge e.1.1 e.1.2 then potentialOutcome θ e.1.2 z * signOf (z e.1.1)
else 0) * (treatment (w e) - q) := by
funext w
exact auditScore_sub_oracle_eq_noise θ q (ne_of_gt hq0) z w
rw [← variance_sub_const (show AEStronglyMeasurable
(fun w => auditScore q (recordOf θ (z, w))) (auditLaw V q) from
(show MemLp _ 2 _ from MemLp.of_discrete).aestronglyMeasurable)
(oracleScore θ z), he, variance_const_mul, variance_weighted_audit_marks q hq,
audit_noise_weights_sq]
have hn0 : (Fintype.card V : ℝ) ≠ 0 := by exact_mod_cast (Nat.ne_of_gt hn)
field_simp
ring
theorem integral_auditScore_sq_given_assignment
Integrating the conditional variance adds the oracle squared conditional mean. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
q :
ℝ
hq :
q ∈ Icc 0 1
hq0 :
0 < q
hn :
0 < Fintype.card V
z :
Assign V
conclusion
(∫ w, (auditScore q (recordOf θ (z, w))) ^ 2 ∂auditLaw V q)
= (oracleScore θ z) ^ 2
+ 4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) * ∑ i, (inNbhd θ i).card * (potentialOutcome θ i z) ^ 2
Proof (Lean source)
-- @node: integral_auditScore_sq_given_assignment
lemma integral_auditScore_sq_given_assignment (θ : Schedule V) (q : ℝ)
(hq : q ∈ Icc 0 1) (hq0 : 0 < q) (hn : 0 < Fintype.card V) (z : Assign V) :
(∫ w, (auditScore q (recordOf θ (z, w))) ^ 2 ∂auditLaw V q) =
(oracleScore θ z) ^ 2 +
4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) *
∑ i, (inNbhd θ i).card * (potentialOutcome θ i z) ^ 2 := by
let := auditLaw_probability (V := V) q hq
have hv := variance_auditScore_given_assignment θ q hq hq0 hn z
rw [variance_eq_sub (show MemLp _ 2 _ from MemLp.of_discrete),
integral_auditScore_given_assignment θ q hq hq0 z] at hv
change (∫ w, (auditScore q (recordOf θ (z, w))) ^ 2 ∂auditLaw V q) -
(oracleScore θ z) ^ 2 = _ at hv
linarith
theorem variance_auditScore_eq_oracle_add
The full assignment-audit variance is the oracle variance plus the expected audit variance. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
q :
ℝ
hn :
0 < Fintype.card V
ha :
hw :
AuditLaw D q
hi :
hq :
0 < q
conclusion
+ 4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) * ∑ i, (inNbhd θ i).card * ∫ z, (potentialOutcome θ i z) ^ 2 ∂halfBernoulli V
Proof (Lean source)
-- @node: variance_auditScore_eq_oracle_add
lemma variance_auditScore_eq_oracle_add (D : Measure (Assign V × Audit V))
(θ : Schedule V) (q : ℝ) (hn : 0 < Fintype.card V)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) (hq : 0 < q) :
variance (fun ω => auditScore q (recordOf θ ω)) D =
variance (oracleScore θ) (halfBernoulli V) +
4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) *
∑ i, (inNbhd θ i).card *
∫ z, (potentialOutcome θ i z) ^ 2 ∂halfBernoulli V := by
let := design_isProbabilityMeasure D q ha hw hi
let := halfBernoulli_probability (V := V)
let := auditLaw_probability (V := V) q ⟨hw.1, hw.2.1⟩
rw [variance_eq_sub (show MemLp _ 2 D from MemLp.of_discrete),
integral_auditScore_eq_tte D θ q ha hw hi hq,
variance_eq_sub (show MemLp _ 2 (halfBernoulli V) from MemLp.of_discrete),
integral_oracleScore_eq_tte]
have hs : (∫ ω, (auditScore q (recordOf θ ω)) ^ 2 ∂D) =
(∫ z, (oracleScore θ z) ^ 2 ∂halfBernoulli V) +
4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) *
∑ i, (inNbhd θ i).card *
∫ z, (potentialOutcome θ i z) ^ 2 ∂halfBernoulli V := by
rw [design_eq_thinnedDesign D q ha hw hi, thinnedDesign,
integral_prod _ (show Integrable _ _ from Integrable.of_finite)]
simp_rw [integral_auditScore_sq_given_assignment θ q ⟨hw.1, hw.2.1⟩ hq hn]
rw [integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_const_mul, integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
simp_rw [integral_const_mul]
change (∫ ω, (auditScore q (recordOf θ ω)) ^ 2 ∂D) - (tte θ) ^ 2 =
(∫ z, (oracleScore θ z) ^ 2 ∂halfBernoulli V) - (tte θ) ^ 2 + _
rw [hs]
ring
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BaselineTranslationAffinity 19 declarations Translation affinity and common-marginal averaging
Translation affinity and common-marginal averaging
def hellingerEnergy
The unhalved squared Hellinger energy of two real densities against a common reference measure.
Definition (Lean source)
parameters
def
hellingerEnergy μ f g :
ℝ
def densityJoint
The joint mixing-coordinate and response law for a common marginal and conditional density.
Definition (Lean source)
parameters
def
def densityMixture
The response mixture after integrating out the common mixing coordinate.
Definition (Lean source)
theorem probability_density_integrable_normalized Lemma probability_density_integrable_normalized in the paper ↗
A measurable nonnegative density defining a probability law is integrable and has integral one; these side conditions are consequences of normalization. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Ω :
Type*
μ :
Measure Ω
f :
Ω → ℝ
hf :
hf0 :
∀ x, 0 ≤ f x
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f x)))
Proof (Lean source)
-- @node: probability_density_integrable_normalized
lemma probability_density_integrable_normalized {Ω : Type*} [MeasurableSpace Ω]
(μ : Measure Ω) (f : Ω → ℝ) (hf : Measurable f) (hf0 : ∀ x, 0 ≤ f x)
[IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f x)))] :
Integrable f μ ∧ (∫ x, f x ∂μ) = 1 := by
have hn : ∫⁻ x, ofReal (f x) ∂μ = 1 := by
have h := measure_univ (μ := μ.withDensity (fun x => ofReal (f x)))
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ] at h
exact h
have hnn : 0 ≤ᵐ[μ] f := Filter.Eventually.of_forall hf0
have hi : Integrable f μ :=
(lintegral_ofReal_ne_top_iff_integrable hf.aestronglyMeasurable hnn).mp
(by rw [hn]; exact ENNReal.one_ne_top)
refine ⟨hi, ?_⟩
have he := ofReal_integral_eq_lintegral_ofReal hi hnn
rw [hn] at he
have hp : 0 ≤ ∫ x, f x ∂μ := integral_nonneg hf0
exact_mod_cast (ENNReal.ofReal_eq_ofReal_iff hp (by norm_num : (0 : ℝ) ≤ 1)).mp
(by simpa using he)
theorem density_tv_le_sqrt_hellingerEnergy
Cauchy–Schwarz bounds total variation by the square root of the unhalved Hellinger energy of two probability densities. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Ω :
Type*
μ :
Measure Ω
f g :
Ω → ℝ
hf :
hg :
hf0 :
∀ x, 0 ≤ f x
hg0 :
∀ x, 0 ≤ g x
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f x)))
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (g x)))
conclusion
≤ sqrt (hellingerEnergy μ f g)
Proof (Lean source)
-- @node: density_tv_le_sqrt_hellingerEnergy
lemma density_tv_le_sqrt_hellingerEnergy {Ω : Type*} [MeasurableSpace Ω]
(μ : Measure Ω) (f g : Ω → ℝ) (hf : Measurable f) (hg : Measurable g)
(hf0 : ∀ x, 0 ≤ f x) (hg0 : ∀ x, 0 ≤ g x)
[IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f x)))]
[IsProbabilityMeasure (μ.withDensity (fun x => ofReal (g x)))] :
tvDist (μ.withDensity (fun x => ofReal (f x)))
(μ.withDensity (fun x => ofReal (g x))) ≤
sqrt (hellingerEnergy μ f g) := by
obtain ⟨hfi, hf1⟩ := probability_density_integrable_normalized μ f hf hf0
obtain ⟨hgi, hg1⟩ := probability_density_integrable_normalized μ g hg hg0
have htv := tvDist_le_sqrt_two_mul_one_sub_affinity
μ f g hfi hgi hf0 hg0 hf1 hg1
rw [← hellingerSqDensity_eq_two_mul_one_sub_affinity
μ f g hfi hgi hf0 hg0 hf1 hg1] at htv
exact htv
The baseline density is nonnegative. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: cosSqDensity_nonneg
lemma cosSqDensity_nonneg (w : ℝ) : 0 ≤ cosSqDensity w := by
unfold cosSqDensity
split <;> positivity
The compactly supported cosine density is integrable and normalized. the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: cosSqDensity_integrable_normalized
lemma cosSqDensity_integrable_normalized :
Integrable cosSqDensity ∧ (∫ w, cosSqDensity w) = 1 := by
have he : cosSqDensity = indicator (Icc (-1 / 4 : ℝ) (1 / 4))
(fun w => 4 * cos (2 * pi * w) ^ 2) := by
funext w
simp only [cosSqDensity, indicator, Set.mem_Icc]
congr 1
exact propext (by
constructor
· intro h
exact ⟨by linarith [(abs_le.mp h).1], (abs_le.mp h).2⟩
· intro h
exact abs_le.mpr ⟨by linarith [h.1], h.2⟩)
have hc : Continuous (fun w : ℝ => 4 * cos (2 * pi * w) ^ 2) := by
fun_prop
rw [he]
refine ⟨(integrable_indicator_iff measurableSet_Icc).mpr hc.continuousOn.integrableOn_Icc, ?_⟩
rw [integral_indicator measurableSet_Icc, integral_Icc_eq_integral_Ioc,
← intervalIntegral.integral_of_le (by norm_num : (-1 / 4 : ℝ) ≤ 1 / 4),
intervalIntegral.integral_const_mul,
intervalIntegral.integral_comp_mul_left (fun x => cos x ^ 2)
(by positivity : 2 * pi ≠ 0), integral_cos_sq]
have hl : 2 * pi * (-1 / 4 : ℝ) = -(pi / 2) := by ring
have hu : 2 * pi * (1 / 4 : ℝ) = pi / 2 := by ring
simp only [hl, hu, Real.cos_neg, Real.cos_pi_div_two, zero_mul, sub_zero,
smul_eq_mul]
field_simp
ring
theorem translated_cosSqDensity_integrable_normalized Lemma translated_cosSqDensity_integrable_normalized in the paper ↗
Translation preserves integrability and normalization of the baseline density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
s :
ℝ
conclusion
conclusion 1
Integrable (fun w => cosSqDensity (w - s))
conclusion 2
(∫ w, cosSqDensity (w - s)) = 1
Proof (Lean source)
-- @node: translated_cosSqDensity_integrable_normalized
lemma translated_cosSqDensity_integrable_normalized (s : ℝ) :
Integrable (fun w => cosSqDensity (w - s)) ∧
(∫ w, cosSqDensity (w - s)) = 1 := by
refine ⟨cosSqDensity_integrable_normalized.1.comp_sub_right s, ?_⟩
rw [integral_sub_right_eq_self]
exact cosSqDensity_integrable_normalized.2
Tensorized affinity and the product-defect bound give subadditivity of squared Hellinger energy for normalized nonnegative coordinate densities. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
ι :
Type*
Fintype ι
f g :
ι → ℝ → ℝ
hf :
∀ i, Integrable (f i)
hg :
∀ i, Integrable (g i)
hf0 :
∀ i x, 0 ≤ f i x
hg0 :
∀ i x, 0 ≤ g i x
hf1 :
∀ i, (∫ x, f i x) = 1
hg1 :
∀ i, (∫ x, g i x) = 1
conclusion
hellingerEnergy volume (fun x : ι → ℝ => ∏ i, f i (x i)) (fun x => ∏ i, g i (x i))
≤ ∑ i, hellingerEnergy volume (f i) (g i)
Proof (Lean source)
-- @node: hellingerEnergy_pi_le_sum
lemma hellingerEnergy_pi_le_sum {ι : Type*} [Fintype ι]
(f g : ι → ℝ → ℝ) (hf : ∀ i, Integrable (f i)) (hg : ∀ i, Integrable (g i))
(hf0 : ∀ i x, 0 ≤ f i x) (hg0 : ∀ i x, 0 ≤ g i x)
(hf1 : ∀ i, (∫ x, f i x) = 1) (hg1 : ∀ i, (∫ x, g i x) = 1) :
hellingerEnergy volume (fun x : ι → ℝ => ∏ i, f i (x i))
(fun x => ∏ i, g i (x i)) ≤ ∑ i, hellingerEnergy volume (f i) (g i) := by
have hFp : Integrable (fun x : ι → ℝ => ∏ i, f i (x i)) :=
Integrable.fintype_prod hf
have hGp : Integrable (fun x : ι → ℝ => ∏ i, g i (x i)) :=
Integrable.fintype_prod hg
have hF0 : ∀ x : ι → ℝ, 0 ≤ ∏ i, f i (x i) :=
fun x => Finset.prod_nonneg (fun i _ => hf0 i (x i))
have hG0 : ∀ x : ι → ℝ, 0 ≤ ∏ i, g i (x i) :=
fun x => Finset.prod_nonneg (fun i _ => hg0 i (x i))
have hF1 : (∫ x : ι → ℝ, ∏ i, f i (x i)) = 1 := by
change (∫ x : ι → ℝ, ∏ i, f i (x i) ∂Measure.pi (fun _ => volume)) = 1
rw [integral_fintype_prod_eq_prod]; simp [hf1]
have hG1 : (∫ x : ι → ℝ, ∏ i, g i (x i)) = 1 := by
change (∫ x : ι → ℝ, ∏ i, g i (x i) ∂Measure.pi (fun _ => volume)) = 1
rw [integral_fintype_prod_eq_prod]; simp [hg1]
have hE (i : ι) : hellingerEnergy volume (f i) (g i) =
2 * (1 - densityAffinity volume (f i) (g i)) :=
hellingerSqDensity_eq_two_mul_one_sub_affinity
volume (f i) (g i) (hf i) (hg i) (hf0 i) (hg0 i) (hf1 i) (hg1 i)
have ha0 (i : ι) : 0 ≤ densityAffinity volume (f i) (g i) :=
integral_nonneg (fun _ => Real.sqrt_nonneg _)
have ha1 (i : ι) : densityAffinity volume (f i) (g i) ≤ 1 := by
have hn : 0 ≤ hellingerEnergy volume (f i) (g i) :=
integral_nonneg (fun _ => sq_nonneg _)
rw [hE i] at hn
linarith
change hellingerSqDensity volume
(fun x : ι → ℝ => ∏ i, f i (x i)) (fun x => ∏ i, g i (x i)) ≤ _
rw [hellingerSqDensity_eq_two_mul_one_sub_affinity
volume _ _ hFp hGp hF0 hG0 hF1 hG1]
change 2 * (1 - densityAffinity (Measure.pi (fun _ : ι => volume))
(fun x => ∏ i, f i (x i)) (fun x => ∏ i, g i (x i))) ≤ _
rw [densityAffinity_pi (fun _ : ι => (volume : Measure ℝ)) f g hf0 hg0]
simp_rw [hE]
rw [← Finset.mul_sum]
exact mul_le_mul_of_nonneg_left (one_sub_prod_le_sum _ ha0 ha1)
(by norm_num)
Joint measurability of the conditional densities gives a measurable family of joint fibers.
Formal statement
Proof (Lean source)
-- @node: measurable_densityJoint_fiber
@[fun_prop] lemma measurable_densityJoint_fiber {Ξ Ω : Type*}
[MeasurableSpace Ξ] [MeasurableSpace Ω] (μ : Measure Ω) [SigmaFinite μ]
(v : Ξ → Ω → ℝ) (hv : Measurable (fun p : Ξ × Ω => v p.1 p.2)) :
Measurable (fun ξ => (μ.withDensity (fun x => ofReal (v ξ x))).map
(Prod.mk ξ)) := by
apply Measure.measurable_of_measurable_coe
intro s hs
simp_rw [Measure.map_apply measurable_prodMk_left hs,
withDensity_apply _ (measurable_prodMk_left hs),
← lintegral_indicator (measurable_prodMk_left hs)]
have hm : Measurable (s.indicator (fun p : Ξ × Ω => ofReal (v p.1 p.2))) :=
(ENNReal.measurable_ofReal.comp hv).indicator hs
simpa only [Set.indicator_apply, Set.mem_preimage] using hm.lintegral_prod_right'
The conditional-density joint law is the density-weighted product reference law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
densityJoint ν μ v = (ν.prod μ).withDensity (fun p => ofReal (v p.1 p.2))
Proof (Lean source)
-- @node: densityJoint_eq_withDensity
lemma densityJoint_eq_withDensity {Ξ Ω : Type*}
[MeasurableSpace Ξ] [MeasurableSpace Ω] (ν : Measure Ξ) [IsProbabilityMeasure ν]
(μ : Measure Ω) [SigmaFinite μ] (v : Ξ → Ω → ℝ)
(hv : Measurable (fun p : Ξ × Ω => v p.1 p.2)) :
densityJoint ν μ v = (ν.prod μ).withDensity
(fun p => ofReal (v p.1 p.2)) := by
ext s hs
rw [densityJoint, Measure.bind_apply hs
(measurable_densityJoint_fiber μ v hv).aemeasurable,
withDensity_apply _ hs, ← lintegral_indicator hs,
lintegral_prod _ (by fun_prop)]
apply lintegral_congr
intro ξ
rw [Measure.map_apply measurable_prodMk_left hs,
withDensity_apply _ (measurable_prodMk_left hs),
← lintegral_indicator (measurable_prodMk_left hs)]
rfl
Normalized conditional fibers give a probability joint law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Ξ Ω :
ν :
μ :
v :
Ξ → Ω → ℝ
hv :
Measurable (fun p : Ξ × Ω => v p.1 p.2)
hp :
∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (v ξ x)))
conclusion
IsProbabilityMeasure (densityJoint ν μ v)
Proof (Lean source)
-- @node: densityJoint_probability
lemma densityJoint_probability {Ξ Ω : Type*}
[MeasurableSpace Ξ] [MeasurableSpace Ω] (ν : Measure Ξ) [IsProbabilityMeasure ν]
(μ : Measure Ω) [SigmaFinite μ] (v : Ξ → Ω → ℝ)
(hv : Measurable (fun p : Ξ × Ω => v p.1 p.2))
(hp : ∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (v ξ x)))) :
IsProbabilityMeasure (densityJoint ν μ v) := by
constructor
rw [densityJoint, Measure.bind_apply MeasurableSet.univ
(measurable_densityJoint_fiber μ v hv).aemeasurable]
have he : ∀ ξ, (Measure.map (Prod.mk ξ)
(μ.withDensity (fun x => ofReal (v ξ x)))) univ = 1 := by
intro ξ
let _ := hp ξ
rw [Measure.map_apply measurable_prodMk_left MeasurableSet.univ]
simp
simp_rw [he]
simp
The sum-reference Hellinger definition agrees with a sigma-finite density reference. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Ω :
Type*
μ :
f g :
Ω → ℝ
hf :
hg :
hf0 :
∀ x, 0 ≤ f x
hg0 :
∀ x, 0 ≤ g x
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f x)))
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (g x)))
conclusion
= hellingerEnergy μ f g
Proof (Lean source)
-- @node: hellingerSqMeasure_eq_density
lemma hellingerSqMeasure_eq_density {Ω : Type*} [MeasurableSpace Ω]
(μ : Measure Ω) [SigmaFinite μ] (f g : Ω → ℝ)
(hf : Measurable f) (hg : Measurable g) (hf0 : ∀ x, 0 ≤ f x) (hg0 : ∀ x, 0 ≤ g x)
[IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f x)))]
[IsProbabilityMeasure (μ.withDensity (fun x => ofReal (g x)))] :
hellingerSqMeasure
(μ.withDensity (fun x => ofReal (f x)))
(μ.withDensity (fun x => ofReal (g x))) = hellingerEnergy μ f g := by
let B := μ.withDensity (fun x => ofReal (f x))
let C := μ.withDensity (fun x => ofReal (g x))
have hb : B ≪ μ := withDensity_absolutelyContinuous μ _
have hc : C ≪ μ := withDensity_absolutelyContinuous μ _
have hbs : B ≪ B + C := Measure.AbsolutelyContinuous.rfl.add_right C
have hcs : C ≪ B + C := Measure.AbsolutelyContinuous.rfl.add_right' B
have hbr := Measure.rnDeriv_mul_rnDeriv (κ := μ) hbs
have hcr := Measure.rnDeriv_mul_rnDeriv (κ := μ) hcs
have hbf : B.rnDeriv μ =ᵐ[μ] fun x => ofReal (f x) :=
Measure.rnDeriv_withDensity μ (by fun_prop)
have hcg : C.rnDeriv μ =ᵐ[μ] fun x => ofReal (g x) :=
Measure.rnDeriv_withDensity μ (by fun_prop)
change (∫ x, (sqrt ((B.rnDeriv (B + C) x).toReal) -
sqrt ((C.rnDeriv (B + C) x).toReal)) ^ 2 ∂(B + C)) = _
rw [← integral_toReal_rnDeriv_mul (hb.add_left hc)]
apply integral_congr_ae
filter_upwards [hbr, hcr, hbf, hcg] with x hxB hxC hxF hxG
have hF : f x = (B.rnDeriv (B + C) x).toReal * ((B + C).rnDeriv μ x).toReal := by
rw [← ENNReal.toReal_ofReal (hf0 x), ← hxF, ← hxB, Pi.mul_apply,
ENNReal.toReal_mul]
have hG : g x = (C.rnDeriv (B + C) x).toReal * ((B + C).rnDeriv μ x).toReal := by
rw [← ENNReal.toReal_ofReal (hg0 x), ← hxG, ← hxC, Pi.mul_apply,
ENNReal.toReal_mul]
change _ = (sqrt (f x) - sqrt (g x)) ^ 2
rw [hF, hG, Real.sqrt_mul ENNReal.toReal_nonneg,
Real.sqrt_mul ENNReal.toReal_nonneg, ← sub_mul, mul_pow,
Real.sq_sqrt ENNReal.toReal_nonneg]
ring
Marginalizing the joint law recovers its response mixture. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: densityJoint_map_snd
lemma densityJoint_map_snd {Ξ Ω : Type*}
[MeasurableSpace Ξ] [MeasurableSpace Ω] (ν : Measure Ξ) [IsProbabilityMeasure ν]
(μ : Measure Ω) [SigmaFinite μ] (v : Ξ → Ω → ℝ)
(hv : Measurable (fun p : Ξ × Ω => v p.1 p.2)) :
(densityJoint ν μ v).map snd = densityMixture ν μ v := by
have hm : Measurable (fun ξ => μ.withDensity (fun x => ofReal (v ξ x))) := by
apply measurable_withDensity
exact ENNReal.measurable_ofReal.comp hv
ext s hs
rw [Measure.map_apply measurable_snd hs, densityJoint, densityMixture,
Measure.bind_apply (measurable_snd hs) (measurable_densityJoint_fiber μ v hv).aemeasurable,
Measure.bind_apply hs hm.aemeasurable]
apply lintegral_congr
intro ξ
rw [Measure.map_apply measurable_prodMk_left (measurable_snd hs)]
rfl
theorem common_marginal_hellinger_averaging
Averaging conditional Hellinger energy over the actual common marginal gives joint Hellinger energy and bounds the response-mixture total variation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Ξ Ω :
ν :
μ :
f g :
Ξ → Ω → ℝ
hf :
Measurable (fun p : Ξ × Ω => f p.1 p.2)
hg :
Measurable (fun p : Ξ × Ω => g p.1 p.2)
hf0 :
∀ ξ x, 0 ≤ f ξ x
hg0 :
∀ ξ x, 0 ≤ g ξ x
hfp :
∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f ξ x)))
hgp :
∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (g ξ x)))
conclusion
conclusion 1
= (∫ ξ, hellingerEnergy μ (f ξ) (g ξ) ∂ν)
conclusion 2
≤ sqrt (∫ ξ, hellingerEnergy μ (f ξ) (g ξ) ∂ν)
Proof (Lean source)
-- @node: common_marginal_hellinger_averaging
lemma common_marginal_hellinger_averaging {Ξ Ω : Type*}
[MeasurableSpace Ξ] [MeasurableSpace Ω] (ν : Measure Ξ) [IsProbabilityMeasure ν]
(μ : Measure Ω) [SigmaFinite μ] (f g : Ξ → Ω → ℝ)
(hf : Measurable (fun p : Ξ × Ω => f p.1 p.2))
(hg : Measurable (fun p : Ξ × Ω => g p.1 p.2))
(hf0 : ∀ ξ x, 0 ≤ f ξ x) (hg0 : ∀ ξ x, 0 ≤ g ξ x)
(hfp : ∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (f ξ x))))
(hgp : ∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (g ξ x)))) :
hellingerSqMeasure (densityJoint ν μ f) (densityJoint ν μ g) =
(∫ ξ, hellingerEnergy μ (f ξ) (g ξ) ∂ν) ∧
tvDist (densityMixture ν μ f) (densityMixture ν μ g) ≤
sqrt (∫ ξ, hellingerEnergy μ (f ξ) (g ξ) ∂ν) := by
let _ := densityJoint_probability ν μ f hf hfp
let _ := densityJoint_probability ν μ g hg hgp
have hpf : IsProbabilityMeasure ((ν.prod μ).withDensity
(fun p => ofReal (f p.1 p.2))) := by
rw [← densityJoint_eq_withDensity ν μ f hf]
infer_instance
have hpg : IsProbabilityMeasure ((ν.prod μ).withDensity
(fun p => ofReal (g p.1 p.2))) := by
rw [← densityJoint_eq_withDensity ν μ g hg]
infer_instance
let _ := hpf
let _ := hpg
have hfi := (probability_density_integrable_normalized (ν.prod μ)
(fun p => f p.1 p.2) hf (fun p => hf0 p.1 p.2)).1
have hgi := (probability_density_integrable_normalized (ν.prod μ)
(fun p => g p.1 p.2) hg (fun p => hg0 p.1 p.2)).1
have he : hellingerEnergy (ν.prod μ) (fun p => f p.1 p.2) (fun p => g p.1 p.2) =
(∫ ξ, hellingerEnergy μ (f ξ) (g ξ) ∂ν) := by
exact integral_prod _ (integrable_sqrt_sum_sub_sq
(ν.prod μ) _ _ hfi hgi (fun p => hf0 p.1 p.2) (fun p => hg0 p.1 p.2)).2
refine ⟨?_, ?_⟩
· rw [densityJoint_eq_withDensity ν μ f hf, densityJoint_eq_withDensity ν μ g hg,
hellingerSqMeasure_eq_density (ν.prod μ) _ _ hf hg
(fun p => hf0 p.1 p.2) (fun p => hg0 p.1 p.2), he]
· have htv : tvDist (densityJoint ν μ f) (densityJoint ν μ g) ≤
sqrt (∫ ξ, hellingerEnergy μ (f ξ) (g ξ) ∂ν) := by
rw [densityJoint_eq_withDensity ν μ f hf, densityJoint_eq_withDensity ν μ g hg,
← he]
exact density_tv_le_sqrt_hellingerEnergy (ν.prod μ) _ _ hf hg
(fun p => hf0 p.1 p.2) (fun p => hg0 p.1 p.2)
rw [← densityJoint_map_snd ν μ f hf, ← densityJoint_map_snd ν μ g hg]
apply le_trans ?_ htv
unfold tvDist
apply ciSup_le
rintro ⟨s, hs⟩
rw [map_measureReal_apply_of_aemeasurable measurable_snd.aemeasurable hs,
map_measureReal_apply_of_aemeasurable measurable_snd.aemeasurable hs]
exact abs_measureReal_sub_le_tvDist (measurable_snd hs)
The pointwise derivative of the square-root baseline away from its two support boundaries, with the arbitrary boundary values fixed to zero.
theorem sqrt_cosSqDensity_hasDerivAt
Away from the two support boundaries, the square-root baseline has the explicit supported sine derivative. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
w :
ℝ
hneg :
w ≠ -(1 / 4 : ℝ)
hpos :
w ≠ (1 / 4 : ℝ)
conclusion
Proof (Lean source)
-- @node: sqrt_cosSqDensity_hasDerivAt
lemma sqrt_cosSqDensity_hasDerivAt (w : ℝ)
(hneg : w ≠ -(1 / 4 : ℝ)) (hpos : w ≠ (1 / 4 : ℝ)) :
HasDerivAt (fun x => sqrt (cosSqDensity x)) (baselineSqrtDerivAt w) w := by
rcases lt_trichotomy w (-(1 / 4 : ℝ)) with hw | hw | hw
· have habs : ¬ |w| < (1 / 4 : ℝ) := by
rw [not_lt]
exact (le_of_lt (lt_neg.mpr hw)).trans (neg_le_abs w)
have heq : (fun x => sqrt (cosSqDensity x)) =ᶠ[nhds w]
(fun _ => (0 : ℝ)) := by
filter_upwards [isOpen_Iio.eventually_mem hw] with x hx
have hxabs : ¬ |x| ≤ (1 / 4 : ℝ) := by
rw [not_le]
exact (lt_neg.mpr hx).trans_le (neg_le_abs x)
rw [cosSqDensity, if_neg hxabs, Real.sqrt_zero]
rw [baselineSqrtDerivAt, if_neg habs]
exact (hasDerivAt_const (x := w) (c := (0 : ℝ))).congr_of_eventuallyEq heq
· exact (hneg hw).elim
· rcases lt_trichotomy w (1 / 4 : ℝ) with hwin | hw | hw
· have habs : |w| < (1 / 4 : ℝ) := abs_lt.mpr ⟨hw, hwin⟩
have heq : (fun x => sqrt (cosSqDensity x)) =ᶠ[nhds w]
(fun x => 2 * cos (2 * pi * x)) := by
filter_upwards [isOpen_Ioo.eventually_mem ⟨hw, hwin⟩] with x hx
rw [sqrt_cosSqDensity_eq_clamped_cos, min_eq_right hx.2.le,
max_eq_right (by linarith [hx.1])]
have hderiv : HasDerivAt (fun x => 2 * cos (2 * pi * x))
(-4 * pi * sin (2 * pi * w)) w := by
convert ((Real.hasDerivAt_cos (2 * pi * w)).comp w
((hasDerivAt_id w).const_mul (2 * pi))).const_mul 2 using 1 <;>
first | rfl | ring
rw [baselineSqrtDerivAt, if_pos habs]
exact hderiv.congr_of_eventuallyEq heq
· exact (hpos hw).elim
· have habs : ¬ |w| < (1 / 4 : ℝ) := by
rw [not_lt]
exact (le_of_lt hw).trans (le_abs_self w)
have heq : (fun x => sqrt (cosSqDensity x)) =ᶠ[nhds w]
(fun _ => (0 : ℝ)) := by
filter_upwards [isOpen_Ioi.eventually_mem hw] with x hx
have hxabs : ¬ |x| ≤ (1 / 4 : ℝ) := by
rw [not_le]
exact hx.trans_le (le_abs_self x)
rw [cosSqDensity, if_neg hxabs, Real.sqrt_zero]
rw [baselineSqrtDerivAt, if_neg habs]
exact (hasDerivAt_const (x := w) (c := (0 : ℝ))).congr_of_eventuallyEq heq
theorem baselineSqrtDerivAt_energy
The explicitly tied square-root derivative has total squared energy four pi squared. the stated conclusion holds.
Formal statement
conclusion
(∫ w, baselineSqrtDerivAt w ^ 2) = 4 * pi ^ 2
Proof (Lean source)
-- @node: baselineSqrtDerivAt_energy
lemma baselineSqrtDerivAt_energy :
(∫ w, baselineSqrtDerivAt w ^ 2) = 4 * pi ^ 2 := by
have he : (fun w => baselineSqrtDerivAt w ^ 2) =
(Ioo (-1 / 4 : ℝ) (1 / 4)).indicator
(fun w => (-4 * pi * sin (2 * pi * w)) ^ 2) := by
funext w
by_cases hw : |w| < (1 / 4 : ℝ)
· have hmem : w ∈ Ioo (-1 / 4 : ℝ) (1 / 4) := by
constructor <;> linarith [(abs_lt.mp hw).1, (abs_lt.mp hw).2]
rw [baselineSqrtDerivAt, if_pos hw, Set.indicator_of_mem hmem]
· have hmem : w ∉ Ioo (-1 / 4 : ℝ) (1 / 4) := by
intro hmem
apply hw
exact abs_lt.mpr ⟨by linarith [hmem.1], hmem.2⟩
rw [baselineSqrtDerivAt, if_neg hw, Set.indicator_of_notMem hmem]
norm_num
rw [he, integral_indicator measurableSet_Ioo, ← integral_Icc_eq_integral_Ioo]
have hsquare : (fun w : ℝ => (-4 * pi * sin (2 * pi * w)) ^ 2) =
(fun w => (4 * pi * sin (2 * pi * w)) ^ 2) := by
funext w
ring
rw [hsquare]
exact baseline_derivative_energy
theorem baseline_translation_affinity
Lipschitz and derivative energy, translation and product bounds, and common-marginal Hellinger averaging. the stated conclusion holds.
Formal statement
conclusion
conclusion 1
conclusion 4
B :
ℕ
s t :
Fin B → ℝ
(∫ y : Fin B → ℝ, (sqrt (∏ ℓ, cosSqDensity (y ℓ - s ℓ)) - sqrt (∏ ℓ, cosSqDensity (y ℓ - t ℓ))) ^ 2)
≤ ∑ ℓ, 4 * pi ^ 2 * (s ℓ - t ℓ) ^ 2
conclusion 5
Ω :
Type
μ :
Measure Ω
vPlus vMinus :
Ω → ℝ
Measurable vPlus
Measurable vMinus
(∀ x, 0 ≤ vPlus x)
(∀ x, 0 ≤ vMinus x)
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vPlus x)))
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vMinus x)))
≤ sqrt (hellingerEnergy μ vPlus vMinus)
conclusion 6
Ξ Ω :
Type
ν :
Measure Ξ
μ :
Measure Ω
vPlus vMinus :
Ξ → Ω → ℝ
Measurable (fun x : Ξ × Ω => vPlus x.1 x.2)
Measurable (fun x : Ξ × Ω => vMinus x.1 x.2)
(∀ ξ x, 0 ≤ vPlus ξ x)
(∀ ξ x, 0 ≤ vMinus ξ x)
(∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vPlus ξ x))))
(∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vMinus ξ x))))
= (∫ ξ, hellingerEnergy μ (vPlus ξ) (vMinus ξ) ∂ν)
≤ sqrt (∫ ξ, hellingerEnergy μ (vPlus ξ) (vMinus ξ) ∂ν)
conclusion 7
w :
ℝ
w ≠ -(1 / 4 : ℝ)
w ≠ (1 / 4 : ℝ)
conclusion 8
(∫ w, baselineSqrtDerivAt w ^ 2) = 4 * pi ^ 2
Proof (Lean source)
-- @node: lem:baseline-translation-affinity
lemma baseline_translation_affinity :
LipschitzWith ((4 * pi).toNNReal) (fun w => sqrt (cosSqDensity w)) ∧
(∫ w in Icc (-1 / 4 : ℝ) (1 / 4), (4 * pi * sin (2 * pi * w)) ^ 2) =
4 * pi ^ 2 ∧
(∀ s t : ℝ, (∫ w, (sqrt (cosSqDensity (w - s)) -
sqrt (cosSqDensity (w - t))) ^ 2) ≤ 4 * pi ^ 2 * (s - t) ^ 2) ∧
(∀ (B : ℕ) (s t : Fin B → ℝ),
(∫ y : Fin B → ℝ, (sqrt (∏ ℓ, cosSqDensity (y ℓ - s ℓ)) -
sqrt (∏ ℓ, cosSqDensity (y ℓ - t ℓ))) ^ 2) ≤
∑ ℓ, 4 * pi ^ 2 * (s ℓ - t ℓ) ^ 2) ∧
(∀ (Ω : Type) [MeasurableSpace Ω] (μ : Measure Ω) [SigmaFinite μ]
(vPlus vMinus : Ω → ℝ), Measurable vPlus → Measurable vMinus →
(∀ x, 0 ≤ vPlus x) → (∀ x, 0 ≤ vMinus x) →
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vPlus x))) →
IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vMinus x))) →
tvDist (μ.withDensity (fun x => ofReal (vPlus x)))
(μ.withDensity (fun x => ofReal (vMinus x))) ≤
sqrt (hellingerEnergy μ vPlus vMinus)) ∧
(∀ (Ξ Ω : Type) [MeasurableSpace Ξ] [MeasurableSpace Ω]
(ν : Measure Ξ) [IsProbabilityMeasure ν] (μ : Measure Ω) [SigmaFinite μ]
(vPlus vMinus : Ξ → Ω → ℝ),
Measurable (fun x : Ξ × Ω => vPlus x.1 x.2) →
Measurable (fun x : Ξ × Ω => vMinus x.1 x.2) →
(∀ ξ x, 0 ≤ vPlus ξ x) → (∀ ξ x, 0 ≤ vMinus ξ x) →
(∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vPlus ξ x)))) →
(∀ ξ, IsProbabilityMeasure (μ.withDensity (fun x => ofReal (vMinus ξ x)))) →
hellingerSqMeasure (densityJoint ν μ vPlus) (densityJoint ν μ vMinus) =
(∫ ξ, hellingerEnergy μ (vPlus ξ) (vMinus ξ) ∂ν) ∧
tvDist (densityMixture ν μ vPlus) (densityMixture ν μ vMinus) ≤
sqrt (∫ ξ, hellingerEnergy μ (vPlus ξ) (vMinus ξ) ∂ν)) ∧
(∀ w : ℝ, w ≠ -(1 / 4 : ℝ) → w ≠ (1 / 4 : ℝ) →
HasDerivAt (fun x => sqrt (cosSqDensity x)) (baselineSqrtDerivAt w) w) ∧
(∫ w, baselineSqrtDerivAt w ^ 2) = 4 * pi ^ 2 := by
refine ⟨baseline_sqrt_lipschitz, baseline_derivative_energy, ?_⟩
have htrans : ∀ s t : ℝ, (∫ w, (sqrt (cosSqDensity (w - s)) -
sqrt (cosSqDensity (w - t))) ^ 2) ≤ 4 * pi ^ 2 * (s - t) ^ 2 := by
exact baseline_translation_energy
refine ⟨htrans, ?_, ?_, ?_, ?_, baselineSqrtDerivAt_energy⟩
· intro B s t
have hp := hellingerEnergy_pi_le_sum
(fun i w => cosSqDensity (w - s i)) (fun i w => cosSqDensity (w - t i))
(fun i => (translated_cosSqDensity_integrable_normalized (s i)).1)
(fun i => (translated_cosSqDensity_integrable_normalized (t i)).1)
(fun _ _ => cosSqDensity_nonneg _) (fun _ _ => cosSqDensity_nonneg _)
(fun i => (translated_cosSqDensity_integrable_normalized (s i)).2)
(fun i => (translated_cosSqDensity_integrable_normalized (t i)).2)
exact hp.trans (Finset.sum_le_sum (fun i _ => htrans (s i) (t i)))
· intro Ω _ μ _ f g hf hg hf0 hg0 hfp hgp
let _ := hfp
let _ := hgp
exact density_tv_le_sqrt_hellingerEnergy μ f g hf hg hf0 hg0
· intro Ξ Ω _ _ ν _ μ _ f g hf hg hf0 hg0 hfp hgp
exact common_marginal_hellinger_averaging ν μ f g hf hg hf0 hg0 hfp hgp
· exact fun w hneg hpos => sqrt_cosSqDensity_hasDerivAt w hneg hpos
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BaselineTranslationEnergy 10 declarations The compactly supported cosine square root has a supported sine derivative.
Square-root baseline derivative and translation energy
The compactly supported cosine square root has a supported sine derivative. Its integral representation, interval Cauchy–Schwarz, and Fubini yield the sharp quadratic translation bound used by the block-testing construction.
Clamping to the support interval expresses the square-root density as a cosine, including its zero endpoint values. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sqrt_cosSqDensity_eq_clamped_cos
lemma sqrt_cosSqDensity_eq_clamped_cos (w : ℝ) :
sqrt (cosSqDensity w) =
2 * cos (2 * pi * max (-1 / 4) (min (1 / 4) w)) := by
by_cases hw : |w| ≤ 1 / 4
· have hb := abs_le.mp hw
rw [min_eq_right hb.2, max_eq_right (by linarith : (-1 / 4 : ℝ) ≤ w), cosSqDensity, if_pos hw]
have hc : 0 ≤ cos (2 * pi * w) := by
apply Real.cos_nonneg_of_mem_Icc
constructor <;> nlinarith [Real.pi_pos]
have hs : 4 * cos (2 * pi * w) ^ 2 =
(2 * cos (2 * pi * w)) ^ 2 := by ring
rw [hs, Real.sqrt_sq (mul_nonneg (by norm_num) hc)]
· rw [cosSqDensity, if_neg hw, Real.sqrt_zero]
rcases lt_or_ge w (-1 / 4) with hlow | hlow
· rw [min_eq_right (by linarith : w ≤ (1 / 4 : ℝ)), max_eq_left hlow.le]
have he : 2 * pi * (-1 / 4 : ℝ) = -(pi / 2) := by ring
rw [he, Real.cos_neg, Real.cos_pi_div_two, mul_zero]
· have hhigh : (1 / 4 : ℝ) ≤ w := by
by_contra hh
exact hw (abs_le.mpr ⟨by linarith, le_of_not_ge hh⟩)
rw [min_eq_left hhigh, max_eq_right (by norm_num : (-1 / 4 : ℝ) ≤ 1 / 4)]
have he : 2 * pi * (1 / 4 : ℝ) = pi / 2 := by ring
rw [he, Real.cos_pi_div_two, mul_zero]
The square-root baseline is globally four-pi Lipschitz, including across the boundary of its compact support. the stated conclusion holds.
Formal statement
conclusion
Proof (Lean source)
-- @node: baseline_sqrt_lipschitz
lemma baseline_sqrt_lipschitz :
LipschitzWith ((4 * pi).toNNReal) (fun w => sqrt (cosSqDensity w)) := by
have hclamp : LipschitzWith 1 (fun w : ℝ => max (-1 / 4) (min (1 / 4) w)) :=
(LipschitzWith.id.const_min (1 / 4)).const_max (-1 / 4)
apply LipschitzWith.of_dist_le_mul
intro x y
rw [sqrt_cosSqDensity_eq_clamped_cos, sqrt_cosSqDensity_eq_clamped_cos,
Real.dist_eq, Real.dist_eq, Real.coe_toNNReal _ (by positivity)]
have hc := hclamp.dist_le_mul x y
simp only [NNReal.coe_one, one_mul, Real.dist_eq] at hc
calc
|2 * cos (2 * pi * max (-1 / 4) (min (1 / 4) x)) -
2 * cos (2 * pi * max (-1 / 4) (min (1 / 4) y))| =
2 * |cos (2 * pi * max (-1 / 4) (min (1 / 4) x)) -
cos (2 * pi * max (-1 / 4) (min (1 / 4) y))| := by
rw [← mul_sub, abs_mul, abs_of_nonneg (by norm_num : (0 : ℝ) ≤ 2)]
_ ≤ 2 * |2 * pi * max (-1 / 4) (min (1 / 4) x) -
2 * pi * max (-1 / 4) (min (1 / 4) y)| :=
mul_le_mul_of_nonneg_left (Real.abs_cos_sub_cos_le _ _) (by norm_num)
_ = 4 * pi * |max (-1 / 4) (min (1 / 4) x) -
max (-1 / 4) (min (1 / 4) y)| := by
rw [← mul_sub, abs_mul, abs_of_nonneg (by positivity : 0 ≤ 2 * pi)]
ring
_ ≤ 4 * pi * |x - y| := mul_le_mul_of_nonneg_left hc (by positivity)
Direct integration of the squared interior derivative gives four pi squared. the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: baseline_derivative_energy
lemma baseline_derivative_energy :
(∫ w in Icc (-1 / 4 : ℝ) (1 / 4),
(4 * pi * sin (2 * pi * w)) ^ 2) = 4 * pi ^ 2 := by
rw [integral_Icc_eq_integral_Ioc,
← intervalIntegral.integral_of_le (by norm_num : (-1 / 4 : ℝ) ≤ 1 / 4)]
simp_rw [mul_pow]
rw [intervalIntegral.integral_const_mul]
rw [intervalIntegral.integral_comp_mul_left (fun x => sin x ^ 2)
(by positivity : 2 * pi ≠ 0)]
rw [integral_sin_sq]
have hl : 2 * pi * (-1 / 4 : ℝ) = -(pi / 2) := by ring
have hu : 2 * pi * (1 / 4 : ℝ) = pi / 2 := by ring
simp only [hl, hu, Real.cos_neg, Real.cos_pi_div_two, mul_zero, sub_zero,
smul_eq_mul]
field_simp
ring
Cauchy–Schwarz on a finite measure bounds the squared integral by mass times energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ x, g x ∂μ) ^ 2 ≤ μ.real univ * ∫ x, g x ^ 2 ∂μ
Proof (Lean source)
-- @node: baseline_interval_cauchy_schwarz
lemma baseline_interval_cauchy_schwarz {α : Type*} [MeasurableSpace α] (μ : Measure α) [IsFiniteMeasure μ]
(g : α → ℝ) (hg : Integrable g μ) (hg2 : Integrable (fun x => g x ^ 2) μ) :
(∫ x, g x ∂μ) ^ 2 ≤ μ.real univ * ∫ x, g x ^ 2 ∂μ := by
let m := μ.real univ
let I := ∫ x, g x ∂μ
have hm0 : 0 ≤ m := measureReal_nonneg
by_cases hm : m = 0
· have hμ : μ = 0 := by
apply Measure.measure_univ_eq_zero.mp
exact ((ENNReal.toReal_eq_zero_iff _).mp hm).resolve_right (measure_ne_top μ univ)
simp [hμ]
· have hmp : 0 < m := lt_of_le_of_ne hm0 (Ne.symm hm)
have hex : (fun x => (m * g x - I) ^ 2) =
(fun x => m ^ 2 * g x ^ 2 - (2 * m * I) * g x + I ^ 2) := by
funext x; ring
have hn : 0 ≤ ∫ x, (m * g x - I) ^ 2 ∂μ := integral_nonneg (fun x => sq_nonneg _)
rw [hex] at hn
have hs : Integrable (fun x => m ^ 2 * g x ^ 2 - (2 * m * I) * g x) μ :=
(hg2.const_mul _).sub (hg.const_mul _)
rw [integral_add hs (integrable_const (I ^ 2)),
integral_sub (hg2.const_mul (m ^ 2)) (hg.const_mul (2 * m * I)),
integral_const_mul, integral_const_mul, integral_const] at hn
change 0 ≤ m ^ 2 * (∫ x, g x ^ 2 ∂μ) - (2 * m * I) * I + m * I ^ 2 at hn
have hx : 0 ≤ m * (m * (∫ x, g x ^ 2 ∂μ) - I ^ 2) := by nlinarith [hn]
have := nonneg_of_mul_nonneg_right hx hmp
dsimp [m, I] at this
linarith
theorem translation_energy_le_of_integral_representation Lemma translation_energy_le_of_integral_representation in the paper ↗
Integrating the interval Cauchy–Schwarz bound and applying Fubini controls translation energy by derivative energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
f g :
ℝ → ℝ
hg :
hgi :
hg2 :
Integrable (fun x => g x ^ 2)
hrep :
∀ a b
if
a ≤ b
then
(∫ x in a..b, g x) = f b - f a
a b :
ℝ
hab :
a ≤ b
conclusion
(∫ w, (f (w + b) - f (w + a)) ^ 2) ≤ (b - a) ^ 2 * (∫ w, g w ^ 2)
Proof (Lean source)
-- @node: translation_energy_le_of_integral_representation
lemma translation_energy_le_of_integral_representation (f g : ℝ → ℝ) (hg : Measurable g) (hgi : Integrable g)
(hg2 : Integrable (fun x => g x ^ 2))
(hrep : ∀ a b, a ≤ b → (∫ x in a..b, g x) = f b - f a)
(a b : ℝ) (hab : a ≤ b) :
(∫ w, (f (w + b) - f (w + a)) ^ 2) ≤
(b - a) ^ 2 * (∫ w, g w ^ 2) := by
let ν := volume.restrict (Ioc a b)
have hν : IsFiniteMeasure ν := inferInstance
have hprod : Integrable (fun p : ℝ × ℝ => g (p.1 + p.2) ^ 2) (volume.prod ν) := by
apply (integrable_prod_iff' ((hg.comp (measurable_fst.add measurable_snd)).pow_const 2).aestronglyMeasurable).mpr
refine ⟨Filter.Eventually.of_forall (fun v => hg2.comp_add_right v), ?_⟩
have he : (fun v => ∫ w, ‖g (w + v) ^ 2‖) = (fun _ : ℝ => ∫ w, g w ^ 2) := by
funext v
simp only [Real.norm_eq_abs, abs_sq]
exact integral_add_right_eq_self (fun w => g w ^ 2) v
change Integrable (fun v => ∫ w, ‖g (w + v) ^ 2‖) ν
rw [he]
exact integrable_const _
have hpoint (w : ℝ) : (f (w + b) - f (w + a)) ^ 2 ≤
(b - a) * ∫ v, g (w + v) ^ 2 ∂ν := by
have hi : Integrable (fun v => g (w + v)) ν :=
(hgi.comp_add_left w).restrict
have hi2 : Integrable (fun v => g (w + v) ^ 2) ν :=
(hg2.comp_add_left w).restrict
have he : (∫ v, g (w + v) ∂ν) = f (w + b) - f (w + a) := by
rw [← intervalIntegral.integral_of_le hab,
intervalIntegral.integral_comp_add_left g w]
exact hrep _ _ (add_le_add_right hab w)
have hc := baseline_interval_cauchy_schwarz ν (fun v => g (w + v)) hi hi2
rw [he] at hc
simpa [ν, measureReal_restrict_apply, Real.volume_real_Ioc_of_le hab] using hc
calc
(∫ w, (f (w + b) - f (w + a)) ^ 2) ≤
∫ w, (b - a) * ∫ v, g (w + v) ^ 2 ∂ν :=
integral_mono_of_nonneg (Filter.Eventually.of_forall (fun _ => sq_nonneg _))
(hprod.integral_prod_left.const_mul _) (Filter.Eventually.of_forall hpoint)
_ = (b - a) ^ 2 * (∫ w, g w ^ 2) := by
rw [integral_const_mul, integral_integral_swap hprod]
simp_rw [integral_add_right_eq_self (fun w => g w ^ 2)]
simp [ν, measureReal_restrict_apply, Real.volume_real_Ioc_of_le hab, pow_two, mul_assoc]
The supported interior derivative of the square-root cosine baseline; endpoint values are immaterial to integration.
theorem baselineSqrtDerivative_integrable
The supported derivative and its square are integrable by compact support. the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: baselineSqrtDerivative_integrable
lemma baselineSqrtDerivative_integrable : Integrable baselineSqrtDerivative ∧ Integrable (fun w => baselineSqrtDerivative w ^ 2) := by
have hc : Continuous (fun w : ℝ => -4 * pi * sin (2 * pi * w)) := by fun_prop
constructor
· exact (integrable_indicator_iff measurableSet_Icc).mpr hc.continuousOn.integrableOn_Icc
· have he : (fun w => baselineSqrtDerivative w ^ 2) =
(Icc (-1 / 4 : ℝ) (1 / 4)).indicator
(fun w => (-4 * pi * sin (2 * pi * w)) ^ 2) := by
funext w
by_cases hw : w ∈ Icc (-1 / 4 : ℝ) (1 / 4)
· simp only [baselineSqrtDerivative, Set.indicator_of_mem hw]
· simp only [baselineSqrtDerivative, Set.indicator_of_notMem hw, zero_pow, ne_eq,
OfNat.ofNat_ne_zero, not_false_eq_true]
rw [he]
exact (integrable_indicator_iff measurableSet_Icc).mpr
(hc.pow 2).continuousOn.integrableOn_Icc
theorem baselineSqrtDerivative_energy
The supported derivative has total squared energy four pi squared. the stated conclusion holds.
Formal statement
conclusion
(∫ w, baselineSqrtDerivative w ^ 2) = 4 * pi ^ 2
Proof (Lean source)
-- @node: baselineSqrtDerivative_energy
lemma baselineSqrtDerivative_energy : (∫ w, baselineSqrtDerivative w ^ 2) = 4 * pi ^ 2 := by
have he : (fun w => baselineSqrtDerivative w ^ 2) =
(Icc (-1 / 4 : ℝ) (1 / 4)).indicator
(fun w => (4 * pi * sin (2 * pi * w)) ^ 2) := by
funext w
by_cases hw : w ∈ Icc (-1 / 4 : ℝ) (1 / 4)
· simp only [baselineSqrtDerivative, Set.indicator_of_mem hw]; ring
· simp only [baselineSqrtDerivative, Set.indicator_of_notMem hw, zero_pow, ne_eq,
OfNat.ofNat_ne_zero, not_false_eq_true]
rw [he, integral_indicator measurableSet_Icc]
exact baseline_derivative_energy
theorem baselineSqrtDerivative_integral
Integration of the supported derivative recovers square-root baseline increments, including across the support boundaries. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: baselineSqrtDerivative_integral
lemma baselineSqrtDerivative_integral (a b : ℝ) (hab : a ≤ b) :
(∫ w in a..b, baselineSqrtDerivative w) =
sqrt (cosSqDensity b) - sqrt (cosSqDensity a) := by
have he : (∫ w in a..b, baselineSqrtDerivative w) =
∫ w in Icc (max (-1 / 4) a) (min (1 / 4) b),
-4 * pi * sin (2 * pi * w) := by
rw [intervalIntegral.integral_of_le hab, ← integral_Icc_eq_integral_Ioc]
unfold baselineSqrtDerivative
rw [integral_indicator measurableSet_Icc, Measure.restrict_restrict measurableSet_Icc,
Set.Icc_inter_Icc]
rw [he, sqrt_cosSqDensity_eq_clamped_cos, sqrt_cosSqDensity_eq_clamped_cos]
rcases lt_or_ge b (-1 / 4) with hb | hb
· have hempty : Icc (max (-1 / 4 : ℝ) a) (min (1 / 4) b) = ∅ :=
Set.Icc_eq_empty_of_lt (lt_of_le_of_lt (min_le_right _ _) (hb.trans_le (le_max_left _ _)))
rw [hempty]
simp only [Measure.restrict_empty, integral_zero_measure]
rw [min_eq_right (by linarith : b ≤ (1 / 4 : ℝ)),
max_eq_left hb.le, min_eq_right (by linarith : a ≤ (1 / 4 : ℝ)),
max_eq_left (by linarith : a ≤ (-1 / 4 : ℝ))]
ring
rcases lt_or_ge (1 / 4 : ℝ) a with ha | ha
· have hempty : Icc (max (-1 / 4 : ℝ) a) (min (1 / 4) b) = ∅ :=
Set.Icc_eq_empty_of_lt (lt_of_le_of_lt (min_le_left _ _) (ha.trans_le (le_max_right _ _)))
rw [hempty]
simp only [Measure.restrict_empty, integral_zero_measure]
rw [min_eq_left (by linarith : (1 / 4 : ℝ) ≤ b),
max_eq_right (by norm_num : (-1 / 4 : ℝ) ≤ 1 / 4), min_eq_left ha.le,
max_eq_right (by norm_num : (-1 / 4 : ℝ) ≤ 1 / 4)]
ring
have hLU : max (-1 / 4 : ℝ) a ≤ min (1 / 4) b := by
exact max_le (le_min (by norm_num) hb) (le_min ha hab)
rw [min_eq_right ha, max_eq_right (le_min (by norm_num) hb),
integral_Icc_eq_integral_Ioc, ← intervalIntegral.integral_of_le hLU]
rw [max_comm (-1 / 4 : ℝ) a]
apply intervalIntegral.integral_eq_sub_of_hasDerivAt
(f := fun w => 2 * cos (2 * pi * w))
· intro w hw
convert ((Real.hasDerivAt_cos (2 * pi * w)).comp w
((hasDerivAt_id w).const_mul (2 * pi))).const_mul 2 using 1 <;> first | rfl | ring
· exact (show Continuous (fun w : ℝ => -4 * pi * sin (2 * pi * w)) by
fun_prop).intervalIntegrable _ _
The square-root baseline translation energy is bounded by four pi squared times the squared shift difference. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: baseline_translation_energy
lemma baseline_translation_energy (s t : ℝ) :
(∫ w, (sqrt (cosSqDensity (w - s)) - sqrt (cosSqDensity (w - t))) ^ 2) ≤
4 * pi ^ 2 * (s - t) ^ 2 := by
have hmeas : Measurable baselineSqrtDerivative := by
unfold baselineSqrtDerivative
have hs : MeasurableSet (Icc (-1 / 4 : ℝ) (1 / 4)) := measurableSet_Icc
fun_prop
have hle (s t : ℝ) (hst : t ≤ s) :
(∫ w, (sqrt (cosSqDensity (w - s)) - sqrt (cosSqDensity (w - t))) ^ 2) ≤
4 * pi ^ 2 * (s - t) ^ 2 := by
have hx := translation_energy_le_of_integral_representation (fun w => sqrt (cosSqDensity w)) baselineSqrtDerivative
hmeas baselineSqrtDerivative_integrable.1 baselineSqrtDerivative_integrable.2 baselineSqrtDerivative_integral (-s) (-t) (by linarith)
rw [baselineSqrtDerivative_energy] at hx
simp only [← sub_eq_add_neg] at hx
have he : (fun w => (sqrt (cosSqDensity (w - s)) -
sqrt (cosSqDensity (w - t))) ^ 2) =
(fun w => (sqrt (cosSqDensity (w - t)) -
sqrt (cosSqDensity (w - s))) ^ 2) := by
funext w; ring
rw [he]
convert hx using 1 <;> ring
rcases le_total t s with h | h
· exact hle s t h
· have hx := hle t s h
have he : (fun w => (sqrt (cosSqDensity (w - s)) -
sqrt (cosSqDensity (w - t))) ^ 2) =
(fun w => (sqrt (cosSqDensity (w - t)) -
sqrt (cosSqDensity (w - s))) ^ 2) := by
funext w; ring
rw [he]
convert hx using 1 <;> ring
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BaselineTransport 1 declarations Transport of the block baseline under translation
Transport of the block baseline under translation
Translation of a baseline law has exactly the translated cosine density. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: baseline_map_add_density
lemma baseline_map_add_density (s : ℝ) :
(volume.withDensity (fun w => ofReal (cosSqDensity w))).map (fun w => w + s) =
volume.withDensity (fun w => ofReal (cosSqDensity (w - s))) := by
have ht := map_withDensity_comp_measurableEquiv
(MeasurableEquiv.addRight s) (measurePreserving_add_right volume s)
(fun w => ofReal (cosSqDensity (w - s)))
simpa [Function.comp_def] using ht
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BlockLaw 1 declarations The fixed-partition density and compatible-completion count recover the full record with every retained edge and assignment coordinate.
Complete original-record block likelihood assembly
The fixed-partition density and compatible-completion count recover the full record with every retained edge and assignment coordinate.
theorem block_law
Support, target, independent source reveals, and the complete-record likelihood reconstruction. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
σ :
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
ha :
hw :
AuditLaw D q
hi :
conclusion
conclusion 1
∀ᵐ ξ ∂(blockParamLaw B d), ScheduleClass (blockSchedule n B d σ h ξ) d ∧ tte (blockSchedule n B d σ h ξ)
= signOf σ * (B * d : ℕ) * h / n
conclusion 2
= Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))
conclusion 3
blockMixtureLawOf n B d D σ h = reconstructedLaw n B d D σ h
conclusion 4
conclusion 5
∀ᵐ hz ∂(graphAssignMarginal n B d D), undiscovered n B d hz.1
= 0 → hiddenTreated n B d hz.1 hz.2 = 0 ∧
(undiscovered n B d hz.1).choose (hiddenTreated n B d hz.1 hz.2) = 1
Proof (Lean source)
-- @node: lem:block-law
lemma block_law (n B d : ℕ) (σ : Bool) (h q : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) :
(∀ᵐ ξ ∂(blockParamLaw B d), ScheduleClass (blockSchedule n B d σ h ξ) d ∧
tte (blockSchedule n B d σ h ξ) = signOf σ * (B * d : ℕ) * h / n) ∧
(retainedGraphMarginal n B d D).map (sourceReveals n B d) =
Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)) ∧
blockMixtureLawOf n B d D σ h = reconstructedLaw n B d D σ h ∧
(blockMixtureLawOf n B d D σ h).map (fun o => (o.1, o.2.1)) =
graphAssignMarginal n B d D ∧
(∀ᵐ hz ∂(graphAssignMarginal n B d D),
undiscovered n B d hz.1 = 0 → hiddenTreated n B d hz.1 hz.2 = 0 ∧
(undiscovered n B d hz.1).choose (hiddenTreated n B d hz.1 hz.2) = 1) := by
let : NeZero n := ⟨by omega⟩
let := design_isProbabilityMeasure D q ha hw hi
refine ⟨block_support n B d σ h hB hd hfit hh,
block_reveal_law n B d q D hB hd hfit hq ha hw hi, ?_,
block_mixture_graphAssignMarginal n B d σ h D,
block_hidden_zero_endpoint n B d D hfit⟩
rw [design_eq_thinnedDesign D q ha hw hi,
blockMixtureLawOf_hidden_completion_bind n B d hd hfit σ h q hq,
reconstructedLaw]
apply Measure.bind_congr_right
have hv : ∀ᵐ hz ∂(graphAssignMarginal n B d (thinnedDesign (Fin n) q)),
ValidRetainedGraph n B d hz.1 :=
(ae_map_iff (by fun_prop) (by measurability)).mp
(retainedGraphMarginal_valid n B d (thinnedDesign (Fin n) q) hfit)
filter_upwards [hv] with hz hH
exact hiddenCompletionKernel_eq_blockDensity n B d hd hfit σ h hz hH
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BlockLawBasics 9 declarations Complete original-record block likelihood
Complete original-record block likelihood
A normalized baseline remains a probability law after taking independent blocks. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: blockBaselineLaw_probability
lemma blockBaselineLaw_probability (B : ℕ) : IsProbabilityMeasure (blockBaselineLaw B) := by
have hnorm : (volume.withDensity (fun w => ofReal (cosSqDensity w))) univ = 1 := by
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal cosSqDensity_integrable_normalized.1
(Filter.Eventually.of_forall cosSqDensity_nonneg),
cosSqDensity_integrable_normalized.2]
simp
let : IsProbabilityMeasure (volume.withDensity (fun w => ofReal (cosSqDensity w))) :=
⟨hnorm⟩
unfold blockBaselineLaw
infer_instance
Mapping a measurable channel commutes with mixing its input laws. For the stated data and conditions, the stated conclusion holds.
Proof (Lean source)
-- @node: block_map_bind
lemma block_map_bind {α β γ : Type*} [MeasurableSpace α] [MeasurableSpace β]
[MeasurableSpace γ] (μ : Measure α) (K : α → Measure β) (g : β → γ)
(hK : Measurable K) (hg : Measurable g) :
(μ.bind K).map g = μ.bind (fun a => (K a).map g) := by
rw [← Measure.bind_dirac_eq_map _ hg,
Measure.bind_bind (g := fun x => Measure.dirac (g x)) hK.aemeasurable
(show AEMeasurable (fun x => Measure.dirac (g x)) _ from
(Measure.measurable_dirac.comp hg).aemeasurable)]
congr 1
funext a
exact Measure.bind_dirac_eq_map _ hg
A jointly measurable recording function yields a measurable family of output laws.
Formal statement
Proof (Lean source)
-- @node: block_measurable_map_parameter
@[fun_prop]
lemma block_measurable_map_parameter {α β γ : Type*} [MeasurableSpace α]
[MeasurableSpace β] [MeasurableSpace γ] (μ : Measure β) [SFinite μ]
(f : α × β → γ) (hf : Measurable f) :
Measurable (fun a => μ.map (fun b => f (a, b))) := by
have h := (Measure.measurable_map f hf).comp
(Measurable.map_prodMk_left (ν := μ))
simpa only [Measure.map_map hf measurable_prodMk_left, Function.comp_def] using h
theorem block_record_measurable
The complete block record depends measurably on its prior parameters and design draw.
Formal statement
hypotheses
conclusion
Measurable (fun x : (SourcePartition B d × (Fin B → ℝ)) × (Assign (Fin n) × Audit (Fin n)) => recordOf (blockSchedule n B d σ h x.1) x.2)
Proof (Lean source)
-- @node: block_record_measurable
@[fun_prop]
lemma block_record_measurable (n B d : ℕ) (σ : Bool) (h : ℝ) :
Measurable (fun x : (SourcePartition B d × (Fin B → ℝ)) ×
(Assign (Fin n) × Audit (Fin n)) => recordOf (blockSchedule n B d σ h x.1) x.2) := by
apply measurable_from_prod_countable_left
intro ω
apply measurable_from_prod_countable_right
intro s
simp only [recordOf, potentialOutcome, blockSchedule, inNbhd, zero_mul, add_zero]
apply measurable_const.prodMk
apply measurable_const.prodMk
apply measurable_pi_lambda
intro i
apply Measurable.add
· apply Finset.measurable_sum
intro ℓ hℓ
by_cases hi : i ∈ recipientBlock n B d ℓ <;> simp only [hi, ite_true, ite_false] <;> fun_prop
· exact measurable_const
theorem block_mixture_graphAssignMarginal
Mixing independent baselines preserves the detailed retained graph and the entire assignment marginal, for either sign and every amplitude. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: block_mixture_graphAssignMarginal
lemma block_mixture_graphAssignMarginal (n B d : ℕ) (σ : Bool) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [SFinite D] :
(blockMixtureLawOf n B d D σ h).map (fun o => (o.1, o.2.1)) =
graphAssignMarginal n B d D := by
let := blockBaselineLaw_probability B
unfold blockMixtureLawOf mixtureLaw
rw [block_map_bind _ _ _
(block_measurable_map_parameter D _ (block_record_measurable n B d σ h))
(by fun_prop)]
have hrecord (ξ : SourcePartition B d × (Fin B → ℝ)) :
(D.map (recordOf (blockSchedule n B d σ h ξ))).map (fun o => (o.1, o.2.1)) =
D.map (fun ω => ((recordOf (blockSchedule n B d true 0
(ξ.1, fun _ => 0)) ω).1, ω.1)) := by
rw [Measure.map_map (by fun_prop) (measurable_of_finite _)]
rfl
simp_rw [hrecord]
unfold blockParamLaw graphAssignMarginal
ext E hE
let K := fun s : SourcePartition B d =>
D.map (fun ω => ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1))
have hK : Measurable K := measurable_of_finite _
change ((partitionLaw B d).prod (blockBaselineLaw B)).bind (K ∘ fst) E =
(partitionLaw B d).bind K E
rw [Measure.bind_apply hE (hK.comp measurable_fst).aemeasurable,
Measure.bind_apply hE hK.aemeasurable]
rw [lintegral_prod (fun ξ : SourcePartition B d × (Fin B → ℝ) => ((K ∘ fst) ξ) E)
(show AEMeasurable (fun ξ : SourcePartition B d × (Fin B → ℝ) => ((K ∘ fst) ξ) E) _ from
((Measure.measurable_coe hE).comp (hK.comp measurable_fst)).aemeasurable)]
simp
theorem block_weighted_baseline_product_integrable_normalized Lemma block_weighted_baseline_product_integrable_normalized in the paper ↗
Every finite product of weighted translated baseline densities is integrable, and its integral is the product of the weights. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
c s :
Fin B → ℝ
conclusion
conclusion 1
conclusion 2
(∫ y : Fin B → ℝ, ∏ ℓ, c ℓ * cosSqDensity (y ℓ - s ℓ)) = ∏ ℓ, c ℓ
Proof (Lean source)
-- @node: block_weighted_baseline_product_integrable_normalized
lemma block_weighted_baseline_product_integrable_normalized (B : ℕ)
(c s : Fin B → ℝ) :
Integrable (fun y : Fin B → ℝ => ∏ ℓ, c ℓ * cosSqDensity (y ℓ - s ℓ)) ∧
(∫ y : Fin B → ℝ, ∏ ℓ, c ℓ * cosSqDensity (y ℓ - s ℓ)) = ∏ ℓ, c ℓ := by
constructor
· exact Integrable.fintype_prod (fun ℓ =>
(translated_cosSqDensity_integrable_normalized (s ℓ)).1.const_mul (c ℓ))
· rw [integral_fintype_prod_volume_eq_prod
(fun ℓ w => c ℓ * cosSqDensity (w - s ℓ))]
simp_rw [integral_const_mul, (translated_cosSqDensity_integrable_normalized _).2,
mul_one]
theorem blockDensity_integrable_normalized
The constrained density integrates to one on every valid retained graph, since its finite allocation weights are normalized; no posterior-law identification is assumed. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
Integrable (blockDensity n B d σ h H z)
conclusion 2
(∫ y, blockDensity n B d σ h H z y) = 1
Proof (Lean source)
-- @node: blockDensity_integrable_normalized
lemma blockDensity_integrable_normalized (n B d : ℕ) (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (hH : ValidRetainedGraph n B d H)
(z : Assign (Fin n)) :
Integrable (blockDensity n B d σ h H z) ∧
(∫ y, blockDensity n B d σ h H z y) = 1 := by
let c := fun (k : ∀ ℓ, Fin (capacity n B d H ℓ + 1)) (ℓ : Fin B) =>
((capacity n B d H ℓ).choose (k ℓ).val : ℝ)
let s := fun (k : ∀ ℓ, Fin (capacity n B d H ℓ + 1)) (ℓ : Fin B) =>
signOf σ * h *
(revealedSignSum n B d H z ℓ + 2 * (k ℓ).val - capacity n B d H ℓ) / (2 * d)
let f := fun (k : ∀ ℓ, Fin (capacity n B d H ℓ + 1)) (y : Fin B → ℝ) =>
if hiddenTreated n B d H z = ∑ ℓ, (k ℓ).val then
∏ ℓ, c k ℓ * cosSqDensity (y ℓ - s k ℓ) else 0
have hf (k : ∀ ℓ, Fin (capacity n B d H ℓ + 1)) : Integrable (f k) := by
dsimp only [f]
split_ifs
· exact (block_weighted_baseline_product_integrable_normalized B (c k) (s k)).1
· exact integrable_zero (Fin B → ℝ) ℝ volume
have he : blockDensity n B d σ h H z = fun y =>
(∑ k, f k y) / ((undiscovered n B d H).choose (hiddenTreated n B d H z) : ℝ) := by
funext y
exact blockDensity_allocation_sum n B d σ h H z y
rw [he]
constructor
· exact (integrable_finsetSum _ (fun k _ => hf k)).div_const _
· rw [integral_div, integral_finsetSum _ (fun k _ => hf k)]
have hfi (k : ∀ ℓ, Fin (capacity n B d H ℓ + 1)) :
(∫ y, f k y) =
if hiddenTreated n B d H z = ∑ ℓ, (k ℓ).val then ∏ ℓ, c k ℓ else 0 := by
dsimp only [f]
split_ifs
· exact (block_weighted_baseline_product_integrable_normalized B (c k) (s k)).2
· simp
simp_rw [hfi]
rw [Finset.sum_div]
exact block_allocation_weights_normalized B (capacity n B d H)
(hiddenTreated n B d H z) (hiddenTreated_le n B d H hH z)
theorem blockDensityLaw_probability
On a valid retained graph, the displayed conditional density defines a probability law of the distinct responses, including the zero-hidden-capacity endpoint. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: blockDensityLaw_probability
lemma blockDensityLaw_probability (n B d : ℕ) (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (hH : ValidRetainedGraph n B d H)
(z : Assign (Fin n)) :
IsProbabilityMeasure (volume.withDensity
(fun y => ofReal (blockDensity n B d σ h H z y))) := by
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal
(blockDensity_integrable_normalized n B d σ h H hH z).1
(Filter.Eventually.of_forall (blockDensity_nonneg n B d σ h H z)),
(blockDensity_integrable_normalized n B d σ h H hH z).2]
simp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BlockPrior 20 declarations Hidden-allocation fixed-schedule prior
Hidden-allocation fixed-schedule prior
def cosSqDensity
The compactly supported cos-squared baseline density.
def SourcePartition
Ordered partitions represented by a label for each source with exactly d sources per label.
def partitionLaw
Uniform finite measure; zero when the finite carrier is empty.
Definition (Lean source)
parameters
B :
ℕ
d :
ℕ
def
given by
def recipientBlock
The recipient block in the second half of the bipartite layout.
def blockBaselineLaw
Independent baseline levels.
Definition (Lean source)
parameters
B :
ℕ
given by
def blockParamLaw
Partition and baselines are drawn independently before assignment.
Definition (Lean source)
parameters
B :
ℕ
d :
ℕ
def
given by
The labeled source-to-recipient block graph, with no diagonal arrows.
Definition (Lean source)
parameters
def
blockEdge n B d s j i :
Prop
given by
clause 2
j ≠ i
theorem blockEdge_irrefl
The source-to-recipient block graph has no diagonal arrows. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
¬ blockEdge n B d s i i
Proof (Lean source)
-- @node: blockEdge_irrefl
lemma blockEdge_irrefl (s : SourcePartition B d) : ∀ i, ¬ blockEdge n B d s i i := by
intro i h
exact h.2 rfl
The amplitude domain of the auxiliary block family.
Definition (Lean source)
parameters
h :
ℝ
def
BlockAmplitude h :
Prop
given by
clause 1
0 ≤ h
clause 2
h ≤ 1 / 4
def blockSchedule
The fixed schedule map. Source and padding rows are zero; each recipient has its block baseline.
Definition (Lean source)
parameters
def
given by
clause 2
decEdge := Classical.decRel _
clause 3
irrefl := blockEdge_irrefl n B d ξ.1
clause 4
a := fun i
=> ∑ ℓ : Fin B, if i ∈ recipientBlock n B d ℓ then ξ.2 ℓ - signOf σ * h / 2 else 0
clause 5
t := fun _ => 0
clause 6
b := fun _ _ => signOf σ * h / d
def blockPrior
The prior is its parameter law paired with the concrete fixed-schedule map.
Definition (Lean source)
parameters
def
blockPrior n B d σ h :
Measure (SourcePartition B d × (Fin B → ℝ)) × ((SourcePartition B d × (Fin B → ℝ)) → Schedule (Fin n))
given by
(blockParamLaw B d, blockSchedule n B d σ h)
def mixtureLaw
Apply the complete-record channel after drawing a fixed schedule from the parameter prior.
Mixture law of the entire original record, without removing any coordinates.
Definition (Lean source)
parameters
given by
The complete-record block mixture under the canonical assignment-audit product design.
Definition (Lean source)
parameters
given by
Probability that at least one of a source's d arrows is retained.
Definition (Lean source)
parameters
d :
ℕ
q :
ℝ
def
retentionP d q :
ℝ
given by
1 - (1 - q) ^ d
def RetentionDomain
Admissible audit probability for the source-association retention formula.
Definition (Lean source)
theorem retentionP_mem_Icc
Source-association retention lies in the probability interval. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: retentionP_mem_Icc
lemma retentionP_mem_Icc (d : ℕ) (q : ℝ) (hq : RetentionDomain q) :
retentionP d q ∈ Icc 0 1 := by
have hnonneg : 0 ≤ 1 - q := sub_nonneg.mpr hq.2
have hle : 1 - q ≤ 1 := sub_le_self _ hq.1
have hp : (1 - q) ^ d ≤ 1 := pow_le_one₀ hnonneg hle
exact ⟨sub_nonneg.mpr hp, sub_le_self _ (pow_nonneg hnonneg d)⟩
The nonlocal testing scale is clipped at one and equals one when no association is retained.
Definition (Lean source)
parameters
B d :
ℕ
q :
ℝ
def
testingScale B d q :
ℝ
given by
def testingAmplitude
One hundred pi inverse times the nonlocal testing scale.
Definition (Lean source)
def testingHandle
Names both constructed priors and their complete original-record mixture laws.
Definition (Lean source)
parameters
n :
ℕ
B :
ℕ
d :
ℕ
q :
ℝ
def
testingHandle n B d q :
((Measure (SourcePartition B d × (Fin B → ℝ)) × ((SourcePartition B d × (Fin B → ℝ)) → Schedule (Fin n))) × (Measure (SourcePartition B d × (Fin B → ℝ)) × ((SourcePartition B d × (Fin B → ℝ)) → Schedule (Fin n)))) × (Measure (Record (Fin n)) × Measure (Record (Fin n)))
given by
((blockPrior n B d true (testingAmplitude B d q), blockPrior n B d false (testingAmplitude B d q)), (blockMixtureLaw n B d q true (testingAmplitude B d q), blockMixtureLaw n B d q false (testingAmplitude B d q)))
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BlockReveal 8 declarations Independent revealed source associations
Independent revealed source associations
Selecting distinct coordinates of independent Bernoulli marks preserves their product law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: bernoulli_select_coordinates
lemma bernoulli_select_coordinates {I J : Type*} [Fintype I] [Fintype J]
(e : J → I) (he : Injective e)
(q : ℝ) (hq : q ∈ Icc 0 1) :
(Measure.pi (fun _ : I => bernoulliLaw q)).map (fun w j => w (e j)) =
Measure.pi (fun _ : J => bernoulliLaw q) := by
let _ := bernoulliLaw_probability q hq
have hind := (ProbabilityTheory.iIndepFun_pi
(μ := fun _ : I => bernoulliLaw q) (X := fun _ => id)
(fun _ => aemeasurable_id)).precomp he
have hm (j : J) : (Measure.pi (fun _ : I => bernoulliLaw q)).map
(fun w => w (e j)) = bernoulliLaw q :=
(measurePreserving_eval (fun _ : I => bernoulliLaw q) (e j)).map_eq
simpa only [Function.comp_def, hm] using
hind.map_fun_eq_pi_map (fun j => (measurable_pi_apply (e j)).aemeasurable)
Regroup a rectangular collection of independent marks into independent rows. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: bernoulli_curry_coordinates
lemma bernoulli_curry_coordinates {I J : Type*} [Fintype I] [Fintype J]
(q : ℝ) (hq : q ∈ Icc 0 1) :
(Measure.pi (fun _ : I × J => bernoulliLaw q)).map (fun w i j => w (i,j)) =
Measure.pi (fun _ : I => Measure.pi (fun _ : J => bernoulliLaw q)) := by
let _ := bernoulliLaw_probability q hq
apply Measure.ext_of_singleton
intro w
rw [Measure.map_apply (by fun_prop) (measurableSet_singleton _)]
have hp : (fun v : I × J → Bool => fun i j => v (i,j)) ⁻¹' {w} =
{fun ij : I × J => w ij.1 ij.2} := by
ext v
simp only [Set.mem_preimage, Set.mem_singleton_iff]
constructor
· intro h
funext ij
exact congrFun (congrFun h ij.1) ij.2
· intro h
subst v
rfl
rw [hp, Measure.pi_singleton, Measure.pi_singleton]
simp_rw [Measure.pi_singleton]
exact Fintype.prod_prod_type _
theorem bernoulli_any_coordinate
The OR of d independent audit marks has success probability one minus their joint failure. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: bernoulli_any_coordinate
lemma bernoulli_any_coordinate (d : ℕ) (q : ℝ) (hq : q ∈ Icc 0 1) :
(Measure.pi (fun _ : Fin d => bernoulliLaw q)).map
(fun w => decide (∃ k, w k = true)) = bernoulliLaw (retentionP d q) := by
let _ := bernoulliLaw_probability q hq
let _ := bernoulliLaw_probability (retentionP d q) (retentionP_mem_Icc d q hq)
let μ := (Measure.pi (fun _ : Fin d => bernoulliLaw q)).map
(fun w => decide (∃ k, w k = true))
let _ : IsProbabilityMeasure μ := Measure.isProbabilityMeasure_map (by fun_prop)
have hf : μ {false} = ofReal ((1 - q) ^ d) := by
dsimp [μ]
rw [Measure.map_apply (by fun_prop) (measurableSet_singleton _)]
have hp : (fun w : Fin d → Bool => decide (∃ k, w k = true)) ⁻¹' {false} =
{fun _ => false} := by
ext w
simp only [Set.mem_preimage, Set.mem_singleton_iff, decide_eq_false_iff_not,
not_exists]
constructor
· intro h
funext k
cases hk : w k
· rfl
· exact False.elim (h k hk)
· intro h k
simp [h]
rw [hp, Measure.pi_singleton]
simp only [bernoulliLaw, Measure.add_apply, Measure.smul_apply,
Measure.dirac_apply' _ (measurableSet_singleton _),
smul_eq_mul]
simp [← ENNReal.ofReal_pow, sub_nonneg.mpr hq.2]
apply Measure.ext_of_singleton
intro b
cases b
· change μ {false} = _
rw [hf]
simp [bernoulliLaw, retentionP]
· have hc : ({false} : Set Bool)ᶜ = {true} := by ext b; cases b <;> simp
change μ {true} = _
rw [← hc, measure_compl (measurableSet_singleton _) (measure_ne_top _ _), measure_univ, hf]
have hp : 0 ≤ (1 - q) ^ d := pow_nonneg (sub_nonneg.mpr hq.2) _
rw [← ENNReal.ofReal_one, ← ENNReal.ofReal_sub 1 hp]
simp [bernoulliLaw, retentionP]
theorem block_audit_rows
For a fixed source partition, its true audit marks can be enumerated in disjoint rows; the full retained graph reveals a source exactly when one mark in its row is true. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: block_audit_rows
lemma block_audit_rows (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) :
∃ e : Fin (B * d) × Fin d → OffDiag (Fin n), Function.Injective e ∧
∀ ω : Assign (Fin n) × Audit (Fin n),
sourceReveals n B d (recordOf
(blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 =
fun j => decide (∃ k : Fin d, ω.2 (e (j,k)) = true) := by
let v : Fin (B * d) → Fin n := fun j => ⟨j.val, by omega⟩
let r (j : Fin (B * d)) : Fin d ≃
{i : Fin n // i ∈ recipientBlock n B d (s.1 j)} :=
Fintype.equivOfCardEq (by simp [recipientBlock_card n B d hfit])
have hr (j : Fin (B * d)) (k : Fin d) :
blockEdge n B d s (v j) (r j k).val :=
(blockEdge_source_iff n B d s (v j) (r j k).val j.isLt).mpr (r j k).property
let e : Fin (B * d) × Fin d → OffDiag (Fin n) :=
fun jk => ⟨(v jk.1, (r jk.1 jk.2).val), (hr jk.1 jk.2).2⟩
refine ⟨e, ?_, ?_⟩
· intro a b hab
have hj : a.1 = b.1 := Fin.ext (congrArg (fun edge => edge.val.1.val) hab)
apply Prod.ext hj
have hi := congrArg (fun edge => edge.val.2) hab
dsimp [e] at hi
rw [← hj] at hi
exact (r a.1).injective (Subtype.ext hi)
· intro ω
funext j
unfold sourceReveals
apply Bool.decide_congr
constructor
· rintro ⟨v', hv', i, hvi, he⟩
have hv : v' = v j := Fin.ext hv'
subst v'
have hedge : blockEdge n B d s (v j) i ∧ ω.2 ⟨(v j,i), hvi⟩ = true := by
simpa only [recordOf, blockSchedule, Bool.and_eq_true, decide_eq_true_eq] using he
have hi := (blockEdge_source_iff n B d s (v j) i j.isLt).mp hedge.1
obtain ⟨k, hk⟩ := (r j).surjective ⟨i, hi⟩
refine ⟨k, ?_⟩
have heq : e (j,k) = ⟨(v j,i), hvi⟩ := by
apply Subtype.ext
exact Prod.ext rfl (congrArg val hk)
rw [heq]
exact hedge.2
· rintro ⟨k, hk⟩
refine ⟨v j, rfl, (r j k).val, (hr j k).2, ?_⟩
change (decide (blockEdge n B d s (v j) (r j k).val) && ω.2 (e (j,k))) = true
simp [hr, hk]
theorem block_reveal_fixed_partition
Conditional on any true partition, the vector of labeled source reveals has the same independent Bernoulli law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(thinnedDesign (Fin n) q).map (fun ω => sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1)
= Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))
Proof (Lean source)
-- @node: block_reveal_fixed_partition
lemma block_reveal_fixed_partition (n B d : ℕ) (q : ℝ)
(hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1) (s : SourcePartition B d) :
(thinnedDesign (Fin n) q).map (fun ω => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1) =
Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)) := by
let _ := bernoulliLaw_probability q hq
let _ : IsProbabilityMeasure ((Measure.pi (fun _ : Fin d => bernoulliLaw q)).map
(fun w => decide (∃ k : Fin d, w k = true))) :=
Measure.isProbabilityMeasure_map (by fun_prop)
obtain ⟨e, he, hrows⟩ := block_audit_rows n B d hfit s
have hselect := bernoulli_select_coordinates e he q hq
have hcurry := bernoulli_curry_coordinates (I := Fin (B * d)) (J := Fin d) q hq
have hany := bernoulli_any_coordinate d q hq
have hmap : (auditLaw (Fin n) q).map
(fun w j => decide (∃ k : Fin d, w (e (j,k)) = true)) =
Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)) := by
have hr := congrArg (fun μ : Measure (Fin (B * d) → Fin d → Bool) =>
μ.map (fun rows j => decide (∃ k : Fin d, rows j k = true))) hcurry
rw [Measure.map_map (by fun_prop) (by fun_prop)] at hr
have hor := Measure.pi_map_pi (μ := fun _ : Fin (B * d) =>
Measure.pi (fun _ : Fin d => bernoulliLaw q))
(f := fun _ => fun w => decide (∃ k : Fin d, w k = true))
(fun _ => (by fun_prop))
rw [hor, hany] at hr
have hs := congrArg (fun μ : Measure (Fin (B * d) × Fin d → Bool) =>
μ.map (fun w j => decide (∃ k : Fin d, w (j,k) = true))) hselect
rw [Measure.map_map (by fun_prop) (by fun_prop)] at hs
exact hs.trans hr
have hm : (thinnedDesign (Fin n) q).map snd = auditLaw (Fin n) q :=
(thinnedDesign_audit q hq).2.2
have ht := congrArg (fun μ : Measure (Audit (Fin n)) => μ.map
(fun w j => decide (∃ k : Fin d, w (e (j,k)) = true))) hm
rw [Measure.map_map (by fun_prop) (by fun_prop), hmap] at ht
simpa only [hrows, Function.comp_def] using ht
A rectangular enumeration provides an ordered source partition for every block layout. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sourcePartition_nonempty
lemma sourcePartition_nonempty (B d : ℕ) : Nonempty (SourcePartition B d) := by
let e : Fin (B * d) ≃ Fin B × Fin d := finProdFinEquiv.symm
refine ⟨⟨fun j => (e j).1, ?_⟩⟩
intro ℓ
have hf : Finset.univ.filter (fun j : Fin (B * d) => (e j).1 = ℓ) =
Finset.univ.image (fun k : Fin d => e.symm (ℓ,k)) := by
ext j
simp only [mem_filter, Finset.mem_univ, true_and, mem_image]
constructor
· intro hj
refine ⟨(e j).2, ?_⟩
rw [← hj, Prod.mk.eta, e.symm_apply_apply]
· rintro ⟨k, rfl⟩
simp
rw [hf, Finset.card_image_of_injective]
· simp
· intro k k' h
exact congrArg snd (e.symm.injective h)
The uniform prior on ordered source partitions has total mass one. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: partitionLaw_probability
lemma partitionLaw_probability (B d : ℕ) : IsProbabilityMeasure (partitionLaw B d) := by
let _ := sourcePartition_nonempty B d
refine ⟨?_⟩
simp only [partitionLaw, Measure.smul_apply, Measure.finsetSum_apply,
Measure.dirac_apply_of_mem (Set.mem_univ _), Finset.sum_const, Finset.card_univ,
smul_eq_mul, nsmul_eq_mul, mul_one]
exact ENNReal.inv_mul_cancel
(by exact_mod_cast Fintype.card_ne_zero) (by simp)
theorem block_reveal_law
Source associations are revealed independently with the stated retention probability. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
q :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hq :
q ∈ Icc 0 1
ha :
hw :
AuditLaw D q
hi :
conclusion
= Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))
Proof (Lean source)
-- @node: block_reveal_law
lemma block_reveal_law (n B d : ℕ) (q : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) :
(retainedGraphMarginal n B d D).map (sourceReveals n B d) =
Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)) := by
rw [design_eq_thinnedDesign D q ha hw hi]
let _ := partitionLaw_probability B d
rw [retainedGraphMarginal, Measure.map_map (by fun_prop) (by fun_prop)]
apply Measure.ext
intro E hE
rw [Measure.map_apply (by fun_prop) hE, graphAssignMarginal,
Measure.bind_apply (hE.preimage (by fun_prop)) (by fun_prop)]
have hfixed (s : SourcePartition B d) :
((thinnedDesign (Fin n) q).map (fun ω =>
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1)))
((sourceReveals n B d ∘ fst) ⁻¹' E) =
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))) E := by
rw [← Measure.map_apply (by fun_prop) hE,
Measure.map_map (by fun_prop) (by fun_prop)]
exact congrArg (fun μ => μ E) (block_reveal_fixed_partition n B d q hfit hq s)
simp_rw [hfixed]
rw [lintegral_const, measure_univ, mul_one]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BlockStatistics 24 declarations Observed block statistics and conditional polynomial density
Observed block statistics and conditional polynomial density
def ValidRetainedGraph
A retained graph uses a single size-d source partition for the entire fitted block layout.
def revealedSources
The labeled sources with at least one retained arrow into the specified recipient block.
def revealedCount
The number of revealed labeled sources in a recipient block.
Definition (Lean source)
def capacity
The remaining hidden-source capacity of a block.
Definition (Lean source)
def revealedSignSum
The sum of observed treatment signs over the revealed labeled sources.
def undiscovered
The total remaining hidden-source capacity over all blocks.
A recipient label belongs to at most one block. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
i :
Fin n
ℓ k :
Fin B
hℓ :
i ∈ recipientBlock n B d ℓ
hk :
i ∈ recipientBlock n B d k
conclusion
ℓ = k
Proof (Lean source)
-- @node: recipientBlock_unique
lemma recipientBlock_unique (i : Fin n) (ℓ k : Fin B)
(hℓ : i ∈ recipientBlock n B d ℓ) (hk : i ∈ recipientBlock n B d k) : ℓ = k := by
have hℓ' := (Finset.mem_filter.mp hℓ).2
have hk' := (Finset.mem_filter.mp hk).2
apply Fin.ext
apply Nat.le_antisymm
· by_contra h
have hmul : (k.val + 1) * d ≤ ℓ.val * d :=
Nat.mul_le_mul_right d (by omega)
omega
· by_contra h
have hmul : (ℓ.val + 1) * d ≤ k.val * d :=
Nat.mul_le_mul_right d (by omega)
omega
Partition-supported retained graphs reveal at most d sources in each block. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
revealedCount n B d H ℓ ≤ d
Proof (Lean source)
-- @node: revealedCount_le
lemma revealedCount_le (H : OffDiag (Fin n) → Bool) (hH : ValidRetainedGraph n B d H)
(ℓ : Fin B) : revealedCount n B d H ℓ ≤ d := by
obtain ⟨hfit, s, hs⟩ := hH
let fiber := Finset.univ.filter (fun j : Fin (B * d) => s.1 j = ℓ)
have hcard : (fiber.image val).card = d := by
rw [Finset.card_image_of_injective _ Fin.val_injective]
exact s.2 ℓ
change (revealedSources n B d H ℓ).card ≤ d
apply le_trans (b := (fiber.image val).card) ?_ (le_of_eq hcard)
apply Finset.card_le_card_of_injOn val
· intro j hj
obtain ⟨hjlt, i, hi, hji, hedge⟩ := (Finset.mem_filter.mp hj).2
obtain ⟨⟨hjlt', k, hsk, hik⟩, _⟩ := hs ⟨(j, i), hji⟩ hedge
have hk : k = ℓ := recipientBlock_unique n B d i k ℓ hik hi
apply Finset.mem_image.mpr
refine ⟨⟨j.val, hjlt⟩, ?_, rfl⟩
apply Finset.mem_filter.mpr
exact ⟨Finset.mem_univ _, by simpa [hk] using hsk⟩
· intro a ha b hb hab
exact Fin.ext hab
def blockDensity
The coefficient-constrained hypergeometric mixture density of distinct block responses.
Definition (Lean source)
parameters
def
blockDensity n B d σ h H z y :
ℝ
given by
(∏ ℓ : Fin B, ∑ k ∈ range (capacity n B d H ℓ + 1), C (((capacity n B d H ℓ).choose k : ℝ) * cosSqDensity (y ℓ - signOf σ * h * (revealedSignSum n B d H z ℓ + 2 * k - capacity n B d H ℓ) / (2 * d))) * X ^ k).coeff (hiddenTreated n B d H z) / ((undiscovered n B d H).choose (hiddenTreated n B d H z) : ℝ)
Expanding the polynomial product retains exactly allocations with the prescribed total degree. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_allocation_coeff
lemma block_allocation_coeff (B : ℕ) (u : Fin B → ℕ) (w : ∀ ℓ, Fin (u ℓ + 1) → ℝ) (K : ℕ) :
(∏ ℓ, ∑ k, C (w ℓ k) * X ^ k.val).coeff K =
∑ k : ∀ ℓ, Fin (u ℓ + 1),
if K = ∑ ℓ, (k ℓ).val then ∏ ℓ, w ℓ (k ℓ) else 0 := by
rw [Fintype.prod_sum]
rw [Polynomial.finsetSum_coeff]
apply Finset.sum_congr rfl
intro k _
rw [Finset.prod_mul_distrib, ← map_prod, Finset.prod_pow_eq_pow_sum,
Polynomial.coeff_C_mul_X_pow]
theorem block_allocation_binomial_polynomial Lemma block_allocation_binomial_polynomial in the paper ↗
The row allocation counting polynomial is the binomial polynomial. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_allocation_binomial_polynomial
lemma block_allocation_binomial_polynomial (u : ℕ) :
(∑ k : Fin (u + 1), C (u.choose k.val : ℝ) * X ^ k.val) =
(X + 1 : Polynomial ℝ) ^ u := by
rw [Fin.sum_univ_eq_sum_range
(fun k => C (u.choose k : ℝ) * X ^ k), add_pow]
apply Finset.sum_congr rfl
intro k hk
simp only [one_pow, mul_one]
rw [mul_comm]
congr 1
The coefficient of the binomial polynomial counts treated subsets of a fixed total size. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_allocation_binomial_coeff
lemma block_allocation_binomial_coeff (u K : ℕ) :
((X + 1 : Polynomial ℝ) ^ u).coeff K = (u.choose K : ℝ) := by
rw [← block_allocation_binomial_polynomial, Polynomial.finsetSum_coeff]
simp only [Polynomial.coeff_C_mul_X_pow]
by_cases hK : K < u + 1
· rw [Finset.sum_eq_single ⟨K, hK⟩]
· simp
· intro k _ hk
rw [if_neg]
intro he
apply hk
exact Fin.ext he.symm
· simp
· have hzero : u.choose K = 0 := Nat.choose_eq_zero_of_lt (by omega)
rw [hzero, Nat.cast_zero]
apply Finset.sum_eq_zero
intro k _
rw [if_neg]
omega
Summing row subset counts with a fixed global treated count gives the global binomial count. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_allocation_weights_sum
lemma block_allocation_weights_sum (B : ℕ) (u : Fin B → ℕ) (K : ℕ) :
(∑ k : ∀ ℓ, Fin (u ℓ + 1),
if K = ∑ ℓ, (k ℓ).val then ∏ ℓ, (u ℓ).choose (k ℓ).val else 0 : ℝ) =
((∑ ℓ, u ℓ).choose K : ℝ) := by
rw [← block_allocation_coeff B u (fun ℓ k => ((u ℓ).choose k.val : ℝ)) K]
simp_rw [block_allocation_binomial_polynomial]
rw [Finset.prod_pow_eq_pow_sum, block_allocation_binomial_coeff]
theorem block_allocation_weights_normalized Lemma block_allocation_weights_normalized in the paper ↗
The constrained row allocation weights sum to one, including zero total capacity. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_allocation_weights_normalized
lemma block_allocation_weights_normalized (B : ℕ) (u : Fin B → ℕ) (K : ℕ)
(hK : K ≤ ∑ ℓ, u ℓ) :
(∑ k : ∀ ℓ, Fin (u ℓ + 1),
(if K = ∑ ℓ, (k ℓ).val then ∏ ℓ, ((u ℓ).choose (k ℓ).val : ℝ) else 0) /
((∑ ℓ, u ℓ).choose K : ℝ)) = 1 := by
rw [← Finset.sum_div, block_allocation_weights_sum]
apply div_self
exact_mod_cast (Nat.choose_pos hK).ne'
theorem blockDensity_allocation_sum
The coefficient-defined density is the exact finite mixture over constrained hidden row counts. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
blockDensity n B d σ h H z y
= (∑ k : ∀ ℓ, Fin (capacity n B d H ℓ + 1), if hiddenTreated n B d H z = ∑ ℓ, (k ℓ).val then ∏ ℓ, ((capacity n B d H ℓ).choose (k ℓ).val : ℝ) * cosSqDensity (y ℓ - signOf σ * h * (revealedSignSum n B d H z ℓ + 2 * (k ℓ).val - capacity n B d H ℓ) / (2 * d)) else 0) / ((undiscovered n B d H).choose (hiddenTreated n B d H z) : ℝ)
Proof (Lean source)
-- @node: blockDensity_allocation_sum
lemma blockDensity_allocation_sum (n B d : ℕ) (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n)) (y : Fin B → ℝ) :
blockDensity n B d σ h H z y =
(∑ k : ∀ ℓ, Fin (capacity n B d H ℓ + 1),
if hiddenTreated n B d H z = ∑ ℓ, (k ℓ).val then
∏ ℓ, ((capacity n B d H ℓ).choose (k ℓ).val : ℝ) *
cosSqDensity (y ℓ - signOf σ * h *
(revealedSignSum n B d H z ℓ + 2 * (k ℓ).val - capacity n B d H ℓ) /
(2 * d)) else 0) /
((undiscovered n B d H).choose (hiddenTreated n B d H z) : ℝ) := by
unfold blockDensity
simp_rw [Finset.sum_range]
rw [block_allocation_coeff B (capacity n B d H)
(fun ℓ k => ((capacity n B d H ℓ).choose k.val : ℝ) *
cosSqDensity (y ℓ - signOf σ * h *
(revealedSignSum n B d H z ℓ + 2 * k.val - capacity n B d H ℓ) / (2 * d)))]
theorem blockDensity_nonneg
The constrained conditional block density is nonnegative for every input. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: blockDensity_nonneg
lemma blockDensity_nonneg (n B d : ℕ) (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n)) (y : Fin B → ℝ) :
0 ≤ blockDensity n B d σ h H z y := by
rw [blockDensity_allocation_sum]
apply div_nonneg
· apply Finset.sum_nonneg
intro k _
split_ifs
· apply Finset.prod_nonneg
intro ℓ _
exact mul_nonneg (Nat.cast_nonneg _) (by unfold cosSqDensity; split_ifs <;> positivity)
· exact le_rfl
· exact Nat.cast_nonneg _
def copyOutcomes
Copy each distinct response to its recipient block and set source and padding outcomes to zero.
def graphAssignMarginal
The actual joint law of the detailed retained graph and every assignment coordinate.
Definition (Lean source)
parameters
given by
(partitionLaw B d).bind (fun s => D.map (fun ω => ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1)))
def retainedGraphMarginal
The actual marginal law of the detailed labeled retained graph.
theorem retainedGraphMarginal_valid
Every retained graph in the actual fitted block experiment has common partition support. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hfit :
2 * (B * d) ≤ n
conclusion
∀ᵐ H ∂(retainedGraphMarginal n B d D), ValidRetainedGraph n B d H
Proof (Lean source)
-- @node: retainedGraphMarginal_valid
lemma retainedGraphMarginal_valid (D : Measure (Assign (Fin n) × Audit (Fin n)))
(hfit : 2 * (B * d) ≤ n) :
∀ᵐ H ∂(retainedGraphMarginal n B d D), ValidRetainedGraph n B d H := by
unfold retainedGraphMarginal
apply (ae_map_iff (by fun_prop) (by measurability)).mpr
change ∀ᵐ hz ∂(graphAssignMarginal n B d D), ValidRetainedGraph n B d hz.1
rw [ae_iff]
apply le_antisymm _ bot_le
refine (Measure.bind_apply_le _ (by measurability)).trans ?_
have hf (s : SourcePartition B d) :
(D.map (fun ω => ((recordOf (blockSchedule n B d true 0
(s, fun _ => 0)) ω).1, ω.1)))
{hz | ¬ ValidRetainedGraph n B d hz.1} = 0 := by
rw [Measure.map_apply (by fun_prop) (by measurability)]
have hempty : (fun ω => ((recordOf (blockSchedule n B d true 0
(s, fun _ => 0)) ω).1, ω.1)) ⁻¹'
{hz | ¬ ValidRetainedGraph n B d hz.1} = ∅ := by
apply Set.eq_empty_iff_forall_notMem.mpr
intro ω hω
apply hω
refine ⟨hfit, s, ?_⟩
intro e he
have hedge : blockEdge n B d s e.1.1 e.1.2 ∧ ω.2 e = true := by
simpa only [recordOf, blockSchedule, Bool.and_eq_true, decide_eq_true_eq] using he
exact hedge.1
rw [hempty, measure_empty]
simp only [hf, lintegral_zero]
exact le_rfl
def sourceReveals
The indicator vector that a labeled source has at least one retained arrow.
def reconstructedLaw
Recover the full original-record mixture from the actual graph-assignment marginal and conditional density.
Definition (Lean source)
parameters
given by
(graphAssignMarginal n B d D).bind (fun hz => (volume.withDensity (fun y => ofReal (blockDensity n B d σ h hz.1 hz.2 y))).map (copyOutcomes n B d hz))
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.BlockSupport 11 declarations Support and causal target of the block prior
Support and causal target of the block prior
Recipient blocks contain exactly d labels when the bipartite layout fits. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: recipientBlock_card
lemma recipientBlock_card (n B d : ℕ) (hfit : 2 * (B * d) ≤ n) (ℓ : Fin B) :
(recipientBlock n B d ℓ).card = d := by
have hlmul : (ℓ.val + 1) * d = ℓ.val * d + d := by simp [Nat.add_mul]
let f : Fin d → Fin n := fun k => ⟨B * d + ℓ.val * d + k.val, by
have hl := ℓ.isLt
have hk := k.isLt
have hb : (ℓ.val + 1) * d ≤ B * d := Nat.mul_le_mul_right d (by omega)
omega⟩
have he : recipientBlock n B d ℓ = Finset.univ.image f := by
ext i
simp only [recipientBlock, mem_filter, Finset.mem_univ, true_and,
mem_image]
constructor
· intro hi
refine ⟨⟨i.val - (B * d + ℓ.val * d), by omega⟩, ?_⟩
apply Fin.ext
dsimp [f]
omega
· rintro ⟨k, _, rfl⟩
dsimp [f]
have hk := k.isLt
constructor <;> omega
rw [he, Finset.card_image_of_injective]
· simp
· intro k k' hk
apply Fin.ext
have hv := congrArg val hk
dsimp [f] at hv
omega
theorem blockEdge_source_iff
The source and recipient labels in a block edge are disjoint. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
blockEdge n B d s j i ↔ i ∈ recipientBlock n B d (s.1 ⟨j.val, hj⟩)
Proof (Lean source)
-- @node: blockEdge_source_iff
lemma blockEdge_source_iff (n B d : ℕ) (s : SourcePartition B d)
(j i : Fin n) (hj : j.val < B * d) :
blockEdge n B d s j i ↔ i ∈ recipientBlock n B d (s.1 ⟨j.val, hj⟩) := by
constructor
· rintro ⟨⟨_, ℓ, he, hi⟩, _⟩
simpa only [he] using hi
· intro hi
refine ⟨⟨hj, _, rfl, hi⟩, ?_⟩
have hb : B * d ≤ i.val := (Finset.mem_filter.mp hi).2.1.trans' (by omega)
intro he
subst i
omega
theorem blockEdge_outgoing_card
A source has exactly d outgoing edges; other labels have none. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(Finset.univ.filter (fun i => blockEdge n B d s j i)).card
= if j.val < B * d then d else 0
Proof (Lean source)
-- @node: blockEdge_outgoing_card
lemma blockEdge_outgoing_card (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (j : Fin n) :
(Finset.univ.filter (fun i => blockEdge n B d s j i)).card =
if j.val < B * d then d else 0 := by
by_cases hj : j.val < B * d
· rw [if_pos hj]
have he : Finset.univ.filter (fun i => blockEdge n B d s j i) =
recipientBlock n B d (s.1 ⟨j.val, hj⟩) := by
ext i
simp only [mem_filter, Finset.mem_univ, true_and,
blockEdge_source_iff n B d s j i hj]
rw [he]
exact recipientBlock_card n B d hfit _
· rw [if_neg hj]
have he : Finset.univ.filter (fun i => blockEdge n B d s j i) = ∅ := by
apply Finset.eq_empty_of_forall_notMem
intro i hi
exact hj (Finset.mem_filter.mp hi).2.1.choose
rw [he, Finset.card_empty]
There are B*d source labels in a fitting block layout. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hfit :
2 * (B * d) ≤ n
conclusion
(Finset.univ.filter (fun j : Fin n => j.val < B * d)).card = B * d
Proof (Lean source)
-- @node: block_source_card
lemma block_source_card (n B d : ℕ) (hfit : 2 * (B * d) ≤ n) :
(Finset.univ.filter (fun j : Fin n => j.val < B * d)).card = B * d := by
by_cases hm : B * d = 0
· simp [hm]
· have hmn : B * d < n := by omega
have he : Finset.univ.filter (fun j : Fin n => j.val < B * d) =
Finset.Iio (⟨B * d, hmn⟩ : Fin n) := by
ext j
simp [Fin.lt_def]
rw [he, Fin.card_Iio]
Every recipient in a block has exactly d incoming source labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
i :
Fin n
ℓ :
Fin B
hi :
i ∈ recipientBlock n B d ℓ
conclusion
(inNbhd (blockSchedule n B d σ h ξ) i).card = d
Proof (Lean source)
-- @node: blockSchedule_inNbhd_card
lemma blockSchedule_inNbhd_card (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ)
(ξ : SourcePartition B d × (Fin B → ℝ)) (i : Fin n) (ℓ : Fin B)
(hi : i ∈ recipientBlock n B d ℓ) :
(inNbhd (blockSchedule n B d σ h ξ) i).card = d := by
let f : Fin (B * d) → Fin n := fun j => ⟨j.val, by have hj := j.isLt; omega⟩
have he : inNbhd (blockSchedule n B d σ h ξ) i =
(Finset.univ.filter (fun j => ξ.1.1 j = ℓ)).image f := by
ext j
change (j ∈ Finset.univ.filter (fun j => blockEdge n B d ξ.1 j i)) ↔ _
simp only [mem_filter, Finset.mem_univ, true_and, mem_image]
constructor
· intro hj
obtain ⟨⟨hjm, k, hjk, hik⟩, _⟩ := hj
have hkl := recipientBlock_unique n B d i k ℓ hik hi
refine ⟨⟨j.val, hjm⟩, ?_, ?_⟩
· exact hjk.trans hkl
· rfl
· rintro ⟨k, hk, rfl⟩
apply (blockEdge_source_iff n B d ξ.1 (f k) i k.isLt).mpr
simpa only [f, hk] using hi
rw [he, Finset.card_image_of_injective]
· exact ξ.1.2 ℓ
· intro j k hjk
exact Fin.ext (congrArg (fun z : Fin n => z.val) hjk)
Labels outside all recipient blocks have no incoming edges. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
i :
Fin n
hi :
∀ ℓ, i ∉ recipientBlock n B d ℓ
conclusion
inNbhd (blockSchedule n B d σ h ξ) i = ∅
Proof (Lean source)
-- @node: blockSchedule_inNbhd_empty
lemma blockSchedule_inNbhd_empty (n B d : ℕ) (σ : Bool) (h : ℝ)
(ξ : SourcePartition B d × (Fin B → ℝ)) (i : Fin n)
(hi : ∀ ℓ, i ∉ recipientBlock n B d ℓ) :
inNbhd (blockSchedule n B d σ h ξ) i = ∅ := by
apply Finset.eq_empty_of_forall_notMem
intro j hj
obtain ⟨⟨_, ℓ, _, hiℓ⟩, _⟩ := (Finset.mem_filter.mp hj).2
exact hi ℓ hiℓ
theorem blockSchedule_mem_class
Baselines in the support interval give the required coefficient-mass and degree bounds. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
σ :
h :
ℝ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
ξ :
SourcePartition B d × (Fin B → ℝ)
hξ :
∀ ℓ, |ξ.2 ℓ| ≤ 1 / 4
conclusion
ScheduleClass (blockSchedule n B d σ h ξ) d
Proof (Lean source)
-- @node: blockSchedule_mem_class
lemma blockSchedule_mem_class (n B d : ℕ) (σ : Bool) (h : ℝ)
(hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hh : h ∈ Icc 0 (1 / 4))
(ξ : SourcePartition B d × (Fin B → ℝ)) (hξ : ∀ ℓ, |ξ.2 ℓ| ≤ 1 / 4) :
ScheduleClass (blockSchedule n B d σ h ξ) d := by
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
have habs : |signOf σ * h / d| = h / d := by
rw [abs_div, abs_mul, show |signOf σ| = 1 by cases σ <;> norm_num [signOf],
abs_of_nonneg hh.1, abs_of_nonneg (Nat.cast_nonneg d), one_mul]
constructor
· intro i
by_cases hi : ∃ ℓ, i ∈ recipientBlock n B d ℓ
· obtain ⟨ℓ, hi⟩ := hi
exact (blockSchedule_inNbhd_card n B d hd hfit σ h ξ i ℓ hi).le
· rw [blockSchedule_inNbhd_empty n B d σ h ξ i (not_exists.mp hi)]
simp
· intro j
have he : (Finset.univ.filter (fun i => j ∈ inNbhd (blockSchedule n B d σ h ξ) i)) =
Finset.univ.filter (fun i => blockEdge n B d ξ.1 j i) := by
ext i
change (i ∈ Finset.univ.filter (fun i => j ∈
Finset.univ.filter (fun j => blockEdge n B d ξ.1 j i))) ↔ _
simp only [mem_filter, Finset.mem_univ, true_and]
rw [he, blockEdge_outgoing_card n B d hfit]
split <;> omega
· intro i
by_cases hi : ∃ ℓ, i ∈ recipientBlock n B d ℓ
· obtain ⟨ℓ, hi⟩ := hi
have ha : (blockSchedule n B d σ h ξ).a i = ξ.2 ℓ - signOf σ * h / 2 := by
change (∑ k : Fin B, if i ∈ recipientBlock n B d k then
ξ.2 k - signOf σ * h / 2 else 0) = _
rw [Finset.sum_eq_single ℓ]
· simp [hi]
· intro k _ hk
rw [if_neg]
exact fun hik => hk (recipientBlock_unique n B d i k ℓ hik hi)
· simp
change |(blockSchedule n B d σ h ξ).a i| + |(0 : ℝ)| +
(∑ j ∈ inNbhd (blockSchedule n B d σ h ξ) i, |signOf σ * h / d|) ≤ 1
rw [ha]
simp only [abs_zero, add_zero, habs, Finset.sum_const, nsmul_eq_mul,
blockSchedule_inNbhd_card n B d hd hfit σ h ξ i ℓ hi]
have he : (d : ℝ) * (h / d) = h := by field_simp
rw [he]
have hs : |signOf σ * h / 2| = h / 2 := by
rw [abs_div, abs_mul, show |signOf σ| = 1 by cases σ <;> norm_num [signOf],
abs_of_nonneg hh.1, abs_of_pos (by norm_num : (0 : ℝ) < 2), one_mul]
have hb := abs_sub (ξ.2 ℓ) (signOf σ * h / 2)
rw [hs] at hb
linarith [hξ ℓ, hh.2]
· have hn := not_exists.mp hi
have ha : (blockSchedule n B d σ h ξ).a i = 0 := by
change (∑ k : Fin B, if i ∈ recipientBlock n B d k then _ else 0) = 0
simp [hn]
change |(blockSchedule n B d σ h ξ).a i| + |(0 : ℝ)| +
(∑ j ∈ inNbhd (blockSchedule n B d σ h ξ) i, |signOf σ * h / d|) ≤ 1
rw [ha, blockSchedule_inNbhd_empty n B d σ h ξ i hn]
norm_num
Summing the outgoing degree counts gives the block prior's constant causal target. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
tte (blockSchedule n B d σ h ξ) = signOf σ * (B * d : ℕ) * h / n
Proof (Lean source)
-- @node: blockSchedule_tte
lemma blockSchedule_tte (n B d : ℕ) (σ : Bool) (h : ℝ)
(hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(ξ : SourcePartition B d × (Fin B → ℝ)) :
tte (blockSchedule n B d σ h ξ) = signOf σ * (B * d : ℕ) * h / n := by
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
simp only [tte, potentialOutcome, blockSchedule, treatment, ite_true,
Bool.false_eq_true, ite_false, mul_one, mul_zero, Finset.sum_const_zero,
add_zero, add_sub_cancel_left, Fintype.card_fin]
simp only [inNbhd, Finset.sum_filter]
rw [Finset.sum_comm]
have hrow (j : Fin n) :
(∑ i : Fin n, if blockEdge n B d ξ.1 j i then signOf σ * h / d else 0) =
if j.val < B * d then signOf σ * h else 0 := by
rw [← Finset.sum_filter]
simp only [Finset.sum_const, nsmul_eq_mul,
blockEdge_outgoing_card n B d hfit]
split
· field_simp
· simp
simp_rw [hrow]
rw [← Finset.sum_filter]
simp only [Finset.sum_const, nsmul_eq_mul, block_source_card n B d hfit]
ring
The compact baseline density gives zero mass outside its declared support. the stated conclusion holds.
Formal statement
conclusion
∀ᵐ w ∂(volume.withDensity (fun w => ofReal (cosSqDensity w))), |w| ≤ (1 / 4 : ℝ)
Proof (Lean source)
-- @node: cosSqDensity_ae_support
lemma cosSqDensity_ae_support :
∀ᵐ w ∂(volume.withDensity (fun w => ofReal (cosSqDensity w))),
|w| ≤ (1 / 4 : ℝ) := by
have hm : Measurable (fun w => ofReal (cosSqDensity w)) := by
apply Measurable.ennreal_ofReal
unfold cosSqDensity
apply Measurable.ite (measurableSet_le (by fun_prop) measurable_const)
· fun_prop
· fun_prop
apply (ae_withDensity_iff hm).mpr
filter_upwards [] with w hw
by_contra h
exact hw (by rw [cosSqDensity, if_neg h, ENNReal.ofReal_zero])
All coordinates of the independent baseline draw lie in the support interval. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: blockBaselineLaw_ae_support
lemma blockBaselineLaw_ae_support (B : ℕ) :
∀ᵐ u ∂(blockBaselineLaw B), ∀ ℓ, |u ℓ| ≤ (1 / 4 : ℝ) := by
unfold blockBaselineLaw
exact Filter.eventually_all.mpr (fun ℓ =>
(Measure.tendsto_eval_ae_ae
(μ := fun _ : Fin B => volume.withDensity (fun w => ofReal (cosSqDensity w)))
(i := ℓ)).eventually cosSqDensity_ae_support)
theorem block_support
Almost every prior schedule belongs to the model class and has the stated constant signed target. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
∀ᵐ ξ ∂(blockParamLaw B d), ScheduleClass (blockSchedule n B d σ h ξ) d ∧ tte (blockSchedule n B d σ h ξ)
= signOf σ * (B * d : ℕ) * h / n
Proof (Lean source)
-- @node: block_support
lemma block_support (n B d : ℕ) (σ : Bool) (h : ℝ)
(hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hh : h ∈ Icc 0 (1 / 4)) :
∀ᵐ ξ ∂(blockParamLaw B d), ScheduleClass (blockSchedule n B d σ h ξ) d ∧
tte (blockSchedule n B d σ h ξ) = signOf σ * (B * d : ℕ) * h / n := by
have hb : ∀ᵐ ξ ∂(blockParamLaw B d), ∀ ℓ, |ξ.2 ℓ| ≤ (1 / 4 : ℝ) := by
have : SigmaFinite (blockBaselineLaw B) := by unfold blockBaselineLaw; infer_instance
exact Measure.quasiMeasurePreserving_snd.ae (blockBaselineLaw_ae_support B)
filter_upwards [hb] with ξ hξ
exact ⟨blockSchedule_mem_class n B d σ h hd hfit hh ξ hξ,
blockSchedule_tte n B d σ h hd hfit ξ⟩
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ChiSqDecomposition 2 declarations Conditional chi-squared decomposition interface
Conditional chi-squared decomposition interface
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.CompatibleCompletions 13 declarations Restriction to the original unrevealed source labels is a bijection between compatible source partitions and words with the observed remaining row capacities.
Compatible partitions as capacity-constrained hidden-label words
Restriction to the original unrevealed source labels is a bijection between compatible source partitions and words with the observed remaining row capacities. The inverse fixes every revealed label at its observed row. Detailed retained edge subsets remain in the compatibility predicate throughout.
Compatible completions of the entire detailed retained graph.
Original source labels whose association was not revealed.
Row words on original hidden labels with exactly the remaining row capacities.
The Boolean reveal statistic is false exactly when the original label has no retained edge. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sourceReveals_false_iff
lemma sourceReveals_false_iff (n B d : ℕ) (H : OffDiag (Fin n) → Bool)
(j : Fin n) (hj : j.val < B * d) :
sourceReveals n B d H ⟨j.val, hj⟩ = false ↔
¬ ∃ i : Fin n, ∃ hji : j ≠ i, H ⟨(j,i),hji⟩ = true := by
simp only [sourceReveals, decide_eq_false_iff_not]
constructor
· intro hn he
exact hn ⟨j, rfl, he⟩
· intro hn
rintro ⟨v, hv, he⟩
have hvj : v = j := Fin.ext hv
subst v
exact hn he
theorem compatiblePartition_agree_revealed
Any two compatible completions agree on every revealed original source label. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
s.1.1 j = t.1.1 j
Proof (Lean source)
-- @node: compatiblePartition_agree_revealed
lemma compatiblePartition_agree_revealed (n B d : ℕ)
(H : OffDiag (Fin n) → Bool) (s t : CompatiblePartition n B d H)
(j : Fin (B * d)) (hj : sourceReveals n B d H j ≠ false) : s.1.1 j = t.1.1 j := by
have hj' : ∃ v : Fin n, v.val = j.val ∧
∃ i : Fin n, ∃ hvi : v ≠ i, H ⟨(v,i),hvi⟩ = true := by
by_contra hn
exact hj (by simp [sourceReveals, hn])
obtain ⟨v, hv, i, hvi, he⟩ := hj'
obtain ⟨⟨hs, ℓ, hsℓ, hiℓ⟩, _⟩ := s.2 ⟨(v,i),hvi⟩ he
obtain ⟨⟨ht, k, htk, hik⟩, _⟩ := t.2 ⟨(v,i),hvi⟩ he
have hsj : (⟨v.val, hs⟩ : Fin (B * d)) = j := Fin.ext hv
have htj : (⟨v.val, ht⟩ : Fin (B * d)) = j := Fin.ext hv
rw [hsj] at hsℓ
rw [htj] at htk
exact hsℓ.trans ((recipientBlock_unique n B d i ℓ k hiℓ hik).trans htk.symm)
def compatiblePartitionRestriction
The actual hidden word of a compatible completion.
Definition (Lean source)
def compatiblePartitionEquivHiddenCompletion
Compatible partitions and capacity-constrained words on the original hidden labels are in bijection; every revealed source remains fixed.
Definition (Lean source)
parameters
def
compatiblePartitionEquivHiddenCompletion n B d hfit H s₀ :
CompatiblePartition n B d H ≃ HiddenCompletion n B d H
given by
clause 1
toFun := compatiblePartitionRestriction n B d hfit H
clause 2
invFun := hiddenCompletionPartition n B d hfit H s₀
clause 3
left_inv := by intro s apply Subtype.ext apply Subtype.ext funext j change (if hj : sourceReveals n B d H j = false then s.1.1 j else s₀.1.1 j)
= s.1.1 j split_ifs with hj · rfl · exact compatiblePartition_agree_revealed n B d H s₀ s j hj
clause 4
right_inv := by intro g apply Subtype.ext funext j exact dif_pos j.2
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.CompleteBlockConstruction 32 declarations Complete-block construction, support, and response moments
Complete-block construction, support, and response moments
The fixed complete directed blocks of augmented degree d plus one.
Definition (Lean source)
parameters
n d :
ℕ
j i :
Fin n
def
completeBlockEdge n d j i :
Prop
given by
clause 1
j.val < (n / (d + 1)) * (d + 1)
i.val < (n / (d + 1)) * (d + 1)
j.val / (d + 1) = i.val / (d + 1)
clause 2
j ≠ i
theorem completeBlockEdge_irrefl
Complete block edges exclude all diagonal arrows. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlockEdge_irrefl
lemma completeBlockEdge_irrefl (n d : ℕ) : ∀ i, ¬ completeBlockEdge n d i i := by
intro i h
exact h.2 rfl
def completeBlockSchedule
Complete directed blocks of size d+1, sharing one fixed graph under both prior signs.
Definition (Lean source)
parameters
def
given by
clause 1
edge := completeBlockEdge n d
clause 2
decEdge := Classical.decRel _
clause 3
irrefl := completeBlockEdge_irrefl n d
clause 4
a := fun i
=> ∑ v : Fin (n / (d + 1)), if i.val / (d + 1) = v.val then U v - signOf σ * h / 2 else 0
clause 5
t := fun i => if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h / (d + 1) else 0
clause 6
b := fun _ _ => signOf σ * h / (d + 1)
The labeled members of one complete block, including its own coordinate.
theorem completeBlockMembers_card
Each full block has exactly d+1 members. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlockMembers_card
lemma completeBlockMembers_card (n d : ℕ) (v : Fin (n / (d + 1))) :
(completeBlockMembers n d v).card = d + 1 := by
let f : Fin (d + 1) → Fin n := fun j => ⟨v.val * (d + 1) + j.val, by
have hv := Nat.mul_le_mul_right (d + 1) (show v.val + 1 ≤ n / (d + 1) by omega)
have hn := Nat.div_mul_le_self n (d + 1)
have hj := j.isLt
rw [Nat.add_mul] at hv
omega⟩
have hf (j : Fin (d + 1)) : (f j).val / (d + 1) = v.val := by
dsimp [f]
rw [Nat.mul_comm v.val (d + 1), Nat.mul_add_div (by omega : 0 < d + 1), Nat.div_eq_of_lt j.isLt, add_zero]
have he : completeBlockMembers n d v = Finset.univ.image f := by
ext i
simp only [completeBlockMembers, mem_filter, Finset.mem_univ, true_and,
mem_image]
constructor
· intro hi
refine ⟨⟨i.val % (d + 1), Nat.mod_lt _ (by omega)⟩, ?_⟩
apply Fin.ext
dsimp [f]
rw [← hi, Nat.mul_comm, Nat.div_add_mod]
· rintro ⟨j, _, rfl⟩
exact hf j
rw [he, Finset.card_image_of_injective]
· simp
· intro j k hjk
apply Fin.ext
have hv := congrArg val hjk
dsimp [f] at hv
omega
Quotients index a full block precisely for nonpadding labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
i :
Fin n
conclusion
i.val < (n / (d + 1)) * (d + 1) ↔ i.val / (d + 1) < n / (d + 1)
Proof (Lean source)
-- @node: completeBlock_active_iff
lemma completeBlock_active_iff (n d : ℕ) (i : Fin n) :
i.val < (n / (d + 1)) * (d + 1) ↔ i.val / (d + 1) < n / (d + 1) := by
exact (Nat.div_lt_iff_lt_mul (by omega : 0 < d + 1)).symm
theorem completeBlockSchedule_inNbhd
A nonpadding label's incoming neighborhood is its block with itself removed. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
inNbhd (completeBlockSchedule n d σ h U) i
Proof (Lean source)
-- @node: completeBlockSchedule_inNbhd
lemma completeBlockSchedule_inNbhd (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n)
(hi : i.val < (n / (d + 1)) * (d + 1)) :
inNbhd (completeBlockSchedule n d σ h U) i =
(completeBlockMembers n d ⟨i.val / (d + 1),
(completeBlock_active_iff n d i).mp hi⟩).erase i := by
ext j
change (j ∈ Finset.univ.filter (fun j => completeBlockEdge n d j i)) ↔ _
simp only [mem_filter, Finset.mem_univ, true_and, Finset.mem_erase]
simp only [completeBlockMembers, mem_filter, Finset.mem_univ, true_and]
change completeBlockEdge n d j i ↔ j ≠ i ∧ j.val / (d + 1) = i.val / (d + 1)
unfold completeBlockEdge
constructor
· rintro ⟨⟨hj, _, he⟩, hne⟩
exact ⟨hne, he⟩
· rintro ⟨hne, he⟩
have hj : j.val < (n / (d + 1)) * (d + 1) :=
(completeBlock_active_iff n d j).mpr (he ▸ (completeBlock_active_iff n d i).mp hi)
exact ⟨⟨hj, hi, he⟩, hne⟩
theorem completeBlockSchedule_inNbhd_padding
Padding labels have no incoming arrows. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
inNbhd (completeBlockSchedule n d σ h U) i = ∅
Proof (Lean source)
-- @node: completeBlockSchedule_inNbhd_padding
lemma completeBlockSchedule_inNbhd_padding (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n)
(hi : ¬ i.val < (n / (d + 1)) * (d + 1)) :
inNbhd (completeBlockSchedule n d σ h U) i = ∅ := by
apply Finset.eq_empty_of_forall_notMem
intro j hj
exact hi (Finset.mem_filter.mp hj).2.1.2.1
theorem completeBlockSchedule_inNbhd_card
Full block rows have off-diagonal degree d; padding rows have degree zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(inNbhd (completeBlockSchedule n d σ h U) i).card
= if i.val < (n / (d + 1)) * (d + 1) then d else 0
Proof (Lean source)
-- @node: completeBlockSchedule_inNbhd_card
lemma completeBlockSchedule_inNbhd_card (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n) :
(inNbhd (completeBlockSchedule n d σ h U) i).card =
if i.val < (n / (d + 1)) * (d + 1) then d else 0 := by
by_cases hi : i.val < (n / (d + 1)) * (d + 1)
· rw [if_pos hi, completeBlockSchedule_inNbhd n d σ h U i hi,
Finset.card_erase_of_mem (by simp [completeBlockMembers]), completeBlockMembers_card]
omega
· rw [if_neg hi, completeBlockSchedule_inNbhd_padding n d σ h U i hi,
Finset.card_empty]
theorem completeBlockEdge_symm
Complete directed block edges are symmetric, so outgoing degrees equal incoming degrees. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlockEdge_symm
lemma completeBlockEdge_symm (n d : ℕ) (j i : Fin n) :
completeBlockEdge n d j i ↔ completeBlockEdge n d i j := by
constructor <;> rintro ⟨⟨hj, hi, he⟩, hne⟩
· exact ⟨⟨hi, hj, he.symm⟩, Ne.symm hne⟩
· exact ⟨⟨hi, hj, he.symm⟩, Ne.symm hne⟩
theorem completeBlockSchedule_baseline
A full block row uses exactly its own baseline. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(completeBlockSchedule n d σ h U).a i
= U ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩ - signOf σ * h / 2
Proof (Lean source)
-- @node: completeBlockSchedule_baseline
lemma completeBlockSchedule_baseline (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n)
(hi : i.val < (n / (d + 1)) * (d + 1)) :
(completeBlockSchedule n d σ h U).a i =
U ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩ - signOf σ * h / 2 := by
let v : Fin (n / (d + 1)) := ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩
change (∑ w, if i.val / (d + 1) = w.val then U w - signOf σ * h / 2 else 0) = _
rw [Finset.sum_eq_single v]
· simp [v]
· intro w _ hw
apply if_neg
intro he
exact hw (Fin.ext he.symm)
· simp
theorem completeBlockSchedule_baseline_padding
Padding rows have zero baseline as well as zero treatment effects. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(completeBlockSchedule n d σ h U).a i = 0
Proof (Lean source)
-- @node: completeBlockSchedule_baseline_padding
lemma completeBlockSchedule_baseline_padding (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n)
(hi : ¬ i.val < (n / (d + 1)) * (d + 1)) :
(completeBlockSchedule n d σ h U).a i = 0 := by
change (∑ w, if i.val / (d + 1) = w.val then U w - signOf σ * h / 2 else 0) = 0
apply Finset.sum_eq_zero
intro w _
apply if_neg
intro he
exact hi ((completeBlock_active_iff n d i).mpr (he ▸ w.isLt))
theorem completeBlockSchedule_mem_class
Baselines and amplitudes in the declared support produce admissible complete-block schedules. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
ScheduleClass (completeBlockSchedule n d σ h U) d
Proof (Lean source)
-- @node: completeBlockSchedule_mem_class
lemma completeBlockSchedule_mem_class (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (hh : h ∈ Icc 0 (1 / 4))
(hU : ∀ v, |U v| ≤ 1 / 4) : ScheduleClass (completeBlockSchedule n d σ h U) d := by
have hk : (0 : ℝ) < (d : ℝ) + 1 := by positivity
have hs : |signOf σ| = 1 := by cases σ <;> norm_num [signOf]
have habs : |signOf σ * h / (d + 1)| = h / (d + 1) := by
rw [abs_div, abs_mul, hs, abs_of_nonneg hh.1, abs_of_pos hk, one_mul]
constructor
· intro i
rw [completeBlockSchedule_inNbhd_card]
split <;> omega
· intro j
have he : Finset.univ.filter (fun i => j ∈ inNbhd (completeBlockSchedule n d σ h U) i) =
inNbhd (completeBlockSchedule n d σ h U) j := by
ext i
change (i ∈ Finset.univ.filter (fun i => j ∈ Finset.univ.filter
(fun j => completeBlockEdge n d j i))) ↔
(i ∈ Finset.univ.filter (fun i => completeBlockEdge n d i j))
simp only [mem_filter, Finset.mem_univ, true_and, completeBlockEdge_symm]
rw [he, completeBlockSchedule_inNbhd_card]
split <;> omega
· intro i
by_cases hi : i.val < (n / (d + 1)) * (d + 1)
· have ha := completeBlockSchedule_baseline n d σ h U i hi
change |(completeBlockSchedule n d σ h U).a i| +
|if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h / (d + 1) else 0| +
(∑ j ∈ inNbhd (completeBlockSchedule n d σ h U) i, |signOf σ * h / (d + 1)|) ≤ 1
rw [ha, if_pos hi]
simp only [habs, Finset.sum_const, nsmul_eq_mul,
completeBlockSchedule_inNbhd_card, if_pos hi, Nat.cast_add, Nat.cast_one]
have hm : h / ((d : ℝ) + 1) + d * (h / (d + 1)) = h := by field_simp; ring
rw [add_assoc, hm]
have ht : |signOf σ * h / 2| = h / 2 := by
rw [abs_div, abs_mul, hs, abs_of_nonneg hh.1]
norm_num
have hb := abs_sub (U ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩)
(signOf σ * h / 2)
rw [ht] at hb
linarith [hU ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩, hh.2]
· change |(completeBlockSchedule n d σ h U).a i| +
|if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h / (d + 1) else 0| +
(∑ j ∈ inNbhd (completeBlockSchedule n d σ h U) i, |signOf σ * h / (d + 1)|) ≤ 1
rw [completeBlockSchedule_baseline_padding n d σ h U i hi, if_neg hi,
completeBlockSchedule_inNbhd_padding n d σ h U i hi]
norm_num
The number of nonpadding labels is the number of full blocks times their size. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
conclusion
(Finset.univ.filter (fun i : Fin n => i.val < (n / (d + 1)) * (d + 1))).card
= (n / (d + 1)) * (d + 1)
Proof (Lean source)
-- @node: completeBlock_active_card
lemma completeBlock_active_card (n d : ℕ) :
(Finset.univ.filter (fun i : Fin n => i.val < (n / (d + 1)) * (d + 1))).card =
(n / (d + 1)) * (d + 1) := by
let m := (n / (d + 1)) * (d + 1)
have hmn : m ≤ n := Nat.div_mul_le_self n (d + 1)
change (Finset.univ.filter (fun i : Fin n => i.val < m)).card = m
by_cases he : m = n
· have hall : ∀ i : Fin n, i.val < m := fun i => by have hi := i.isLt; omega
simp [hall, he]
· have hm : m < n := by omega
have hset : Finset.univ.filter (fun i : Fin n => i.val < m) =
Finset.Iio (⟨m, hm⟩ : Fin n) := by
ext i
simp [Fin.lt_def]
change (Finset.univ.filter (fun i : Fin n => i.val < m)).card = m
rw [hset, Fin.card_Iio]
theorem completeBlockSchedule_response
A complete-block response is its baseline plus the translated centered block sign sum. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
potentialOutcome (completeBlockSchedule n d σ h U) i z
= U ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩
+ signOf σ * h / (2 * (d + 1)) * ∑ j ∈ completeBlockMembers n d ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩, signOf (z j)
Proof (Lean source)
-- @node: completeBlockSchedule_response
lemma completeBlockSchedule_response (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n) (z : Assign (Fin n))
(hi : i.val < (n / (d + 1)) * (d + 1)) :
potentialOutcome (completeBlockSchedule n d σ h U) i z =
U ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩ +
signOf σ * h / (2 * (d + 1)) *
∑ j ∈ completeBlockMembers n d
⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩, signOf (z j) := by
let v : Fin (n / (d + 1)) := ⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩
let C := completeBlockMembers n d v
have hmem : i ∈ C := by simp [C, completeBlockMembers, v]
have hsum := Finset.sum_erase_add C (fun j => treatment (z j)) hmem
have hcenter : (∑ j ∈ C, signOf (z j)) =
2 * (∑ j ∈ C, treatment (z j)) - (d + 1) := by
have hsign (j : Fin n) : signOf (z j) = 2 * treatment (z j) - 1 := by
cases z j <;> norm_num [signOf, treatment]
simp_rw [hsign]
rw [Finset.sum_sub_distrib, ← Finset.mul_sum]
simp [C, completeBlockMembers_card]
unfold potentialOutcome
rw [completeBlockSchedule_baseline n d σ h U i hi,
completeBlockSchedule_inNbhd n d σ h U i hi]
change U v - signOf σ * h / 2 +
(if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h / (d + 1) else 0) *
treatment (z i) +
(∑ j ∈ C.erase i, signOf σ * h / (d + 1) * treatment (z j)) =
U v + signOf σ * h / (2 * (d + 1)) * ∑ j ∈ C, signOf (z j)
rw [if_pos hi, ← Finset.mul_sum, hcenter, ← hsum]
field_simp
<;> ring
theorem completeBlockSchedule_row_contrast
Each nonpadding row has total-treatment contrast sigma*h, and padding rows have zero contrast. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h else 0
Proof (Lean source)
-- @node: completeBlockSchedule_row_contrast
lemma completeBlockSchedule_row_contrast (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n) :
potentialOutcome (completeBlockSchedule n d σ h U) i (fun _ => true) -
potentialOutcome (completeBlockSchedule n d σ h U) i (fun _ => false) =
if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h else 0 := by
simp only [potentialOutcome, treatment, ite_true, Bool.false_eq_true, ite_false,
mul_one, mul_zero, Finset.sum_const_zero, add_zero, add_assoc, add_sub_cancel_left]
change (if i.val < (n / (d + 1)) * (d + 1) then signOf σ * h / (d + 1) else 0) +
(∑ j ∈ inNbhd (completeBlockSchedule n d σ h U) i, signOf σ * h / (d + 1)) = _
simp only [Finset.sum_const, nsmul_eq_mul, completeBlockSchedule_inNbhd_card]
by_cases hi : i.val < (n / (d + 1)) * (d + 1)
· rw [if_pos hi, if_pos hi, if_pos hi]
push_cast
field_simp
<;> ring
· simp [hi]
theorem completeBlockSchedule_tte
The complete-block construction has its constant signed target on every baseline draw. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
tte (completeBlockSchedule n d σ h U) = signOf σ * ((n / (d + 1)) * (d + 1) : ℕ) * h / n
Proof (Lean source)
-- @node: completeBlockSchedule_tte
lemma completeBlockSchedule_tte (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) :
tte (completeBlockSchedule n d σ h U) =
signOf σ * ((n / (d + 1)) * (d + 1) : ℕ) * h / n := by
simp only [tte, Fintype.card_fin, completeBlockSchedule_row_contrast]
rw [← Finset.sum_filter]
simp only [Finset.sum_const, nsmul_eq_mul, completeBlock_active_card]
ring
theorem completeBlock_support
The independent compactly supported baselines put the complete-block prior in the model class. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
∀ᵐ U ∂(blockBaselineLaw (n / (d + 1))), ScheduleClass (completeBlockSchedule n d σ h U) d ∧ tte (completeBlockSchedule n d σ h U)
= signOf σ * ((n / (d + 1)) * (d + 1) : ℕ) * h / n
Proof (Lean source)
-- @node: completeBlock_support
lemma completeBlock_support (n d : ℕ) (σ : Bool) (h : ℝ)
(hh : h ∈ Icc 0 (1 / 4)) :
∀ᵐ U ∂(blockBaselineLaw (n / (d + 1))),
ScheduleClass (completeBlockSchedule n d σ h U) d ∧
tte (completeBlockSchedule n d σ h U) =
signOf σ * ((n / (d + 1)) * (d + 1) : ℕ) * h / n := by
filter_upwards [blockBaselineLaw_ae_support (n / (d + 1))] with U hU
exact ⟨completeBlockSchedule_mem_class n d σ h U hh hU,
completeBlockSchedule_tte n d σ h U⟩
At least one complete block fits for every admissible degree. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
hn :
4 ≤ n
hdu :
d ≤ n - 1
conclusion
1 ≤ n / (d + 1)
Proof (Lean source)
-- @node: completeBlock_count_pos
lemma completeBlock_count_pos (n d : ℕ) (hn : 4 ≤ n) (hdu : d ≤ n - 1) :
1 ≤ n / (d + 1) := by
apply (Nat.le_div_iff_mul_le (by omega : 0 < d + 1)).mpr
omega
The full blocks cover at least half the population, as required by the testing separation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
hn :
4 ≤ n
hdu :
d ≤ n - 1
conclusion
(1 / 2 : ℝ) ≤ (((n / (d + 1)) * (d + 1) : ℕ) : ℝ) / n
Proof (Lean source)
-- @node: completeBlock_coverage_ge_half
lemma completeBlock_coverage_ge_half (n d : ℕ) (hn : 4 ≤ n) (hdu : d ≤ n - 1) :
(1 / 2 : ℝ) ≤ (((n / (d + 1)) * (d + 1) : ℕ) : ℝ) / n := by
have hr := completeBlock_count_pos n d hn hdu
have hk : d + 1 ≤ (n / (d + 1)) * (d + 1) := by
simpa using Nat.mul_le_mul_right (d + 1) hr
have hrem := Nat.mod_lt n (by omega : 0 < d + 1)
have hdiv := Nat.div_add_mod n (d + 1)
have htwice : n ≤ 2 * ((n / (d + 1)) * (d + 1)) := by
rw [Nat.mul_comm (d + 1)] at hdiv
omega
have htwiceR : (n : ℝ) ≤ 2 * (((n / (d + 1)) * (d + 1) : ℕ) : ℝ) := by
exact_mod_cast htwice
have hnR : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
apply (le_div_iff₀ hnR).mpr
linarith
theorem completeBlock_sign_second_moment
The centered sign sum in a full labeled block has second moment equal to its size. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
v :
Fin (n / (d + 1))
conclusion
(∫ z, (∑ j ∈ completeBlockMembers n d v, signOf (z j)) ^ 2 ∂halfBernoulli (Fin n))
= (d : ℝ) + 1
Proof (Lean source)
-- @node: completeBlock_sign_second_moment
lemma completeBlock_sign_second_moment (n d : ℕ) (v : Fin (n / (d + 1))) :
(∫ z, (∑ j ∈ completeBlockMembers n d v, signOf (z j)) ^ 2
∂halfBernoulli (Fin n)) = (d : ℝ) + 1 := by
rw [reverse_revealed_second_moment, completeBlockMembers_card]
simp
theorem completeBlock_design_second_moment
The actual design has the same block sign moment, with all audit coordinates retained. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ ω, (∑ j ∈ completeBlockMembers n d v, signOf (ω.1 j)) ^ 2 ∂D) = (d : ℝ) + 1
Proof (Lean source)
-- @node: completeBlock_design_second_moment
lemma completeBlock_design_second_moment (n d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (ha : AssignmentLaw D)
(v : Fin (n / (d + 1))) :
(∫ ω, (∑ j ∈ completeBlockMembers n d v, signOf (ω.1 j)) ^ 2 ∂D) =
(d : ℝ) + 1 := by
rw [← integral_map measurable_fst.aemeasurable
(show AEStronglyMeasurable (fun z : Assign (Fin n) =>
(∑ j ∈ completeBlockMembers n d v, signOf (z j)) ^ 2) (D.map fst) from
(measurable_of_finite _).aestronglyMeasurable), ha]
exact completeBlock_sign_second_moment n d v
def completeBlockShift
The baseline translation in a complete block retains every member's observed sign.
Definition (Lean source)
parameters
def
completeBlockShift n d σ h z v :
ℝ
given by
signOf σ * h / (2 * (d + 1)) * ∑ j ∈ completeBlockMembers n d v, signOf (z j)
theorem completeBlock_shift_separation_moment
The squared separation of opposite-sign translations has its exact design expectation. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlock_shift_separation_moment
lemma completeBlock_shift_separation_moment (n d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (ha : AssignmentLaw D)
(v : Fin (n / (d + 1))) :
(∫ ω, (completeBlockShift n d true h ω.1 v -
completeBlockShift n d false h ω.1 v) ^ 2 ∂D) = h ^ 2 / (d + 1) := by
have he (z : Assign (Fin n)) :
(completeBlockShift n d true h z v - completeBlockShift n d false h z v) ^ 2 =
(h / (d + 1)) ^ 2 * (∑ j ∈ completeBlockMembers n d v, signOf (z j)) ^ 2 := by
simp only [completeBlockShift, signOf, Bool.true_eq, Bool.false_eq_true,
ite_true, ite_false]
field_simp
<;> ring
simp_rw [he]
rw [integral_const_mul, completeBlock_design_second_moment n d D ha v]
have hk : (d : ℝ) + 1 ≠ 0 := by positivity
field_simp
def completeBlockResponseDensity
Product of the baseline densities translated by the observed complete-block sign sums.
Definition (Lean source)
parameters
def
completeBlockResponseDensity n d σ h z y :
ℝ
given by
∏ v, cosSqDensity (y v - completeBlockShift n d σ h z v)
theorem completeBlock_response_energy_le
Tensorized baseline translation bounds the distinct-response energy for every assignment. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
hellingerEnergy volume (completeBlockResponseDensity n d true h z) (completeBlockResponseDensity n d false h z)
≤ ∑ v : Fin (n / (d + 1)), 4 * pi ^ 2 * (completeBlockShift n d true h z v - completeBlockShift n d false h z v) ^ 2
Proof (Lean source)
-- @node: completeBlock_response_energy_le
lemma completeBlock_response_energy_le (n d : ℕ) (h : ℝ) (z : Assign (Fin n)) :
hellingerEnergy volume (completeBlockResponseDensity n d true h z)
(completeBlockResponseDensity n d false h z) ≤
∑ v : Fin (n / (d + 1)), 4 * pi ^ 2 *
(completeBlockShift n d true h z v - completeBlockShift n d false h z v) ^ 2 := by
exact baseline_translation_affinity.2.2.2.1 (n / (d + 1))
(completeBlockShift n d true h z) (completeBlockShift n d false h z)
theorem completeBlock_response_energy_mean_le
Averaging over the full assignment-audit design gives the sharp complete-block energy bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ ω, hellingerEnergy volume (completeBlockResponseDensity n d true h ω.1) (completeBlockResponseDensity n d false h ω.1) ∂D)
≤ 4 * pi ^ 2 * h ^ 2 * (n / (d + 1) : ℕ) / (d + 1)
Proof (Lean source)
-- @node: completeBlock_response_energy_mean_le
lemma completeBlock_response_energy_mean_le (n d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (ha : AssignmentLaw D) :
(∫ ω, hellingerEnergy volume (completeBlockResponseDensity n d true h ω.1)
(completeBlockResponseDensity n d false h ω.1) ∂D) ≤
4 * pi ^ 2 * h ^ 2 * (n / (d + 1) : ℕ) / (d + 1) := by
let := halfBernoulli_probability (V := Fin n)
have hp : IsProbabilityMeasure (D.map fst) := by rw [ha]; infer_instance
let := hp
let : IsProbabilityMeasure D := Measure.isProbabilityMeasure_of_map fst
calc
_ ≤ ∫ ω, ∑ v : Fin (n / (d + 1)), 4 * pi ^ 2 *
(completeBlockShift n d true h ω.1 v -
completeBlockShift n d false h ω.1 v) ^ 2 ∂D :=
integral_mono Integrable.of_finite Integrable.of_finite
(fun ω => completeBlock_response_energy_le n d h ω.1)
_ = _ := by
rw [integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
simp_rw [integral_const_mul, completeBlock_shift_separation_moment n d h D ha]
simp only [Finset.sum_const, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul]
ring
theorem completeBlockResponseDensity_measurable
The complete-block response density is jointly measurable in the full design and responses.
Formal statement
hypotheses
conclusion
Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin (n / (d + 1)) → ℝ) => completeBlockResponseDensity n d σ h x.1.1 x.2)
Proof (Lean source)
-- @node: completeBlockResponseDensity_measurable
@[fun_prop]
lemma completeBlockResponseDensity_measurable (n d : ℕ) (σ : Bool) (h : ℝ) :
Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) ×
(Fin (n / (d + 1)) → ℝ) => completeBlockResponseDensity n d σ h x.1.1 x.2) := by
apply measurable_from_prod_countable_right
intro ω
unfold completeBlockResponseDensity cosSqDensity
apply Finset.measurable_prod
intro v _
apply Measurable.ite (measurableSet_le (by fun_prop) measurable_const) <;> fun_prop
theorem completeBlockResponseDensity_nonneg
Every conditional complete-block density is nonnegative. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
0 ≤ completeBlockResponseDensity n d σ h z y
Proof (Lean source)
-- @node: completeBlockResponseDensity_nonneg
lemma completeBlockResponseDensity_nonneg (n d : ℕ) (σ : Bool) (h : ℝ)
(z : Assign (Fin n)) (y : Fin (n / (d + 1)) → ℝ) :
0 ≤ completeBlockResponseDensity n d σ h z y := by
exact Finset.prod_nonneg (fun _ _ => cosSqDensity_nonneg _)
theorem completeBlockResponseDensity_integrable_normalized
Independent translated baselines give an integrable, normalized product density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
Integrable (completeBlockResponseDensity n d σ h z)
conclusion 2
(∫ y, completeBlockResponseDensity n d σ h z y) = 1
Proof (Lean source)
-- @node: completeBlockResponseDensity_integrable_normalized
lemma completeBlockResponseDensity_integrable_normalized (n d : ℕ) (σ : Bool) (h : ℝ)
(z : Assign (Fin n)) :
Integrable (completeBlockResponseDensity n d σ h z) ∧
(∫ y, completeBlockResponseDensity n d σ h z y) = 1 := by
constructor
· exact Integrable.fintype_prod (fun v =>
(translated_cosSqDensity_integrable_normalized (completeBlockShift n d σ h z v)).1)
· change (∫ y : Fin (n / (d + 1)) → ℝ,
∏ v, cosSqDensity (y v - completeBlockShift n d σ h z v)
∂Measure.pi (fun _ => volume)) = 1
rw [integral_fintype_prod_eq_prod
(fun v w => cosSqDensity (w - completeBlockShift n d σ h z v))]
simp only [(translated_cosSqDensity_integrable_normalized _).2, Finset.prod_const_one]
theorem completeBlockResponseDensity_probability
Each conditional product response law is a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (volume.withDensity (fun y => ofReal (completeBlockResponseDensity n d σ h z y)))
Proof (Lean source)
-- @node: completeBlockResponseDensity_probability
lemma completeBlockResponseDensity_probability (n d : ℕ) (σ : Bool) (h : ℝ)
(z : Assign (Fin n)) :
IsProbabilityMeasure (volume.withDensity
(fun y => ofReal (completeBlockResponseDensity n d σ h z y))) := by
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal
(completeBlockResponseDensity_integrable_normalized n d σ h z).1
(Filter.Eventually.of_forall (completeBlockResponseDensity_nonneg n d σ h z)),
(completeBlockResponseDensity_integrable_normalized n d σ h z).2]
simp
theorem completeBlock_response_energy_population_le
The averaged complete-block response energy is at most the population-scale envelope. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ ω, hellingerEnergy volume (completeBlockResponseDensity n d true h ω.1) (completeBlockResponseDensity n d false h ω.1) ∂D)
≤ 4 * pi ^ 2 * h ^ 2 * n / ((d : ℝ) + 1) ^ 2
Proof (Lean source)
-- @node: completeBlock_response_energy_population_le
lemma completeBlock_response_energy_population_le (n d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (ha : AssignmentLaw D) :
(∫ ω, hellingerEnergy volume (completeBlockResponseDensity n d true h ω.1)
(completeBlockResponseDensity n d false h ω.1) ∂D) ≤
4 * pi ^ 2 * h ^ 2 * n / ((d : ℝ) + 1) ^ 2 := by
refine (completeBlock_response_energy_mean_le n d h D ha).trans ?_
have hk : (0 : ℝ) < (d : ℝ) + 1 := by positivity
have hr : (n / (d + 1) : ℕ) * ((d : ℝ) + 1) ≤ n := by
exact_mod_cast Nat.div_mul_le_self n (d + 1)
apply (div_le_div_iff₀ hk (sq_pos_of_pos hk)).mpr
nlinarith [mul_le_mul_of_nonneg_left hr (show 0 ≤ 4 * pi ^ 2 * h ^ 2 by positivity)]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.CompleteBlockDensity 8 declarations Translation of the independent prior baselines gives the normalized product response density, conditional on every coordinate of the actual design.
Complete-block conditional density identification
Translation of the independent prior baselines gives the normalized product response density, conditional on every coordinate of the actual design.
The product of translated baseline laws is the product density against vector volume. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
s :
Fin B → ℝ
conclusion
Measure.pi (fun v => volume.withDensity (fun w => ofReal (cosSqDensity (w - s v))))
Proof (Lean source)
-- @node: baseline_pi_translated_density
lemma baseline_pi_translated_density (B : ℕ) (s : Fin B → ℝ) :
Measure.pi (fun v => volume.withDensity
(fun w => ofReal (cosSqDensity (w - s v)))) =
volume.withDensity (fun y : Fin B → ℝ =>
ofReal (∏ v, cosSqDensity (y v - s v))) := by
apply Measure.pi_eq
intro E hE
rw [withDensity_apply _ (.univ_pi hE), ← lintegral_indicator (.univ_pi hE)]
have he : (Set.univ.pi E).indicator
(fun y : Fin B → ℝ => ofReal (∏ v, cosSqDensity (y v - s v))) =
fun y => ofReal (∏ v, (E v).indicator
(fun w => cosSqDensity (w - s v)) (y v)) := by
funext y
by_cases hy : y ∈ Set.univ.pi E
· have hy' : ∀ v, y v ∈ E v := by simpa using hy
simp [hy']
· have hy' : ∃ v, y v ∉ E v := by simpa using hy
obtain ⟨v, hv⟩ := hy'
rw [Set.indicator_of_notMem hy]
have hz : (∏ v, (E v).indicator (fun w => cosSqDensity (w - s v)) (y v)) = 0 := by
apply Finset.prod_eq_zero (Finset.mem_univ v)
exact Set.indicator_of_notMem hv _
rw [hz, ENNReal.ofReal_zero]
rw [he]
change (∫⁻ y : Fin B → ℝ, ofReal (∏ v,
(E v).indicator (fun w => cosSqDensity (w - s v)) (y v))
∂Measure.pi (fun _ => volume)) = _
rw [← ofReal_integral_eq_lintegral_ofReal
(Integrable.fintype_prod (fun v =>
((translated_cosSqDensity_integrable_normalized (s v)).1.indicator (hE v))))
(Filter.Eventually.of_forall (fun y => Finset.prod_nonneg (fun v _ =>
Set.indicator_nonneg (fun _ _ => cosSqDensity_nonneg _) _))),
integral_fintype_prod_eq_prod]
rw [ENNReal.ofReal_prod_of_nonneg (fun v _ => integral_nonneg
(fun w => Set.indicator_nonneg (fun _ _ => cosSqDensity_nonneg _) w))]
apply Finset.prod_congr rfl
intro v _
rw [withDensity_apply _ (hE v), ← lintegral_indicator (hE v)]
have he' : (E v).indicator (fun w => ofReal (cosSqDensity (w - s v))) =
fun w => ofReal ((E v).indicator (fun w => cosSqDensity (w - s v)) w) := by
funext w
by_cases hw : w ∈ E v <;> simp [indicator, hw]
rw [he']
exact ofReal_integral_eq_lintegral_ofReal
((translated_cosSqDensity_integrable_normalized (s v)).1.indicator (hE v))
(Filter.Eventually.of_forall (fun w =>
Set.indicator_nonneg (fun _ _ => cosSqDensity_nonneg _) w))
Independently translating all prior baselines gives the exact vector response density. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: blockBaselineLaw_map_translation
lemma blockBaselineLaw_map_translation (B : ℕ) (s : Fin B → ℝ) :
(blockBaselineLaw B).map (fun U v => U v + s v) =
volume.withDensity (fun y : Fin B → ℝ =>
ofReal (∏ v, cosSqDensity (y v - s v))) := by
have hp (v : Fin B) : IsProbabilityMeasure
((volume.withDensity (fun w => ofReal (cosSqDensity w))).map
(fun w => w + s v)) := by
rw [baseline_map_add_density]
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal
(translated_cosSqDensity_integrable_normalized (s v)).1
(Filter.Eventually.of_forall (fun _ => cosSqDensity_nonneg _)),
(translated_cosSqDensity_integrable_normalized (s v)).2]
simp
let _ := hp
rw [blockBaselineLaw, Measure.pi_map_pi (f := fun v w => w + s v) (fun _ => (by fun_prop))]
simp_rw [baseline_map_add_density]
exact baseline_pi_translated_density B s
theorem completeBlock_response_law
Conditional on the entire assignment, complete-block responses have the stated density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= volume.withDensity (fun y => ofReal (completeBlockResponseDensity n d σ h z y))
Proof (Lean source)
-- @node: completeBlock_response_law
lemma completeBlock_response_law (n d : ℕ) (σ : Bool) (h : ℝ) (z : Assign (Fin n)) :
(blockBaselineLaw (n / (d + 1))).map
(fun U v => U v + completeBlockShift n d σ h z v) =
volume.withDensity (fun y => ofReal (completeBlockResponseDensity n d σ h z y)) := by
exact blockBaselineLaw_map_translation _ (completeBlockShift n d σ h z)
theorem mixtureLaw_eq_product_map
Mixing a jointly measurable recording channel is the pushforward of the independent prior-design product law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: mixtureLaw_eq_product_map
lemma mixtureLaw_eq_product_map {V Ξ : Type*} [Fintype V] [DecidableEq V]
[MeasurableSpace Ξ] (D : Measure (Assign V × Audit V)) [IsProbabilityMeasure D]
(π : Measure Ξ) (s : Ξ → Schedule V)
(hm : Measurable (fun x : Ξ × (Assign V × Audit V) => recordOf (s x.1) x.2)) :
mixtureLaw D π s = (π.prod D).map (fun x => recordOf (s x.1) x.2) := by
ext E hE
rw [mixtureLaw, Measure.bind_apply hE (measurable_record_components D s hm).aemeasurable,
Measure.map_apply hm hE, Measure.prod_apply (hm hE)]
apply lintegral_congr
intro ξ
have hr : Measurable (recordOf (s ξ)) :=
hm.comp (measurable_const.prodMk measurable_id)
rw [Measure.map_apply hr hE]
rfl
theorem completeBlock_densityJoint_eq_product_map
Retaining the entire design draw, the conditional product-density joint law is exactly independent baselines translated by the observed signs. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (D.prod (blockBaselineLaw (n / (d + 1)))).map (fun x => (x.1, fun v => x.2 v + completeBlockShift n d σ h x.1.1 v))
Proof (Lean source)
-- @node: completeBlock_densityJoint_eq_product_map
lemma completeBlock_densityJoint_eq_product_map (n d : ℕ) (σ : Bool) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) :
densityJoint D volume (fun ω y => completeBlockResponseDensity n d σ h ω.1 y) =
(D.prod (blockBaselineLaw (n / (d + 1)))).map
(fun x => (x.1, fun v => x.2 v + completeBlockShift n d σ h x.1.1 v)) := by
let := blockBaselineLaw_probability (n / (d + 1))
have hm : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) ×
(Fin (n / (d + 1)) → ℝ) =>
(x.1, fun v => x.2 v + completeBlockShift n d σ h x.1.1 v)) := by
apply measurable_fst.prodMk
apply measurable_pi_lambda
intro v
apply add
· fun_prop
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
completeBlockShift n d σ h ω.1 v)).comp measurable_fst
ext E hE
rw [densityJoint, Measure.bind_apply hE
(measurable_densityJoint_fiber volume _
(completeBlockResponseDensity_measurable n d σ h)).aemeasurable,
Measure.map_apply hm hE, Measure.prod_apply (hm hE)]
apply lintegral_congr
intro ω
rw [← completeBlock_response_law,
Measure.map_map measurable_prodMk_left (by fun_prop),
Measure.map_apply (show Measurable (Prod.mk ω ∘
(fun (U : Fin (n / (d + 1)) → ℝ) v =>
U v + completeBlockShift n d σ h ω.1 v)) from
measurable_prodMk_left.comp (by fun_prop)) hE]
rfl
theorem completeBlock_mixture_density_representation
The actual full-record prior mixture is the copying of the conditional-density joint law with its full assignment-audit marginal. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (densityJoint D volume (fun ω y => completeBlockResponseDensity n d σ h ω.1 y)).map (fun x => completeBlockCopy n d (((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) x.1).1, x.1.1), x.2))
Proof (Lean source)
-- @node: completeBlock_mixture_density_representation
lemma completeBlock_mixture_density_representation (n d : ℕ) (σ : Bool) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d σ h) =
(densityJoint D volume (fun ω y => completeBlockResponseDensity n d σ h ω.1 y)).map
(fun x => completeBlockCopy n d
(((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) x.1).1,
x.1.1), x.2)) := by
let := blockBaselineLaw_probability (n / (d + 1))
have hc : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) ×
(Fin (n / (d + 1)) → ℝ) => completeBlockCopy n d
(((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) x.1).1,
x.1.1), x.2)) := by
apply (completeBlockCopy_measurable n d).comp
apply prodMk
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) ω).1, ω.1))).comp
measurable_fst
· fun_prop
rw [mixtureLaw_eq_product_map D _ _ (completeBlock_record_measurable n d σ h),
completeBlock_densityJoint_eq_product_map, ← Measure.prod_swap]
rw [Measure.map_map (completeBlock_record_measurable n d σ h) measurable_swap,
Measure.map_map hc (by
apply measurable_fst.prodMk
apply measurable_pi_lambda
intro v
apply add
· fun_prop
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
completeBlockShift n d σ h ω.1 v)).comp measurable_fst)]
congr 1
funext x
dsimp only [Function.comp_apply, swap]
rw [completeBlock_record_reconstruction, completeBlock_record_graph]
theorem completeBlock_densityJoint_tv_le
The common full-design marginal makes the joint response distance at most the square root of the averaged translation energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
tvDist (densityJoint D volume (fun ω y => completeBlockResponseDensity n d true h ω.1 y)) (densityJoint D volume (fun ω y => completeBlockResponseDensity n d false h ω.1 y))
≤ sqrt (∫ ω, hellingerEnergy volume (completeBlockResponseDensity n d true h ω.1) (completeBlockResponseDensity n d false h ω.1) ∂D)
Proof (Lean source)
-- @node: completeBlock_densityJoint_tv_le
lemma completeBlock_densityJoint_tv_le (n d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
tvDist
(densityJoint D volume (fun ω y => completeBlockResponseDensity n d true h ω.1 y))
(densityJoint D volume (fun ω y => completeBlockResponseDensity n d false h ω.1 y)) ≤
sqrt (∫ ω, hellingerEnergy volume (completeBlockResponseDensity n d true h ω.1)
(completeBlockResponseDensity n d false h ω.1) ∂D) := by
let P := densityJoint D volume (fun ω y => completeBlockResponseDensity n d true h ω.1 y)
let M := densityJoint D volume (fun ω y => completeBlockResponseDensity n d false h ω.1 y)
let := densityJoint_probability D volume _
(completeBlockResponseDensity_measurable n d true h)
(fun ω => completeBlockResponseDensity_probability n d true h ω.1)
let := densityJoint_probability D volume _
(completeBlockResponseDensity_measurable n d false h)
(fun ω => completeBlockResponseDensity_probability n d false h ω.1)
have he := (common_marginal_hellinger_averaging D volume _ _
(completeBlockResponseDensity_measurable n d true h)
(completeBlockResponseDensity_measurable n d false h)
(fun ω => completeBlockResponseDensity_nonneg n d true h ω.1)
(fun ω => completeBlockResponseDensity_nonneg n d false h ω.1)
(fun ω => completeBlockResponseDensity_probability n d true h ω.1)
(fun ω => completeBlockResponseDensity_probability n d false h ω.1)).1
have he0 : 0 ≤ hellingerSqMeasure P M := by
rw [he]
exact integral_nonneg (fun _ => integral_nonneg (fun _ => sq_nonneg _))
calc
_ ≤ sqrt (hellingerSqMeasure P M *
(1 - hellingerSqMeasure P M / 4)) :=
tvDist_le_sharp_hellinger P M
_ ≤ sqrt (hellingerSqMeasure P M) := by
apply Real.sqrt_le_sqrt
nlinarith [sq_nonneg (hellingerSqMeasure P M)]
_ = _ := by rw [he]
theorem completeBlock_mixture_tv_population_le
The genuine complete original-record mixtures satisfy the population translation bound; projection retains the labeled graph and every treatment coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
h :
ℝ
ha :
conclusion
tvDist (mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d true h)) (mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d false h))
Proof (Lean source)
-- @node: completeBlock_mixture_tv_population_le
lemma completeBlock_mixture_tv_population_le (n d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(ha : AssignmentLaw D) :
tvDist
(mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d true h))
(mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d false h)) ≤
sqrt (4 * pi ^ 2 * h ^ 2 * n / ((d : ℝ) + 1) ^ 2) := by
let := densityJoint_probability D volume _
(completeBlockResponseDensity_measurable n d true h)
(fun ω => completeBlockResponseDensity_probability n d true h ω.1)
let := densityJoint_probability D volume _
(completeBlockResponseDensity_measurable n d false h)
(fun ω => completeBlockResponseDensity_probability n d false h ω.1)
have hc : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) ×
(Fin (n / (d + 1)) → ℝ) => completeBlockCopy n d
(((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) x.1).1,
x.1.1), x.2)) := by
apply (completeBlockCopy_measurable n d).comp
apply prodMk
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) ω).1, ω.1))).comp
measurable_fst
· fun_prop
rw [completeBlock_mixture_density_representation, completeBlock_mixture_density_representation]
exact (block_tvDist_map_le _ _ _ hc).trans
((completeBlock_densityJoint_tv_le n d h D).trans
(Real.sqrt_le_sqrt (completeBlock_response_energy_population_le n d h D ha)))
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.CompleteBlockRecord 16 declarations The distinct responses determine every labeled outcome, including padding.
Complete-block copying and reconstruction
The distinct responses determine every labeled outcome, including padding. Reading one fixed member of each block recovers the responses without discarding the graph or any treatment coordinate.
theorem completeBlockSchedule_response_padding
Padding labels have identically zero outcome under the complete-block prior. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
potentialOutcome (completeBlockSchedule n d σ h U) i z = 0
Proof (Lean source)
-- @node: completeBlockSchedule_response_padding
lemma completeBlockSchedule_response_padding (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (i : Fin n) (z : Assign (Fin n))
(hi : ¬ i.val < (n / (d + 1)) * (d + 1)) :
potentialOutcome (completeBlockSchedule n d σ h U) i z = 0 := by
unfold potentialOutcome
rw [completeBlockSchedule_baseline_padding n d σ h U i hi,
completeBlockSchedule_inNbhd_padding n d σ h U i hi]
simp [completeBlockSchedule, hi]
A fixed labeled representative of each full block.
theorem completeBlockRepresentative_quotient
The chosen member belongs to its intended block. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
v :
Fin (n / (d + 1))
conclusion
(completeBlockRepresentative n d v).val / (d + 1) = v.val
Proof (Lean source)
-- @node: completeBlockRepresentative_quotient
lemma completeBlockRepresentative_quotient (n d : ℕ) (v : Fin (n / (d + 1))) :
(completeBlockRepresentative n d v).val / (d + 1) = v.val := by
simp [completeBlockRepresentative]
def completeBlockCopy
Copy each distinct block response to all its labeled members; padding stays zero.
def completeBlockExtract
Read the distinct responses while preserving all graph and treatment coordinates.
theorem completeBlockCopy_measurable
Copying is measurable on the full labeled record space.
Formal statement
Proof (Lean source)
-- @node: completeBlockCopy_measurable
@[fun_prop]
lemma completeBlockCopy_measurable (n d : ℕ) : Measurable (completeBlockCopy n d) := by
unfold completeBlockCopy
apply measurable_fst.fst.prodMk
apply measurable_fst.snd.prodMk
apply measurable_pi_lambda
intro i
apply Finset.measurable_sum
intro v _
by_cases hi : i.val / (d + 1) = v.val <;> simp only [hi, ite_true, ite_false] <;> fun_prop
theorem completeBlockExtract_measurable
Reading the fixed representatives is measurable.
Formal statement
Proof (Lean source)
-- @node: completeBlockExtract_measurable
@[fun_prop]
lemma completeBlockExtract_measurable (n d : ℕ) : Measurable (completeBlockExtract n d) := by
unfold completeBlockExtract
fun_prop
theorem completeBlockCopy_member
At a full block member the copying map returns exactly that block's response. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlockCopy_member
lemma completeBlockCopy_member (n d : ℕ)
(x : ((OffDiag (Fin n) → Bool) × Assign (Fin n)) × (Fin (n / (d + 1)) → ℝ))
(i : Fin n) (v : Fin (n / (d + 1))) (hi : i.val / (d + 1) = v.val) :
(completeBlockCopy n d x).2.2 i = x.2 v := by
change (∑ w, if i.val / (d + 1) = w.val then x.2 w else 0) = x.2 v
rw [Finset.sum_eq_single v]
· simp [hi]
· intro w _ hw
apply if_neg
intro he
exact hw (Fin.ext (he.symm.trans hi))
· simp
theorem completeBlockCopy_padding
At padding labels all terms in the copying sum vanish. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlockCopy_padding
lemma completeBlockCopy_padding (n d : ℕ)
(x : ((OffDiag (Fin n) → Bool) × Assign (Fin n)) × (Fin (n / (d + 1)) → ℝ))
(i : Fin n) (hi : ¬ i.val < (n / (d + 1)) * (d + 1)) :
(completeBlockCopy n d x).2.2 i = 0 := by
change (∑ v : Fin (n / (d + 1)), if i.val / (d + 1) = v.val then x.2 v else 0) = 0
apply Finset.sum_eq_zero
intro v _
apply if_neg
intro he
exact hi ((completeBlock_active_iff n d i).mpr (he ▸ v.isLt))
theorem completeBlockExtract_copy
Reading the copied outcomes is an exact left inverse, including graph labels and assignment. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
completeBlockExtract n d (completeBlockCopy n d x) = x
Proof (Lean source)
-- @node: completeBlockExtract_copy
lemma completeBlockExtract_copy (n d : ℕ)
(x : ((OffDiag (Fin n) → Bool) × Assign (Fin n)) × (Fin (n / (d + 1)) → ℝ)) :
completeBlockExtract n d (completeBlockCopy n d x) = x := by
apply Prod.ext
· rfl
· funext v
exact completeBlockCopy_member n d x _ v (completeBlockRepresentative_quotient n d v)
theorem completeBlock_record_reconstruction
The original record is exactly the copying of its translated block baselines. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
recordOf (completeBlockSchedule n d σ h U) ω
= completeBlockCopy n d (((recordOf (completeBlockSchedule n d σ h U) ω).1, ω.1), fun v => U v + completeBlockShift n d σ h ω.1 v)
Proof (Lean source)
-- @node: completeBlock_record_reconstruction
lemma completeBlock_record_reconstruction (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (ω : Assign (Fin n) × Audit (Fin n)) :
recordOf (completeBlockSchedule n d σ h U) ω =
completeBlockCopy n d
(((recordOf (completeBlockSchedule n d σ h U) ω).1, ω.1),
fun v => U v + completeBlockShift n d σ h ω.1 v) := by
apply Prod.ext
· rfl
· apply Prod.ext
· rfl
· funext i
by_cases hi : i.val < (n / (d + 1)) * (d + 1)
· let v : Fin (n / (d + 1)) :=
⟨i.val / (d + 1), (completeBlock_active_iff n d i).mp hi⟩
rw [completeBlockCopy_member n d _ i v rfl]
exact completeBlockSchedule_response n d σ h U i ω.1 hi
· rw [completeBlockCopy_padding n d _ i hi]
exact completeBlockSchedule_response_padding n d σ h U i ω.1 hi
theorem completeBlock_record_graph
The complete retained graph is independent of the sign, amplitude, and baseline draws. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlock_record_graph
lemma completeBlock_record_graph (n d : ℕ) (σ : Bool) (h : ℝ)
(U : Fin (n / (d + 1)) → ℝ) (ω : Assign (Fin n) × Audit (Fin n)) :
(recordOf (completeBlockSchedule n d σ h U) ω).1 =
(recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) ω).1 := by
rfl
theorem completeBlock_record_measurable
The full recording channel is jointly measurable in baselines and the actual design draw.
Formal statement
hypotheses
conclusion
Measurable (fun x : (Fin (n / (d + 1)) → ℝ) × (Assign (Fin n) × Audit (Fin n)) => recordOf (completeBlockSchedule n d σ h x.1) x.2)
Proof (Lean source)
-- @node: completeBlock_record_measurable
@[fun_prop]
lemma completeBlock_record_measurable (n d : ℕ) (σ : Bool) (h : ℝ) :
Measurable (fun x : (Fin (n / (d + 1)) → ℝ) ×
(Assign (Fin n) × Audit (Fin n)) => recordOf (completeBlockSchedule n d σ h x.1) x.2) := by
have he : (fun x : (Fin (n / (d + 1)) → ℝ) ×
(Assign (Fin n) × Audit (Fin n)) => recordOf (completeBlockSchedule n d σ h x.1) x.2) =
fun x => completeBlockCopy n d
(((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) x.2).1, x.2.1),
fun v => x.1 v + completeBlockShift n d σ h x.2.1 v) := by
funext x
rw [completeBlock_record_reconstruction, completeBlock_record_graph]
rw [he]
apply (completeBlockCopy_measurable n d).comp
apply prodMk
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
((recordOf (completeBlockSchedule n d true 0 (fun _ => 0)) ω).1, ω.1))).comp measurable_snd
· apply measurable_pi_lambda
intro v
apply add
· fun_prop
· exact (measurable_of_finite (fun z : Assign (Fin n) =>
completeBlockShift n d σ h z v)).comp measurable_snd.fst
theorem completeBlockCopy_map_extract
Extracting after copying recovers any law of distinct responses and ancillary coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: completeBlockCopy_map_extract
lemma completeBlockCopy_map_extract (n d : ℕ)
(μ : Measure (((OffDiag (Fin n) → Bool) × Assign (Fin n)) ×
(Fin (n / (d + 1)) → ℝ))) :
(μ.map (completeBlockCopy n d)).map (completeBlockExtract n d) = μ := by
rw [Measure.map_map (completeBlockExtract_measurable n d) (completeBlockCopy_measurable n d)]
have he : completeBlockExtract n d ∘ completeBlockCopy n d = id := by
funext x
exact completeBlockExtract_copy n d x
rw [he, Measure.map_id]
theorem completeBlockCopy_tvDist_eq
No testing information is lost when distinct responses are copied into the original record. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: completeBlockCopy_tvDist_eq
lemma completeBlockCopy_tvDist_eq (n d : ℕ)
(μ ν : Measure (((OffDiag (Fin n) → Bool) × Assign (Fin n)) ×
(Fin (n / (d + 1)) → ℝ))) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν] :
tvDist (μ.map (completeBlockCopy n d)) (ν.map (completeBlockCopy n d)) =
tvDist μ ν := by
let : IsProbabilityMeasure (μ.map (completeBlockCopy n d)) :=
Measure.isProbabilityMeasure_map (completeBlockCopy_measurable n d).aemeasurable
let : IsProbabilityMeasure (ν.map (completeBlockCopy n d)) :=
Measure.isProbabilityMeasure_map (completeBlockCopy_measurable n d).aemeasurable
apply le_antisymm
· exact block_tvDist_map_le μ ν _ (completeBlockCopy_measurable n d)
· have ht := block_tvDist_map_le (μ.map (completeBlockCopy n d))
(ν.map (completeBlockCopy n d)) _ (completeBlockExtract_measurable n d)
simpa only [completeBlockCopy_map_extract] using ht
theorem completeBlock_record_map_reconstruction
For any joint baseline-design law, the original-record law is exactly the copied reduced law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
μ.map (fun x => recordOf (completeBlockSchedule n d σ h x.1) x.2)
= (μ.map (fun x => completeBlockExtract n d (recordOf (completeBlockSchedule n d σ h x.1) x.2))).map (completeBlockCopy n d)
Proof (Lean source)
-- @node: completeBlock_record_map_reconstruction
lemma completeBlock_record_map_reconstruction (n d : ℕ) (σ : Bool) (h : ℝ)
(μ : Measure ((Fin (n / (d + 1)) → ℝ) × (Assign (Fin n) × Audit (Fin n)))) :
μ.map (fun x => recordOf (completeBlockSchedule n d σ h x.1) x.2) =
(μ.map (fun x => completeBlockExtract n d
(recordOf (completeBlockSchedule n d σ h x.1) x.2))).map (completeBlockCopy n d) := by
rw [Measure.map_map (completeBlockCopy_measurable n d)
(show Measurable (fun x : (Fin (n / (d + 1)) → ℝ) ×
(Assign (Fin n) × Audit (Fin n)) => completeBlockExtract n d
(recordOf (completeBlockSchedule n d σ h x.1) x.2)) from
(completeBlockExtract_measurable n d).comp (completeBlock_record_measurable n d σ h))]
congr 1
funext x
have hr := completeBlock_record_reconstruction n d σ h x.1 x.2
dsimp only [Function.comp_apply]
conv_rhs => rw [hr, completeBlockExtract_copy]
exact hr
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.CompletionCounting 7 declarations The multinomial word-fiber theorem counts capacity-constrained allocations of original hidden source labels.
Exact multiplicities of labeled hidden completions
The multinomial word-fiber theorem counts capacity-constrained allocations of original hidden source labels. The counts include all compatible partitions, rather than unlabeled or reduced-information observations.
Functions with specified row-fiber sizes on an arbitrary finite label type.
Definition (Lean source)
Reindexing original labels preserves every row-fiber cardinality. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: row_fiber_card_equiv
lemma row_fiber_card_equiv {ι κ : Type*} [Fintype ι] [Fintype κ]
{B : ℕ} (e : ι ≃ κ) (g : κ → Fin B) (ℓ : Fin B) :
(Finset.univ.filter (fun j : ι => g (e j) = ℓ)).card =
(Finset.univ.filter (fun j : κ => g j = ℓ)).card := by
classical
simpa only [Fintype.card_subtype] using
Fintype.card_congr (e.subtypeEquiv (p := fun j => g (e j) = ℓ)
(q := fun j => g j = ℓ) (fun _ => Iff.rfl))
def finiteRowWordEquiv
Capacity-constrained words are invariant under bijections of the original labels.
Definition (Lean source)
parameters
def
given by
clause 1
toFun g := ⟨fun j
=> g.1 (e.symm j), fun ℓ => (row_fiber_card_equiv e.symm g.1 ℓ).trans (g.2 ℓ)⟩
clause 2
invFun g := ⟨fun j => g.1 (e j), fun ℓ => (row_fiber_card_equiv e g.1 ℓ).trans (g.2 ℓ)⟩
clause 3
left_inv g := by apply Subtype.ext; funext j; simp
clause 4
right_inv g := by apply Subtype.ext; funext j; simp
theorem finiteRowWord_card_mul_factorial
The exact multinomial count on an arbitrary finite original-label type, with no division or rounding ambiguity: the count times the row factorials is the total factorial. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: finiteRowWord_card_mul_factorial
lemma finiteRowWord_card_mul_factorial {ι : Type*} [Fintype ι] [DecidableEq ι]
(B : ℕ) (u : Fin B → ℕ) (hu : ∑ ℓ, u ℓ = card ι) :
card (FiniteRowWord ι B u) * (∏ ℓ, (u ℓ).factorial) =
(card ι).factorial := by
let e := finiteRowWordEquiv (Fintype.equivFin ι) u
rw [Fintype.card_congr e]
have hc := word_fiber_card_mul_factorial
u hu
simpa [FiniteRowWord, Fintype.card_subtype,
wordFiber,
coordCount,
multiFactorial, funext_iff] using hc
theorem compatiblePartition_card_mul_factorial
The exact count of compatible partitions times the remaining row factorials is M!. This is the common labeled-completion denominator, including M = 0. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (undiscovered n B d H).factorial
Proof (Lean source)
-- @node: compatiblePartition_card_mul_factorial
lemma compatiblePartition_card_mul_factorial (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(H : OffDiag (Fin n) → Bool) (s : CompatiblePartition n B d H) :
card (CompatiblePartition n B d H) *
(∏ ℓ, (capacity n B d H ℓ).factorial) = (undiscovered n B d H).factorial := by
classical
rw [Fintype.card_congr (compatiblePartitionEquivHiddenCompletion n B d hfit H s)]
have hc := finiteRowWord_card_mul_factorial (ι := HiddenSource n B d H) B
(capacity n B d H) (hiddenSource_card n B d hfit H s).symm
simpa only [hiddenSource_card n B d hfit H s] using hc
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ConditionalDensityBridge 9 declarations Normalize the full retained-graph fiber without discarding assignment coordinates.
The actual positive-graph conditional density
Normalize the full retained-graph fiber without discarding assignment coordinates.
Reduction of the reconstructed original record preserves its exact density kernel. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
σ :
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
conclusion
= (graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (volume.withDensity (fun y => ofReal (blockDensity n B d σ h hz.1 hz.2 y))).map (fun y => (hz.1, hz.2, y)))
Proof (Lean source)
lemma reducedBlockLaw_density_bind (n B d : ℕ) (σ : Bool) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1) :
reducedBlockLaw n B d (thinnedDesign (Fin n) q) σ h =
(graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(volume.withDensity (fun y => ofReal
(blockDensity n B d σ h hz.1 hz.2 y))).map
(fun y => (hz.1, hz.2, y))) := by
have hlaw := (block_law n B d σ h q (thinnedDesign (Fin n) q)
hn hB hd hfit hh hq (thinnedDesign_assignment (V := Fin n) q hq)
(thinnedDesign_audit (V := Fin n) q hq)
(thinnedDesign_independent (V := Fin n) q hq)).2.2.1
rw [reducedBlockLaw, hlaw, reconstructedLaw,
block_map_bind _ _ _ (measurable_of_finite _) (by fun_prop)]
apply Measure.bind_congr_right
filter_upwards [] with hz
rw [Measure.map_map (by fun_prop)
(show Measurable (copyOutcomes n B d hz) from
(copyOutcomes_measurable n B d).comp
(measurable_const.prodMk (measurable_const.prodMk measurable_id)))]
congr 1
funext y
dsimp only [Function.comp_apply]
rw [distinctResponses_copyOutcomes n B d hd hfit]
rfl
theorem graphAssignMarginal_atom_factor
Graph and full assignment atoms factor under the actual independent design. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: graphAssignMarginal_atom_factor
lemma graphAssignMarginal_atom_factor (n B d : ℕ) (q : ℝ)
(hq : q ∈ Icc 0 1) (H : OffDiag (Fin n) → Bool) (z : Assign (Fin n)) :
graphAssignMarginal n B d (thinnedDesign (Fin n) q) {(H,z)} =
retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} *
halfBernoulli (Fin n) {z} := by
let g := fun s : SourcePartition B d =>
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H}
have ha : graphAssignMarginal n B d (thinnedDesign (Fin n) q) {(H,z)} =
halfBernoulli (Fin n) {z} * ∫⁻ s, g s ∂partitionLaw B d := by
rw [graphAssignMarginal, Measure.bind_apply (measurableSet_singleton _) (by fun_prop)]
change (∫⁻ s, partitionGraphAssignLaw n B d (thinnedDesign (Fin n) q) s {(H,z)}
∂partitionLaw B d) = _
simp_rw [partitionGraphAssignLaw_atom n B d q hq]
exact lintegral_const_mul _ (measurable_of_finite _)
have hg : retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} =
∫⁻ s, g s ∂partitionLaw B d := by
rw [retainedGraphMarginal, Measure.map_apply (by fun_prop) (measurableSet_singleton _),
graphAssignMarginal, Measure.bind_apply (by measurability) (by fun_prop)]
apply lintegral_congr
intro s
rw [← Measure.map_apply (by fun_prop) (measurableSet_singleton _),
Measure.map_map (by fun_prop) (by fun_prop)]
have he := congrArg (fun μ => μ.map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e))
(thinnedDesign_audit (V := Fin n) q hq).2.2
rw [Measure.map_map (by fun_prop) (by fun_prop)] at he
exact congrArg (fun μ => μ {H}) he
rw [ha, hg, mul_comm]
theorem blockDensity_assignment_measurable
The actual finite response density is measurable jointly with the assignment.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: blockDensity_assignment_measurable
@[fun_prop] lemma blockDensity_assignment_measurable (n B d : ℕ) (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) :
Measurable (fun p : Assign (Fin n) × (Fin B → ℝ) =>
blockDensity n B d σ h H p.1 p.2) := by
apply measurable_from_prod_countable_right
intro z
simp_rw [blockDensity_allocation_sum]
apply Measurable.div_const
apply Finset.measurable_sum
intro k hk
split_ifs
· apply Finset.measurable_prod
intro ℓ hℓ
apply measurable_const.mul
unfold cosSqDensity
apply Measurable.ite
· exact measurableSet_le (by fun_prop) measurable_const
· fun_prop
· fun_prop
· fun_prop
Independent reference responses have the product reference density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
h :
ℝ
conclusion
Measure.pi (fun v => volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: reference_pi_density
lemma reference_pi_density (B d : ℕ) (h : ℝ) :
Measure.pi (fun v => volume.withDensity
(fun w => ofReal (refDensity d h w))) =
volume.withDensity (fun y : Fin B → ℝ =>
ofReal (∏ v, refDensity d h (y v))) := by
apply Measure.pi_eq
intro E hE
rw [withDensity_apply _ (.univ_pi hE), ← lintegral_indicator (.univ_pi hE)]
have he : (Set.univ.pi E).indicator
(fun y : Fin B → ℝ => ofReal (∏ v, refDensity d h (y v))) =
fun y => ofReal (∏ v, (E v).indicator
(fun w => refDensity d h w) (y v)) := by
funext y
by_cases hy : y ∈ Set.univ.pi E
· have hy' : ∀ v, y v ∈ E v := by simpa using hy
simp [hy']
· have hy' : ∃ v, y v ∉ E v := by simpa using hy
obtain ⟨v, hv⟩ := hy'
rw [Set.indicator_of_notMem hy]
have hz : (∏ v, (E v).indicator (fun w => refDensity d h w) (y v)) = 0 := by
apply Finset.prod_eq_zero (Finset.mem_univ v)
exact Set.indicator_of_notMem hv _
rw [hz, ENNReal.ofReal_zero]
rw [he]
change (∫⁻ y : Fin B → ℝ, ofReal (∏ v,
(E v).indicator (fun w => refDensity d h w) (y v))
∂Measure.pi (fun _ => volume)) = _
rw [← ofReal_integral_eq_lintegral_ofReal
(Integrable.fintype_prod (fun v =>
((refDensity_integrable_normalized d h).1.indicator (hE v))))
(Filter.Eventually.of_forall (fun y => Finset.prod_nonneg (fun v _ =>
Set.indicator_nonneg (fun _ _ => refDensity_nonneg d h _) _))),
integral_fintype_prod_eq_prod]
rw [ENNReal.ofReal_prod_of_nonneg (fun v _ => integral_nonneg
(fun w => Set.indicator_nonneg (fun _ _ => refDensity_nonneg d h _) w))]
apply Finset.prod_congr rfl
intro v _
rw [withDensity_apply _ (hE v), ← lintegral_indicator (hE v)]
have he' : (E v).indicator (fun w => ofReal (refDensity d h w)) =
fun w => ofReal ((E v).indicator (fun w => refDensity d h w) w) := by
funext w
by_cases hw : w ∈ E v <;> simp [indicator, hw]
rw [he']
exact ofReal_integral_eq_lintegral_ofReal
((refDensity_integrable_normalized d h).1.indicator (hE v))
(Filter.Eventually.of_forall (fun w =>
Set.indicator_nonneg (fun _ _ => refDensity_nonneg d h _) w))
theorem conditionalBlockLaw_eq_densityJoint Lemma conditionalBlockLaw_eq_densityJoint in the paper ↗
The normalized positive-graph fiber has the displayed response density and keeps the entire actual assignment vector. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
lemma conditionalBlockLaw_eq_densityJoint (n B d : ℕ) (σ : Bool) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(H : OffDiag (Fin n) → Bool)
(hH : retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} ≠ 0) :
conditionalBlockLaw n B d (thinnedDesign (Fin n) q) σ h H =
densityJoint (halfBernoulli (Fin n)) volume (blockDensity n B d σ h H) := by
let ρ := retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H}
have hρ : ρ ≠ ∞ := by
let := design_isProbabilityMeasure (thinnedDesign (Fin n) q) q
(thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)
let := retainedGraphMarginal_probability n B d (thinnedDesign (Fin n) q)
exact measure_ne_top _ _
ext E hE
rw [conditionalBlockLaw, Measure.smul_apply, smul_eq_mul,
Measure.map_apply measurable_snd hE, Measure.restrict_apply (measurable_snd hE),
reducedBlockLaw_density_bind n B d σ h q hn hB hd hfit hh hq,
Measure.bind_apply (by measurability) (measurable_of_finite _).aemeasurable,
lintegral_fintype]
have hf (hz : (OffDiag (Fin n) → Bool) × Assign (Fin n)) :
((volume.withDensity (fun y => ofReal
(blockDensity n B d σ h hz.1 hz.2 y))).map (fun y => (hz.1, hz.2, y)))
(snd ⁻¹' E ∩ {x | x.1 = H}) =
if hz.1 = H then
(volume.withDensity (fun y => ofReal
(blockDensity n B d σ h hz.1 hz.2 y))) (Prod.mk hz.2 ⁻¹' E) else 0 := by
rw [Measure.map_apply (by fun_prop) (by measurability)]
by_cases he : hz.1 = H
· rw [if_pos he]
congr 1
ext y
simp [he]
· rw [if_neg he]
have hempty : (fun y => (hz.1, hz.2, y)) ⁻¹'
(snd ⁻¹' E ∩ {x | x.1 = H}) = ∅ := by
ext y
simp [he]
rw [hempty, measure_empty]
simp_rw [hf, graphAssignMarginal_atom_factor n B d q hq]
rw [Fintype.sum_prod_type]
have hsum : (∑ G : OffDiag (Fin n) → Bool, ∑ z : Assign (Fin n),
(if G = H then
(volume.withDensity (fun y => ofReal (blockDensity n B d σ h G z y)))
(Prod.mk z ⁻¹' E) else 0) *
(retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {G} *
halfBernoulli (Fin n) {z})) =
ρ * ∑ z : Assign (Fin n),
(volume.withDensity (fun y => ofReal (blockDensity n B d σ h H z y)))
(Prod.mk z ⁻¹' E) * halfBernoulli (Fin n) {z} := by
simp only [ite_mul, zero_mul]
simp_rw [Finset.sum_ite_irrel]
simp only [Finset.sum_const_zero]
rw [Finset.sum_ite_eq', if_pos (Finset.mem_univ H), Finset.mul_sum]
apply Finset.sum_congr rfl
intro z _
dsimp [ρ]
ring
rw [hsum, ← mul_assoc, ENNReal.inv_mul_cancel hH hρ, one_mul,
densityJoint, Measure.bind_apply hE
(measurable_of_finite _).aemeasurable, lintegral_fintype]
apply Finset.sum_congr rfl
intro z _
rw [Measure.map_apply measurable_prodMk_left hE]
theorem conditionalBlockLaw_eq_reference_withDensity Lemma conditionalBlockLaw_eq_reference_withDensity in the paper ↗
The displayed posterior density is a density relative to the actual assignment and independent reference responses, including reference-zero sets. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
hH :
s :
CompatiblePartition n B d H
conclusion
= (hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h).withDensity (fun p => ofReal (blockDensity n B d true h H p.1 p.2 / (∏ ℓ, refDensity d h (p.2 ℓ))))
Proof (Lean source)
lemma conditionalBlockLaw_eq_reference_withDensity (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(H : OffDiag (Fin n) → Bool)
(hH : retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} ≠ 0)
(s : CompatiblePartition n B d H) :
conditionalBlockLaw n B d (thinnedDesign (Fin n) q) true h H =
(hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h).withDensity
(fun p => ofReal
(blockDensity n B d true h H p.1 p.2 / (∏ ℓ, refDensity d h (p.2 ℓ)))) := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (halfBernoulli (Fin n)) := by
unfold halfBernoulli; infer_instance
let := bernoulliLaw_probability q hq
let : IsProbabilityMeasure (auditLaw (Fin n) q) := by unfold auditLaw; infer_instance
rw [conditionalBlockLaw_eq_densityJoint n B d true h q hn hB hd hfit hh hq H hH,
densityJoint_eq_withDensity _ _ _ (blockDensity_assignment_measurable n B d true h H),
hiddenReferenceLaw, show (thinnedDesign (Fin n) q).map fst =
halfBernoulli (Fin n) from thinnedDesign_assignment q hq,
reference_pi_density, prod_withDensity_right (by fun_prop),
← withDensity_mul _ (by fun_prop) (by fun_prop)]
congr 1
funext p
dsimp only [Pi.mul_apply]
have hprod : 0 ≤ ∏ ℓ, refDensity d h (p.2 ℓ) :=
Finset.prod_nonneg (fun ℓ _ => refDensity_nonneg d h _)
by_cases hz : (∏ ℓ, refDensity d h (p.2 ℓ)) = 0
· obtain ⟨ℓ, _, hℓ⟩ := Finset.prod_eq_zero_iff.mp hz
rw [blockDensity_zero_of_reference_zero n B d hd hfit h H p.1 s p.2 ℓ hℓ,
hz]
simp
· rw [← ENNReal.ofReal_mul hprod, mul_div_cancel₀ _ hz]
theorem conditionalBlockLaw_rnDeriv_density_ratio Lemma conditionalBlockLaw_rnDeriv_density_ratio in the paper ↗
The actual conditional Radon--Nikodym likelihood is the complete-label posterior density ratio almost everywhere under the reference. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
hH :
s :
CompatiblePartition n B d H
conclusion
(fun p => ((conditionalBlockLaw n B d (thinnedDesign (Fin n) q) true h H).rnDeriv (hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h) p).toReal)
=ᵐ[ hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h] fun p => blockDensity n B d true h H p.1 p.2 / (∏ ℓ, refDensity d h (p.2 ℓ))
Proof (Lean source)
lemma conditionalBlockLaw_rnDeriv_density_ratio (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(H : OffDiag (Fin n) → Bool)
(hH : retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} ≠ 0)
(s : CompatiblePartition n B d H) :
(fun p => ((conditionalBlockLaw n B d (thinnedDesign (Fin n) q) true h H).rnDeriv
(hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h) p).toReal) =ᵐ[
hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h]
fun p => blockDensity n B d true h H p.1 p.2 / (∏ ℓ, refDensity d h (p.2 ℓ)) := by
let := design_isProbabilityMeasure (thinnedDesign (Fin n) q) q
(thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)
let := hiddenReferenceLaw_probability n B d h (thinnedDesign (Fin n) q)
rw [conditionalBlockLaw_eq_reference_withDensity n B d h q hn hB hd hfit hh hq H hH s]
have he := Measure.rnDeriv_withDensity
(hiddenReferenceLaw n B d (thinnedDesign (Fin n) q) h)
(show Measurable (fun p : Assign (Fin n) × (Fin B → ℝ) => ofReal
(blockDensity n B d true h H p.1 p.2 / (∏ ℓ, refDensity d h (p.2 ℓ)))) by fun_prop)
filter_upwards [he] with p hp
rw [hp, ENNReal.toReal_ofReal (div_nonneg
(blockDensity_nonneg n B d true h H p.1 p.2)
(Finset.prod_nonneg (fun ℓ _ => refDensity_nonneg d h _)))]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ConditionalGraphMixture 31 declarations The normalized graph fibers have unit mass on every positive-probability graph.
Finite retained-graph conditioning and conditional testing bounds
The normalized graph fibers have unit mass on every positive-probability graph. Common finite mixtures average conditional total-variation bounds, and a common reference converts two conditional chi-squared budgets into a testing bound.
theorem reducedBlockLaw_graph_marginal
The distinct-response record has exactly the actual complete retained-graph marginal. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reducedBlockLaw_graph_marginal
lemma reducedBlockLaw_graph_marginal (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [SFinite D] (σ : Bool) (h : ℝ) :
(reducedBlockLaw n B d D σ h).map fst = retainedGraphMarginal n B d D := by
rw [reducedBlockLaw, Measure.map_map (by fun_prop) (by fun_prop)]
have he := block_mixture_graphAssignMarginal n B d σ h D
rw [retainedGraphMarginal, ← he, Measure.map_map (by fun_prop) (by fun_prop)]
rfl
theorem reducedBlockLaw_graph_fiber_mass
The unnormalized graph fiber has mass equal to the actual graph-atom probability. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reducedBlockLaw_graph_fiber_mass
lemma reducedBlockLaw_graph_fiber_mass (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [SFinite D] (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) :
((reducedBlockLaw n B d D σ h).restrict {x | x.1 = H}).map snd univ =
retainedGraphMarginal n B d D {H} := by
rw [Measure.map_apply (by fun_prop) MeasurableSet.univ, Set.preimage_univ,
Measure.restrict_apply_univ, ← reducedBlockLaw_graph_marginal n B d D σ h,
Measure.map_apply (by fun_prop) (measurableSet_singleton H)]
rfl
theorem conditionalBlockLaw_probability
On every graph atom of positive probability, the actual conditional law is a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (conditionalBlockLaw n B d D σ h H)
Proof (Lean source)
-- @node: conditionalBlockLaw_probability
lemma conditionalBlockLaw_probability (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(σ : Bool) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(hH : retainedGraphMarginal n B d D {H} ≠ 0) :
IsProbabilityMeasure (conditionalBlockLaw n B d D σ h H) := by
let := blockMixtureLawOf_probability n B d D σ h
have hm : Measurable (fun o : Record (Fin n) =>
(o.1, o.2.1, distinctResponses n B d o)) := by fun_prop
let : IsProbabilityMeasure (reducedBlockLaw n B d D σ h) :=
Measure.isProbabilityMeasure_map hm.aemeasurable
let : IsProbabilityMeasure (retainedGraphMarginal n B d D) := by
rw [← reducedBlockLaw_graph_marginal n B d D σ h]
exact Measure.isProbabilityMeasure_map (by fun_prop)
constructor
rw [conditionalBlockLaw, Measure.smul_apply, smul_eq_mul,
reducedBlockLaw_graph_fiber_mass, ENNReal.inv_mul_cancel hH (measure_ne_top _ _)]
theorem conditionalBlockLaw_null
A null retained graph has zero conditional law under the chosen null-atom convention. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: conditionalBlockLaw_null
lemma conditionalBlockLaw_null (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [SFinite D]
(σ : Bool) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(hH : retainedGraphMarginal n B d D {H} = 0) :
conditionalBlockLaw n B d D σ h H = 0 := by
have hz : ((reducedBlockLaw n B d D σ h).restrict {x | x.1 = H}).map snd = 0 := by
apply Measure.measure_univ_eq_zero.mp
rw [reducedBlockLaw_graph_fiber_mass, hH]
rw [conditionalBlockLaw, hz, smul_zero]
theorem conditionalBlockLaw_weighted
Multiplying a conditional law by its graph probability recovers the unnormalized fiber. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
σ :
h :
ℝ
conclusion
retainedGraphMarginal n B d D {H} • conditionalBlockLaw n B d D σ h H
Proof (Lean source)
-- @node: conditionalBlockLaw_weighted
lemma conditionalBlockLaw_weighted (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(σ : Bool) (h : ℝ) (H : OffDiag (Fin n) → Bool) :
retainedGraphMarginal n B d D {H} • conditionalBlockLaw n B d D σ h H =
((reducedBlockLaw n B d D σ h).restrict {x | x.1 = H}).map snd := by
by_cases hH : retainedGraphMarginal n B d D {H} = 0
· rw [hH, zero_smul]
apply symm
apply Measure.measure_univ_eq_zero.mp
rw [reducedBlockLaw_graph_fiber_mass, hH]
· let := blockMixtureLawOf_probability n B d D σ h
let : IsProbabilityMeasure (reducedBlockLaw n B d D σ h) :=
Measure.isProbabilityMeasure_map (by fun_prop)
let : IsProbabilityMeasure (retainedGraphMarginal n B d D) := by
rw [← reducedBlockLaw_graph_marginal n B d D σ h]
exact Measure.isProbabilityMeasure_map (by fun_prop)
rw [conditionalBlockLaw, smul_smul,
ENNReal.mul_inv_cancel hH (measure_ne_top _ _), one_smul]
Restoring the fixed graph coordinate after projecting a graph fiber is exact. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: block_graph_fiber_map_copy
lemma block_graph_fiber_map_copy {α β : Type*} [MeasurableSpace α]
[MeasurableSingletonClass α] [MeasurableSpace β] (μ : Measure (α × β)) (a : α) :
((μ.restrict {x | x.1 = a}).map snd).map (Prod.mk a) =
μ.restrict {x | x.1 = a} := by
rw [Measure.map_map measurable_prodMk_left measurable_snd]
have hs : MeasurableSet {x : α × β | x.1 = a} := by
exact measurable_fst (measurableSet_singleton a)
have he : (Prod.mk a ∘ snd) =ᵐ[μ.restrict {x | x.1 = a}] id := by
filter_upwards [ae_restrict_mem hs] with x hx
exact Prod.ext hx.symm rfl
rw [Measure.map_congr he, Measure.map_id]
theorem reducedBlockLaw_bind_conditional
The actual distinct-response record is the mixture of its normalized complete-graph fibers. Null graph atoms contribute zero, so no conditional law on a null event is assumed. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
reducedBlockLaw n B d D σ h
= (retainedGraphMarginal n B d D).bind (fun H => (conditionalBlockLaw n B d D σ h H).map (Prod.mk H))
Proof (Lean source)
-- @node: reducedBlockLaw_bind_conditional
lemma reducedBlockLaw_bind_conditional (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(σ : Bool) (h : ℝ) :
reducedBlockLaw n B d D σ h =
(retainedGraphMarginal n B d D).bind (fun H =>
(conditionalBlockLaw n B d D σ h H).map (Prod.mk H)) := by
let μ := reducedBlockLaw n B d D σ h
let K := fun H => (conditionalBlockLaw n B d D σ h H).map (Prod.mk H)
have hK : Measurable K := measurable_of_finite _
have hcopy (H : OffDiag (Fin n) → Bool) :
retainedGraphMarginal n B d D {H} • K H = μ.restrict {x | x.1 = H} := by
dsimp only [K]
rw [← Measure.map_smul, conditionalBlockLaw_weighted,
block_graph_fiber_map_copy]
have hd : Pairwise (Disjoint on (fun H : OffDiag (Fin n) → Bool =>
{x : (OffDiag (Fin n) → Bool) × (Assign (Fin n) × (Fin B → ℝ)) | x.1 = H})) := by
intro H G hHG
apply Set.disjoint_left.mpr
intro x hx hy
exact hHG (hx.symm.trans hy)
have hs (H : OffDiag (Fin n) → Bool) :
MeasurableSet {x : (OffDiag (Fin n) → Bool) ×
(Assign (Fin n) × (Fin B → ℝ)) | x.1 = H} :=
measurable_fst (measurableSet_singleton H)
have hu : (⋃ H : OffDiag (Fin n) → Bool,
{x : (OffDiag (Fin n) → Bool) × (Assign (Fin n) × (Fin B → ℝ)) | x.1 = H}) =
univ := by
ext x
simp
have hsum := Measure.restrict_iUnion (μ := μ) hd hs
rw [hu, Measure.restrict_univ] at hsum
ext E hE
rw [Measure.bind_apply hE hK.aemeasurable, lintegral_fintype]
change μ E = _
rw [hsum, Measure.sum_apply _ hE, tsum_fintype]
apply Finset.sum_congr rfl
intro H _
have he := congrArg (fun ν => ν E) (hcopy H)
simpa only [Measure.smul_apply, smul_eq_mul, mul_comm] using he.symm
theorem block_tvDist_bind_integral_le
A common finite mixing marginal averages conditional total-variation distances. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
α β :
μ :
P Q :
α → Measure β
hP :
∀ᵐ a ∂μ, IsProbabilityMeasure (P a)
hQ :
∀ᵐ a ∂μ, IsProbabilityMeasure (Q a)
hPm :
hQm :
Proof (Lean source)
-- @node: block_tvDist_bind_integral_le
lemma block_tvDist_bind_integral_le {α β : Type*} [Finite α] [MeasurableSpace α]
[MeasurableSingletonClass α] [MeasurableSpace β]
(μ : Measure α) [IsProbabilityMeasure μ] (P Q : α → Measure β)
(hP : ∀ᵐ a ∂μ, IsProbabilityMeasure (P a))
(hQ : ∀ᵐ a ∂μ, IsProbabilityMeasure (Q a))
(hPm : Measurable P) (hQm : Measurable Q) :
tvDist (μ.bind P) (μ.bind Q) ≤
∫ a, tvDist (P a) (Q a) ∂μ := by
unfold tvDist at ⊢
apply ciSup_le
rintro ⟨E, hE⟩
have hreal (K : α → Measure β) (hK : ∀ᵐ a ∂μ, IsProbabilityMeasure (K a))
(hKm : Measurable K) : (μ.bind K).real E = ∫ a, (K a).real E ∂μ := by
rw [measureReal_def, Measure.bind_apply hE hKm.aemeasurable]
exact (integral_toReal ((Measure.measurable_coe hE).comp hKm).aemeasurable
(hK.mono (fun a ha => by
let _ := ha
exact measure_lt_top (K a) E))).symm
rw [hreal P hP hPm, hreal Q hQ hQm,
← integral_sub Integrable.of_finite Integrable.of_finite]
exact abs_integral_le_integral_abs.trans
(integral_mono_ae Integrable.of_finite Integrable.of_finite (by
filter_upwards [hP, hQ] with a ha hb
let _ := ha
let _ := hb
exact abs_measureReal_sub_le_tvDist hE))
theorem blockMixture_tvDist_le_mean_conditional
The full original-record distance is bounded by the mean conditional distance over its actual complete retained-graph marginal, with all assignment coordinates retained. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
≤ ∫ H, tvDist (conditionalBlockLaw n B d D true h H) (conditionalBlockLaw n B d D false h H) ∂(retainedGraphMarginal n B d D)
Proof (Lean source)
-- @node: blockMixture_tvDist_le_mean_conditional
lemma blockMixture_tvDist_le_mean_conditional (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] (h : ℝ) :
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤
∫ H, tvDist (conditionalBlockLaw n B d D true h H)
(conditionalBlockLaw n B d D false h H) ∂(retainedGraphMarginal n B d D) := by
let μ := retainedGraphMarginal n B d D
let := blockMixtureLawOf_probability n B d D true h
let : IsProbabilityMeasure (reducedBlockLaw n B d D true h) :=
Measure.isProbabilityMeasure_map (by fun_prop)
let : IsProbabilityMeasure μ := by
dsimp only [μ]
rw [← reducedBlockLaw_graph_marginal n B d D true h]
exact Measure.isProbabilityMeasure_map (by fun_prop)
have hpos : ∀ᵐ H ∂μ, μ {H} ≠ 0 := ae_iff_of_countable.mpr (fun _ => id)
have hprob (σ : Bool) : ∀ᵐ H ∂μ,
IsProbabilityMeasure ((conditionalBlockLaw n B d D σ h H).map (Prod.mk H)) := by
filter_upwards [hpos] with H hH
let := conditionalBlockLaw_probability n B d D σ h H hH
exact Measure.isProbabilityMeasure_map (by fun_prop)
rw [← reducedBlockLaw_tvDist_eq n B d hd hfit D h,
reducedBlockLaw_bind_conditional, reducedBlockLaw_bind_conditional]
apply (block_tvDist_bind_integral_le μ _ _ (hprob true) (hprob false)
(measurable_of_finite _) (measurable_of_finite _)).trans
apply integral_mono_ae Integrable.of_finite Integrable.of_finite
filter_upwards [hpos] with H hH
let := conditionalBlockLaw_probability n B d D true h H hH
let := conditionalBlockLaw_probability n B d D false h H hH
exact block_tvDist_map_le _ _ (Prod.mk H) (by fun_prop)
theorem block_tvDist_le_sqrt_common_chiSq
Two conditional laws with the same chi-squared budget against a common reference have distance at most the square root of that budget. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
α :
Type*
hP :
P ≪ R
hQ :
Q ≪ R
hiP :
Integrable (fun x => ((P.rnDeriv R x).toReal - 1) ^ 2) R
hiQ :
Integrable (fun x => ((Q.rnDeriv R x).toReal - 1) ^ 2) R
Proof (Lean source)
-- @node: block_tvDist_le_sqrt_common_chiSq
lemma block_tvDist_le_sqrt_common_chiSq {α : Type*} [MeasurableSpace α]
(P Q R : Measure α) [IsProbabilityMeasure P] [IsProbabilityMeasure Q]
[IsProbabilityMeasure R] (hP : P ≪ R) (hQ : Q ≪ R)
(hiP : Integrable (fun x => ((P.rnDeriv R x).toReal - 1) ^ 2) R)
(hiQ : Integrable (fun x => ((Q.rnDeriv R x).toReal - 1) ^ 2) R)
(heq : chiSqDiv Q R = chiSqDiv P R) :
tvDist P Q ≤ sqrt (chiSqDiv P R) := by
have hp := tvDist_le_half_sqrt_chiSqDiv P R hP hiP
have hq := tvDist_le_half_sqrt_chiSqDiv Q R hQ hiQ
rw [heq] at hq
unfold tvDist
apply ciSup_le
rintro ⟨E, hE⟩
calc
|P.real E - Q.real E| ≤ |P.real E - R.real E| + |Q.real E - R.real E| := by
have ht := abs_add_le (P.real E - R.real E) (R.real E - Q.real E)
rw [abs_sub_comm (R.real E) (Q.real E)] at ht
simpa only [sub_add_sub_cancel] using ht
_ ≤ (1 / 2) * sqrt (chiSqDiv P R) +
(1 / 2) * sqrt (chiSqDiv P R) :=
add_le_add ((abs_measureReal_sub_le_tvDist hE).trans hp)
((abs_measureReal_sub_le_tvDist hE).trans hq)
_ = _ := by ring
theorem retainedGraphMarginal_probability
The actual complete retained-graph marginal is a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (retainedGraphMarginal n B d D)
Proof (Lean source)
-- @node: retainedGraphMarginal_probability
lemma retainedGraphMarginal_probability (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
IsProbabilityMeasure (retainedGraphMarginal n B d D) := by
let := blockMixtureLawOf_probability n B d D true 0
let : IsProbabilityMeasure (reducedBlockLaw n B d D true 0) :=
Measure.isProbabilityMeasure_map (by fun_prop)
rw [← reducedBlockLaw_graph_marginal n B d D true 0]
exact Measure.isProbabilityMeasure_map (by fun_prop)
On a finite probability space, averaging square roots costs at most the square root of the mean. This is Cauchy--Schwarz with the constant function one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
α :
μ :
g :
α → ℝ
hg :
∀ a, 0 ≤ g a
Proof (Lean source)
-- @node: block_integral_sqrt_le
lemma block_integral_sqrt_le {α : Type*} [Finite α] [MeasurableSpace α]
[MeasurableSingletonClass α] (μ : Measure α) [IsProbabilityMeasure μ]
(g : α → ℝ) (hg : ∀ a, 0 ≤ g a) :
(∫ a, sqrt (g a) ∂μ) ≤ sqrt (∫ a, g a ∂μ) := by
let c := ∫ a, sqrt (g a) ∂μ
have hvar : 0 ≤ ∫ a, (sqrt (g a) - c) ^ 2 ∂μ :=
integral_nonneg (fun _ => sq_nonneg _)
have he : (fun a => (sqrt (g a) - c) ^ 2) =
fun a => g a - 2 * c * sqrt (g a) + c ^ 2 := by
funext a
nlinarith [Real.sq_sqrt (hg a)]
rw [he, integral_add Integrable.of_finite Integrable.of_finite,
integral_sub Integrable.of_finite Integrable.of_finite, integral_const_mul] at hvar
have hsq : c ^ 2 ≤ ∫ a, g a ∂μ := by
simp [integral_const] at hvar
change 0 ≤ (∫ a, g a ∂μ) - 2 * c * c + c ^ 2 at hvar
nlinarith
have hroot := Real.sq_sqrt (integral_nonneg hg : 0 ≤ ∫ a, g a ∂μ)
have hc : 0 ≤ c := integral_nonneg (fun _ => Real.sqrt_nonneg _)
have hr : 0 ≤ sqrt (∫ a, g a ∂μ) := Real.sqrt_nonneg _
change c ≤ _
nlinarith
A finite probability mixture splits into a bad-event mass and the square root of its good-event second-moment budget. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_integral_good_event_le
lemma block_integral_good_event_le {α : Type*} [Finite α] [MeasurableSpace α]
[MeasurableSingletonClass α] (μ : Measure α) [IsProbabilityMeasure μ]
(E : Set α) (t g : α → ℝ) (hg : ∀ a, 0 ≤ g a)
(ht : ∀ᵐ a ∂μ, t a ≤ 1)
(hgood : ∀ᵐ a ∂μ, a ∈ E → t a ≤ sqrt (g a)) :
(∫ a, t a ∂μ) ≤ μ.real Eᶜ + sqrt (∫ a, E.indicator g a ∂μ) := by
classical
have hs : MeasurableSet E := Set.to_countable E |>.measurableSet
have hpoint : ∀ᵐ a ∂μ, t a ≤ Eᶜ.indicator (fun _ => (1 : ℝ)) a +
sqrt (E.indicator g a) := by
filter_upwards [ht, hgood] with a ha hb
by_cases he : a ∈ E
· simpa [he] using hb he
· simpa [he] using ha
have havg := integral_mono_ae (μ := μ) Integrable.of_finite
Integrable.of_finite hpoint
rw [integral_add Integrable.of_finite Integrable.of_finite,
integral_indicator hs.compl] at havg
have hb := block_integral_sqrt_le μ (E.indicator g)
(fun a => Set.indicator_nonneg (fun a _ => hg a) a)
have hmass : (∫ a in Eᶜ, (1 : ℝ) ∂μ) = μ.real Eᶜ := by simp
rw [hmass] at havg
exact havg.trans (add_le_add_right hb _)
theorem blockMixture_tvDist_good_event_le
Conditional square-root bounds give the roadmap's good-event bound for the complete original record, retaining the full graph and assignment marginal. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
hcond :
∀ᵐ H ∂(retainedGraphMarginal n B d D), H ∈ E → tvDist (conditionalBlockLaw n B d D true h H) (conditionalBlockLaw n B d D false h H)
≤ sqrt (hiddenChiSq n B d D h H)
conclusion
≤ (retainedGraphMarginal n B d D).real Eᶜ
Proof (Lean source)
-- @node: blockMixture_tvDist_good_event_le
lemma blockMixture_tvDist_good_event_le (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(h : ℝ) (E : Set (OffDiag (Fin n) → Bool))
(hcond : ∀ᵐ H ∂(retainedGraphMarginal n B d D), H ∈ E →
tvDist (conditionalBlockLaw n B d D true h H)
(conditionalBlockLaw n B d D false h H) ≤ sqrt (hiddenChiSq n B d D h H)) :
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤
(retainedGraphMarginal n B d D).real Eᶜ +
sqrt (∫ H, E.indicator (hiddenChiSq n B d D h) H
∂(retainedGraphMarginal n B d D)) := by
let := retainedGraphMarginal_probability n B d D
apply (blockMixture_tvDist_le_mean_conditional n B d hd hfit D h).trans
apply block_integral_good_event_le _ E _ _ (hiddenChiSq_nonneg n B d D h) _ hcond
have hpos : ∀ᵐ H ∂(retainedGraphMarginal n B d D),
retainedGraphMarginal n B d D {H} ≠ 0 := ae_iff_of_countable.mpr (fun _ => id)
filter_upwards [hpos] with H hH
let := conditionalBlockLaw_probability n B d D true h H hH
let := conditionalBlockLaw_probability n B d D false h H hH
exact tvDist_le_one
The cosine baseline density is even. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: cosSqDensity_neg
lemma cosSqDensity_neg (w : ℝ) : cosSqDensity (-w) = cosSqDensity w := by
simp [cosSqDensity, abs_neg, mul_neg, Real.cos_neg]
Reflecting a baseline draw preserves its probability law. the stated conclusion holds.
Formal statement
conclusion
= volume.withDensity (fun w => ofReal (cosSqDensity w))
Proof (Lean source)
-- @node: block_baseline_map_neg
lemma block_baseline_map_neg :
(volume.withDensity (fun w => ofReal (cosSqDensity w))).map
(fun w : ℝ => -w) = volume.withDensity (fun w => ofReal (cosSqDensity w)) := by
have ht := map_withDensity_comp_measurableEquiv
(MeasurableEquiv.neg ℝ) (Measure.measurePreserving_neg volume)
(fun w => ofReal (cosSqDensity w))
simpa [Function.comp_def, cosSqDensity_neg] using ht
Reflecting independent block baselines preserves their joint law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: blockBaselineLaw_map_neg
lemma blockBaselineLaw_map_neg (B : ℕ) :
(blockBaselineLaw B).map (fun U ℓ => -U ℓ) = blockBaselineLaw B := by
let : IsProbabilityMeasure (volume.withDensity (fun w => ofReal (cosSqDensity w))) := by
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal cosSqDensity_integrable_normalized.1
(Filter.Eventually.of_forall cosSqDensity_nonneg), cosSqDensity_integrable_normalized.2]
simp
let : IsProbabilityMeasure ((volume.withDensity
(fun w => ofReal (cosSqDensity w))).map (fun w : ℝ => -w)) :=
Measure.isProbabilityMeasure_map (by fun_prop)
rw [blockBaselineLaw, Measure.pi_map_pi (f := fun _ w => -w) (fun _ => by fun_prop)]
simp_rw [block_baseline_map_neg]
The prior is invariant when all baselines are reflected, with the labeled partition fixed. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
conclusion
Proof (Lean source)
-- @node: blockParamLaw_map_neg
lemma blockParamLaw_map_neg (B d : ℕ) :
(blockParamLaw B d).map (fun ξ => (ξ.1, fun ℓ => -ξ.2 ℓ)) = blockParamLaw B d := by
let := partitionLaw_probability B d
let := blockBaselineLaw_probability B
change (blockParamLaw B d).map (map id (fun U ℓ => -U ℓ)) = _
rw [blockParamLaw, ← Measure.map_prod_map _ _ measurable_id (by fun_prop),
Measure.map_id, blockBaselineLaw_map_neg]
theorem block_record_sign_reflection
Reversing the sign and the baselines negates every response, while preserving the complete retained graph and all observed treatment coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: block_record_sign_reflection
lemma block_record_sign_reflection (n B d : ℕ) (h : ℝ)
(ξ : SourcePartition B d × (Fin B → ℝ)) (ω : Assign (Fin n) × Audit (Fin n)) :
recordOf (blockSchedule n B d false h (ξ.1, fun ℓ => -ξ.2 ℓ)) ω =
let o := recordOf (blockSchedule n B d true h ξ) ω
(o.1, o.2.1, fun i => -o.2.2 i) := by
apply Prod.ext
· rfl
apply Prod.ext
· rfl
funext i
simp only [recordOf, potentialOutcome, blockSchedule, signOf, Bool.false_eq_true,
if_false, if_true, zero_mul, add_zero, inNbhd]
rw [neg_add, ← Finset.sum_neg_distrib, ← Finset.sum_neg_distrib]
congr 1
· apply Finset.sum_congr rfl
intro ℓ _
split_ifs <;> ring
· apply Finset.sum_congr rfl
intro j _
ring
theorem blockMixtureLawOf_sign_reflection
The entire negative-sign record law is the response reflection of the positive-sign law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= blockMixtureLawOf n B d D false h
Proof (Lean source)
-- @node: blockMixtureLawOf_sign_reflection
lemma blockMixtureLawOf_sign_reflection (n B d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
(blockMixtureLawOf n B d D true h).map
(fun o => (o.1, o.2.1, fun i => -o.2.2 i)) = blockMixtureLawOf n B d D false h := by
let π := blockParamLaw B d
let := partitionLaw_probability B d
let := blockBaselineLaw_probability B
let : IsProbabilityMeasure π := by dsimp [π, blockParamLaw]; infer_instance
let f := fun ξ : SourcePartition B d × (Fin B → ℝ) => (ξ.1, fun ℓ => -ξ.2 ℓ)
have hf : Measurable f := by fun_prop
have hpi : π.map f = π := blockParamLaw_map_neg B d
have hprod : (π.prod D).map (map f id) = π.prod D := by
rw [← Measure.map_prod_map _ _ hf measurable_id, hpi, Measure.map_id]
rw [blockMixtureLawOf, mixtureLaw_eq_product_map D _ _
(block_record_measurable n B d true h),
Measure.map_map (by fun_prop) (block_record_measurable n B d true h)]
rw [blockMixtureLawOf, mixtureLaw_eq_product_map D _ _
(block_record_measurable n B d false h), ← hprod,
Measure.map_map (block_record_measurable n B d false h) (hf.prodMap measurable_id)]
congr 1
funext x
exact (block_record_sign_reflection n B d h x.1 x.2).symm
Averaging identical recipients commutes with reflection of every outcome. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
distinctResponses n B d (o.1, o.2.1, fun i => -o.2.2 i)
= fun ℓ => -distinctResponses n B d o ℓ
Proof (Lean source)
-- @node: distinctResponses_neg
lemma distinctResponses_neg (n B d : ℕ) (o : Record (Fin n)) :
distinctResponses n B d (o.1, o.2.1, fun i => -o.2.2 i) =
fun ℓ => -distinctResponses n B d o ℓ := by
funext ℓ
simp [distinctResponses, Finset.sum_neg_distrib]
theorem reducedBlockLaw_sign_reflection
Response reflection preserves the full detailed retained-graph coordinate and assignments in the reduced experiment. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= reducedBlockLaw n B d D false h
Proof (Lean source)
-- @node: reducedBlockLaw_sign_reflection
lemma reducedBlockLaw_sign_reflection (n B d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
(reducedBlockLaw n B d D true h).map
(fun x => (x.1, x.2.1, fun ℓ => -x.2.2 ℓ)) = reducedBlockLaw n B d D false h := by
rw [reducedBlockLaw, reducedBlockLaw, ← blockMixtureLawOf_sign_reflection n B d h D,
Measure.map_map (by fun_prop) (by fun_prop),
Measure.map_map (by fun_prop) (by fun_prop)]
congr 1
funext o
simp only [Function.comp_apply, distinctResponses_neg]
theorem conditionalBlockLaw_sign_reflection
Conditioning on any complete retained graph commutes with response reflection. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: conditionalBlockLaw_sign_reflection
lemma conditionalBlockLaw_sign_reflection (n B d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(H : OffDiag (Fin n) → Bool) :
(conditionalBlockLaw n B d D true h H).map
(fun x => (x.1, fun ℓ => -x.2 ℓ)) = conditionalBlockLaw n B d D false h H := by
rw [conditionalBlockLaw, conditionalBlockLaw, Measure.map_smul,
← reducedBlockLaw_sign_reflection n B d h D,
Measure.restrict_map (by fun_prop) (by measurability),
Measure.map_map (by fun_prop) (by fun_prop),
Measure.map_map (by fun_prop) (by fun_prop)]
rfl
Reflecting every response preserves the uniform-sign reference density. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: refDensity_neg
lemma refDensity_neg (d : ℕ) (h w : ℝ) : refDensity d h (-w) = refDensity d h w := by
classical
let e : (Fin d → Bool) ≃ (Fin d → Bool) :=
{ toFun := fun ε j => !(ε j)
invFun := fun ε j => !(ε j)
left_inv := fun ε => by funext j; simp
right_inv := fun ε => by funext j; simp }
have hsign (b : Bool) : signOf (!b) = -signOf b := by cases b <;> norm_num [signOf]
have hrow (ε : Fin d → Bool) : rowDensity d h ε (-w) = rowDensity d h (e ε) w := by
have hsum : (∑ j, signOf ((e ε) j)) = -(∑ j, signOf (ε j)) := by
simp only [e, Equiv.coe_fn_mk, hsign, Finset.sum_neg_distrib]
rw [rowDensity, rowDensity, hsum]
have he : -w - h / (2 * d) * (∑ j, signOf (ε j)) =
-(w - h / (2 * d) * -(∑ j, signOf (ε j))) := by ring
rw [he, cosSqDensity_neg]
unfold refDensity
congr 1
exact Fintype.sum_equiv e _ _ hrow
Reflecting a reference response preserves its law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_referenceRow_map_neg
lemma block_referenceRow_map_neg (d : ℕ) (h : ℝ) :
(volume.withDensity (fun w => ofReal (refDensity d h w))).map
(fun w : ℝ => -w) = volume.withDensity (fun w => ofReal (refDensity d h w)) := by
have ht := map_withDensity_comp_measurableEquiv
(MeasurableEquiv.neg ℝ) (Measure.measurePreserving_neg volume)
(fun w => ofReal (refDensity d h w))
simpa [Function.comp_def, refDensity_neg] using ht
The independent reference responses are jointly invariant under reflection. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
h :
ℝ
conclusion
(Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))).map (fun y ℓ => -y ℓ)
Proof (Lean source)
-- @node: block_referencePi_map_neg
lemma block_referencePi_map_neg (B d : ℕ) (h : ℝ) :
(Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))).map (fun y ℓ => -y ℓ) =
Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
let := referenceRowLaw_probability d h
let : IsProbabilityMeasure ((volume.withDensity
(fun w => ofReal (refDensity d h w))).map (fun w : ℝ => -w)) :=
Measure.isProbabilityMeasure_map (by fun_prop)
rw [Measure.pi_map_pi (f := fun _ w => -w) (fun _ => by fun_prop)]
simp_rw [block_referenceRow_map_neg]
theorem conditionalBlockLaw_chiSq_sign_eq
The two conditional sign laws have equal chi-squared divergence against the actual assignment-reference product, by measurable response reflection. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= hiddenChiSq n B d D h H
Proof (Lean source)
-- @node: conditionalBlockLaw_chiSq_sign_eq
lemma conditionalBlockLaw_chiSq_sign_eq (n B d : ℕ) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(H : OffDiag (Fin n) → Bool) (hH : retainedGraphMarginal n B d D {H} ≠ 0) :
chiSqDiv (conditionalBlockLaw n B d D false h H)
(hiddenReferenceLaw n B d D h) = hiddenChiSq n B d D h H := by
let := conditionalBlockLaw_probability n B d D true h H hH
let := hiddenReferenceLaw_probability n B d h D
let e := (MeasurableEquiv.refl (Assign (Fin n))).prodCongr
(MeasurableEquiv.piCongrRight (fun _ : Fin B => MeasurableEquiv.neg ℝ))
have he := chiSqDiv_map_measurableEquiv e
(conditionalBlockLaw n B d D true h H) (hiddenReferenceLaw n B d D h)
change chiSqDiv
((conditionalBlockLaw n B d D true h H).map (fun x => (x.1, fun ℓ => -x.2 ℓ)))
((hiddenReferenceLaw n B d D h).map (fun x => (x.1, fun ℓ => -x.2 ℓ))) = _ at he
rwa [conditionalBlockLaw_sign_reflection, hiddenReferenceLaw_map_neg] at he
theorem conditionalBlockLaw_false_absolutelyContinuous
The negative-sign conditional law is also dominated by the reference, since reflection preserves that reference and transports the actual positive-sign law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
Proof (Lean source)
-- @node: conditionalBlockLaw_false_absolutelyContinuous
lemma conditionalBlockLaw_false_absolutelyContinuous (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(H : OffDiag (Fin n) → Bool) :
conditionalBlockLaw n B d D false h H ≪ hiddenReferenceLaw n B d D h := by
have hac := (conditionalBlockLaw_absolutelyContinuous_of_probability n B d hd hfit D h H).map
(f := fun x => (x.1, fun ℓ => -x.2 ℓ)) (by fun_prop)
rwa [conditionalBlockLaw_sign_reflection, hiddenReferenceLaw_map_neg] at hac
theorem conditionalBlockLaw_tvDist_le_sqrt_chiSq
On any positive-probability detailed graph, square-integrable conditional likelihoods convert the common chi-squared divergence into the conditional testing bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
hH :
retainedGraphMarginal n B d D {H} ≠ 0
hi :
∀ σ : Bool,
Integrable (fun x => (((conditionalBlockLaw n B d D σ h H).rnDeriv (hiddenReferenceLaw n B d D h) x).toReal - 1) ^ 2) (hiddenReferenceLaw n B d D h)
conclusion
≤ sqrt (hiddenChiSq n B d D h H)
Proof (Lean source)
-- @node: conditionalBlockLaw_tvDist_le_sqrt_chiSq
lemma conditionalBlockLaw_tvDist_le_sqrt_chiSq (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(H : OffDiag (Fin n) → Bool) (hH : retainedGraphMarginal n B d D {H} ≠ 0)
(hi : ∀ σ : Bool, Integrable (fun x =>
(((conditionalBlockLaw n B d D σ h H).rnDeriv (hiddenReferenceLaw n B d D h) x).toReal
- 1) ^ 2) (hiddenReferenceLaw n B d D h)) :
tvDist (conditionalBlockLaw n B d D true h H)
(conditionalBlockLaw n B d D false h H) ≤ sqrt (hiddenChiSq n B d D h H) := by
let := conditionalBlockLaw_probability n B d D true h H hH
let := conditionalBlockLaw_probability n B d D false h H hH
let := hiddenReferenceLaw_probability n B d h D
exact block_tvDist_le_sqrt_common_chiSq _ _ _
(conditionalBlockLaw_absolutelyContinuous_of_probability n B d hd hfit D h H)
(conditionalBlockLaw_false_absolutelyContinuous n B d hd hfit h D H)
(hi true) (hi false) (conditionalBlockLaw_chiSq_sign_eq n B d h D H hH)
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ConditionalLikelihoodBounds 9 declarations The finite row-channel mixture bounds actual conditional likelihoods and supplies square integrability without an extra regularity premise.
Finite conditional likelihood bounds
The finite row-channel mixture bounds actual conditional likelihoods and supplies square integrability without an extra regularity premise.
Tensorizing the finite row bound gives a finite bound for every fixed allocation. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: translatedRows_le_reference
lemma translatedRows_le_reference (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) :
Measure.pi (fun ℓ => volume.withDensity
(fun w => ofReal (rowDensity d h (ε ℓ) w))) ≤
(ofReal ((2 : ℝ) ^ d)) ^ B • Measure.pi (fun _ : Fin B =>
volume.withDensity (fun w => ofReal (refDensity d h w))) := by
let ν := volume.withDensity (fun w => ofReal (refDensity d h w))
let μ := fun δ : Fin d → Bool => volume.withDensity
(fun w => ofReal (rowDensity d h δ w))
let c := ofReal ((2 : ℝ) ^ d)
let := referenceRowLaw_probability d h
have (δ : Fin d → Bool) : IsProbabilityMeasure (μ δ) := translatedRowLaw_probability d h δ
change Measure.pi (fun ℓ => μ (ε ℓ)) ≤ c ^ B • Measure.pi (fun _ : Fin B => ν)
induction B with
| zero => rw [Measure.pi_of_empty, Measure.pi_of_empty]; simp
| succ B ih =>
let e := piFinSuccAbove (fun _ : Fin (B + 1) => ℝ) 0
have hμ := (measurePreserving_piFinSuccAbove (fun ℓ => μ (ε ℓ)) 0).map_eq
have hν := (measurePreserving_piFinSuccAbove (fun _ : Fin (B + 1) => ν) 0).map_eq
have hp := Measure.prod_mono (rowLaw_le_reference d h (ε 0))
(ih (fun ℓ => ε ((0 : Fin (B + 1)).succAbove ℓ)))
rw [Measure.prod_smul_left, Measure.prod_smul_right, smul_smul] at hp
have hm : (Measure.pi (fun ℓ => μ (ε ℓ))).map e ≤
(c ^ (B + 1) • Measure.pi (fun _ : Fin (B + 1) => ν)).map e := by
rw [Measure.map_smul, hμ, hν, pow_succ, mul_comm (c ^ B) c]
exact hp
have hm' := Measure.map_mono hm e.symm.measurable
simpa only [Measure.map_map e.symm.measurable e.measurable,
MeasurableEquiv.symm_comp_self, Measure.map_id] using hm'
theorem block_responseLaw_le_reference
Every actual fixed-partition response law obeys the finite row-channel bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(blockBaselineLaw B).map (fun U => distinctResponses n B d (recordOf (blockSchedule n B d true h (s, U)) ω))
≤ (ofReal ((2 : ℝ) ^ d)) ^ B • Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: block_responseLaw_le_reference
lemma block_responseLaw_le_reference (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) :
(blockBaselineLaw B).map (fun U =>
distinctResponses n B d (recordOf (blockSchedule n B d true h (s, U)) ω)) ≤
(ofReal ((2 : ℝ) ^ d)) ^ B • Measure.pi (fun _ : Fin B =>
volume.withDensity (fun w => ofReal (refDensity d h w))) := by
obtain ⟨ε, he⟩ := block_responseLaw_eq_translatedRows n B d hd hfit h s ω
rw [he]
exact translatedRows_le_reference B d h ε
theorem reducedBlockLaw_snd_le_reference
Averaging actual partition and design draws preserves the uniform density bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
≤ (ofReal ((2 : ℝ) ^ d)) ^ B • hiddenReferenceLaw n B d D h
Proof (Lean source)
-- @node: reducedBlockLaw_snd_le_reference
lemma reducedBlockLaw_snd_le_reference (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] (h : ℝ) :
(reducedBlockLaw n B d D true h).map snd ≤
(ofReal ((2 : ℝ) ^ d)) ^ B • hiddenReferenceLaw n B d D h := by
let ν := Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))
let c := (ofReal ((2 : ℝ) ^ d)) ^ B
let := referenceRowLaw_probability d h
let := blockBaselineLaw_probability B
let := partitionLaw_probability B d
let f := fun o : Record (Fin n) => (o.2.1, distinctResponses n B d o)
have hf : Measurable f := by fun_prop
have hrec : Measurable (fun x : (SourcePartition B d × (Fin B → ℝ)) ×
(Assign (Fin n) × Audit (Fin n)) => f (recordOf (blockSchedule n B d true h x.1) x.2)) :=
hf.comp (block_record_measurable n B d true h)
let K := fun ξ : SourcePartition B d × (Fin B → ℝ) =>
D.map (fun ω => f (recordOf (blockSchedule n B d true h ξ) ω))
have hK : Measurable K := block_measurable_map_parameter D _ hrec
have heq : (reducedBlockLaw n B d D true h).map snd = (blockParamLaw B d).bind K := by
rw [reducedBlockLaw, Measure.map_map (by fun_prop) (by fun_prop)]
change (blockMixtureLawOf n B d D true h).map f = _
rw [blockMixtureLawOf, mixtureLaw, block_map_bind _ _ _
(block_measurable_map_parameter D _ (block_record_measurable n B d true h)) hf]
congr 1
funext ξ
exact Measure.map_map hf (measurable_of_finite _)
rw [heq]
apply Measure.le_iff.mpr
intro E hE
have hs (s : SourcePartition B d) :
(∫⁻ U, K (s, U) E ∂blockBaselineLaw B) ≤ c * hiddenReferenceLaw n B d D h E := by
let g := fun x : (Fin B → ℝ) × (Assign (Fin n) × Audit (Fin n)) =>
f (recordOf (blockSchedule n B d true h (s, x.1)) x.2)
have hg : Measurable g := hrec.comp
((measurable_const.prodMk measurable_fst).prodMk measurable_snd)
have he : (∫⁻ U, K (s, U) E ∂blockBaselineLaw B) =
((blockBaselineLaw B).prod D) (g ⁻¹' E) := by
rw [Measure.prod_apply (hg hE)]
apply lintegral_congr
intro U
exact Measure.map_apply (measurable_of_finite _) hE
rw [he, Measure.prod_apply_symm (hg hE)]
have href : hiddenReferenceLaw n B d D h E = ∫⁻ ω, ν (Prod.mk ω.1 ⁻¹' E) ∂D := by
rw [hiddenReferenceLaw, Measure.prod_apply hE,
lintegral_map (measurable_of_finite _) (by fun_prop)]
rw [href, ← lintegral_const_mul' c _ (by simp [c])]
apply lintegral_mono
intro ω
have hle := block_responseLaw_le_reference n B d hd hfit h s ω
have hm : Measurable (fun U => distinctResponses n B d
(recordOf (blockSchedule n B d true h (s, U)) ω)) :=
(distinctResponses_measurable n B d).comp
((block_record_measurable n B d true h).comp
((measurable_const.prodMk measurable_id).prodMk measurable_const))
have hp := hle (Prod.mk ω.1 ⁻¹' E)
rw [Measure.map_apply hm (measurable_prodMk_left hE), Measure.smul_apply,
smul_eq_mul] at hp
exact hp
rw [Measure.bind_apply hE hK.aemeasurable, blockParamLaw,
lintegral_prod (fun ξ => K ξ E) ((Measure.measurable_coe hE).comp hK).aemeasurable]
change _ ≤ c * hiddenReferenceLaw n B d D h E
exact (lintegral_mono hs).trans_eq (by simp)
theorem conditionalBlockLaw_le_reference
Restricting to a graph atom costs only its inverse probability in the density bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
conditionalBlockLaw n B d D true h H
≤ ((retainedGraphMarginal n B d D {H})⁻¹ * (ofReal ((2 : ℝ) ^ d)) ^ B) • hiddenReferenceLaw n B d D h
Proof (Lean source)
-- @node: conditionalBlockLaw_le_reference
lemma conditionalBlockLaw_le_reference (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(h : ℝ) (H : OffDiag (Fin n) → Bool) :
conditionalBlockLaw n B d D true h H ≤
((retainedGraphMarginal n B d D {H})⁻¹ * (ofReal ((2 : ℝ) ^ d)) ^ B) •
hiddenReferenceLaw n B d D h := by
have hle : ((reducedBlockLaw n B d D true h).restrict {x | x.1 = H}).map snd ≤
(reducedBlockLaw n B d D true h).map snd :=
Measure.map_mono Measure.restrict_le_self (by fun_prop)
rw [conditionalBlockLaw, ← smul_smul]
exact smul_le_smul_left _ (hle.trans (reducedBlockLaw_snd_le_reference n B d hd hfit D h))
A finite domination constant bounds the Radon--Nikodym derivative almost everywhere. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
P.rnDeriv Q ≤ᵐ[Q] fun _ => c
Proof (Lean source)
-- @node: block_rnDeriv_le_of_le_smul
lemma block_rnDeriv_le_of_le_smul {α : Type*} [MeasurableSpace α]
(P Q : Measure α) [SigmaFinite Q] (c : ℝ≥0∞)
(hle : P ≤ c • Q) : P.rnDeriv Q ≤ᵐ[Q] fun _ => c := by
apply ae_le_of_forall_setLIntegral_le_of_sigmaFinite (P.measurable_rnDeriv Q)
intro E hE hfin
rw [lintegral_const, Measure.restrict_apply_univ]
exact (Measure.setLIntegral_rnDeriv_le E).trans (hle E)
theorem block_rnDeriv_sub_one_sq_integrable Lemma block_rnDeriv_sub_one_sq_integrable in the paper ↗
Finite measure domination supplies the square-integrability used by the testing step. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable (fun x => ((P.rnDeriv Q x).toReal - 1) ^ 2) Q
Proof (Lean source)
-- @node: block_rnDeriv_sub_one_sq_integrable
lemma block_rnDeriv_sub_one_sq_integrable {α : Type*} [MeasurableSpace α]
(P Q : Measure α) [IsFiniteMeasure Q] (c : ℝ≥0∞) (hc : c ≠ ∞)
(hle : P ≤ c • Q) :
Integrable (fun x => ((P.rnDeriv Q x).toReal - 1) ^ 2) Q := by
have hb := block_rnDeriv_le_of_le_smul P Q c hle
apply (integrable_const ((c.toReal + 1) ^ 2)).mono'
(((P.measurable_rnDeriv Q).ennreal_toReal.sub measurable_const).pow_const
2).aestronglyMeasurable
filter_upwards [hb] with x hx
have ht : (P.rnDeriv Q x).toReal ≤ c.toReal := ENNReal.toReal_mono hc hx
have hn : 0 ≤ (P.rnDeriv Q x).toReal := ENNReal.toReal_nonneg
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
change ((P.rnDeriv Q x).toReal - 1) ^ 2 ≤ (c.toReal + 1) ^ 2
have hab : |(P.rnDeriv Q x).toReal - 1| ≤ c.toReal + 1 :=
(by
have ha := abs_add_le (P.rnDeriv Q x).toReal (-1)
simpa [sub_eq_add_neg, abs_of_nonneg hn] using ha.trans (by
simpa [abs_of_nonneg hn] using add_le_add_right ht 1))
simpa only [sq_abs] using pow_le_pow_left₀ (abs_nonneg _) hab 2
theorem conditionalBlockLaw_likelihood_sq_integrable
Conditional likelihoods have finite second moments on each positive graph atom. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable (fun x => (((conditionalBlockLaw n B d D σ h H).rnDeriv (hiddenReferenceLaw n B d D h) x).toReal - 1) ^ 2) (hiddenReferenceLaw n B d D h)
Proof (Lean source)
-- @node: conditionalBlockLaw_likelihood_sq_integrable
lemma conditionalBlockLaw_likelihood_sq_integrable (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(h : ℝ) (H : OffDiag (Fin n) → Bool)
(hH : retainedGraphMarginal n B d D {H} ≠ 0) (σ : Bool) :
Integrable (fun x =>
(((conditionalBlockLaw n B d D σ h H).rnDeriv (hiddenReferenceLaw n B d D h) x).toReal
- 1) ^ 2) (hiddenReferenceLaw n B d D h) := by
let := hiddenReferenceLaw_probability n B d h D
let c := (retainedGraphMarginal n B d D {H})⁻¹ * (ofReal ((2 : ℝ) ^ d)) ^ B
have hc : c ≠ ∞ := ENNReal.mul_ne_top (by simpa using hH) (by simp)
have hp := conditionalBlockLaw_le_reference n B d hd hfit D h H
have hbound : conditionalBlockLaw n B d D σ h H ≤ c • hiddenReferenceLaw n B d D h := by
cases σ
· have hm := Measure.map_mono hp
(show Measurable (fun x : Assign (Fin n) × (Fin B → ℝ) =>
(x.1, fun ℓ => -x.2 ℓ)) by fun_prop)
rwa [conditionalBlockLaw_sign_reflection, Measure.map_smul, hiddenReferenceLaw_map_neg] at hm
· exact hp
exact block_rnDeriv_sub_one_sq_integrable _ _ c hc hbound
theorem conditionalBlockLaw_tvDist_le_sqrt_chiSq_unconditional
The roadmap's conditional testing bound requires no extra square-integrability assumption. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
≤ sqrt (hiddenChiSq n B d D h H)
Proof (Lean source)
-- @node: conditionalBlockLaw_tvDist_le_sqrt_chiSq_unconditional
lemma conditionalBlockLaw_tvDist_le_sqrt_chiSq_unconditional (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(H : OffDiag (Fin n) → Bool) (hH : retainedGraphMarginal n B d D {H} ≠ 0) :
tvDist (conditionalBlockLaw n B d D true h H)
(conditionalBlockLaw n B d D false h H) ≤ sqrt (hiddenChiSq n B d D h H) := by
exact conditionalBlockLaw_tvDist_le_sqrt_chiSq n B d hd hfit h D H hH
(conditionalBlockLaw_likelihood_sq_integrable n B d hd hfit D h H hH)
theorem blockMixture_tvDist_good_event_chiSq_le
Averaging over the actual complete graph marginal gives the good-event testing bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
≤ (retainedGraphMarginal n B d D).real Eᶜ
Proof (Lean source)
-- @node: blockMixture_tvDist_good_event_chiSq_le
lemma blockMixture_tvDist_good_event_chiSq_le (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(h : ℝ) (E : Set (OffDiag (Fin n) → Bool)) :
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤
(retainedGraphMarginal n B d D).real Eᶜ +
sqrt (∫ H, E.indicator (hiddenChiSq n B d D h) H
∂(retainedGraphMarginal n B d D)) := by
apply blockMixture_tvDist_good_event_le n B d hd hfit D h E
have hpos : ∀ᵐ H ∂(retainedGraphMarginal n B d D),
retainedGraphMarginal n B d D {H} ≠ 0 := ae_iff_of_countable.mpr (fun _ => id)
filter_upwards [hpos] with H hH
intro _
exact conditionalBlockLaw_tvDist_le_sqrt_chiSq_unconditional n B d hd hfit h D H hH
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ConditionalWalshBridge 13 declarations Enumerate each completed row using its original source labels, expand its actual translated response density, and average over the exact compatible posterior.
Actual labeled-completion Walsh likelihood
Enumerate each completed row using its original source labels, expand its actual translated response density, and average over the exact compatible posterior. The reference-zero set is kept separate from the positive-density expansion.
def partitionRowSigns
A row sign vector reads the original assignment at every completed source label.
Definition (Lean source)
parameters
given by
fun j
=> z ((Finset.equivFinOfCardEq (partitionSourceLabels_card n B d hfit s ℓ)).symm j).val
theorem partitionRowSigns_sum
Enumerating a completed row preserves its full labeled sign sum. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j, signOf (partitionRowSigns n B d hfit s z ℓ j))
= ∑ j ∈ partitionSourceLabels n B d s ℓ, signOf (z j)
Proof (Lean source)
-- @node: partitionRowSigns_sum
lemma partitionRowSigns_sum (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (z : Assign (Fin n)) (ℓ : Fin B) :
(∑ j, signOf (partitionRowSigns n B d hfit s z ℓ j)) =
∑ j ∈ partitionSourceLabels n B d s ℓ, signOf (z j) := by
conv_rhs => rw [← Finset.sum_attach]
exact (Fintype.sum_equiv
(Finset.equivFinOfCardEq (partitionSourceLabels_card n B d hfit s ℓ))
(fun j => signOf (z j.val))
(fun j => signOf (partitionRowSigns n B d hfit s z ℓ j))
(fun j => by simp [partitionRowSigns])).symm
theorem partitionResponseDensity_eq_rowDensity
The positive-prior density is a product of actual completed-row channels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
partitionResponseDensity n B d true h s ω y
= ∏ ℓ, rowDensity d h (partitionRowSigns n B d hfit s ω.1 ℓ) (y ℓ)
Proof (Lean source)
-- @node: partitionResponseDensity_eq_rowDensity
lemma partitionResponseDensity_eq_rowDensity (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (y : Fin B → ℝ) :
partitionResponseDensity n B d true h s ω y =
∏ ℓ, rowDensity d h (partitionRowSigns n B d hfit s ω.1 ℓ) (y ℓ) := by
apply Finset.prod_congr rfl
intro ℓ _
obtain ⟨i, hi⟩ := Finset.card_pos.mp (by
rw [recipientBlock_card n B d hfit ℓ]; omega)
rw [partitionResponseShift_eq_source_sum n B d hd hfit true h s ω ℓ i hi]
rw [rowDensity, partitionRowSigns_sum,
partitionSourceLabels_eq_inNbhd n B d s ℓ i hi]
simp only [signOf, ite_true, one_mul]
theorem partitionResponseDensity_walsh_expansion
The full fixed-partition likelihood expands row by row without losing labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s :
SourcePartition B d
y :
Fin B → ℝ
hy :
∀ ℓ, 0 < refDensity d h (y ℓ)
conclusion
= ∏ ℓ, ∑ E : Finset (Fin d), walshCoeff d h E.card (y ℓ) * ∏ j ∈ E, signOf (partitionRowSigns n B d hfit s ω.1 ℓ j)
Proof (Lean source)
-- @node: partitionResponseDensity_walsh_expansion
lemma partitionResponseDensity_walsh_expansion (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
partitionResponseDensity n B d true h s ω y / (∏ ℓ, refDensity d h (y ℓ)) =
∏ ℓ, ∑ E : Finset (Fin d), walshCoeff d h E.card (y ℓ) *
∏ j ∈ E, signOf (partitionRowSigns n B d hfit s ω.1 ℓ j) := by
rw [partitionResponseDensity_eq_rowDensity n B d hd hfit,
← Finset.prod_div_distrib]
apply Finset.prod_congr rfl
intro ℓ _
exact rowDensity_walsh_expansion d h (y ℓ) (hy ℓ) _
theorem partitionResponseDensity_zero_of_reference_zero
Every completed density vanishes when any reference row density is zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s :
SourcePartition B d
y :
Fin B → ℝ
ℓ :
Fin B
hy :
refDensity d h (y ℓ) = 0
conclusion
partitionResponseDensity n B d true h s ω y = 0
Proof (Lean source)
-- @node: partitionResponseDensity_zero_of_reference_zero
lemma partitionResponseDensity_zero_of_reference_zero (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (y : Fin B → ℝ)
(ℓ : Fin B) (hy : refDensity d h (y ℓ) = 0) :
partitionResponseDensity n B d true h s ω y = 0 := by
rw [partitionResponseDensity_eq_rowDensity n B d hd hfit]
exact Finset.prod_eq_zero (Finset.mem_univ ℓ)
(rowDensity_eq_zero_of_refDensity_eq_zero d h (y ℓ) hy _)
theorem blockDensity_completion_walsh_expansion
On positive reference density, the actual conditional response likelihood is exactly the uniform posterior average of products of labeled Walsh expansions. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s₀ :
CompatiblePartition n B d H
y :
Fin B → ℝ
hy :
∀ ℓ, 0 < refDensity d h (y ℓ)
conclusion
= (card (HiddenCompletion n B d H) : ℝ)⁻¹ * ∑ g : HiddenCompletion n B d H, ∏ ℓ, ∑ E : Finset (Fin d), walshCoeff d h E.card (y ℓ) * ∏ j ∈ E, signOf (partitionRowSigns n B d hfit (hiddenCompletionPartition n B d hfit H s₀ g).1 z ℓ j)
Proof (Lean source)
-- @node: blockDensity_completion_walsh_expansion
lemma blockDensity_completion_walsh_expansion (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n))
(s₀ : CompatiblePartition n B d H) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) =
(card (HiddenCompletion n B d H) : ℝ)⁻¹ *
∑ g : HiddenCompletion n B d H,
∏ ℓ, ∑ E : Finset (Fin d), walshCoeff d h E.card (y ℓ) *
∏ j ∈ E, signOf (partitionRowSigns n B d hfit
(hiddenCompletionPartition n B d hfit H s₀ g).1 z ℓ j) := by
rw [← hiddenCompletion_density_average_real n B d hd hfit h H z s₀ y,
mul_div_assoc, Finset.sum_div]
congr 1
apply Finset.sum_congr rfl
intro g _
exact partitionResponseDensity_walsh_expansion n B d hd hfit h _ _ y hy
theorem blockDensity_zero_of_reference_zero
The averaged actual density vanishes on the reference-zero set as well. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s₀ :
CompatiblePartition n B d H
y :
Fin B → ℝ
ℓ :
Fin B
hy :
refDensity d h (y ℓ) = 0
conclusion
blockDensity n B d true h H z y = 0
Proof (Lean source)
-- @node: blockDensity_zero_of_reference_zero
lemma blockDensity_zero_of_reference_zero (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n))
(s₀ : CompatiblePartition n B d H) (y : Fin B → ℝ)
(ℓ : Fin B) (hy : refDensity d h (y ℓ) = 0) :
blockDensity n B d true h H z y = 0 := by
rw [← hiddenCompletion_density_average_real n B d hd hfit h H z s₀ y]
simp_rw [partitionResponseDensity_zero_of_reference_zero n B d hd hfit h _ _ y ℓ hy]
simp
def completionWalshDensity
The exact completion-averaged Walsh density, extended by zero off its reference support and off compatible graphs. Completion averaging retains the assignment labels.
Definition (Lean source)
parameters
def
completionWalshDensity n B d hfit h hz y :
ℝ
given by
if hs : Nonempty (CompatiblePartition n B d hz.1) then if ∀ ℓ, 0 < refDensity d h (y ℓ) then (∏ ℓ, refDensity d h (y ℓ)) * ((card (HiddenCompletion n B d hz.1) : ℝ)⁻¹ * ∑ g : HiddenCompletion n B d hz.1, ∏ ℓ, ∑ E : Finset (Fin d), walshCoeff d h E.card (y ℓ) * ∏ j ∈ E, signOf (partitionRowSigns n B d hfit (hiddenCompletionPartition n B d hfit hz.1 hs.some g).1 hz.2 ℓ j)) else 0 else 0
theorem completionWalshDensity_eq_blockDensity
On the actual graph support this finite Walsh density equals the actual density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: completionWalshDensity_eq_blockDensity
lemma completionWalshDensity_eq_blockDensity (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n))
(hH : ValidRetainedGraph n B d hz.1) (y : Fin B → ℝ) :
completionWalshDensity n B d hfit h hz y = blockDensity n B d true h hz.1 hz.2 y := by
have hs : Nonempty (CompatiblePartition n B d hz.1) := by
obtain ⟨_, s, hs⟩ := hH
exact ⟨⟨s, hs⟩⟩
rw [completionWalshDensity, dif_pos hs]
split_ifs with hy
· rw [← blockDensity_completion_walsh_expansion n B d hd hfit h hz.1 hz.2 hs.some y hy]
exact mul_div_cancel₀ _ (ne_of_gt (Finset.prod_pos (fun ℓ _ => hy ℓ)))
· obtain ⟨ℓ, hℓ⟩ := not_forall.mp hy
have hz0 : refDensity d h (y ℓ) = 0 :=
le_antisymm (not_lt.mp hℓ) (refDensity_nonneg d h (y ℓ))
exact (blockDensity_zero_of_reference_zero n B d hd hfit h hz.1 hz.2 hs.some y ℓ hz0).symm
theorem blockMixtureLawOf_completion_walsh_representation Lemma blockMixtureLawOf_completion_walsh_representation in the paper ↗
Replacing the posterior density by its finite Walsh expansion gives exactly the full original-record law, with all edge subsets and outcome copies. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
conclusion
= (graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (volume.withDensity (fun y => ofReal (completionWalshDensity n B d hfit h hz y))).map (copyOutcomes n B d hz))
Proof (Lean source)
lemma blockMixtureLawOf_completion_walsh_representation (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1) :
blockMixtureLawOf n B d (thinnedDesign (Fin n) q) true h =
(graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(volume.withDensity (fun y => ofReal
(completionWalshDensity n B d hfit h hz y))).map (copyOutcomes n B d hz)) := by
rw [(block_law n B d true h q (thinnedDesign (Fin n) q) hn hB hd hfit hh hq
(thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)).2.2.1, reconstructedLaw]
apply Measure.bind_congr_right
have hv : ∀ᵐ hz ∂(graphAssignMarginal n B d (thinnedDesign (Fin n) q)),
ValidRetainedGraph n B d hz.1 :=
(ae_map_iff (by fun_prop) (by measurability)).mp
(retainedGraphMarginal_valid n B d (thinnedDesign (Fin n) q) hfit)
filter_upwards [hv] with hz hH
simp_rw [completionWalshDensity_eq_blockDensity n B d hd hfit h hz hH]
theorem reducedBlockLaw_completion_walsh_representation Lemma reducedBlockLaw_completion_walsh_representation in the paper ↗
Recovering distinct responses from the Walsh density preserves the complete retained graph and every assignment coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
conclusion
= (graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (volume.withDensity (fun y => ofReal (completionWalshDensity n B d hfit h hz y))).map (fun y => (hz.1, hz.2, y)))
Proof (Lean source)
lemma reducedBlockLaw_completion_walsh_representation (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1) :
reducedBlockLaw n B d (thinnedDesign (Fin n) q) true h =
(graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(volume.withDensity (fun y => ofReal
(completionWalshDensity n B d hfit h hz y))).map
(fun y => (hz.1, hz.2, y))) := by
rw [reducedBlockLaw, blockMixtureLawOf_completion_walsh_representation n B d h q
hn hB hd hfit hh hq, block_map_bind _ _ _ (measurable_of_finite _) (by fun_prop)]
congr 1
funext hz
have hm : Measurable (copyOutcomes n B d hz) := by
simpa only [Function.comp_def] using (copyOutcomes_measurable n B d).comp
(show Measurable (fun y : Fin B → ℝ => (hz.1, hz.2, y)) from by fun_prop)
rw [Measure.map_map (by fun_prop) hm]
congr 1
funext y
simp only [Function.comp_apply, distinctResponses_copyOutcomes n B d hd hfit]
rfl
theorem conditionalBlockLaw_completion_walsh_representation Lemma conditionalBlockLaw_completion_walsh_representation in the paper ↗
The actual conditional positive-prior law, expressed using its finite completion-averaged Walsh density. Restriction conditions on the entire graph. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
conclusion
= (retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H})⁻¹ • (((graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (volume.withDensity (fun y => ofReal (completionWalshDensity n B d hfit h hz y))).map (fun y => (hz.1, hz.2, y)))).restrict {x | x.1 = H}).map snd
Proof (Lean source)
lemma conditionalBlockLaw_completion_walsh_representation (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(H : OffDiag (Fin n) → Bool) :
conditionalBlockLaw n B d (thinnedDesign (Fin n) q) true h H =
(retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H})⁻¹ •
(((graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(volume.withDensity (fun y => ofReal
(completionWalshDensity n B d hfit h hz y))).map
(fun y => (hz.1, hz.2, y)))).restrict {x | x.1 = H}).map snd := by
rw [conditionalBlockLaw,
reducedBlockLaw_completion_walsh_representation n B d h q hn hB hd hfit hh hq]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ConditionalWalshRegrouping 16 declarations Split each completed row into its fixed revealed labels and original hidden labels, then group hidden selections by degree before posterior averaging.
Revealed and hidden label regrouping of the actual Walsh likelihood
Split each completed row into its fixed revealed labels and original hidden labels, then group hidden selections by degree before posterior averaging.
Reindex selected Walsh coordinates without changing their cardinalities. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: labeled_walsh_subset_sum_reindex
lemma labeled_walsh_subset_sum_reindex {ι κ : Type*} [Fintype ι] [DecidableEq κ]
(e : ι ↪ κ) (a : ℕ → ℝ) (x : κ → ℝ) :
(∑ E : Finset ι, a E.card * ∏ j ∈ E, x (e j)) =
∑ E ∈ (Finset.univ.image e).powerset, a E.card * ∏ j ∈ E, x j := by
classical
rw [Finset.powerset_image, Finset.sum_image
(Finset.image_injOn_powerset_of_injOn (fun _ _ _ _ h => e.injective h))]
simp only [Finset.card_image_of_injective _ e.injective,
Finset.prod_image (fun _ _ _ _ h => e.injective h), Finset.powerset_univ]
theorem labeled_walsh_powerset_split
Split selected labels uniquely across disjoint revealed and hidden sets. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ E ∈ (R ∪ U).powerset, f E) = ∑ J ∈ R.powerset, ∑ K ∈ U.powerset, f (J ∪ K)
Proof (Lean source)
-- @node: labeled_walsh_powerset_split
lemma labeled_walsh_powerset_split {ι : Type*} [DecidableEq ι] (R U : Finset ι)
(hRU : Disjoint R U) (f : Finset ι → ℝ) :
(∑ E ∈ (R ∪ U).powerset, f E) =
∑ J ∈ R.powerset, ∑ K ∈ U.powerset, f (J ∪ K) := by
rw [← Finset.sum_product R.powerset U.powerset (fun i => f (i.1 ∪ i.2))]
symm
apply Finset.sum_bij (fun i _ => i.1 ∪ i.2)
· intro i hi
obtain ⟨hJ, hK⟩ := Finset.mem_product.mp hi
exact Finset.mem_powerset.mpr (Finset.union_subset_union
(Finset.mem_powerset.mp hJ) (Finset.mem_powerset.mp hK))
· intro i hi j hj he
obtain ⟨hI₁, hI₂⟩ := Finset.mem_product.mp hi
obtain ⟨hJ₁, hJ₂⟩ := Finset.mem_product.mp hj
have hI₁ := Finset.mem_powerset.mp hI₁
have hI₂ := Finset.mem_powerset.mp hI₂
have hJ₁ := Finset.mem_powerset.mp hJ₁
have hJ₂ := Finset.mem_powerset.mp hJ₂
apply Prod.ext <;> ext x
· have hn : x ∈ R → x ∉ U := fun hx hu => Finset.disjoint_left.mp hRU hx hu
have hm := congrArg (fun S : Finset ι => x ∈ S) he
simp only [mem_union] at hm
constructor
· intro hx
rcases hm.mp (inl hx) with h | h
· exact h
· exact False.elim (hn (hI₁ hx) (hJ₂ h))
· intro hx
rcases hm.mpr (inl hx) with h | h
· exact h
· exact False.elim (hn (hJ₁ hx) (hI₂ h))
· have hn : x ∈ U → x ∉ R := fun hx hr => Finset.disjoint_left.mp hRU hr hx
have hm := congrArg (fun S : Finset ι => x ∈ S) he
simp only [mem_union] at hm
constructor
· intro hx
rcases hm.mp (inr hx) with h | h
· exact False.elim (hn (hI₂ hx) (hJ₁ h))
· exact h
· intro hx
rcases hm.mpr (inr hx) with h | h
· exact False.elim (hn (hJ₂ hx) (hI₁ h))
· exact h
· intro E hE
refine ⟨(E ∩ R, E ∩ U), ?_, ?_⟩
· exact Finset.mem_product.mpr ⟨Finset.mem_powerset.mpr inter_subset_right,
Finset.mem_powerset.mpr Finset.inter_subset_right⟩
· rw [← Finset.inter_union_distrib_left]
exact Finset.inter_eq_left.mpr (Finset.mem_powerset.mp hE)
· intro i hi
rfl
theorem partitionRowSigns_walsh_sum_labels
A completed row Walsh expansion indexed by original population labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ E : Finset (Fin d), walshCoeff d h E.card w * ∏ j ∈ E, signOf (partitionRowSigns n B d hfit s z ℓ j))
Proof (Lean source)
-- @node: partitionRowSigns_walsh_sum_labels
lemma partitionRowSigns_walsh_sum_labels (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(h : ℝ) (s : SourcePartition B d) (z : Assign (Fin n)) (ℓ : Fin B) (w : ℝ) :
(∑ E : Finset (Fin d), walshCoeff d h E.card w *
∏ j ∈ E, signOf (partitionRowSigns n B d hfit s z ℓ j)) =
∑ E ∈ (partitionSourceLabels n B d s ℓ).powerset,
walshCoeff d h E.card w * ∏ j ∈ E, signOf (z j) := by
let e : Fin d ↪ Fin n :=
(Finset.equivFinOfCardEq (partitionSourceLabels_card n B d hfit s ℓ)).symm.toEmbedding.trans
⟨Subtype.val, Subtype.val_injective⟩
have he : Finset.univ.image e = partitionSourceLabels n B d s ℓ := by
ext j
simp only [mem_image, Finset.mem_univ, true_and]
constructor
· rintro ⟨k, rfl⟩
exact ((Finset.equivFinOfCardEq (partitionSourceLabels_card n B d hfit s ℓ)).symm k).2
· intro hj
refine ⟨(Finset.equivFinOfCardEq (partitionSourceLabels_card n B d hfit s ℓ)) ⟨j,hj⟩, ?_⟩
simp [e]
simpa only [he, partitionRowSigns, e, Function.Embedding.trans_apply,
Equiv.toEmbedding_apply, Function.Embedding.coeFn_mk] using
labeled_walsh_subset_sum_reindex e (fun k => walshCoeff d h k w) (fun j => signOf (z j))
def completedRowWalshSum
The actual completed-row Walsh sum, with the revealed monomial separated and hidden original-label subsets grouped by their selected degree.
Definition (Lean source)
parameters
def
completedRowWalshSum n B d h H s z ℓ w :
ℝ
given by
∑ J ∈ (revealedSources n B d H ℓ).powerset, (∏ j ∈ J, signOf (z j)) * ∑ k ∈ range (capacity n B d H ℓ + 1), walshCoeff d h (J.card + k) w * ∑ E ∈ (partitionHiddenLabels n B d s H ℓ).powersetCard k, ∏ j ∈ E, signOf (z j)
theorem partitionRowSigns_walsh_sum_regrouped
Exact revealed/hidden splitting and degree regrouping for a compatible row. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ E : Finset (Fin d), walshCoeff d h E.card w * ∏ j ∈ E, signOf (partitionRowSigns n B d hfit s.1 z ℓ j))
= completedRowWalshSum n B d h H s.1 z ℓ w
Proof (Lean source)
-- @node: partitionRowSigns_walsh_sum_regrouped
lemma partitionRowSigns_walsh_sum_regrouped (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(s : CompatiblePartition n B d H) (z : Assign (Fin n)) (ℓ : Fin B) (w : ℝ) :
(∑ E : Finset (Fin d), walshCoeff d h E.card w *
∏ j ∈ E, signOf (partitionRowSigns n B d hfit s.1 z ℓ j)) =
completedRowWalshSum n B d h H s.1 z ℓ w := by
rw [partitionRowSigns_walsh_sum_labels,
partitionSourceLabels_eq_revealed_union_hidden n B d H s ℓ,
labeled_walsh_powerset_split _ _ (partition_revealed_hidden_disjoint n B d H s.1 ℓ)]
unfold completedRowWalshSum
apply Finset.sum_congr rfl
intro J hJ
have hJU (E : Finset (Fin n)) (hE : E ∈ (partitionHiddenLabels n B d s.1 H ℓ).powerset) :
Disjoint J E :=
(partition_revealed_hidden_disjoint n B d H s.1 ℓ).mono
(Finset.mem_powerset.mp hJ) (Finset.mem_powerset.mp hE)
calc
_ = (∏ j ∈ J, signOf (z j)) *
∑ E ∈ (partitionHiddenLabels n B d s.1 H ℓ).powerset,
walshCoeff d h (J.card + E.card) w * ∏ j ∈ E, signOf (z j) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro E hE
rw [Finset.card_union_of_disjoint (hJU E hE), Finset.prod_union (hJU E hE)]
ring
_ = _ := by
congr 1
rw [Finset.sum_powerset, partitionHiddenLabels_card n B d hfit s.1 H s.2 ℓ]
apply Finset.sum_congr rfl
intro k hk
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro E hE
rw [(Finset.mem_powersetCard.mp hE).2]
theorem blockDensity_completion_regrouped_walsh_expansion
The posterior average of the actual likelihood, regrouped into revealed monomials and hidden degree selections while preserving original labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s₀ :
CompatiblePartition n B d H
y :
Fin B → ℝ
hy :
∀ ℓ, 0 < refDensity d h (y ℓ)
conclusion
= (card (HiddenCompletion n B d H) : ℝ)⁻¹ * ∑ g : HiddenCompletion n B d H, ∏ ℓ, completedRowWalshSum n B d h H (hiddenCompletionPartition n B d hfit H s₀ g).1 z ℓ (y ℓ)
Proof (Lean source)
-- @node: blockDensity_completion_regrouped_walsh_expansion
lemma blockDensity_completion_regrouped_walsh_expansion (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n))
(s₀ : CompatiblePartition n B d H) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) =
(card (HiddenCompletion n B d H) : ℝ)⁻¹ *
∑ g : HiddenCompletion n B d H,
∏ ℓ, completedRowWalshSum n B d h H
(hiddenCompletionPartition n B d hfit H s₀ g).1 z ℓ (y ℓ) := by
rw [blockDensity_completion_walsh_expansion n B d hd hfit h H z s₀ y hy]
congr 1
apply Finset.sum_congr rfl
intro g hg
apply Finset.prod_congr rfl
intro ℓ hℓ
exact partitionRowSigns_walsh_sum_regrouped n B d hfit h H _ z ℓ (y ℓ)
theorem completedRowWalshSum_product_expansion
Expanding the completed likelihood across rows separates the fixed revealed monomial and coefficient product from the selected hidden-label monomials. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∏ ℓ, completedRowWalshSum n B d h H s z ℓ (y ℓ))
= ∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) * ∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d H ℓ + 1)), (∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) * ∏ ℓ, ∑ E ∈ (partitionHiddenLabels n B d s H ℓ).powersetCard (k ℓ), ∏ j ∈ E, signOf (z j)
Proof (Lean source)
-- @node: completedRowWalshSum_product_expansion
lemma completedRowWalshSum_product_expansion (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (s : SourcePartition B d)
(z : Assign (Fin n)) (y : Fin B → ℝ) :
(∏ ℓ, completedRowWalshSum n B d h H s z ℓ (y ℓ)) =
∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset),
(∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) *
∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d H ℓ + 1)),
(∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) *
∏ ℓ, ∑ E ∈ (partitionHiddenLabels n B d s H ℓ).powersetCard (k ℓ),
∏ j ∈ E, signOf (z j) := by
classical
simp only [completedRowWalshSum]
rw [Finset.prod_univ_sum]
apply Finset.sum_congr rfl
intro J hJ
simp only [Finset.prod_mul_distrib]
congr 1
rw [Finset.prod_univ_sum]
apply Finset.sum_congr rfl
intro k hk
exact Finset.prod_mul_distrib
def completionHiddenSelectionMoment
The exact uniform-completion moment of hidden selected-label monomials. The row degrees are fixed, while their original labels remain jointly allocated.
Definition (Lean source)
parameters
def
completionHiddenSelectionMoment n B d hfit H s₀ z k :
ℝ
given by
(card (HiddenCompletion n B d H) : ℝ)⁻¹ * ∑ g : HiddenCompletion n B d H, ∏ ℓ, ∑ E ∈ (partitionHiddenLabels n B d (hiddenCompletionPartition n B d hfit H s₀ g).1 H ℓ).powersetCard (k ℓ), ∏ j ∈ E, signOf (z j)
theorem completion_walsh_average_selection_expansion
Move the posterior average onto hidden monomials after expanding across rows. All revealed selections and response coefficients are fixed by the full record. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(card (HiddenCompletion n B d H) : ℝ)⁻¹ * (∑ g : HiddenCompletion n B d H, ∏ ℓ, completedRowWalshSum n B d h H (hiddenCompletionPartition n B d hfit H s₀ g).1 z ℓ (y ℓ))
= ∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) * ∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d H ℓ + 1)), (∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) * completionHiddenSelectionMoment n B d hfit H s₀ z k
Proof (Lean source)
-- @node: completion_walsh_average_selection_expansion
lemma completion_walsh_average_selection_expansion (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (H : OffDiag (Fin n) → Bool)
(s₀ : CompatiblePartition n B d H) (z : Assign (Fin n)) (y : Fin B → ℝ) :
(card (HiddenCompletion n B d H) : ℝ)⁻¹ *
(∑ g : HiddenCompletion n B d H,
∏ ℓ, completedRowWalshSum n B d h H
(hiddenCompletionPartition n B d hfit H s₀ g).1 z ℓ (y ℓ)) =
∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset),
(∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) *
∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d H ℓ + 1)),
(∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) *
completionHiddenSelectionMoment n B d hfit H s₀ z k := by
classical
simp_rw [completedRowWalshSum_product_expansion]
rw [Finset.sum_comm]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro J hJ
rw [← Finset.mul_sum, Finset.sum_comm, Finset.mul_sum]
simp only [completionHiddenSelectionMoment, Finset.mul_sum]
apply Finset.sum_congr rfl
intro k hk
apply Finset.sum_congr rfl
intro g hg
ring
theorem blockDensity_completion_selection_expansion
The actual density ratio with its posterior average confined to the hidden selected-label moment; no graph or treatment coordinate has been discarded. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s₀ :
CompatiblePartition n B d H
y :
Fin B → ℝ
hy :
∀ ℓ, 0 < refDensity d h (y ℓ)
conclusion
= ∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) * ∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d H ℓ + 1)), (∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) * completionHiddenSelectionMoment n B d hfit H s₀ z k
Proof (Lean source)
-- @node: blockDensity_completion_selection_expansion
lemma blockDensity_completion_selection_expansion (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n))
(s₀ : CompatiblePartition n B d H) (y : Fin B → ℝ)
(hy : ∀ ℓ, 0 < refDensity d h (y ℓ)) :
blockDensity n B d true h H z y / (∏ ℓ, refDensity d h (y ℓ)) =
∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset),
(∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) *
∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d H ℓ + 1)),
(∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) *
completionHiddenSelectionMoment n B d hfit H s₀ z k := by
rw [blockDensity_completion_regrouped_walsh_expansion n B d hd hfit h H z s₀ y hy,
completion_walsh_average_selection_expansion]
def regroupedCompletionWalshDensity
The actual posterior density after splitting revealed selections and grouping hidden selections by degree, extended by zero on the reference-zero set.
Definition (Lean source)
parameters
def
regroupedCompletionWalshDensity n B d hfit h hz y :
ℝ
given by
if hs : Nonempty (CompatiblePartition n B d hz.1) then if ∀ ℓ, 0 < refDensity d h (y ℓ) then (∏ ℓ, refDensity d h (y ℓ)) * (∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d hz.1 ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (hz.2 j)) * ∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d hz.1 ℓ + 1)), (∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) * completionHiddenSelectionMoment n B d hfit hz.1 hs.some hz.2 k) else 0 else 0
theorem regroupedCompletionWalshDensity_eq_blockDensity
Regrouping gives the actual conditional response density on every compatible retained graph, including zero reference-density endpoints. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: regroupedCompletionWalshDensity_eq_blockDensity
lemma regroupedCompletionWalshDensity_eq_blockDensity (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n))
(hH : ValidRetainedGraph n B d hz.1) (y : Fin B → ℝ) :
regroupedCompletionWalshDensity n B d hfit h hz y =
blockDensity n B d true h hz.1 hz.2 y := by
have hs : Nonempty (CompatiblePartition n B d hz.1) := by
obtain ⟨_, s, hs⟩ := hH
exact ⟨⟨s, hs⟩⟩
rw [regroupedCompletionWalshDensity, dif_pos hs]
split_ifs with hy
· rw [← blockDensity_completion_selection_expansion
n B d hd hfit h hz.1 hz.2 hs.some y hy]
exact mul_div_cancel₀ _ (ne_of_gt (Finset.prod_pos (fun ℓ _ => hy ℓ)))
· obtain ⟨ℓ, hℓ⟩ := not_forall.mp hy
have hz0 : refDensity d h (y ℓ) = 0 :=
le_antisymm (not_lt.mp hℓ) (refDensity_nonneg d h (y ℓ))
exact (blockDensity_zero_of_reference_zero
n B d hd hfit h hz.1 hz.2 hs.some y ℓ hz0).symm
theorem reducedBlockLaw_regrouped_completion_walsh_representation Lemma reducedBlockLaw_regrouped_completion_walsh_representation in the paper ↗
The reduced actual law uses the regrouped density and retains the complete graph together with every assignment coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
conclusion
= (graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (volume.withDensity (fun y => ofReal (regroupedCompletionWalshDensity n B d hfit h hz y))).map (fun y => (hz.1, hz.2, y)))
Proof (Lean source)
lemma reducedBlockLaw_regrouped_completion_walsh_representation (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1) :
reducedBlockLaw n B d (thinnedDesign (Fin n) q) true h =
(graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(volume.withDensity (fun y => ofReal
(regroupedCompletionWalshDensity n B d hfit h hz y))).map
(fun y => (hz.1, hz.2, y))) := by
rw [reducedBlockLaw_completion_walsh_representation n B d h q hn hB hd hfit hh hq]
apply Measure.bind_congr_right
have hv : ∀ᵐ hz ∂(graphAssignMarginal n B d (thinnedDesign (Fin n) q)),
ValidRetainedGraph n B d hz.1 :=
(ae_map_iff (by fun_prop) (by measurability)).mp
(retainedGraphMarginal_valid n B d (thinnedDesign (Fin n) q) hfit)
filter_upwards [hv] with hz hH
simp_rw [completionWalshDensity_eq_blockDensity n B d hd hfit h hz hH,
regroupedCompletionWalshDensity_eq_blockDensity n B d hd hfit h hz hH]
theorem conditionalBlockLaw_regrouped_completion_walsh_representation Lemma conditionalBlockLaw_regrouped_completion_walsh_representation in the paper ↗
Conditioning on the entire detailed graph uses exactly the regrouped posterior likelihood, with no independence substitution for hidden treatments. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
conclusion
= (retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H})⁻¹ • (((graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (volume.withDensity (fun y => ofReal (regroupedCompletionWalshDensity n B d hfit h hz y))).map (fun y => (hz.1, hz.2, y)))).restrict {x | x.1 = H}).map snd
Proof (Lean source)
lemma conditionalBlockLaw_regrouped_completion_walsh_representation (n B d : ℕ) (h q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(H : OffDiag (Fin n) → Bool) :
conditionalBlockLaw n B d (thinnedDesign (Fin n) q) true h H =
(retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H})⁻¹ •
(((graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(volume.withDensity (fun y => ofReal
(regroupedCompletionWalshDensity n B d hfit h hz y))).map
(fun y => (hz.1, hz.2, y)))).restrict {x | x.1 = H}).map snd := by
rw [conditionalBlockLaw,
reducedBlockLaw_regrouped_completion_walsh_representation n B d h q hn hB hd hfit hh hq]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ContractionSeries 8 declarations Geometric and exponential series for contraction
Geometric and exponential series for contraction
The factorial dominates the exponential-normalized power needed for the contraction series. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: factorial_exponential_bound
lemma factorial_exponential_bound (b : ℕ) :
((b : ℝ) ^ b) / (b.factorial : ℝ) ≤ exp (b : ℝ) := by
exact Real.pow_div_factorial_le_exp (b : ℝ) (Nat.cast_nonneg b) b
Counting disjoint active revealed and hidden row sets gives the coefficient bound used before extending the contraction sum to infinite series. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: contraction_active_row_coefficient
lemma contraction_active_row_coefficient (B a b : ℕ) (hB : 0 < B) :
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b ≤
((exp 1 * B) ^ a / (a.factorial : ℝ)) * (4 * exp 1) ^ b := by
have hBr : (0 : ℝ) < B := by exact_mod_cast hB
have hfa : (0 : ℝ) < a.factorial := by exact_mod_cast a.factorial_pos
have hfb : (0 : ℝ) < b.factorial := by exact_mod_cast b.factorial_pos
have hchooseA : (B.choose a : ℝ) ≤ (B : ℝ) ^ a / a.factorial :=
Nat.choose_le_pow_div a B
have hchooseB : ((B - a).choose b : ℝ) ≤ (B : ℝ) ^ b / b.factorial := by
calc
_ ≤ ((B - a : ℕ) : ℝ) ^ b / b.factorial := Nat.choose_le_pow_div b (B - a)
_ ≤ _ := div_le_div_of_nonneg_right
(pow_le_pow_left₀ (by positivity) (by exact_mod_cast sub_le B a) b) hfb.le
have hmin0 : (0 : ℝ) ≤ min 1 (4 * (a + b : ℕ) / (B : ℝ)) := by positivity
calc
_ ≤ ((B : ℝ) ^ a / a.factorial) * ((B : ℝ) ^ b / b.factorial) *
(4 * (a + b : ℕ) / (B : ℝ)) ^ b := by
apply mul_le_mul
· exact mul_le_mul hchooseA hchooseB (by positivity) (by positivity)
· exact pow_le_pow_left₀ hmin0 (min_le_right _ _) b
· positivity
· positivity
_ = ((B : ℝ) ^ a / a.factorial) * 4 ^ b *
((a + b : ℕ) : ℝ) ^ b / b.factorial := by
rw [div_pow, mul_pow]
field_simp
_ ≤ ((B : ℝ) ^ a / a.factorial) * 4 ^ b * exp (a + b : ℕ) := by
rw [mul_div_assoc]
exact mul_le_mul_of_nonneg_left
(Real.pow_div_factorial_le_exp (a + b : ℕ) (by positivity) b) (by positivity)
_ = _ := by
rw [Nat.cast_add, Real.exp_add,
show exp (a : ℝ) = exp 1 ^ a by simp [← Real.exp_nat_mul],
show exp (b : ℝ) = exp 1 ^ b by simp [← Real.exp_nat_mul],
mul_pow, mul_pow]
ring
The nonnegative contraction geometric series sums to the reciprocal gap. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
r :
ℝ
hr :
0 ≤ r
hr1 :
r < 1
conclusion
(∑' k : ℕ, r ^ k) = (1 - r)⁻¹
Proof (Lean source)
-- @node: contraction_geometric_series
lemma contraction_geometric_series (r : ℝ) (hr : 0 ≤ r) (hr1 : r < 1) :
(∑' k : ℕ, r ^ k) = (1 - r)⁻¹ := by
exact tsum_geometric_of_abs_lt_one (by simpa only [abs_of_nonneg hr] using hr1)
The finite active-row contraction sum is dominated by an exponential series in the revealed energy times a geometric series in the hidden energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: contraction_finite_rectangle_bound
lemma contraction_finite_rectangle_bound (B A C : ℕ) (x y : ℝ)
(hB : 0 < B) (hx : 0 ≤ x) (hy : 0 ≤ y)
(hsmall : 4 * exp 1 * y < 1) :
(∑ a ∈ range A, ∑ b ∈ range C,
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b * x ^ a * y ^ b) ≤
exp (exp 1 * B * x) / (1 - 4 * exp 1 * y) := by
have hr0 : 0 ≤ 4 * exp 1 * y := by positivity
have hgeom : (∑ b ∈ range C, (4 * exp 1 * y) ^ b) ≤
(1 - 4 * exp 1 * y)⁻¹ := by
rw [← contraction_geometric_series _ hr0 hsmall]
exact (summable_geometric_of_lt_one hr0 hsmall).sum_le_tsum _
(fun b _ => pow_nonneg hr0 b)
have hexp : (∑ a ∈ range A,
(exp 1 * B * x) ^ a / (a.factorial : ℝ)) ≤
exp (exp 1 * B * x) :=
Real.sum_le_exp_of_nonneg (by positivity) A
calc
_ ≤ ∑ a ∈ range A, ∑ b ∈ range C,
((exp 1 * B * x) ^ a / (a.factorial : ℝ)) *
(4 * exp 1 * y) ^ b := by
apply Finset.sum_le_sum
intro a ha
apply Finset.sum_le_sum
intro b hb
have h := mul_le_mul_of_nonneg_right
(mul_le_mul_of_nonneg_right (contraction_active_row_coefficient B a b hB)
(pow_nonneg hx a)) (pow_nonneg hy b)
convert h using 1
simp only [mul_pow]
ring
_ = (∑ a ∈ range A, (exp 1 * B * x) ^ a / (a.factorial : ℝ)) *
(∑ b ∈ range C, (4 * exp 1 * y) ^ b) := by
simp only [← Finset.mul_sum, ← Finset.sum_mul]
_ ≤ exp (exp 1 * B * x) * (1 - 4 * exp 1 * y)⁻¹ :=
mul_le_mul hexp hgeom (Finset.sum_nonneg (fun b _ => pow_nonneg hr0 b))
(Real.exp_nonneg _)
_ = _ := by rw [div_eq_mul_inv]
Removing the constant Walsh term from the finite active-row sum produces exactly the minus-one bound in the contraction roadmap. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: contraction_nonconstant_sum_bound
lemma contraction_nonconstant_sum_bound (B : ℕ) (x y : ℝ)
(hB : 0 < B) (hx : 0 ≤ x) (hy : 0 ≤ y)
(hsmall : 4 * exp 1 * y < 1) :
(∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a + b = 0 then 0 else
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b * x ^ a * y ^ b) ≤
exp (exp 1 * B * x) / (1 - 4 * exp 1 * y) - 1 := by
let F : ℕ → ℕ → ℝ := fun a b =>
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b * x ^ a * y ^ b
have hzero : F 0 0 = 1 := by simp [F]
have hremove (a b : ℕ) : (if a + b = 0 then 0 else F a b) =
F a b - if a = 0 ∧ b = 0 then 1 else 0 := by
by_cases ha : a = 0
· subst a
by_cases hb : b = 0
· subst b; simp [hzero]
· simp [hb]
· simp [ha]
change (∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a + b = 0 then 0 else F a b) ≤ _
simp_rw [hremove, Finset.sum_sub_distrib]
have hone : (∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a = 0 ∧ b = 0 then (1 : ℝ) else 0) = 1 := by
simp_rw [ite_and]
simp
rw [hone]
exact sub_le_sub_right (contraction_finite_rectangle_bound B (B + 1) (B + 1)
x y hB hx hy hsmall) 1
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.DesignBridge 6 declarations Assignment and audit bridges
Assignment and audit bridges
theorem thinnedDesign_assignment
The canonical product design has the required assignment marginal. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: thinnedDesign_assignment
lemma thinnedDesign_assignment (q : ℝ) (hq : q ∈ Icc 0 1) :
AssignmentLaw (thinnedDesign V q) := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (bernoulliLaw q) := bernoulliLaw_probability q hq
unfold AssignmentLaw thinnedDesign halfBernoulli auditLaw
rw [Measure.map_fst_prod, measure_univ, one_smul]
theorem thinnedDesign_audit
The canonical product design has the required audit marginal. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: thinnedDesign_audit
lemma thinnedDesign_audit (q : ℝ) (hq : q ∈ Icc 0 1) :
AuditLaw (thinnedDesign V q) q := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (bernoulliLaw q) := bernoulliLaw_probability q hq
refine ⟨hq.1, hq.2, ?_⟩
unfold thinnedDesign halfBernoulli auditLaw
rw [Measure.map_snd_prod, measure_univ, one_smul]
theorem thinnedDesign_independent
The two coordinates of the canonical product design are independent. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: thinnedDesign_independent
lemma thinnedDesign_independent (q : ℝ) (hq : q ∈ Icc 0 1) :
DesignIndependent (thinnedDesign V q) := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (bernoulliLaw q) := bernoulliLaw_probability q hq
unfold DesignIndependent thinnedDesign halfBernoulli auditLaw
exact ProbabilityTheory.indepFun_prod measurable_id measurable_id
theorem design_eq_thinnedDesign
The three design atoms determine the canonical assignment-audit product law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: design_eq_thinnedDesign
lemma design_eq_thinnedDesign (D : Measure (Assign V × Audit V)) (q : ℝ)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) :
D = thinnedDesign V q := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
have hm : IsProbabilityMeasure (D.map fst) := by
rw [ha]
unfold halfBernoulli
infer_instance
let : IsProbabilityMeasure D := Measure.isProbabilityMeasure_of_map fst
have heq := hi.map_prod_eq_prod_map_map measurable_fst.aemeasurable
measurable_snd.aemeasurable
change D.map id = (D.map fst).prod (D.map snd) at heq
rw [Measure.map_id, ha, hw.2.2] at heq
exact heq
theorem design_isProbabilityMeasure
The three design atoms imply that the joint design is a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: design_isProbabilityMeasure
lemma design_isProbabilityMeasure (D : Measure (Assign V × Audit V)) (q : ℝ)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) :
IsProbabilityMeasure D := by
rw [design_eq_thinnedDesign D q ha hw hi]
exact thinnedDesign_probabilityDesign q ⟨hw.1, hw.2.1⟩
theorem integral_halfBernoulli_eq_bernoulliDesign_E
Measure-valued assignment expectations equal finite-design Bernoulli expectations. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
f :
Assign V → ℝ
conclusion
∫ z, f z ∂(halfBernoulli V)
= (bernoulliDesign (fun _ : V => (1 / 2 : ℝ)) (fun _ => div_nonneg (show (0 : ℝ) ≤ 1 from zero_le_one) (show (0 : ℝ) ≤ 2 from zero_le_two)) (fun _ => half_le_self (show (0 : ℝ) ≤ 1 from zero_le_one))).E f
Proof (Lean source)
-- @node: integral_halfBernoulli_eq_bernoulliDesign_E
lemma integral_halfBernoulli_eq_bernoulliDesign_E (f : Assign V → ℝ) :
∫ z, f z ∂(halfBernoulli V) =
(bernoulliDesign (fun _ : V => (1 / 2 : ℝ))
(fun _ => div_nonneg (show (0 : ℝ) ≤ 1 from zero_le_one) (show (0 : ℝ) ≤ 2 from
zero_le_two))
(fun _ => half_le_self (show (0 : ℝ) ≤ 1 from zero_le_one))).E f := by
let D := bernoulliDesign
(fun _ : V => (1 / 2 : ℝ))
(fun _ => div_nonneg (show (0 : ℝ) ≤ 1 from zero_le_one)
(show (0 : ℝ) ≤ 2 from zero_le_two))
(fun _ => half_le_self (show (0 : ℝ) ≤ 1 from zero_le_one))
have hcoin : ∀ i : V,
(coinDesign (1 / 2)
(by norm_num) (by norm_num)).toMeasure = bernoulliLaw (1 / 2) := by
intro i
simp [toMeasure,
coinDesign, bernoulliLaw, add_comm]
have hD : D.toMeasure = halfBernoulli V := by
unfold D bernoulliDesign
rw [prodDesign_toMeasure_eq_pi]
exact congrArg Measure.pi (funext hcoin)
rw [← hD]
exact D.integral_toMeasure f
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.DetailedGraphLikelihood 13 declarations Independent audit marks give a product likelihood on every labeled edge.
Likelihood of a complete detailed retained graph
Independent audit marks give a product likelihood on every labeled edge. For compatible graphs this likelihood depends only on the true and retained edge counts, including at audit probabilities zero and one.
A retained coordinate is Bernoulli on a true edge and identically false on a nonedge. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: retained_coordinate_mass
lemma retained_coordinate_mass (q : ℝ) (hq : q ∈ Icc 0 1) (e b : Bool) :
((bernoulliLaw q).map (fun w => e && w)) {b} =
if e then (if b then ofReal q else ofReal (1 - q))
else (if b then 0 else 1) := by
let := bernoulliLaw_probability q hq
cases e
· have hu : ({false, true} : Set Bool) = univ := by
ext b; cases b <;> simp
cases b
· simp only [Bool.false_and, Measure.map_const, Bool.univ_eq, Measure.smul_apply,
MeasurableSpace.measurableSet_top, Measure.dirac_apply', Set.mem_singleton_iff,
Set.indicator_of_mem, Pi.one_apply, smul_eq_mul, mul_one, Bool.false_eq_true,
↓reduceIte]
rw [hu, measure_univ]
· simp [Measure.map_const]
· cases b <;> simp [bernoulliLaw]
The probability of the full retained indicator vector is the product over all labeled coordinates, with no summation or deletion of retained-edge subsets. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: detailed_thinning_mass_product
lemma detailed_thinning_mass_product {I : Type*} [Fintype I]
(q : ℝ) (hq : q ∈ Icc 0 1) (E H : I → Bool) :
((Measure.pi (fun _ : I => bernoulliLaw q)).map
(fun w i => E i && w i)) {H} =
∏ i, if E i then (if H i then ofReal q else ofReal (1 - q))
else (if H i then 0 else 1) := by
let := bernoulliLaw_probability q hq
let _ (i : I) : IsProbabilityMeasure ((bernoulliLaw q).map (fun w => E i && w)) :=
Measure.isProbabilityMeasure_map (measurable_of_finite _).aemeasurable
rw [Measure.pi_map_pi (fun _ => (measurable_of_finite _).aemeasurable), Measure.pi_singleton]
exact Finset.prod_congr rfl (fun i _ => retained_coordinate_mass q hq (E i) (H i))
theorem detailed_thinning_mass_incompatible Lemma detailed_thinning_mass_incompatible in the paper ↗
A graph containing an arrow absent from the true graph has zero probability. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: detailed_thinning_mass_incompatible
lemma detailed_thinning_mass_incompatible {I : Type*} [Fintype I]
(q : ℝ) (hq : q ∈ Icc 0 1) (E H : I → Bool)
(hi : ∃ i, H i = true ∧ E i = false) :
((Measure.pi (fun _ : I => bernoulliLaw q)).map
(fun w i => E i && w i)) {H} = 0 := by
rw [detailed_thinning_mass_product q hq E H]
obtain ⟨i, hH, hE⟩ := hi
apply Finset.prod_eq_zero (Finset.mem_univ i)
simp [hH, hE]
For a compatible detailed graph the likelihood is q to the retained-edge count times one minus q to the number of unretained true edges. This formula includes both endpoints. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: detailed_thinning_mass_compatible
lemma detailed_thinning_mass_compatible {I : Type*} [Fintype I]
(q : ℝ) (hq : q ∈ Icc 0 1) (E H : I → Bool)
(hc : ∀ i, H i = true → E i = true) :
((Measure.pi (fun _ : I => bernoulliLaw q)).map
(fun w i => E i && w i)) {H} =
ofReal q ^ (Finset.univ.filter (fun i => H i = true)).card *
ofReal (1 - q) ^
((Finset.univ.filter (fun i => E i = true)).card -
(Finset.univ.filter (fun i => H i = true)).card) := by
classical
rw [detailed_thinning_mass_product q hq E H]
have hp : (∏ i, if E i then
(if H i then ofReal q else ofReal (1 - q))
else (if H i then 0 else 1)) =
∏ i, if H i = true then ofReal q
else if E i = true then ofReal (1 - q) else 1 := by
apply Finset.prod_congr rfl
intro i _
cases he : E i <;> cases hh : H i <;> simp_all
rw [hp, Finset.prod_ite]
simp only [Finset.prod_const]
rw [Finset.prod_ite]
simp only [Finset.prod_const, one_pow, mul_one]
have hcard : (Finset.univ.filter (fun i => ¬ H i = true)).filter
(fun i => E i = true) =
(Finset.univ.filter (fun i => E i = true)).filter (fun i => ¬ H i = true) := by
ext i
simp only [mem_filter, Finset.mem_univ, true_and]
tauto
rw [hcard]
have heq : (Finset.univ.filter (fun i => E i = true)).filter
(fun i => H i = true) = Finset.univ.filter (fun i => H i = true) := by
ext i
simp only [mem_filter, Finset.mem_univ, true_and]
exact ⟨And.right, fun hh => ⟨hc i hh, hh⟩⟩
have hsum := Finset.card_filter_add_card_filter_not
(s := Finset.univ.filter (fun i => E i = true)) (fun i => H i = true)
rw [heq] at hsum
rw [show ((Finset.univ.filter (fun i => E i = true)).filter
(fun i => ¬ H i = true)).card =
(Finset.univ.filter (fun i => E i = true)).card -
(Finset.univ.filter (fun i => H i = true)).card by omega]
theorem blockEdge_total_card
Every source partition has exactly B*d*d true arrows, counted with their full labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: blockEdge_total_card
lemma blockEdge_total_card (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) :
(Finset.univ.filter (fun e : OffDiag (Fin n) =>
blockEdge n B d s e.val.1 e.val.2)).card = B * d * d := by
classical
have he : (Finset.univ.filter (fun e : OffDiag (Fin n) =>
blockEdge n B d s e.val.1 e.val.2)).card =
(Finset.univ.filter (fun ji : Fin n × Fin n =>
blockEdge n B d s ji.1 ji.2)).card := by
apply Finset.card_bij (fun e _ => e.val)
· intro e he
simpa only [mem_filter, Finset.mem_univ, true_and] using he
· intro e he f hf hef
exact Subtype.ext hef
· intro ji hji
have hj : blockEdge n B d s ji.1 ji.2 := (Finset.mem_filter.mp hji).2
refine ⟨⟨ji, hj.2⟩, ?_, rfl⟩
simpa only [mem_filter, Finset.mem_univ, true_and] using hj
rw [he, Finset.card_filter, Fintype.sum_prod_type]
simp_rw [← Finset.card_filter, blockEdge_outgoing_card n B d hfit s]
rw [Finset.sum_ite]
simp only [Finset.sum_const, nsmul_eq_mul, mul_zero, add_zero,
block_source_card n B d hfit, Nat.cast_id]
theorem block_detailed_graph_mass
Conditional on a true partition, the complete retained-graph probability has the common likelihood q^|H| (1-q)^(B*d*d-|H|) for every compatible completion. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: block_detailed_graph_mass
lemma block_detailed_graph_mass (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hc : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2) :
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H} =
ofReal q ^ (Finset.univ.filter (fun e => H e = true)).card *
ofReal (1 - q) ^
(B * d * d - (Finset.univ.filter (fun e => H e = true)).card) := by
rw [auditLaw, detailed_thinning_mass_compatible q hq
(fun e => decide (blockEdge n B d s e.val.1 e.val.2)) H
(fun e he => by simpa using hc e he)]
simp only [decide_eq_true_eq, blockEdge_total_card n B d hfit s]
theorem block_detailed_graph_mass_eq
Detailed retained subsets have the same likelihood under any two compatible source partitions, even at the audit endpoints. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: block_detailed_graph_mass_eq
lemma block_detailed_graph_mass_eq (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) (s t : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)
(ht : ∀ e, H e = true → blockEdge n B d t e.val.1 e.val.2) :
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H} =
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d t e.val.1 e.val.2) && w e)) {H} := by
rw [block_detailed_graph_mass n B d hfit q hq s H hs,
block_detailed_graph_mass n B d hfit q hq t H ht]
theorem block_detailed_graph_mass_ite
The full fixed-partition graph likelihood vanishes off compatible completions and is constant on them. Compatibility records every observed arrow with both endpoint labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: block_detailed_graph_mass_ite
lemma block_detailed_graph_mass_ite (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool) :
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H} =
if ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2 then
ofReal q ^ (Finset.univ.filter (fun e => H e = true)).card *
ofReal (1 - q) ^
(B * d * d - (Finset.univ.filter (fun e => H e = true)).card)
else 0 := by
classical
by_cases hc : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2
· rw [if_pos hc]
exact block_detailed_graph_mass n B d hfit q hq s H hc
· rw [if_neg hc, auditLaw]
apply detailed_thinning_mass_incompatible q hq
push Not at hc
obtain ⟨e, he, hn⟩ := hc
exact ⟨e, he, by simp [hn]⟩
Integrating any function over the uniform partition prior is its finite arithmetic mean. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
f :
SourcePartition B d → ℝ≥0∞
conclusion
(∫⁻ s, f s ∂partitionLaw B d) = (card (SourcePartition B d) : ℝ≥0∞)⁻¹ * ∑ s, f s
Proof (Lean source)
-- @node: partitionLaw_lintegral
lemma partitionLaw_lintegral (B d : ℕ) (f : SourcePartition B d → ℝ≥0∞) :
(∫⁻ s, f s ∂partitionLaw B d) =
(card (SourcePartition B d) : ℝ≥0∞)⁻¹ * ∑ s, f s := by
rw [partitionLaw, lintegral_smul_measure, lintegral_finsetSum_measure]
simp only [lintegral_dirac, smul_eq_mul]
theorem block_retained_graph_mass
Under the actual audit law, averaging the full detailed graph likelihood over the uniform source partitions gives the compatible-completion count times its common likelihood. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (card (SourcePartition B d) : ℝ≥0∞)⁻¹ * ((Finset.univ.filter (fun s : SourcePartition B d => ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)).card : ℝ≥0∞) * (ofReal q ^ (Finset.univ.filter (fun e => H e = true)).card * ofReal (1 - q) ^ (B * d * d - (Finset.univ.filter (fun e => H e = true)).card))
Proof (Lean source)
-- @node: block_retained_graph_mass
lemma block_retained_graph_mass (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) (H : OffDiag (Fin n) → Bool) :
retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} =
(card (SourcePartition B d) : ℝ≥0∞)⁻¹ *
((Finset.univ.filter (fun s : SourcePartition B d =>
∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)).card : ℝ≥0∞) *
(ofReal q ^ (Finset.univ.filter (fun e => H e = true)).card *
ofReal (1 - q) ^
(B * d * d - (Finset.univ.filter (fun e => H e = true)).card)) := by
classical
rw [retainedGraphMarginal, Measure.map_apply (by fun_prop) (measurableSet_singleton _),
graphAssignMarginal, Measure.bind_apply
((measurableSet_singleton H).preimage (by fun_prop)) (by fun_prop)]
have hfixed (s : SourcePartition B d) :
((thinnedDesign (Fin n) q).map (fun ω =>
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1)))
(fst ⁻¹' {H}) =
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H} := by
rw [← Measure.map_apply (by fun_prop) (measurableSet_singleton H),
Measure.map_map (by fun_prop) (by fun_prop)]
have hm := congrArg (fun μ : Measure (Audit (Fin n)) => μ.map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e))
(thinnedDesign_audit (V := Fin n) q hq).2.2
rw [Measure.map_map (by fun_prop) (by fun_prop)] at hm
exact congrArg (fun μ => μ {H}) hm
simp_rw [hfixed, block_detailed_graph_mass_ite n B d hfit q hq]
rw [partitionLaw_lintegral, Finset.sum_ite]
simp only [Finset.sum_const, nsmul_eq_mul, mul_zero, add_zero, mul_assoc]
Each labeled partition has the same atom mass in the construction prior. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: partitionLaw_singleton
lemma partitionLaw_singleton (B d : ℕ) (s : SourcePartition B d) :
partitionLaw B d {s} = (card (SourcePartition B d) : ℝ≥0∞)⁻¹ := by
classical
simp [partitionLaw, Measure.finsetSum_apply, Measure.dirac_apply' _
(measurableSet_singleton s)]
theorem block_partition_graph_joint_mass
The joint atom for a fixed labeled partition and detailed retained graph is the prior partition mass times its exact full graph likelihood. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: block_partition_graph_joint_mass
lemma block_partition_graph_joint_mass (n B d : ℕ) (q : ℝ) (hq : q ∈ Icc 0 1)
(s : SourcePartition B d) (H : OffDiag (Fin n) → Bool) :
(((partitionLaw B d).prod (auditLaw (Fin n) q)).map (fun x =>
(x.1, fun e => decide (blockEdge n B d x.1 e.val.1 e.val.2) && x.2 e))) {(s, H)} =
(card (SourcePartition B d) : ℝ≥0∞)⁻¹ *
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H} := by
classical
let := bernoulliLaw_probability q hq
let _ : IsProbabilityMeasure (auditLaw (Fin n) q) := by unfold auditLaw; infer_instance
rw [Measure.map_apply (measurable_of_finite _) (measurableSet_singleton _)]
have he : (fun x : SourcePartition B d × Audit (Fin n) =>
(x.1, fun e => decide (blockEdge n B d x.1 e.val.1 e.val.2) && x.2 e)) ⁻¹'
{(s,H)} = {s} ×ˢ ((fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e) ⁻¹' {H}) := by
ext x
simp only [Set.mem_preimage, Set.mem_singleton_iff, Prod.mk.injEq, Set.mem_prod]
constructor
· rintro ⟨hx, hh⟩
exact ⟨hx, by simpa only [hx] using hh⟩
· rintro ⟨hx, hh⟩
exact ⟨hx, by simpa only [hx] using hh⟩
rw [he, Measure.prod_prod, partitionLaw_singleton,
Measure.map_apply (measurable_of_finite _) (measurableSet_singleton _)]
theorem block_partition_posterior_atom
At every positive-probability detailed graph, Bayes' rule assigns each compatible labeled partition exactly the reciprocal of the number of compatible completions. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
((((partitionLaw B d).prod (auditLaw (Fin n) q)).map (fun x => (x.1, fun e => decide (blockEdge n B d x.1 e.val.1 e.val.2) && x.2 e))) {(s, H)}) / retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H}
= ((Finset.univ.filter (fun t : SourcePartition B d => ∀ e, H e = true → blockEdge n B d t e.val.1 e.val.2)).card : ℝ≥0∞)⁻¹
Proof (Lean source)
-- @node: block_partition_posterior_atom
lemma block_partition_posterior_atom (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hpos : retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} ≠ 0)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2) :
((((partitionLaw B d).prod (auditLaw (Fin n) q)).map (fun x =>
(x.1, fun e => decide (blockEdge n B d x.1 e.val.1 e.val.2) && x.2 e))) {(s, H)}) /
retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} =
((Finset.univ.filter (fun t : SourcePartition B d =>
∀ e, H e = true → blockEdge n B d t e.val.1 e.val.2)).card : ℝ≥0∞)⁻¹ := by
classical
let N : ℝ≥0∞ := card (SourcePartition B d)
let C : ℝ≥0∞ := (Finset.univ.filter (fun t : SourcePartition B d =>
∀ e, H e = true → blockEdge n B d t e.val.1 e.val.2)).card
let c : ℝ≥0∞ := ofReal q ^ (Finset.univ.filter (fun e => H e = true)).card *
ofReal (1 - q) ^
(B * d * d - (Finset.univ.filter (fun e => H e = true)).card)
have hg : retainedGraphMarginal n B d (thinnedDesign (Fin n) q) {H} = N⁻¹ * C * c :=
block_retained_graph_mass n B d hfit q hq H
have hc : c ≠ 0 := by
intro hc
apply hpos
rw [hg, hc, mul_zero]
let := sourcePartition_nonempty B d
have hN : N ≠ 0 := by
dsimp [N]
exact_mod_cast Fintype.card_ne_zero
have hf : N⁻¹ * c ≠ 0 := mul_ne_zero (by simp [N]) hc
have hft : N⁻¹ * c ≠ ⊤ := by
have hn : N⁻¹ ≠ ⊤ := ENNReal.inv_ne_top.mpr hN
have hct : c ≠ ⊤ := by dsimp [c]; finiteness
exact ENNReal.mul_ne_top hn hct
rw [block_partition_graph_joint_mass n B d q hq,
block_detailed_graph_mass n B d hfit q hq s H hs, hg]
change (N⁻¹ * c) / (N⁻¹ * C * c) = C⁻¹
rw [show N⁻¹ * C * c = (N⁻¹ * c) * C by ac_rfl]
simpa only [mul_one, one_div] using ENNReal.mul_div_mul_left 1 C hf hft
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ElbowAlgebra 4 declarations Precision-scale endpoint and elbow algebra
Precision-scale endpoint and elbow algebra
The source reveal probability is bounded above by the union bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reveal_probability_le_mul
lemma reveal_probability_le_mul (d : ℕ) (q : ℝ) (hq : q ∈ Icc 0 1) :
1 - (1 - q) ^ d ≤ (d : ℝ) * q := by
induction d with
| zero => simp
| succ d ih =>
have hp := pow_nonneg (sub_nonneg.mpr hq.2) d
have hm := mul_le_mul_of_nonneg_right ih (sub_nonneg.mpr hq.2)
rw [pow_succ, Nat.cast_succ]
nlinarith [mul_nonneg (by positivity : (0 : ℝ) ≤ d) (sq_nonneg q)]
Exponential convexity gives a uniform chord lower bound for reveal probabilities. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reveal_probability_ge_chord
lemma reveal_probability_ge_chord (d : ℕ) (q : ℝ) (hq : q ∈ Icc 0 1) :
(1 - exp (-1)) * min 1 ((d : ℝ) * q) ≤ 1 - (1 - q) ^ d := by
have hbase : 1 - q ≤ exp (-q) := by
linarith [Real.add_one_le_exp (-q)]
have hpow := pow_le_pow_left₀ (sub_nonneg.mpr hq.2) hbase d
rw [← Real.exp_nat_mul] at hpow
have hx : 0 ≤ (d : ℝ) * q := mul_nonneg (Nat.cast_nonneg _) hq.1
by_cases h : (d : ℝ) * q ≤ 1
· have hc := convexOn_exp.2 (Set.mem_univ (0 : ℝ)) (Set.mem_univ (-1 : ℝ))
(sub_nonneg.mpr h) hx (by ring : (1 - (d : ℝ) * q) + (d : ℝ) * q = 1)
simp only [smul_eq_mul, mul_zero, zero_add, mul_neg_one, Real.exp_zero,
mul_one] at hc
rw [min_eq_right h]
have he : (d : ℝ) * -q = -((d : ℝ) * q) := by ring
rw [he] at hpow
nlinarith
· have he : (d : ℝ) * -q ≤ -1 := by nlinarith
have hc := Real.exp_le_exp.mpr he
rw [min_eq_left (le_of_not_ge h)]
linarith
theorem frontier_elbow
Endpoint formulas and the explicit positive-retention elbow comparison. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
hn :
4 ≤ n
hd :
1 ≤ d
hdu :
d ≤ n - 1
conclusion
conclusion 1
frontierScale n d 0 = 1
conclusion 2
frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n)
conclusion 3
q :
ℝ
0 < q
q ≤ 1
(1 / 2 : ℝ) * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q
Proof (Lean source)
-- @node: frontier_elbow
lemma frontier_elbow (n d : ℕ) (hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) :
frontierScale n d 0 = 1 ∧ frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n) ∧
∀ q : ℝ, 0 < q → q ≤ 1 →
(1 / 2 : ℝ) * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q ∧
frontierScale n d q ≤ (1 - exp (-1))⁻¹ * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) := by
have hd0 : d ≠ 0 := by omega
refine ⟨by simp [frontierScale], by simp [frontierScale, hd0], ?_⟩
intro q hq hq1
have hnpos : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hdpos : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
let p := 1 - (1 - q) ^ d
let c := 1 - exp (-1)
have hc : 0 < c := by
dsimp [c]
linarith [Real.exp_lt_one_iff.mpr (by norm_num : (-1 : ℝ) < 0)]
have hc1 : c ≤ 1 := by dsimp [c]; linarith [Real.exp_pos (-1)]
have hp1 : p ≤ 1 := sub_le_self _ (pow_nonneg (by linarith) d)
have hpdq : p ≤ (d : ℝ) * q := reveal_probability_le_mul d q ⟨hq.le, hq1⟩
have hpc : c * min 1 ((d : ℝ) * q) ≤ p :=
reveal_probability_ge_chord d q ⟨hq.le, hq1⟩
have hp : 0 < p := lt_of_lt_of_le (mul_pos hc (lt_min (by norm_num) (mul_pos hdpos hq))) hpc
let x := (d : ℝ) ^ 2 / n
let y := (d : ℝ) / (n * q)
let v := (d : ℝ) ^ 2 / (n * p)
have hx : 0 ≤ x := by dsimp [x]; positivity
have hy : 0 ≤ y := by dsimp [y]; positivity
have hv : 0 ≤ v := by dsimp [v]; positivity
have hxv : x ≤ v := by
dsimp [x, v]
exact div_le_div_of_nonneg_left (sq_nonneg _) (mul_pos hnpos hp)
(by nlinarith)
have hyv : y ≤ v := by
dsimp [y, v]
apply (div_le_div_iff₀ (mul_pos hnpos hq) (mul_pos hnpos hp)).2
nlinarith [mul_le_mul_of_nonneg_left hpdq (mul_pos hnpos hdpos).le]
have hcv : c * v ≤ x + y := by
by_cases ht : (d : ℝ) * q ≤ 1
· rw [min_eq_right ht] at hpc
have hcy : c * v ≤ y := by
dsimp [v, y]
rw [← mul_div_assoc]
apply (div_le_div_iff₀ (mul_pos hnpos hp) (mul_pos hnpos hq)).2
nlinarith [mul_le_mul_of_nonneg_left hpc (mul_pos hnpos hdpos).le]
linarith
· rw [min_eq_left (le_of_not_ge ht)] at hpc
have hcx : c * v ≤ x := by
dsimp [v, x]
rw [← mul_div_assoc]
apply (div_le_div_iff₀ (mul_pos hnpos hp) hnpos).2
nlinarith [mul_le_mul_of_nonneg_left hpc (mul_nonneg hnpos.le (sq_nonneg (d : ℝ)))]
linarith
simp only [frontierScale, if_pos hq]
change (1 / 2 : ℝ) * min 1 (x + y) ≤ min 1 v ∧
min 1 v ≤ c⁻¹ * min 1 (x + y)
constructor
· by_cases ht : 1 ≤ v
· rw [min_eq_left ht]
linarith [min_le_left (1 : ℝ) (x + y)]
· rw [min_eq_right (le_of_not_ge ht)]
linarith [min_le_right (1 : ℝ) (x + y)]
· apply (le_inv_mul_iff₀ hc).2
apply le_min
· exact (mul_le_mul_of_nonneg_right hc1 (le_min (by norm_num) hv)).trans
(by simpa using min_le_left (1 : ℝ) v)
· exact (mul_le_mul_of_nonneg_left (min_le_right 1 v) hc.le).trans hcv
theorem upperEnvelope_le_frontierScale
The observable risk envelope is at most twenty-four times the explicit precision scale after truncation at one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: upperEnvelope_le_frontierScale
lemma upperEnvelope_le_frontierScale (n d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hd : 1 ≤ d) (hq : q ∈ Icc 0 1) :
min 1 (upperEnvelope n d q) ≤ ofReal (24 * frontierScale n d q) := by
by_cases hpos : 0 < q
· have hnpos : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hdpos : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
have hd1 : (1 : ℝ) ≤ d := by exact_mod_cast hd
let p := 1 - (1 - q) ^ d
have hp1 : p ≤ 1 := sub_le_self _ (pow_nonneg (sub_nonneg.mpr hq.2) d)
have hpdq : p ≤ (d : ℝ) * q := reveal_probability_le_mul d q hq
have hc : 0 < 1 - exp (-1) := by
linarith [Real.exp_lt_one_iff.mpr (by norm_num : (-1 : ℝ) < 0)]
have hp : 0 < p := lt_of_lt_of_le
(mul_pos hc (lt_min (by norm_num) (mul_pos hdpos hpos)))
(reveal_probability_ge_chord d q hq)
let v := (d : ℝ) ^ 2 / (n * p)
have hx : (d : ℝ) ^ 2 / n ≤ v := by
apply div_le_div_of_nonneg_left (sq_nonneg _) (mul_pos hnpos hp)
nlinarith
have hy : (d : ℝ) / (n * q) ≤ v := by
apply (div_le_div_iff₀ (mul_pos hnpos hpos) (mul_pos hnpos hp)).2
nlinarith [mul_le_mul_of_nonneg_left hpdq (mul_pos hnpos hdpos).le]
have ha : 5 * ((d : ℝ) + 1) ^ 2 / n ≤ 20 * ((d : ℝ) ^ 2 / n) := by
rw [← mul_div_assoc]
apply div_le_div_of_nonneg_right _ hnpos.le
nlinarith
have hb : 4 * d * (1 - q) / (n * q) ≤ 4 * ((d : ℝ) / (n * q)) := by
rw [← mul_div_assoc]
apply div_le_div_of_nonneg_right _ (mul_pos hnpos hpos).le
nlinarith [mul_nonneg hdpos.le hq.1]
have hu : 5 * ((d : ℝ) + 1) ^ 2 / n +
4 * d * (1 - q) / (n * q) ≤ 24 * v := by
linarith
rw [upperEnvelope, if_pos hpos, frontierScale, if_pos hpos]
change min 1 (ofReal _) ≤ ofReal (24 * min 1 v)
by_cases hv : v ≤ 1
· rw [min_eq_right hv]
exact (min_le_right _ _).trans (ENNReal.ofReal_le_ofReal hu)
· rw [min_eq_left (le_of_not_ge hv)]
exact (min_le_left _ _).trans (by norm_num)
· simp [upperEnvelope, frontierScale, hpos]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.FrontierConsistency 2 declarations The elbow comparison turns vanishing precision scale into the two ratio limits, including the infinite value of the inverse-retention ratio at zero retention.
Sequence limits for the explicit precision scale
The elbow comparison turns vanishing precision scale into the two ratio limits, including the infinite value of the inverse-retention ratio at zero retention.
theorem frontierScale_nonneg
The precision scale is nonnegative on the retention domain. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: frontierScale_nonneg
lemma frontierScale_nonneg (n d : ℕ) (q : ℝ) (hq : q ∈ Icc 0 1) :
0 ≤ frontierScale n d q := by
have hp : 0 ≤ 1 - (1 - q) ^ d := by
exact sub_nonneg.mpr (pow_le_one₀ (sub_nonneg.mpr hq.2) (by linarith [hq.1]))
unfold frontierScale
split_ifs <;> positivity
theorem frontierScale_tendsto_iff
The scale vanishes exactly when both population and audit ratios vanish. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: frontierScale_tendsto_iff
lemma frontierScale_tendsto_iff (dseq : ℕ → ℕ) (qseq : ℕ → ℝ)
(ha : AdmissibleSequences dseq qseq) :
Tendsto (fun n => frontierScale n (dseq n) (qseq n)) atTop (nhds 0) ↔
(Tendsto (fun n => (dseq n : ℝ) ^ 2 / n) atTop (nhds 0) ∧
Tendsto (fun n => if qseq n = 0 then (⊤ : ℝ≥0∞)
else ofReal (dseq n / (n * qseq n))) atTop (nhds 0)) := by
have hadm : ∀ᶠ n in atTop, 4 ≤ n := eventually_ge_atTop 4
constructor
· intro hF
have hsmall := (tendsto_order.mp hF).2 (1 / 2) (by norm_num)
have hpos : ∀ᶠ n in atTop, 0 < qseq n := by
filter_upwards [hsmall] with n hn
by_contra h
norm_num [frontierScale, h] at hn
have hsum : ∀ᶠ n in atTop,
(dseq n : ℝ) ^ 2 / n + dseq n / (n * qseq n) ≤
2 * frontierScale n (dseq n) (qseq n) := by
filter_upwards [hadm, hpos, hsmall] with n hn hq hs
obtain ⟨hd, hdu, hqdom⟩ := ha n hn
have hb := ((frontier_elbow n (dseq n) hn hd hdu).2.2 (qseq n) hq hqdom.2).1
have hx : (dseq n : ℝ) ^ 2 / n + dseq n / (n * qseq n) < 1 := by
by_contra h
rw [min_eq_left (le_of_not_gt h)] at hb
linarith
rw [min_eq_right hx.le] at hb
linarith
have htwice : Tendsto (fun n => 2 * frontierScale n (dseq n) (qseq n))
atTop (nhds 0) := by simpa using hF.const_mul 2
have hxnonneg : ∀ n, 0 ≤ (dseq n : ℝ) ^ 2 / n := by intro n; positivity
have hynonneg : ∀ᶠ n in atTop, 0 ≤ (dseq n : ℝ) / (n * qseq n) := by
filter_upwards [hpos] with n hq
positivity
have hx := squeeze_zero' (Eventually.of_forall hxnonneg) (by
filter_upwards [hsum, hynonneg] with n hs hy
linarith) htwice
have hy : Tendsto (fun n => (dseq n : ℝ) / (n * qseq n)) atTop (nhds 0) :=
squeeze_zero' hynonneg (by
filter_upwards [hsum] with n hs
linarith [hxnonneg n]) htwice
refine ⟨hx, ?_⟩
have hy' := ENNReal.tendsto_ofReal hy
simp only [ENNReal.ofReal_zero] at hy'
apply hy'.congr'
filter_upwards [hpos] with n hq
simp [ne_of_gt hq]
· rintro ⟨hx, hy⟩
have hsmall := (tendsto_order.mp hy).2 1 (by norm_num)
have hpos : ∀ᶠ n in atTop, 0 < qseq n := by
filter_upwards [hadm, hsmall] with n hn hs
have hq := (ha n hn).2.2.1
have hne : qseq n ≠ 0 := by
intro h
simp [h] at hs
exact lt_of_le_of_ne hq (Ne.symm hne)
have hyreal : Tendsto (fun n => (dseq n : ℝ) / (n * qseq n))
atTop (nhds 0) := by
have ht := (ENNReal.tendsto_toReal ENNReal.zero_ne_top).comp hy
simp only [ENNReal.toReal_zero] at ht
apply ht.congr'
filter_upwards [hpos] with n hq
simp [ne_of_gt hq, ENNReal.toReal_ofReal (by positivity :
0 ≤ (dseq n : ℝ) / (n * qseq n))]
have hsum := hx.add hyreal
simp only [zero_add] at hsum
have hbound : ∀ᶠ n in atTop, frontierScale n (dseq n) (qseq n) ≤
(1 - exp (-1))⁻¹ * ((dseq n : ℝ) ^ 2 / n + dseq n / (n * qseq n)) := by
filter_upwards [hadm, hpos] with n hn hq
obtain ⟨hd, hdu, hqdom⟩ := ha n hn
have hb := ((frontier_elbow n (dseq n) hn hd hdu).2.2 (qseq n) hq hqdom.2).2
exact hb.trans (mul_le_mul_of_nonneg_left (min_le_right _ _) (by
have hc : 0 < 1 - exp (-1) := by
linarith [Real.exp_lt_one_iff.mpr (by norm_num : (-1 : ℝ) < 0)]
positivity))
apply squeeze_zero' ?_ hbound
(by simpa using hsum.const_mul (1 - exp (-1))⁻¹)
filter_upwards [hadm] with n hn
exact frontierScale_nonneg n (dseq n) (qseq n) (ha n hn).2.2
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.FrontierLower 4 declarations The supported two-prior reduction and integer block rounding assemble the complete-record testing bound into the explicit minimax scale.
Explicit frontier lower bound
The supported two-prior reduction and integer block rounding assemble the complete-record testing bound into the explicit minimax scale.
theorem block_testing_minimax_lower
At the prescribed testing amplitude, the opposite supported priors give a squared-target minimax lower bound, including estimator randomization. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: block_testing_minimax_lower
lemma block_testing_minimax_lower (n B d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hdu : d ≤ n - 1)
(hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1) :
ofReal (((B * d : ℕ) * ((100 * pi)⁻¹ * testingScale B d q) / n) ^ 2 / 4) ≤
R (Fin n) d q := by
let h := (100 * pi)⁻¹ * testingScale B d q
have hs := testingScale_pos_le_one B d q hB hd
have hk : 0 < (100 * pi)⁻¹ := inv_pos.mpr (by positivity)
have hkquarter : (100 * pi)⁻¹ ≤ (1 / 4 : ℝ) := by
have hp := Real.pi_gt_three
rw [← one_div]
apply (div_le_div_iff₀ (by positivity) (by norm_num : (0 : ℝ) < 4)).2
nlinarith
have hh : h ∈ Icc 0 (1 / 4) :=
⟨(mul_pos hk hs.1).le, (mul_le_of_le_one_right hk.le hs.2).trans hkquarter⟩
let D := thinnedDesign (Fin n) q
let := partitionLaw_probability B d
let := blockBaselineLaw_probability B
let : IsProbabilityMeasure (blockParamLaw B d) := by unfold blockParamLaw; infer_instance
have htv := (block_testing_scale n B d q D hn hB hd hfit hq
(thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)).1 h hh.1 le_rfl
have hp := block_support n B d true h hB hd hfit hh
have hm := block_support n B d false h hB hd hfit hh
simp only [signOf, Bool.false_eq_true, ↓reduceIte, one_mul, neg_one_mul] at hp hm
have hm' : ∀ᵐ ξ ∂blockParamLaw B d, ScheduleClass (blockSchedule n B d false h ξ) d ∧
tte (blockSchedule n B d false h ξ) = -((B * d : ℕ) * h / n) := by
filter_upwards [hm] with ξ hξ
exact ⟨hξ.1, by rw [hξ.2]; ring⟩
have hl := full_record_two_prior_risk n d q hn hd hdu hq
(blockParamLaw B d) (blockParamLaw B d)
(blockSchedule n B d true h) (blockSchedule n B d false h)
(block_record_measurable n B d true h) (block_record_measurable n B d false h)
((B * d : ℕ) * h / n) (by
have hn' : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hm' : (0 : ℝ) < (B * d : ℕ) := by exact_mod_cast (show 0 < B * d by positivity)
exact div_pos (mul_pos hm' (mul_pos hk hs.1)) hn') hp hm'
apply le_trans (ENNReal.ofReal_le_ofReal ?_) hl
change _ ≤ ((B * d : ℕ) * h / n) ^ 2 / 2 * (1 -
tvDist (blockMixtureLawOf n B d D true h) (blockMixtureLawOf n B d D false h))
nlinarith [sq_nonneg ((B * d : ℕ) * h / n)]
theorem frontierScale_le_testingScale_sq
Squaring the testing scale dominates the frontier scale whenever the block sources fit inside the population. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
frontierScale n d q ≤ (testingScale B d q) ^ 2
Proof (Lean source)
-- @node: frontierScale_le_testingScale_sq
lemma frontierScale_le_testingScale_sq (n B d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hmn : B * d ≤ n)
(hq : q ∈ Icc 0 1) :
frontierScale n d q ≤ (testingScale B d q) ^ 2 := by
have hn' : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hm' : (0 : ℝ) < (B * d : ℕ) := by exact_mod_cast (show 0 < B * d by positivity)
have hd' : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
by_cases hqpos : 0 < q
· have hp : 0 < retentionP d q := by
unfold retentionP
exact sub_pos.mpr (pow_lt_one₀ (sub_nonneg.mpr hq.2) (by linarith) (by omega))
have hmp : 0 < (B * d : ℕ) * retentionP d q := mul_pos hm' hp
have hnp : 0 < (n : ℝ) * retentionP d q := mul_pos hn' hp
have hsqrt : 0 < sqrt ((B * d : ℕ) * retentionP d q) := Real.sqrt_pos.2 hmp
have hsq : ((d : ℝ) / sqrt ((B * d : ℕ) * retentionP d q)) ^ 2 =
(d : ℝ) ^ 2 / ((B * d : ℕ) * retentionP d q) := by
rw [div_pow, Real.sq_sqrt hmp.le]
have hv : (d : ℝ) ^ 2 / ((n : ℝ) * retentionP d q) ≤
(d : ℝ) ^ 2 / ((B * d : ℕ) * retentionP d q) := by
apply div_le_div_of_nonneg_left (sq_nonneg _) hmp
exact mul_le_mul_of_nonneg_right (by exact_mod_cast hmn) hp.le
rw [frontierScale, if_pos hqpos, testingScale, if_pos hp]
change min 1 ((d : ℝ) ^ 2 / ((n : ℝ) * retentionP d q)) ≤ _
by_cases ht : (d : ℝ) / sqrt ((B * d : ℕ) * retentionP d q) ≤ 1
· rw [min_eq_right ht, hsq]
exact (min_le_right _ _).trans hv
· rw [min_eq_left (le_of_not_ge ht), one_pow]
exact min_le_left _ _
· have hzero : q = 0 := by linarith [hq.1]
subst q
simp [frontierScale, testingScale, retentionP]
Floor rounding supplies between a quarter and a half population of sources in the small-degree regime. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
hd :
1 ≤ d
hsmall :
4 * d < n
conclusion
conclusion 1
1 ≤ n / (2 * d)
conclusion 2
2 * (n / (2 * d) * d) ≤ n
conclusion 3
(n : ℝ) / 4 ≤ (n / (2 * d) * d : ℕ)
Proof (Lean source)
-- @node: frontier_block_rounding
lemma frontier_block_rounding (n d : ℕ) (hd : 1 ≤ d) (hsmall : 4 * d < n) :
1 ≤ n / (2 * d) ∧ 2 * (n / (2 * d) * d) ≤ n ∧
(n : ℝ) / 4 ≤ (n / (2 * d) * d : ℕ) := by
have hd0 : 0 < 2 * d := by omega
have hmul := Nat.div_mul_le_self n (2 * d)
have hmod := Nat.mod_lt n hd0
have heq := Nat.mod_add_div n (2 * d)
have hfit : 2 * (n / (2 * d) * d) ≤ n := by nlinarith
have hl : n < 2 * (n / (2 * d) * d) + 2 * d := by nlinarith
have hquarter : n ≤ 4 * (n / (2 * d) * d) := by nlinarith
refine ⟨?_, hfit, ?_⟩
· exact (Nat.le_div_iff_mul_le hd0).2 (by omega)
· have hc : (n : ℝ) ≤ 4 * (n / (2 * d) * d : ℕ) := by exact_mod_cast hquarter
linarith
theorem frontier_minimax_lower
The explicit frontier minimax lower constant follows from supplied blocks at large degree and rounded hidden-partition blocks at small degree. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: frontier_minimax_lower
lemma frontier_minimax_lower (n d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) (hq : q ∈ Icc 0 1) :
ofReal (frontierScale n d q / (2560000 * pi ^ 2)) ≤ R (Fin n) d q := by
have hn' : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hd' : (1 : ℝ) ≤ d := by exact_mod_cast hd
have hc : 0 < 160000 * pi ^ 2 := by positivity
have hc' : 0 < 2560000 * pi ^ 2 := by positivity
have hF : frontierScale n d q ≤ 1 := by
unfold frontierScale
split_ifs <;> first | exact min_le_left _ _ | exact le_rfl
by_cases hlarge : (1 / 16 : ℝ) ≤ (d : ℝ) ^ 2 / n
· have hl := supplied_block_record_lower n d q (thinnedDesign (Fin n) q)
hn hd hdu hq (thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)
apply le_trans (ENNReal.ofReal_le_ofReal ?_) hl
have hdeg : (1 / 16 : ℝ) ≤ ((d : ℝ) + 1) ^ 2 / n := by
apply hlarge.trans
exact div_le_div_of_nonneg_right (by nlinarith) hn'.le
have hmin : (1 / 16 : ℝ) ≤ min 1 (((d : ℝ) + 1) ^ 2 / n) := le_min (by norm_num) hdeg
rw [← div_eq_inv_mul]
apply (div_le_div_iff₀ hc' hc).2
have heq : 2560000 * pi ^ 2 = 16 * (160000 * pi ^ 2) := by ring
rw [heq]
nlinarith [mul_le_mul_of_nonneg_right hmin hc.le]
· have hsmall : 4 * d < n := by
have hs : (d : ℝ) ^ 2 < (n : ℝ) / 16 := by
have := (div_lt_iff₀ hn').mp (lt_of_not_ge hlarge)
linarith
have hreal : 4 * (d : ℝ) < n := by nlinarith
exact_mod_cast hreal
obtain ⟨hB, hfit, hquarter⟩ := frontier_block_rounding n d hd hsmall
let B := n / (2 * d)
let m : ℝ := (B * d : ℕ)
let s := testingScale B d q
let k := (100 * pi)⁻¹
have hs := testingScale_pos_le_one B d q hB hd
have hk : 0 < k := by dsimp [k]; positivity
have hspos : 0 < s := hs.1
have hmpos : 0 < m := by dsimp [m]; exact_mod_cast (show 0 < B * d by positivity)
have hmn : B * d ≤ n := by dsimp [B]; omega
have hFs : frontierScale n d q ≤ s ^ 2 :=
frontierScale_le_testingScale_sq n B d q hn hB hd hmn hq
have hl := block_testing_minimax_lower n B d q hn hB hd hdu hfit hq
apply le_trans (ENNReal.ofReal_le_ofReal ?_) hl
change frontierScale n d q / (2560000 * pi ^ 2) ≤ (m * (k * s) / n) ^ 2 / 4
have hratio : (1 / 4 : ℝ) ≤ m / n := (le_div_iff₀ hn').2 (by linarith)
have htarget : k * s / 4 ≤ m * (k * s) / n := by
calc
k * s / 4 = (1 / 4 : ℝ) * (k * s) := by ring
_ ≤ (m / n) * (k * s) := mul_le_mul_of_nonneg_right hratio (mul_pos hk hs.1).le
_ = m * (k * s) / n := by ring
have hsquared : (k * s / 4) ^ 2 ≤ (m * (k * s) / n) ^ 2 :=
(sq_le_sq₀ (by positivity) (by positivity)).2 htarget
have hidentity : (k * s / 4) ^ 2 / 4 = s ^ 2 / (640000 * pi ^ 2) := by
dsimp [k]
field_simp
<;> ring
calc
frontierScale n d q / (2560000 * pi ^ 2) ≤
s ^ 2 / (2560000 * pi ^ 2) := div_le_div_of_nonneg_right hFs hc'.le
_ ≤ s ^ 2 / (640000 * pi ^ 2) :=
div_le_div_of_nonneg_left (sq_nonneg s) (by positivity) (by nlinarith [sq_pos_of_pos Real.pi_pos])
_ = (k * s / 4) ^ 2 / 4 := hidentity.symm
_ ≤ (m * (k * s) / n) ^ 2 / 4 := div_le_div_of_nonneg_right hsquared (by norm_num)
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.GeniePartition 2 declarations Genie-partition translation bound
Genie-partition translation bound
theorem block_tvDist_bind_uniform_le
Mixing probability laws over a common finite marginal preserves a uniform TV bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
α β :
μ :
P Q :
α → Measure β
hP :
∀ a, IsProbabilityMeasure (P a)
hQ :
∀ a, IsProbabilityMeasure (Q a)
hPm :
hQm :
c :
ℝ
hc :
∀ a, tvDist (P a) (Q a) ≤ c
conclusion
tvDist (μ.bind P) (μ.bind Q) ≤ c
Proof (Lean source)
-- @node: block_tvDist_bind_uniform_le
lemma block_tvDist_bind_uniform_le {α β : Type*} [Finite α] [MeasurableSpace α]
[MeasurableSingletonClass α]
[MeasurableSpace β] (μ : Measure α) [IsProbabilityMeasure μ]
(P Q : α → Measure β) (hP : ∀ a, IsProbabilityMeasure (P a))
(hQ : ∀ a, IsProbabilityMeasure (Q a)) (hPm : Measurable P) (hQm : Measurable Q)
(c : ℝ) (hc : ∀ a, tvDist (P a) (Q a) ≤ c) :
tvDist (μ.bind P) (μ.bind Q) ≤ c := by
let _ := hP
let _ := hQ
unfold tvDist
apply ciSup_le
rintro ⟨E, hE⟩
have hreal (K : α → Measure β) (hK : ∀ a, IsProbabilityMeasure (K a))
(hKm : Measurable K) : (μ.bind K).real E = ∫ a, (K a).real E ∂μ := by
let _ := hK
rw [measureReal_def, Measure.bind_apply hE hKm.aemeasurable]
exact (integral_toReal ((Measure.measurable_coe hE).comp hKm).aemeasurable
(Filter.Eventually.of_forall (fun a => measure_lt_top (K a) E))).symm
rw [hreal P hP hPm, hreal Q hQ hQm,
← integral_sub Integrable.of_finite Integrable.of_finite]
calc
_ ≤ ∫ a, |(P a).real E - (Q a).real E| ∂μ := abs_integral_le_integral_abs
_ ≤ ∫ _ : α, c ∂μ := integral_mono Integrable.of_finite (integrable_const c)
(fun a => (abs_measureReal_sub_le_tvDist hE).trans (hc a))
_ = c := by simp
theorem genie_partition_tv_bound
Giving the partition to the observer yields the product-translation upper bound on the original-record distance. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: genie_partition_tv_bound
lemma genie_partition_tv_bound (n B d : ℕ) (h q : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (hB : 1 ≤ B) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) :
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤ 2 * pi * h * sqrt ((B : ℝ) / d) := by
let := design_isProbabilityMeasure D q ha hw hi
let := partitionLaw_probability B d
let := blockBaselineLaw_probability B
have hp (σ : Bool) (s : SourcePartition B d) : IsProbabilityMeasure
(mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d σ h (s, U))) :=
mixtureLaw_probability D _ _ ((block_record_measurable n B d σ h).comp
((measurable_const.prodMk measurable_fst).prodMk measurable_snd))
rw [blockMixtureLawOf_bind_partition, blockMixtureLawOf_bind_partition]
exact block_tvDist_bind_uniform_le _ _ _ (hp true) (hp false)
(measurable_of_finite _) (measurable_of_finite _) _
(fun s => partition_mixture_tv_le n B d hd hfit h hh.1 s D ha)
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.GoodEvent 6 declarations Good-event probability for hidden source counts
Good-event probability for hidden source counts
Unrevealed source labels, counted in the source index type, are bounded by total capacity. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(Finset.univ.filter (fun j => sourceReveals n B d H j = false)).card
≤ undiscovered n B d H
Proof (Lean source)
-- @node: unrevealed_count_le_capacity
lemma unrevealed_count_le_capacity (n B d : ℕ) (H : OffDiag (Fin n) → Bool)
(hH : ValidRetainedGraph n B d H) :
(Finset.univ.filter (fun j => sourceReveals n B d H j = false)).card ≤
undiscovered n B d H := by
have hmn : B * d ≤ n := by have := hH.1; omega
let emb : Fin (B * d) → Fin n := fun j => ⟨j.val, lt_of_lt_of_le j.isLt hmn⟩
have hc : (Finset.univ.filter (fun j => sourceReveals n B d H j = false)).card =
hiddenTreated n B d H (fun _ => true) := by
unfold hiddenTreated
apply Finset.card_bij (fun j _ => emb j)
· intro j hj
have hnoreveal : ¬ ∃ i : Fin n, ∃ hji : emb j ≠ i, H ⟨(emb j,i),hji⟩ = true := by
intro he
have hf := (Finset.mem_filter.mp hj).2
have ht : sourceReveals n B d H j = true := by
simp only [sourceReveals, decide_eq_true_eq]
exact ⟨emb j, rfl, he⟩
rw [ht] at hf
contradiction
exact Finset.mem_filter.mpr ⟨Finset.mem_univ _, j.isLt, rfl, hnoreveal⟩
· intro a ha b hb hab
exact Fin.ext (congrArg (fun j : Fin n => j.val) hab)
· intro j hj
obtain ⟨hjlt, _, hjnone⟩ := (Finset.mem_filter.mp hj).2
refine ⟨⟨j.val, hjlt⟩, ?_, Fin.ext rfl⟩
apply Finset.mem_filter.mpr
refine ⟨Finset.mem_univ _, ?_⟩
simp only [sourceReveals, decide_eq_false_iff_not]
rintro ⟨v, hv, i, hvi, he⟩
have hvj : v = j := Fin.ext hv
subst v
exact hjnone ⟨i, hvi, he⟩
rw [hc]
exact hiddenTreated_le n B d H hH (fun _ => true)
The inverse-third moment of the hidden count factors over independent source reveals. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: unrevealed_inverse_third_moment
lemma unrevealed_inverse_third_moment (m : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r : Fin m → Bool,
(1 / 3 : ℝ) ^ (Finset.univ.filter (fun j => r j = false)).card
∂(Measure.pi (fun _ : Fin m => bernoulliLaw p))) =
(p + (1 - p) / 3) ^ m := by
let := bernoulliLaw_probability p hp
have hprod (r : Fin m → Bool) :
(1 / 3 : ℝ) ^ (Finset.univ.filter (fun j => r j = false)).card =
∏ j, (if r j = false then (1 / 3 : ℝ) else 1) := by
rw [← Finset.prod_const]
rw [Finset.prod_filter]
simp_rw [hprod]
rw [integral_fintype_prod_eq_prod (fun (_ : Fin m) (b : Bool) =>
if b = false then (1 / 3 : ℝ) else 1)]
simp only [Finset.prod_const, Finset.card_univ, Fintype.card_fin]
congr 1
rw [bernoulliLaw, integral_add_measure, integral_smul_measure, integral_smul_measure]
· simp [ENNReal.toReal_ofReal hp.1, ENNReal.toReal_ofReal (sub_nonneg.mpr hp.2)]
ring
· exact (Integrable.of_finite (μ := Measure.dirac false)).smul_measure (by finiteness)
· exact (Integrable.of_finite (μ := Measure.dirac true)).smul_measure (by finiteness)
The fourth-power form avoids numerical approximations in the binomial tail constant. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
m :
ℕ
hm :
32 ≤ m
conclusion
(2 / 3 : ℝ) ^ m / (1 / 3 : ℝ) ^ ((m : ℝ) / 4) ≤ 1 / 16
Proof (Lean source)
-- @node: inverse_third_tail_constant
lemma inverse_third_tail_constant (m : ℕ) (hm : 32 ≤ m) :
(2 / 3 : ℝ) ^ m / (1 / 3 : ℝ) ^ ((m : ℝ) / 4) ≤ 1 / 16 := by
have ht : 0 < (1 / 3 : ℝ) ^ ((m : ℝ) / 4) := Real.rpow_pos_of_pos (by norm_num) _
have hfour : ((2 / 3 : ℝ) ^ m / (1 / 3 : ℝ) ^ ((m : ℝ) / 4)) ^ 4 =
(16 / 27 : ℝ) ^ m := by
rw [div_pow, ← Real.rpow_mul_natCast (by norm_num)]
have he : (m : ℝ) / 4 * (4 : ℕ) = m := by push_cast; ring
rw [he, Real.rpow_natCast, ← pow_mul, mul_comm m 4, pow_mul, ← div_pow]
norm_num
have hb : (16 / 27 : ℝ) ^ m ≤ (16 / 27 : ℝ) ^ 32 :=
pow_le_pow_of_le_one (by norm_num) (by norm_num) hm
have hbound : (16 / 27 : ℝ) ^ 32 ≤ (1 / 16 : ℝ) ^ 4 := by norm_num
have hpow := hfour ▸ hb.trans hbound
exact le_of_pow_le_pow_left₀ (by norm_num : (4 : ℕ) ≠ 0)
(by norm_num : (0 : ℝ) ≤ 1 / 16) hpow
Markov's inequality applied to the inverse-third count moment gives the required tail. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: unrevealed_count_small_probability
lemma unrevealed_count_small_probability (m : ℕ) (p : ℝ)
(hm : 32 ≤ m) (hp : p ∈ Icc 0 1) (hphalf : p ≤ 1 / 2) :
(Measure.pi (fun _ : Fin m => bernoulliLaw p)).real
{r | ((Finset.univ.filter (fun j => r j = false)).card : ℝ) < (m : ℝ) / 4} ≤
1 / 16 := by
let := bernoulliLaw_probability p hp
let μ := Measure.pi (fun _ : Fin m => bernoulliLaw p)
let t : ℝ := (1 / 3 : ℝ) ^ ((m : ℝ) / 4)
let f : (Fin m → Bool) → ℝ := fun r =>
(1 / 3 : ℝ) ^ (Finset.univ.filter (fun j => r j = false)).card
have ht : 0 < t := Real.rpow_pos_of_pos (by norm_num) _
have hmarkov := mul_meas_ge_le_integral_of_nonneg (μ := μ) (f := f)
(Filter.Eventually.of_forall (fun r => by dsimp [f]; positivity))
(show Integrable f μ from Integrable.of_finite) t
have hsub : {r : Fin m → Bool |
((Finset.univ.filter (fun j => r j = false)).card : ℝ) < (m : ℝ) / 4} ⊆
{r | t ≤ f r} := by
intro r hr
dsimp [t, f]
rw [← Real.rpow_natCast]
exact Real.rpow_le_rpow_of_exponent_ge (by norm_num) (by norm_num) hr.le
have htail := measureReal_mono (μ := μ) hsub
have hmean : (∫ r, f r ∂μ) ≤ (2 / 3 : ℝ) ^ m := by
rw [unrevealed_inverse_third_moment m p hp]
apply pow_le_pow_left₀
· have := hp.1; linarith
· linarith
have hdiv : μ.real {r | t ≤ f r} ≤ (2 / 3 : ℝ) ^ m / t := by
apply (le_div_iff₀ ht).mpr
simpa only [mul_comm] using hmarkov.trans hmean
exact htail.trans (hdiv.trans (inverse_third_tail_constant m hm))
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.GroupedConditionalLikelihood 8 declarations Regroup the exact labeled posterior expansion by total hidden degree.
Total-hidden-degree coefficients of the actual conditional likelihood
Regroup the exact labeled posterior expansion by total hidden degree. The coefficient fibers keep all row capacities and binomial slot multiplicities.
The response coefficient for a revealed-subset family and a total hidden degree, with the exact capacity-constrained binomial multiplicities.
Definition (Lean source)
parameters
def
groupedHiddenWalshCoeff n B d h H J t y :
ℝ
theorem groupedHiddenWalshCoeff_eq_bounded
The same actual coefficient indexed by bounded row degrees, exactly matching the allocation-energy lemmas' finite domain. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
groupedHiddenWalshCoeff n B d h H J t y
Proof (Lean source)
-- @node: groupedHiddenWalshCoeff_eq_bounded
lemma groupedHiddenWalshCoeff_eq_bounded (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (J : Fin B → Finset (Fin n))
(t : ℕ) (y : Fin B → ℝ) :
groupedHiddenWalshCoeff n B d h H J t y =
∑ k ∈ Finset.univ.filter
(fun k : ∀ ℓ : Fin B, Fin (capacity n B d H ℓ + 1) =>
t = ∑ ℓ, (k ℓ).val),
(∏ ℓ, (choose (capacity n B d H ℓ) (k ℓ).val : ℝ)) *
∏ ℓ, walshCoeff d h ((J ℓ).card + (k ℓ).val) (y ℓ) := by
unfold groupedHiddenWalshCoeff
symm
apply Finset.sum_bij (fun k _ ℓ => (k ℓ).val)
· intro k hk
refine Finset.mem_filter.mpr ⟨?_, (Finset.mem_filter.mp hk).2⟩
exact Fintype.mem_piFinset.mpr (fun ℓ => Finset.mem_range.mpr (k ℓ).isLt)
· intro a ha b hb hab
funext ℓ
exact Fin.ext (congrFun hab ℓ)
· intro k hk
have hbound (ℓ : Fin B) : k ℓ < capacity n B d H ℓ + 1 :=
Finset.mem_range.mp (Fintype.mem_piFinset.mp (Finset.mem_filter.mp hk).1 ℓ)
refine ⟨fun ℓ => ⟨k ℓ, hbound ℓ⟩, ?_, rfl⟩
exact Finset.mem_filter.mpr ⟨Finset.mem_univ _, (Finset.mem_filter.mp hk).2⟩
· intro k hk
rfl
def groupedHiddenWalshDensity
The grouped likelihood density keeps the actual labels and the reference-zero extension while making the hidden-degree coefficient fibers explicit.
Definition (Lean source)
parameters
def
groupedHiddenWalshDensity n B d hfit h hz y :
ℝ
given by
if hs : Nonempty (CompatiblePartition n B d hz.1) then if ∀ ℓ, 0 < refDensity d h (y ℓ) then (∏ ℓ, refDensity d h (y ℓ)) * (∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d hz.1 ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (hz.2 j)) * ∑ t ∈ range (undiscovered n B d hz.1 + 1), symmetricSignPoly (undiscovered n B d hz.1) t (fun i => hz.2 (hiddenSourceLabelEmbedding n B d hfit hz.1 ((hiddenSourceEnumeration n B d hfit hz.1 hs.some).symm i))) * groupedHiddenWalshCoeff n B d h hz.1 J t y) else 0 else 0
theorem symmetricHiddenWalshDensity_eq_grouped
Regrouping preserves the entire posterior density, including incompatible graphs and zero reference-density endpoints. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
symmetricHiddenWalshDensity n B d hfit h hz y
= groupedHiddenWalshDensity n B d hfit h hz y
Proof (Lean source)
-- @node: symmetricHiddenWalshDensity_eq_grouped
lemma symmetricHiddenWalshDensity_eq_grouped (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n)) (y : Fin B → ℝ) :
symmetricHiddenWalshDensity n B d hfit h hz y =
groupedHiddenWalshDensity n B d hfit h hz y := by
by_cases hs : Nonempty (CompatiblePartition n B d hz.1)
· have hH : ValidRetainedGraph n B d hz.1 := ⟨hfit, hs.some.1, hs.some.2⟩
rw [symmetricHiddenWalshDensity_eq_blockDensity n B d hd hfit h hz hH,
groupedHiddenWalshDensity, dif_pos hs]
split_ifs with hy
· rw [← blockDensity_grouped_hidden_walsh_expansion
n B d hd hfit h hz.1 hz.2 hs.some y hy]
exact (mul_div_cancel₀ _ (ne_of_gt (Finset.prod_pos (fun ℓ _ => hy ℓ)))).symm
· obtain ⟨ℓ, hℓ⟩ := not_forall.mp hy
have hz0 : refDensity d h (y ℓ) = 0 :=
le_antisymm (not_lt.mp hℓ) (refDensity_nonneg d h (y ℓ))
exact blockDensity_zero_of_reference_zero
n B d hd hfit h hz.1 hz.2 hs.some y ℓ hz0
· simp only [symmetricHiddenWalshDensity, groupedHiddenWalshDensity, dif_neg hs]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.HiddenAllocationContraction 1 declarations Hidden-allocation chi-squared contraction
Hidden-allocation chi-squared contraction
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.HiddenAllocationEnergy 12 declarations Finite weighted Cauchy–Schwarz and the exact allocation weight count bound each active-row coefficient's squared norm.
Weighted energy of constrained hidden allocations
Finite weighted Cauchy–Schwarz and the exact allocation weight count bound each active-row coefficient's squared norm. The good-event binomial ratio then gives one contraction factor per hidden active row. These bounds do not assume the unfinished identification of the conditional likelihood.
The squared Walsh coefficient is integrable under the normalized row reference; boundedness supplies this regularity without an added premise. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
s :
ℕ
conclusion
Integrable (fun w => walshCoeff d h s w ^ 2) (volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: walshCoeff_sq_reference_integrable
lemma walshCoeff_sq_reference_integrable (d : ℕ) (h : ℝ) (s : ℕ) :
Integrable (fun w => walshCoeff d h s w ^ 2)
(volume.withDensity (fun w => ofReal (refDensity d h w))) := by
let := referenceRowLaw_probability d h
apply Integrable.of_bound (by fun_prop) 1
filter_upwards [] with w
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
exact (sq_le_one_iff_abs_le_one _).mpr (walshCoeff_abs_le_one d h w s)
The paper's row energy is exactly the coefficient's squared norm under the row reference law, including reference-density zeros. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: walshCoeff_sq_reference_integral
lemma walshCoeff_sq_reference_integral (d : ℕ) (h : ℝ) (s : ℕ) :
(∫ w, walshCoeff d h s w ^ 2
∂volume.withDensity (fun w => ofReal (refDensity d h w))) = gamma d h s := by
rw [integral_withDensity_eq_integral_toReal_smul (by fun_prop)
(Filter.Eventually.of_forall (fun _ => ENNReal.ofReal_lt_top))]
simp only [ENNReal.toReal_ofReal (refDensity_nonneg d h _), smul_eq_mul]
simp_rw [mul_comm (refDensity d h _)]
rfl
theorem walshCoeff_product_reference_energy Lemma walshCoeff_product_reference_energy in the paper ↗
Products of row coefficients have finite squared norm, and independence factors that norm into the product of the row energies. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
Integrable (fun y : Fin B → ℝ => (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) ^ 2) (Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
conclusion 2
(∫ y : Fin B → ℝ, (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) ^ 2 ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= ∏ ℓ, gamma d h (s ℓ)
Proof (Lean source)
-- @node: walshCoeff_product_reference_energy
lemma walshCoeff_product_reference_energy (B d : ℕ) (h : ℝ) (s : Fin B → ℕ) :
Integrable (fun y : Fin B → ℝ => (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) ^ 2)
(Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) ∧
(∫ y : Fin B → ℝ, (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) ^ 2
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) = ∏ ℓ, gamma d h (s ℓ) := by
let := referenceRowLaw_probability d h
have he (y : Fin B → ℝ) : (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) ^ 2 =
∏ ℓ, walshCoeff d h (s ℓ) (y ℓ) ^ 2 := (Finset.prod_pow _ _ _).symm
simp_rw [he]
constructor
· exact Integrable.fintype_prod (fun ℓ => walshCoeff_sq_reference_integrable d h (s ℓ))
· rw [integral_fintype_prod_eq_prod (fun ℓ w => walshCoeff d h (s ℓ) w ^ 2)]
simp_rw [walshCoeff_sq_reference_integral]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.HiddenAllocationOrthogonality 11 declarations Different sets of active response rows are orthogonal under the independent row reference.
Orthogonality of hidden-allocation response supports
Different sets of active response rows are orthogonal under the independent row reference. Positive degrees within an active row are not assumed orthogonal. The finite weighted and Pythagorean identities assemble the support-group step of the hidden-allocation contraction roadmap.
The constant row coefficient is one almost everywhere under the reference law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
conclusion
walshCoeff d h 0
=ᵐ[volume.withDensity (fun w => ofReal (refDensity d h w))] (fun _ => 1)
Proof (Lean source)
-- @node: walshCoeff_zero_reference_ae
lemma walshCoeff_zero_reference_ae (d : ℕ) (h : ℝ) :
walshCoeff d h 0 =ᵐ[volume.withDensity
(fun w => ofReal (refDensity d h w))] (fun _ => 1) := by
change ∀ᵐ w ∂volume.withDensity (fun w => ofReal (refDensity d h w)),
walshCoeff d h 0 w = 1
rw [ae_withDensity_iff (by fun_prop)]
filter_upwards [] with w hw
exact walshCoeff_zero d h w (lt_of_le_of_ne (refDensity_nonneg d h w)
(by intro hz; apply hw; rw [← hz]; simp))
The nonconstant row coefficients have zero mean under the actual reference law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
hd :
1 ≤ d
s :
ℕ
hs :
1 ≤ s
conclusion
= 0
Proof (Lean source)
-- @node: walshCoeff_reference_centered
lemma walshCoeff_reference_centered (d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(s : ℕ) (hs : 1 ≤ s) :
(∫ w, walshCoeff d h s w ∂volume.withDensity
(fun w => ofReal (refDensity d h w))) = 0 := by
rw [integral_withDensity_eq_integral_toReal_smul (by fun_prop)
(Filter.Eventually.of_forall (fun _ => ENNReal.ofReal_lt_top))]
simp only [ENNReal.toReal_ofReal (refDensity_nonneg d h _), smul_eq_mul]
simpa only [mul_comm] using walshCoeff_centered d h hd s hs
A nonconstant coefficient is orthogonal to the constant coefficient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
hd :
1 ≤ d
s t :
ℕ
hst :
(1 ≤ s ∧ t = 0) ∨ (s = 0 ∧ 1 ≤ t)
conclusion
(∫ w, walshCoeff d h s w * walshCoeff d h t w ∂volume.withDensity (fun w => ofReal (refDensity d h w)))
= 0
Proof (Lean source)
-- @node: walshCoeff_reference_cross_zero
lemma walshCoeff_reference_cross_zero (d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(s t : ℕ) (hst : (1 ≤ s ∧ t = 0) ∨ (s = 0 ∧ 1 ≤ t)) :
(∫ w, walshCoeff d h s w * walshCoeff d h t w ∂volume.withDensity
(fun w => ofReal (refDensity d h w))) = 0 := by
rcases hst with ⟨hs, rfl⟩ | ⟨rfl, ht⟩
· calc
_ = ∫ w, walshCoeff d h s w ∂volume.withDensity
(fun w => ofReal (refDensity d h w)) := by
apply integral_congr_ae
filter_upwards [walshCoeff_zero_reference_ae d h] with w hw
rw [hw, mul_one]
_ = 0 := walshCoeff_reference_centered d h hd s hs
· calc
_ = ∫ w, walshCoeff d h t w ∂volume.withDensity
(fun w => ofReal (refDensity d h w)) := by
apply integral_congr_ae
filter_upwards [walshCoeff_zero_reference_ae d h] with w hw
rw [hw, one_mul]
_ = 0 := walshCoeff_reference_centered d h hd t ht
theorem walshCoeff_product_cross_integrable Lemma walshCoeff_product_cross_integrable in the paper ↗
Cross products of arbitrary row coefficient products are integrable, since each row coefficient is bounded by one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable (fun y : Fin B → ℝ => (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) * (∏ ℓ, walshCoeff d h (t ℓ) (y ℓ))) (Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
Proof (Lean source)
-- @node: walshCoeff_product_cross_integrable
lemma walshCoeff_product_cross_integrable (B d : ℕ) (h : ℝ) (s t : Fin B → ℕ) :
Integrable (fun y : Fin B → ℝ =>
(∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) *
(∏ ℓ, walshCoeff d h (t ℓ) (y ℓ)))
(Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) := by
let := referenceRowLaw_probability d h
apply Integrable.of_bound (by fun_prop) 1
filter_upwards [] with y
rw [Real.norm_eq_abs, abs_mul]
apply mul_le_one₀
· rw [Finset.abs_prod]
exact Finset.prod_le_one (fun _ _ => abs_nonneg _)
(fun ℓ _ => walshCoeff_abs_le_one d h (y ℓ) (s ℓ))
· exact abs_nonneg _
· rw [Finset.abs_prod]
exact Finset.prod_le_one (fun _ _ => abs_nonneg _)
(fun ℓ _ => walshCoeff_abs_le_one d h (y ℓ) (t ℓ))
theorem walshCoeff_product_support_orthogonal Lemma walshCoeff_product_support_orthogonal in the paper ↗
A row active in exactly one coefficient product forces their reference cross integral to vanish; degrees on shared active rows may differ. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ y : Fin B → ℝ, (∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) * (∏ ℓ, walshCoeff d h (t ℓ) (y ℓ)) ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= 0
Proof (Lean source)
-- @node: walshCoeff_product_support_orthogonal
lemma walshCoeff_product_support_orthogonal (B d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(s t : Fin B → ℕ)
(hst : ∃ ℓ, (1 ≤ s ℓ ∧ t ℓ = 0) ∨ (s ℓ = 0 ∧ 1 ≤ t ℓ)) :
(∫ y : Fin B → ℝ,
(∏ ℓ, walshCoeff d h (s ℓ) (y ℓ)) *
(∏ ℓ, walshCoeff d h (t ℓ) (y ℓ))
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) = 0 := by
let := referenceRowLaw_probability d h
simp_rw [← Finset.prod_mul_distrib]
rw [integral_fintype_prod_eq_prod
(fun ℓ w => walshCoeff d h (s ℓ) w * walshCoeff d h (t ℓ) w)]
obtain ⟨ℓ, hℓ⟩ := hst
exact Finset.prod_eq_zero (Finset.mem_univ ℓ)
(walshCoeff_reference_cross_zero d h hd (s ℓ) (t ℓ) hℓ)
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.HiddenAllocationSummation 5 declarations Summing over the observed total hidden degree removes the degree constraint exactly.
Summing constrained allocation energies
Summing over the observed total hidden degree removes the degree constraint exactly. The remaining nonnegative allocation energies factor over rows, including restrictions requiring positive degree on hidden active rows. This supplies the degree-summation step of the hidden-allocation contraction proof without assuming a posterior identity.
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.HiddenSelectionAveraging 13 declarations Reindex the completion likelihood by capacity-constrained words on the original hidden source type.
Original hidden-label selection averages
Reindex the completion likelihood by capacity-constrained words on the original hidden source type. Permutations of those labels preserve the posterior average.
theorem partitionHiddenLabels_eq_word_fiber
A completed row's hidden labels are exactly its hidden-word fiber. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hfit :
2 * (B * d) ≤ n
s₀ :
CompatiblePartition n B d H
g :
HiddenCompletion n B d H
ℓ :
Fin B
conclusion
partitionHiddenLabels n B d (hiddenCompletionPartition n B d hfit H s₀ g).1 H ℓ
= (Finset.univ.filter (fun j => g.1 j = ℓ)).map (hiddenSourceLabelEmbedding n B d hfit H)
Proof (Lean source)
-- @node: partitionHiddenLabels_eq_word_fiber
lemma partitionHiddenLabels_eq_word_fiber (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(H : OffDiag (Fin n) → Bool) (s₀ : CompatiblePartition n B d H)
(g : HiddenCompletion n B d H) (ℓ : Fin B) :
partitionHiddenLabels n B d (hiddenCompletionPartition n B d hfit H s₀ g).1 H ℓ =
(Finset.univ.filter (fun j => g.1 j = ℓ)).map
(hiddenSourceLabelEmbedding n B d hfit H) := by
ext j
rw [partitionHiddenLabels_mem_iff n B d _ H
(hiddenCompletionPartition n B d hfit H s₀ g).2]
constructor
· rintro ⟨⟨hj, hrow⟩, hhidden⟩
have hr := (sourceReveals_false_iff n B d H j hj).mpr hhidden
refine Finset.mem_map.mpr ⟨⟨⟨j.val,hj⟩,hr⟩, ?_, ?_⟩
· simpa only [mem_filter, Finset.mem_univ, true_and,
hiddenCompletionPartition, hiddenCompletionExtension, dif_pos hr] using hrow
· rfl
· rintro hmap
obtain ⟨v, hv, rfl⟩ := Finset.mem_map.mp hmap
refine ⟨⟨v.1.isLt, ?_⟩, ?_⟩
· simpa only [mem_filter, Finset.mem_univ, true_and,
hiddenSourceLabelEmbedding, Function.Embedding.coeFn_mk,
hiddenCompletionPartition, hiddenCompletionExtension, dif_pos v.2] using hv
· exact (sourceReveals_false_iff n B d H _ v.1.isLt).mp v.2
Selected-subset monomials commute with an injective relabeling. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ E ∈ (S.map e).powersetCard k, ∏ j ∈ E, x j) = ∑ E ∈ S.powersetCard k, ∏ j ∈ E, x (e j)
Proof (Lean source)
-- @node: powersetCard_monomial_sum_map
lemma powersetCard_monomial_sum_map {ι κ : Type*} [DecidableEq ι] [DecidableEq κ]
(e : ι ↪ κ) (S : Finset ι) (k : ℕ) (x : κ → ℝ) :
(∑ E ∈ (S.map e).powersetCard k, ∏ j ∈ E, x j) =
∑ E ∈ S.powersetCard k, ∏ j ∈ E, x (e j) := by
rw [Finset.powersetCard_map]
rw [Finset.sum_map]
apply Finset.sum_congr rfl
intro E hE
change (∏ j ∈ E.map e, x j) = _
rw [Finset.prod_map]
theorem completionHiddenSelectionMoment_eq_word_average
The hidden moment in the actual likelihood is the original-label word average. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
completionHiddenSelectionMoment n B d hfit H s₀ z k
= hiddenWordSelectionAverage n B d H (fun j => signOf (z (hiddenSourceLabelEmbedding n B d hfit H j))) k
Proof (Lean source)
-- @node: completionHiddenSelectionMoment_eq_word_average
lemma completionHiddenSelectionMoment_eq_word_average (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (H : OffDiag (Fin n) → Bool)
(s₀ : CompatiblePartition n B d H) (z : Assign (Fin n)) (k : Fin B → ℕ) :
completionHiddenSelectionMoment n B d hfit H s₀ z k =
hiddenWordSelectionAverage n B d H
(fun j => signOf (z (hiddenSourceLabelEmbedding n B d hfit H j))) k := by
unfold completionHiddenSelectionMoment hiddenWordSelectionAverage
congr 1
apply Finset.sum_congr rfl
intro g hg
apply Finset.prod_congr rfl
intro ℓ hℓ
rw [partitionHiddenLabels_eq_word_fiber,
powersetCard_monomial_sum_map]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.HiddenSignEnergy 6 declarations The exact hidden-sign normalization survives adjoining the independent revealed signs.
Joint revealed and hidden sign energies
The exact hidden-sign normalization survives adjoining the independent revealed signs. Finite likelihood expansions therefore have squared norms weighted by inverse binomial coefficients, without treating hidden row counts as independent.
The degree-zero symmetric hidden polynomial is exactly the constant one, including the endpoint with no hidden labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: symmetricSignPoly_zero
lemma symmetricSignPoly_zero (M : ℕ) (z : Fin M → Bool) :
symmetricSignPoly M 0 z = 1 := by
simp [symmetricSignPoly]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.LabelSubsetConditionalLikelihood 9 declarations Disjoint revealed rows identify their subset families bijectively with subsets of the union of revealed labels.
Revealed-label indexing of the conditional Walsh likelihood
Disjoint revealed rows identify their subset families bijectively with subsets of the union of revealed labels. This puts the actual posterior likelihood in the label-monomial indexing used by the joint sign energy decomposition.
theorem revealedSources_pairwise_disjoint
Compatibility assigns each revealed source to a unique row. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: revealedSources_pairwise_disjoint
lemma revealedSources_pairwise_disjoint (n B d : ℕ) (H : OffDiag (Fin n) → Bool)
(s : CompatiblePartition n B d H) :
Pairwise (fun a b : Fin B => Disjoint
(revealedSources n B d H a) (revealedSources n B d H b)) := by
intro a b hab
apply Finset.disjoint_left.mpr
intro j hja hjb
have ha := revealedSources_subset_partitionSourceLabels n B d s.1 H s.2 a hja
have hb := revealedSources_subset_partitionSourceLabels n B d s.1 H s.2 b hjb
obtain ⟨h₁, h₁a⟩ := (Finset.mem_filter.mp ha).2
obtain ⟨h₂, h₂b⟩ := (Finset.mem_filter.mp hb).2
exact hab (h₁a.symm.trans h₂b)
theorem disjoint_rowSubset_union_inter
Intersecting the union of a disjoint row-subset family with one row recovers exactly that row's selected labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: disjoint_rowSubset_union_inter
lemma disjoint_rowSubset_union_inter {ι α : Type*} [Fintype ι]
[DecidableEq ι] [DecidableEq α] (R J : ι → Finset α)
(hR : Pairwise (fun a b => Disjoint (R a) (R b)))
(hJ : ∀ a, J a ⊆ R a) (a : ι) :
(Finset.univ.biUnion J) ∩ R a = J a := by
ext j
simp only [Finset.mem_inter, Finset.mem_biUnion, Finset.mem_univ, true_and]
constructor
· rintro ⟨⟨b, hj⟩, hra⟩
by_cases hba : b = a
· simpa [hba] using hj
· exact False.elim (Finset.disjoint_left.mp (hR hba) (hJ b hj) hra)
· intro hj
exact ⟨⟨a, hj⟩, hJ a hj⟩
theorem rowSubset_inter_union
Every subset of a row union is recovered from its intersections with the rows. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowSubset_inter_union
lemma rowSubset_inter_union {ι α : Type*} [Fintype ι]
[DecidableEq ι] [DecidableEq α] (R : ι → Finset α) (J : Finset α)
(hJ : J ⊆ Finset.univ.biUnion R) :
Finset.univ.biUnion (fun a => J ∩ R a) = J := by
ext j
simp only [Finset.mem_biUnion, Finset.mem_univ, true_and, Finset.mem_inter]
constructor
· rintro ⟨a, hj, _⟩
exact hj
· intro hj
obtain ⟨a, _, hra⟩ := Finset.mem_biUnion.mp (hJ hj)
exact ⟨a, hj, hra⟩
theorem disjoint_rowSubset_sum_reindex
Summing over disjoint row-subset families equals summing over subsets of all revealed labels, with row intersections as the inverse map. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: disjoint_rowSubset_sum_reindex
lemma disjoint_rowSubset_sum_reindex {ι α : Type*} [Fintype ι]
[DecidableEq ι] [DecidableEq α] (R : ι → Finset α)
(hR : Pairwise (fun a b => Disjoint (R a) (R b)))
(f : (ι → Finset α) → ℝ) :
(∑ J ∈ piFinset (fun a => (R a).powerset), f J) =
∑ J ∈ (Finset.univ.biUnion R).powerset, f (fun a => J ∩ R a) := by
apply Finset.sum_bij (fun J _ => Finset.univ.biUnion J)
· intro J hJ
apply Finset.mem_powerset.mpr
intro j hj
obtain ⟨a, ha, hja⟩ := Finset.mem_biUnion.mp hj
exact Finset.mem_biUnion.mpr ⟨a, ha,
Finset.mem_powerset.mp (Fintype.mem_piFinset.mp hJ a) hja⟩
· intro J hJ K hK he
funext a
have hsubJ := fun a => Finset.mem_powerset.mp (Fintype.mem_piFinset.mp hJ a)
have hsubK := fun a => Finset.mem_powerset.mp (Fintype.mem_piFinset.mp hK a)
rw [← disjoint_rowSubset_union_inter R J hR hsubJ a,
← disjoint_rowSubset_union_inter R K hR hsubK a, he]
· intro J hJ
refine ⟨fun a => J ∩ R a, ?_, rowSubset_inter_union R J (Finset.mem_powerset.mp hJ)⟩
exact Fintype.mem_piFinset.mpr (fun a =>
Finset.mem_powerset.mpr inter_subset_right)
· intro J hJ
congr 1
funext a
exact (disjoint_rowSubset_union_inter R J hR
(fun a => Finset.mem_powerset.mp (Fintype.mem_piFinset.mp hJ a)) a).symm
theorem disjoint_rowSubset_signProduct
Products over a row-subset family are precisely monomials in its union of original revealed labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: disjoint_rowSubset_signProduct
lemma disjoint_rowSubset_signProduct {ι α : Type*} [Fintype ι]
[DecidableEq ι] [DecidableEq α] (R J : ι → Finset α)
(hR : Pairwise (fun a b => Disjoint (R a) (R b)))
(hJ : ∀ a, J a ⊆ R a) (z : α → Bool) :
(∏ a, ∏ j ∈ J a, signOf (z j)) =
∏ j ∈ Finset.univ.biUnion J, signOf (z j) := by
symm
apply Finset.prod_biUnion
intro a _ b _ hab
exact (hR hab).mono (hJ a) (hJ b)
theorem groupedHiddenWalsh_sum_labelSubsets
The grouped actual likelihood is a sum of distinct revealed-label monomials and symmetric hidden polynomials with the exact row-intersection coefficients. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) * ∑ t ∈ range (undiscovered n B d H + 1), symmetricSignPoly (undiscovered n B d H) t x * groupedHiddenWalshCoeff n B d h H J t y)
= ∑ J ∈ (Finset.univ.biUnion (revealedSources n B d H)).powerset, (∏ j ∈ J, signOf (z j)) * ∑ t ∈ range (undiscovered n B d H + 1), symmetricSignPoly (undiscovered n B d H) t x * groupedHiddenWalshCoeff n B d h H (fun ℓ => J ∩ revealedSources n B d H ℓ) t y
Proof (Lean source)
-- @node: groupedHiddenWalsh_sum_labelSubsets
lemma groupedHiddenWalsh_sum_labelSubsets (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (s : CompatiblePartition n B d H)
(z : Assign (Fin n)) (y : Fin B → ℝ)
(x : Fin (undiscovered n B d H) → Bool) :
(∑ J ∈ piFinset
(fun ℓ : Fin B => (revealedSources n B d H ℓ).powerset),
(∏ ℓ, ∏ j ∈ J ℓ, signOf (z j)) *
∑ t ∈ range (undiscovered n B d H + 1),
symmetricSignPoly (undiscovered n B d H) t x *
groupedHiddenWalshCoeff n B d h H J t y) =
∑ J ∈ (Finset.univ.biUnion (revealedSources n B d H)).powerset,
(∏ j ∈ J, signOf (z j)) *
∑ t ∈ range (undiscovered n B d H + 1),
symmetricSignPoly (undiscovered n B d H) t x *
groupedHiddenWalshCoeff n B d h H
(fun ℓ => J ∩ revealedSources n B d H ℓ) t y := by
rw [disjoint_rowSubset_sum_reindex _ (revealedSources_pairwise_disjoint n B d H s)]
apply Finset.sum_congr rfl
intro J hJ
rw [disjoint_rowSubset_signProduct _ _ (revealedSources_pairwise_disjoint n B d H s)
(fun _ => inter_subset_right),
rowSubset_inter_union _ J (Finset.mem_powerset.mp hJ)]
def labelSubsetHiddenWalshDensity
The actual posterior density with each revealed monomial indexed by its source-label subset, retaining the reference-zero extension.
Definition (Lean source)
parameters
def
labelSubsetHiddenWalshDensity n B d hfit h hz y :
ℝ
given by
if hs : Nonempty (CompatiblePartition n B d hz.1) then if ∀ ℓ, 0 < refDensity d h (y ℓ) then (∏ ℓ, refDensity d h (y ℓ)) * (∑ J ∈ (Finset.univ.biUnion (revealedSources n B d hz.1)).powerset, (∏ j ∈ J, signOf (hz.2 j)) * ∑ t ∈ range (undiscovered n B d hz.1 + 1), symmetricSignPoly (undiscovered n B d hz.1) t (fun i => hz.2 (hiddenSourceLabelEmbedding n B d hfit hz.1 ((hiddenSourceEnumeration n B d hfit hz.1 hs.some).symm i))) * groupedHiddenWalshCoeff n B d h hz.1 (fun ℓ => J ∩ revealedSources n B d hz.1 ℓ) t y) else 0 else 0
theorem groupedHiddenWalshDensity_eq_labelSubset
The label-subset density equals the actual grouped density on every graph and response, including impossible graphs and reference-density zeros. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
groupedHiddenWalshDensity n B d hfit h hz y
= labelSubsetHiddenWalshDensity n B d hfit h hz y
Proof (Lean source)
-- @node: groupedHiddenWalshDensity_eq_labelSubset
lemma groupedHiddenWalshDensity_eq_labelSubset (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n)) (y : Fin B → ℝ) :
groupedHiddenWalshDensity n B d hfit h hz y =
labelSubsetHiddenWalshDensity n B d hfit h hz y := by
unfold groupedHiddenWalshDensity labelSubsetHiddenWalshDensity
split_ifs with hs hy
· rw [groupedHiddenWalsh_sum_labelSubsets n B d h hz.1 hs.some]
all_goals rfl
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.LikelihoodExpansion 24 declarations Conditional likelihood for the hidden-allocation record
Conditional likelihood for the hidden-allocation record
def distinctResponses
Extract distinct block responses by averaging identical recipient outcomes.
Definition (Lean source)
def reducedBlockLaw
Full retained graph together with all assignments and distinct block responses.
Definition (Lean source)
parameters
def
given by
theorem copyOutcomes_recipient
Copying a block response gives that same value at every recipient in its block. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
y :
Fin B → ℝ
ℓ :
Fin B
i :
Fin n
hi :
i ∈ recipientBlock n B d ℓ
conclusion
(copyOutcomes n B d hz y).2.2 i = y ℓ
Proof (Lean source)
-- @node: copyOutcomes_recipient
lemma copyOutcomes_recipient (hz : (OffDiag (Fin n) → Bool) × Assign (Fin n))
(y : Fin B → ℝ) (ℓ : Fin B) (i : Fin n) (hi : i ∈ recipientBlock n B d ℓ) :
(copyOutcomes n B d hz y).2.2 i = y ℓ := by
change (∑ k : Fin B, if i ∈ recipientBlock n B d k then y k else 0) = y ℓ
rw [Finset.sum_eq_single ℓ]
· simp [hi]
· intro k _ hk
rw [if_neg]
exact fun hik => hk (recipientBlock_unique n B d i k ℓ hik hi)
· simp
theorem distinctResponses_copyOutcomes
Averaging the copied recipients recovers the distinct response exactly. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
distinctResponses n B d (copyOutcomes n B d hz y) = y
Proof (Lean source)
-- @node: distinctResponses_copyOutcomes
lemma distinctResponses_copyOutcomes (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n)) (y : Fin B → ℝ) :
distinctResponses n B d (copyOutcomes n B d hz y) = y := by
funext ℓ
unfold distinctResponses
rw [Finset.sum_congr rfl (fun i hi => copyOutcomes_recipient n B d hz y ℓ i hi)]
simp only [Finset.sum_const, nsmul_eq_mul, recipientBlock_card n B d hfit]
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
rw [← mul_assoc, inv_mul_cancel₀ hd0, one_mul]
theorem blockSchedule_response_constant
Two recipients in the same block have identical potential outcomes for every assignment. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
ℓ :
Fin B
i k :
Fin n
hi :
i ∈ recipientBlock n B d ℓ
hk :
k ∈ recipientBlock n B d ℓ
conclusion
potentialOutcome (blockSchedule n B d σ h ξ) i z
= potentialOutcome (blockSchedule n B d σ h ξ) k z
Proof (Lean source)
-- @node: blockSchedule_response_constant
lemma blockSchedule_response_constant (σ : Bool) (h : ℝ)
(ξ : SourcePartition B d × (Fin B → ℝ)) (z : Assign (Fin n))
(ℓ : Fin B) (i k : Fin n)
(hi : i ∈ recipientBlock n B d ℓ) (hk : k ∈ recipientBlock n B d ℓ) :
potentialOutcome (blockSchedule n B d σ h ξ) i z =
potentialOutcome (blockSchedule n B d σ h ξ) k z := by
have hmem (r : Fin n) (hr : r ∈ recipientBlock n B d ℓ) (t : Fin B) :
r ∈ recipientBlock n B d t ↔ t = ℓ := by
constructor
· intro ht
exact recipientBlock_unique n B d r t ℓ ht hr
· rintro rfl
exact hr
have hedge (r : Fin n) (hr : r ∈ recipientBlock n B d ℓ) (j : Fin n) :
blockEdge n B d ξ.1 j r ↔ ∃ hj : j.val < B * d, ξ.1.1 ⟨j.val, hj⟩ = ℓ := by
constructor
· rintro ⟨⟨hj, t, ht, hrt⟩, _⟩
exact ⟨hj, ht.trans ((hmem r hr t).mp hrt)⟩
· rintro ⟨hj, ht⟩
apply (blockEdge_source_iff n B d ξ.1 j r hj).mpr
simpa [ht] using hr
simp only [potentialOutcome, blockSchedule, inNbhd, zero_mul, add_zero,
hmem i hi, hmem k hk, hedge i hi, hedge k hk]
theorem blockSchedule_response_zero
Sources and padding labels have zero response under every constructed schedule. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
i :
Fin n
hi :
∀ ℓ, i ∉ recipientBlock n B d ℓ
conclusion
potentialOutcome (blockSchedule n B d σ h ξ) i z = 0
Proof (Lean source)
-- @node: blockSchedule_response_zero
lemma blockSchedule_response_zero (σ : Bool) (h : ℝ)
(ξ : SourcePartition B d × (Fin B → ℝ)) (z : Assign (Fin n)) (i : Fin n)
(hi : ∀ ℓ, i ∉ recipientBlock n B d ℓ) :
potentialOutcome (blockSchedule n B d σ h ξ) i z = 0 := by
rw [potentialOutcome, blockSchedule_inNbhd_empty n B d σ h ξ i hi]
simp [blockSchedule, hi]
theorem distinctResponses_record_recipient
On a genuine block record, the distinct-response average is each recipient's response. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
ℓ :
Fin B
i :
Fin n
hi :
i ∈ recipientBlock n B d ℓ
conclusion
= potentialOutcome (blockSchedule n B d σ h ξ) i ω.1
Proof (Lean source)
-- @node: distinctResponses_record_recipient
lemma distinctResponses_record_recipient (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h : ℝ) (ξ : SourcePartition B d × (Fin B → ℝ))
(ω : Assign (Fin n) × Audit (Fin n)) (ℓ : Fin B) (i : Fin n)
(hi : i ∈ recipientBlock n B d ℓ) :
distinctResponses n B d (recordOf (blockSchedule n B d σ h ξ) ω) ℓ =
potentialOutcome (blockSchedule n B d σ h ξ) i ω.1 := by
unfold distinctResponses
change (d : ℝ)⁻¹ * (∑ k ∈ recipientBlock n B d ℓ,
potentialOutcome (blockSchedule n B d σ h ξ) k ω.1) = _
rw [Finset.sum_congr rfl (fun k hk =>
blockSchedule_response_constant n B d σ h ξ ω.1 ℓ k i hk hi)]
simp only [Finset.sum_const, nsmul_eq_mul, recipientBlock_card n B d hfit]
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
rw [← mul_assoc, inv_mul_cancel₀ hd0, one_mul]
theorem copyOutcomes_distinctResponses_record
Copying the recovered distinct responses reconstructs the entire original record, including every retained edge and treatment coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
conclusion
copyOutcomes n B d ((recordOf (blockSchedule n B d σ h ξ) ω).1, ω.1) (distinctResponses n B d (recordOf (blockSchedule n B d σ h ξ) ω))
= recordOf (blockSchedule n B d σ h ξ) ω
Proof (Lean source)
-- @node: copyOutcomes_distinctResponses_record
lemma copyOutcomes_distinctResponses_record (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h : ℝ) (ξ : SourcePartition B d × (Fin B → ℝ))
(ω : Assign (Fin n) × Audit (Fin n)) :
copyOutcomes n B d
((recordOf (blockSchedule n B d σ h ξ) ω).1, ω.1)
(distinctResponses n B d (recordOf (blockSchedule n B d σ h ξ) ω)) =
recordOf (blockSchedule n B d σ h ξ) ω := by
apply Prod.ext
· rfl
apply Prod.ext
· rfl
funext i
by_cases hi : ∃ ℓ, i ∈ recipientBlock n B d ℓ
· obtain ⟨ℓ, hi⟩ := hi
exact (copyOutcomes_recipient n B d _ _ ℓ i hi).trans
(distinctResponses_record_recipient n B d hd hfit σ h ξ ω ℓ i hi)
· have hn := not_exists.mp hi
change (∑ ℓ : Fin B, if i ∈ recipientBlock n B d ℓ then _ else 0) = _
simp only [hn, ite_false, Finset.sum_const_zero]
exact (blockSchedule_response_zero n B d σ h ξ ω.1 i hn).symm
theorem copyOutcomes_measurable
The response-copying channel is measurable jointly in the full finite marginal and the distinct real responses.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
conclusion
Measurable (fun x : (OffDiag (Fin n) → Bool) × (Assign (Fin n) × (Fin B → ℝ)) => copyOutcomes n B d (x.1, x.2.1) x.2.2)
Proof (Lean source)
-- @node: copyOutcomes_measurable
@[fun_prop] lemma copyOutcomes_measurable :
Measurable (fun x : (OffDiag (Fin n) → Bool) × (Assign (Fin n) × (Fin B → ℝ)) =>
copyOutcomes n B d (x.1, x.2.1) x.2.2) := by
unfold copyOutcomes
apply measurable_fst.prodMk
apply (measurable_fst.comp measurable_snd).prodMk
apply measurable_pi_lambda
intro i
apply Finset.measurable_sum
intro ℓ _
by_cases hi : i ∈ recipientBlock n B d ℓ <;> simp only [hi, ite_true, ite_false] <;> fun_prop
Recovering distinct responses is a measurable statistic of the complete original record.
Formal statement
Proof (Lean source)
-- @node: distinctResponses_measurable
@[fun_prop] lemma distinctResponses_measurable : Measurable (distinctResponses n B d) := by
unfold distinctResponses
fun_prop
theorem reducedBlockLaw_copy_eq
Applying the measurable copying channel to the reduced law recovers the complete original-record mixture, without using the unfinished conditional-density identity. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
conclusion
= blockMixtureLawOf n B d D σ h
Proof (Lean source)
-- @node: reducedBlockLaw_copy_eq
lemma reducedBlockLaw_copy_eq (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [SFinite D] (σ : Bool) (h : ℝ) :
(reducedBlockLaw n B d D σ h).map
(fun x => copyOutcomes n B d (x.1, x.2.1) x.2.2) =
blockMixtureLawOf n B d D σ h := by
unfold reducedBlockLaw
rw [Measure.map_map (copyOutcomes_measurable n B d) (by fun_prop)]
unfold blockMixtureLawOf mixtureLaw
rw [block_map_bind _ _ _
(block_measurable_map_parameter D _ (block_record_measurable n B d σ h))
((copyOutcomes_measurable n B d).comp (by fun_prop))]
congr 1
funext ξ
rw [Measure.map_map
((copyOutcomes_measurable n B d).comp (by fun_prop)) (measurable_of_finite _)]
congr 1
funext ω
exact copyOutcomes_distinctResponses_record n B d hd hfit σ h ξ ω
theorem block_tvDist_map_le
A measurable map contracts total variation by taking preimages of test events. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
α β :
μ ν :
f :
α → β
hf :
Proof (Lean source)
-- @node: block_tvDist_map_le
lemma block_tvDist_map_le {α β : Type*} [MeasurableSpace α] [MeasurableSpace β]
(μ ν : Measure α) [IsProbabilityMeasure μ] [IsProbabilityMeasure ν]
(f : α → β) (hf : Measurable f) :
tvDist (μ.map f) (ν.map f) ≤ tvDist μ ν := by
unfold tvDist
apply ciSup_le
rintro ⟨A, hA⟩
rw [map_measureReal_apply_of_aemeasurable hf.aemeasurable hA,
map_measureReal_apply_of_aemeasurable hf.aemeasurable hA]
exact abs_measureReal_sub_le_tvDist (hf hA)
theorem blockMixtureLawOf_probability
Complete block mixtures are probability laws for every sign and amplitude. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
σ :
h :
ℝ
conclusion
IsProbabilityMeasure (blockMixtureLawOf n B d D σ h)
Proof (Lean source)
-- @node: blockMixtureLawOf_probability
lemma blockMixtureLawOf_probability
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(σ : Bool) (h : ℝ) : IsProbabilityMeasure (blockMixtureLawOf n B d D σ h) := by
let := partitionLaw_probability B d
let := blockBaselineLaw_probability B
let : IsProbabilityMeasure (blockParamLaw B d) := by unfold blockParamLaw; infer_instance
exact mixtureLaw_probability D (blockParamLaw B d) (blockSchedule n B d σ h)
(block_record_measurable n B d σ h)
theorem reducedBlockLaw_tvDist_eq
Recovering and copying distinct responses preserve total variation exactly. Thus working with distinct outcomes retains all the information in the original record. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
Proof (Lean source)
-- @node: reducedBlockLaw_tvDist_eq
lemma reducedBlockLaw_tvDist_eq (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(h : ℝ) :
tvDist (reducedBlockLaw n B d D true h)
(reducedBlockLaw n B d D false h) =
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) := by
let (σ : Bool) : IsProbabilityMeasure (blockMixtureLawOf n B d D σ h) :=
blockMixtureLawOf_probability n B d D σ h
have hm : Measurable (fun o : Record (Fin n) =>
(o.1, o.2.1, distinctResponses n B d o)) := by fun_prop
let (σ : Bool) : IsProbabilityMeasure (reducedBlockLaw n B d D σ h) :=
Measure.isProbabilityMeasure_map hm.aemeasurable
apply le_antisymm
· exact block_tvDist_map_le _ _ _ hm
· rw [← reducedBlockLaw_copy_eq n B d hd hfit D true h,
← reducedBlockLaw_copy_eq n B d hd hfit D false h]
exact block_tvDist_map_le _ _ _ (copyOutcomes_measurable n B d)
theorem blockSchedule_response_centered
A recipient response is its baseline plus the centered source-sign shift. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
ξ :
SourcePartition B d × (Fin B → ℝ)
ℓ :
Fin B
i :
Fin n
hi :
i ∈ recipientBlock n B d ℓ
conclusion
potentialOutcome (blockSchedule n B d σ h ξ) i z
= ξ.2 ℓ
+ signOf σ * h / (2 * d) * ∑ j ∈ inNbhd (blockSchedule n B d σ h ξ) i, signOf (z j)
Proof (Lean source)
-- @node: blockSchedule_response_centered
lemma blockSchedule_response_centered (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h : ℝ) (ξ : SourcePartition B d × (Fin B → ℝ))
(z : Assign (Fin n)) (ℓ : Fin B) (i : Fin n)
(hi : i ∈ recipientBlock n B d ℓ) :
potentialOutcome (blockSchedule n B d σ h ξ) i z =
ξ.2 ℓ + signOf σ * h / (2 * d) *
∑ j ∈ inNbhd (blockSchedule n B d σ h ξ) i, signOf (z j) := by
have ha : (blockSchedule n B d σ h ξ).a i = ξ.2 ℓ - signOf σ * h / 2 := by
change (∑ k : Fin B, if i ∈ recipientBlock n B d k then _ else 0) = _
rw [Finset.sum_eq_single ℓ]
· simp [hi]
· intro k _ hk
rw [if_neg]
exact fun hik => hk (recipientBlock_unique n B d i k ℓ hik hi)
· simp
have hz (j : Fin n) : (if z j then (1 : ℝ) else 0) = (signOf (z j) + 1) / 2 := by
cases z j <;> norm_num [signOf]
rw [potentialOutcome, ha]
simp only [blockSchedule, zero_mul, add_zero]
simp_rw [treatment, hz]
simp only [add_div, mul_add, Finset.sum_add_distrib, Finset.sum_div,
← Finset.mul_sum, Finset.sum_const, nsmul_eq_mul]
have hc := blockSchedule_inNbhd_card n B d hd hfit σ h ξ i ℓ hi
simp only [blockSchedule] at hc
rw [hc, ← Finset.sum_div]
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
field_simp
<;> ring
theorem blockSchedule_response_row_signs
Enumerating a recipient's actual d source labels gives a row-channel sign vector. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
s :
SourcePartition B d
ℓ :
Fin B
i :
Fin n
hi :
i ∈ recipientBlock n B d ℓ
conclusion
∀ U : Fin B → ℝ,
potentialOutcome (blockSchedule n B d σ h (s, U)) i z
= U ℓ + signOf σ * h / (2 * d) * ∑ j, signOf (ε j)
Proof (Lean source)
-- @node: blockSchedule_response_row_signs
lemma blockSchedule_response_row_signs (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h : ℝ) (s : SourcePartition B d) (z : Assign (Fin n))
(ℓ : Fin B) (i : Fin n) (hi : i ∈ recipientBlock n B d ℓ) :
∃ ε : Fin d → Bool, ∀ U : Fin B → ℝ,
potentialOutcome (blockSchedule n B d σ h (s, U)) i z =
U ℓ + signOf σ * h / (2 * d) * ∑ j, signOf (ε j) := by
let N := inNbhd (blockSchedule n B d σ h (s, fun _ => 0)) i
have hc : N.card = d := blockSchedule_inNbhd_card n B d hd hfit σ h _ i ℓ hi
let e : N ≃ Fin d := Finset.equivFinOfCardEq hc
refine ⟨fun j => z (e.symm j).val, ?_⟩
intro U
rw [blockSchedule_response_centered n B d hd hfit σ h (s, U) z ℓ i hi]
have he : (∑ j ∈ N, signOf (z j)) = ∑ j : Fin d, signOf (z (e.symm j).val) := by
rw [← Finset.sum_attach]
exact Fintype.sum_equiv e _ _ (fun j => by simp)
change U ℓ + signOf σ * h / (2 * d) * (∑ j ∈ N, signOf (z j)) = _
rw [he]
theorem distinctResponses_record_row_signs
All distinct responses, conditional on a fixed partition and the entire design draw, are independent baseline translations by actual source signs. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
s :
SourcePartition B d
conclusion
∀ U : Fin B → ℝ,
= fun ℓ => U ℓ + signOf σ * h / (2 * d) * ∑ j, signOf (ε ℓ j)
Proof (Lean source)
-- @node: distinctResponses_record_row_signs
lemma distinctResponses_record_row_signs (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) :
∃ ε : Fin B → Fin d → Bool, ∀ U : Fin B → ℝ,
distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, U)) ω) =
fun ℓ => U ℓ + signOf σ * h / (2 * d) * ∑ j, signOf (ε ℓ j) := by
have hrow (ℓ : Fin B) : ∃ ε : Fin d → Bool, ∀ U : Fin B → ℝ,
distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, U)) ω) ℓ =
U ℓ + signOf σ * h / (2 * d) * ∑ j, signOf (ε j) := by
have hc := recipientBlock_card n B d hfit ℓ
have hn : (recipientBlock n B d ℓ).Nonempty := Finset.card_pos.mp (by omega)
obtain ⟨i, hi⟩ := hn
obtain ⟨ε, hε⟩ := blockSchedule_response_row_signs n B d hd hfit σ h s ω.1 ℓ i hi
refine ⟨ε, fun U => ?_⟩
rw [distinctResponses_record_recipient n B d hd hfit σ h (s, U) ω ℓ i hi]
exact hε U
choose ε hε using hrow
exact ⟨ε, fun U => funext (fun ℓ => hε ℓ U)⟩
theorem block_responseLaw_eq_translatedRows
For a fixed partition and complete design draw, the positive-sign response law is exactly a product of row-channel components indexed by the actual source labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s :
SourcePartition B d
conclusion
(blockBaselineLaw B).map (fun U => distinctResponses n B d (recordOf (blockSchedule n B d true h (s, U)) ω))
= Measure.pi (fun ℓ => volume.withDensity (fun w => ofReal (rowDensity d h (ε ℓ) w)))
Proof (Lean source)
-- @node: block_responseLaw_eq_translatedRows
lemma block_responseLaw_eq_translatedRows (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(h : ℝ) (s : SourcePartition B d) (ω : Assign (Fin n) × Audit (Fin n)) :
∃ ε : Fin B → Fin d → Bool,
(blockBaselineLaw B).map (fun U =>
distinctResponses n B d (recordOf (blockSchedule n B d true h (s, U)) ω)) =
Measure.pi (fun ℓ => volume.withDensity
(fun w => ofReal (rowDensity d h (ε ℓ) w))) := by
obtain ⟨ε, hε⟩ := distinctResponses_record_row_signs n B d hd hfit true h s ω
refine ⟨ε, ?_⟩
have he : (fun U => distinctResponses n B d
(recordOf (blockSchedule n B d true h (s, U)) ω)) =
fun U ℓ => U ℓ + h / (2 * d) * ∑ j, signOf (ε ℓ j) := by
funext U
simpa [signOf] using hε U
let : IsProbabilityMeasure (volume.withDensity
(fun w => ofReal (cosSqDensity w))) := by
simpa [rowDensity] using translatedRowLaw_probability 0 0 (fun _ => false)
let (ℓ : Fin B) : IsProbabilityMeasure
((volume.withDensity (fun w => ofReal (cosSqDensity w))).map
(fun w => w + h / (2 * d) * ∑ j, signOf (ε ℓ j))) :=
Measure.isProbabilityMeasure_map (by fun_prop)
rw [he, blockBaselineLaw,
Measure.pi_map_pi (f := fun ℓ w => w + h / (2 * d) * ∑ j, signOf (ε ℓ j))
(fun _ => by fun_prop)]
simp_rw [baseline_map_add_density]
rfl
theorem block_responseLaw_absolutelyContinuous_reference
Every fixed-partition, fixed-design response law is dominated by the independent reference responses, before any conditional allocation calculation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
s :
SourcePartition B d
conclusion
(blockBaselineLaw B).map (fun U => distinctResponses n B d (recordOf (blockSchedule n B d true h (s, U)) ω)) ≪ Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w)))
Proof (Lean source)
-- @node: block_responseLaw_absolutelyContinuous_reference
lemma block_responseLaw_absolutelyContinuous_reference
(hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(h : ℝ) (s : SourcePartition B d) (ω : Assign (Fin n) × Audit (Fin n)) :
(blockBaselineLaw B).map (fun U =>
distinctResponses n B d (recordOf (blockSchedule n B d true h (s, U)) ω)) ≪
Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
obtain ⟨ε, he⟩ := block_responseLaw_eq_translatedRows n B d hd hfit h s ω
rw [he]
exact translatedRows_absolutelyContinuous_reference B d h ε
def conditionalBlockLaw
Conditioning on a finite retained-graph atom; values on null atoms are immaterial.
Definition (Lean source)
parameters
n :
ℕ
B :
ℕ
d :
ℕ
σ :
h :
ℝ
given by
((retainedGraphMarginal n B d D) {H})⁻¹ • ((reducedBlockLaw n B d D σ h).restrict {x | x.1 = H}).map snd
theorem reducedBlockLaw_snd_absolutelyContinuous_reference
After mixing the partition and baselines, the assignment-response marginal is still dominated by the actual assignment marginal times the reference responses. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
Proof (Lean source)
-- @node: reducedBlockLaw_snd_absolutelyContinuous_reference
lemma reducedBlockLaw_snd_absolutelyContinuous_reference
(hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] (h : ℝ) :
(reducedBlockLaw n B d D true h).map snd ≪ hiddenReferenceLaw n B d D h := by
let ν := Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))
let := referenceRowLaw_probability d h
let := blockBaselineLaw_probability B
let f := fun o : Record (Fin n) => (o.2.1, distinctResponses n B d o)
have hf : Measurable f := by fun_prop
have hrec : Measurable (fun x : (SourcePartition B d × (Fin B → ℝ)) ×
(Assign (Fin n) × Audit (Fin n)) => f (recordOf (blockSchedule n B d true h x.1) x.2)) :=
hf.comp (block_record_measurable n B d true h)
let K := fun ξ : SourcePartition B d × (Fin B → ℝ) =>
D.map (fun ω => f (recordOf (blockSchedule n B d true h ξ) ω))
have hK : Measurable K := block_measurable_map_parameter D _ hrec
have heq : (reducedBlockLaw n B d D true h).map snd = (blockParamLaw B d).bind K := by
rw [reducedBlockLaw, Measure.map_map (by fun_prop) (by fun_prop)]
change (blockMixtureLawOf n B d D true h).map f = _
rw [blockMixtureLawOf, mixtureLaw, block_map_bind _ _ _
(block_measurable_map_parameter D _ (block_record_measurable n B d true h)) hf]
congr 1
funext ξ
exact Measure.map_map hf (measurable_of_finite _)
rw [heq]
apply Measure.AbsolutelyContinuous.mk
intro E hE hzero
have hz : ∀ᵐ z ∂D.map fst, ν (Prod.mk z ⁻¹' E) = 0 :=
Measure.measure_ae_null_of_prod_null hzero
have hω : ∀ᵐ ω ∂D, ν (Prod.mk ω.1 ⁻¹' E) = 0 :=
ae_of_ae_map (by fun_prop) hz
have hs (s : SourcePartition B d) :
(∫⁻ U, K (s, U) E ∂blockBaselineLaw B) = 0 := by
let g := fun x : (Fin B → ℝ) × (Assign (Fin n) × Audit (Fin n)) =>
f (recordOf (blockSchedule n B d true h (s, x.1)) x.2)
have hg : Measurable g := hrec.comp
((measurable_const.prodMk measurable_fst).prodMk measurable_snd)
have hp : ((blockBaselineLaw B).prod D) (g ⁻¹' E) = 0 := by
rw [Measure.prod_apply_symm (hg hE)]
apply lintegral_eq_zero_of_ae_eq_zero
filter_upwards [hω] with ω hω
have hac := block_responseLaw_absolutelyContinuous_reference n B d hd hfit h s ω
have hz' := hac hω
have hm : Measurable (fun U => distinctResponses n B d
(recordOf (blockSchedule n B d true h (s, U)) ω)) :=
(distinctResponses_measurable n B d).comp
((block_record_measurable n B d true h).comp
((measurable_const.prodMk measurable_id).prodMk measurable_const))
rw [Measure.map_apply hm (measurable_prodMk_left hE)] at hz'
exact hz'
rw [Measure.prod_apply (hg hE)] at hp
convert hp using 1
apply lintegral_congr
intro U
exact Measure.map_apply (measurable_of_finite _) hE
have hm : Measurable (fun ξ => K ξ E) := (Measure.measurable_coe hE).comp hK
rw [Measure.bind_apply hE hK.aemeasurable, blockParamLaw,
lintegral_prod (fun ξ => K ξ E) hm.aemeasurable]
simp_rw [hs]
simp
theorem conditionalBlockLaw_absolutelyContinuous_of_probability
Restriction to a retained-graph atom and rescaling preserve reference domination; this also covers null atoms under the total ENNReal convention. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n :
ℕ
B :
ℕ
d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
h :
ℝ
conclusion
Proof (Lean source)
-- @node: conditionalBlockLaw_absolutelyContinuous_of_probability
lemma conditionalBlockLaw_absolutelyContinuous_of_probability
(hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(h : ℝ) (H : OffDiag (Fin n) → Bool) :
conditionalBlockLaw n B d D true h H ≪ hiddenReferenceLaw n B d D h := by
have hac := reducedBlockLaw_snd_absolutelyContinuous_reference n B d hd hfit D h
have hle : ((reducedBlockLaw n B d D true h).restrict {x | x.1 = H}).map snd ≤
(reducedBlockLaw n B d D true h).map snd :=
Measure.map_mono Measure.restrict_le_self (by fun_prop)
exact (hle.absolutelyContinuous.trans hac).smul_left _
theorem conditionalBlockLaw_absolutelyContinuous
The positive-sign conditional block law is dominated by the reference law on almost every retained graph. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
h ∈ Icc 0 (1 / 4)
hq :
q ∈ Icc 0 1
ha :
conclusion
∀ᵐ H ∂(retainedGraphMarginal n B d (μZ.prod (auditLaw (Fin n) q))), conditionalBlockLaw n B d (μZ.prod (auditLaw (Fin n) q)) true h H ≪ hiddenReferenceLaw n B d (μZ.prod (auditLaw (Fin n) q)) h
Proof (Lean source)
-- @node: conditionalBlockLaw_absolutelyContinuous
lemma conditionalBlockLaw_absolutelyContinuous (n B d : ℕ) (h q : ℝ)
(μZ : Measure (Assign (Fin n))) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(hh : h ∈ Icc 0 (1 / 4)) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw (μZ.prod (auditLaw (Fin n) q))) :
∀ᵐ H ∂(retainedGraphMarginal n B d (μZ.prod (auditLaw (Fin n) q))),
conditionalBlockLaw n B d (μZ.prod (auditLaw (Fin n) q)) true h H ≪
hiddenReferenceLaw n B d (μZ.prod (auditLaw (Fin n) q)) h := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
let : IsProbabilityMeasure (halfBernoulli (Fin n)) := by
unfold halfBernoulli
infer_instance
let : IsProbabilityMeasure (μZ.prod (auditLaw (Fin n) q)) := by
constructor
have ht := congrArg (fun μ : Measure (Assign (Fin n)) => μ univ) ha
rw [Measure.map_apply (by fun_prop) MeasurableSet.univ] at ht
simpa using ht
exact Filter.Eventually.of_forall (fun H =>
conditionalBlockLaw_absolutelyContinuous_of_probability n B d hd hfit _ h H)
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.OracleQuadraticEnergy 9 declarations Finite quadratic coefficient energy
Finite quadratic coefficient energy
theorem oracleBeta_mass_le
The augmented coefficient mass is at most one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j ∈ oracleNbhd θ i, |oracleBeta θ i j|) ≤ 1
Proof (Lean source)
-- @node: oracleBeta_mass_le
lemma oracleBeta_mass_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) (i : V) :
(∑ j ∈ oracleNbhd θ i, |oracleBeta θ i j|) ≤ 1 := by
rw [oracleBeta_abs_sum]
have h := hc.coeffMass i
linarith [abs_nonneg (θ.a i)]
theorem oracleBeta_pair_sq_le
A pair square is bounded by its absolute coefficient mass. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
d :
ℕ
hc :
ScheduleClass θ d
i j k :
V
hj :
j ∈ oracleNbhd θ i
hk :
k ∈ (oracleNbhd θ i).erase j
conclusion
Proof (Lean source)
-- @node: oracleBeta_pair_sq_le
lemma oracleBeta_pair_sq_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d)
(i j k : V) (hj : j ∈ oracleNbhd θ i) (hk : k ∈ (oracleNbhd θ i).erase j) :
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 ≤
|oracleBeta θ i j| + |oracleBeta θ i k| := by
have hkj := (Finset.mem_erase.mp hk).1
have hk' := (Finset.mem_erase.mp hk).2
have hp : |oracleBeta θ i j| + |oracleBeta θ i k| ≤ 1 := by
calc
_ = ∑ l ∈ ({j, k} : Finset V), |oracleBeta θ i l| := by
simp [Ne.symm hkj]
_ ≤ ∑ l ∈ oracleNbhd θ i, |oracleBeta θ i l| := by
apply Finset.sum_le_sum_of_subset_of_nonneg
· intro l hl
simp only [Finset.mem_insert, Finset.mem_singleton] at hl
rcases hl with rfl | rfl <;> assumption
· intro l _ _; exact abs_nonneg _
_ ≤ 1 := oracleBeta_mass_le θ d hc i
have ha := abs_add_le (oracleBeta θ i j) (oracleBeta θ i k)
have hn := abs_nonneg (oracleBeta θ i j + oracleBeta θ i k)
nlinarith [sq_abs (oracleBeta θ i j + oracleBeta θ i k)]
theorem oracle_pair_incidence_symm
Swapping the two coordinates preserves the row-pair incidence set. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
j k :
V
conclusion
Proof (Lean source)
-- @node: oracle_pair_incidence_symm
lemma oracle_pair_incidence_symm (θ : Schedule V) (j k : V) :
(Finset.univ.filter fun i => k ∈ oracleNbhd θ i ∧ j ∈ (oracleNbhd θ i).erase k) =
(Finset.univ.filter fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j) := by
ext i
simp only [mem_filter, Finset.mem_univ, true_and, Finset.mem_erase]
constructor <;> rintro ⟨h₁, h₂, h₃⟩ <;> exact ⟨h₃, Ne.symm h₂, h₁⟩
theorem oracle_pair_coefficient_sum
The symmetrized collected coefficient is the rowwise pair-coefficient sum. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
j k :
V
conclusion
oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j
= ∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j), (oracleBeta θ i j + oracleBeta θ i k)
Proof (Lean source)
-- @node: oracle_pair_coefficient_sum
lemma oracle_pair_coefficient_sum (θ : Schedule V) (j k : V) :
oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j =
∑ i ∈ Finset.univ.filter
(fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j),
(oracleBeta θ i j + oracleBeta θ i k) := by
unfold oracleQuadraticCoefficient
rw [oracle_pair_incidence_symm, Finset.sum_add_distrib]
theorem oracle_pair_coefficient_sq_le
Out-degree bounds the number of rows sharing a coordinate pair. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) ^ 2
≤ ((d : ℝ) + 1) * ∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j), (oracleBeta θ i j + oracleBeta θ i k) ^ 2
Proof (Lean source)
-- @node: oracle_pair_coefficient_sq_le
lemma oracle_pair_coefficient_sq_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d)
(j k : V) :
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) ^ 2 ≤
((d : ℝ) + 1) * ∑ i ∈ Finset.univ.filter
(fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j),
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 := by
rw [oracle_pair_coefficient_sum]
have hcard : ((Finset.univ.filter
(fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j)).card : ℝ) ≤
(d : ℝ) + 1 := by
have hs : (Finset.univ.filter
(fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j)).card ≤ d + 1 :=
(Finset.card_le_card (by
intro i hi
exact Finset.mem_filter.mpr ⟨Finset.mem_univ _, (Finset.mem_filter.mp hi).2.1⟩)).trans
(oracleNbhd_blockDegree θ d hc j)
exact_mod_cast hs
exact sq_sum_le_card_mul_sum_sq.trans
(mul_le_mul_of_nonneg_right hcard (Finset.sum_nonneg fun _ _ => sq_nonneg _))
theorem oracle_row_pair_energy_le
Summing the absolute pair masses over a row uses at most twice its augmented degree. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
≤ 2 * ((d : ℝ) + 1)
Proof (Lean source)
-- @node: oracle_row_pair_energy_le
lemma oracle_row_pair_energy_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) (i : V) :
(∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
(oracleBeta θ i j + oracleBeta θ i k) ^ 2) ≤ 2 * ((d : ℝ) + 1) := by
calc
_ ≤ ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
(|oracleBeta θ i j| + |oracleBeta θ i k|) := by
apply Finset.sum_le_sum
intro j hj
exact Finset.sum_le_sum fun k hk => oracleBeta_pair_sq_le θ d hc i j k hj hk
_ ≤ ∑ j ∈ oracleNbhd θ i, ∑ k ∈ oracleNbhd θ i,
(|oracleBeta θ i j| + |oracleBeta θ i k|) := by
apply Finset.sum_le_sum
intro j _
exact Finset.sum_le_sum_of_subset_of_nonneg (Finset.erase_subset _ _) (by
intro k _ _; positivity)
_ = 2 * (oracleNbhd θ i).card * (∑ j ∈ oracleNbhd θ i, |oracleBeta θ i j|) := by
simp_rw [Finset.sum_add_distrib, Finset.sum_const, nsmul_eq_mul]
rw [← Finset.mul_sum]
ring
_ ≤ 2 * ((d : ℝ) + 1) := by
have hcard : ((oracleNbhd θ i).card : ℝ) ≤ (d : ℝ) + 1 := by
exact_mod_cast oracleNbhd_card_le θ d hc i
calc
_ ≤ 2 * ((oracleNbhd θ i).card : ℝ) * 1 :=
mul_le_mul_of_nonneg_left (oracleBeta_mass_le θ d hc i) (by positivity)
_ ≤ 2 * ((d : ℝ) + 1) := by
simpa only [mul_one] using mul_le_mul_of_nonneg_left hcard (by norm_num : (0:ℝ) ≤ 2)
theorem oracle_pair_energy_reindex
Reindexing pairwise row energies preserves all neighborhood coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
(∑ j, ∑ k, ∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j), (oracleBeta θ i j + oracleBeta θ i k) ^ 2)
= ∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j, (oracleBeta θ i j + oracleBeta θ i k) ^ 2
Proof (Lean source)
-- @node: oracle_pair_energy_reindex
lemma oracle_pair_energy_reindex (θ : Schedule V) :
(∑ j, ∑ k, ∑ i ∈ Finset.univ.filter
(fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j),
(oracleBeta θ i j + oracleBeta θ i k) ^ 2) =
∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 := by
simp only [Finset.sum_filter]
have hswap (j : V) :
(∑ k, ∑ i, if j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j then
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 else 0) =
∑ i, ∑ k, if j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j then
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 else 0 := Finset.sum_comm
simp_rw [hswap]
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro i _
have hs (s : Finset V) (f : V → ℝ) :
(∑ j ∈ s, f j) = ∑ j : V, if j ∈ s then f j else 0 := by
rw [← Finset.sum_filter]
congr 1
ext j
simp
rw [hs (oracleNbhd θ i)]
apply Finset.sum_congr rfl
intro j _
by_cases hj : j ∈ oracleNbhd θ i
· simp only [hj, true_and, if_true]
exact (hs _ _).symm
· simp [hj]
theorem oracle_quadratic_energy_symmetrize
Ordered pairs count each symmetrized quadratic coefficient twice. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
(∑ j, ∑ k, (oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) ^ 2)
= 2 * ∑ j, ∑ k, oracleQuadraticCoefficient θ j k * (oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j)
Proof (Lean source)
-- @node: oracle_quadratic_energy_symmetrize
lemma oracle_quadratic_energy_symmetrize (θ : Schedule V) :
(∑ j, ∑ k, (oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) ^ 2) =
2 * ∑ j, ∑ k, oracleQuadraticCoefficient θ j k *
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) := by
have he (j k : V) :
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) ^ 2 =
oracleQuadraticCoefficient θ j k *
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) +
oracleQuadraticCoefficient θ k j *
(oracleQuadraticCoefficient θ k j + oracleQuadraticCoefficient θ j k) := by ring
simp_rw [he, Finset.sum_add_distrib]
rw [Finset.sum_comm (f := fun j k => oracleQuadraticCoefficient θ k j *
(oracleQuadraticCoefficient θ k j + oracleQuadraticCoefficient θ j k))]
ring
theorem oracle_quadratic_energy_le
Finite Cauchy–Schwarz, out-degree multiplicity and row mass control the quadratic energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j, ∑ k, oracleQuadraticCoefficient θ j k * (oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j))
≤ (Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2
Proof (Lean source)
-- @node: oracle_quadratic_energy_le
lemma oracle_quadratic_energy_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) :
(∑ j, ∑ k, oracleQuadraticCoefficient θ j k *
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j)) ≤
(Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2 := by
have h : (∑ j, ∑ k,
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) ^ 2) ≤
2 * (Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2 := by
calc
_ ≤ ∑ j, ∑ k, ((d : ℝ) + 1) * ∑ i ∈ Finset.univ.filter
(fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j),
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 :=
Finset.sum_le_sum fun j _ => Finset.sum_le_sum fun k _ =>
oracle_pair_coefficient_sq_le θ d hc j k
_ = ((d : ℝ) + 1) * ∑ i, ∑ j ∈ oracleNbhd θ i,
∑ k ∈ (oracleNbhd θ i).erase j,
(oracleBeta θ i j + oracleBeta θ i k) ^ 2 := by
simp_rw [← Finset.mul_sum]
rw [oracle_pair_energy_reindex]
_ ≤ ((d : ℝ) + 1) * ∑ _i : V, 2 * ((d : ℝ) + 1) :=
mul_le_mul_of_nonneg_left
(Finset.sum_le_sum fun i _ => oracle_row_pair_energy_le θ d hc i) (by positivity)
_ = _ := by simp; ring
rw [oracle_quadratic_energy_symmetrize] at h
linarith
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.OracleVariance 1 declarations Supplied-graph score variance bound
Supplied-graph score variance bound
theorem oracle_variance_le
The supplied-graph additive score has variance at most five times squared augmented degree over population size. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
≤ 5 * ((d : ℝ) + 1) ^ 2 / card V
Proof (Lean source)
-- @node: oracle_variance_le
lemma oracle_variance_le (θ : Schedule V) (d : ℕ) (hn : 4 ≤ Fintype.card V)
(hd : 1 ≤ d) (hclass : ScheduleClass θ d) :
variance (oracleScore θ) (halfBernoulli V) ≤
5 * ((d : ℝ) + 1) ^ 2 / Fintype.card V := by
rw [ProbabilityTheory.variance_eq_integral (by fun_prop)]
rw [integral_oracleScore_eq_tte]
change (∫ z, (oracleScore θ z - tte θ) ^ 2 ∂halfBernoulli V) ≤ _
simp_rw [oracleScore_centered_expansion]
let L : Assign V → ℝ := fun z => (2 / (Fintype.card V : ℝ)) *
(∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j))
let Q : Assign V → ℝ := fun z => (Fintype.card V : ℝ)⁻¹ *
∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k)
change (∫ z, (L z + Q z) ^ 2 ∂halfBernoulli V) ≤ _
have hsplit : (∫ z, (L z + Q z) ^ 2 ∂halfBernoulli V) =
(∫ z, L z ^ 2 ∂halfBernoulli V) +
(∫ z, 2 * L z * Q z ∂halfBernoulli V) +
(∫ z, Q z ^ 2 ∂halfBernoulli V) := by
simp_rw [show ∀ z, (L z + Q z) ^ 2 = L z ^ 2 + 2 * L z * Q z + Q z ^ 2 by
intro z; ring]
rw [integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _)]
have hmixed : (∫ z, 2 * L z * Q z ∂halfBernoulli V) = 0 := by
have he : (fun z => 2 * L z * Q z) = fun z =>
(2 * (2 / (Fintype.card V : ℝ)) * (Fintype.card V : ℝ)⁻¹) *
((∑ i, oracleMu θ i * ∑ l ∈ oracleNbhd θ i, signOf (z l)) *
(∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k))) := by
funext z
dsimp [L, Q]
ring
rw [he, integral_const_mul, oracle_mixed_second_moment_zero, mul_zero]
rw [hsplit, hmixed, add_zero]
refine le_trans (add_le_add
(oracle_linear_scaled_second_moment_le θ d hn hclass) le_rfl) ?_
change 4 * ((d : ℝ) + 1) ^ 2 / Fintype.card V +
(∫ z, ((Fintype.card V : ℝ)⁻¹ *
∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k)) ^ 2 ∂halfBernoulli V) ≤ _
simp_rw [mul_pow]
rw [integral_const_mul, oracle_quadratic_second_moment]
have hn0 : (0 : ℝ) < Fintype.card V := by
exact_mod_cast (by omega : 0 < Fintype.card V)
calc
_ ≤ 4 * ((d : ℝ) + 1) ^ 2 / Fintype.card V +
(Fintype.card V : ℝ)⁻¹ ^ 2 *
((Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2) :=
add_le_add le_rfl
(mul_le_mul_of_nonneg_left (oracle_quadratic_energy_le θ d hclass) (sq_nonneg _))
_ = _ := by field_simp; ring
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.OracleWalsh 29 declarations Oracle variance chain
Oracle variance chain
def oracleNbhd
The compulsory own coordinate adjoined to the off-diagonal sources.
def oracleBeta
The additive coefficient on an augmented source coordinate.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
i j :
V
def
oracleBeta θ i j :
ℝ
given by
if j = i then θ.t i else θ.b i j
def oracleMu
The assignment-averaged row outcome in the centered-sign expansion.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
i :
V
def
oracleMu θ i :
ℝ
given by
θ.a i + (∑ j ∈ oracleNbhd θ i, oracleBeta θ i j) / 2
theorem oracle_self_not_mem
No own coordinate occurs among the off-diagonal sources. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: oracle_self_not_mem
lemma oracle_self_not_mem (θ : Schedule V) (i : V) : i ∉ inNbhd θ i := by
simp [inNbhd, θ.irrefl i]
theorem oracleBeta_sum
Summing augmented coefficients separates the own treatment coefficient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
i :
V
f :
V → ℝ
conclusion
(∑ j ∈ oracleNbhd θ i, oracleBeta θ i j * f j)
= θ.t i * f i + ∑ j ∈ inNbhd θ i, θ.b i j * f j
Proof (Lean source)
-- @node: oracleBeta_sum
lemma oracleBeta_sum (θ : Schedule V) (i : V) (f : V → ℝ) :
(∑ j ∈ oracleNbhd θ i, oracleBeta θ i j * f j) =
θ.t i * f i + ∑ j ∈ inNbhd θ i, θ.b i j * f j := by
rw [oracleNbhd, Finset.sum_insert (oracle_self_not_mem θ i)]
simp only [oracleBeta, ite_true]
congr 1
apply Finset.sum_congr rfl
intro j hj
have hji : j ≠ i := fun he => oracle_self_not_mem θ i (he ▸ hj)
simp [hji]
theorem oracleBeta_abs_sum
The augmented coefficient mass is precisely the own plus spillover mass. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
i :
V
conclusion
Proof (Lean source)
-- @node: oracleBeta_abs_sum
lemma oracleBeta_abs_sum (θ : Schedule V) (i : V) :
(∑ j ∈ oracleNbhd θ i, |oracleBeta θ i j|) =
|θ.t i| + ∑ j ∈ inNbhd θ i, |θ.b i j| := by
rw [oracleNbhd, Finset.sum_insert (oracle_self_not_mem θ i)]
simp only [oracleBeta, ite_true]
congr 1
apply Finset.sum_congr rfl
intro j hj
have hji : j ≠ i := fun he => oracle_self_not_mem θ i (he ▸ hj)
simp [hji]
theorem oracleNbhd_card_le
The augmented row has at most d+1 coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(oracleNbhd θ i).card ≤ d + 1
Proof (Lean source)
-- @node: oracleNbhd_card_le
lemma oracleNbhd_card_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) (i : V) :
(oracleNbhd θ i).card ≤ d + 1 := by
rw [oracleNbhd, Finset.card_insert_of_notMem (oracle_self_not_mem θ i)]
exact Nat.add_le_add_right (hc.inDegree i) 1
theorem oracleNbhd_blockDegree
Adjoining the diagonal increases coordinate multiplicity by at most one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
BlockDegreeLE (oracleNbhd θ) (d + 1)
Proof (Lean source)
-- @node: oracleNbhd_blockDegree
lemma oracleNbhd_blockDegree (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) :
BlockDegreeLE (oracleNbhd θ) (d + 1) := by
intro j
have he : (Finset.univ.filter fun i => j ∈ oracleNbhd θ i) =
insert j (Finset.univ.filter fun i => j ∈ inNbhd θ i) := by
ext i
simp [oracleNbhd, eq_comm]
rw [he, Finset.card_insert_of_notMem]
· exact Nat.add_le_add_right (hc.outDegree j) 1
· simp [oracle_self_not_mem θ j]
theorem oracleMu_abs_le
The centered row baseline has magnitude at most one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
|oracleMu θ i| ≤ 1
Proof (Lean source)
-- @node: oracleMu_abs_le
lemma oracleMu_abs_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) (i : V) :
|oracleMu θ i| ≤ 1 := by
have hs := Finset.abs_sum_le_sum_abs (fun j => oracleBeta θ i j) (oracleNbhd θ i)
have hm := hc.coeffMass i
rw [add_assoc, ← oracleBeta_abs_sum] at hm
calc
|oracleMu θ i| ≤ |θ.a i| + |∑ j ∈ oracleNbhd θ i, oracleBeta θ i j| / 2 := by
simpa [oracleMu, abs_div] using abs_add_le (θ.a i)
((∑ j ∈ oracleNbhd θ i, oracleBeta θ i j) / 2)
_ ≤ 1 := by linarith [abs_nonneg (∑ j ∈ oracleNbhd θ i, oracleBeta θ i j)]
theorem potentialOutcome_oracle_centered
Substituting treatment=(1+sign)/2 gives the actual centered row response. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
potentialOutcome θ i z
Proof (Lean source)
-- @node: potentialOutcome_oracle_centered
lemma potentialOutcome_oracle_centered (θ : Schedule V) (i : V) (z : Assign V) :
potentialOutcome θ i z = oracleMu θ i +
(∑ j ∈ oracleNbhd θ i, oracleBeta θ i j * signOf (z j)) / 2 := by
have hz (j : V) : treatment (z j) = (1 + signOf (z j)) / 2 := by
cases z j <;> norm_num [treatment, signOf]
have ho : potentialOutcome θ i z = θ.a i +
∑ j ∈ oracleNbhd θ i, oracleBeta θ i j * treatment (z j) := by
rw [oracleBeta_sum]
change θ.a i + θ.t i * treatment (z i) +
(∑ j ∈ inNbhd θ i, θ.b i j * treatment (z j)) = _
ring
rw [ho]
simp_rw [hz, mul_div, mul_add, mul_one]
rw [← Finset.sum_div, Finset.sum_add_distrib, oracleMu]
ring
theorem oracleNbhd_sign_sum
The sum of augmented signs is the recorded supplied-graph weight. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j ∈ oracleNbhd θ i, signOf (z j)) = signOf (z i) + ∑ j ∈ inNbhd θ i, signOf (z j)
Proof (Lean source)
-- @node: oracleNbhd_sign_sum
lemma oracleNbhd_sign_sum (θ : Schedule V) (i : V) (z : Assign V) :
(∑ j ∈ oracleNbhd θ i, signOf (z j)) =
signOf (z i) + ∑ j ∈ inNbhd θ i, signOf (z j) := by
exact Finset.sum_insert (oracle_self_not_mem θ i)
theorem tte_oracleBeta_sum
The constant coefficient of the score is the all-treated contrast. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
Proof (Lean source)
-- @node: tte_oracleBeta_sum
lemma tte_oracleBeta_sum (θ : Schedule V) :
tte θ = (Fintype.card V : ℝ)⁻¹ *
∑ i, ∑ j ∈ oracleNbhd θ i, oracleBeta θ i j := by
unfold tte
congr 1
apply Finset.sum_congr rfl
intro i _
have hb := oracleBeta_sum θ i (fun _ => 1)
simp only [mul_one] at hb
rw [hb]
simp [potentialOutcome, treatment]
ring
theorem oracle_row_quadratic_split
Removing the diagonal sign squares leaves a genuinely quadratic, distinct-coordinate term. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (∑ j ∈ oracleNbhd θ i, oracleBeta θ i j)
+ ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j, oracleBeta θ i j * signOf (z j) * signOf (z k)
Proof (Lean source)
-- @node: oracle_row_quadratic_split
lemma oracle_row_quadratic_split (θ : Schedule V) (i : V) (z : Assign V) :
(∑ j ∈ oracleNbhd θ i, oracleBeta θ i j * signOf (z j)) *
(∑ k ∈ oracleNbhd θ i, signOf (z k)) =
(∑ j ∈ oracleNbhd θ i, oracleBeta θ i j) +
∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k) := by
rw [Finset.sum_mul, ← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro j hj
rw [Finset.mul_sum, ← Finset.sum_erase_add _ _ hj]
have hs : signOf (z j) * signOf (z j) = 1 := by
cases z j <;> norm_num [signOf]
simp only [mul_assoc, hs, mul_one]
ring
theorem oracleScore_centered_expansion
Exact degree-at-most-two expansion of the centered supplied-graph score. The quadratic sum uses ordered pairs so it requires no arbitrary ordering of the population. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
oracleScore θ z - tte θ
= (2 / (Fintype.card V : ℝ)) * (∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j))
+ (Fintype.card V : ℝ)⁻¹ * ∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j, oracleBeta θ i j * signOf (z j) * signOf (z k)
Proof (Lean source)
-- @node: oracleScore_centered_expansion
lemma oracleScore_centered_expansion (θ : Schedule V) (z : Assign V) :
oracleScore θ z - tte θ =
(2 / (Fintype.card V : ℝ)) *
(∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j)) +
(Fintype.card V : ℝ)⁻¹ *
∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k) := by
have hrow (i : V) :
potentialOutcome θ i z * (∑ j ∈ oracleNbhd θ i, signOf (z j)) =
oracleMu θ i * (∑ j ∈ oracleNbhd θ i, signOf (z j)) +
((∑ j ∈ oracleNbhd θ i, oracleBeta θ i j) +
∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k)) / 2 := by
rw [potentialOutcome_oracle_centered, add_mul, div_mul_eq_mul_div,
oracle_row_quadratic_split]
rw [oracleScore, tte_oracleBeta_sum]
simp_rw [← oracleNbhd_sign_sum, hrow]
simp_rw [add_div]
simp only [Finset.sum_add_distrib, ← Finset.sum_div]
ring
def oracleLinearCoefficient
The collected coefficient of a source sign in the oracle score.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
j :
V
def
oracleLinearCoefficient θ j :
ℝ
given by
∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i), oracleMu θ i
theorem oracle_linear_collect
Collecting the linear terms by source preserves all augmented neighborhoods. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j))
= ∑ j, oracleLinearCoefficient θ j * signOf (z j)
Proof (Lean source)
-- @node: oracle_linear_collect
lemma oracle_linear_collect (θ : Schedule V) (z : Assign V) :
(∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j)) =
∑ j, oracleLinearCoefficient θ j * signOf (z j) := by
have hs (i : V) : (∑ j ∈ oracleNbhd θ i, signOf (z j)) =
∑ j : V, if j ∈ oracleNbhd θ i then signOf (z j) else 0 := by
rw [← Finset.sum_filter]
congr 1
ext j
simp
simp_rw [hs]
simp only [oracleLinearCoefficient, Finset.sum_mul, Finset.mul_sum, Finset.sum_filter,
mul_ite, ite_mul, mul_zero, zero_mul]
rw [Finset.sum_comm]
Two centered signs are orthonormal under the actual assignment measure. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
j k :
V
conclusion
(∫ z, signOf (z j) * signOf (z k) ∂halfBernoulli V) = if j = k then 1 else 0
Proof (Lean source)
-- @node: integral_oracle_sign_pair
lemma integral_oracle_sign_pair (j k : V) :
(∫ z, signOf (z j) * signOf (z k) ∂halfBernoulli V) =
if j = k then 1 else 0 := by
have he : (fun z : Assign V => signOf (z j) * signOf (z k)) =
fun z => 2 * (treatment (z k) * signOf (z j)) - signOf (z j) := by
funext z
cases z j <;> cases z k <;> norm_num [signOf, treatment]
rw [he, integral_sub (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_const_mul, integral_treatment_mul_signOf, integral_signOf_halfBernoulli]
by_cases h : j = k
· simp [h]
· simp [h, Ne.symm h]
theorem oracle_linear_second_moment
Orthogonality gives the exact second moment of the linear sign polynomial. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
(∫ z, (∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j)) ^ 2 ∂halfBernoulli V)
= ∑ j, oracleLinearCoefficient θ j ^ 2
Proof (Lean source)
-- @node: oracle_linear_second_moment
lemma oracle_linear_second_moment (θ : Schedule V) :
(∫ z, (∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j)) ^ 2
∂halfBernoulli V) = ∑ j, oracleLinearCoefficient θ j ^ 2 := by
simp_rw [oracle_linear_collect, pow_two, Finset.sum_mul, Finset.mul_sum]
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_congr rfl
intro j _
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
have he (k : V) :
(fun z : Assign V => (oracleLinearCoefficient θ j * signOf (z j)) *
(oracleLinearCoefficient θ k * signOf (z k))) =
fun z => (oracleLinearCoefficient θ j * oracleLinearCoefficient θ k) *
(signOf (z j) * signOf (z k)) := by funext z; ring
simp_rw [he, integral_const_mul, integral_oracle_sign_pair]
simp [mul_ite]
theorem oracle_linear_energy_le
Cauchy-Schwarz and the out-degree bound control the linear coefficient energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j, oracleLinearCoefficient θ j ^ 2) ≤ (Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2
Proof (Lean source)
-- @node: oracle_linear_energy_le
lemma oracle_linear_energy_le (θ : Schedule V) (d : ℕ) (hc : ScheduleClass θ d) :
(∑ j, oracleLinearCoefficient θ j ^ 2) ≤
(Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2 := by
have hmu (i : V) : oracleMu θ i ^ 2 ≤ 1 := by
have h := oracleMu_abs_le θ d hc i
have h' := abs_nonneg (oracleMu θ i)
nlinarith [sq_abs (oracleMu θ i)]
have hdegree := oracleNbhd_blockDegree θ d hc
have hrow (j : V) : oracleLinearCoefficient θ j ^ 2 ≤
((d : ℝ) + 1) * ∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i),
oracleMu θ i ^ 2 := by
have hcard : ((Finset.univ.filter (fun i => j ∈ oracleNbhd θ i)).card : ℝ) ≤
(d : ℝ) + 1 := by exact_mod_cast hdegree j
exact (show oracleLinearCoefficient θ j ^ 2 ≤
((Finset.univ.filter (fun i => j ∈ oracleNbhd θ i)).card : ℝ) *
∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i), oracleMu θ i ^ 2 from
sq_sum_le_card_mul_sum_sq).trans
(mul_le_mul_of_nonneg_right hcard (Finset.sum_nonneg fun _ _ => sq_nonneg _))
calc
(∑ j, oracleLinearCoefficient θ j ^ 2) ≤
∑ j, ((d : ℝ) + 1) *
∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i), oracleMu θ i ^ 2 :=
Finset.sum_le_sum fun j _ => hrow j
_ = ((d : ℝ) + 1) * ∑ i, (oracleNbhd θ i).card * oracleMu θ i ^ 2 := by
rw [← Finset.mul_sum]
congr 1
simp only [Finset.sum_filter]
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro i _
simp [Finset.univ_inter]
_ ≤ ((d : ℝ) + 1) * ∑ _i : V, ((d : ℝ) + 1) := by
apply mul_le_mul_of_nonneg_left _ (by positivity)
apply Finset.sum_le_sum
intro i _
have hcard : ((oracleNbhd θ i).card : ℝ) ≤ (d : ℝ) + 1 := by
exact_mod_cast oracleNbhd_card_le θ d hc i
exact (mul_le_mul_of_nonneg_left (hmu i) (by positivity)).trans (by simpa using hcard)
_ = _ := by simp; ring
theorem oracle_linear_scaled_second_moment_le
The complete linear contribution to the oracle variance is at most 4D²/n. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ z, ((2 / (Fintype.card V : ℝ)) * (∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j))) ^ 2 ∂halfBernoulli V)
≤ 4 * ((d : ℝ) + 1) ^ 2 / Fintype.card V
Proof (Lean source)
-- @node: oracle_linear_scaled_second_moment_le
lemma oracle_linear_scaled_second_moment_le (θ : Schedule V) (d : ℕ)
(hn : 4 ≤ Fintype.card V) (hc : ScheduleClass θ d) :
(∫ z, ((2 / (Fintype.card V : ℝ)) *
(∑ i, oracleMu θ i * ∑ j ∈ oracleNbhd θ i, signOf (z j))) ^ 2
∂halfBernoulli V) ≤ 4 * ((d : ℝ) + 1) ^ 2 / Fintype.card V := by
have hn0 : (0 : ℝ) < Fintype.card V := by exact_mod_cast (by omega : 0 < Fintype.card V)
simp_rw [mul_pow]
rw [integral_const_mul, oracle_linear_second_moment]
calc
(2 / (Fintype.card V : ℝ)) ^ 2 * (∑ j, oracleLinearCoefficient θ j ^ 2) ≤
(2 / (Fintype.card V : ℝ)) ^ 2 *
((Fintype.card V : ℝ) * ((d : ℝ) + 1) ^ 2) :=
mul_le_mul_of_nonneg_left (oracle_linear_energy_le θ d hc) (sq_nonneg _)
_ = _ := by field_simp; ring
A singleton sign is orthogonal to every distinct-coordinate quadratic monomial. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
l j k :
V
hjk :
j ≠ k
conclusion
(∫ z, signOf (z l) * (signOf (z j) * signOf (z k)) ∂halfBernoulli V) = 0
Proof (Lean source)
-- @node: integral_oracle_sign_triple
lemma integral_oracle_sign_triple (l j k : V) (hjk : j ≠ k) :
(∫ z, signOf (z l) * (signOf (z j) * signOf (z k))
∂halfBernoulli V) = 0 := by
have hsets : ({l} : Finset V) ≠ {j, k} := by
intro he
have hj : j = l := by
have : j ∈ ({l} : Finset V) := he ▸ (by simp : j ∈ ({j, k} : Finset V))
simpa using this
have hk : k = l := by
have : k ∈ ({l} : Finset V) := he ▸ (by simp : k ∈ ({j, k} : Finset V))
simpa using this
exact hjk (hj.trans hk.symm)
simpa [Finset.prod_insert, hjk, hsets] using
signProduct_orthogonality ({l} : Finset V) {j, k}
theorem oracle_mixed_second_moment_zero
The actual linear and quadratic parts of the supplied score have zero cross moment. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
(∫ z, (∑ i, oracleMu θ i * ∑ l ∈ oracleNbhd θ i, signOf (z l)) * (∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j, oracleBeta θ i j * signOf (z j) * signOf (z k)) ∂halfBernoulli V)
= 0
Proof (Lean source)
-- @node: oracle_mixed_second_moment_zero
lemma oracle_mixed_second_moment_zero (θ : Schedule V) :
(∫ z, (∑ i, oracleMu θ i * ∑ l ∈ oracleNbhd θ i, signOf (z l)) *
(∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k))
∂halfBernoulli V) = 0 := by
simp_rw [oracle_linear_collect, Finset.sum_mul, Finset.mul_sum]
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_eq_zero
intro l _
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_eq_zero
intro i _
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_eq_zero
intro j hj
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_eq_zero
intro k hk
have he : (fun z : Assign V => (oracleLinearCoefficient θ l * signOf (z l)) *
(oracleBeta θ i j * signOf (z j) * signOf (z k))) =
fun z => (oracleLinearCoefficient θ l * oracleBeta θ i j) *
(signOf (z l) * (signOf (z j) * signOf (z k))) := by funext z; ring
rw [he, integral_const_mul, integral_oracle_sign_triple l j k
(Ne.symm (Finset.mem_erase.mp hk).1), mul_zero]
def oracleQuadraticCoefficient
The ordered-pair coefficient retains the true augmented-neighborhood incidences.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
j k :
V
def
oracleQuadraticCoefficient θ j k :
ℝ
given by
∑ i ∈ Finset.univ.filter (fun i => j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j), oracleBeta θ i j
theorem oracleQuadraticCoefficient_diag
Diagonal coordinates have already contributed to the causal target. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: oracleQuadraticCoefficient_diag
lemma oracleQuadraticCoefficient_diag (θ : Schedule V) (j : V) :
oracleQuadraticCoefficient θ j j = 0 := by
simp [oracleQuadraticCoefficient]
theorem oracle_quadratic_collect
Collecting the quadratic monomials by ordered source pair changes no graph incidence. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j, oracleBeta θ i j * signOf (z j) * signOf (z k))
= ∑ j, ∑ k, oracleQuadraticCoefficient θ j k * signOf (z j) * signOf (z k)
Proof (Lean source)
-- @node: oracle_quadratic_collect
lemma oracle_quadratic_collect (θ : Schedule V) (z : Assign V) :
(∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k)) =
∑ j, ∑ k, oracleQuadraticCoefficient θ j k * signOf (z j) * signOf (z k) := by
have hs (s : Finset V) (f : V → ℝ) :
(∑ j ∈ s, f j) = ∑ j : V, if j ∈ s then f j else 0 := by
rw [← Finset.sum_filter]
congr 1
ext j
simp
have hrow (i : V) :
(∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k)) =
∑ j : V, ∑ k : V, if j ∈ oracleNbhd θ i ∧ k ∈ (oracleNbhd θ i).erase j
then oracleBeta θ i j * signOf (z j) * signOf (z k) else 0 := by
rw [hs]
apply Finset.sum_congr rfl
intro j _
by_cases hj : j ∈ oracleNbhd θ i
· rw [if_pos hj, hs]
apply Finset.sum_congr rfl
intro k _
simp only [hj, true_and]
· simp [hj]
simp_rw [hrow]
simp only [oracleQuadraticCoefficient, Finset.sum_filter, Finset.sum_mul,
ite_mul, zero_mul]
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro j _
rw [Finset.sum_comm]
Equal unordered pairs are precisely the two orientations of an ordered pair. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
j k a b :
V
conclusion
({j, k} : Finset V) = {a, b} ↔ (j = a ∧ k = b) ∨ (j = b ∧ k = a)
Proof (Lean source)
-- @node: oracle_pair_eq_iff
lemma oracle_pair_eq_iff (j k a b : V) :
({j, k} : Finset V) = {a, b} ↔ (j = a ∧ k = b) ∨ (j = b ∧ k = a) := by
rw [← Finset.coe_inj]
simpa only [Finset.coe_pair] using
(Set.pair_eq_pair_iff : ({j, k} : Set V) = {a, b} ↔ _)
The fourth sign moment matches the two orientations, with no other surviving pairs. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
j k a b :
V
hjk :
j ≠ k
hab :
a ≠ b
conclusion
(∫ z, (signOf (z j) * signOf (z k)) * (signOf (z a) * signOf (z b)) ∂halfBernoulli V)
= if (j = a ∧ k = b) ∨ (j = b ∧ k = a) then 1 else 0
Proof (Lean source)
-- @node: integral_oracle_sign_four
lemma integral_oracle_sign_four (j k a b : V) (hjk : j ≠ k) (hab : a ≠ b) :
(∫ z, (signOf (z j) * signOf (z k)) * (signOf (z a) * signOf (z b))
∂halfBernoulli V) =
if (j = a ∧ k = b) ∨ (j = b ∧ k = a) then 1 else 0 := by
simpa [Finset.prod_insert, hjk, hab, oracle_pair_eq_iff] using
signProduct_orthogonality ({j, k} : Finset V) {a, b}
theorem oracle_quadratic_inner_moment
Summing the fourth moments leaves both orientations of each non-diagonal pair. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
j k :
V
hjk :
j ≠ k
conclusion
(∫ z, (signOf (z j) * signOf (z k)) * (∑ a, ∑ b, oracleQuadraticCoefficient θ a b * signOf (z a) * signOf (z b)) ∂halfBernoulli V)
= oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j
Proof (Lean source)
-- @node: oracle_quadratic_inner_moment
lemma oracle_quadratic_inner_moment (θ : Schedule V) (j k : V) (hjk : j ≠ k) :
(∫ z, (signOf (z j) * signOf (z k)) *
(∑ a, ∑ b, oracleQuadraticCoefficient θ a b * signOf (z a) * signOf (z b))
∂halfBernoulli V) =
oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j := by
simp_rw [Finset.mul_sum]
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
have hterm (a b : V) :
(∫ z, (signOf (z j) * signOf (z k)) *
(oracleQuadraticCoefficient θ a b * signOf (z a) * signOf (z b))
∂halfBernoulli V) =
(if j = a ∧ k = b then oracleQuadraticCoefficient θ a b else 0) +
(if j = b ∧ k = a then oracleQuadraticCoefficient θ a b else 0) := by
by_cases hab : a = b
· subst b
simp only [oracleQuadraticCoefficient_diag, zero_mul, mul_zero, integral_zero]
simp
· have he : (fun z : Assign V => (signOf (z j) * signOf (z k)) *
(oracleQuadraticCoefficient θ a b * signOf (z a) * signOf (z b))) =
fun z => oracleQuadraticCoefficient θ a b *
((signOf (z j) * signOf (z k)) * (signOf (z a) * signOf (z b))) := by
funext z; ring
rw [he, integral_const_mul, integral_oracle_sign_four j k a b hjk hab]
by_cases h1 : j = a ∧ k = b <;> by_cases h2 : j = b ∧ k = a
· exact False.elim (hjk (h1.1.trans h2.2.symm))
all_goals simp only [h1, h2, true_or, or_true, false_or, or_false,
if_true, if_false, mul_one, mul_zero, add_zero, zero_add,
hab, Ne.symm hab, and_self, false_and, and_false]
simp_rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _), hterm,
Finset.sum_add_distrib]
simp_rw [ite_and]
simp [Finset.sum_ite_eq, Finset.sum_ite_eq']
theorem oracle_quadratic_second_moment
Exact quadratic energy: ordered coefficients pair with their reverse orientation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
(∫ z, (∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j, oracleBeta θ i j * signOf (z j) * signOf (z k)) ^ 2 ∂halfBernoulli V)
= ∑ j, ∑ k, oracleQuadraticCoefficient θ j k * (oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j)
Proof (Lean source)
-- @node: oracle_quadratic_second_moment
lemma oracle_quadratic_second_moment (θ : Schedule V) :
(∫ z, (∑ i, ∑ j ∈ oracleNbhd θ i, ∑ k ∈ (oracleNbhd θ i).erase j,
oracleBeta θ i j * signOf (z j) * signOf (z k)) ^ 2 ∂halfBernoulli V) =
∑ j, ∑ k, oracleQuadraticCoefficient θ j k *
(oracleQuadraticCoefficient θ j k + oracleQuadraticCoefficient θ k j) := by
simp_rw [oracle_quadratic_collect, pow_two, Finset.sum_mul]
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_congr rfl
intro j _
rw [integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
apply Finset.sum_congr rfl
intro k _
by_cases hjk : j = k
· subst k
simp [oracleQuadraticCoefficient_diag]
· have he : (fun z : Assign V =>
(oracleQuadraticCoefficient θ j k * signOf (z j) * signOf (z k)) *
(∑ a, ∑ b, oracleQuadraticCoefficient θ a b * signOf (z a) * signOf (z b))) =
fun z => oracleQuadraticCoefficient θ j k *
((signOf (z j) * signOf (z k)) *
(∑ a, ∑ b, oracleQuadraticCoefficient θ a b * signOf (z a) * signOf (z b))) := by
funext z; ring
rw [he, integral_const_mul, oracle_quadratic_inner_moment θ j k hjk]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.PartitionResponseDensity 10 declarations Before averaging compatible source partitions, the actual responses are independent translations of the prior baselines.
Fixed-partition density of the original hidden-allocation experiment
Before averaging compatible source partitions, the actual responses are independent translations of the prior baselines. The identities here preserve the complete retained graph and every assignment coordinate for either sign.
def partitionResponseShift
The actual distinct response when all block baselines are zero.
Definition (Lean source)
parameters
def
given by
theorem distinctResponses_record_translation
Adding independent baselines adds them coordinatewise to the actual distinct responses. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= fun ℓ => U ℓ + partitionResponseShift n B d σ h s ω ℓ
Proof (Lean source)
-- @node: distinctResponses_record_translation
lemma distinctResponses_record_translation (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (U : Fin B → ℝ) :
distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, U)) ω) =
fun ℓ => U ℓ + partitionResponseShift n B d σ h s ω ℓ := by
obtain ⟨ε, hε⟩ := distinctResponses_record_row_signs n B d hd hfit σ h s ω
have hzero := hε (fun _ => 0)
unfold partitionResponseShift
rw [hε U, hzero]
simp only [zero_add]
def partitionResponseDensity
Product baseline densities translated by the actual fixed-partition response shifts.
Definition (Lean source)
parameters
def
partitionResponseDensity n B d σ h s ω y :
ℝ
given by
∏ ℓ, cosSqDensity (y ℓ - partitionResponseShift n B d σ h s ω ℓ)
theorem partitionResponseDensity_measurable
The fixed-partition density is measurable jointly in the full design and responses.
Formal statement
hypotheses
conclusion
Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) => partitionResponseDensity n B d σ h s x.1 x.2)
Proof (Lean source)
-- @node: partitionResponseDensity_measurable
@[fun_prop]
lemma partitionResponseDensity_measurable (n B d : ℕ) (σ : Bool) (h : ℝ)
(s : SourcePartition B d) :
Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
partitionResponseDensity n B d σ h s x.1 x.2) := by
apply measurable_from_prod_countable_right
intro ω
unfold partitionResponseDensity cosSqDensity
apply Finset.measurable_prod
intro ℓ _
apply Measurable.ite (measurableSet_le (by fun_prop) measurable_const) <;> fun_prop
theorem partition_response_density_law
Conditional on any fixed partition and complete design draw, independent prior baselines have exactly the translated product response density, for both signs. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(blockBaselineLaw B).map (fun U => distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, U)) ω))
= volume.withDensity (fun y => ofReal (partitionResponseDensity n B d σ h s ω y))
Proof (Lean source)
-- @node: partition_response_density_law
lemma partition_response_density_law (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) :
(blockBaselineLaw B).map (fun U =>
distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, U)) ω)) =
volume.withDensity (fun y => ofReal (partitionResponseDensity n B d σ h s ω y)) := by
simp_rw [distinctResponses_record_translation n B d hd hfit σ h s ω]
exact blockBaselineLaw_map_translation B (partitionResponseShift n B d σ h s ω)
theorem partition_record_translation
Copying the shifted baselines reconstructs the original record for any actual design realization; detailed edges and ancillary assignments remain in the record. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
recordOf (blockSchedule n B d σ h (s, U)) ω
= copyOutcomes n B d ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1) (fun ℓ => U ℓ + partitionResponseShift n B d σ h s ω ℓ)
Proof (Lean source)
-- @node: partition_record_translation
lemma partition_record_translation (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (U : Fin B → ℝ) :
recordOf (blockSchedule n B d σ h (s, U)) ω =
copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1)
(fun ℓ => U ℓ + partitionResponseShift n B d σ h s ω ℓ) := by
rw [← distinctResponses_record_translation n B d hd hfit σ h s ω U]
exact (copyOutcomes_distinctResponses_record n B d hd hfit σ h (s, U) ω).symm
theorem partition_densityJoint_eq_product_map
Integrating the fixed-partition density against the complete design is equivalent to independent baselines followed by their actual response translations. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (D.prod (blockBaselineLaw B)).map (fun x => (x.1, distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, x.2)) x.1)))
Proof (Lean source)
-- @node: partition_densityJoint_eq_product_map
lemma partition_densityJoint_eq_product_map (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) :
densityJoint D volume (partitionResponseDensity n B d σ h s) =
(D.prod (blockBaselineLaw B)).map (fun x => (x.1,
distinctResponses n B d (recordOf (blockSchedule n B d σ h (s, x.2)) x.1))) := by
let := blockBaselineLaw_probability B
have hm : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
(x.1, distinctResponses n B d
(recordOf (blockSchedule n B d σ h (s, x.2)) x.1))) := by
apply measurable_fst.prodMk
exact (distinctResponses_measurable n B d).comp
((block_record_measurable n B d σ h).comp
((measurable_const.prodMk measurable_snd).prodMk measurable_fst))
ext E hE
rw [densityJoint, Measure.bind_apply hE
(measurable_densityJoint_fiber volume _
(partitionResponseDensity_measurable n B d σ h s)).aemeasurable,
Measure.map_apply hm hE, Measure.prod_apply (hm hE)]
apply lintegral_congr
intro ω
rw [← partition_response_density_law n B d hd hfit σ h s ω,
Measure.map_map measurable_prodMk_left (by
exact (distinctResponses_measurable n B d).comp
((block_record_measurable n B d σ h).comp
((measurable_const.prodMk measurable_id).prodMk measurable_const))),
Measure.map_apply (measurable_prodMk_left.comp (by
exact (distinctResponses_measurable n B d).comp
((block_record_measurable n B d σ h).comp
((measurable_const.prodMk measurable_id).prodMk measurable_const)))) hE]
rfl
theorem partition_mixture_density_representation
For a fixed source partition, the complete original-record mixture is exactly obtained by drawing the actual design and the translated product response density, then copying. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (densityJoint D volume (partitionResponseDensity n B d σ h s)).map (fun x => copyOutcomes n B d ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1) x.2)
Proof (Lean source)
-- @node: partition_mixture_density_representation
lemma partition_mixture_density_representation (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d σ h (s, U)) =
(densityJoint D volume (partitionResponseDensity n B d σ h s)).map
(fun x => copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1)
x.2) := by
let := blockBaselineLaw_probability B
have hr : Measurable (fun x : (Fin B → ℝ) × (Assign (Fin n) × Audit (Fin n)) =>
recordOf (blockSchedule n B d σ h (s, x.1)) x.2) :=
(block_record_measurable n B d σ h).comp
((measurable_const.prodMk measurable_fst).prodMk measurable_snd)
have hc : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1) x.2) := by
let f : ((Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ)) →
(OffDiag (Fin n) → Bool) × (Assign (Fin n) × (Fin B → ℝ)) := fun x =>
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1, x.2)
have hf : Measurable f := by
apply prodMk
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1)).comp measurable_fst
· exact measurable_fst.fst.prodMk measurable_snd
exact (copyOutcomes_measurable n B d).comp hf
have hm : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
(x.1, distinctResponses n B d
(recordOf (blockSchedule n B d σ h (s, x.2)) x.1))) :=
measurable_fst.prodMk ((distinctResponses_measurable n B d).comp (hr.comp measurable_swap))
rw [mixtureLaw_eq_product_map D _ _ hr,
partition_densityJoint_eq_product_map n B d hd hfit, ← Measure.prod_swap,
Measure.map_map hr measurable_swap, Measure.map_map hc hm]
congr 1
funext x
dsimp only [Function.comp_apply, swap]
exact (copyOutcomes_distinctResponses_record n B d hd hfit σ h (s, x.2) x.1).symm
theorem blockMixtureLawOf_bind_partition
Drawing partition and baselines independently is equivalent to first mixing the fixed-partition original-record laws. This identity uses the actual complete design. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
blockMixtureLawOf n B d D σ h
= (partitionLaw B d).bind (fun s => mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d σ h (s, U)))
Proof (Lean source)
-- @node: blockMixtureLawOf_bind_partition
lemma blockMixtureLawOf_bind_partition (n B d : ℕ) (σ : Bool) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
blockMixtureLawOf n B d D σ h = (partitionLaw B d).bind (fun s =>
mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d σ h (s, U))) := by
let := blockBaselineLaw_probability B
let K := fun ξ : SourcePartition B d × (Fin B → ℝ) =>
D.map (recordOf (blockSchedule n B d σ h ξ))
have hK : Measurable K :=
block_measurable_map_parameter D _ (block_record_measurable n B d σ h)
have hL : Measurable (fun s : SourcePartition B d =>
mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d σ h (s, U))) :=
measurable_of_finite _
ext E hE
change ((partitionLaw B d).prod (blockBaselineLaw B)).bind K E = _
rw [Measure.bind_apply hE hK.aemeasurable,
lintegral_prod (fun ξ => K ξ E) ((Measure.measurable_coe hE).comp hK).aemeasurable,
Measure.bind_apply hE hL.aemeasurable]
apply lintegral_congr
intro s
change _ = (blockBaselineLaw B).bind (fun U => K (s, U)) E
exact (Measure.bind_apply hE
(show AEMeasurable (fun U => K (s, U)) (blockBaselineLaw B) from
(hK.comp (measurable_const.prodMk measurable_id)).aemeasurable)).symm
theorem blockMixtureLawOf_partition_density_representation
The full original-record block mixture has an exact density representation conditional on the true partition and actual design, prior to the posterior-completion calculation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
conclusion
blockMixtureLawOf n B d D σ h
= (partitionLaw B d).bind (fun s => (densityJoint D volume (partitionResponseDensity n B d σ h s)).map (fun x => copyOutcomes n B d ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1) x.2))
Proof (Lean source)
-- @node: blockMixtureLawOf_partition_density_representation
lemma blockMixtureLawOf_partition_density_representation (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
blockMixtureLawOf n B d D σ h = (partitionLaw B d).bind (fun s =>
(densityJoint D volume (partitionResponseDensity n B d σ h s)).map
(fun x => copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1)
x.2)) := by
rw [blockMixtureLawOf_bind_partition]
congr 1
funext s
exact partition_mixture_density_representation n B d hd hfit σ h s D
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.PartitionTranslationBound 8 declarations The actual labeled source sums have second moment d under the assignment marginal.
Translation testing bound conditional on the true partition
The actual labeled source sums have second moment d under the assignment marginal. Tensorized baseline translation and common-marginal Hellinger averaging then bound the complete original-record distance for each fixed partition.
theorem partitionResponseShift_eq_source_sum
The fixed-partition shift uses the actual incoming source labels of any recipient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
s :
SourcePartition B d
ℓ :
Fin B
i :
Fin n
hi :
i ∈ recipientBlock n B d ℓ
conclusion
partitionResponseShift n B d σ h s ω ℓ
= signOf σ * h / (2 * d) * ∑ j ∈ inNbhd (blockSchedule n B d true 0 (s, fun _ => 0)) i, signOf (ω.1 j)
Proof (Lean source)
-- @node: partitionResponseShift_eq_source_sum
lemma partitionResponseShift_eq_source_sum (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (ℓ : Fin B) (i : Fin n)
(hi : i ∈ recipientBlock n B d ℓ) :
partitionResponseShift n B d σ h s ω ℓ = signOf σ * h / (2 * d) *
∑ j ∈ inNbhd (blockSchedule n B d true 0 (s, fun _ => 0)) i, signOf (ω.1 j) := by
unfold partitionResponseShift
rw [distinctResponses_record_recipient n B d hd hfit σ h (s, fun _ => 0) ω ℓ i hi,
blockSchedule_response_centered n B d hd hfit σ h (s, fun _ => 0) ω.1 ℓ i hi]
simp only [zero_add]
rfl
theorem partition_shift_separation_moment
Opposite signs have exactly h squared over d expected squared shift separation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ ω, (partitionResponseShift n B d true h s ω ℓ - partitionResponseShift n B d false h s ω ℓ) ^ 2 ∂D)
= h ^ 2 / d
Proof (Lean source)
-- @node: partition_shift_separation_moment
lemma partition_shift_separation_moment (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (ha : AssignmentLaw D) (ℓ : Fin B) :
(∫ ω, (partitionResponseShift n B d true h s ω ℓ -
partitionResponseShift n B d false h s ω ℓ) ^ 2 ∂D) = h ^ 2 / d := by
obtain ⟨i, hi⟩ := Finset.card_pos.mp (show 0 < (recipientBlock n B d ℓ).card by
rw [recipientBlock_card n B d hfit ℓ]; omega)
let N := inNbhd (blockSchedule n B d true 0 (s, fun _ => 0)) i
have hc : N.card = d := blockSchedule_inNbhd_card n B d hd hfit true 0 _ i ℓ hi
have hm : (∫ ω, (∑ j ∈ N, signOf (ω.1 j)) ^ 2 ∂D) = (d : ℝ) := by
rw [← integral_map measurable_fst.aemeasurable
(show AEStronglyMeasurable (fun z : Assign (Fin n) =>
(∑ j ∈ N, signOf (z j)) ^ 2) (D.map fst) from
(measurable_of_finite _).aestronglyMeasurable), ha,
reverse_revealed_second_moment, hc]
have he (ω : Assign (Fin n) × Audit (Fin n)) :
(partitionResponseShift n B d true h s ω ℓ -
partitionResponseShift n B d false h s ω ℓ) ^ 2 =
(h / d) ^ 2 * (∑ j ∈ N, signOf (ω.1 j)) ^ 2 := by
rw [partitionResponseShift_eq_source_sum n B d hd hfit true h s ω ℓ i hi,
partitionResponseShift_eq_source_sum n B d hd hfit false h s ω ℓ i hi]
simp only [signOf, Bool.false_eq_true, ite_true, ite_false]
dsimp only [N]
ring
simp_rw [he]
rw [integral_const_mul, hm]
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
field_simp
theorem partition_response_energy_le
Tensorized translation controls fixed-partition response energy at each design draw. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
hellingerEnergy volume (partitionResponseDensity n B d true h s ω) (partitionResponseDensity n B d false h s ω)
≤ ∑ ℓ : Fin B, 4 * pi ^ 2 * (partitionResponseShift n B d true h s ω ℓ - partitionResponseShift n B d false h s ω ℓ) ^ 2
Proof (Lean source)
-- @node: partition_response_energy_le
lemma partition_response_energy_le (n B d : ℕ) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) :
hellingerEnergy volume (partitionResponseDensity n B d true h s ω)
(partitionResponseDensity n B d false h s ω) ≤
∑ ℓ : Fin B, 4 * pi ^ 2 *
(partitionResponseShift n B d true h s ω ℓ -
partitionResponseShift n B d false h s ω ℓ) ^ 2 := by
exact baseline_translation_affinity.2.2.2.1 B
(partitionResponseShift n B d true h s ω) (partitionResponseShift n B d false h s ω)
theorem partition_response_energy_mean_le
Averaging the translation bound keeps every assignment and audit coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ ω, hellingerEnergy volume (partitionResponseDensity n B d true h s ω) (partitionResponseDensity n B d false h s ω) ∂D)
≤ 4 * pi ^ 2 * h ^ 2 * B / d
Proof (Lean source)
-- @node: partition_response_energy_mean_le
lemma partition_response_energy_mean_le (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(ha : AssignmentLaw D) :
(∫ ω, hellingerEnergy volume (partitionResponseDensity n B d true h s ω)
(partitionResponseDensity n B d false h s ω) ∂D) ≤
4 * pi ^ 2 * h ^ 2 * B / d := by
calc
_ ≤ ∫ ω, ∑ ℓ : Fin B, 4 * pi ^ 2 *
(partitionResponseShift n B d true h s ω ℓ -
partitionResponseShift n B d false h s ω ℓ) ^ 2 ∂D :=
integral_mono Integrable.of_finite Integrable.of_finite
(fun ω => partition_response_energy_le n B d h s ω)
_ = _ := by
rw [integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
simp_rw [integral_const_mul, partition_shift_separation_moment n B d hd hfit h s D ha]
simp only [Finset.sum_const, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul]
ring
theorem partitionResponseDensity_nonneg
Fixed-partition product densities are nonnegative. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
0 ≤ partitionResponseDensity n B d σ h s ω y
Proof (Lean source)
-- @node: partitionResponseDensity_nonneg
lemma partitionResponseDensity_nonneg (n B d : ℕ) (σ : Bool) (h : ℝ)
(s : SourcePartition B d) (ω : Assign (Fin n) × Audit (Fin n)) (y : Fin B → ℝ) :
0 ≤ partitionResponseDensity n B d σ h s ω y := by
exact Finset.prod_nonneg (fun _ _ => cosSqDensity_nonneg _)
theorem partitionResponseDensity_probability
Independent translated baselines normalize every fixed-partition response density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (volume.withDensity (fun y => ofReal (partitionResponseDensity n B d σ h s ω y)))
Proof (Lean source)
-- @node: partitionResponseDensity_probability
lemma partitionResponseDensity_probability (n B d : ℕ) (σ : Bool) (h : ℝ)
(s : SourcePartition B d) (ω : Assign (Fin n) × Audit (Fin n)) :
IsProbabilityMeasure (volume.withDensity
(fun y => ofReal (partitionResponseDensity n B d σ h s ω y))) := by
unfold partitionResponseDensity
rw [← blockBaselineLaw_map_translation B (partitionResponseShift n B d σ h s ω)]
let := blockBaselineLaw_probability B
exact Measure.isProbabilityMeasure_map (show AEMeasurable
(fun (U : Fin B → ℝ) v => U v + partitionResponseShift n B d σ h s ω v) _ from
(show Measurable (fun (U : Fin B → ℝ) v => U v + partitionResponseShift n B d σ h s ω v) by
fun_prop).aemeasurable)
theorem partition_densityJoint_tv_le
Common full-design marginals turn the averaged response energy into a joint TV bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
tvDist (densityJoint D volume (partitionResponseDensity n B d true h s)) (densityJoint D volume (partitionResponseDensity n B d false h s))
≤ sqrt (∫ ω, hellingerEnergy volume (partitionResponseDensity n B d true h s ω) (partitionResponseDensity n B d false h s ω) ∂D)
Proof (Lean source)
-- @node: partition_densityJoint_tv_le
lemma partition_densityJoint_tv_le (n B d : ℕ) (h : ℝ) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
tvDist
(densityJoint D volume (partitionResponseDensity n B d true h s))
(densityJoint D volume (partitionResponseDensity n B d false h s)) ≤
sqrt (∫ ω, hellingerEnergy volume (partitionResponseDensity n B d true h s ω)
(partitionResponseDensity n B d false h s ω) ∂D) := by
rw [densityJoint_eq_withDensity D volume _ (partitionResponseDensity_measurable n B d true h s),
densityJoint_eq_withDensity D volume _ (partitionResponseDensity_measurable n B d false h s)]
let := densityJoint_probability D volume _
(partitionResponseDensity_measurable n B d true h s)
(partitionResponseDensity_probability n B d true h s)
let := densityJoint_probability D volume _
(partitionResponseDensity_measurable n B d false h s)
(partitionResponseDensity_probability n B d false h s)
have hp : IsProbabilityMeasure ((D.prod volume).withDensity
(fun x => ofReal (partitionResponseDensity n B d true h s x.1 x.2))) := by
rw [← densityJoint_eq_withDensity D volume _
(partitionResponseDensity_measurable n B d true h s)]
infer_instance
have hm : IsProbabilityMeasure ((D.prod volume).withDensity
(fun x => ofReal (partitionResponseDensity n B d false h s x.1 x.2))) := by
rw [← densityJoint_eq_withDensity D volume _
(partitionResponseDensity_measurable n B d false h s)]
infer_instance
let := hp
let := hm
have hb := density_tv_le_sqrt_hellingerEnergy (D.prod volume) _ _
(partitionResponseDensity_measurable n B d true h s)
(partitionResponseDensity_measurable n B d false h s)
(fun x => partitionResponseDensity_nonneg n B d true h s x.1 x.2)
(fun x => partitionResponseDensity_nonneg n B d false h s x.1 x.2)
have hf := (probability_density_integrable_normalized (D.prod volume) _
(partitionResponseDensity_measurable n B d true h s)
(fun x => partitionResponseDensity_nonneg n B d true h s x.1 x.2)).1
have hg := (probability_density_integrable_normalized (D.prod volume) _
(partitionResponseDensity_measurable n B d false h s)
(fun x => partitionResponseDensity_nonneg n B d false h s x.1 x.2)).1
have he := integral_prod _ (integrable_sqrt_sum_sub_sq
(D.prod volume) _ _ hf hg
(fun x => partitionResponseDensity_nonneg n B d true h s x.1 x.2)
(fun x => partitionResponseDensity_nonneg n B d false h s x.1 x.2)).2
exact hb.trans_eq (congrArg sqrt he)
theorem partition_mixture_tv_le
The full original record conditional on the true partition satisfies the genie bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
tvDist (mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d true h (s, U))) (mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d false h (s, U)))
Proof (Lean source)
-- @node: partition_mixture_tv_le
lemma partition_mixture_tv_le (n B d : ℕ) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n)
(h : ℝ) (hh : 0 ≤ h) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(ha : AssignmentLaw D) :
tvDist
(mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d true h (s, U)))
(mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d false h (s, U))) ≤
2 * pi * h * sqrt ((B : ℝ) / d) := by
let := densityJoint_probability D volume _
(partitionResponseDensity_measurable n B d true h s)
(partitionResponseDensity_probability n B d true h s)
let := densityJoint_probability D volume _
(partitionResponseDensity_measurable n B d false h s)
(partitionResponseDensity_probability n B d false h s)
have hc : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1) x.2) := by
let f : ((Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ)) →
(OffDiag (Fin n) → Bool) × (Assign (Fin n) × (Fin B → ℝ)) := fun x =>
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1, x.2)
have hf : Measurable f := by
apply prodMk
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1)).comp measurable_fst
· exact measurable_fst.fst.prodMk measurable_snd
exact (copyOutcomes_measurable n B d).comp hf
rw [partition_mixture_density_representation n B d hd hfit,
partition_mixture_density_representation n B d hd hfit]
refine (block_tvDist_map_le _ _ _ hc).trans
((partition_densityJoint_tv_le n B d h s D).trans
((Real.sqrt_le_sqrt (partition_response_energy_mean_le n B d hd hfit h s D ha)).trans_eq ?_))
have he : 4 * pi ^ 2 * h ^ 2 * B / d =
(2 * pi * h) ^ 2 * ((B : ℝ) / d) := by ring
rw [he, Real.sqrt_mul (sq_nonneg _), Real.sqrt_sq (by positivity)]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.PosteriorCompletionLaw 10 declarations Disintegration over the complete retained graph and assignment preserves every coordinate.
Full-record posterior completion mixture
Disintegration over the complete retained graph and assignment preserves every coordinate. The compatible-partition posterior is uniform and its denominator is the exact labeled multinomial count.
def partitionGraphAssignLaw
The finite full graph-assignment law for a fixed labeled source partition.
def partitionResponseKernel
A fixed-partition response channel depends on the assignment but not the audit marks.
Definition (Lean source)
parameters
given by
(volume.withDensity (fun y => ofReal (partitionResponseDensity n B d σ h s (hz.2, fun _ => false) y))).map (copyOutcomes n B d hz)
theorem partition_mixture_graphAssign_bind
The fixed-partition original-record law factors through its full graph-assignment law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: partition_mixture_graphAssign_bind
lemma partition_mixture_graphAssign_bind (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
mixtureLaw D (blockBaselineLaw B) (fun U => blockSchedule n B d σ h (s, U)) =
(partitionGraphAssignLaw n B d D s).bind (partitionResponseKernel n B d σ h s) := by
rw [partition_mixture_density_representation n B d hd hfit]
ext E hE
have hc : Measurable (fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1) x.2) :=
(copyOutcomes_measurable n B d).comp (show Measurable
(fun x : (Assign (Fin n) × Audit (Fin n)) × (Fin B → ℝ) =>
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1, x.2)) from by
apply prodMk
· exact (measurable_of_finite (fun ω : Assign (Fin n) × Audit (Fin n) =>
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1)).comp measurable_fst
· exact measurable_fst.fst.prodMk measurable_snd)
rw [Measure.map_apply hc hE, densityJoint, Measure.bind_apply (hc hE)
(measurable_densityJoint_fiber volume _
(partitionResponseDensity_measurable n B d σ h s)).aemeasurable,
Measure.bind_apply hE (measurable_of_finite _).aemeasurable,
partitionGraphAssignLaw, lintegral_map (measurable_of_finite _) (measurable_of_finite _)]
apply lintegral_congr
intro ω
have hmω : Measurable (copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1)) :=
hc.comp measurable_prodMk_left
rw [Measure.map_apply measurable_prodMk_left (hc hE), partitionResponseKernel,
Measure.map_apply hmω hE]
rfl
theorem partitionGraphAssignLaw_atom
A graph-assignment atom factors into its exact detailed graph likelihood and the probability of the entire assignment vector. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: partitionGraphAssignLaw_atom
lemma partitionGraphAssignLaw_atom (n B d : ℕ) (q : ℝ) (hq : q ∈ Icc 0 1)
(s : SourcePartition B d) (H : OffDiag (Fin n) → Bool) (z : Assign (Fin n)) :
partitionGraphAssignLaw n B d (thinnedDesign (Fin n) q) s {(H,z)} =
halfBernoulli (Fin n) {z} *
((auditLaw (Fin n) q).map (fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e)) {H} := by
let := bernoulliLaw_probability q hq
let _ : IsProbabilityMeasure (auditLaw (Fin n) q) := by unfold auditLaw; infer_instance
rw [partitionGraphAssignLaw, Measure.map_apply (measurable_of_finite _)
(measurableSet_singleton _)]
have he : (fun ω : Assign (Fin n) × Audit (Fin n) =>
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1)) ⁻¹'
{(H,z)} = {z} ×ˢ ((fun w e =>
decide (blockEdge n B d s e.val.1 e.val.2) && w e) ⁻¹' {H}) := by
ext ω
simp only [Set.mem_preimage, Set.mem_singleton_iff, Prod.mk.injEq, Set.mem_prod,
recordOf, blockSchedule]
exact and_comm
rw [he, thinnedDesign, Measure.prod_prod,
Measure.map_apply (measurable_of_finite _) (measurableSet_singleton _)]
theorem partitionGraphAssignLaw_atom_ite
The common likelihood includes every assignment coordinate and retained-edge subset. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: partitionGraphAssignLaw_atom_ite
lemma partitionGraphAssignLaw_atom_ite (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) (s : SourcePartition B d)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n)) :
partitionGraphAssignLaw n B d (thinnedDesign (Fin n) q) s {hz} =
if (∀ e, hz.1 e = true → blockEdge n B d s e.val.1 e.val.2) then
halfBernoulli (Fin n) {hz.2} *
(ofReal q ^ (Finset.univ.filter (fun e => hz.1 e = true)).card *
ofReal (1-q) ^
(B*d*d - (Finset.univ.filter (fun e => hz.1 e = true)).card)) else 0 := by
rw [partitionGraphAssignLaw_atom n B d q hq s hz.1 hz.2,
block_detailed_graph_mass_ite n B d hfit q hq]
split_ifs <;> simp only [mul_zero]
theorem partitionLaw_bind_uniform_fibers
Uniform finite mixing followed by a common-likelihood observation channel gives the uniform posterior over compatible completions, including null observation atoms. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
β γ :
B d :
ℕ
μ :
SourcePartition B d → Measure β
K :
SourcePartition B d → β → Measure γ
R :
SourcePartition B d → β → Prop
∀ s b, Decidable (R s b)
a :
β → ℝ≥0∞
hμ :
∀ s b, μ s {b} = if R s b then a b else 0
conclusion
= ((partitionLaw B d).bind μ).bind (fun b => (((Finset.univ.filter (fun s => R s b)).card : ℝ≥0∞)⁻¹) • ∑ s ∈ Finset.univ.filter (fun s => R s b), K s b)
Proof (Lean source)
-- @node: partitionLaw_bind_uniform_fibers
lemma partitionLaw_bind_uniform_fibers {β γ : Type*} [Fintype β]
[MeasurableSpace β] [MeasurableSingletonClass β] [MeasurableSpace γ]
(B d : ℕ) (μ : SourcePartition B d → Measure β)
(K : SourcePartition B d → β → Measure γ) (R : SourcePartition B d → β → Prop)
[∀ s b, Decidable (R s b)] (a : β → ℝ≥0∞)
(hμ : ∀ s b, μ s {b} = if R s b then a b else 0) :
(partitionLaw B d).bind (fun s => (μ s).bind (K s)) =
((partitionLaw B d).bind μ).bind (fun b =>
(((Finset.univ.filter (fun s => R s b)).card : ℝ≥0∞)⁻¹) •
∑ s ∈ Finset.univ.filter (fun s => R s b), K s b) := by
ext E hE
rw [Measure.bind_apply hE (measurable_of_finite _).aemeasurable,
partitionLaw_lintegral,
Measure.bind_apply hE (measurable_of_finite _).aemeasurable,
lintegral_countable', tsum_fintype]
simp_rw [Measure.bind_apply hE (measurable_of_finite _).aemeasurable,
lintegral_countable', tsum_fintype, hμ]
rw [Finset.sum_comm, Finset.mul_sum]
apply Finset.sum_congr rfl
intro b _
rw [Measure.smul_apply, smul_eq_mul, Measure.finsetSum_apply,
Measure.bind_apply (measurableSet_singleton _) (measurable_of_finite _).aemeasurable,
partitionLaw_lintegral]
simp_rw [hμ]
rw [Finset.sum_ite]
simp only [Finset.sum_const, nsmul_eq_mul, mul_zero, add_zero]
have hs : (∑ s, (K s b) E * (if R s b then a b else 0)) =
(∑ s ∈ Finset.univ.filter (fun s => R s b), (K s b) E) * a b := by
rw [Finset.sum_mul]
simp_rw [mul_ite, mul_zero]
rw [Finset.sum_ite]
simp
rw [hs]
let C := Finset.univ.filter (fun s : SourcePartition B d => R s b)
by_cases hC : C.card = 0
· have he : C = ∅ := Finset.card_eq_zero.mp hC
change _ * ((∑ s ∈ C, (K s b) E) * a b) = _
simp only [he, Finset.sum_empty, zero_mul, mul_zero, C] at *
· have hc : (C.card : ℝ≥0∞)⁻¹ * C.card = 1 :=
ENNReal.inv_mul_cancel (by exact_mod_cast hC) (by finiteness)
change _ * ((∑ s ∈ C, (K s b) E) * a b) = _
calc
_ = ((C.card : ℝ≥0∞)⁻¹ * C.card) *
((card (SourcePartition B d) : ℝ≥0∞)⁻¹ *
((∑ s ∈ C, (K s b) E) * a b)) := by rw [hc, one_mul]
_ = _ := by ac_rfl
theorem blockMixtureLawOf_uniform_completion_bind
The complete original-record mixture is the actual full graph-assignment marginal followed by a uniform mixture of all compatible fixed-partition response kernels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= (graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz => (((Finset.univ.filter (fun s : SourcePartition B d => ∀ e, hz.1 e = true → blockEdge n B d s e.val.1 e.val.2)).card : ℝ≥0∞)⁻¹) • ∑ s ∈ Finset.univ.filter (fun s : SourcePartition B d => ∀ e, hz.1 e = true → blockEdge n B d s e.val.1 e.val.2), partitionResponseKernel n B d σ h s hz)
Proof (Lean source)
-- @node: blockMixtureLawOf_uniform_completion_bind
lemma blockMixtureLawOf_uniform_completion_bind (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q : ℝ) (hq : q ∈ Icc 0 1) :
blockMixtureLawOf n B d (thinnedDesign (Fin n) q) σ h =
(graphAssignMarginal n B d (thinnedDesign (Fin n) q)).bind (fun hz =>
(((Finset.univ.filter (fun s : SourcePartition B d =>
∀ e, hz.1 e = true → blockEdge n B d s e.val.1 e.val.2)).card : ℝ≥0∞)⁻¹) •
∑ s ∈ Finset.univ.filter (fun s : SourcePartition B d =>
∀ e, hz.1 e = true → blockEdge n B d s e.val.1 e.val.2),
partitionResponseKernel n B d σ h s hz) := by
let : IsProbabilityMeasure (thinnedDesign (Fin n) q) :=
thinnedDesign_probabilityDesign (V := Fin n) q hq
rw [blockMixtureLawOf_bind_partition]
simp_rw [partition_mixture_graphAssign_bind n B d hd hfit]
have he := partitionLaw_bind_uniform_fibers
(β := (OffDiag (Fin n) → Bool) × Assign (Fin n)) (γ := Record (Fin n)) B d
(partitionGraphAssignLaw n B d (thinnedDesign (Fin n) q))
(partitionResponseKernel n B d σ h)
(fun s hz => ∀ e, hz.1 e = true → blockEdge n B d s e.val.1 e.val.2)
(fun hz => halfBernoulli (Fin n) {hz.2} *
(ofReal q ^ (Finset.univ.filter (fun e => hz.1 e = true)).card *
ofReal (1-q) ^
(B*d*d - (Finset.univ.filter (fun e => hz.1 e = true)).card)))
(partitionGraphAssignLaw_atom_ite n B d hfit q hq)
exact he
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.PosteriorResponseBridge 19 declarations A compatible labeled partition splits each source row into the observed signs and its hidden labels.
Compatible-completion response shifts
A compatible labeled partition splits each source row into the observed signs and its hidden labels. The hidden treated count supplies the exact shift used in the polynomial conditional density; no treatment coordinate is discarded.
The original source labels assigned to a specified block by a completion.
Definition (Lean source)
parameters
given by
Finset.univ.filter (fun j => ∃ hj : j.val < B * d, s.1 ⟨j.val, hj⟩ = ℓ)
Hidden labels in a completed row retain their original population labels.
Definition (Lean source)
parameters
given by
partitionSourceLabels n B d s ℓ \ revealedSources n B d H ℓ
def partitionHiddenTreated
Count treated hidden labels in a specified compatible completed row.
Definition (Lean source)
parameters
def
partitionHiddenTreated n B d s H z ℓ :
ℕ
given by
theorem partitionSourceLabels_card
Every completed row has exactly d original source labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(partitionSourceLabels n B d s ℓ).card = d
Proof (Lean source)
-- @node: partitionSourceLabels_card
lemma partitionSourceLabels_card (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) :
(partitionSourceLabels n B d s ℓ).card = d := by
let emb : Fin (B * d) → Fin n := fun j => ⟨j.val, by omega⟩
have hemb : Injective emb := by
intro a b hab
exact Fin.ext (congrArg (fun j : Fin n => j.val) hab)
have he : partitionSourceLabels n B d s ℓ =
(Finset.univ.filter (fun j => s.1 j = ℓ)).image emb := by
ext j
simp only [partitionSourceLabels, mem_filter, Finset.mem_univ,
true_and, mem_image]
constructor
· rintro ⟨hj, hs⟩
exact ⟨⟨j.val, hj⟩, hs, Fin.ext rfl⟩
· rintro ⟨k, hk, rfl⟩
exact ⟨k.isLt, hk⟩
rw [he, Finset.card_image_of_injective _ hemb]
exact s.2 ℓ
theorem revealedSources_subset_partitionSourceLabels
Compatibility with the complete graph fixes the row of every revealed label. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
revealedSources n B d H ℓ ⊆ partitionSourceLabels n B d s ℓ
Proof (Lean source)
-- @node: revealedSources_subset_partitionSourceLabels
lemma revealedSources_subset_partitionSourceLabels (n B d : ℕ)
(s : SourcePartition B d) (H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2) (ℓ : Fin B) :
revealedSources n B d H ℓ ⊆ partitionSourceLabels n B d s ℓ := by
intro j hj
obtain ⟨hjlt, i, hi, hji, he⟩ := (Finset.mem_filter.mp hj).2
obtain ⟨⟨hjlt', k, hsk, hik⟩, _⟩ := hs ⟨(j, i), hji⟩ he
have hk : k = ℓ := recipientBlock_unique n B d i k ℓ hik hi
exact Finset.mem_filter.mpr ⟨Finset.mem_univ _, hjlt, by simpa [hk] using hsk⟩
theorem partitionHiddenLabels_card
A compatible completion has exactly the observed remaining capacity in each row. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(partitionHiddenLabels n B d s H ℓ).card = capacity n B d H ℓ
Proof (Lean source)
-- @node: partitionHiddenLabels_card
lemma partitionHiddenLabels_card (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2) (ℓ : Fin B) :
(partitionHiddenLabels n B d s H ℓ).card = capacity n B d H ℓ := by
rw [partitionHiddenLabels, Finset.card_sdiff_of_subset
(revealedSources_subset_partitionSourceLabels n B d s H hs ℓ),
partitionSourceLabels_card n B d hfit]
rfl
theorem partitionHiddenTreated_le_capacity
The completed row's treated hidden count is an admissible polynomial index. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: partitionHiddenTreated_le_capacity
lemma partitionHiddenTreated_le_capacity (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)
(z : Assign (Fin n)) (ℓ : Fin B) :
partitionHiddenTreated n B d s H z ℓ ≤ capacity n B d H ℓ := by
rw [← partitionHiddenLabels_card n B d hfit s H hs ℓ]
exact Finset.card_filter_le _ _
theorem partitionHiddenLabels_mem_iff
Within a compatible completion, a hidden row label is precisely a source in that row with no retained arrow anywhere in the complete graph. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: partitionHiddenLabels_mem_iff
lemma partitionHiddenLabels_mem_iff (n B d : ℕ) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)
(ℓ : Fin B) (j : Fin n) :
j ∈ partitionHiddenLabels n B d s H ℓ ↔
(∃ hj : j.val < B * d, s.1 ⟨j.val, hj⟩ = ℓ) ∧
¬ ∃ i : Fin n, ∃ hji : j ≠ i, H ⟨(j,i), hji⟩ = true := by
simp only [partitionHiddenLabels, mem_sdiff, partitionSourceLabels, revealedSources,
mem_filter, Finset.mem_univ, true_and]
constructor
· rintro ⟨⟨hj, hrow⟩, hn⟩
refine ⟨⟨hj, hrow⟩, ?_⟩
rintro ⟨i, hji, he⟩
obtain ⟨⟨hj', k, hsk, hik⟩, _⟩ := hs ⟨(j,i), hji⟩ he
have hk : k = ℓ := hsk.symm.trans hrow
apply hn
exact ⟨hj, i, by simpa [hk] using hik, hji, he⟩
· rintro ⟨hrow, hn⟩
refine ⟨hrow, ?_⟩
rintro ⟨_, i, _, hji, he⟩
exact hn ⟨i, hji, he⟩
theorem partitionHiddenTreated_sum
Completed row treated counts sum to the observed global hidden treated count; this equality uses every assignment coordinate of each original hidden source label. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
s :
SourcePartition B d
conclusion
(∑ ℓ, partitionHiddenTreated n B d s H z ℓ) = hiddenTreated n B d H z
Proof (Lean source)
-- @node: partitionHiddenTreated_sum
lemma partitionHiddenTreated_sum (n B d : ℕ) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)
(z : Assign (Fin n)) :
(∑ ℓ, partitionHiddenTreated n B d s H z ℓ) = hiddenTreated n B d H z := by
by_cases hB : B = 0
· subst B
simp [hiddenTreated]
let label : Fin n → Fin B := fun j =>
if hj : j.val < B * d then s.1 ⟨j.val, hj⟩ else ⟨0, Nat.pos_of_ne_zero hB⟩
let hidden := Finset.univ.filter (fun j : Fin n => j.val < B * d ∧ z j = true ∧
¬ ∃ i : Fin n, ∃ hji : j ≠ i, H ⟨(j,i),hji⟩ = true)
have hrow (ℓ : Fin B) : hidden.filter (fun j => label j = ℓ) =
(partitionHiddenLabels n B d s H ℓ).filter (fun j => z j = true) := by
ext j
simp only [mem_filter, hidden, Finset.mem_univ, true_and,
partitionHiddenLabels_mem_iff n B d s H hs]
constructor
· rintro ⟨⟨hj, hz, hn⟩, hℓ⟩
exact ⟨⟨⟨hj, by simpa only [label, dif_pos hj] using hℓ⟩, hn⟩, hz⟩
· rintro ⟨⟨⟨hj, hℓ⟩, hn⟩, hz⟩
exact ⟨⟨hj, hz, hn⟩, by simpa only [label, dif_pos hj] using hℓ⟩
have hsplit : hidden.card = ∑ ℓ : Fin B, (hidden.filter (fun j => label j = ℓ)).card :=
Finset.card_eq_sum_card_fiberwise (fun _ _ => Finset.mem_univ _)
simpa only [hrow, partitionHiddenTreated, hiddenTreated, hidden] using hsplit.symm
theorem partitionSourceLabels_eq_inNbhd
Incoming neighbors of any block recipient are the completion's original source labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: partitionSourceLabels_eq_inNbhd
lemma partitionSourceLabels_eq_inNbhd (n B d : ℕ) (s : SourcePartition B d)
(ℓ : Fin B) (i : Fin n) (hi : i ∈ recipientBlock n B d ℓ) :
partitionSourceLabels n B d s ℓ =
inNbhd (blockSchedule n B d true 0 (s, fun _ => 0)) i := by
ext j
simp only [partitionSourceLabels, inNbhd, blockSchedule, mem_filter,
Finset.mem_univ, true_and]
constructor
· rintro ⟨hj, hs⟩
exact (blockEdge_source_iff n B d s j i hj).mpr (by simpa [hs] using hi)
· rintro ⟨⟨hj, k, hsk, hik⟩, _⟩
exact ⟨hj, hsk.trans (recipientBlock_unique n B d i k ℓ hik hi)⟩
A sum of centered Boolean signs is twice the treated-label count minus the row size. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j ∈ S, signOf (z j)) = 2 * ((S.filter (fun j => z j = true)).card : ℝ) - S.card
Proof (Lean source)
-- @node: partition_sign_sum_eq_treated
lemma partition_sign_sum_eq_treated {ι : Type*} [DecidableEq ι]
(S : Finset ι) (z : ι → Bool) :
(∑ j ∈ S, signOf (z j)) = 2 * ((S.filter (fun j => z j = true)).card : ℝ) - S.card := by
have he (j : ι) : signOf (z j) = 2 * (if z j = true then (1 : ℝ) else 0) - 1 := by
cases z j <;> norm_num [signOf]
simp_rw [he]
rw [Finset.sum_sub_distrib, ← Finset.mul_sum, ← Finset.sum_filter]
simp
theorem partitionSourceLabels_sign_sum
The completed source sum is exactly the revealed signs plus twice the hidden treated count minus the remaining capacity. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j ∈ partitionSourceLabels n B d s ℓ, signOf (z j))
= revealedSignSum n B d H z ℓ
+ 2 * (partitionHiddenTreated n B d s H z ℓ : ℝ)
- capacity n B d H ℓ
Proof (Lean source)
-- @node: partitionSourceLabels_sign_sum
lemma partitionSourceLabels_sign_sum (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)
(z : Assign (Fin n)) (ℓ : Fin B) :
(∑ j ∈ partitionSourceLabels n B d s ℓ, signOf (z j)) =
revealedSignSum n B d H z ℓ + 2 * (partitionHiddenTreated n B d s H z ℓ : ℝ) -
capacity n B d H ℓ := by
have hsplit := Finset.sum_sdiff (f := fun j => signOf (z j))
(revealedSources_subset_partitionSourceLabels n B d s H hs ℓ)
change (∑ j ∈ partitionHiddenLabels n B d s H ℓ, signOf (z j)) +
revealedSignSum n B d H z ℓ = _ at hsplit
rw [partition_sign_sum_eq_treated, partitionHiddenLabels_card n B d hfit s H hs ℓ]
at hsplit
change 2 * (partitionHiddenTreated n B d s H z ℓ : ℝ) - capacity n B d H ℓ +
revealedSignSum n B d H z ℓ = _ at hsplit
linarith
def partitionHiddenAllocation
The actual completed hidden counts are admissible polynomial indices, retaining all original labels and the detailed retained edge subsets.
Definition (Lean source)
parameters
def
given by
fun ℓ
=> ⟨partitionHiddenTreated n B d s (fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 ℓ, Nat.lt_succ_of_le (partitionHiddenTreated_le_capacity n B d hfit s _ (fun e he => by have he' : blockEdge n B d s e.val.1 e.val.2 ∧ ω.2 e = true := by simpa only [Bool.and_eq_true, decide_eq_true_eq] using he exact he'.1) ω.1 ℓ)⟩
theorem partitionHiddenAllocation_sum
The actual admissible allocation satisfies the coefficient extraction's global degree constraint, including the endpoint of zero hidden capacity. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ ℓ, (partitionHiddenAllocation n B d hfit s ω ℓ).val)
Proof (Lean source)
-- @node: partitionHiddenAllocation_sum
lemma partitionHiddenAllocation_sum (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ω : Assign (Fin n) × Audit (Fin n)) :
(∑ ℓ, (partitionHiddenAllocation n B d hfit s ω ℓ).val) =
hiddenTreated n B d
(fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 := by
exact partitionHiddenTreated_sum n B d s
(fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) (fun e he => by
have he' : blockEdge n B d s e.val.1 e.val.2 ∧ ω.2 e = true := by
simpa only [Bool.and_eq_true, decide_eq_true_eq] using he
exact he'.1) ω.1
def partitionAllocationDensity
The product row density associated with the actual compatible allocation. Its shifts are precisely those appearing in the coefficient-extraction formula.
Definition (Lean source)
parameters
def
partitionAllocationDensity n B d hfit σ h s ω y :
ℝ
given by
∏ ℓ, cosSqDensity (y ℓ - signOf σ * h * (revealedSignSum n B d (fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 ℓ + 2 * ((partitionHiddenAllocation n B d hfit s ω ℓ).val : ℝ) - capacity n B d (fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ℓ) / (2 * d))
theorem partitionResponseDensity_eq_allocationDensity
The original fixed-partition density is exactly the row product at the completed hidden treated allocation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
partitionResponseDensity n B d σ h s ω y = partitionAllocationDensity n B d hfit σ h s ω y
Proof (Lean source)
-- @node: partitionResponseDensity_eq_allocationDensity
lemma partitionResponseDensity_eq_allocationDensity (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (y : Fin B → ℝ) :
partitionResponseDensity n B d σ h s ω y =
partitionAllocationDensity n B d hfit σ h s ω y := by
unfold partitionResponseDensity partitionAllocationDensity partitionHiddenAllocation
simp_rw [partitionResponseShift_eq_hiddenTreated n B d hd hfit]
theorem partitionResponseDensity_eq_constrainedAllocation
The fixed-partition product density satisfies exactly the total-degree constraint used in the hypergeometric polynomial formula. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
partitionResponseDensity n B d σ h s ω y
= if hiddenTreated n B d (fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 = ∑ ℓ, (partitionHiddenAllocation n B d hfit s ω ℓ).val then partitionAllocationDensity n B d hfit σ h s ω y else 0
Proof (Lean source)
-- @node: partitionResponseDensity_eq_constrainedAllocation
lemma partitionResponseDensity_eq_constrainedAllocation (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(ω : Assign (Fin n) × Audit (Fin n)) (y : Fin B → ℝ) :
partitionResponseDensity n B d σ h s ω y =
if hiddenTreated n B d
(fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 =
∑ ℓ, (partitionHiddenAllocation n B d hfit s ω ℓ).val then
partitionAllocationDensity n B d hfit σ h s ω y else 0 := by
rw [if_pos (partitionHiddenAllocation_sum n B d hfit s ω).symm]
exact partitionResponseDensity_eq_allocationDensity n B d hd hfit σ h s ω y
theorem blockMixtureLawOf_allocation_density_representation
Before the compatible-allocation counting step, the actual full mixture is exactly the partition average of the allocation row density followed by copying every outcome. This representation preserves all retained edges and all assignment coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
conclusion
blockMixtureLawOf n B d D σ h
= (partitionLaw B d).bind (fun s => (densityJoint D volume (fun ω y => if hiddenTreated n B d (fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 = ∑ ℓ, (partitionHiddenAllocation n B d hfit s ω ℓ).val then partitionAllocationDensity n B d hfit σ h s ω y else 0)).map (fun x => copyOutcomes n B d ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1) x.2))
Proof (Lean source)
-- @node: blockMixtureLawOf_allocation_density_representation
lemma blockMixtureLawOf_allocation_density_representation (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D] :
blockMixtureLawOf n B d D σ h = (partitionLaw B d).bind (fun s =>
(densityJoint D volume (fun ω y =>
if hiddenTreated n B d
(fun e => decide (blockEdge n B d s e.val.1 e.val.2) && ω.2 e) ω.1 =
∑ ℓ, (partitionHiddenAllocation n B d hfit s ω ℓ).val then
partitionAllocationDensity n B d hfit σ h s ω y else 0)).map
(fun x => copyOutcomes n B d
((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) x.1).1, x.1.1)
x.2)) := by
rw [blockMixtureLawOf_partition_density_representation n B d hd hfit]
simp_rw [densityJoint, partitionResponseDensity_eq_constrainedAllocation n B d hd hfit]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.PosteriorTreatedDensity 8 declarations The treated-label word average is the coefficient-extraction density on the complete retained graph and the entire assignment vector.
Exact posterior response density
The treated-label word average is the coefficient-extraction density on the complete retained graph and the entire assignment vector.
theorem compatiblePartition_treatedRowCount
The actual compatible partition's row treated count is the restricted word's row count. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
treatedRowCount (hiddenSourceTreated n B d hfit H z) (fun j => s.1.1 j.1) ℓ
= partitionHiddenTreated n B d s.1 H z ℓ
Proof (Lean source)
-- @node: compatiblePartition_treatedRowCount
lemma compatiblePartition_treatedRowCount (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n))
(s : CompatiblePartition n B d H) (ℓ : Fin B) :
treatedRowCount (hiddenSourceTreated n B d hfit H z) (fun j => s.1.1 j.1) ℓ =
partitionHiddenTreated n B d s.1 H z ℓ := by
let e : {j : {j : HiddenSource n B d H // hiddenSourceTreated n B d hfit H z j} //
s.1.1 j.1.1 = ℓ} ≃
{j : Fin n // j ∈ partitionHiddenLabels n B d s.1 H ℓ ∧ z j = true} := {
toFun := fun j => ⟨⟨j.1.1.1.val, by have := j.1.1.1.isLt; omega⟩,
(partitionHiddenLabels_mem_iff n B d s.1 H s.2 ℓ _).mpr
⟨⟨j.1.1.1.isLt,j.2⟩,
(sourceReveals_false_iff n B d H _ j.1.1.1.isLt).mp j.1.1.2⟩,j.1.2⟩
invFun := fun j =>
let hj := (partitionHiddenLabels_mem_iff n B d s.1 H s.2 ℓ j.1).mp j.2.1
⟨⟨⟨⟨j.1.val,hj.1.choose⟩,
(sourceReveals_false_iff n B d H j.1 hj.1.choose).mpr hj.2⟩,j.2.2⟩,
hj.1.choose_spec⟩
left_inv := by intro j; rfl
right_inv := by intro j; rfl }
simpa only [treatedRowCount, partitionHiddenTreated, Fintype.card_subtype,
← Finset.filter_filter, Finset.filter_univ_mem] using Fintype.card_congr e
theorem compatiblePartition_responseShift
Compatible partitions have the displayed response shift for the observed detailed retained graph, independently of the audit representative used to write the density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
partitionResponseShift n B d σ h s.1 (z, fun _ => false) ℓ
= signOf σ * h * (revealedSignSum n B d H z ℓ + 2 * (partitionHiddenTreated n B d s.1 H z ℓ : ℝ) - capacity n B d H ℓ) / (2 * d)
Proof (Lean source)
-- @node: compatiblePartition_responseShift
lemma compatiblePartition_responseShift (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (z : Assign (Fin n))
(s : CompatiblePartition n B d H) (ℓ : Fin B) :
partitionResponseShift n B d σ h s.1 (z, fun _ => false) ℓ =
signOf σ * h * (revealedSignSum n B d H z ℓ +
2 * (partitionHiddenTreated n B d s.1 H z ℓ : ℝ) - capacity n B d H ℓ) /
(2 * d) := by
obtain ⟨i, hi⟩ := Finset.card_pos.mp (by
rw [recipientBlock_card n B d hfit ℓ]
omega)
rw [partitionResponseShift_eq_source_sum n B d hd hfit σ h s.1 _ ℓ i hi,
← partitionSourceLabels_eq_inNbhd n B d s.1 ℓ i hi,
partitionSourceLabels_sign_sum n B d hfit s.1 H s.2]
ring
def treatedAllocationResponseDensity
The product response at an admissible row treated-count vector.
Definition (Lean source)
parameters
def
treatedAllocationResponseDensity n B d σ h H z k y :
ℝ
given by
∏ ℓ, cosSqDensity (y ℓ - signOf σ * h * (revealedSignSum n B d H z ℓ + 2 * (k ℓ).val - capacity n B d H ℓ) / (2 * d))
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.PublishedModel 47 declarations Published SNIPE convention gate and specialization
Published SNIPE convention gate and specialization
structure PublishedSchedule
Published raw-polynomial schedules allow diagonal arrows and zero edge coefficients.
Definition (Lean source)
def pubNbhd
The published in-neighborhood, including diagonal arrows when present.
Definition (Lean source)
parameters
def
given by
Finset.univ.filter (fun j => ϑ.edge j i)
def pubOutNbhd
The published out-neighborhood, including diagonal arrows when present.
Definition (Lean source)
parameters
def
given by
Finset.univ.filter (fun i => ϑ.edge j i)
def PublishedOrderOne
Order-one neighborhood interference restricts only unsupported and higher-order coefficients; loops and zero singleton coefficients are permitted.
Definition (Lean source)
parameters
V :
Type u_1
ϑ :
def
PublishedOrderOne ϑ :
Prop
def pubOutcome
The published raw-monomial polynomial potential outcome, including the empty monomial.
Definition (Lean source)
parameters
def
pubOutcome ϑ i z :
ℝ
def pubEnvelope
The coefficient-mass envelope is the maximum row sum of absolute polynomial coefficients, including the empty monomial. The empty-population extension is zero.
Definition (Lean source)
parameters
V :
Type u_1
ϑ :
def
pubEnvelope ϑ :
ℝ≥0
Cortez-Rodriguez, Eichhorn and Yu (2023), arXiv:2208.05553v4, Sections 3.1–3.2, Assumptions 1–2 and equations (3.2)–(3.3). At interaction order one the polynomial has a constant and neighborhood singleton terms, and its defined envelope is their maximum absolute-coefficient mass. Neighborhoods may contain their own vertices, and singleton coefficients may vanish. No normalization or degree bound is part of this convention gate. Source: https://arxiv.org/html/2208.05553v4#S3 .
Definition (Lean source)
parameters
V :
Type u_1
def
SnipeModelConventions :
Sort 0
given by
∀ ϑ : PublishedSchedule V
then
= Finset.univ.sup (fun i => (|ϑ.coef i ∅| + ∑ j ∈ pubNbhd ϑ i, |ϑ.coef i {j}|).toNNReal)
def PublishedClass
The paper's bounded-envelope and bounded-total-degree specialization of the order-one published model. These restrictions belong to the specialization, not the cited conventions.
Definition (Lean source)
def pubTte
The all-treated minus all-control average in the published polynomial carrier.
Definition (Lean source)
parameters
V :
Type u_1
ϑ :
def
pubTte ϑ :
ℝ
given by
def augment
Adjoin all diagonal arrows and encode the baseline, own effect, and off-diagonal coefficients as order-one monomials.
Definition (Lean source)
parameters
V :
Type u_1
θ :
Schedule V
def
given by
clause 1
edge := fun j i => θ.edge j i ∨ j = i
clause 2
decEdge := Classical.decRel _
clause 3
coef := fun i S
=> if S = ∅ then θ.a i else ∑ j : V, if S = {j} then (if j = i then θ.t i else if θ.edge j i then θ.b i j else 0) else 0
Removing diagonal arrows gives an irreflexive published graph. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: strip_edge_irrefl
lemma strip_edge_irrefl (ϑ : PublishedSchedule V) :
∀ i, ¬ (ϑ.edge i i ∧ i ≠ i) := by
intro i h
exact h.2 rfl
def strip
Remove diagonal arrows and read the empty and singleton polynomial coefficients as additive schedule coefficients.
Definition (Lean source)
parameters
V :
Type u_1
ϑ :
def
strip ϑ :
Schedule V
given by
clause 1
edge := fun j i => ϑ.edge j i ∧ j ≠ i
clause 2
decEdge := Classical.decRel _
clause 3
irrefl := strip_edge_irrefl ϑ
clause 4
a := fun i => ϑ.coef i ∅
clause 5
t := fun i => ϑ.coef i {i}
clause 6
b := fun i j => ϑ.coef i {j}
def PublishedEquivalent
Equality of graph and all supported polynomial coefficients.
Definition (Lean source)
parameters
V :
Type u_1
ϑ ψ :
def
PublishedEquivalent ϑ ψ :
Prop
given by
clause 1
j i :
ϑ.edge j i ↔ ψ.edge j i
def ScheduleEquivalent
Equality of schedules modulo unused off-graph values of the coefficient array.
Definition (Lean source)
parameters
V :
Type u_1
θ ψ :
Schedule V
def
ScheduleEquivalent θ ψ :
Prop
given by
clause 1
j i :
θ.edge j i ↔ ψ.edge j i
clause 2
θ.a = ψ.a
clause 3
θ.t = ψ.t
clause 4
i j :
θ.edge j i
θ.b i j = ψ.b i j
def AugGraph
The measurable augmented graph carrier contains every known diagonal arrow.
def AugRecord
The augmented complete record consists of the diagonal-complete graph, assignments, and outcomes.
def adjoinGraph
Add the known diagonal arrows to a recorded off-diagonal labeled graph.
Every diagonal arrow is present after graph augmentation. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: adjoinGraph_diagonal
lemma adjoinGraph_diagonal (H : OffDiag V → Bool) : ∀ i, adjoinGraph H (i,i) = true := by
intro i
simp [adjoinGraph]
def adjoinRecord
Augment only the graph coordinate of the complete record.
Definition (Lean source)
parameters
V :
Type u_1
o :
Record V
def
given by
(⟨adjoinGraph o.1, adjoinGraph_diagonal o.1⟩, o.2)
def removeRecord
Remove known diagonal arrows while preserving all assignments and outcomes.
Removing and re-adjoining diagonals is the identity on augmented records. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: adjoin_remove
lemma adjoin_remove (o : AugRecord V) : adjoinRecord (removeRecord o) = o := by
apply Prod.ext
· apply Subtype.ext
funext e
rcases e with ⟨j, i⟩
by_cases h : j ≠ i
· simp [adjoinRecord, removeRecord, adjoinGraph, h]
· have heq : j = i := not_ne_iff.mp h
subst j
simp [adjoinRecord, removeRecord, adjoinGraph, o.1.2 i]
· rfl
Adjoining and then removing diagonals is the identity on original records. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: remove_adjoin
lemma remove_adjoin (o : Record V) : removeRecord (adjoinRecord o) = o := by
apply Prod.ext
· funext e
simp [removeRecord, adjoinRecord, adjoinGraph, e.2]
· rfl
Adjoining known diagonal arrows is measurable.
Formal statement
Proof (Lean source)
-- @node: adjoinRecord_measurable
@[fun_prop] lemma adjoinRecord_measurable : Measurable (adjoinRecord (V := V)) := by
unfold adjoinRecord
apply Measurable.prodMk _ measurable_snd
apply Measurable.subtype_mk
apply measurable_pi_lambda
intro e
by_cases h : e.1 ≠ e.2
· simpa only [adjoinGraph, dif_pos h] using
(measurable_fst.eval : Measurable (fun o : Record V => o.1 ⟨e, h⟩))
· simpa only [adjoinGraph, dif_neg h] using
(measurable_const : Measurable (fun _ : Record V => true))
Removing known diagonal arrows is measurable.
Formal statement
Proof (Lean source)
-- @node: removeRecord_measurable
@[fun_prop] lemma removeRecord_measurable : Measurable (removeRecord (V := V)) := by
unfold removeRecord
apply prodMk _ measurable_snd
apply measurable_pi_lambda
intro e
exact (measurable_subtype_coe.comp measurable_fst).eval
def augmentRecord
The deterministic measurable equivalence adds the known diagonals.
Definition (Lean source)
parameters
V :
Type u_1
given by
clause 1
toFun := adjoinRecord
clause 2
invFun := removeRecord
clause 3
left_inv := remove_adjoin
clause 4
right_inv := adjoin_remove
clause 5
measurable_toFun := adjoinRecord_measurable
clause 6
measurable_invFun := removeRecord_measurable
def pubRecordOf
The published schedule under the same off-diagonal audit and assignment channel, with known diagonals adjoined.
Definition (Lean source)
parameters
def
given by
def pubRecordLaw
The complete augmented record and independent uniform seed law at a published schedule.
Definition (Lean source)
parameters
given by
def PubEstimator
All measurable seeded real-valued estimators of the augmented record.
Definition (Lean source)
def pubMinimaxRisk
The extended minimax risk over a specified published class and all measurable seeded augmented-record estimators.
Definition (Lean source)
parameters
def
pubMinimaxRisk D 𝔠 :
ℝ≥0∞
given by
minimaxValueENNReal (fun (T : PubEstimator V) (ϑ : 𝔠) => ∫⁻ x, ofReal ((T.1 x - pubTte ϑ.1) ^ 2) ∂(pubRecordLaw D ϑ.1))
theorem augment_coef_empty
Augmentation puts the baseline at the empty monomial. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: augment_coef_empty
lemma augment_coef_empty (θ : Schedule V) (i : V) :
(augment θ).coef i ∅ = θ.a i := by
simp [augment]
theorem augment_coef_singleton
Augmentation puts each treatment effect at its singleton monomial. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
i j :
V
conclusion
(augment θ).coef i {j} = if j = i then θ.t i else if θ.edge j i then θ.b i j else 0
Proof (Lean source)
-- @node: augment_coef_singleton
lemma augment_coef_singleton (θ : Schedule V) (i j : V) :
(augment θ).coef i {j} =
if j = i then θ.t i else if θ.edge j i then θ.b i j else 0 := by
classical
simp [augment, Finset.singleton_inj]
theorem pubNbhd_augment
The augmented neighborhood is the original one with the own coordinate inserted. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: pubNbhd_augment
lemma pubNbhd_augment (θ : Schedule V) (i : V) :
pubNbhd (augment θ) i = insert i (inNbhd θ i) := by
ext j
simp [pubNbhd, augment, inNbhd, or_comm]
theorem strip_augment_equivalent
Stripping the augmented schedule recovers every coefficient that is used by an outcome. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: strip_augment_equivalent
lemma strip_augment_equivalent (θ : Schedule V) :
ScheduleEquivalent (strip (augment θ)) θ := by
refine ⟨?_, ?_, ?_, ?_⟩
· intro j i
change ((θ.edge j i ∨ j = i) ∧ j ≠ i) ↔ θ.edge j i
constructor
· intro h
exact h.1.resolve_right h.2
· intro h
exact ⟨Or.inl h, fun heq => θ.irrefl i (heq ▸ h)⟩
· funext i
exact augment_coef_empty θ i
· funext i
exact (augment_coef_singleton θ i i).trans (if_pos rfl)
· intro i j hij
have hji : j ≠ i := hij.2
have hedge : θ.edge j i := hij.1.resolve_right hji
exact (augment_coef_singleton θ i j).trans (by rw [if_neg hji, if_pos hedge])
theorem augment_orderOne
Only the empty and supported singleton monomials survive augmentation. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: augment_orderOne
lemma augment_orderOne (θ : Schedule V) : PublishedOrderOne (augment θ) := by
constructor
· intro i S hS
have hne : S ≠ ∅ := by
intro heq
exact hS (heq ▸ empty_subset _)
simp only [augment, if_neg hne]
apply Finset.sum_eq_zero
intro j _
by_cases hsj : S = {j}
· have hnot : ¬ (θ.edge j i ∨ j = i) := by
intro hj
apply hS
rw [hsj, singleton_subset_iff, pubNbhd_augment]
simpa [inNbhd, or_comm] using hj
rw [if_pos hsj, if_neg (fun h => hnot (inr h)),
if_neg (fun h => hnot (inl h))]
· exact if_neg hsj
· intro i S hcard
have hne : S ≠ ∅ := by
intro heq
simp [heq] at hcard
simp only [augment, if_neg hne]
apply Finset.sum_eq_zero
intro j _
apply if_neg
intro heq
simp [heq] at hcard
theorem pubOutcome_augment
Under the published polynomial convention, augmentation preserves every outcome. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: pubOutcome_augment
lemma pubOutcome_augment (hconv : SnipeModelConventions (V := V))
(θ : Schedule V) (i : V) (z : Assign V) :
pubOutcome (augment θ) i z = potentialOutcome θ i z := by
rw [(hconv (augment θ) (augment_orderOne θ)).1 i z, augment_coef_empty, pubNbhd_augment]
have hnot : i ∉ inNbhd θ i := by simp [inNbhd, θ.irrefl]
rw [Finset.sum_insert hnot]
rw [augment_coef_singleton θ i i, if_pos rfl]
unfold potentialOutcome
rw [← add_assoc]
congr 1
apply Finset.sum_congr rfl
intro j hj
have hij : θ.edge j i := (Finset.mem_filter.mp hj).2
have hji : j ≠ i := fun heq => θ.irrefl i (heq ▸ hij)
rw [augment_coef_singleton, if_neg hji, if_pos hij]
theorem pubTte_augment
Preserving all outcomes preserves the all-treated causal contrast. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: pubTte_augment
lemma pubTte_augment (hconv : SnipeModelConventions (V := V)) (θ : Schedule V) :
pubTte (augment θ) = tte θ := by
simp only [pubTte, tte, pubOutcome_augment hconv]
theorem pubRecordOf_augment
The augmented original record is exactly the published record under the same channel. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: pubRecordOf_augment
lemma pubRecordOf_augment (hconv : SnipeModelConventions (V := V))
(θ : Schedule V) (ω : Assign V × Audit V) :
augmentRecord (recordOf θ ω) = pubRecordOf (augment θ) ω := by
change adjoinRecord (recordOf θ ω) = pubRecordOf (augment θ) ω
unfold pubRecordOf
congr 1
apply Prod.ext
· funext e
have hne : e.1.1 ≠ e.1.2 := e.2
simp [recordOf, augment, hne]
· apply Prod.ext
· rfl
funext i
exact (pubOutcome_augment hconv θ i ω.1).symm
theorem pubOutNbhd_augment
Augmentation adds exactly the own recipient to each out-neighborhood. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
j :
V
conclusion
Proof (Lean source)
-- @node: pubOutNbhd_augment
lemma pubOutNbhd_augment (θ : Schedule V) (j : V) :
pubOutNbhd (augment θ) j =
insert j (Finset.univ.filter (fun i => j ∈ inNbhd θ i)) := by
ext i
simp [pubOutNbhd, augment, inNbhd, or_comm, eq_comm]
theorem augment_coefficientMass
The published coefficient mass equals the original row coefficient mass. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: augment_coefficientMass
lemma augment_coefficientMass (θ : Schedule V) (i : V) :
|(augment θ).coef i ∅| + ∑ j ∈ pubNbhd (augment θ) i, |(augment θ).coef i {j}| =
|θ.a i| + |θ.t i| + ∑ j ∈ inNbhd θ i, |θ.b i j| := by
rw [augment_coef_empty, pubNbhd_augment]
have hnot : i ∉ inNbhd θ i := by simp [inNbhd, θ.irrefl]
rw [Finset.sum_insert hnot, augment_coef_singleton θ i i, if_pos rfl, ← add_assoc]
congr 1
apply Finset.sum_congr rfl
intro j hj
have hij : θ.edge j i := (Finset.mem_filter.mp hj).2
have hji : j ≠ i := fun heq => θ.irrefl i (heq ▸ hij)
rw [augment_coef_singleton, if_neg hji, if_pos hij]
theorem augment_publishedClass
A degree-d additive schedule maps to a diagonal-complete degree-(d+1) published schedule. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
hconv :
SnipeModelConventions (V := V)
θ :
Schedule V
d :
ℕ
hclass :
ScheduleClass θ d
Proof (Lean source)
-- @node: augment_publishedClass
lemma augment_publishedClass (hconv : SnipeModelConventions (V := V))
(θ : Schedule V) (d : ℕ) (hclass : ScheduleClass θ d) :
PublishedClass (augment θ) (d + 1) ∧ ∀ i, (augment θ).edge i i := by
refine ⟨⟨augment_orderOne θ, ?_, ?_, ?_⟩, fun _ => inr rfl⟩
· rw [(hconv (augment θ) (augment_orderOne θ)).2]
apply Finset.sup_le
intro i _
rw [augment_coefficientMass]
exact Real.toNNReal_le_one.mpr (hclass.coeffMass i)
· intro i
rw [pubNbhd_augment, Finset.card_insert_of_notMem (by simp [inNbhd, θ.irrefl])]
exact Nat.add_le_add_right (hclass.inDegree i) 1
· intro j
rw [pubOutNbhd_augment, Finset.card_insert_of_notMem (by simp [inNbhd, θ.irrefl])]
exact Nat.add_le_add_right (hclass.outDegree j) 1
A diagonal-complete published neighborhood is the stripped neighborhood plus its own vertex. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: pubNbhd_strip
lemma pubNbhd_strip (ϑ : PublishedSchedule V) (hdiag : ∀ i, ϑ.edge i i) (i : V) :
pubNbhd ϑ i = insert i (inNbhd (strip ϑ) i) := by
ext j
constructor
· intro hj
have he := (Finset.mem_filter.mp hj).2
by_cases h : j = i
· exact Finset.mem_insert.mpr (inl h)
· exact Finset.mem_insert.mpr (inr
(Finset.mem_filter.mpr ⟨Finset.mem_univ j, he, h⟩))
· intro hj
rcases Finset.mem_insert.mp hj with h | h
· subst j
exact Finset.mem_filter.mpr ⟨Finset.mem_univ i, hdiag i⟩
· exact Finset.mem_filter.mpr ⟨Finset.mem_univ j, (Finset.mem_filter.mp h).2.1⟩
The same diagonal removal identity holds for out-neighborhoods. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: pubOutNbhd_strip
lemma pubOutNbhd_strip (ϑ : PublishedSchedule V) (hdiag : ∀ i, ϑ.edge i i) (j : V) :
pubOutNbhd ϑ j = insert j (Finset.univ.filter (fun i => j ∈ inNbhd (strip ϑ) i)) := by
ext i
constructor
· intro hi
have he := (Finset.mem_filter.mp hi).2
by_cases h : i = j
· exact Finset.mem_insert.mpr (inl h)
· exact Finset.mem_insert.mpr (inr
(Finset.mem_filter.mpr ⟨Finset.mem_univ i,
Finset.mem_filter.mpr ⟨Finset.mem_univ j, he, Ne.symm h⟩⟩))
· intro hi
rcases Finset.mem_insert.mp hi with h | h
· subst i
exact Finset.mem_filter.mpr ⟨Finset.mem_univ j, hdiag j⟩
· exact Finset.mem_filter.mpr ⟨Finset.mem_univ i,
(Finset.mem_filter.mp (Finset.mem_filter.mp h).2).2.1⟩
theorem strip_scheduleClass
Stripping removes one degree and retains exactly the published row coefficient mass. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
hconv :
SnipeModelConventions (V := V)
ϑ :
d :
ℕ
hclass :
PublishedClass ϑ (d + 1)
hdiag :
∀ i, ϑ.edge i i
conclusion
ScheduleClass (strip ϑ) d
Proof (Lean source)
-- @node: strip_scheduleClass
lemma strip_scheduleClass (hconv : SnipeModelConventions (V := V))
(ϑ : PublishedSchedule V) (d : ℕ) (hclass : PublishedClass ϑ (d + 1))
(hdiag : ∀ i, ϑ.edge i i) : ScheduleClass (strip ϑ) d := by
constructor
· intro i
have hi := hclass.2.2.1 i
rw [pubNbhd_strip ϑ hdiag,
Finset.card_insert_of_notMem (fun h => (Finset.mem_filter.mp h).2.2 rfl)] at hi
exact Nat.le_of_add_le_add_right hi
· intro j
have hj := hclass.2.2.2 j
rw [pubOutNbhd_strip ϑ hdiag,
Finset.card_insert_of_notMem (fun h =>
(Finset.mem_filter.mp (Finset.mem_filter.mp h).2).2.2 rfl)] at hj
exact Nat.le_of_add_le_add_right hj
· intro i
have hm := hclass.2.1
rw [(hconv ϑ hclass.1).2] at hm
have hi := (Finset.le_sup (f := fun i =>
(|ϑ.coef i ∅| + ∑ j ∈ pubNbhd ϑ i, |ϑ.coef i {j}|).toNNReal)
(Finset.mem_univ i)).trans hm
have hr := Real.toNNReal_le_one.mp hi
rw [pubNbhd_strip ϑ hdiag,
Finset.sum_insert (fun h => (Finset.mem_filter.mp h).2.2 rfl), ← add_assoc] at hr
exact hr
Re-augmentation recovers the graph and every supported coefficient of an order-one schedule. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: augment_strip_equivalent
lemma augment_strip_equivalent (ϑ : PublishedSchedule V)
(horder : PublishedOrderOne ϑ) (hdiag : ∀ i, ϑ.edge i i) :
PublishedEquivalent (augment (strip ϑ)) ϑ := by
refine ⟨?_, ?_⟩
· intro j i
change ((ϑ.edge j i ∧ j ≠ i) ∨ j = i) ↔ ϑ.edge j i
by_cases h : j = i
· subst j
simp [hdiag]
· simp [h]
· intro i S hS
by_cases hempty : S = ∅
· subst S
exact augment_coef_empty (strip ϑ) i
by_cases hcard : 1 < S.card
· rw [(augment_orderOne (strip ϑ)).2 i S hcard, horder.2 i S hcard]
· obtain ⟨j, hj⟩ := Finset.nonempty_iff_ne_empty.mpr hempty
have hsingle : S = {j} := by
ext k
constructor
· intro hk
exact Finset.mem_singleton.mpr
(Finset.card_le_one.mp (Nat.le_of_not_gt hcard) k hk j hj)
· intro hk
simpa only [Finset.mem_singleton.mp hk] using hj
subst S
rw [augment_coef_singleton]
by_cases hji : j = i
· subst j
rw [if_pos rfl]
rfl
· have hedge : ϑ.edge j i := by
have hjmem := hS (mem_singleton_self j)
have he := (Finset.mem_filter.mp hjmem).2
exact (he.resolve_right hji).1
rw [if_neg hji, if_pos (show (strip ϑ).edge j i from ⟨hedge, hji⟩)]
rfl
theorem augmented_estimator_loss
Composing a published estimator with record augmentation preserves squared loss. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
sqLoss D θ ⟨fun x
=> T.1 (adjoinRecord x.1, x.2), T.2.comp (adjoinRecord_measurable.comp measurable_fst |>.prodMk measurable_snd)⟩ = ∫⁻ x, ofReal ((T.1 x - pubTte (augment θ)) ^ 2) ∂(pubRecordLaw D (augment θ))
Proof (Lean source)
-- @node: augmented_estimator_loss
lemma augmented_estimator_loss (hconv : SnipeModelConventions (V := V))
(D : Measure (Assign V × Audit V)) (θ : Schedule V) (T : PubEstimator V) :
sqLoss D θ ⟨fun x => T.1 (adjoinRecord x.1, x.2),
T.2.comp (adjoinRecord_measurable.comp measurable_fst |>.prodMk measurable_snd)⟩ =
∫⁻ x, ofReal ((T.1 x - pubTte (augment θ)) ^ 2)
∂(pubRecordLaw D (augment θ)) := by
have hm : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(recordOf θ ω.1, ω.2)) := by
exact ((Measurable.of_discrete : Measurable (recordOf θ)).comp
measurable_fst).prodMk measurable_snd
have hpub : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(pubRecordOf (augment θ) ω.1, ω.2)) := by
exact ((Measurable.of_discrete : Measurable (pubRecordOf (augment θ))).comp
measurable_fst).prodMk measurable_snd
have hT := T.2
have hcomp : Measurable (fun x : Record V × ℝ => T.1 (adjoinRecord x.1, x.2)) :=
T.2.comp (adjoinRecord_measurable.comp measurable_fst |>.prodMk measurable_snd)
unfold sqLoss recordLaw pubRecordLaw
dsimp only
rw [lintegral_map (by fun_prop) hm, lintegral_map (by fun_prop) hpub]
simp only [pubTte_augment hconv, ← pubRecordOf_augment hconv]
rfl
theorem minimaxRisk_le_pubMinimaxRisk
Class inclusion and the measurable record map transfer the minimax lower bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
hconv :
SnipeModelConventions (V := V)
d :
ℕ
𝔠 :
Set (PublishedSchedule V)
hcontains :
∀ θ
if
ScheduleClass θ d
then
augment θ ∈ 𝔠
conclusion
minimaxRisk D d ≤ pubMinimaxRisk D 𝔠
Proof (Lean source)
-- @node: minimaxRisk_le_pubMinimaxRisk
lemma minimaxRisk_le_pubMinimaxRisk (hconv : SnipeModelConventions (V := V))
(D : Measure (Assign V × Audit V)) (d : ℕ) (𝔠 : Set (PublishedSchedule V))
(hcontains : ∀ θ, ScheduleClass θ d → augment θ ∈ 𝔠) :
minimaxRisk D d ≤ pubMinimaxRisk D 𝔠 := by
unfold minimaxRisk pubMinimaxRisk
apply Causalean.Stat.minimaxValueENNReal_le_minimaxValue
intro T
let T' : Estimator V := ⟨fun x => T.1 (adjoinRecord x.1, x.2),
T.2.comp (adjoinRecord_measurable.comp measurable_fst |>.prodMk measurable_snd)⟩
refine ⟨T', ?_⟩
apply worstCaseRiskENNReal_le
intro θ
rw [show sqLoss D θ.1 T' =
∫⁻ x, ofReal ((T.1 x - pubTte (augment θ.1)) ^ 2)
∂(pubRecordLaw D (augment θ.1)) from augmented_estimator_loss hconv D θ.1 T]
exact le_iSup_of_le (⟨augment θ.1, hcontains θ.1 θ.2⟩ : 𝔠) le_rfl
theorem published_model
Conditional on the cited conventions, diagonal augmentation and stripping are inverse specializations, preserve all outcomes and the target, give an invertible record transformation, and transfer lower bounds by class inclusion under the same assignment-audit channel. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
hconv_of_gate :
SnipeModelConventions (V := V)
conclusion
conclusion 2
ScheduleClass (strip ϑ) d
conclusion 6
conclusion 7
minimaxRisk D d ≤ pubMinimaxRisk D 𝔠
Proof (Lean source)
-- @node: lem:published-model
lemma published_model (hconv_of_gate : SnipeModelConventions (V := V)) :
(∀ (θ : Schedule V) d, ScheduleClass θ d → PublishedClass (augment θ) (d + 1) ∧
∀ i, (augment θ).edge i i) ∧
(∀ (ϑ : PublishedSchedule V) d, PublishedClass ϑ (d + 1) → (∀ i, ϑ.edge i i) →
ScheduleClass (strip ϑ) d ∧ PublishedEquivalent (augment (strip ϑ)) ϑ) ∧
(∀ θ : Schedule V, ScheduleEquivalent (strip (augment θ)) θ) ∧
(∀ (θ : Schedule V) (i : V) (z : Assign V),
pubOutcome (augment θ) i z = potentialOutcome θ i z) ∧
(∀ θ : Schedule V, pubTte (augment θ) = tte θ) ∧
(∀ (θ : Schedule V) (ω : Assign V × Audit V),
augmentRecord (recordOf θ ω) = pubRecordOf (augment θ) ω) ∧
(∀ (D : Measure (Assign V × Audit V)) (q : ℝ) (d : ℕ),
AssignmentLaw D → AuditLaw D q → DesignIndependent D →
∀ 𝔠 : Set (PublishedSchedule V), (∀ θ, ScheduleClass θ d → augment θ ∈ 𝔠) →
minimaxRisk D d ≤ pubMinimaxRisk D 𝔠) := by
refine ⟨augment_publishedClass hconv_of_gate, ?_, strip_augment_equivalent,
pubOutcome_augment hconv_of_gate,
pubTte_augment hconv_of_gate, pubRecordOf_augment hconv_of_gate, ?_⟩
· intro ϑ d hclass hdiag
exact ⟨strip_scheduleClass hconv_of_gate ϑ d hclass hdiag,
augment_strip_equivalent ϑ hclass.1 hdiag⟩
· intro D q d _ha _hw _hi 𝔠 hcontains
exact minimaxRisk_le_pubMinimaxRisk hconv_of_gate D d 𝔠 hcontains
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.RetainedRowAverage 9 declarations Enumerate the source fiber of each original partition before reducing the full retained graph to its row counts.
Averaging actual partition-dependent retained rows
Enumerate the source fiber of each original partition before reducing the full retained graph to its row counts.
theorem fixed_partition_revealed_source_iff
Under a fixed original partition, membership in a revealed row is exactly source revelation together with membership in that partition's source fiber. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
↔ ∃ a : Fin (B * d), a.val = j.val ∧ s.1 a = ℓ ∧ sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 a = true
Proof (Lean source)
-- @node: fixed_partition_revealed_source_iff
lemma fixed_partition_revealed_source_iff (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ω : Assign (Fin n) × Audit (Fin n))
(ℓ : Fin B) (j : Fin n) :
j ∈ revealedSources n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 ℓ ↔
∃ a : Fin (B * d), a.val = j.val ∧ s.1 a = ℓ ∧
sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 a = true := by
classical
let H := (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1
have hs (e : OffDiag (Fin n)) (he : H e = true) :
blockEdge n B d s e.1.1 e.1.2 := by
change (decide (blockEdge n B d s e.1.1 e.1.2) && ω.2 e) = true at he
simp only [Bool.and_eq_true, decide_eq_true_eq] at he
exact he.1
constructor
· intro hj
obtain ⟨hjl, i, hi, hji, he⟩ := (Finset.mem_filter.mp hj).2
let a : Fin (B * d) := ⟨j.val, hjl⟩
have hedge := hs ⟨(j,i),hji⟩ he
obtain ⟨_, k, hk, hik⟩ := hedge.1
have hkl := recipientBlock_unique n B d i k ℓ hik hi
refine ⟨a, rfl, ?_, ?_⟩
· simpa [a, hkl] using hk
· simp only [sourceReveals, decide_eq_true_eq]
exact ⟨j, rfl, i, hji, he⟩
· rintro ⟨a, haj, haℓ, ha⟩
obtain ⟨v, hv, i, hvi, he⟩ := of_decide_eq_true ha
have hvj : v = j := Fin.ext (hv.trans haj)
subst v
obtain ⟨_, k, hk, hik⟩ := (hs ⟨(j,i),hvi⟩ he).1
have hka : (⟨j.val, by omega⟩ : Fin (B * d)) = a := Fin.ext haj.symm
have hkℓ : k = ℓ := by simpa [hka, haℓ] using hk.symm
apply Finset.mem_filter.mpr
refine ⟨Finset.mem_univ _, ?_, i, ?_, hvi, he⟩
· simpa [← haj] using a.isLt
· simpa [hkℓ] using hik
theorem fixed_partition_row_enumeration
A size-d source fiber can be enumerated without dropping any labels. The revealed row count then equals the count in its enumerated source-reveal row. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
= (Finset.univ.filter (fun k : Fin d => sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)) = true)).card
Proof (Lean source)
-- @node: fixed_partition_row_enumeration
lemma fixed_partition_row_enumeration (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) :
∃ e : Fin B × Fin d → Fin (B * d), Function.Injective e ∧
∀ ω : Assign (Fin n) × Audit (Fin n), ∀ ℓ : Fin B,
revealedCount n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 ℓ =
(Finset.univ.filter (fun k : Fin d => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)) = true)).card := by
classical
let r (ℓ : Fin B) : Fin d ≃ {a : Fin (B * d) // s.1 a = ℓ} :=
Fintype.equivOfCardEq (by simpa only [Fintype.card_fin, Fintype.card_subtype] using (s.2 ℓ).symm)
let e : Fin B × Fin d → Fin (B * d) := fun x => (r x.1 x.2).val
have he : Function.Injective e := by
intro x y hxy
have hℓ : x.1 = y.1 := by
rw [← (r x.1 x.2).property, ← (r y.1 y.2).property]
exact congrArg s.1 hxy
apply Prod.ext hℓ
have hv : (r x.1 x.2).val = (r x.1 y.2).val := by
change (r x.1 x.2).val = (r y.1 y.2).val at hxy
rw [← hℓ] at hxy
exact hxy
exact (r x.1).injective (Subtype.ext hv)
refine ⟨e, he, ?_⟩
intro ω ℓ
let H := (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1
let v (k : Fin d) : Fin n := ⟨(e (ℓ,k)).val, by have := (e (ℓ,k)).isLt; omega⟩
have hv : Function.Injective v := by
intro k k' hkk
have hh : e (ℓ,k) = e (ℓ,k') := Fin.ext (congrArg (fun j : Fin n => j.val) hkk)
exact congrArg snd (he hh)
have hset : revealedSources n B d H ℓ =
(Finset.univ.filter (fun k : Fin d => sourceReveals n B d H (e (ℓ,k)) = true)).image v := by
ext j
rw [fixed_partition_revealed_source_iff n B d hfit s ω ℓ j]
simp only [mem_image, mem_filter, Finset.mem_univ, true_and]
constructor
· rintro ⟨a, haj, haℓ, ha⟩
obtain ⟨k, hk⟩ := (r ℓ).surjective ⟨a, haℓ⟩
have hek : e (ℓ,k) = a := congrArg val hk
exact ⟨k, by simpa [hek] using ha, Fin.ext (by simpa [v, hek] using haj)⟩
· rintro ⟨k, hk, rfl⟩
exact ⟨e (ℓ,k), rfl, (r ℓ k).property, hk⟩
change (revealedSources n B d H ℓ).card = _
rw [hset, Finset.card_image_of_injective _ hv]
theorem fixed_partition_reveal_rows_law
The enumerated reveals in any fixed original partition have the independent row law, even though the observed retained graph keeps all edge subsets. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(thinnedDesign (Fin n) q).map (fun ω ℓ k => sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)))
Proof (Lean source)
-- @node: fixed_partition_reveal_rows_law
lemma fixed_partition_reveal_rows_law (n B d : ℕ) (q : ℝ)
(hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(s : SourcePartition B d) (e : Fin B × Fin d → Fin (B * d))
(he : Injective e) :
(thinnedDesign (Fin n) q).map (fun ω ℓ k => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k))) =
Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d =>
bernoulliLaw (retentionP d q))) := by
have hsource := block_reveal_fixed_partition n B d q hfit hq s
have hselect := bernoulli_select_coordinates e he (retentionP d q)
(retentionP_mem_Icc d q hq)
have hrows := bernoulli_curry_coordinates (I := Fin B) (J := Fin d)
(retentionP d q) (retentionP_mem_Icc d q hq)
have hfirst := congrArg (fun μ : Measure (Fin (B * d) → Bool) =>
μ.map (fun w x => w (e x))) hsource
rw [Measure.map_map (by fun_prop) (by fun_prop), hselect] at hfirst
have hsecond := congrArg (fun μ : Measure (Fin B × Fin d → Bool) =>
μ.map (fun w ℓ k => w (ℓ,k))) hfirst
rw [Measure.map_map (by fun_prop) (by fun_prop), hrows] at hsecond
exact hsecond
theorem countRevealRow_energy_eq
Canonical count patterns preserve both row energies for any actual reveal row. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: countRevealRow_energy_eq
lemma countRevealRow_energy_eq (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
rowRevealedEnergy d h (countRevealRow d
(Finset.univ.filter (fun j => r j = true)).card) = rowRevealedEnergy d h r ∧
rowHiddenEnergy d h (countRevealRow d
(Finset.univ.filter (fun j => r j = true)).card) = rowHiddenEnergy d h r := by
classical
apply rowRevealEnergy_eq_of_hidden_count_eq
rw [(countRevealRow_counts d _ (by
exact (Finset.card_filter_le _ _).trans_eq (by simp))).2]
have hc : Finset.univ.filter (fun j : Fin d => r j = false) =
univ \ Finset.univ.filter (fun j : Fin d => r j = true) := by
ext j
cases hj : r j <;> simp [hj]
rw [hc, Finset.card_sdiff_of_subset (filter_subset _ _)]
simp
theorem fixed_partition_envelope_eq
The nonconstant retained envelope is the independent-row expression under any label-preserving enumeration of the original source partition. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h :
ℝ
s :
SourcePartition B d
hc :
∀ ℓ : Fin B,
= (Finset.univ.filter (fun k : Fin d => sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)) = true)).card
conclusion
+ E.card
= 0 then 0 else (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, rowRevealedEnergy d h (fun k => sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)))) * (∏ ℓ ∈ E, rowHiddenEnergy d h (fun k => sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)))))
Proof (Lean source)
-- @node: fixed_partition_envelope_eq
lemma fixed_partition_envelope_eq (n B d : ℕ) (h : ℝ)
(s : SourcePartition B d) (e : Fin B × Fin d → Fin (B * d))
(hc : ∀ ω : Assign (Fin n) × Audit (Fin n), ∀ ℓ : Fin B,
revealedCount n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 ℓ =
(Finset.univ.filter (fun k : Fin d => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)) = true)).card)
(ω : Assign (Fin n) × Audit (Fin n)) :
retainedRowEnergyEnvelope n B d h
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 =
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then 0 else
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (fun k => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k)))) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (fun k => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k))))) := by
unfold retainedRowEnergyEnvelope
simp_rw [hc ω, (countRevealRow_energy_eq d h _).1,
(countRevealRow_energy_eq d h _).2]
theorem retainedGraphMarginal_partition_sum
The exact graph marginal is the uniform mixture of the fixed-partition graph laws, before any row-count reduction. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
retainedGraphMarginal n B d D
= (card (SourcePartition B d) : ℝ≥0∞)⁻¹ • ∑ s : SourcePartition B d, D.map (fun ω => (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1)
Proof (Lean source)
-- @node: retainedGraphMarginal_partition_sum
lemma retainedGraphMarginal_partition_sum (n B d : ℕ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) :
retainedGraphMarginal n B d D =
(card (SourcePartition B d) : ℝ≥0∞)⁻¹ •
∑ s : SourcePartition B d, D.map (fun ω =>
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1) := by
unfold retainedGraphMarginal graphAssignMarginal partitionLaw
rw [Measure.bind_smul, ← Measure.sum_fintype,
Measure.bind_sum _ _ (by fun_prop), Measure.sum_fintype]
have hm : Measurable (fun s : SourcePartition B d =>
D.map (fun ω => ((recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1, ω.1))) := by fun_prop
simp_rw [Measure.dirac_bind hm]
rw [Measure.map_smul, ← Measure.sum_fintype, Measure.map_sum (by fun_prop),
Measure.sum_fintype]
congr 1
apply Finset.sum_congr rfl
intro s _
rw [Measure.map_map (by fun_prop) (by fun_prop)]
rfl
theorem retainedRowEnergyEnvelope_nonneg
Each actual retained-row contribution is nonnegative, so removing the good indicator is valid before partitionwise averaging. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
0 ≤ retainedRowEnergyEnvelope n B d h H
Proof (Lean source)
-- @node: retainedRowEnergyEnvelope_nonneg
lemma retainedRowEnergyEnvelope_nonneg (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) : 0 ≤ retainedRowEnergyEnvelope n B d h H := by
unfold retainedRowEnergyEnvelope
apply Finset.sum_nonneg
intro A _
apply Finset.sum_nonneg
intro E _
split_ifs
· exact le_rfl
· apply mul_nonneg (by positivity)
apply mul_nonneg
· exact Finset.prod_nonneg (fun ℓ _ => (rowRevealEnergy_bounds d h _).1)
· exact Finset.prod_nonneg (fun ℓ _ => (rowRevealEnergy_bounds d h _).2.1)
theorem fixed_partition_row_energy_average_le
Fixed-partition averaging of the actual row-count envelope uses independent reveals only after its original labels have been enumerated. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ ω, retainedRowEnergyEnvelope n B d h (recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 ∂thinnedDesign (Fin n) q)
+ b
= 0 then 0 else (B.choose a : ℝ) * ((B - a).choose b : ℝ) * (min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b * (retentionP d q * etaOne d h) ^ a * eta d h ^ b
Proof (Lean source)
-- @node: fixed_partition_row_energy_average_le
lemma fixed_partition_row_energy_average_le (n B d : ℕ) (h q : ℝ)
(hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(s : SourcePartition B d) :
(∫ ω, retainedRowEnergyEnvelope n B d h
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1
∂thinnedDesign (Fin n) q) ≤
∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a + b = 0 then 0 else
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b *
(retentionP d q * etaOne d h) ^ a * eta d h ^ b := by
classical
obtain ⟨e, he, hc⟩ := fixed_partition_row_enumeration n B d hfit s
let F (rows : Fin B → Fin d → Bool) : ℝ :=
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then 0 else
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)))
let rows (ω : Assign (Fin n) × Audit (Fin n)) := fun ℓ k => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 (e (ℓ,k))
have hpoint (ω) : retainedRowEnergyEnvelope n B d h
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 = F (rows ω) :=
fixed_partition_envelope_eq n B d h s e hc ω
simp_rw [hpoint]
have hlaw := fixed_partition_reveal_rows_law n B d q hfit hq s e he
change (∫ ω, F (rows ω) ∂thinnedDesign (Fin n) q) ≤ _
rw [← integral_map (by fun_prop : AEMeasurable rows (thinnedDesign (Fin n) q))
(by fun_prop : AEStronglyMeasurable F ((thinnedDesign (Fin n) q).map rows)), hlaw]
simpa only [Set.indicator_univ] using
active_row_nonconstant_average_le B d h (retentionP d q)
(retentionP_mem_Icc d q hq) univ
theorem retained_row_energy_average_le
Averaging the full detailed graph mixture gives the finite nonconstant contraction series. No source or edge labels are discarded in forming the law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ H, indicator {H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H} (retainedRowEnergyEnvelope n B d h) H ∂retainedGraphMarginal n B d (thinnedDesign (Fin n) q))
+ b
= 0 then 0 else (B.choose a : ℝ) * ((B - a).choose b : ℝ) * (min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b * (retentionP d q * etaOne d h) ^ a * eta d h ^ b
Proof (Lean source)
-- @node: retained_row_energy_average_le
lemma retained_row_energy_average_le (n B d : ℕ) (h q : ℝ)
(hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1) :
(∫ H, indicator {H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H}
(retainedRowEnergyEnvelope n B d h) H
∂retainedGraphMarginal n B d (thinnedDesign (Fin n) q)) ≤
∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a + b = 0 then 0 else
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b *
(retentionP d q * etaOne d h) ^ a * eta d h ^ b := by
classical
let _ := bernoulliLaw_probability q hq
let _ : IsProbabilityMeasure (thinnedDesign (Fin n) q) := by
exact design_isProbabilityMeasure (thinnedDesign (Fin n) q) q
(thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)
let C : ℝ := ∑ a ∈ range (B + 1), ∑ b ∈ range (B + 1),
if a + b = 0 then 0 else
(B.choose a : ℝ) * ((B - a).choose b : ℝ) *
(min 1 (4 * (a + b : ℕ) / (B : ℝ))) ^ b *
(retentionP d q * etaOne d h) ^ a * eta d h ^ b
let _ := sourcePartition_nonempty B d
have hprob := partitionLaw_probability B d
rw [retainedGraphMarginal_partition_sum, integral_smul_measure,
integral_finsetSum_measure (fun _ _ => Integrable.of_finite)]
have havg : ∀ s : SourcePartition B d,
(∫ H, indicator {H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H}
(retainedRowEnergyEnvelope n B d h) H
∂(thinnedDesign (Fin n) q).map (fun ω =>
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1)) ≤ C := by
intro s
rw [integral_map (by fun_prop) (by fun_prop)]
apply le_trans _ (fixed_partition_row_energy_average_le n B d h q hfit hq s)
apply integral_mono
· first | fun_prop | exact Integrable.of_finite
· first | fun_prop | exact Integrable.of_finite
intro ω
by_cases hg : (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1
· simp only [indicator, Set.mem_setOf_eq, hg, if_true, le_refl]
· simpa only [indicator, Set.mem_setOf_eq, hg, if_false] using
retainedRowEnergyEnvelope_nonneg n B d h _
calc
_ ≤ ((card (SourcePartition B d) : ℝ≥0∞)⁻¹).toReal *
∑ _s : SourcePartition B d, C :=
mul_le_mul_of_nonneg_left (Finset.sum_le_sum (fun s _ => havg s)) (by positivity)
_ = C := by
rw [Finset.sum_const, Finset.card_univ, nsmul_eq_mul, ENNReal.toReal_inv,
ENNReal.toReal_natCast]
have hn : (card (SourcePartition B d) : ℝ) ≠ 0 := by
exact_mod_cast Fintype.card_ne_zero
field_simp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.RevealedSubsetEnergyBridge 13 declarations Remove masked inactive rows and reindex the original revealed-label subsets by families of subsets of the disjoint revealed rows.
Exact row factors in the revealed-label energy envelope
Remove masked inactive rows and reindex the original revealed-label subsets by families of subsets of the disjoint revealed rows. All original labels remain in the bijection; only their rowwise energy factors are simplified.
The constant Walsh coefficient has unit reference energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: gamma_zero_energy
lemma gamma_zero_energy (d : ℕ) (h : ℝ) : gamma d h 0 = 1 := by
unfold gamma
calc
_ = ∫ w, refDensity d h w := by
apply integral_congr_ae
filter_upwards [] with w
by_cases hz : refDensity d h w = 0
· simp [hz]
· have hp := lt_of_le_of_ne (refDensity_nonneg d h w) (Ne.symm hz)
rw [walshCoeff_zero d h w hp]
ring
_ = 1 := (refDensity_integrable_normalized d h).2
theorem activeSupportEnergyBound_row_factors
On disjoint active rows, the masked product consists of the unrestricted revealed-degree factors and the positive-degree hidden factors; all other rows contribute exactly one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
activeSupportEnergyBound B d h u j
= let A := Finset.univ.filter (fun ℓ => j ℓ ≠ 0) ∑ E ∈ (univ \ A).powerset, (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, ∑ t : Fin (u ℓ + 1), ((u ℓ).choose t.val : ℝ) * gamma d h (j ℓ + t.val)) * (∏ ℓ ∈ E, ∑ t : Fin (u ℓ + 1), if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0))
Proof (Lean source)
-- @node: activeSupportEnergyBound_row_factors
lemma activeSupportEnergyBound_row_factors (B d : ℕ) (h : ℝ)
(u j : Fin B → ℕ) :
activeSupportEnergyBound B d h u j =
let A := Finset.univ.filter (fun ℓ => j ℓ ≠ 0)
∑ E ∈ (univ \ A).powerset,
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, ∑ t : Fin (u ℓ + 1),
((u ℓ).choose t.val : ℝ) * gamma d h (j ℓ + t.val)) *
(∏ ℓ ∈ E, ∑ t : Fin (u ℓ + 1),
if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0)) := by
classical
unfold activeSupportEnergyBound
dsimp only
apply Finset.sum_congr rfl
intro E hE
congr 1
let A := Finset.univ.filter (fun ℓ => j ℓ ≠ 0)
let R (ℓ : Fin B) := ∑ t : Fin (u ℓ + 1),
((u ℓ).choose t.val : ℝ) * gamma d h (j ℓ + t.val)
let U (ℓ : Fin B) := ∑ t : Fin (u ℓ + 1),
if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0
have hEA : ∀ ℓ ∈ E, ℓ ∉ A := by
intro ℓ hℓ
exact (Finset.mem_sdiff.mp (Finset.mem_powerset.mp hE hℓ)).2
calc
_ = ∏ ℓ : Fin B, (if ℓ ∈ A then R ℓ else 1) *
(if ℓ ∈ E then U ℓ else 1) := by
apply Finset.prod_congr rfl
intro ℓ _
by_cases hA : ℓ ∈ A
· have hj : j ℓ ≠ 0 := (Finset.mem_filter.mp hA).2
have hn : ℓ ∉ E := fun hℓ => hEA ℓ hℓ hA
have hc : activeCapacity B u (A ∪ E) ℓ = u ℓ := by
simp [activeCapacity, hA]
change (∑ t : Fin (activeCapacity B u (A ∪ E) ℓ + 1),
if (if j ℓ ≠ 0 then True else if ℓ ∈ A ∪ E then 0 < t.val else t.val = 0) then
((activeCapacity B u (A ∪ E) ℓ).choose t.val : ℝ) *
gamma d h (j ℓ + t.val) else 0) = _
rw [hc]
simp [hj, hA, hn, R]
· have hj : j ℓ = 0 := by simpa [A] using hA
by_cases hℓ : ℓ ∈ E
· have hc : activeCapacity B u (A ∪ E) ℓ = u ℓ := by
simp [activeCapacity, hℓ]
change (∑ t : Fin (activeCapacity B u (A ∪ E) ℓ + 1),
if (if j ℓ ≠ 0 then True else if ℓ ∈ A ∪ E then 0 < t.val else t.val = 0) then
((activeCapacity B u (A ∪ E) ℓ).choose t.val : ℝ) *
gamma d h (j ℓ + t.val) else 0) = _
rw [hc]
simp [hj, hA, hℓ, U]
· have hc : activeCapacity B u (A ∪ E) ℓ = 0 := by
simp [activeCapacity, hA, hℓ]
change (∑ t : Fin (activeCapacity B u (A ∪ E) ℓ + 1),
if (if j ℓ ≠ 0 then True else if ℓ ∈ A ∪ E then 0 < t.val else t.val = 0) then
((activeCapacity B u (A ∪ E) ℓ).choose t.val : ℝ) *
gamma d h (j ℓ + t.val) else 0) = _
rw [hc]
simp [hj, hA, hℓ, gamma_zero_energy]
_ = _ := by
rw [Finset.prod_mul_distrib]
rw [Fintype.prod_ite_mem, Fintype.prod_ite_mem]
theorem rowRevealedEnergy_subset_degree_sum
Summing nonempty revealed subsets and every hidden degree is precisely revealed row energy, rather than an independent hidden-row approximation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: rowRevealedEnergy_subset_degree_sum
lemma rowRevealedEnergy_subset_degree_sum (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
(∑ J ∈ (Finset.univ.filter (fun j : Fin d => r j = true)).powerset,
if J = ∅ then 0 else
∑ t : Fin ((Finset.univ.filter (fun j : Fin d => r j = false)).card + 1),
(((Finset.univ.filter (fun j : Fin d => r j = false)).card).choose t.val : ℝ) *
gamma d h (J.card + t.val)) = rowRevealedEnergy d h r := by
classical
let R := Finset.univ.filter (fun j : Fin d => r j = true)
let U := Finset.univ.filter (fun j : Fin d => r j = false)
have hRU : Disjoint R U := by
apply Finset.disjoint_left.mpr
intro j hj hk
have ht := (Finset.mem_filter.mp hj).2
have hf := (Finset.mem_filter.mp hk).2
simp [ht] at hf
have hcover : R ∪ U = univ := by
ext j
cases hr : r j <;> simp [R, U, hr]
have hsum (J : Finset (Fin d)) :
(∑ t : Fin (U.card + 1), (U.card.choose t.val : ℝ) *
gamma d h (J.card + t.val)) = ∑ K ∈ U.powerset, gamma d h (J.card + K.card) := by
rw [Fin.sum_univ_eq_sum_range (fun k => (U.card.choose k : ℝ) *
gamma d h (J.card + k))]
simpa only [nsmul_eq_mul] using
(Finset.sum_powerset_apply_card (fun k => gamma d h (J.card + k)) (x := U)).symm
change (∑ J ∈ R.powerset, if J = ∅ then 0 else
∑ t : Fin (U.card + 1), (U.card.choose t.val : ℝ) * gamma d h (J.card + t.val)) = _
simp_rw [hsum]
rw [rowRevealedEnergy, ← Finset.powerset_univ, ← hcover,
labeled_walsh_powerset_split R U hRU]
apply Finset.sum_congr rfl
intro J hJ
have hJR := Finset.mem_powerset.mp hJ
by_cases hz : J = ∅
· subst J
simp only [if_true, Finset.empty_union]
symm
apply Finset.sum_eq_zero
intro K hK
have hn : ¬ ∃ j ∈ K, r j = true := by
rintro ⟨j, hj, ht⟩
have hf := (Finset.mem_filter.mp (Finset.mem_powerset.mp hK hj)).2
simp [ht] at hf
simp [hn]
· simp only [hz, if_false]
apply Finset.sum_congr rfl
intro K hK
have hKU := Finset.mem_powerset.mp hK
obtain ⟨j, hj⟩ := Finset.nonempty_iff_ne_empty.mpr hz
have hh : ∃ j ∈ J ∪ K, r j = true :=
⟨j, mem_union_left K hj, (Finset.mem_filter.mp (hJR hj)).2⟩
have hne : J ∪ K ≠ ∅ := by
exact Finset.nonempty_iff_ne_empty.mp ⟨j, mem_union_left K hj⟩
rw [if_pos hh, rowSubsetEnergy, if_neg hne,
Finset.card_union_of_disjoint (hRU.mono hJR hKU)]
theorem labeled_revealed_subset_energy_eq
The revealed-subset energy factor depends only on the revealed and hidden counts, and therefore agrees with any row reveal pattern of those counts. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ J ∈ R.powerset, if J = ∅ then 0 else ∑ t : Fin (u + 1), (u.choose t.val : ℝ) * gamma d h (J.card + t.val))
= rowRevealedEnergy d h r
Proof (Lean source)
-- @node: labeled_revealed_subset_energy_eq
lemma labeled_revealed_subset_energy_eq {α : Type*} [DecidableEq α]
(d : ℕ) (h : ℝ) (R : Finset α) (u : ℕ) (r : Fin d → Bool)
(hR : R.card = (Finset.univ.filter (fun j : Fin d => r j = true)).card)
(hu : u = (Finset.univ.filter (fun j : Fin d => r j = false)).card) :
(∑ J ∈ R.powerset, if J = ∅ then 0 else
∑ t : Fin (u + 1), (u.choose t.val : ℝ) * gamma d h (J.card + t.val)) =
rowRevealedEnergy d h r := by
classical
rw [← rowRevealedEnergy_subset_degree_sum d h r, ← hu]
let F (k : ℕ) : ℝ := if k = 0 then 0 else
∑ t : Fin (u + 1), (u.choose t.val : ℝ) * gamma d h (k + t.val)
simp_rw [← Finset.card_eq_zero]
change (∑ J ∈ R.powerset, F J.card) =
∑ J ∈ (Finset.univ.filter (fun j : Fin d => r j = true)).powerset, F J.card
simp_rw [Finset.sum_powerset_apply_card]
rw [hR]
A canonical reveal pattern with r revealed slots preserves the two row counts. It is used only to express energies, after the full-label reindexing.
theorem countRevealRow_counts
The canonical count pattern has exactly r revealed slots and d-r hidden slots. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d r :
ℕ
hr :
r ≤ d
conclusion
conclusion 1
conclusion 2
Proof (Lean source)
-- @node: countRevealRow_counts
lemma countRevealRow_counts (d r : ℕ) (hr : r ≤ d) :
(Finset.univ.filter (fun j : Fin d => countRevealRow d r j = true)).card = r ∧
(Finset.univ.filter (fun j : Fin d => countRevealRow d r j = false)).card = d - r := by
classical
let e : Fin r ↪ Fin d := ⟨fun j => ⟨j.val, by omega⟩, fun a b hab =>
Fin.ext (congrArg (fun k : Fin d => k.val) hab)⟩
have he : Finset.univ.filter (fun j : Fin d => countRevealRow d r j = true) =
Finset.univ.map e := by
ext j
simp only [mem_filter, Finset.mem_univ, true_and, countRevealRow,
decide_eq_true_eq, Finset.mem_map]
constructor
· intro hj
exact ⟨⟨j.val, hj⟩, rfl⟩
· rintro ⟨k, rfl⟩
exact k.isLt
have hc : (Finset.univ.filter (fun j : Fin d => countRevealRow d r j = true)).card = r := by
rw [he, Finset.card_map]; simp
refine ⟨hc, ?_⟩
have hcomp : Finset.univ.filter (fun j : Fin d => countRevealRow d r j = false) =
univ \ Finset.univ.filter (fun j : Fin d => countRevealRow d r j = true) := by
ext j
cases hj : countRevealRow d r j <;> simp [hj]
rw [hcomp, Finset.card_sdiff_of_subset (filter_subset _ _), hc]
simp
theorem revealed_active_subset_product
At a fixed revealed-active set, the row-subset sum factors into independent nonempty-subset sums; inactive rows have the unique empty selection. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: revealed_active_subset_product
lemma revealed_active_subset_product {α : Type*} [DecidableEq α]
(B : ℕ) (R : Fin B → Finset α) (A : Finset (Fin B))
(F : Fin B → Finset α → ℝ) :
(∑ J ∈ (piFinset (fun ℓ => (R ℓ).powerset)).filter
(fun J => Finset.univ.filter (fun ℓ => J ℓ ≠ ∅) = A),
∏ ℓ ∈ A, F ℓ (J ℓ)) =
∏ ℓ ∈ A, ∑ J ∈ (R ℓ).powerset, if J = ∅ then 0 else F ℓ J := by
classical
let S (ℓ : Fin B) := if ℓ ∈ A then (R ℓ).powerset.erase ∅ else {∅}
have hsets : (piFinset (fun ℓ => (R ℓ).powerset)).filter
(fun J => Finset.univ.filter (fun ℓ => J ℓ ≠ ∅) = A) = piFinset S := by
ext J
simp only [mem_filter, Fintype.mem_piFinset]
constructor
· rintro ⟨hJ, hA⟩ ℓ
have hm : J ℓ ≠ ∅ ↔ ℓ ∈ A := by
rw [← hA]; simp
by_cases hℓ : ℓ ∈ A
· simp [S, hℓ, (hm.mpr hℓ), hJ ℓ]
· have hz : J ℓ = ∅ := by tauto
simp [S, hℓ, hz]
· intro hJ
have hm (ℓ) : J ℓ ≠ ∅ ↔ ℓ ∈ A := by
have hh := hJ ℓ
by_cases hℓ : ℓ ∈ A
· have hn := (Finset.mem_erase.mp
(show J ℓ ∈ (R ℓ).powerset.erase ∅ by simpa [S, hℓ] using hh)).1
simp [hn, hℓ]
· simp only [S, hℓ, if_false, Finset.mem_singleton] at hh
simp [hh, hℓ]
refine ⟨?_, ?_⟩
· intro ℓ
have hh := hJ ℓ
by_cases hℓ : ℓ ∈ A
· exact (Finset.mem_erase.mp (by simpa [S, hℓ] using hh)).2
· have hz : J ℓ = ∅ := by simpa [S, hℓ] using hh
simp [hz]
· ext ℓ
simp [hm ℓ]
rw [hsets]
calc
_ = ∑ J ∈ piFinset S, ∏ ℓ : Fin B,
if ℓ ∈ A then F ℓ (J ℓ) else 1 := by
simp_rw [Fintype.prod_ite_mem]
_ = ∏ ℓ : Fin B, ∑ J ∈ S ℓ, if ℓ ∈ A then F ℓ J else 1 :=
(Finset.prod_univ_sum S (fun ℓ J => if ℓ ∈ A then F ℓ J else 1)).symm
_ = ∏ ℓ : Fin B, if ℓ ∈ A then
(∑ J ∈ (R ℓ).powerset, if J = ∅ then 0 else F ℓ J) else 1 := by
apply Finset.prod_congr rfl
intro ℓ _
by_cases hℓ : ℓ ∈ A
· simp only [S, hℓ, if_true]
rw [Finset.sum_ite]
simp only [Finset.sum_const_zero, zero_add]
congr 1
ext J
simp [Finset.mem_erase, and_comm]
· simp [S, hℓ]
_ = _ := Fintype.prod_ite_mem A _
def rowSubsetActiveEnvelope
The exact row-subset envelope before revealed-active rows are grouped.
Definition (Lean source)
parameters
def
rowSubsetActiveEnvelope B d h R u :
ℝ
given by
∑ J ∈ piFinset (fun ℓ => (R ℓ).powerset), let A := Finset.univ.filter (fun ℓ => J ℓ ≠ ∅) ∑ E ∈ (univ \ A).powerset, (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, ∑ t : Fin (u ℓ + 1), ((u ℓ).choose t.val : ℝ) * gamma d h ((J ℓ).card + t.val)) * (∏ ℓ ∈ E, ∑ t : Fin (u ℓ + 1), if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0))
theorem rowSubsetActiveEnvelope_active_rows
Grouping the exact revealed-subset families by their nonempty rows gives products of row-subset energies for every disjoint pair of active sets. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
rowSubsetActiveEnvelope B d h R u
= ∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset, (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, ∑ J ∈ (R ℓ).powerset, if J = ∅ then 0 else ∑ t : Fin (u ℓ + 1), ((u ℓ).choose t.val : ℝ) * gamma d h (J.card + t.val)) * (∏ ℓ ∈ E, ∑ t : Fin (u ℓ + 1), if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0))
Proof (Lean source)
-- @node: rowSubsetActiveEnvelope_active_rows
lemma rowSubsetActiveEnvelope_active_rows {α : Type*} [DecidableEq α]
(B d : ℕ) (h : ℝ) (R : Fin B → Finset α) (u : Fin B → ℕ) :
rowSubsetActiveEnvelope B d h R u =
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, ∑ J ∈ (R ℓ).powerset, if J = ∅ then 0 else
∑ t : Fin (u ℓ + 1), ((u ℓ).choose t.val : ℝ) *
gamma d h (J.card + t.val)) *
(∏ ℓ ∈ E, ∑ t : Fin (u ℓ + 1),
if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0)) := by
classical
let F (ℓ : Fin B) (J : Finset α) := ∑ t : Fin (u ℓ + 1),
((u ℓ).choose t.val : ℝ) * gamma d h (J.card + t.val)
let U (ℓ : Fin B) := ∑ t : Fin (u ℓ + 1),
if 0 < t.val then ((u ℓ).choose t.val : ℝ) * gamma d h t.val else 0
let g (J : Fin B → Finset α) := Finset.univ.filter (fun ℓ => J ℓ ≠ ∅)
let c (A E : Finset (Fin B)) :=
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card
unfold rowSubsetActiveEnvelope
change (∑ J ∈ piFinset (fun ℓ => (R ℓ).powerset),
∑ E ∈ (univ \ g J).powerset,
c (g J) E * ((∏ ℓ ∈ g J, F ℓ (J ℓ)) * ∏ ℓ ∈ E, U ℓ)) = _
rw [← Finset.sum_fiberwise (piFinset (fun ℓ => (R ℓ).powerset)) g]
apply Finset.sum_congr rfl
intro A _
calc
_ = ∑ J ∈ (piFinset (fun ℓ => (R ℓ).powerset)).filter (fun J => g J = A),
∑ E ∈ (univ \ A).powerset,
c A E * ((∏ ℓ ∈ A, F ℓ (J ℓ)) * ∏ ℓ ∈ E, U ℓ) := by
apply Finset.sum_congr rfl
intro J hJ
rw [(Finset.mem_filter.mp hJ).2]
_ = _ := by
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro E _
simp_rw [← mul_assoc]
rw [← Finset.sum_mul, ← Finset.mul_sum]
rw [revealed_active_subset_product B R A F]
theorem rowSubsetActiveEnvelope_eq_row_energies
When row counts match, the exact active-set envelope is the product of the revealed and hidden row energies used by independent-row averaging. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
α :
Type*
B d :
ℕ
h :
ℝ
u :
Fin B → ℕ
hR :
∀ ℓ, (R ℓ).card = (Finset.univ.filter (fun j => r ℓ j = true)).card
hu :
∀ ℓ, u ℓ = (Finset.univ.filter (fun j => r ℓ j = false)).card
conclusion
rowSubsetActiveEnvelope B d h R u
= ∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset, (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, rowRevealedEnergy d h (r ℓ)) * (∏ ℓ ∈ E, rowHiddenEnergy d h (r ℓ)))
Proof (Lean source)
-- @node: rowSubsetActiveEnvelope_eq_row_energies
lemma rowSubsetActiveEnvelope_eq_row_energies {α : Type*} [DecidableEq α]
(B d : ℕ) (h : ℝ) (R : Fin B → Finset α) (u : Fin B → ℕ)
(r : Fin B → Fin d → Bool)
(hR : ∀ ℓ, (R ℓ).card = (Finset.univ.filter (fun j => r ℓ j = true)).card)
(hu : ∀ ℓ, u ℓ = (Finset.univ.filter (fun j => r ℓ j = false)).card) :
rowSubsetActiveEnvelope B d h R u =
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (r ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (r ℓ))) := by
rw [rowSubsetActiveEnvelope_active_rows]
simp_rw [labeled_revealed_subset_energy_eq d h _ _ _ (hR _) (hu _),
rowHiddenEnergy_fin_degree_sum d h _ _ (hu _)]
theorem actualLabelActiveEnvelope_eq_rowSubset
The original label enumeration and the disjoint row-subset family have identical envelopes, including the constant coefficient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
actualLabelActiveEnvelope n B d h H
Proof (Lean source)
-- @node: actualLabelActiveEnvelope_eq_rowSubset
lemma actualLabelActiveEnvelope_eq_rowSubset (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (s : CompatiblePartition n B d H) :
actualLabelActiveEnvelope n B d h H =
rowSubsetActiveEnvelope B d h (revealedSources n B d H) (capacity n B d H) := by
unfold actualLabelActiveEnvelope rowSubsetActiveEnvelope
have hr := disjoint_rowSubset_sum_reindex (revealedSources n B d H)
(revealedSources_pairwise_disjoint n B d H s)
(fun J => activeSupportEnergyBound B d h (capacity n B d H)
(fun ℓ => (J ℓ).card))
rw [← revealedLabelEmbedding_univ_map n B d H, labeled_powerset_map,
Finset.sum_map] at hr
simp only [Finset.powerset_univ] at hr
trans ∑ J ∈ piFinset (fun ℓ => (revealedSources n B d H ℓ).powerset),
activeSupportEnergyBound B d h (capacity n B d H) (fun ℓ => (J ℓ).card)
· exact hr.symm
apply Finset.sum_congr rfl
intro J _
rw [activeSupportEnergyBound_row_factors]
simp only [ne_eq, Finset.card_eq_zero]
Nonconstant disjoint-row envelope expressed through the actual row counts.
Definition (Lean source)
parameters
def
retainedRowEnergyEnvelope n B d h H :
ℝ
given by
= 0 then 0 else (min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card * ((∏ ℓ ∈ A, rowRevealedEnergy d h (countRevealRow d (revealedCount n B d H ℓ))) * (∏ ℓ ∈ E, rowHiddenEnergy d h (countRevealRow d (revealedCount n B d H ℓ))))
theorem actualLabelActiveEnvelope_sub_one_eq_row_energy
Summing all actual revealed labels gives the nonconstant disjoint-row energy envelope, with exactly one constant term removed. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
actualLabelActiveEnvelope n B d h H - 1 = retainedRowEnergyEnvelope n B d h H
Proof (Lean source)
-- @node: actualLabelActiveEnvelope_sub_one_eq_row_energy
lemma actualLabelActiveEnvelope_sub_one_eq_row_energy (n B d : ℕ) (h : ℝ)
(H : OffDiag (Fin n) → Bool) (hH : ValidRetainedGraph n B d H) :
actualLabelActiveEnvelope n B d h H - 1 = retainedRowEnergyEnvelope n B d h H := by
classical
obtain ⟨hfit, s, hs⟩ := hH
have hc (ℓ : Fin B) := countRevealRow_counts d (revealedCount n B d H ℓ)
(revealedCount_le n B d H ⟨hfit, s, hs⟩ ℓ)
rw [actualLabelActiveEnvelope_eq_rowSubset n B d h H ⟨s, hs⟩,
rowSubsetActiveEnvelope_eq_row_energies B d h (revealedSources n B d H) (capacity n B d H)
(fun ℓ => countRevealRow d (revealedCount n B d H ℓ))
(fun ℓ => (hc ℓ).1.symm) (fun ℓ => (hc ℓ).2.symm)]
let F (A E : Finset (Fin B)) : ℝ :=
(min 1 (4 * (A.card + E.card : ℕ) / (B : ℝ))) ^ E.card *
((∏ ℓ ∈ A, rowRevealedEnergy d h (countRevealRow d (revealedCount n B d H ℓ))) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (countRevealRow d (revealedCount n B d H ℓ))))
change (∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset, F A E) - 1 =
∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then 0 else F A E
have hcst : (∑ A : Finset (Fin B), ∑ E ∈ (univ \ A).powerset,
if A.card + E.card = 0 then (1 : ℝ) else 0) = 1 := by
simp [Finset.card_eq_zero, ite_and]
have ht (A E : Finset (Fin B)) : F A E =
(if A.card + E.card = 0 then 1 else 0) +
(if A.card + E.card = 0 then 0 else F A E) := by
by_cases hz : A.card + E.card = 0
· have hA : A = ∅ := Finset.card_eq_zero.mp (by omega)
have hE : E = ∅ := Finset.card_eq_zero.mp (by omega)
simp [F, hA, hE]
· simp only [hz, if_false, zero_add]
conv_lhs => arg 1; arg 2; ext A; arg 2; ext E; rw [ht A E]
simp only [Finset.sum_add_distrib]
rw [hcst]
ring
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ReverseBaselineMoments 7 declarations Evenness removes the baseline mean and independent baseline coordinates remove cross-row covariance.
Integrating the independent baselines in the observable reverse test
Evenness removes the baseline mean and independent baseline coordinates remove cross-row covariance. Compact support bounds the remaining diagonal energy.
theorem reverse_baseline_coordinate_integrable Lemma reverse_baseline_coordinate_integrable in the paper ↗
A polynomial of a baseline coordinate is integrable on its compact support. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
i :
Fin B
conclusion
conclusion 1
Integrable (fun U => U i) (blockBaselineLaw B)
conclusion 2
Integrable (fun U => (U i) ^ 2) (blockBaselineLaw B)
Proof (Lean source)
-- @node: reverse_baseline_coordinate_integrable
lemma reverse_baseline_coordinate_integrable (B : ℕ) (i : Fin B) :
Integrable (fun U => U i) (blockBaselineLaw B) ∧
Integrable (fun U => (U i) ^ 2) (blockBaselineLaw B) := by
let := blockBaselineLaw_probability B
constructor
· apply (integrable_const (1 / 4 : ℝ)).mono' (measurable_pi_apply i).aestronglyMeasurable
filter_upwards [blockBaselineLaw_ae_support B] with U hU
simpa only [Real.norm_eq_abs] using hU i
· apply (integrable_const (1 / 16 : ℝ)).mono' ((measurable_pi_apply i).pow_const 2).aestronglyMeasurable
filter_upwards [blockBaselineLaw_ae_support B] with U hU
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hi := hU i
nlinarith [sq_abs (U i), abs_nonneg (U i)]
Evenness of the actual joint baseline law centers each coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_baseline_coordinate_mean
lemma reverse_baseline_coordinate_mean (B : ℕ) (i : Fin B) :
(∫ U, U i ∂blockBaselineLaw B) = 0 := by
have ht := integral_map (show AEMeasurable (fun U : Fin B → ℝ => fun ℓ => -U ℓ)
(blockBaselineLaw B) from (measurable_pi_lambda _ (fun ℓ =>
(measurable_pi_apply ℓ).neg)).aemeasurable)
(measurable_pi_apply i).aestronglyMeasurable
rw [blockBaselineLaw_map_neg, integral_neg] at ht
linarith
Distinct independent baseline coordinates have zero mixed moment. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_baseline_cross_moment
lemma reverse_baseline_cross_moment (B : ℕ) (i j : Fin B) (hij : i ≠ j) :
(∫ U, U i * U j ∂blockBaselineLaw B) = 0 := by
let μ := volume.withDensity (fun w => ofReal (cosSqDensity w))
let : IsProbabilityMeasure μ := by
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal cosSqDensity_integrable_normalized.1
(Filter.Eventually.of_forall cosSqDensity_nonneg),
cosSqDensity_integrable_normalized.2]
simp
have hind := (iIndepFun_pi (μ := fun _ : Fin B => μ)
(X := fun _ => (id : ℝ → ℝ)) (fun _ => measurable_id.aemeasurable)).indepFun hij
have he := hind.integral_mul_eq_mul_integral
(measurable_pi_apply i).aestronglyMeasurable
(measurable_pi_apply j).aestronglyMeasurable
change (∫ U, U i * U j ∂blockBaselineLaw B) = _ at he
change (∫ U, U i * U j ∂blockBaselineLaw B) =
(∫ U, U i ∂blockBaselineLaw B) * (∫ U, U j ∂blockBaselineLaw B) at he
rw [he, reverse_baseline_coordinate_mean, zero_mul]
theorem reverse_baseline_coordinate_second_le Lemma reverse_baseline_coordinate_second_le in the paper ↗
The baseline's diagonal second moment is bounded by its support radius. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_baseline_coordinate_second_le
lemma reverse_baseline_coordinate_second_le (B : ℕ) (i : Fin B) :
(∫ U, (U i) ^ 2 ∂blockBaselineLaw B) ≤ 1 / 16 := by
let := blockBaselineLaw_probability B
calc
_ ≤ ∫ _U, (1 / 16 : ℝ) ∂blockBaselineLaw B := by
apply integral_mono_ae (reverse_baseline_coordinate_integrable B i).2
(integrable_const _)
filter_upwards [blockBaselineLaw_ae_support B] with U hU
have hi := hU i
nlinarith [sq_abs (U i), abs_nonneg (U i)]
_ = _ := by simp
A weighted baseline sum and its square are integrable without new premises. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
a :
Fin B → ℝ
conclusion
conclusion 1
Integrable (fun U => ∑ i, a i * U i) (blockBaselineLaw B)
conclusion 2
Integrable (fun U => (∑ i, a i * U i) ^ 2) (blockBaselineLaw B)
Proof (Lean source)
-- @node: reverse_baseline_sum_integrable
lemma reverse_baseline_sum_integrable (B : ℕ) (a : Fin B → ℝ) :
Integrable (fun U => ∑ i, a i * U i) (blockBaselineLaw B) ∧
Integrable (fun U => (∑ i, a i * U i) ^ 2) (blockBaselineLaw B) := by
let := blockBaselineLaw_probability B
have hb : ∀ᵐ U ∂blockBaselineLaw B,
|∑ i, a i * U i| ≤ ∑ i, |a i| / 4 := by
filter_upwards [blockBaselineLaw_ae_support B] with U hU
apply (Finset.abs_sum_le_sum_abs _ _).trans
apply Finset.sum_le_sum
intro i _
rw [abs_mul, div_eq_mul_inv]
simpa using mul_le_mul_of_nonneg_left (hU i) (abs_nonneg (a i))
constructor
· exact integrable_finset_sum _ (fun i _ =>
(reverse_baseline_coordinate_integrable B i).1.const_mul _)
· apply (integrable_const ((∑ i, |a i| / 4) ^ 2)).mono'
((Finset.measurable_sum _ (fun i _ => (show Measurable (fun U : Fin B → ℝ => U i) from measurable_pi_apply i).const_mul (a i))).pow_const 2).aestronglyMeasurable
filter_upwards [hb] with U hU
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
nlinarith [sq_abs (∑ i, a i * U i), abs_nonneg (∑ i, a i * U i)]
Independent centered baselines contribute only their diagonal energies. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
a :
Fin B → ℝ
conclusion
(∫ U, (∑ i, a i * U i) ^ 2 ∂blockBaselineLaw B)
= ∑ i, a i ^ 2 * ∫ U, (U i) ^ 2 ∂blockBaselineLaw B
Proof (Lean source)
-- @node: reverse_baseline_sum_second
lemma reverse_baseline_sum_second (B : ℕ) (a : Fin B → ℝ) :
(∫ U, (∑ i, a i * U i) ^ 2 ∂blockBaselineLaw B) =
∑ i, a i ^ 2 * ∫ U, (U i) ^ 2 ∂blockBaselineLaw B := by
let := blockBaselineLaw_probability B
have hcross (i j : Fin B) : Integrable (fun U => U i * U j) (blockBaselineLaw B) := by
apply (integrable_const (1 / 16 : ℝ)).mono'
((measurable_pi_apply i).mul (measurable_pi_apply j)).aestronglyMeasurable
filter_upwards [blockBaselineLaw_ae_support B] with U hU
simp only [Real.norm_eq_abs, Pi.mul_apply, abs_mul]
nlinarith [hU i, hU j, abs_nonneg (U i), abs_nonneg (U j)]
simp_rw [pow_two, Finset.sum_mul, Finset.mul_sum]
rw [integral_finset_sum _ (fun i _ => integrable_finset_sum _ (fun j _ =>
by
apply ((hcross i j).const_mul (a i * a j)).congr
exact Filter.Eventually.of_forall (fun U => by ring)))]
apply Finset.sum_congr rfl
intro i _
rw [integral_finset_sum _ (fun j _ =>
by
apply ((hcross i j).const_mul (a i * a j)).congr
exact Filter.Eventually.of_forall (fun U => by ring))]
have he (j : Fin B) :
(∫ U, a i * U i * (a j * U j) ∂blockBaselineLaw B) =
if j = i then a i * a i * ∫ U, U i * U i ∂blockBaselineLaw B else 0 := by
have hp (U : Fin B → ℝ) : a i * U i * (a j * U j) =
(a i * a j) * (U i * U j) := by ring
simp_rw [hp, integral_const_mul]
by_cases hj : j = i
· simp [hj]
· simp [hj, reverse_baseline_cross_moment B i j (Ne.symm hj)]
simp_rw [he]
simp
Adding any fixed signal leaves only its squared value and baseline energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ U, ((∑ i, a i * U i) + c) ^ 2 ∂blockBaselineLaw B) ≤ (∑ i, a i ^ 2) / 16 + c ^ 2
Proof (Lean source)
-- @node: reverse_baseline_shifted_second_le
lemma reverse_baseline_shifted_second_le (B : ℕ) (a : Fin B → ℝ) (c : ℝ) :
(∫ U, ((∑ i, a i * U i) + c) ^ 2 ∂blockBaselineLaw B) ≤
(∑ i, a i ^ 2) / 16 + c ^ 2 := by
let := blockBaselineLaw_probability B
have hm : (∫ U, (∑ i, a i * U i) ∂blockBaselineLaw B) = 0 := by
rw [integral_finset_sum _ (fun i _ =>
(reverse_baseline_coordinate_integrable B i).1.const_mul _)]
simp_rw [integral_const_mul, reverse_baseline_coordinate_mean, mul_zero]
simp
have he (U : Fin B → ℝ) : ((∑ i, a i * U i) + c) ^ 2 =
(∑ i, a i * U i) ^ 2 + 2 * c * (∑ i, a i * U i) + c ^ 2 := by ring
simp_rw [he]
have hlin := (reverse_baseline_sum_integrable B a).1
have hsq := (reverse_baseline_sum_integrable B a).2
integral_linearity
rw [hm, mul_zero, add_zero, reverse_baseline_sum_second]
simp only [integral_const, probReal_univ, one_smul]
have henergy : (∑ i, a i ^ 2 * (∫ U, (U i) ^ 2 ∂blockBaselineLaw B)) ≤
(∑ i, a i ^ 2) / 16 := by
rw [Finset.sum_div]
apply Finset.sum_le_sum
intro i _
simpa only [div_eq_mul_inv, one_mul] using
mul_le_mul_of_nonneg_left (reverse_baseline_coordinate_second_le B i) (sq_nonneg (a i))
linarith
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ReverseRowMoments 6 declarations Injective restrictions of the full independent assignments and reveals have the size-d product law.
Actual source-row marginals for the reverse test
Injective restrictions of the full independent assignments and reveals have the size-d product law. This connects the finite sign moments to labeled source rows.
Restricting an independent Boolean vector to distinct labels preserves its law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_pi_injection_law
lemma reverse_pi_injection_law {N d : ℕ} (p : ℝ) (hp : p ∈ Icc 0 1)
(e : Fin d → Fin N) (he : Injective e) :
(Measure.pi (fun _ : Fin N => bernoulliLaw p)).map (fun z k => z (e k)) =
Measure.pi (fun _ : Fin d => bernoulliLaw p) := by
let := bernoulliLaw_probability p hp
have hi := (iIndepFun_pi (μ := fun _ : Fin N => bernoulliLaw p)
(X := fun _ => (id : Bool → Bool)) (fun _ => measurable_id.aemeasurable)).precomp he
have hm := hi.map_fun_eq_pi_map (fun k => (measurable_pi_apply (e k)).aemeasurable)
simpa only [Function.id_def, (measurePreserving_eval
(fun _ : Fin N => bernoulliLaw p) _).map_eq] using hm
A row selected at distinct assignment and reveal labels has the actual row law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_row_injection_law
lemma reverse_row_injection_law {n m d : ℕ} (p : ℝ) (hp : p ∈ Icc 0 1)
(v : Fin d → Fin n) (j : Fin d → Fin m)
(hv : Injective v) (hj : Injective j) :
((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin m => bernoulliLaw p))).map
(fun zr => (fun k => zr.1 (v k), fun k => zr.2 (j k))) =
(halfBernoulli (Fin d)).prod
(Measure.pi (fun _ : Fin d => bernoulliLaw p)) := by
have hm := Measure.map_prod_map (halfBernoulli (Fin n))
(Measure.pi (fun _ : Fin m => bernoulliLaw p))
(measurable_of_finite (fun z k => z (v k)))
(measurable_of_finite (fun r k => r (j k)))
rw [show (halfBernoulli (Fin n)).map (fun z k => z (v k)) =
halfBernoulli (Fin d) from
reverse_pi_injection_law (1 / 2) (by constructor <;> norm_num) v hv,
reverse_pi_injection_law p hp j hj] at hm
exact hm.symm
theorem reverse_row_injection_integral
Integrating a row function against its actual labeled inputs uses the row marginal. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, f (fun k => zr.1 (v k), fun k => zr.2 (j k)) ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin m => bernoulliLaw p))))
Proof (Lean source)
-- @node: reverse_row_injection_integral
lemma reverse_row_injection_integral {n m d : ℕ} (p : ℝ) (hp : p ∈ Icc 0 1)
(v : Fin d → Fin n) (j : Fin d → Fin m)
(hv : Injective v) (hj : Injective j)
(f : Assign (Fin d) × (Fin d → Bool) → ℝ) :
(∫ zr, f (fun k => zr.1 (v k), fun k => zr.2 (j k))
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin m => bernoulliLaw p)))) =
∫ zr, f zr ∂((halfBernoulli (Fin d)).prod
(Measure.pi (fun _ : Fin d => bernoulliLaw p))) := by
rw [← reverse_row_injection_law p hp v j hv hj,
integral_map (measurable_of_finite _).aemeasurable
(measurable_of_finite f).aestronglyMeasurable]
The actual row product law gives the revealed sign energy dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_row_second_moment
lemma reverse_row_second_moment (d : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, (∑ k : Fin d, if zr.2 k then signOf (zr.1 k) else 0) ^ 2
∂((halfBernoulli (Fin d)).prod
(Measure.pi (fun _ : Fin d => bernoulliLaw p)))) = d * p := by
let := halfBernoulli_probability (V := Fin d)
let := bernoulliLaw_probability p hp
rw [integral_prod_symm _ Integrable.of_finite]
simp_rw [← Finset.sum_filter]
exact reverse_revealed_second_moment_average d p hp
The actual row product law gives the signal mean dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_row_product_mean
lemma reverse_row_product_mean (d : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, (∑ k : Fin d, if zr.2 k then signOf (zr.1 k) else 0) *
(∑ k : Fin d, signOf (zr.1 k))
∂((halfBernoulli (Fin d)).prod
(Measure.pi (fun _ : Fin d => bernoulliLaw p)))) = d * p := by
let := halfBernoulli_probability (V := Fin d)
let := bernoulliLaw_probability p hp
rw [integral_prod_symm _ Integrable.of_finite]
simp_rw [← Finset.sum_filter]
exact reverse_revealed_total_product_mean_average d p hp
The actual row product law bounds the signal second moment by 3d²p. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_row_product_second_le
lemma reverse_row_product_second_le (d : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, ((∑ k : Fin d, if zr.2 k then signOf (zr.1 k) else 0) *
(∑ k : Fin d, signOf (zr.1 k))) ^ 2
∂((halfBernoulli (Fin d)).prod
(Measure.pi (fun _ : Fin d => bernoulliLaw p)))) ≤ 3 * (d : ℝ) ^ 2 * p := by
let := halfBernoulli_probability (V := Fin d)
let := bernoulliLaw_probability p hp
rw [integral_prod_symm _ Integrable.of_finite]
simp_rw [mul_pow, ← Finset.sum_filter]
exact reverse_revealed_total_mixed_moment_average_le d p hp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ReverseSourceRows 9 declarations Actual source fibers have the row marginals used in the sign calculations.
Labeled source-row variance in the reverse test
Actual source fibers have the row marginals used in the sign calculations. Their disjoint labels give independent row signals and additive variances.
def reverseRowA
The revealed sign sum at the original labels of a source row.
def reverseRowT
The total sign sum at the original labels of a source row.
theorem reverse_source_row_integral
Reindexing a true source fiber connects its two sums to the actual row law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hfit :
2 * (B * d) ≤ n
s :
SourcePartition B d
ℓ :
Fin B
p :
ℝ
hp :
p ∈ Icc 0 1
f :
ℝ → ℝ → ℝ
conclusion
(∫ zr, f (reverseRowA n B d hfit s ℓ zr) (reverseRowT n B d hfit s ℓ zr) ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
Proof (Lean source)
-- @node: reverse_source_row_integral
lemma reverse_source_row_integral (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) (p : ℝ) (hp : p ∈ Icc 0 1)
(f : ℝ → ℝ → ℝ) :
(∫ zr, f (reverseRowA n B d hfit s ℓ zr) (reverseRowT n B d hfit s ℓ zr)
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) =
∫ zr, f (∑ k : Fin d, if zr.2 k then signOf (zr.1 k) else 0)
(∑ k : Fin d, signOf (zr.1 k))
∂((halfBernoulli (Fin d)).prod
(Measure.pi (fun _ : Fin d => bernoulliLaw p))) := by
let R := Finset.univ.filter (fun j => s.1 j = ℓ)
let e : Fin d ≃ R := Fintype.equivOfCardEq (by
rw [Fintype.card_fin, Fintype.card_coe]
exact (s.2 ℓ).symm)
let j : Fin d → Fin (B * d) := fun k => (e k).val
let v : Fin d → Fin n := fun k => ⟨(j k).val, by have := (j k).isLt; omega⟩
have hj : Injective j := Subtype.val_injective.comp e.injective
have hv : Injective v := by
intro a b hab
apply hj
exact Fin.ext (congrArg (fun x : Fin n => x.val) hab)
have hs (g : Fin (B * d) → ℝ) : (∑ k : Fin d, g (j k)) = ∑ k ∈ R, g k := by
rw [← Finset.sum_coe_sort R g]
exact e.sum_comp (fun k : R => g k.val)
have hA (zr : Assign (Fin n) × (Fin (B * d) → Bool)) :
reverseRowA n B d hfit s ℓ zr =
∑ k : Fin d, if zr.2 (j k) then signOf (zr.1 (v k)) else 0 := by
exact (hs (fun k => if zr.2 k then signOf (zr.1 ⟨k.val, by omega⟩) else 0)).symm
have hT (zr : Assign (Fin n) × (Fin (B * d) → Bool)) :
reverseRowT n B d hfit s ℓ zr = ∑ k : Fin d, signOf (zr.1 (v k)) := by
exact (hs (fun k => signOf (zr.1 ⟨k.val, by omega⟩))).symm
simp_rw [hA, hT]
exact reverse_row_injection_integral p hp v j hv hj
(fun zr => f (∑ k, if zr.2 k then signOf (zr.1 k) else 0)
(∑ k, signOf (zr.1 k)))
theorem reverse_source_row_second
The actual revealed row energy equals dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, (reverseRowA n B d hfit s ℓ zr) ^ 2 ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
= d * p
Proof (Lean source)
-- @node: reverse_source_row_second
lemma reverse_source_row_second (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, (reverseRowA n B d hfit s ℓ zr) ^ 2
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) = d * p := by
rw [reverse_source_row_integral n B d hfit s ℓ p hp (fun a _ => a ^ 2)]
exact reverse_row_second_moment d p hp
theorem reverse_source_row_product_mean
The actual row signal mean equals dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
= d * p
Proof (Lean source)
-- @node: reverse_source_row_product_mean
lemma reverse_source_row_product_mean (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) = d * p := by
rw [reverse_source_row_integral n B d hfit s ℓ p hp (fun a t => a * t)]
exact reverse_row_product_mean d p hp
theorem reverse_source_row_product_second_le
The actual row signal second moment is at most 3d²p. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, (reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr) ^ 2 ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
≤ 3 * (d : ℝ) ^ 2 * p
Proof (Lean source)
-- @node: reverse_source_row_product_second_le
lemma reverse_source_row_product_second_le (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, (reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr) ^ 2
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) ≤ 3 * (d : ℝ) ^ 2 * p := by
rw [reverse_source_row_integral n B d hfit s ℓ p hp (fun a t => (a * t) ^ 2)]
exact reverse_row_product_second_le d p hp
theorem reverse_source_pairs_independent
Each source's sign and reveal form an independent paired coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_source_pairs_independent
lemma reverse_source_pairs_independent (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(p : ℝ) (hp : p ∈ Icc 0 1) :
iIndepFun (fun j : Fin (B * d) =>
fun zr : Assign (Fin n) × (Fin (B * d) → Bool) =>
(zr.1 ⟨j.val, by omega⟩, zr.2 j))
((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))) := by
let := halfBernoulli_probability (V := Fin n)
let := bernoulliLaw_probability (1 / 2) (by constructor <;> norm_num)
let := bernoulliLaw_probability p hp
let v : Fin (B * d) → Fin n := fun j => ⟨j.val, by omega⟩
have hv : Injective v := by
intro a b hab
exact Fin.ext (congrArg (fun x : Fin n => x.val) hab)
let ν := (halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))
let g := fun zr : Assign (Fin n) × (Fin (B * d) → Bool) =>
fun j => (zr.1 (v j), zr.2 j)
let ρ := (bernoulliLaw (1 / 2)).prod (bernoulliLaw p)
have hm : ν.map g = Measure.pi (fun _ : Fin (B * d) => ρ) := by
have hr := reverse_row_injection_law p hp v id hv Function.injective_id
have he := MeasurePreserving.symm
(MeasurableEquiv.arrowProdEquivProdArrow Bool Bool (Fin (B * d)))
(measurePreserving_arrowProdEquivProdArrow Bool Bool (Fin (B * d))
(fun _ => bernoulliLaw (1 / 2)) (fun _ => bernoulliLaw p))
have ht := congrArg (fun μ => μ.map
(MeasurableEquiv.arrowProdEquivProdArrow Bool Bool (Fin (B * d))).symm) hr
rw [Measure.map_map (measurable_of_finite _) (measurable_of_finite _)] at ht
change ν.map g = _ at ht
exact ht.trans he.map_eq
have hc (j : Fin (B * d)) : ν.map (fun zr => g zr j) = ρ := by
change ν.map ((fun x : Fin (B * d) → Bool × Bool => x j) ∘ g) = ρ
rw [← Measure.map_map (measurable_pi_apply j) (measurable_of_finite g), hm]
exact (measurePreserving_eval (fun _ : Fin (B * d) => ρ) j).map_eq
apply (iIndepFun_iff_map_fun_eq_pi_map (fun _ =>
(measurable_of_finite _).aemeasurable)).mpr
change ν.map g = Measure.pi (fun j => ν.map (fun zr => g zr j))
simp only [hc, hm]
theorem reverse_source_row_signals_independent
Disjoint source fibers give independent row signals in the original-input law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hfit :
2 * (B * d) ≤ n
s :
SourcePartition B d
p :
ℝ
hp :
p ∈ Icc 0 1
i j :
Fin B
hij :
i ≠ j
conclusion
IndepFun (fun zr => reverseRowA n B d hfit s i zr * reverseRowT n B d hfit s i zr) (fun zr => reverseRowA n B d hfit s j zr * reverseRowT n B d hfit s j zr) ((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))
Proof (Lean source)
-- @node: reverse_source_row_signals_independent
lemma reverse_source_row_signals_independent (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (p : ℝ) (hp : p ∈ Icc 0 1)
(i j : Fin B) (hij : i ≠ j) :
IndepFun
(fun zr => reverseRowA n B d hfit s i zr * reverseRowT n B d hfit s i zr)
(fun zr => reverseRowA n B d hfit s j zr * reverseRowT n B d hfit s j zr)
((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))) := by
let R (ℓ : Fin B) := Finset.univ.filter (fun k => s.1 k = ℓ)
have hd : Disjoint (R i) (R j) := by
apply Finset.disjoint_left.mpr
intro k hki hkj
exact hij ((Finset.mem_filter.mp hki).2.symm.trans (Finset.mem_filter.mp hkj).2)
have hi := (reverse_source_pairs_independent n B d hfit p hp).indepFun_finset
(R i) (R j) hd (fun _ => measurable_of_finite _)
let f (ℓ : Fin B) (x : R ℓ → Bool × Bool) : ℝ :=
(∑ k : R ℓ, if (x k).2 then signOf (x k).1 else 0) *
(∑ k : R ℓ, signOf (x k).1)
have hf (ℓ : Fin B) (zr : Assign (Fin n) × (Fin (B * d) → Bool)) :
f ℓ (fun k => (zr.1 ⟨k.val.val, by have := k.val.isLt; omega⟩, zr.2 k.val)) =
reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr := by
dsimp only [f]
rw [Finset.sum_coe_sort (R ℓ) (fun k =>
if zr.2 k then signOf (zr.1 ⟨k.val, by omega⟩) else 0),
Finset.sum_coe_sort (R ℓ) (fun k => signOf (zr.1 ⟨k.val, by omega⟩))]
rfl
have ht := hi.comp (measurable_of_finite (f i)) (measurable_of_finite (f j))
simpa only [Function.comp_def, hf] using ht
theorem reverse_source_signal_variance_le
Independent row variances sum to at most B times the row second-moment bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, ((∑ ℓ, reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr) - (B : ℝ) * d * p) ^ 2 ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
≤ (B : ℝ) * (3 * (d : ℝ) ^ 2 * p)
Proof (Lean source)
-- @node: reverse_source_signal_variance_le
lemma reverse_source_signal_variance_le (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, ((∑ ℓ, reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr) -
(B : ℝ) * d * p) ^ 2
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) ≤
(B : ℝ) * (3 * (d : ℝ) ^ 2 * p) := by
let := halfBernoulli_probability (V := Fin n)
let := bernoulliLaw_probability p hp
let ν := (halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))
let X := fun ℓ zr => reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr
have hX (ℓ : Fin B) : MemLp (X ℓ) 2 ν :=
⟨(measurable_of_finite _).aestronglyMeasurable, eLpNorm_lt_top_of_finite⟩
have hm : (∫ zr, ∑ ℓ, X ℓ zr ∂ν) = (B : ℝ) * d * p := by
rw [integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
simp only [X, ν, reverse_source_row_product_mean n B d hfit s _ p hp]
simp
ring
have hv : variance (fun zr => ∑ ℓ, X ℓ zr) ν = ∑ ℓ, variance (X ℓ) ν := by
have hv := IndepFun.variance_sum (s := univ) (fun ℓ _ => hX ℓ)
(fun i _ j _ hij => reverse_source_row_signals_independent n B d hfit s p hp i j hij)
have hsum : (∑ ℓ, X ℓ) = (fun zr => ∑ ℓ, X ℓ zr) := by
funext zr
simp
rw [hsum] at hv
exact hv
have he : (∫ zr, ((∑ ℓ, X ℓ zr) - (B : ℝ) * d * p) ^ 2 ∂ν) =
variance (fun zr => ∑ ℓ, X ℓ zr) ν := by
rw [variance_eq_integral (measurable_of_finite _).aemeasurable, hm]
change (∫ zr, ((∑ ℓ, X ℓ zr) - (B : ℝ) * d * p) ^ 2 ∂ν) ≤ _
rw [he]
rw [hv]
calc
_ ≤ ∑ _ℓ : Fin B, 3 * (d : ℝ) ^ 2 * p := by
apply Finset.sum_le_sum
intro ℓ _
exact (variance_le_expectation_sq (measurable_of_finite _).aestronglyMeasurable).trans
(reverse_source_row_product_second_le n B d hfit s ℓ p hp)
_ = _ := by simp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ReverseTest 32 declarations The test uses the entire recorded graph and assignment to compute revealed sign sums, and reads the distinct responses from the copied outcomes.
Observable reverse-test statistic and its actual partition channel
The test uses the entire recorded graph and assignment to compute revealed sign sums, and reads the distinct responses from the copied outcomes. Its latent representation keeps the original source labels and independent reveal vector.
The sign test's real-valued statistic, computed from the original record.
Definition (Lean source)
parameters
def
reverseTestStatistic n B d o :
ℝ
given by
∑ ℓ, revealedSignSum n B d o.1 o.2.1 ℓ * distinctResponses n B d o ℓ
theorem reverseTestStatistic_measurable
The original-record statistic is Borel measurable.
Formal statement
Proof (Lean source)
-- @node: reverseTestStatistic_measurable
@[fun_prop]
lemma reverseTestStatistic_measurable (n B d : ℕ) :
Measurable (reverseTestStatistic n B d) := by
unfold reverseTestStatistic
apply Finset.measurable_sum
intro ℓ _
apply mul
· exact (measurable_of_finite (fun hz :
(OffDiag (Fin n) → Bool) × Assign (Fin n) =>
revealedSignSum n B d hz.1 hz.2 ℓ)).comp
(measurable_fst.prodMk measurable_snd.fst)
· exact (measurable_pi_apply ℓ).comp (distinctResponses_measurable n B d)
theorem reverse_record_graph
Retained graphs of the block channel do not depend on baselines or the prior sign. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(recordOf (blockSchedule n B d σ h (s, U)) ω).1
Proof (Lean source)
-- @node: reverse_record_graph
lemma reverse_record_graph (n B d : ℕ) (σ : Bool) (h : ℝ)
(s : SourcePartition B d) (U : Fin B → ℝ)
(ω : Assign (Fin n) × Audit (Fin n)) :
(recordOf (blockSchedule n B d σ h (s, U)) ω).1 =
(recordOf (blockSchedule n B d true 0 (s, fun _ => 0)) ω).1 := by
rfl
theorem reverse_revealedSources_filter
For a compatible detailed graph, a row's revealed set is its source set filtered by whether an arrow from the original label was retained anywhere. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
revealedSources n B d H ℓ
= (partitionSourceLabels n B d s ℓ).filter (fun j => ∃ i : Fin n, ∃ hji : j ≠ i, H ⟨(j,i),hji⟩ = true)
Proof (Lean source)
-- @node: reverse_revealedSources_filter
lemma reverse_revealedSources_filter (n B d : ℕ) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2) (ℓ : Fin B) :
revealedSources n B d H ℓ = (partitionSourceLabels n B d s ℓ).filter
(fun j => ∃ i : Fin n, ∃ hji : j ≠ i, H ⟨(j,i),hji⟩ = true) := by
ext j
simp only [mem_filter]
constructor
· intro hj
refine ⟨revealedSources_subset_partitionSourceLabels n B d s H hs ℓ hj, ?_⟩
obtain ⟨_, i, _, hji, he⟩ := (Finset.mem_filter.mp hj).2
exact ⟨i, hji, he⟩
· rintro ⟨hj, i, hji, he⟩
obtain ⟨hjlt, hrow⟩ := (Finset.mem_filter.mp hj).2
obtain ⟨⟨hjlt', k, hsk, hik⟩, _⟩ := hs ⟨(j,i),hji⟩ he
have hk : k = ℓ := hsk.symm.trans hrow
exact Finset.mem_filter.mpr ⟨Finset.mem_univ _, hjlt, i,
by simpa only [hk] using hik, hji, he⟩
theorem reverse_distinctResponses_source_sum
Each recorded response has the true row's total sign shift. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
s :
SourcePartition B d
U :
Fin B → ℝ
ℓ :
Fin B
conclusion
= U ℓ + signOf σ * h / (2 * d) * ∑ j ∈ partitionSourceLabels n B d s ℓ, signOf (ω.1 j)
Proof (Lean source)
-- @node: reverse_distinctResponses_source_sum
lemma reverse_distinctResponses_source_sum (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(U : Fin B → ℝ) (ω : Assign (Fin n) × Audit (Fin n)) (ℓ : Fin B) :
distinctResponses n B d (recordOf (blockSchedule n B d σ h (s,U)) ω) ℓ =
U ℓ + signOf σ * h / (2 * d) *
∑ j ∈ partitionSourceLabels n B d s ℓ, signOf (ω.1 j) := by
obtain ⟨i, hi⟩ := Finset.card_pos.mp (show 0 < (recipientBlock n B d ℓ).card by
rw [recipientBlock_card n B d hfit]; omega)
rw [distinctResponses_record_translation n B d hd hfit σ h s ω U]
dsimp only
rw [partitionResponseShift_eq_source_sum n B d hd hfit σ h s ω ℓ i hi,
← partitionSourceLabels_eq_inNbhd n B d s ℓ i hi]
theorem reverse_sourceLabels_image
The true row's original labels are the image of its finite source fiber. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
partitionSourceLabels n B d s ℓ
Proof (Lean source)
-- @node: reverse_sourceLabels_image
lemma reverse_sourceLabels_image (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) :
partitionSourceLabels n B d s ℓ =
(Finset.univ.filter (fun j => s.1 j = ℓ)).image
(fun j : Fin (B * d) => (⟨j.val, by omega⟩ : Fin n)) := by
ext j
simp only [partitionSourceLabels, mem_filter, Finset.mem_univ,
true_and, mem_image]
constructor
· rintro ⟨hj, hs⟩
exact ⟨⟨j.val,hj⟩, hs, Fin.ext rfl⟩
· rintro ⟨k, hk, rfl⟩
exact ⟨k.isLt,hk⟩
theorem reverse_sourceLabels_sum
Reindexing a row sum preserves each original source label. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ j ∈ partitionSourceLabels n B d s ℓ, f j)
= ∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), f ⟨j.val, by omega⟩
Proof (Lean source)
-- @node: reverse_sourceLabels_sum
lemma reverse_sourceLabels_sum (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (ℓ : Fin B) (f : Fin n → ℝ) :
(∑ j ∈ partitionSourceLabels n B d s ℓ, f j) =
∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
f ⟨j.val, by omega⟩ := by
rw [reverse_sourceLabels_image n B d hfit, Finset.sum_image]
intro a _ b _ hab
exact Fin.ext (congrArg (fun j : Fin n => j.val) hab)
theorem reverse_revealedSignSum_source_fiber
A compatible row's revealed sum is the source-fiber sum with its Bernoulli reveal indicators, keeping assignments at their original population labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
revealedSignSum n B d H z ℓ
= ∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), if sourceReveals n B d H j then signOf (z ⟨j.val,by omega⟩) else 0
Proof (Lean source)
-- @node: reverse_revealedSignSum_source_fiber
lemma reverse_revealedSignSum_source_fiber (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (s : SourcePartition B d)
(H : OffDiag (Fin n) → Bool)
(hs : ∀ e, H e = true → blockEdge n B d s e.val.1 e.val.2)
(z : Assign (Fin n)) (ℓ : Fin B) :
revealedSignSum n B d H z ℓ =
∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
if sourceReveals n B d H j then signOf (z ⟨j.val,by omega⟩) else 0 := by
rw [revealedSignSum, reverse_revealedSources_filter n B d s H hs,
Finset.sum_filter, reverse_sourceLabels_sum n B d hfit]
apply Finset.sum_congr rfl
intro j _
have he : (∃ i : Fin n, ∃ hji : (⟨j.val,by omega⟩ : Fin n) ≠ i,
H ⟨((⟨j.val,by omega⟩ : Fin n),i),hji⟩ = true) ↔
sourceReveals n B d H j = true := by
simp only [sourceReveals, decide_eq_true_eq]
constructor
· rintro ⟨i,hji,hi⟩
exact ⟨⟨j.val,by omega⟩,rfl,i,hji,hi⟩
· rintro ⟨v,hv,i,hvi,hi⟩
have hv' : v = (⟨j.val,by omega⟩ : Fin n) := Fin.ext hv
exact ⟨i, hv' ▸ hvi, by simpa only [hv'] using hi⟩
simp only [he]
def reverseLatentStatistic
The reverse statistic as a function of baselines, assignments and source reveals, conditional on the true source partition.
Definition (Lean source)
parameters
def
reverseLatentStatistic n B d hfit σ h s U zr :
ℝ
given by
∑ ℓ, (∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), if zr.2 j then signOf (zr.1 ⟨j.val,by omega⟩) else 0) * (U ℓ + signOf σ * h / (2 * d) * ∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), signOf (zr.1 ⟨j.val,by omega⟩))
theorem reverseTestStatistic_eq_latent
The original-record test is exactly the independent-input statistic; the partition is used for the proof but is never supplied to the test. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= reverseLatentStatistic n B d hfit σ h s U (ω.1, sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s,fun _ => 0)) ω).1)
Proof (Lean source)
-- @node: reverseTestStatistic_eq_latent
lemma reverseTestStatistic_eq_latent (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) (s : SourcePartition B d)
(U : Fin B → ℝ) (ω : Assign (Fin n) × Audit (Fin n)) :
reverseTestStatistic n B d (recordOf (blockSchedule n B d σ h (s,U)) ω) =
reverseLatentStatistic n B d hfit σ h s U (ω.1, sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s,fun _ => 0)) ω).1) := by
unfold reverseTestStatistic reverseLatentStatistic
apply Finset.sum_congr rfl
intro ℓ _
rw [reverse_distinctResponses_source_sum n B d hd hfit,
reverse_sourceLabels_sum n B d hfit]
congr 1
change revealedSignSum n B d _ ω.1 ℓ = _
rw [reverse_record_graph, reverse_revealedSignSum_source_fiber n B d hfit]
intro e he
have he' : blockEdge n B d s e.val.1 e.val.2 ∧ ω.2 e = true := by
simpa only [recordOf, blockSchedule, Bool.and_eq_true, decide_eq_true_eq] using he
exact he'.1
theorem reverse_assignment_reveals_law
Actual assignments and source reveals have a product law conditional on every true partition; the complete assignment vector remains available. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(thinnedDesign (Fin n) q).map (fun ω => (ω.1, sourceReveals n B d (recordOf (blockSchedule n B d true 0 (s,fun _ => 0)) ω).1))
Proof (Lean source)
-- @node: reverse_assignment_reveals_law
lemma reverse_assignment_reveals_law (n B d : ℕ) (q : ℝ)
(hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1) (s : SourcePartition B d) :
(thinnedDesign (Fin n) q).map (fun ω => (ω.1, sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s,fun _ => 0)) ω).1)) =
(halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))) := by
let := bernoulliLaw_probability q hq
let := bernoulliLaw_probability (1 / 2) (by constructor <;> norm_num)
let g : Audit (Fin n) → (Fin (B * d) → Bool) := fun w => sourceReveals n B d
(recordOf (blockSchedule n B d true 0 (s,fun _ => 0)) (fun _ => false,w)).1
have hg : Measurable g := measurable_of_finite _
have hgm : (auditLaw (Fin n) q).map g =
Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)) := by
have hm := block_reveal_fixed_partition n B d q hfit hq s
have hsnd := (thinnedDesign_audit (V := Fin n) q hq).2.2
have ht := congrArg (fun μ : Measure (Audit (Fin n)) => μ.map g) hsnd
rw [Measure.map_map hg measurable_snd] at ht
exact ht.symm.trans hm
have ht := Measure.map_prod_map (halfBernoulli (Fin n)) (auditLaw (Fin n) q)
measurable_id hg
rw [Measure.map_id, hgm] at ht
exact ht.symm
theorem reverseTestStatistic_fixed_partition_law
For each fixed partition and baseline draw, the actual statistic's law is obtained from independent assignments and independent source reveals. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
σ :
h q :
ℝ
hq :
q ∈ Icc 0 1
s :
SourcePartition B d
U :
Fin B → ℝ
conclusion
(thinnedDesign (Fin n) q).map (fun ω => reverseTestStatistic n B d (recordOf (blockSchedule n B d σ h (s,U)) ω))
= ((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))).map (reverseLatentStatistic n B d hfit σ h s U)
Proof (Lean source)
-- @node: reverseTestStatistic_fixed_partition_law
lemma reverseTestStatistic_fixed_partition_law (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q : ℝ) (hq : q ∈ Icc 0 1)
(s : SourcePartition B d) (U : Fin B → ℝ) :
(thinnedDesign (Fin n) q).map (fun ω =>
reverseTestStatistic n B d (recordOf (blockSchedule n B d σ h (s,U)) ω)) =
((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))).map
(reverseLatentStatistic n B d hfit σ h s U) := by
rw [← reverse_assignment_reveals_law n B d q hfit hq s,
Measure.map_map (measurable_of_finite _) (measurable_of_finite _)]
congr 1
funext ω
exact reverseTestStatistic_eq_latent n B d hd hfit σ h s U ω
Mixing the independent-input reverse statistic over the genuine prior parameters.
Definition (Lean source)
parameters
def
given by
(blockParamLaw B d).bind (fun ξ => ((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))).map (reverseLatentStatistic n B d hfit σ h ξ.1 ξ.2))
theorem reverseTestStatistic_mixture_law
The actual full-record statistic has exactly the independent-input mixture law. Every conditional law retains the true source-label fiber and all assignment coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
= reverseLatentLaw n B d hfit σ h q
Proof (Lean source)
-- @node: reverseTestStatistic_mixture_law
lemma reverseTestStatistic_mixture_law (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q : ℝ) (hq : q ∈ Icc 0 1) :
(blockMixtureLawOf n B d (thinnedDesign (Fin n) q) σ h).map
(reverseTestStatistic n B d) = reverseLatentLaw n B d hfit σ h q := by
rw [blockMixtureLawOf, mixtureLaw, block_map_bind _ _ _
(block_measurable_map_parameter _ _ (block_record_measurable n B d σ h))
(reverseTestStatistic_measurable n B d)]
unfold reverseLatentLaw
congr 1
funext ξ
rw [Measure.map_map (reverseTestStatistic_measurable n B d)
(measurable_of_finite _)]
exact reverseTestStatistic_fixed_partition_law n B d hd hfit σ h q hq ξ.1 ξ.2
Chebyshev bounds the plus-sign error, including the strict tie convention. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
μ.real (Iio 0) ≤ v / a ^ 2
Proof (Lean source)
-- @node: reverse_plus_sign_error_le
lemma reverse_plus_sign_error_le (μ : Measure ℝ) [IsProbabilityMeasure μ]
(a v : ℝ) (ha : 0 < a)
(hi : Integrable (fun x => (x - a) ^ 2) μ)
(hv : (∫ x, (x - a) ^ 2 ∂μ) ≤ v) :
μ.real (Iio 0) ≤ v / a ^ 2 := by
have hs : Iio (0 : ℝ) ⊆ {x : ℝ | a ^ 2 ≤ (x - a) ^ 2} := by
intro x hx
have hx' : x < 0 := hx
dsimp
nlinarith [sq_nonneg x]
have hm := mul_meas_ge_le_integral_of_nonneg
(Filter.Eventually.of_forall (fun x : ℝ => sq_nonneg (x - a))) hi (a ^ 2)
have he := measureReal_mono (μ := μ) hs (measure_ne_top μ _)
apply (le_div_iff₀ (sq_pos_of_pos ha)).mpr
calc
μ.real (Iio 0) * a ^ 2 ≤
μ.real {x : ℝ | a ^ 2 ≤ (x - a) ^ 2} * a ^ 2 :=
mul_le_mul_of_nonneg_right he (sq_nonneg a)
_ ≤ v := by linarith
Chebyshev bounds the minus-sign error, with zero assigned to plus. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
μ.real (Ici 0) ≤ v / a ^ 2
Proof (Lean source)
-- @node: reverse_minus_sign_error_le
lemma reverse_minus_sign_error_le (μ : Measure ℝ) [IsProbabilityMeasure μ]
(a v : ℝ) (ha : 0 < a)
(hi : Integrable (fun x => (x + a) ^ 2) μ)
(hv : (∫ x, (x + a) ^ 2 ∂μ) ≤ v) :
μ.real (Ici 0) ≤ v / a ^ 2 := by
have hs : Ici (0 : ℝ) ⊆ {x : ℝ | a ^ 2 ≤ (x + a) ^ 2} := by
intro x hx
have hx' : 0 ≤ x := hx
dsimp
nlinarith [sq_nonneg x]
have hm := mul_meas_ge_le_integral_of_nonneg
(Filter.Eventually.of_forall (fun x : ℝ => sq_nonneg (x + a))) hi (a ^ 2)
have he := measureReal_mono (μ := μ) hs (measure_ne_top μ _)
apply (le_div_iff₀ (sq_pos_of_pos ha)).mpr
calc
μ.real (Ici 0) * a ^ 2 ≤
μ.real {x : ℝ | a ^ 2 ≤ (x + a) ^ 2} * a ^ 2 :=
mul_le_mul_of_nonneg_right he (sq_nonneg a)
_ ≤ v := by linarith
theorem reverse_scalar_sign_test_tv_le
The observable sign test separates scalar laws with opposite centers and controlled squared deviations; no assertion about those moments is assumed in the paper theorem. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
μ ν :
a v :
ℝ
ha :
0 < a
hiμ :
Integrable (fun x => (x - a) ^ 2) μ
hiν :
Integrable (fun x => (x + a) ^ 2) ν
hvμ :
(∫ x, (x - a) ^ 2 ∂μ) ≤ v
hvν :
(∫ x, (x + a) ^ 2 ∂ν) ≤ v
conclusion
1 - 2 * v / a ^ 2 ≤ tvDist μ ν
Proof (Lean source)
-- @node: reverse_scalar_sign_test_tv_le
lemma reverse_scalar_sign_test_tv_le (μ ν : Measure ℝ)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν]
(a v : ℝ) (ha : 0 < a)
(hiμ : Integrable (fun x => (x - a) ^ 2) μ)
(hiν : Integrable (fun x => (x + a) ^ 2) ν)
(hvμ : (∫ x, (x - a) ^ 2 ∂μ) ≤ v)
(hvν : (∫ x, (x + a) ^ 2 ∂ν) ≤ v) :
1 - 2 * v / a ^ 2 ≤ tvDist μ ν := by
have htest := one_sub_tvDist_le_test
(μ := μ) (ν := ν) (A := Iio (0 : ℝ)) measurableSet_Iio
rw [Set.compl_Iio] at htest
have hplus := reverse_plus_sign_error_le μ a v ha hiμ hvμ
have hminus := reverse_minus_sign_error_le ν a v ha hiν hvν
rw [mul_div_assoc]
linarith
theorem reverseLatentLaw_probability
The scalar law is the pushforward of the actual probability experiment. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (reverseLatentLaw n B d hfit σ h q)
Proof (Lean source)
-- @node: reverseLatentLaw_probability
lemma reverseLatentLaw_probability (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q : ℝ) (hq : q ∈ Icc 0 1) :
IsProbabilityMeasure (reverseLatentLaw n B d hfit σ h q) := by
let : IsProbabilityMeasure (thinnedDesign (Fin n) q) :=
thinnedDesign_probabilityDesign (V := Fin n) q hq
let := blockMixtureLawOf_probability n B d (thinnedDesign (Fin n) q) σ h
rw [← reverseTestStatistic_mixture_law n B d hd hfit σ h q hq]
exact Measure.isProbabilityMeasure_map (reverseTestStatistic_measurable n B d).aemeasurable
A bounded sign sum over the true source fiber is bounded by its row size. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
|∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), f j| ≤ d
Proof (Lean source)
-- @node: reverse_source_fiber_sum_abs_le
lemma reverse_source_fiber_sum_abs_le (B d : ℕ) (s : SourcePartition B d)
(ℓ : Fin B) (f : Fin (B * d) → ℝ) (hf : ∀ j, |f j| ≤ 1) :
|∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), f j| ≤ d := by
calc
_ ≤ ∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), |f j| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), (1 : ℝ) :=
Finset.sum_le_sum (fun j _ => hf j)
_ = d := by simp [s.2 ℓ]
theorem reverseLatentStatistic_abs_le
Every fixed-input scalar statistic is bounded on the genuine baseline support. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hfit :
2 * (B * d) ≤ n
σ :
h :
ℝ
s :
SourcePartition B d
U :
Fin B → ℝ
hU :
∀ ℓ, |U ℓ| ≤ (1 / 4 : ℝ)
conclusion
|reverseLatentStatistic n B d hfit σ h s U zr|
≤ B * ((d : ℝ) * (1 / 4 + |signOf σ * h / (2 * d)| * d))
Proof (Lean source)
-- @node: reverseLatentStatistic_abs_le
lemma reverseLatentStatistic_abs_le (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h : ℝ) (s : SourcePartition B d) (U : Fin B → ℝ)
(zr : Assign (Fin n) × (Fin (B * d) → Bool))
(hU : ∀ ℓ, |U ℓ| ≤ (1 / 4 : ℝ)) :
|reverseLatentStatistic n B d hfit σ h s U zr| ≤
B * ((d : ℝ) * (1 / 4 + |signOf σ * h / (2 * d)| * d)) := by
have hsign (b : Bool) : |signOf b| ≤ 1 := by cases b <;> norm_num [signOf]
have hA (ℓ : Fin B) := reverse_source_fiber_sum_abs_le B d s ℓ
(fun j => if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0)
(fun j => by split_ifs <;> simp_all only [abs_zero, zero_le_one])
have hT (ℓ : Fin B) := reverse_source_fiber_sum_abs_le B d s ℓ
(fun j => signOf (zr.1 ⟨j.val, by omega⟩)) (fun j => hsign _)
unfold reverseLatentStatistic
calc
_ ≤ ∑ ℓ, |(∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0) *
(U ℓ + signOf σ * h / (2 * d) *
∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
signOf (zr.1 ⟨j.val, by omega⟩))| := Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _ℓ : Fin B, (d : ℝ) * (1 / 4 + |signOf σ * h / (2 * d)| * d) := by
apply Finset.sum_le_sum
intro ℓ _
rw [abs_mul]
apply mul_le_mul (hA ℓ) _ (abs_nonneg _) (Nat.cast_nonneg d)
calc
_ ≤ |U ℓ| + |signOf σ * h / (2 * d) *
∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
signOf (zr.1 ⟨j.val, by omega⟩)| := abs_add_le _ _
_ ≤ 1 / 4 + |signOf σ * h / (2 * d)| * d := by
rw [abs_mul]
exact add_le_add (hU ℓ) (mul_le_mul_of_nonneg_left (hT ℓ) (abs_nonneg _))
_ = _ := by simp
theorem reverseLatentLaw_ae_abs_le
The bound on every fixed-input statistic survives the actual parameter mixture. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverseLatentLaw_ae_abs_le
lemma reverseLatentLaw_ae_abs_le (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h q : ℝ) (_hq : q ∈ Icc 0 1) :
∀ᵐ x ∂reverseLatentLaw n B d hfit σ h q,
|x| ≤ B * ((d : ℝ) * (1 / 4 + |signOf σ * h / (2 * d)| * d)) := by
let C : ℝ := B * ((d : ℝ) * (1 / 4 + |signOf σ * h / (2 * d)| * d))
let := blockBaselineLaw_probability B
have hparam : ∀ᵐ ξ ∂blockParamLaw B d, ∀ ℓ, |ξ.2 ℓ| ≤ (1 / 4 : ℝ) := by
unfold blockParamLaw
apply (Measure.ae_prod_iff_ae_ae (by
rw [Set.ofPred_forall]
apply MeasurableSet.iInter
intro ℓ
exact measurableSet_le (by fun_prop) measurable_const)).mpr
exact Filter.Eventually.of_forall (fun _ => blockBaselineLaw_ae_support B)
have hfibre : ∀ᵐ ξ ∂blockParamLaw B d,
∀ᵐ x ∂(((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))).map
(reverseLatentStatistic n B d hfit σ h ξ.1 ξ.2)), |x| ≤ C := by
filter_upwards [hparam] with ξ hξ
apply (ae_map_iff (measurable_of_finite _).aemeasurable
(measurableSet_le (by fun_prop) measurable_const)).mpr
exact Filter.Eventually.of_forall (fun zr =>
reverseLatentStatistic_abs_le n B d hfit σ h ξ.1 ξ.2 zr hξ)
have hz := hfibre.mono (fun ξ hξ => ae_iff.mp hξ)
change ∀ᵐ x ∂(blockParamLaw B d).bind _, |x| ≤ C
rw [ae_iff]
apply le_antisymm _ bot_le
calc
_ ≤ ∫⁻ ξ, (((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))).map
(reverseLatentStatistic n B d hfit σ h ξ.1 ξ.2)) {x | ¬ |x| ≤ C}
∂blockParamLaw B d := Measure.bind_apply_le _
(measurableSet_le (by fun_prop : Measurable (fun x : ℝ => |x|))
(measurable_const : Measurable (fun _ : ℝ => C))).compl
_ = 0 := by rw [lintegral_congr_ae hz, lintegral_zero]
theorem reverseLatentLaw_squared_deviation_integrable
Every squared deviation under the actual scalar law is integrable by bounded support, without a new regularity hypothesis. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable (fun x => (x - a) ^ 2) (reverseLatentLaw n B d hfit σ h q)
Proof (Lean source)
-- @node: reverseLatentLaw_squared_deviation_integrable
lemma reverseLatentLaw_squared_deviation_integrable (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q a : ℝ) (hq : q ∈ Icc 0 1) :
Integrable (fun x => (x - a) ^ 2) (reverseLatentLaw n B d hfit σ h q) := by
let := reverseLatentLaw_probability n B d hd hfit σ h q hq
let C : ℝ := B * ((d : ℝ) * (1 / 4 + |signOf σ * h / (2 * d)| * d))
apply (integrable_const ((C + |a|) ^ 2)).mono' (by fun_prop)
filter_upwards [reverseLatentLaw_ae_abs_le n B d hfit σ h q hq] with x hx
have hb : |x - a| ≤ C + |a| := (abs_sub x a).trans (add_le_add hx (le_refl |a|))
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
nlinarith [sq_abs (x - a), abs_nonneg (x - a), abs_nonneg a]
def reverseSignalEnergy
The baseline-free squared signal plus the diagonal baseline budget, retaining all original assignment coordinates and source-label fibers.
Definition (Lean source)
parameters
def
reverseSignalEnergy n B d hfit σ h a s zr :
ℝ
given by
(∑ ℓ, (∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ), if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0) ^ 2) / 16
+ (reverseLatentStatistic n B d hfit σ h s (fun _ => 0) zr - a) ^ 2
theorem reverseLatentStatistic_baseline_energy_le
Integrating the actual statistic's baselines leaves the finite signal energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ U, (reverseLatentStatistic n B d hfit σ h s U zr - a) ^ 2 ∂blockBaselineLaw B)
≤ reverseSignalEnergy n B d hfit σ h a s zr
Proof (Lean source)
-- @node: reverseLatentStatistic_baseline_energy_le
lemma reverseLatentStatistic_baseline_energy_le (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h a : ℝ) (s : SourcePartition B d)
(zr : Assign (Fin n) × (Fin (B * d) → Bool)) :
(∫ U, (reverseLatentStatistic n B d hfit σ h s U zr - a) ^ 2
∂blockBaselineLaw B) ≤ reverseSignalEnergy n B d hfit σ h a s zr := by
have he (U : Fin B → ℝ) : reverseLatentStatistic n B d hfit σ h s U zr - a =
(∑ ℓ, (∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0) * U ℓ) +
(reverseLatentStatistic n B d hfit σ h s (fun _ => 0) zr - a) := by
simp only [reverseLatentStatistic, mul_add, Finset.sum_add_distrib, zero_add]
ring
calc
_ = ∫ U, ((∑ ℓ, (∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0) * U ℓ) +
(reverseLatentStatistic n B d hfit σ h s (fun _ => 0) zr - a)) ^ 2
∂blockBaselineLaw B := integral_congr_ae
(Filter.Eventually.of_forall (fun U => congrArg (fun x : ℝ => x ^ 2) (he U)))
_ ≤ _ := by
simpa only [reverseSignalEnergy] using reverse_baseline_shifted_second_le B
(fun ℓ => ∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0)
(reverseLatentStatistic n B d hfit σ h s (fun _ => 0) zr - a)
theorem reverseLatentStatistic_joint_measurable
Under the stated population-size condition, the actual independent-input statistic is jointly Borel measurable.
Formal statement
hypotheses
conclusion
Measurable (fun x : (SourcePartition B d × (Fin B → ℝ)) × (Assign (Fin n) × (Fin (B * d) → Bool)) => reverseLatentStatistic n B d hfit σ h x.1.1 x.1.2 x.2)
Proof (Lean source)
-- @node: reverseLatentStatistic_joint_measurable
@[fun_prop]
lemma reverseLatentStatistic_joint_measurable (n B d : ℕ)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h : ℝ) :
Measurable (fun x : (SourcePartition B d × (Fin B → ℝ)) ×
(Assign (Fin n) × (Fin (B * d) → Bool)) =>
reverseLatentStatistic n B d hfit σ h x.1.1 x.1.2 x.2) := by
apply measurable_from_prod_countable_left
intro zr
apply measurable_from_prod_countable_right
intro s
dsimp only [reverseLatentStatistic]
apply Finset.measurable_sum
intro ℓ _
apply mul
· exact (measurable_const : Measurable (fun _ : Fin B → ℝ =>
∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
if zr.2 j then signOf (zr.1 ⟨j.val, by omega⟩) else 0))
· apply add
· exact measurable_pi_apply ℓ
· exact (measurable_const : Measurable (fun _ : Fin B → ℝ =>
signOf σ * h / (2 * d) * ∑ j ∈ Finset.univ.filter (fun j => s.1 j = ℓ),
signOf (zr.1 ⟨j.val, by omega⟩)))
theorem reverseLatentLaw_joint_map
The scalar mixture is the pushforward of its actual independent joint inputs. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
reverseLatentLaw n B d hfit σ h q
= ((blockParamLaw B d).prod ((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))))).map (fun x => reverseLatentStatistic n B d hfit σ h x.1.1 x.1.2 x.2)
Proof (Lean source)
-- @node: reverseLatentLaw_joint_map
lemma reverseLatentLaw_joint_map (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(σ : Bool) (h q : ℝ) :
reverseLatentLaw n B d hfit σ h q =
((blockParamLaw B d).prod ((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))))).map
(fun x => reverseLatentStatistic n B d hfit σ h x.1.1 x.1.2 x.2) := by
have hf := reverseLatentStatistic_joint_measurable n B d hfit σ h
rw [Measure.prod, block_map_bind _ _ _ Measurable.map_prodMk_left hf]
unfold reverseLatentLaw
congr 1
funext ξ
rw [Measure.map_map hf measurable_prodMk_left]
rfl
theorem reverseLatentLaw_baseline_energy_le
The remaining scalar moment is bounded by a finite partition, assignment and reveal average; the continuous baseline variables have been integrated out. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ x, (x - a) ^ 2 ∂reverseLatentLaw n B d hfit σ h q)
≤ ∫ zr, ∫ s, reverseSignalEnergy n B d hfit σ h a s zr ∂partitionLaw B d ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))))
Proof (Lean source)
-- @node: reverseLatentLaw_baseline_energy_le
lemma reverseLatentLaw_baseline_energy_le (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q a : ℝ) (hq : q ∈ Icc 0 1) :
(∫ x, (x - a) ^ 2 ∂reverseLatentLaw n B d hfit σ h q) ≤
∫ zr, ∫ s, reverseSignalEnergy n B d hfit σ h a s zr ∂partitionLaw B d
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))) := by
let := blockBaselineLaw_probability B
let := partitionLaw_probability B d
let : IsFiniteMeasure (blockParamLaw B d) := by
unfold blockParamLaw
infer_instance
let := halfBernoulli_probability (V := Fin n)
let := bernoulliLaw_probability (retentionP d q) (retentionP_mem_Icc d q hq)
let ν := (halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))
let f := fun x : (SourcePartition B d × (Fin B → ℝ)) ×
(Assign (Fin n) × (Fin (B * d) → Bool)) =>
reverseLatentStatistic n B d hfit σ h x.1.1 x.1.2 x.2
have hf : Measurable f := reverseLatentStatistic_joint_measurable n B d hfit σ h
have hi := reverseLatentLaw_squared_deviation_integrable n B d hd hfit σ h q a hq
rw [reverseLatentLaw_joint_map] at hi ⊢
have hfi : Integrable (fun x => (f x - a) ^ 2) ((blockParamLaw B d).prod ν) :=
(integrable_map_measure (by fun_prop) hf.aemeasurable).mp hi
rw [integral_map hf.aemeasurable (by fun_prop), integral_prod_symm _ hfi]
apply integral_mono_ae hfi.integral_prod_right Integrable.of_finite
filter_upwards [hfi.prod_left_ae] with zr hz
change Integrable (fun ξ => (f (ξ, zr) - a) ^ 2)
((partitionLaw B d).prod (blockBaselineLaw B)) at hz
unfold blockParamLaw
rw [integral_prod _ hz]
apply integral_mono_ae hz.integral_prod_left Integrable.of_finite
exact Filter.Eventually.of_forall (fun s =>
reverseLatentStatistic_baseline_energy_le n B d hfit σ h a s zr)
theorem reverse_source_baseline_energy
Summing the exact actual-row energies gives the finite baseline contribution. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, (∑ ℓ, (reverseRowA n B d hfit s ℓ zr) ^ 2) / 16 ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
= (B : ℝ) * d * p / 16
Proof (Lean source)
-- @node: reverse_source_baseline_energy
lemma reverse_source_baseline_energy (n B d : ℕ) (hfit : 2 * (B * d) ≤ n)
(s : SourcePartition B d) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ zr, (∑ ℓ, (reverseRowA n B d hfit s ℓ zr) ^ 2) / 16
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) =
(B : ℝ) * d * p / 16 := by
let := halfBernoulli_probability (V := Fin n)
let := bernoulliLaw_probability p hp
rw [integral_div, integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
simp only [reverse_source_row_second n B d hfit s _ p hp]
simp
ring
theorem reverseSignalEnergy_fixed_partition_le
The finite signal energy of every genuine partition satisfies the roadmap bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
s :
SourcePartition B d
σ :
h p :
ℝ
hh :
0 < h
hhu :
h ≤ 1 / 4
hp :
p ∈ Icc 0 1
conclusion
(∫ zr, reverseSignalEnergy n B d hfit σ h (signOf σ * (h * (B * d : ℕ) * p / (2 * d))) s zr ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))))
≤ 7 * (B * d : ℕ) * p / 64
Proof (Lean source)
-- @node: reverseSignalEnergy_fixed_partition_le
lemma reverseSignalEnergy_fixed_partition_le (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (s : SourcePartition B d)
(σ : Bool) (h p : ℝ) (hh : 0 < h) (hhu : h ≤ 1 / 4)
(hp : p ∈ Icc 0 1) :
(∫ zr, reverseSignalEnergy n B d hfit σ h
(signOf σ * (h * (B * d : ℕ) * p / (2 * d))) s zr
∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p)))) ≤
7 * (B * d : ℕ) * p / 64 := by
let := halfBernoulli_probability (V := Fin n)
let := bernoulliLaw_probability p hp
let ν := (halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw p))
let c : ℝ := signOf σ * h / (2 * d)
have he (zr : Assign (Fin n) × (Fin (B * d) → Bool)) :
reverseLatentStatistic n B d hfit σ h s (fun _ => 0) zr -
signOf σ * (h * (B * d : ℕ) * p / (2 * d)) =
c * ((∑ ℓ, reverseRowA n B d hfit s ℓ zr * reverseRowT n B d hfit s ℓ zr) -
(B : ℝ) * d * p) := by
simp only [reverseLatentStatistic, zero_add, reverseRowA, reverseRowT, Nat.cast_mul]
simp_rw [mul_left_comm _ (signOf σ * h / (2 * d)), ← Finset.mul_sum]
dsimp [c]
ring
have hs : c ^ 2 = h ^ 2 / (4 * (d : ℝ) ^ 2) := by
cases σ <;> simp [c, signOf, div_pow] <;> ring
have hd' : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
have hv := reverse_source_signal_variance_le n B d hfit s p hp
have hv' := mul_le_mul_of_nonneg_left hv (sq_nonneg c)
have henergy : (∫ zr, reverseSignalEnergy n B d hfit σ h
(signOf σ * (h * (B * d : ℕ) * p / (2 * d))) s zr ∂ν) ≤
(B : ℝ) * d * p / 16 + c ^ 2 * ((B : ℝ) * (3 * (d : ℝ) ^ 2 * p)) := by
unfold reverseSignalEnergy
simp_rw [he, mul_pow]
rw [integral_add Integrable.of_finite Integrable.of_finite, integral_const_mul]
change (∫ zr, (∑ ℓ, (reverseRowA n B d hfit s ℓ zr) ^ 2) / 16 ∂ν) + _ ≤ _
rw [reverse_source_baseline_energy n B d hfit s p hp]
exact add_le_add le_rfl hv'
apply henergy.trans
rw [hs, Nat.cast_mul]
have hc : h ^ 2 / (4 * (d : ℝ) ^ 2) * ((B : ℝ) * (3 * (d : ℝ) ^ 2 * p)) =
3 * h ^ 2 * B * p / 4 := by
field_simp
rw [hc]
have hh2 : h ^ 2 ≤ 1 / 16 := by nlinarith
have hm : 0 ≤ (B : ℝ) * p := mul_nonneg (Nat.cast_nonneg _) hp.1
have hb := mul_le_mul_of_nonneg_right hh2 hm
have hd1 : (1 : ℝ) ≤ d := by exact_mod_cast hd
nlinarith [mul_le_mul_of_nonneg_right hd1 hm]
theorem reverseSignalEnergy_average_le
Averaging the uniform fixed-partition bound retains its exact constant. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ zr, ∫ s, reverseSignalEnergy n B d hfit σ h (signOf σ * (h * (B * d : ℕ) * retentionP d q / (2 * d))) s zr ∂partitionLaw B d ∂((halfBernoulli (Fin n)).prod (Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q)))))
≤ 7 * (B * d : ℕ) * retentionP d q / 64
Proof (Lean source)
-- @node: reverseSignalEnergy_average_le
lemma reverseSignalEnergy_average_le (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (σ : Bool) (h q : ℝ)
(hh : 0 < h) (hhu : h ≤ 1 / 4) (hq : q ∈ Icc 0 1) :
(∫ zr, ∫ s, reverseSignalEnergy n B d hfit σ h
(signOf σ * (h * (B * d : ℕ) * retentionP d q / (2 * d))) s zr
∂partitionLaw B d ∂((halfBernoulli (Fin n)).prod
(Measure.pi (fun _ : Fin (B * d) => bernoulliLaw (retentionP d q))))) ≤
7 * (B * d : ℕ) * retentionP d q / 64 := by
let := halfBernoulli_probability (V := Fin n)
let := bernoulliLaw_probability (retentionP d q) (retentionP_mem_Icc d q hq)
let := partitionLaw_probability B d
rw [integral_integral_swap Integrable.of_finite]
calc
_ ≤ ∫ _s, (7 * (B * d : ℕ) * retentionP d q / 64 : ℝ) ∂partitionLaw B d := by
apply integral_mono Integrable.of_finite (integrable_const _)
intro s
exact reverseSignalEnergy_fixed_partition_le n B d hd hfit s σ h
(retentionP d q) hh hhu (retentionP_mem_Icc d q hq)
_ = _ := by simp
Substituting the exact center and row-variance bound gives the paper constant. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
m h p :
ℝ
hm :
0 < m
hh :
0 < h
hp :
0 < p
hd :
1 ≤ d
conclusion
2 * (7 * m * p / 64) / (h * m * p / (2 * d)) ^ 2 = 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * m * p)
Proof (Lean source)
-- @node: reverse_sign_test_constant
lemma reverse_sign_test_constant (d : ℕ) (m h p : ℝ)
(hm : 0 < m) (hh : 0 < h) (hp : 0 < p) (hd : 1 ≤ d) :
2 * (7 * m * p / 64) / (h * m * p / (2 * d)) ^ 2 =
7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * m * p) := by
have hd' : (d : ℝ) ≠ 0 := by exact_mod_cast (by omega : d ≠ 0)
field_simp
ring
theorem reverse_test_tv_bound
The revealed-sign outcome test supplies the reverse total-variation bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
h q :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hh :
0 < h
hhu :
h ≤ 1 / 4
hq :
q ∈ Icc 0 1
hp :
0 < retentionP d q
ha :
hw :
AuditLaw D q
hi :
conclusion
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q)
Proof (Lean source)
-- @node: reverse_test_tv_bound
lemma reverse_test_tv_bound (n B d : ℕ) (h q : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n))) (hB : 1 ≤ B) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (hh : 0 < h) (hhu : h ≤ 1 / 4) (hq : q ∈ Icc 0 1)
(hp : 0 < retentionP d q) (ha : AssignmentLaw D) (hw : AuditLaw D q)
(hi : DesignIndependent D) :
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) := by
let := design_isProbabilityMeasure D q ha hw hi
let := blockMixtureLawOf_probability n B d D true h
let := blockMixtureLawOf_probability n B d D false h
have hprojection := block_tvDist_map_le
(blockMixtureLawOf n B d D true h) (blockMixtureLawOf n B d D false h)
(reverseTestStatistic n B d) (reverseTestStatistic_measurable n B d)
rw [design_eq_thinnedDesign D q ha hw hi,
reverseTestStatistic_mixture_law n B d hd hfit true h q hq,
reverseTestStatistic_mixture_law n B d hd hfit false h q hq] at hprojection
rw [design_eq_thinnedDesign D q ha hw hi]
apply le_trans _ hprojection
let μ := reverseLatentLaw n B d hfit true h q
let ν := reverseLatentLaw n B d hfit false h q
let a : ℝ := h * (B * d : ℕ) * retentionP d q / (2 * d)
let v : ℝ := 7 * (B * d : ℕ) * retentionP d q / 64
let := reverseLatentLaw_probability n B d hd hfit true h q hq
let := reverseLatentLaw_probability n B d hd hfit false h q hq
have hm : (0 : ℝ) < (B * d : ℕ) := by
exact_mod_cast (Nat.mul_pos (by omega : 0 < B) (by omega : 0 < d))
have ha' : 0 < a := by
dsimp [a]
have hd' : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
positivity
-- The baseline bridge below reduces (4) in the roadmap to finite sign/reveal
-- moments and independent row variances. The sign test is fully proved.
have hmoment : ∀ σ : Bool,
Integrable (fun x => (x - signOf σ * a) ^ 2)
(reverseLatentLaw n B d hfit σ h q) ∧
(∫ x, (x - signOf σ * a) ^ 2
∂reverseLatentLaw n B d hfit σ h q) ≤ v := by
intro σ
refine ⟨reverseLatentLaw_squared_deviation_integrable n B d hd hfit σ h q
(signOf σ * a) hq, ?_⟩
apply (reverseLatentLaw_baseline_energy_le n B d hd hfit σ h q
(signOf σ * a) hq).trans
exact reverseSignalEnergy_average_le n B d hd hfit σ h q hh hhu hq
have hplus := hmoment true
have hminus := hmoment false
simp only [signOf, ↓reduceIte, one_mul] at hplus
simp only [signOf, Bool.false_eq_true, ↓reduceIte, neg_one_mul, sub_neg_eq_add] at hminus
have ht := reverse_scalar_sign_test_tv_le μ ν a v ha'
hplus.1 hminus.1 hplus.2 hminus.2
have hc := reverse_sign_test_constant d (B * d : ℕ) h (retentionP d q)
hm hh hp hd
simpa only [μ, ν, a, v, hc] using ht
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ReverseTestMoments 13 declarations The independent assignment signs have exact weighted second and fourth moments.
Exact sign moments for the revealed-source reverse test
The independent assignment signs have exact weighted second and fourth moments. These identities supply the finite-row moment calculations in the reverse test.
theorem integral_halfBernoulli_eq_signAverage
The actual independent assignment expectation equals the uniform Boolean average. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ z, f z ∂halfBernoulli (Fin d)) = signAverage f
Proof (Lean source)
-- @node: integral_halfBernoulli_eq_signAverage
lemma integral_halfBernoulli_eq_signAverage (d : ℕ) (f : Assign (Fin d) → ℝ) :
(∫ z, f z ∂halfBernoulli (Fin d)) = signAverage f := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
have hmass (z : Assign (Fin d)) :
(halfBernoulli (Fin d)) {z} = ofReal ((2 : ℝ)⁻¹ ^ d) := by
rw [halfBernoulli, Measure.pi_singleton]
have hc (b : Bool) : bernoulliLaw (1 / 2) {b} = ofReal (1 / 2) := by
cases b <;> simp [bernoulliLaw, Measure.add_apply, Measure.smul_apply,
Measure.dirac_apply', measurableSet_singleton] <;> norm_num [ENNReal.ofReal_div_of_pos]
simp only [hc, Finset.prod_const, Finset.card_univ, Fintype.card_fin]
rw [← ENNReal.ofReal_pow (by norm_num)]
norm_num
rw [integral_fintype (halfBernoulli_integrable f)]
simp only [measureReal_def, hmass, ENNReal.toReal_ofReal (by positivity :
0 ≤ (2 : ℝ)⁻¹ ^ d), smul_eq_mul, ← Finset.mul_sum, signAverage]
rw [div_eq_mul_inv, inv_pow]
ring
Weighted independent centered signs have their coefficient energy as second moment. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_sign_second_moment
lemma reverse_sign_second_moment (d : ℕ) (a : Fin d → ℝ) :
(∫ z, (∑ j, signOf (z j) * a j) ^ 2 ∂halfBernoulli (Fin d)) = ∑ j, a j ^ 2 := by
rw [integral_halfBernoulli_eq_signAverage]
have hs (b : Bool) : signOf b = if b then (1 : ℝ) else -1 := by cases b <;> rfl
simp_rw [hs]
rw [signAverage, (signSum_moments d a).1]
exact mul_div_cancel_left₀ _ (pow_ne_zero d (by norm_num))
The fourth moment retains the exact diagonal correction, rather than a loose bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_sign_fourth_moment
lemma reverse_sign_fourth_moment (d : ℕ) (a : Fin d → ℝ) :
(∫ z, (∑ j, signOf (z j) * a j) ^ 4 ∂halfBernoulli (Fin d)) =
3 * (∑ j, a j ^ 2) ^ 2 - 2 * ∑ j, a j ^ 4 := by
rw [integral_halfBernoulli_eq_signAverage]
have hs (b : Bool) : signOf b = if b then (1 : ℝ) else -1 := by cases b <;> rfl
simp_rw [hs]
exact signAverage_fourth_exact a
theorem reverse_two_level_sum
Summing two coefficient levels counts the revealed coordinates exactly. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_two_level_sum
lemma reverse_two_level_sum (d : ℕ) (R : Finset (Fin d)) (u v : ℝ) :
(∑ j : Fin d, if j ∈ R then u else v) = d * v + R.card * (u - v) := by
have he (j : Fin d) : (if j ∈ R then u else v) =
v + (if j ∈ R then u - v else 0) := by split_ifs <;> ring
simp_rw [he]
simp [Finset.sum_add_distrib, Finset.sum_ite_mem]
ring
theorem reverse_revealed_second_moment
A revealed sum over r coordinates has second moment r. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_second_moment
lemma reverse_revealed_second_moment (d : ℕ) (R : Finset (Fin d)) :
(∫ z, (∑ j ∈ R, signOf (z j)) ^ 2 ∂halfBernoulli (Fin d)) = R.card := by
have h := reverse_sign_second_moment d (fun j => if j ∈ R then 1 else 0)
simpa [mul_ite, Finset.sum_ite_mem] using h
theorem reverse_revealed_total_product_mean
Polarizing second moments gives E[AT] = r for revealed and total sign sums. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_total_product_mean
lemma reverse_revealed_total_product_mean (d : ℕ) (R : Finset (Fin d)) :
(∫ z, (∑ j ∈ R, signOf (z j)) * (∑ j : Fin d, signOf (z j))
∂halfBernoulli (Fin d)) = R.card := by
let A : Assign (Fin d) → ℝ := fun z => ∑ j ∈ R, signOf (z j)
let T : Assign (Fin d) → ℝ := fun z => ∑ j : Fin d, signOf (z j)
have h := reverse_sign_second_moment d (fun j => if j ∈ R then 2 else 1)
have hA := reverse_revealed_second_moment d R
have hT := reverse_sign_second_moment d (fun _ => 1)
have he (z : Assign (Fin d)) :
(∑ j, signOf (z j) * (if j ∈ R then 2 else 1)) = A z + T z := by
have hj (j : Fin d) : signOf (z j) * (if j ∈ R then 2 else 1) =
(if j ∈ R then signOf (z j) else 0) + signOf (z j) := by split_ifs <;> ring
simp_rw [hj]
simp [Finset.sum_add_distrib, Finset.sum_ite_mem, A, T]
simp_rw [he] at h
norm_num [ite_pow, reverse_two_level_sum] at h
simp only [mul_one, one_pow, Finset.sum_const, Finset.card_univ,
Fintype.card_fin, nsmul_eq_mul, mul_one] at hT
change (∫ z, A z ^ 2 ∂halfBernoulli (Fin d)) = _ at hA
change (∫ z, T z ^ 2 ∂halfBernoulli (Fin d)) = _ at hT
have hp : (∫ z, (A z + T z) ^ 2 ∂halfBernoulli (Fin d)) =
(∫ z, A z ^ 2 ∂halfBernoulli (Fin d)) +
2 * (∫ z, A z * T z ∂halfBernoulli (Fin d)) +
(∫ z, T z ^ 2 ∂halfBernoulli (Fin d)) := by
have he (z : Assign (Fin d)) : (A z + T z) ^ 2 =
A z ^ 2 + 2 * (A z * T z) + T z ^ 2 := by ring
simp_rw [he]
rw [integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_const_mul]
change (∫ z, A z * T z ∂halfBernoulli (Fin d)) = _
rw [h, hA, hT] at hp
linarith
theorem reverse_revealed_fourth_moment
A revealed sum over r coordinates has fourth moment 3r² − 2r. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_fourth_moment
lemma reverse_revealed_fourth_moment (d : ℕ) (R : Finset (Fin d)) :
(∫ z, (∑ j ∈ R, signOf (z j)) ^ 4 ∂halfBernoulli (Fin d)) =
3 * (R.card : ℝ) ^ 2 - 2 * R.card := by
have h := reverse_sign_fourth_moment d (fun j => if j ∈ R then 1 else 0)
simpa [mul_ite, Finset.sum_ite_mem] using h
theorem reverse_revealed_total_mixed_moment
Polarizing exact fourth moments yields the mixed revealed/total second moment. The coordinates outside R remain observed assignment coordinates. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_total_mixed_moment
lemma reverse_revealed_total_mixed_moment (d : ℕ) (R : Finset (Fin d)) :
(∫ z, (∑ j ∈ R, signOf (z j)) ^ 2 * (∑ j : Fin d, signOf (z j)) ^ 2
∂halfBernoulli (Fin d)) =
(R.card : ℝ) * d + 2 * R.card * (R.card - 1 : ℝ) := by
let A : Assign (Fin d) → ℝ := fun z => ∑ j ∈ R, signOf (z j)
let T : Assign (Fin d) → ℝ := fun z => ∑ j : Fin d, signOf (z j)
have hplus := reverse_sign_fourth_moment d (fun j => if j ∈ R then 2 else 1)
have hminus := reverse_sign_fourth_moment d (fun j => if j ∈ R then 0 else -1)
have hA := reverse_revealed_fourth_moment d R
have hT := reverse_sign_fourth_moment d (fun _ => 1)
have heplus (z : Assign (Fin d)) :
(∑ j, signOf (z j) * (if j ∈ R then 2 else 1)) = A z + T z := by
have he (j : Fin d) : signOf (z j) * (if j ∈ R then 2 else 1) =
(if j ∈ R then signOf (z j) else 0) + signOf (z j) := by split_ifs <;> ring
simp_rw [he]
simp [Finset.sum_add_distrib, Finset.sum_ite_mem, A, T]
have heminus (z : Assign (Fin d)) :
(∑ j, signOf (z j) * (if j ∈ R then 0 else -1)) = A z - T z := by
have he (j : Fin d) : signOf (z j) * (if j ∈ R then 0 else -1) =
(if j ∈ R then signOf (z j) else 0) - signOf (z j) := by split_ifs <;> ring
simp_rw [he]
simp [Finset.sum_sub_distrib, Finset.sum_ite_mem, A, T]
simp_rw [heplus] at hplus
simp_rw [heminus] at hminus
norm_num [ite_pow, reverse_two_level_sum] at hplus hminus
simp only [mul_one, one_pow, Finset.sum_const, Finset.card_univ,
Fintype.card_fin, nsmul_eq_mul, mul_one] at hT
change (∫ z, A z ^ 4 ∂halfBernoulli (Fin d)) = _ at hA
change (∫ z, T z ^ 4 ∂halfBernoulli (Fin d)) = _ at hT
have hpol : (∫ z, (A z + T z) ^ 4 ∂halfBernoulli (Fin d)) +
(∫ z, (A z - T z) ^ 4 ∂halfBernoulli (Fin d)) =
2 * (∫ z, A z ^ 4 ∂halfBernoulli (Fin d)) +
12 * (∫ z, A z ^ 2 * T z ^ 2 ∂halfBernoulli (Fin d)) +
2 * (∫ z, T z ^ 4 ∂halfBernoulli (Fin d)) := by
rw [← integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _)]
have he (z : Assign (Fin d)) :
(A z + T z) ^ 4 + (A z - T z) ^ 4 =
2 * A z ^ 4 + 12 * (A z ^ 2 * T z ^ 2) + 2 * T z ^ 4 := by ring
simp_rw [he]
rw [integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _)]
simp only [integral_const_mul]
change (∫ z, A z ^ 2 * T z ^ 2 ∂halfBernoulli (Fin d)) = _
rw [hplus, hminus, hA, hT] at hpol
nlinarith only [hpol]
theorem reverse_revealed_total_mixed_moment_le
The mixed row moment is at most 3rd, including the empty revealed set. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_total_mixed_moment_le
lemma reverse_revealed_total_mixed_moment_le (d : ℕ) (R : Finset (Fin d)) :
(∫ z, (∑ j ∈ R, signOf (z j)) ^ 2 * (∑ j : Fin d, signOf (z j)) ^ 2
∂halfBernoulli (Fin d)) ≤ 3 * R.card * d := by
rw [reverse_revealed_total_mixed_moment]
have hr : (R.card : ℝ) ≤ d := by
exact_mod_cast (show R.card ≤ d by simpa using R.card_le_univ)
nlinarith [mul_nonneg (Nat.cast_nonneg R.card) (sub_nonneg.mpr hr)]
The independent reveal count in a size-d row has expectation dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_reveal_count_mean
lemma reverse_reveal_count_mean (d : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r : Fin d → Bool, ((Finset.univ.filter (fun j => r j = true)).card : ℝ)
∂Measure.pi (fun _ : Fin d => bernoulliLaw p)) = d * p := by
let := bernoulliLaw_probability p hp
have he (r : Fin d → Bool) :
((Finset.univ.filter (fun j => r j = true)).card : ℝ) =
∑ j : Fin d, treatment (r j) := by
simp [treatment, Finset.sum_boole]
simp_rw [he]
rw [integral_finset_sum _ (fun _ _ => Integrable.of_finite)]
have hc (j : Fin d) :
(∫ r : Fin d → Bool, treatment (r j)
∂Measure.pi (fun _ : Fin d => bernoulliLaw p)) = p := by
rw [integral_comp_eval (by fun_prop : AEStronglyMeasurable treatment (bernoulliLaw p))]
rw [show bernoulliLaw p = bernoulliBool p by
unfold bernoulliLaw bernoulliBool
exact add_comm _ _]
rw [bernoulliBool_integral hp.1 hp.2]
simp [treatment]
simp [hc]
theorem reverse_revealed_second_moment_average Lemma reverse_revealed_second_moment_average in the paper ↗
Averaging the conditional sign second moment gives E A² = dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_second_moment_average
lemma reverse_revealed_second_moment_average (d : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r : Fin d → Bool, (∫ z, (∑ j ∈ Finset.univ.filter (fun j => r j = true),
signOf (z j)) ^ 2 ∂halfBernoulli (Fin d))
∂Measure.pi (fun _ : Fin d => bernoulliLaw p)) = d * p := by
simp_rw [reverse_revealed_second_moment]
exact reverse_reveal_count_mean d p hp
theorem reverse_revealed_total_product_mean_average Lemma reverse_revealed_total_product_mean_average in the paper ↗
Averaging the revealed/total sign product gives E[AT] = dp. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_total_product_mean_average
lemma reverse_revealed_total_product_mean_average (d : ℕ) (p : ℝ)
(hp : p ∈ Icc 0 1) :
(∫ r : Fin d → Bool, (∫ z, (∑ j ∈ Finset.univ.filter (fun j => r j = true),
signOf (z j)) * (∑ j : Fin d, signOf (z j)) ∂halfBernoulli (Fin d))
∂Measure.pi (fun _ : Fin d => bernoulliLaw p)) = d * p := by
simp_rw [reverse_revealed_total_product_mean]
exact reverse_reveal_count_mean d p hp
theorem reverse_revealed_total_mixed_moment_average_le Lemma reverse_revealed_total_mixed_moment_average_le in the paper ↗
Independent reveals and exact sign moments give E[A²T²] ≤ 3d²p. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: reverse_revealed_total_mixed_moment_average_le
lemma reverse_revealed_total_mixed_moment_average_le (d : ℕ) (p : ℝ)
(hp : p ∈ Icc 0 1) :
(∫ r : Fin d → Bool, (∫ z, (∑ j ∈ Finset.univ.filter (fun j => r j = true),
signOf (z j)) ^ 2 * (∑ j : Fin d, signOf (z j)) ^ 2 ∂halfBernoulli (Fin d))
∂Measure.pi (fun _ : Fin d => bernoulliLaw p)) ≤ 3 * (d : ℝ) ^ 2 * p := by
let := bernoulliLaw_probability p hp
calc
_ ≤ ∫ r : Fin d → Bool, 3 *
((Finset.univ.filter (fun j => r j = true)).card : ℝ) * d
∂Measure.pi (fun _ : Fin d => bernoulliLaw p) :=
integral_mono (Integrable.of_finite) (Integrable.of_finite)
(fun r => reverse_revealed_total_mixed_moment_le d _)
_ = 3 * (d : ℝ) ^ 2 * p := by
rw [integral_mul_const, integral_const_mul, reverse_reveal_count_mean d p hp]
ring
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.RowEnergyCounts 5 declarations Counting subsets of the actual hidden labels identifies the hidden active-row energy with the binomial degree sum in the contraction roadmap.
Row energies from hidden capacities
Counting subsets of the actual hidden labels identifies the hidden active-row energy with the binomial degree sum in the contraction roadmap. The revealed energy is its complement in the total nonconstant channel energy.
theorem rowHiddenEnergy_eq_powerset
Hidden energy sums exactly the nonconstant subsets of the hidden label set. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
rowHiddenEnergy d h r
Proof (Lean source)
-- @node: rowHiddenEnergy_eq_powerset
lemma rowHiddenEnergy_eq_powerset (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
rowHiddenEnergy d h r =
∑ E ∈ (Finset.univ.filter (fun j : Fin d => r j = false)).powerset,
rowSubsetEnergy d h E := by
classical
rw [rowHiddenEnergy]
have he (E : Finset (Fin d)) :
(¬ ∃ j ∈ E, r j = true) ↔
E ⊆ Finset.univ.filter (fun j : Fin d => r j = false) := by
simp only [Finset.subset_iff, mem_filter, Finset.mem_univ, true_and]
constructor
· intro hn j hj
cases hr : r j
· rfl
· exact False.elim (hn ⟨j, hj, hr⟩)
· intro hn ⟨j, hj, ht⟩
have := hn hj
simp [ht] at this
calc
_ = ∑ E : Finset (Fin d), if E ⊆ Finset.univ.filter
(fun j : Fin d => r j = false) then rowSubsetEnergy d h E else 0 := by
apply Finset.sum_congr rfl
intro E _
by_cases hr : ∃ j ∈ E, r j = true
· simp [hr, show ¬ E ⊆ Finset.univ.filter (fun j : Fin d => r j = false)
from fun hs => (he E).mpr hs hr]
· simp [hr, (he E).mp hr]
_ = _ := by
rw [← Finset.sum_filter]
congr 1
ext E
simp
theorem rowHiddenEnergy_by_count
Counting hidden subsets by cardinality gives the exact positive-degree binomial sum, including an empty hidden label set. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowHiddenEnergy_by_count
lemma rowHiddenEnergy_by_count (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
rowHiddenEnergy d h r =
∑ k ∈ Icc 1 (Finset.univ.filter (fun j : Fin d => r j = false)).card,
(((Finset.univ.filter (fun j : Fin d => r j = false)).card).choose k : ℝ) *
gamma d h k := by
classical
rw [rowHiddenEnergy_eq_powerset]
let U := Finset.univ.filter (fun j : Fin d => r j = false)
have hc := Finset.sum_powerset_apply_card
(fun k => if k = 0 then (0 : ℝ) else gamma d h k) (x := U)
simp only [Finset.card_eq_zero, nsmul_eq_mul] at hc
change (∑ E ∈ U.powerset, if E = ∅ then 0 else gamma d h E.card) = _
rw [hc]
have hr : range (U.card + 1) = insert 0 (Icc 1 U.card) := by
ext k
simp only [Finset.mem_range, Finset.mem_insert, Finset.mem_Icc]
omega
rw [hr, Finset.sum_insert (by simp)]
simp only [ite_true, mul_zero, zero_add]
apply Finset.sum_congr rfl
intro k hk
rw [if_neg (by have := (Finset.mem_Icc.mp hk).1; omega)]
theorem rowRevealedEnergy_by_count
The revealed energy is the remainder after the exact hidden-degree sum. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowRevealedEnergy_by_count
lemma rowRevealedEnergy_by_count (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
rowRevealedEnergy d h r = eta d h -
∑ k ∈ Icc 1 (Finset.univ.filter (fun j : Fin d => r j = false)).card,
(((Finset.univ.filter (fun j : Fin d => r j = false)).card).choose k : ℝ) *
gamma d h k := by
have hs := rowRevealEnergy_split d h r
rw [rowHiddenEnergy_by_count] at hs
linarith
theorem rowHiddenEnergy_fin_degree_sum
The positive-degree factor used by the restricted allocation sum is exactly hidden row energy, with the zero-degree allocation excluded. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowHiddenEnergy_fin_degree_sum
lemma rowHiddenEnergy_fin_degree_sum (d : ℕ) (h : ℝ) (r : Fin d → Bool)
(u : ℕ) (hu : u = (Finset.univ.filter (fun j : Fin d => r j = false)).card) :
(∑ k : Fin (u + 1), if 0 < k.val then
(u.choose k.val : ℝ) * gamma d h k.val else 0) = rowHiddenEnergy d h r := by
classical
rw [rowHiddenEnergy_by_count, ← hu]
rw [Fin.sum_univ_eq_sum_range (fun k => if 0 < k then
(u.choose k : ℝ) * gamma d h k else 0)]
have hr : range (u + 1) = insert 0 (Icc 1 u) := by
ext k
simp only [Finset.mem_range, Finset.mem_insert, Finset.mem_Icc]
omega
rw [hr, Finset.sum_insert (by simp)]
simp only [lt_self_iff_false, if_false, zero_add]
apply Finset.sum_congr rfl
intro k hk
rw [if_pos (by have := (Finset.mem_Icc.mp hk).1; omega)]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.RowReferenceDomination 15 declarations The finite reference mixture controls each row likelihood, including the zero-density set.
Domination of translated rows by their reference law
The finite reference mixture controls each row likelihood, including the zero-density set. These bounds justify the finite integrals in the hidden-allocation roadmap.
Each component of the uniform reference mixture is bounded by its number of components times the mixture density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
rowDensity d h ε w ≤ (2 : ℝ) ^ d * refDensity d h w
Proof (Lean source)
-- @node: rowDensity_le_reference
lemma rowDensity_le_reference (d : ℕ) (h w : ℝ) (ε : Fin d → Bool) :
rowDensity d h ε w ≤ (2 : ℝ) ^ d * refDensity d h w := by
have hs := Finset.single_le_sum (fun δ _ => cosSqDensity_nonneg
(w - h / (2 * d) * ∑ j, signOf (δ j))) (Finset.mem_univ ε)
simpa only [refDensity, rowDensity, ← mul_assoc, mul_inv_cancel₀ (by positivity : (2 : ℝ) ^ d ≠ 0),
one_mul] using hs
The row likelihood is nonnegative and bounded on the entire real line, with the conventional zero value at a zero of the reference density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
0 ≤ rowDensity d h ε w / refDensity d h w
conclusion 2
rowDensity d h ε w / refDensity d h w ≤ (2 : ℝ) ^ d
Proof (Lean source)
-- @node: rowLikelihood_bounds
lemma rowLikelihood_bounds (d : ℕ) (h w : ℝ) (ε : Fin d → Bool) :
0 ≤ rowDensity d h ε w / refDensity d h w ∧
rowDensity d h ε w / refDensity d h w ≤ (2 : ℝ) ^ d := by
constructor
· exact div_nonneg (cosSqDensity_nonneg _) (refDensity_nonneg d h w)
· by_cases hz : refDensity d h w = 0
· rw [hz, div_zero]; positivity
· exact (div_le_iff₀ (lt_of_le_of_ne (refDensity_nonneg d h w) (Ne.symm hz))).mpr
(rowDensity_le_reference d h w ε)
Multiplying the likelihood by the reference recovers the component density even on the zero-density set. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(rowDensity d h ε w / refDensity d h w) * refDensity d h w = rowDensity d h ε w
Proof (Lean source)
-- @node: rowLikelihood_mul_reference
lemma rowLikelihood_mul_reference (d : ℕ) (h w : ℝ) (ε : Fin d → Bool) :
(rowDensity d h ε w / refDensity d h w) * refDensity d h w =
rowDensity d h ε w := by
by_cases hz : refDensity d h w = 0
· rw [hz, mul_zero, rowDensity_eq_zero_of_refDensity_eq_zero d h w hz ε]
· exact div_mul_cancel₀ _ hz
Each translated row law is bounded by a finite multiple of the reference law. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowLaw_le_reference
lemma rowLaw_le_reference (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
volume.withDensity (fun w => ofReal (rowDensity d h ε w)) ≤
ofReal ((2 : ℝ) ^ d) •
volume.withDensity (fun w => ofReal (refDensity d h w)) := by
rw [← withDensity_smul' _ _ ENNReal.ofReal_ne_top]
apply withDensity_mono
filter_upwards [] with w
change ofReal (rowDensity d h ε w) ≤
ofReal ((2 : ℝ) ^ d) * ofReal (refDensity d h w)
rw [← ENNReal.ofReal_mul (by positivity : 0 ≤ (2 : ℝ) ^ d)]
exact ENNReal.ofReal_le_ofReal (rowDensity_le_reference d h w ε)
theorem rowLaw_absolutelyContinuous_reference Lemma rowLaw_absolutelyContinuous_reference in the paper ↗
The reference law dominates every sign-conditioned translated row law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
volume.withDensity (fun w => ofReal (rowDensity d h ε w)) ≪ volume.withDensity (fun w => ofReal (refDensity d h w))
Proof (Lean source)
-- @node: rowLaw_absolutelyContinuous_reference
lemma rowLaw_absolutelyContinuous_reference (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
volume.withDensity (fun w => ofReal (rowDensity d h ε w)) ≪
volume.withDensity (fun w => ofReal (refDensity d h w)) := by
exact Measure.absolutelyContinuous_of_le_smul (rowLaw_le_reference d h ε)
Reweighting the reference by the bounded row likelihood gives the exact translated law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
volume.withDensity (fun w => ofReal (rowDensity d h ε w))
= (volume.withDensity (fun w => ofReal (refDensity d h w))).withDensity (fun w => ofReal (rowDensity d h ε w / refDensity d h w))
Proof (Lean source)
-- @node: rowLaw_eq_reference_withDensity
lemma rowLaw_eq_reference_withDensity (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
volume.withDensity (fun w => ofReal (rowDensity d h ε w)) =
(volume.withDensity (fun w => ofReal (refDensity d h w))).withDensity
(fun w => ofReal (rowDensity d h ε w / refDensity d h w)) := by
rw [← withDensity_mul volume (by fun_prop) (by fun_prop)]
apply withDensity_congr_ae
filter_upwards [] with w
simp only [Pi.mul_apply]
rw [← ENNReal.ofReal_mul (refDensity_nonneg d h w), mul_comm,
rowLikelihood_mul_reference]
A normalized reference density defines a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
conclusion
Proof (Lean source)
-- @node: referenceRowLaw_probability
lemma referenceRowLaw_probability (d : ℕ) (h : ℝ) :
IsProbabilityMeasure (volume.withDensity (fun w => ofReal (refDensity d h w))) := by
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal (refDensity_integrable_normalized d h).1
(Filter.Eventually.of_forall (refDensity_nonneg d h)),
(refDensity_integrable_normalized d h).2]
simp
Each translated component is a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (volume.withDensity (fun w => ofReal (rowDensity d h ε w)))
Proof (Lean source)
-- @node: translatedRowLaw_probability
lemma translatedRowLaw_probability (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
IsProbabilityMeasure (volume.withDensity (fun w => ofReal (rowDensity d h ε w))) := by
constructor
rw [withDensity_apply _ MeasurableSet.univ, Measure.restrict_univ,
← ofReal_integral_eq_lintegral_ofReal (rowDensity_integrable_normalized d h ε).1
(Filter.Eventually.of_forall (fun w => cosSqDensity_nonneg
(w - h / (2 * d) * ∑ j, signOf (ε j)))),
(rowDensity_integrable_normalized d h ε).2]
simp
The exact bounded likelihood integrates to one under the reference law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ w, rowDensity d h ε w / refDensity d h w ∂volume.withDensity (fun w => ofReal (refDensity d h w)))
= 1
Proof (Lean source)
-- @node: rowLikelihood_integral_reference
lemma rowLikelihood_integral_reference (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
(∫ w, rowDensity d h ε w / refDensity d h w
∂volume.withDensity (fun w => ofReal (refDensity d h w))) = 1 := by
rw [integral_withDensity_eq_integral_toReal_smul (by fun_prop)
(Filter.Eventually.of_forall (fun _ => ENNReal.ofReal_lt_top))]
simp only [ENNReal.toReal_ofReal (refDensity_nonneg d h _), smul_eq_mul]
simp_rw [mul_comm (refDensity d h _), rowLikelihood_mul_reference]
exact (rowDensity_integrable_normalized d h ε).2
theorem translatedRows_absolutelyContinuous_reference Lemma translatedRows_absolutelyContinuous_reference in the paper ↗
Independent rows with arbitrary fixed sign vectors are dominated by the independent reference responses. This does not replace the constrained allocation by independent signs. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: translatedRows_absolutelyContinuous_reference
lemma translatedRows_absolutelyContinuous_reference (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) :
Measure.pi (fun ℓ => volume.withDensity (fun w => ofReal (rowDensity d h (ε ℓ) w))) ≪
Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w))) := by
let ν := volume.withDensity (fun w => ofReal (refDensity d h w))
let μ := fun δ : Fin d → Bool => volume.withDensity
(fun w => ofReal (rowDensity d h δ w))
let := referenceRowLaw_probability d h
haveI (δ : Fin d → Bool) : IsProbabilityMeasure (μ δ) := translatedRowLaw_probability d h δ
change Measure.pi (fun ℓ => μ (ε ℓ)) ≪ Measure.pi (fun _ : Fin B => ν)
induction B with
| zero => rw [Measure.pi_of_empty, Measure.pi_of_empty]
| succ B ih =>
let e := piFinSuccAbove (fun _ : Fin (B + 1) => ℝ) 0
have hμ := (measurePreserving_piFinSuccAbove (fun ℓ => μ (ε ℓ)) 0).map_eq
have hν := (measurePreserving_piFinSuccAbove (fun _ : Fin (B + 1) => ν) 0).map_eq
have hprod := (rowLaw_absolutelyContinuous_reference d h (ε 0)).prod
(ih (fun ℓ => ε ((0 : Fin (B + 1)).succAbove ℓ)))
have hmap : (Measure.pi (fun ℓ => μ (ε ℓ))).map e ≪
(Measure.pi (fun _ : Fin (B + 1) => ν)).map e := by
rw [hμ, hν]
exact hprod
have hmapped := hmap.map (f := e.symm) e.symm.measurable
rwa [Measure.map_map e.symm.measurable e.measurable,
MeasurableEquiv.symm_comp_self, Measure.map_id,
Measure.map_map e.symm.measurable e.measurable,
MeasurableEquiv.symm_comp_self, Measure.map_id] at hmapped
A product of row likelihoods is bounded by the product of their finite row bounds. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
0 ≤ (∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ))
conclusion 2
(∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) ≤ ((2 : ℝ) ^ d) ^ B
Proof (Lean source)
-- @node: rowLikelihood_product_bounds
lemma rowLikelihood_product_bounds (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) (y : Fin B → ℝ) :
0 ≤ (∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) ∧
(∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) ≤
((2 : ℝ) ^ d) ^ B := by
constructor
· exact Finset.prod_nonneg (fun ℓ _ => (rowLikelihood_bounds d h (y ℓ) (ε ℓ)).1)
· calc
_ ≤ ∏ _ : Fin B, (2 : ℝ) ^ d := Finset.prod_le_prod
(fun ℓ _ => (rowLikelihood_bounds d h (y ℓ) (ε ℓ)).1)
(fun ℓ _ => (rowLikelihood_bounds d h (y ℓ) (ε ℓ)).2)
_ = _ := by simp
A finite product of the total row likelihoods is measurable, including at reference zeros.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: rowLikelihood_product_measurable
@[fun_prop]
lemma rowLikelihood_product_measurable (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) :
Measurable (fun y : Fin B → ℝ =>
∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) := by
apply Finset.measurable_prod
intro ℓ _
exact ((rowDensity_measurable d h (ε ℓ)).comp (measurable_pi_apply ℓ)).div
((refDensity_measurable d h).comp (measurable_pi_apply ℓ))
A fixed-allocation likelihood is integrable under the independent reference responses. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable (fun y : Fin B → ℝ => ∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) (Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
Proof (Lean source)
-- @node: rowLikelihood_product_integrable
lemma rowLikelihood_product_integrable (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) :
Integrable (fun y : Fin B → ℝ =>
∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ))
(Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) := by
let := referenceRowLaw_probability d h
apply (integrable_const (((2 : ℝ) ^ d) ^ B)).mono'
(rowLikelihood_product_measurable B d h ε).aestronglyMeasurable
filter_upwards [] with y
rw [Real.norm_eq_abs, abs_of_nonneg (rowLikelihood_product_bounds B d h ε y).1]
exact (rowLikelihood_product_bounds B d h ε y).2
theorem rowLikelihood_product_sq_integrable Lemma rowLikelihood_product_sq_integrable in the paper ↗
Fixed-allocation likelihoods have finite second moments under the reference law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable (fun y : Fin B → ℝ => (∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) ^ 2) (Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
Proof (Lean source)
-- @node: rowLikelihood_product_sq_integrable
lemma rowLikelihood_product_sq_integrable (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) :
Integrable (fun y : Fin B → ℝ =>
(∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)) ^ 2)
(Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) := by
let := referenceRowLaw_probability d h
apply (integrable_const ((((2 : ℝ) ^ d) ^ B) ^ 2)).mono'
((rowLikelihood_product_measurable B d h ε).pow_const 2).aestronglyMeasurable
filter_upwards [] with y
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
exact pow_le_pow_left₀ (rowLikelihood_product_bounds B d h ε y).1
(rowLikelihood_product_bounds B d h ε y).2 2
theorem rowLikelihood_product_integral_reference Lemma rowLikelihood_product_integral_reference in the paper ↗
The product likelihood remains normalized because its reference responses are independent. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ y : Fin B → ℝ, ∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ) ∂Measure.pi (fun _ : Fin B => volume.withDensity (fun w => ofReal (refDensity d h w))))
= 1
Proof (Lean source)
-- @node: rowLikelihood_product_integral_reference
lemma rowLikelihood_product_integral_reference (B d : ℕ) (h : ℝ)
(ε : Fin B → Fin d → Bool) :
(∫ y : Fin B → ℝ, ∏ ℓ, rowDensity d h (ε ℓ) (y ℓ) / refDensity d h (y ℓ)
∂Measure.pi (fun _ : Fin B => volume.withDensity
(fun w => ofReal (refDensity d h w)))) = 1 := by
let := referenceRowLaw_probability d h
rw [integral_fintype_prod_eq_prod
(fun ℓ w => rowDensity d h (ε ℓ) w / refDensity d h w)]
simp_rw [rowLikelihood_integral_reference]
exact Finset.prod_const_one
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.RowRevealEnergy 17 declarations Splits the nonconstant Walsh energy by whether a source subset contains a revealed label.
Revealed and hidden row energies
Splits the nonconstant Walsh energy by whether a source subset contains a revealed label. Under independent source reveals, the expected revealed energy is at most the reveal probability times the order-weighted channel energy.
Nonconstant energy attached to a labeled source subset; the empty subset is excluded.
Energy of source subsets containing at least one actually revealed label.
Energy of nonempty source subsets containing only hidden labels.
theorem rowSubsetEnergy_nonneg
Each source subset contributes nonnegative energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowSubsetEnergy_nonneg
lemma rowSubsetEnergy_nonneg (d : ℕ) (h : ℝ) (E : Finset (Fin d)) :
0 ≤ rowSubsetEnergy d h E := by
unfold rowSubsetEnergy
split_ifs
· exact le_rfl
· exact gamma_nonneg d h E.card
theorem rowSubsetEnergy_sum
Counting subsets by cardinality recovers the total nonconstant channel energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowSubsetEnergy_sum
lemma rowSubsetEnergy_sum (d : ℕ) (h : ℝ) :
(∑ E : Finset (Fin d), rowSubsetEnergy d h E) = eta d h := by
have hc := Finset.sum_powerset_apply_card
(fun s => if s = 0 then (0 : ℝ) else gamma d h s)
(x := (Finset.univ : Finset (Fin d)))
simp only [Finset.powerset_univ, Finset.card_univ, Fintype.card_fin,
Finset.card_eq_zero, nsmul_eq_mul] at hc
change (∑ E : Finset (Fin d), if E = ∅ then 0 else gamma d h E.card) = _
rw [hc]
have hr : range (d + 1) = insert 0 (Icc 1 d) := by
ext s
simp only [Finset.mem_range, Finset.mem_insert, Finset.mem_Icc]
omega
rw [hr, Finset.sum_insert (by simp)]
simp only [ite_true, mul_zero, zero_add]
apply Finset.sum_congr rfl
intro s hs
rw [if_neg (by have := (Finset.mem_Icc.mp hs).1; omega)]
theorem rowRevealEnergy_split
Revealed and hidden subsets partition the full nonconstant row energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: rowRevealEnergy_split
lemma rowRevealEnergy_split (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
rowRevealedEnergy d h r + rowHiddenEnergy d h r = eta d h := by
rw [rowRevealedEnergy, rowHiddenEnergy, ← Finset.sum_add_distrib,
← rowSubsetEnergy_sum d h]
apply Finset.sum_congr rfl
intro E _
split_ifs <;> simp
theorem rowRevealEnergy_bounds
Both parts are nonnegative, and hidden row energy is bounded by total energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
0 ≤ rowRevealedEnergy d h r
conclusion 2
0 ≤ rowHiddenEnergy d h r
conclusion 3
rowHiddenEnergy d h r ≤ eta d h
Proof (Lean source)
-- @node: rowRevealEnergy_bounds
lemma rowRevealEnergy_bounds (d : ℕ) (h : ℝ) (r : Fin d → Bool) :
0 ≤ rowRevealedEnergy d h r ∧ 0 ≤ rowHiddenEnergy d h r ∧
rowHiddenEnergy d h r ≤ eta d h := by
have hR : 0 ≤ rowRevealedEnergy d h r := by
apply Finset.sum_nonneg
intro E _
split_ifs
· exact rowSubsetEnergy_nonneg d h E
· exact le_rfl
have hH : 0 ≤ rowHiddenEnergy d h r := by
apply Finset.sum_nonneg
intro E _
split_ifs
· exact le_rfl
· exact rowSubsetEnergy_nonneg d h E
exact ⟨hR, hH, by linarith [rowRevealEnergy_split d h r]⟩
theorem rowSubset_reveal_law
A subset of independently revealed source labels is hit with its exact OR probability. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowSubset_reveal_law
lemma rowSubset_reveal_law (d : ℕ) (E : Finset (Fin d)) (p : ℝ)
(hp : p ∈ Icc 0 1) :
(Measure.pi (fun _ : Fin d => bernoulliLaw p)).map
(fun r => decide (∃ j ∈ E, r j = true)) = bernoulliLaw (retentionP E.card p) := by
let e : Fin E.card ≃ E := (Fintype.equivFinOfCardEq (by simp)).symm
let f : Fin E.card → Fin d := fun j => (e j).val
have hf : Injective f := by
intro j k hjk
exact e.injective (Subtype.ext hjk)
have hs := bernoulli_select_coordinates f hf p hp
have hm := congrArg (fun μ : Measure (Fin E.card → Bool) =>
μ.map (fun r => decide (∃ j, r j = true))) hs
rw [Measure.map_map (by fun_prop) (by fun_prop), bernoulli_any_coordinate E.card p hp] at hm
convert hm using 1
congr 1
funext r
apply Bool.decide_congr
constructor
· rintro ⟨j, hj, hr⟩
obtain ⟨k, hk⟩ := e.surjective ⟨j, hj⟩
exact ⟨k, by simpa [f, hk] using hr⟩
· rintro ⟨k, hk⟩
exact ⟨f k, (e k).property, hk⟩
theorem rowSubset_reveal_mean
Exact mean of a subset's reveal indicator. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowSubset_reveal_mean
lemma rowSubset_reveal_mean (d : ℕ) (E : Finset (Fin d)) (p : ℝ)
(hp : p ∈ Icc 0 1) :
(∫ r, (if ∃ j ∈ E, r j = true then (1 : ℝ) else 0)
∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) = retentionP E.card p := by
have he := integral_map (μ := Measure.pi (fun _ : Fin d => bernoulliLaw p))
(φ := fun r => decide (∃ j ∈ E, r j = true)) (by fun_prop)
(f := fun b : Bool => if b = true then (1 : ℝ) else 0) (by fun_prop)
rw [rowSubset_reveal_law d E p hp] at he
simp only [decide_eq_true_eq] at he
have hp' := retentionP_mem_Icc E.card p hp
rw [← he]
rw [bernoulliLaw, integral_add_measure, integral_smul_measure, integral_smul_measure]
· simp [ENNReal.toReal_ofReal hp'.1]
· exact (Integrable.of_finite (μ := Measure.dirac false)).smul_measure (by finiteness)
· exact (Integrable.of_finite (μ := Measure.dirac true)).smul_measure (by finiteness)
theorem retentionP_le_order_mul
The elementary union bound controls a subset's probability of containing a reveal. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: retentionP_le_order_mul
lemma retentionP_le_order_mul (s : ℕ) (p : ℝ) (hp : p ∈ Icc 0 1) :
retentionP s p ≤ (s : ℝ) * p := by
have hpow := one_add_mul_le_pow (show (-2 : ℝ) ≤ -p by linarith [hp.2]) s
unfold retentionP
convert (show 1 - (1 + -p) ^ s ≤ (s : ℝ) * p by linarith [hpow]) using 1
ring
theorem rowRevealedEnergy_mean
Independent source reveals give exactly the cardinality-weighted revealed row energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p)))
Proof (Lean source)
-- @node: rowRevealedEnergy_mean
lemma rowRevealedEnergy_mean (d : ℕ) (h p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) =
∑ E : Finset (Fin d), retentionP E.card p * rowSubsetEnergy d h E := by
let := bernoulliLaw_probability p hp
unfold rowRevealedEnergy
rw [integral_finsetSum _ (fun _ _ => Integrable.of_finite)]
apply Finset.sum_congr rfl
intro E _
have hf : (fun r : Fin d → Bool => if ∃ j ∈ E, r j = true then rowSubsetEnergy d h E else 0) =
fun r => (if ∃ j ∈ E, r j = true then (1 : ℝ) else 0) * rowSubsetEnergy d h E := by
funext r
split_ifs <;> simp
rw [hf, integral_mul_const, rowSubset_reveal_mean d E p hp]
theorem rowRevealedEnergy_mean_by_order
Counting source subsets by order gives the exact revealed-energy mean from the roadmap. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p)))
Proof (Lean source)
-- @node: rowRevealedEnergy_mean_by_order
lemma rowRevealedEnergy_mean_by_order (d : ℕ) (h p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) =
∑ s ∈ Icc 1 d, (d.choose s : ℝ) * retentionP s p * gamma d h s := by
rw [rowRevealedEnergy_mean d h p hp]
have hf (E : Finset (Fin d)) : retentionP E.card p * rowSubsetEnergy d h E =
retentionP E.card p * gamma d h E.card := by
by_cases hE : E = ∅
· simp [hE, retentionP]
· simp [rowSubsetEnergy, hE]
simp_rw [hf]
have hc := Finset.sum_powerset_apply_card (fun s => retentionP s p * gamma d h s)
(x := (Finset.univ : Finset (Fin d)))
simp only [Finset.powerset_univ, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul] at hc
rw [hc]
have hr : range (d + 1) = insert 0 (Icc 1 d) := by
ext s
simp only [Finset.mem_range, Finset.mem_insert, Finset.mem_Icc]
omega
rw [hr, Finset.sum_insert (by simp)]
simp only [retentionP, pow_zero, sub_self, zero_mul, mul_zero, zero_add, mul_assoc]
theorem rowSubsetEnergy_weighted_sum
Counting order-weighted subset energies recovers the weighted Walsh channel energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
conclusion
Proof (Lean source)
-- @node: rowSubsetEnergy_weighted_sum
lemma rowSubsetEnergy_weighted_sum (d : ℕ) (h : ℝ) :
(∑ E : Finset (Fin d), (E.card : ℝ) * rowSubsetEnergy d h E) = etaOne d h := by
have hc := Finset.sum_powerset_apply_card (fun s => (s : ℝ) * gamma d h s)
(x := (Finset.univ : Finset (Fin d)))
simp only [Finset.powerset_univ, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul] at hc
have hf (E : Finset (Fin d)) : (E.card : ℝ) * rowSubsetEnergy d h E =
(E.card : ℝ) * gamma d h E.card := by
by_cases hE : E = ∅
· simp [hE]
· simp [rowSubsetEnergy, hE]
simp_rw [hf]
rw [hc]
have hr : range (d + 1) = insert 0 (Icc 1 d) := by
ext s
simp only [Finset.mem_range, Finset.mem_insert, Finset.mem_Icc]
omega
rw [hr, Finset.sum_insert (by simp)]
simp only [Nat.cast_zero, zero_mul, mul_zero, zero_add]
apply Finset.sum_congr rfl
intro s _
ring
theorem rowRevealedEnergy_mean_le
The expected revealed row energy is at most reveal probability times weighted energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p)))
≤ p * etaOne d h
Proof (Lean source)
-- @node: rowRevealedEnergy_mean_le
lemma rowRevealedEnergy_mean_le (d : ℕ) (h p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) ≤
p * etaOne d h := by
rw [rowRevealedEnergy_mean d h p hp, ← rowSubsetEnergy_weighted_sum d h, Finset.mul_sum]
apply Finset.sum_le_sum
intro E _
calc
_ ≤ ((E.card : ℝ) * p) * rowSubsetEnergy d h E :=
mul_le_mul_of_nonneg_right (retentionP_le_order_mul E.card p hp)
(rowSubsetEnergy_nonneg d h E)
_ = _ := by ring
theorem rowHiddenEnergy_mean_split
The mean hidden energy is the complement of the mean revealed energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p)))
+ (∫ r, rowHiddenEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p)))
= eta d h
Proof (Lean source)
-- @node: rowHiddenEnergy_mean_split
lemma rowHiddenEnergy_mean_split (d : ℕ) (h p : ℝ) (hp : p ∈ Icc 0 1) :
(∫ r, rowRevealedEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) +
(∫ r, rowHiddenEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) =
eta d h := by
let := bernoulliLaw_probability p hp
rw [← integral_add Integrable.of_finite Integrable.of_finite]
simp_rw [rowRevealEnergy_split]
simp
theorem rowHiddenEnergy_mean_bounds
The mean hidden row energy lies between zero and total channel energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 1
0 ≤ (∫ r, rowHiddenEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p)))
conclusion 2
Proof (Lean source)
-- @node: rowHiddenEnergy_mean_bounds
lemma rowHiddenEnergy_mean_bounds (d : ℕ) (h p : ℝ) (hp : p ∈ Icc 0 1) :
0 ≤ (∫ r, rowHiddenEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) ∧
(∫ r, rowHiddenEnergy d h r ∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) ≤
eta d h := by
have hR : 0 ≤ (∫ r, rowRevealedEnergy d h r
∂(Measure.pi (fun _ : Fin d => bernoulliLaw p))) :=
integral_nonneg (fun r => (rowRevealEnergy_bounds d h r).1)
exact ⟨integral_nonneg (fun r => (rowRevealEnergy_bounds d h r).2.1),
by linarith [rowHiddenEnergy_mean_split d h p hp]⟩
theorem independent_rowRevealEnergy_product_bound
Independent row reveals factor the energies of disjoint active revealed and hidden row sets, giving the product bound in the contraction roadmap. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ rows : Fin B → Fin d → Bool, (∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) * (∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)) ∂(Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p))))
Proof (Lean source)
-- @node: independent_rowRevealEnergy_product_bound
lemma independent_rowRevealEnergy_product_bound (B d : ℕ) (h p : ℝ)
(hp : p ∈ Icc 0 1) (A E : Finset (Fin B)) (hAE : Disjoint A E) :
(∫ rows : Fin B → Fin d → Bool,
(∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ))
∂(Measure.pi (fun _ : Fin B => Measure.pi (fun _ : Fin d => bernoulliLaw p)))) ≤
(p * etaOne d h) ^ A.card * (eta d h) ^ E.card := by
let := bernoulliLaw_probability p hp
let μ := Measure.pi (fun _ : Fin d => bernoulliLaw p)
let F (ℓ : Fin B) (r : Fin d → Bool) : ℝ :=
(if ℓ ∈ A then rowRevealedEnergy d h r else 1) *
(if ℓ ∈ E then rowHiddenEnergy d h r else 1)
have hF (rows : Fin B → Fin d → Bool) :
(∏ ℓ ∈ A, rowRevealedEnergy d h (rows ℓ)) *
(∏ ℓ ∈ E, rowHiddenEnergy d h (rows ℓ)) = ∏ ℓ, F ℓ (rows ℓ) := by
simp only [F, Finset.prod_mul_distrib]
simp
simp_rw [hF]
rw [integral_fintype_prod_eq_prod F]
calc
(∏ ℓ : Fin B, ∫ r, F ℓ r ∂μ) ≤
∏ ℓ : Fin B, (if ℓ ∈ A then p * etaOne d h else 1) *
(if ℓ ∈ E then eta d h else 1) := by
apply Finset.prod_le_prod
· intro ℓ _
apply integral_nonneg
intro r
exact mul_nonneg
(by split_ifs <;> first | exact (rowRevealEnergy_bounds d h r).1 | positivity)
(by split_ifs <;> first | exact (rowRevealEnergy_bounds d h r).2.1 | positivity)
· intro ℓ _
by_cases hA : ℓ ∈ A
· have hE : ℓ ∉ E := fun hE => Finset.disjoint_left.mp hAE hA hE
simpa [F, μ, hA, hE] using rowRevealedEnergy_mean_le d h p hp
· by_cases hE : ℓ ∈ E
· simpa [F, μ, hA, hE] using (rowHiddenEnergy_mean_bounds d h p hp).2
· simp [F, μ, hA, hE]
_ = _ := by
rw [Finset.prod_mul_distrib]
simp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ScoreAudit 1 declarations Exact audit score moments
Exact audit score moments
theorem score_audit
Unbiasedness, the exact assignment-audit variance identity, and the explicit oracle bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
d :
ℕ
q :
ℝ
hn :
4 ≤ Fintype.card V
hd :
1 ≤ d
hdu :
d ≤ Fintype.card V - 1
hclass :
ScheduleClass θ d
ha :
hw :
AuditLaw D q
hi :
hq :
0 < q
conclusion
conclusion 1
conclusion 2
+ 4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) * ∑ i, (inNbhd θ i).card * ∫ z, (potentialOutcome θ i z) ^ 2 ∂(halfBernoulli V)
conclusion 3
≤ 5 * ((d : ℝ) + 1) ^ 2 / Fintype.card V
Proof (Lean source)
-- @node: lem:score-audit
lemma score_audit (D : Measure (Assign V × Audit V)) (θ : Schedule V) (d : ℕ) (q : ℝ)
(hn : 4 ≤ Fintype.card V) (hd : 1 ≤ d) (hdu : d ≤ Fintype.card V - 1)
(hclass : ScheduleClass θ d) (ha : AssignmentLaw D) (hw : AuditLaw D q)
(hi : DesignIndependent D) (hq : 0 < q) :
(∫ ω, auditScore q (recordOf θ ω) ∂D) = tte θ ∧
variance (fun ω => auditScore q (recordOf θ ω)) D =
variance (oracleScore θ) (halfBernoulli V) +
4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) *
∑ i, (inNbhd θ i).card *
∫ z, (potentialOutcome θ i z) ^ 2 ∂(halfBernoulli V) ∧
variance (oracleScore θ) (halfBernoulli V) ≤
5 * ((d : ℝ) + 1) ^ 2 / Fintype.card V := by
refine ⟨integral_auditScore_eq_tte D θ q ha hw hi hq, ?_,
oracle_variance_le θ d hn hd hclass⟩
exact variance_auditScore_eq_oracle_add D θ q (by omega) ha hw hi hq
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.ScoreMean 12 declarations Assignment moments and conditional audit means
Assignment moments and conditional audit means
The assignment law is a probability measure. the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: halfBernoulli_probability
lemma halfBernoulli_probability : IsProbabilityMeasure (halfBernoulli V) := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
unfold halfBernoulli
infer_instance
theorem halfBernoulli_integrable
Every real statistic on the finite assignment space is integrable. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: halfBernoulli_integrable
lemma halfBernoulli_integrable (f : Assign V → ℝ) : Integrable f (halfBernoulli V) := by
let := halfBernoulli_probability (V := V)
exact Integrable.of_finite
A half-Bernoulli centered sign has mean zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
j :
V
conclusion
(∫ z, signOf (z j) ∂halfBernoulli V) = 0
Proof (Lean source)
-- @node: integral_signOf_halfBernoulli
lemma integral_signOf_halfBernoulli (j : V) :
(∫ z, signOf (z j) ∂halfBernoulli V) = 0 := by
have he : (fun z : Assign V => signOf (z j)) =
fun z => 2 * centeredMonomial
(fun _ : V => (1 / 2 : ℝ)) {j} z := by
funext z
cases hz : z j <;>
simp [centeredMonomial,
treatInd, signOf, hz] <;> norm_num
rw [he, integral_halfBernoulli_eq_bernoulliDesign_E,
E_const_mul,
bernoulliDesign_E_centeredMonomial]
simp
Pairing a treatment indicator with a sign isolates its own coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
k j :
V
conclusion
(∫ z, treatment (z k) * signOf (z j) ∂halfBernoulli V) = if k = j then (1 / 2 : ℝ) else 0
Proof (Lean source)
-- @node: integral_treatment_mul_signOf
lemma integral_treatment_mul_signOf (k j : V) :
(∫ z, treatment (z k) * signOf (z j) ∂halfBernoulli V) =
if k = j then (1 / 2 : ℝ) else 0 := by
have he : (fun z : Assign V => treatment (z k) * signOf (z j)) =
fun z => 2 * (centeredMonomial
(fun _ : V => (1 / 2 : ℝ)) {j} z *
∏ l ∈ ({k} : Finset V), treatInd l z) := by
funext z
cases hk : z k <;> cases hj : z j <;>
simp [centeredMonomial,
treatInd, signOf, treatment, hk, hj] <;>
norm_num
rw [he, integral_halfBernoulli_eq_bernoulliDesign_E,
E_const_mul,
bernoulliDesign_E_centeredMonomial_mul_treatInd_prod]
by_cases h : k = j
· subst k; norm_num
· simp [h, Ne.symm h]
theorem integral_potentialOutcome_mul_signOf
A row outcome paired with a sign has half its corresponding additive coefficient. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
i j :
V
conclusion
(∫ z, potentialOutcome θ i z * signOf (z j) ∂halfBernoulli V)
= (θ.t i * (if i = j then 1 else 0) + ∑ k ∈ inNbhd θ i, θ.b i k * (if k = j then 1 else 0)) / 2
Proof (Lean source)
-- @node: integral_potentialOutcome_mul_signOf
lemma integral_potentialOutcome_mul_signOf (θ : Schedule V) (i j : V) :
(∫ z, potentialOutcome θ i z * signOf (z j) ∂halfBernoulli V) =
(θ.t i * (if i = j then 1 else 0) +
∑ k ∈ inNbhd θ i, θ.b i k * (if k = j then 1 else 0)) / 2 := by
simp_rw [potentialOutcome, add_mul, Finset.sum_mul, mul_assoc]
rw [integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_const_mul, integral_signOf_halfBernoulli, mul_zero, zero_add,
integral_const_mul, integral_treatment_mul_signOf,
integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
simp_rw [integral_const_mul, integral_treatment_mul_signOf]
simp only [div_eq_mul_inv, add_mul, Finset.sum_mul]
congr 1
· split <;> ring
· apply Finset.sum_congr rfl
intro k _
split <;> ring
theorem integral_oracleScore_eq_tte
Orthogonal sign moments identify the oracle score's constant term with the causal target. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
conclusion
(∫ z, oracleScore θ z ∂halfBernoulli V) = tte θ
Proof (Lean source)
-- @node: integral_oracleScore_eq_tte
lemma integral_oracleScore_eq_tte (θ : Schedule V) :
(∫ z, oracleScore θ z ∂halfBernoulli V) = tte θ := by
unfold oracleScore
rw [integral_const_mul, integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
have hrow (i : V) :
(∫ z, potentialOutcome θ i z * (signOf (z i) +
∑ j ∈ inNbhd θ i, signOf (z j)) ∂halfBernoulli V) =
(θ.t i + ∑ j ∈ inNbhd θ i, θ.b i j) / 2 := by
simp_rw [mul_add, Finset.mul_sum]
rw [integral_add (halfBernoulli_integrable _) (halfBernoulli_integrable _),
integral_finsetSum _ (fun _ _ => halfBernoulli_integrable _)]
simp_rw [integral_potentialOutcome_mul_signOf]
have hii : i ∉ inNbhd θ i := by simp [inNbhd, θ.irrefl i]
simp only [ite_true, mul_one]
simp_rw [mul_ite, mul_one, mul_zero]
simp only [Finset.sum_ite_eq', hii, ite_false, add_zero]
have hs : (∑ j ∈ inNbhd θ i,
((if i = j then θ.t i else 0) + if j ∈ inNbhd θ i then θ.b i j else 0) / 2) =
(∑ j ∈ inNbhd θ i, θ.b i j) / 2 := by
rw [Finset.sum_div]
apply Finset.sum_congr rfl
intro j hj
have hji : i ≠ j := fun h => hii (h ▸ hj)
simp [hji, hj]
rw [hs]
ring
simp_rw [hrow]
simp only [tte, potentialOutcome, treatment, ite_true, Bool.false_eq_true,
ite_false, mul_one, mul_zero, Finset.sum_const_zero, add_zero]
simp_rw [show ∀ i, θ.a i + θ.t i + ∑ j ∈ inNbhd θ i, θ.b i j - θ.a i =
θ.t i + ∑ j ∈ inNbhd θ i, θ.b i j by intro i; ring]
rw [← Finset.sum_div]
ring
The audit product is a probability law at admissible retention. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (auditLaw V q)
Proof (Lean source)
-- @node: auditLaw_probability
lemma auditLaw_probability (q : ℝ) (hq : q ∈ Icc 0 1) :
IsProbabilityMeasure (auditLaw V q) := by
let := bernoulliLaw_probability q hq
unfold auditLaw
infer_instance
theorem auditLaw_integrable
Every real audit statistic is integrable on the finite audit space. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Integrable f (auditLaw V q)
Proof (Lean source)
-- @node: auditLaw_integrable
lemma auditLaw_integrable (q : ℝ) (hq : q ∈ Icc 0 1) (f : Audit V → ℝ) :
Integrable f (auditLaw V q) := by
let := auditLaw_probability (V := V) q hq
exact Integrable.of_finite
The mean of a single numeric audit mark is its retention probability. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: integral_audit_treatment
lemma integral_audit_treatment (q : ℝ) (hq : q ∈ Icc 0 1) (e : OffDiag V) :
(∫ w, treatment (w e) ∂auditLaw V q) = q := by
let := bernoulliLaw_probability q hq
have hm := (measurePreserving_eval (fun _ : OffDiag V => bernoulliLaw q) e).map_eq
have he := integral_map (μ := auditLaw V q)
(φ := fun w => w e) ((measurable_pi_apply e).aemeasurable) (f := treatment) (by fun_prop)
rw [show (auditLaw V q).map (fun w => w e) = bernoulliLaw q from hm] at he
rw [← he]
rw [show bernoulliLaw q = bernoulliBool q by
unfold bernoulliLaw bernoulliBool
exact add_comm _ _]
rw [bernoulliBool_integral hq.1 hq.2]
simp [treatment]
theorem integral_recorded_sign_sum
Conditional on assignment, the retained-neighbor sign sum has mean q times the true sum. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ w, ∑ j ∈ recordedNbhd (recordOf θ (z, w)).1 i, signOf (z j) ∂auditLaw V q)
= q * ∑ j ∈ inNbhd θ i, signOf (z j)
Proof (Lean source)
-- @node: integral_recorded_sign_sum
lemma integral_recorded_sign_sum (θ : Schedule V) (q : ℝ) (hq : q ∈ Icc 0 1)
(z : Assign V) (i : V) :
(∫ w, ∑ j ∈ recordedNbhd (recordOf θ (z, w)).1 i, signOf (z j)
∂auditLaw V q) = q * ∑ j ∈ inNbhd θ i, signOf (z j) := by
simp only [recordedNbhd]
simp_rw [Finset.sum_filter]
dsimp only [recordOf]
rw [integral_finsetSum _ (fun _ _ => auditLaw_integrable q hq _)]
have ht (j : V) :
(∫ w, if (∃ hji : j ≠ i, (decide (θ.edge j i) && w ⟨(j, i), hji⟩) = true)
then signOf (z j) else 0 ∂auditLaw V q) =
if θ.edge j i then q * signOf (z j) else 0 := by
by_cases hj : θ.edge j i
· have hji : j ≠ i := by intro he; subst j; exact θ.irrefl i hj
have he : (fun w : Audit V => if (∃ hji : j ≠ i,
(decide (θ.edge j i) && w ⟨(j, i), hji⟩) = true) then signOf (z j) else 0) =
fun w => treatment (w ⟨(j, i), hji⟩) * signOf (z j) := by
funext w
cases hw : w ⟨(j, i), hji⟩ <;> simp [hj, hji, hw, treatment]
rw [he, integral_mul_const, integral_audit_treatment q hq, if_pos hj]
· simp [hj]
trans ∑ j : V, if θ.edge j i then q * signOf (z j) else 0
· exact Finset.sum_congr rfl (fun j _ => ht j)
· rw [inNbhd, Finset.sum_filter, Finset.mul_sum]
apply Finset.sum_congr rfl
intro j _
split <;> simp_all
theorem integral_auditScore_given_assignment
Inverse inclusion weighting recovers the oracle score's conditional mean. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: integral_auditScore_given_assignment
lemma integral_auditScore_given_assignment (θ : Schedule V) (q : ℝ)
(hq : q ∈ Icc 0 1) (hq0 : 0 < q) (z : Assign V) :
(∫ w, auditScore q (recordOf θ (z, w)) ∂auditLaw V q) = oracleScore θ z := by
let := auditLaw_probability (V := V) q hq
simp only [auditScore, recordOf]
rw [integral_const_mul, integral_finsetSum _ (fun _ _ => auditLaw_integrable q hq _)]
unfold oracleScore
congr 1
apply Finset.sum_congr rfl
intro i _
rw [integral_const_mul, integral_add (auditLaw_integrable q hq _)
(auditLaw_integrable q hq _), integral_const, probReal_univ, one_smul,
integral_const_mul]
have he := integral_recorded_sign_sum θ q hq z i
change (∫ w, ∑ j ∈ recordedNbhd
(fun e => decide (θ.edge e.1.1 e.1.2) && w e) i, signOf (z j) ∂auditLaw V q) =
q * ∑ j ∈ inNbhd θ i, signOf (z j) at he
rw [he, ← mul_assoc, inv_mul_cancel₀ (ne_of_gt hq0), one_mul]
theorem integral_auditScore_eq_tte
The complete-record inverse-inclusion score is unbiased under the three design atoms. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: integral_auditScore_eq_tte
lemma integral_auditScore_eq_tte (D : Measure (Assign V × Audit V)) (θ : Schedule V)
(q : ℝ) (ha : AssignmentLaw D) (hw : AuditLaw D q)
(hi : DesignIndependent D) (hq : 0 < q) :
(∫ ω, auditScore q (recordOf θ ω) ∂D) = tte θ := by
rw [design_eq_thinnedDesign D q ha hw hi]
let := halfBernoulli_probability (V := V)
let := auditLaw_probability (V := V) q ⟨hw.1, hw.2.1⟩
rw [thinnedDesign, integral_prod _ (show Integrable _ _ from Integrable.of_finite)]
simp_rw [integral_auditScore_given_assignment θ q ⟨hw.1, hw.2.1⟩ hq]
exact integral_oracleScore_eq_tte θ
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.SuppliedBlockRisk 10 declarations The translated full-record laws at the bounded testing amplitude give the unrestricted randomized minimax converse through the two-prior testing inequality.
Supplied-block risk assembly
The translated full-record laws at the bounded testing amplitude give the unrestricted randomized minimax converse through the two-prior testing inequality.
The roadmap amplitude, expressed using the square root of the risk scale.
theorem completeBlockLowerAmplitude_properties
The testing amplitude is positive, admissible, and has exactly the required squared scale. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
hn :
0 < n
conclusion
conclusion 1
0 < completeBlockLowerAmplitude n d
conclusion 2
completeBlockLowerAmplitude n d ∈ Icc 0 (1 / 4)
conclusion 3
(completeBlockLowerAmplitude n d) ^ 2
Proof (Lean source)
-- @node: completeBlockLowerAmplitude_properties
lemma completeBlockLowerAmplitude_properties (n d : ℕ) (hn : 0 < n) :
0 < completeBlockLowerAmplitude n d ∧
completeBlockLowerAmplitude n d ∈ Icc 0 (1 / 4) ∧
(completeBlockLowerAmplitude n d) ^ 2 =
(10000 * pi ^ 2)⁻¹ * min 1 (((d : ℝ) + 1) ^ 2 / n) := by
have hnR : (0 : ℝ) < n := by exact_mod_cast hn
have hk : (0 : ℝ) < (d : ℝ) + 1 := by positivity
have hs : 0 < min 1 (((d : ℝ) + 1) ^ 2 / n) := by positivity
have hκ : 0 < (100 * pi)⁻¹ := by positivity
have hroot : sqrt (min 1 (((d : ℝ) + 1) ^ 2 / n)) ≤ 1 := by
exact (Real.sqrt_le_one).mpr (min_le_left _ _)
have hκquarter : (100 * pi)⁻¹ ≤ (1 / 4 : ℝ) := by
rw [inv_eq_one_div]
apply (div_le_iff₀ (by positivity : 0 < 100 * pi)).mpr
nlinarith [Real.two_le_pi]
refine ⟨mul_pos hκ (Real.sqrt_pos.2 hs), ⟨(mul_pos hκ (Real.sqrt_pos.2 hs)).le, ?_⟩, ?_⟩
· exact (mul_le_of_le_one_right hκ.le hroot).trans hκquarter
· unfold completeBlockLowerAmplitude
rw [mul_pow, Real.sq_sqrt hs.le]
congr 1
field_simp
ring
theorem completeBlock_testing_distance
The genuine original-record distance at the roadmap amplitude is at most one fiftieth. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
hn :
0 < n
ha :
conclusion
tvDist (mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d true (completeBlockLowerAmplitude n d))) (mixtureLaw D (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d false (completeBlockLowerAmplitude n d)))
≤ 1 / 50
Proof (Lean source)
-- @node: completeBlock_testing_distance
lemma completeBlock_testing_distance (n d : ℕ) (hn : 0 < n)
(D : Measure (Assign (Fin n) × Audit (Fin n))) [IsProbabilityMeasure D]
(ha : AssignmentLaw D) :
tvDist
(mixtureLaw D (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true (completeBlockLowerAmplitude n d)))
(mixtureLaw D (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d false (completeBlockLowerAmplitude n d))) ≤ 1 / 50 := by
have hnR : (0 : ℝ) < n := by exact_mod_cast hn
have hk : (0 : ℝ) < (d : ℝ) + 1 := by positivity
have hsq := (completeBlockLowerAmplitude_properties n d hn).2.2
refine (completeBlock_mixture_tv_population_le n d _ D ha).trans ?_
apply (Real.sqrt_le_iff).mpr
refine ⟨by norm_num, ?_⟩
rw [hsq]
have hmin := min_le_right (1 : ℝ) (((d : ℝ) + 1) ^ 2 / n)
have hp : 0 < pi ^ 2 := sq_pos_of_pos Real.pi_pos
have he : 4 * pi ^ 2 * ((10000 * pi ^ 2)⁻¹ *
min 1 (((d : ℝ) + 1) ^ 2 / n)) * n / ((d : ℝ) + 1) ^ 2 =
min 1 (((d : ℝ) + 1) ^ 2 / n) * n / (2500 * ((d : ℝ) + 1) ^ 2) := by
field_simp
ring
rw [he]
apply (div_le_iff₀ (by positivity : 0 < 2500 * ((d : ℝ) + 1) ^ 2)).mpr
have hm := (le_div_iff₀ hnR).mp hmin
nlinarith
theorem completeBlock_risk_lower_assembly
Support, full-record testing distance, and coverage yield the supplied-block risk scale. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: completeBlock_risk_lower_assembly
lemma completeBlock_risk_lower_assembly (n d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) (hq : q ∈ Icc 0 1) :
ofReal ((160000 * pi ^ 2)⁻¹ * min 1 (((d : ℝ) + 1) ^ 2 / n)) ≤
minimaxRisk (thinnedDesign (Fin n) q) d := by
let h := completeBlockLowerAmplitude n d
let ρ : ℝ := (((n / (d + 1)) * (d + 1) : ℕ) : ℝ) / n
have hprop := completeBlockLowerAmplitude_properties n d (by omega)
have hcov : (1 / 2 : ℝ) ≤ ρ := completeBlock_coverage_ge_half n d hn hdu
have hh : 0 < h := hprop.1
have hΔ : 0 < ρ * h := mul_pos (by linarith) hh
let := blockBaselineLaw_probability (n / (d + 1))
have hs (σ : Bool) : ∀ᵐ U ∂(blockBaselineLaw (n / (d + 1))),
ScheduleClass (completeBlockSchedule n d σ h U) d ∧
tte (completeBlockSchedule n d σ h U) = signOf σ * (ρ * h) := by
filter_upwards [completeBlock_support n d σ h hprop.2.1] with U hU
refine ⟨hU.1, hU.2.trans ?_⟩
dsimp [ρ]
ring
let : IsProbabilityMeasure (thinnedDesign (Fin n) q) := thinnedDesign_probabilityDesign q hq
have ht := full_record_two_prior_risk n d q hn hd hdu hq
(blockBaselineLaw (n / (d + 1))) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true h) (completeBlockSchedule n d false h)
(completeBlock_record_measurable n d true h) (completeBlock_record_measurable n d false h)
(ρ * h) hΔ (by simpa [signOf] using hs true) (by simpa [signOf] using hs false)
have htv := completeBlock_testing_distance n d (by omega) (thinnedDesign (Fin n) q)
(thinnedDesign_assignment q hq)
refine le_trans (ENNReal.ofReal_le_ofReal ?_) ht
have hsep : h / 2 ≤ ρ * h := by nlinarith
have hsqsep : h ^ 2 / 4 ≤ (ρ * h) ^ 2 := by nlinarith [sq_nonneg (ρ * h - h / 2)]
have hscale : (160000 * pi ^ 2)⁻¹ * min 1 (((d : ℝ) + 1) ^ 2 / n) =
h ^ 2 / 16 := by
rw [show h ^ 2 = _ from hprop.2.2]
field_simp
ring
rw [hscale]
change tvDist
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true h))
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d false h)) ≤ 1 / 50 at htv
nlinarith [mul_nonneg (sq_nonneg (ρ * h))
(show 0 ≤ 1 / 50 - tvDist
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true h))
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d false h)) from sub_nonneg.mpr htv)]
def hardcodeSuppliedEstimator
Hard-coding a graph makes a supplied-graph estimator a measurable original-record rule.
Definition (Lean source)
parameters
def
given by
⟨fun x
theorem hardcodeSuppliedEstimator_sqLoss
On a schedule with the hard-coded graph, the original and supplied squared risks agree. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
lemma hardcodeSuppliedEstimator_sqLoss
(D : Measure (Assign V × Audit V)) (θ : Schedule V)
(g : OffDiag V → Bool) (T : SuppliedEstimator V)
(hg : suppliedGraphOf θ = g) :
sqLoss D θ (hardcodeSuppliedEstimator g T) = suppliedSqLoss D θ T := by
have hr : Measurable (recordOf θ) := measurable_of_finite _
have ho : Measurable (fun ω : (Assign V × Audit V) × ℝ => (recordOf θ ω.1, ω.2)) :=
(hr.comp measurable_fst).prodMk measurable_snd
have hs : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(suppliedRecordOf θ ω.1, ω.2)) :=
((hr.comp measurable_fst).prodMk measurable_const).prodMk measurable_snd
have hf : Measurable (fun x : Record V × ℝ =>
ofReal (((hardcodeSuppliedEstimator g T).1 x - tte θ) ^ 2)) :=
(((hardcodeSuppliedEstimator g T).2.sub measurable_const).pow_const 2).ennreal_ofReal
have hfG : Measurable (fun x : SuppliedRecord V × ℝ =>
ofReal ((T.1 x - tte θ) ^ 2)) :=
((T.2.sub measurable_const).pow_const 2).ennreal_ofReal
unfold sqLoss recordLaw suppliedSqLoss suppliedRecordLaw
rw [lintegral_map hf ho, lintegral_map hfG hs]
simp only [hardcodeSuppliedEstimator, suppliedRecordOf, hg]
theorem mixture_seeded_risk_le_supplied_worst
A constant-target, fixed-graph prior's hard-coded risks are bounded by supplied worst risk. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
Ξ :
Type u_2
D :
π :
s :
Ξ → Schedule V
hm :
d :
ℕ
a :
ℝ
hs :
∀ᵐ u ∂π, ScheduleClass (s u) d ∧ tte (s u) = a
hg :
∀ u, suppliedGraphOf (s u) = g
T :
conclusion
≤ suppliedWorstRisk D d T
Proof (Lean source)
lemma mixture_seeded_risk_le_supplied_worst
(D : Measure (Assign V × Audit V)) [IsProbabilityMeasure D]
(π : Measure Ξ) [IsProbabilityMeasure π] (s : Ξ → Schedule V)
(hm : Measurable (fun x : Ξ × (Assign V × Audit V) => recordOf (s x.1) x.2))
(d : ℕ) (a : ℝ) (g : OffDiag V → Bool)
(hs : ∀ᵐ u ∂π, ScheduleClass (s u) d ∧ tte (s u) = a)
(hg : ∀ u, suppliedGraphOf (s u) = g) (T : SuppliedEstimator V) :
(∫⁻ x, ofReal (((hardcodeSuppliedEstimator g T).1 x - a) ^ 2)
∂((mixtureLaw D π s).prod seedLaw)) ≤ suppliedWorstRisk D d T := by
let : IsProbabilityMeasure seedLaw := ⟨by simp [seedLaw, Real.volume_Icc]⟩
let T₀ := hardcodeSuppliedEstimator g T
have hf : Measurable (fun x : Record V × ℝ => ofReal ((T₀.1 x - a) ^ 2)) :=
((T₀.2.sub measurable_const).pow_const 2).ennreal_ofReal
rw [lintegral_prod _ hf.aemeasurable]
unfold mixtureLaw
rw [Measure.lintegral_bind (measurable_record_components D s hm).aemeasurable
hf.lintegral_prod_right'.aemeasurable]
calc
_ ≤ ∫⁻ _ : Ξ, suppliedWorstRisk D d T ∂π := by
apply lintegral_mono_ae
filter_upwards [hs] with u hu
have hr : Measurable (recordOf (s u)) :=
hm.comp (measurable_const.prodMk measurable_id)
rw [lintegral_map hf.lintegral_prod_right' hr]
have heq : (∫⁻ ω, ∫⁻ v, ofReal ((T₀.1 (recordOf (s u) ω, v) - a) ^ 2)
∂seedLaw ∂D) = sqLoss D (s u) T₀ := by
unfold sqLoss recordLaw
rw [hu.2]
have hmseed : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(recordOf (s u) ω.1, ω.2)) :=
(hr.comp measurable_fst).prodMk measurable_snd
rw [lintegral_map hf hmseed]
exact (lintegral_prod _ (hf.comp hmseed).aemeasurable).symm
rw [heq, hardcodeSuppliedEstimator_sqLoss D (s u) g T (hg u)]
exact le_iSup
(fun θ : {θ : Schedule V // ScheduleClass θ d} => suppliedSqLoss D θ.1 T)
⟨s u, hu.1⟩
_ = suppliedWorstRisk D d T := by simp
theorem completeBlock_supplied_two_prior
The common deterministic complete-block graph allows testing against supplied worst risk. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n d :
ℕ
q h Δ :
ℝ
hq :
q ∈ Icc 0 1
hΔ :
0 < Δ
hs :
∀ σ : Bool,
∀ᵐ U ∂(blockBaselineLaw (n / (d + 1))), ScheduleClass (completeBlockSchedule n d σ h U) d ∧ tte (completeBlockSchedule n d σ h U)
= signOf σ * Δ
conclusion
ofReal (Δ ^ 2 / 2 * (1 - tvDist (mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d true h)) (mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1))) (completeBlockSchedule n d false h))))
Proof (Lean source)
lemma completeBlock_supplied_two_prior
(n d : ℕ) (q h Δ : ℝ) (hq : q ∈ Icc 0 1) (hΔ : 0 < Δ)
(hs : ∀ σ : Bool, ∀ᵐ U ∂(blockBaselineLaw (n / (d + 1))),
ScheduleClass (completeBlockSchedule n d σ h U) d ∧
tte (completeBlockSchedule n d σ h U) = signOf σ * Δ) :
ofReal (Δ ^ 2 / 2 * (1 - tvDist
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true h))
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d false h)))) ≤
suppliedMinimaxRisk (thinnedDesign (Fin n) q) d := by
let D := thinnedDesign (Fin n) q
let π := blockBaselineLaw (n / (d + 1))
let g := suppliedGraphOf (completeBlockSchedule n d true 0 (fun _ => 0))
let P := mixtureLaw D π (completeBlockSchedule n d true h)
let M := mixtureLaw D π (completeBlockSchedule n d false h)
let : IsProbabilityMeasure D := thinnedDesign_probabilityDesign q hq
let : IsProbabilityMeasure π := blockBaselineLaw_probability _
let : IsProbabilityMeasure seedLaw := ⟨by simp [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure P :=
mixtureLaw_probability D π _ (completeBlock_record_measurable n d true h)
let : IsProbabilityMeasure M :=
mixtureLaw_probability D π _ (completeBlock_record_measurable n d false h)
have hg (σ : Bool) (U : Fin (n / (d + 1)) → ℝ) :
suppliedGraphOf (completeBlockSchedule n d σ h U) = g := rfl
have htv : tvDist (P.prod seedLaw) (M.prod seedLaw) =
tvDist P M := by
simpa using tvDist_compProd_eq P M (const _ seedLaw)
change _ ≤ ⨅ T : SuppliedEstimator (Fin n), suppliedWorstRisk D d T
apply le_iInf
intro T
have hP := mixture_seeded_risk_le_supplied_worst D π _
(completeBlock_record_measurable n d true h) d Δ g
(by simpa [signOf] using hs true) (hg true) T
have hM := mixture_seeded_risk_le_supplied_worst D π _
(completeBlock_record_measurable n d false h) d (-Δ) g
(by simpa [signOf] using hs false) (hg false) T
have htest := two_prior_squared_testing (P.prod seedLaw) (M.prod seedLaw)
(hardcodeSuppliedEstimator g T).1 (hardcodeSuppliedEstimator g T).2 Δ hΔ
(suppliedWorstRisk D d T) hP hM
rw [htv] at htest
exact htest
theorem completeBlock_supplied_risk_lower_assembly
The unchanged support, distance, and algebra certify supplied-graph minimax risk. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma completeBlock_supplied_risk_lower_assembly (n d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) (hq : q ∈ Icc 0 1) :
ofReal ((160000 * pi ^ 2)⁻¹ * min 1 (((d : ℝ) + 1) ^ 2 / n)) ≤
suppliedMinimaxRisk (thinnedDesign (Fin n) q) d := by
let h := completeBlockLowerAmplitude n d
let ρ : ℝ := (((n / (d + 1)) * (d + 1) : ℕ) : ℝ) / n
have hprop := completeBlockLowerAmplitude_properties n d (by omega)
have hcov : (1 / 2 : ℝ) ≤ ρ := completeBlock_coverage_ge_half n d hn hdu
have hh : 0 < h := hprop.1
have hΔ : 0 < ρ * h := mul_pos (by linarith) hh
let := blockBaselineLaw_probability (n / (d + 1))
have hs (σ : Bool) : ∀ᵐ U ∂(blockBaselineLaw (n / (d + 1))),
ScheduleClass (completeBlockSchedule n d σ h U) d ∧
tte (completeBlockSchedule n d σ h U) = signOf σ * (ρ * h) := by
filter_upwards [completeBlock_support n d σ h hprop.2.1] with U hU
refine ⟨hU.1, hU.2.trans ?_⟩
dsimp [ρ]
ring
let : IsProbabilityMeasure (thinnedDesign (Fin n) q) := thinnedDesign_probabilityDesign q hq
have ht := completeBlock_supplied_two_prior n d q h (ρ * h) hq hΔ hs
have htv := completeBlock_testing_distance n d (by omega) (thinnedDesign (Fin n) q)
(thinnedDesign_assignment q hq)
refine le_trans (ENNReal.ofReal_le_ofReal ?_) ht
have hsep : h / 2 ≤ ρ * h := by nlinarith
have hsqsep : h ^ 2 / 4 ≤ (ρ * h) ^ 2 := by nlinarith [sq_nonneg (ρ * h - h / 2)]
have hscale : (160000 * pi ^ 2)⁻¹ * min 1 (((d : ℝ) + 1) ^ 2 / n) =
h ^ 2 / 16 := by
rw [show h ^ 2 = _ from hprop.2.2]
field_simp
ring
rw [hscale]
change tvDist
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true h))
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d false h)) ≤ 1 / 50 at htv
nlinarith [mul_nonneg (sq_nonneg (ρ * h))
(show 0 ≤ 1 / 50 - tvDist
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d true h))
(mixtureLaw (thinnedDesign (Fin n) q) (blockBaselineLaw (n / (d + 1)))
(completeBlockSchedule n d false h)) from sub_nonneg.mpr htv)]
theorem supplied_block_record_lower
The same lower bound holds for the entire original record and for that record with the true graph additionally supplied. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
≤ (if supplied then suppliedMinimaxRisk D d else minimaxRisk D d)
Proof (Lean source)
-- @node: lem:supplied-block-record-lower
lemma supplied_block_record_lower (n d : ℕ) (q : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D)
(supplied : Bool := false) :
ofReal ((160000 * pi ^ 2)⁻¹ * min 1 (((d : ℝ) + 1) ^ 2 / n)) ≤
(if supplied then suppliedMinimaxRisk D d else minimaxRisk D d) := by
rw [design_eq_thinnedDesign D q ha hw hi]
cases supplied
· exact completeBlock_risk_lower_assembly n d q hn hd hdu hq
· exact completeBlock_supplied_risk_lower_assembly n d q hn hd hdu hq
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.SymmetricConditionalLikelihood 4 declarations The uniform original-label allocation average is replaced by its exact binomial-weighted symmetric polynomial.
Conditional likelihood with symmetric hidden polynomials
The uniform original-label allocation average is replaced by its exact binomial-weighted symmetric polynomial. The full graph and assignment marginal and all reference-zero endpoints remain those of the actual experiment.
def symmetricHiddenWalshDensity
The posterior response density using exact binomial-weighted symmetric hidden polynomials, extended by zero at incompatible graphs and reference-density zeros.
Definition (Lean source)
parameters
def
symmetricHiddenWalshDensity n B d hfit h hz y :
ℝ
given by
if hs : Nonempty (CompatiblePartition n B d hz.1) then if ∀ ℓ, 0 < refDensity d h (y ℓ) then (∏ ℓ, refDensity d h (y ℓ)) * (∑ J ∈ piFinset (fun ℓ : Fin B => (revealedSources n B d hz.1 ℓ).powerset), (∏ ℓ, ∏ j ∈ J ℓ, signOf (hz.2 j)) * ∑ k ∈ piFinset (fun ℓ : Fin B => range (capacity n B d hz.1 ℓ + 1)), (∏ ℓ, walshCoeff d h ((J ℓ).card + k ℓ) (y ℓ)) * ((∏ ℓ, (choose (capacity n B d hz.1 ℓ) (k ℓ) : ℝ)) * symmetricSignPoly (undiscovered n B d hz.1) (∑ ℓ, k ℓ) (fun i => hz.2 (hiddenSourceLabelEmbedding n B d hfit hz.1 ((hiddenSourceEnumeration n B d hfit hz.1 hs.some).symm i))))) else 0 else 0
theorem symmetricHiddenWalshDensity_eq_blockDensity
The symmetric hidden-subset expansion gives the full actual response density, including every zero-density endpoint. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: symmetricHiddenWalshDensity_eq_blockDensity
lemma symmetricHiddenWalshDensity_eq_blockDensity (n B d : ℕ) (hd : 1 ≤ d)
(hfit : 2 * (B * d) ≤ n) (h : ℝ)
(hz : (OffDiag (Fin n) → Bool) × Assign (Fin n))
(hH : ValidRetainedGraph n B d hz.1) (y : Fin B → ℝ) :
symmetricHiddenWalshDensity n B d hfit h hz y =
blockDensity n B d true h hz.1 hz.2 y := by
have hs : Nonempty (CompatiblePartition n B d hz.1) := by
obtain ⟨_, s, hs⟩ := hH
exact ⟨⟨s, hs⟩⟩
rw [symmetricHiddenWalshDensity, dif_pos hs]
split_ifs with hy
· have he := blockDensity_symmetric_hidden_selection_expansion
n B d hd hfit h hz.1 hz.2 hs.some y hy
rw [← he]
exact mul_div_cancel₀ _ (ne_of_gt (Finset.prod_pos (fun ℓ _ => hy ℓ)))
· obtain ⟨ℓ, hℓ⟩ := not_forall.mp hy
have hz0 : refDensity d h (y ℓ) = 0 :=
le_antisymm (not_lt.mp hℓ) (refDensity_nonneg d h (y ℓ))
exact (blockDensity_zero_of_reference_zero
n B d hd hfit h hz.1 hz.2 hs.some y ℓ hz0).symm
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.SymmetricSignPolys 3 declarations Symmetric hidden-sign polynomials
Symmetric hidden-sign polynomials
def symmetricSignPoly
Symmetric sign polynomial on a Boolean slice.
Products of independent centered signs form an orthonormal family. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
S T :
Finset V
conclusion
(∫ z, (∏ j ∈ S, signOf (z j)) * (∏ j ∈ T, signOf (z j)) ∂(halfBernoulli V))
= if S = T then 1 else 0
Proof (Lean source)
-- @node: signProduct_orthogonality
lemma signProduct_orthogonality {V : Type*} [Fintype V] [DecidableEq V]
(S T : Finset V) :
(∫ z, (∏ j ∈ S, signOf (z j)) * (∏ j ∈ T, signOf (z j))
∂(halfBernoulli V)) = if S = T then 1 else 0 := by
open Causalean.Experimentation.DesignBased in
have hscale (R : Finset V) (z : Assign V) :
(∏ j ∈ R, signOf (z j)) =
(2 : ℝ) ^ R.card * centeredMonomial (fun _ => (1 / 2 : ℝ)) R z := by
unfold centeredMonomial
rw [← Finset.prod_const, ← Finset.prod_mul_distrib]
apply Finset.prod_congr rfl
intro j hj
cases h : z j <;> norm_num [signOf, treatInd, h]
rw [integral_halfBernoulli_eq_bernoulliDesign_E]
simp_rw [hscale]
have hfun : (fun z : Assign V =>
((2 : ℝ) ^ S.card * centeredMonomial
(fun _ => (1 / 2 : ℝ)) S z) *
((2 : ℝ) ^ T.card * centeredMonomial
(fun _ => (1 / 2 : ℝ)) T z)) =
(fun z => ((2 : ℝ) ^ S.card * 2 ^ T.card) *
(centeredMonomial
(fun _ => (1 / 2 : ℝ)) S z *
centeredMonomial
(fun _ => (1 / 2 : ℝ)) T z)) := by funext z; ring
rw [hfun, E_const_mul,
bernoulliDesign_E_centeredMonomial_mul]
by_cases hST : S = T
· subst T
simp only [Finset.prod_const]
norm_num
rw [← mul_pow, ← mul_pow]
norm_num
· simp [hST]
When both polynomial degrees do not exceed the ambient dimension, their Bernoulli inner product is the reciprocal binomial coefficient when the degrees agree and zero otherwise.
Formal statement
hypotheses
M k t :
ℕ
hk :
k ≤ M
ht :
t ≤ M
conclusion
= if k = t then (M.choose k : ℝ)⁻¹ else 0
Proof (Lean source)
-- @node: symmetricSignPoly_orthogonality
lemma symmetricSignPoly_orthogonality (M k t : ℕ) (hk : k ≤ M) (ht : t ≤ M) :
(∫ z, symmetricSignPoly M k z * symmetricSignPoly M t z ∂(halfBernoulli (Fin M))) =
if k = t then (M.choose k : ℝ)⁻¹ else 0 := by
rw [integral_halfBernoulli_eq_bernoulliDesign_E]
unfold symmetricSignPoly
have hfun : (fun z : Assign (Fin M) =>
((M.choose k : ℝ)⁻¹ * ∑ E ∈ Finset.univ.powersetCard k, ∏ j ∈ E, signOf (z j)) *
((M.choose t : ℝ)⁻¹ * ∑ F ∈ Finset.univ.powersetCard t, ∏ j ∈ F, signOf (z j))) =
(fun z => ((M.choose k : ℝ)⁻¹ * (M.choose t : ℝ)⁻¹) *
∑ E ∈ Finset.univ.powersetCard k, ∑ F ∈ Finset.univ.powersetCard t,
(∏ j ∈ E, signOf (z j)) * (∏ j ∈ F, signOf (z j))) := by
funext z
simp only [← Finset.mul_sum, ← Finset.sum_mul]
ring
rw [hfun, E_const_mul]
simp only [E_sum]
simp_rw [← integral_halfBernoulli_eq_bernoulliDesign_E, signProduct_orthogonality]
by_cases hkt : k = t
· subst t
have hnonzero : (M.choose k : ℝ) ≠ 0 := by
exact_mod_cast (Nat.ne_of_gt (Nat.choose_pos hk))
simp only [ite_true]
have hsum : (∑ E ∈ (Finset.univ : Finset (Fin M)).powersetCard k,
∑ F ∈ (Finset.univ : Finset (Fin M)).powersetCard k,
if E = F then (1 : ℝ) else 0) = M.choose k := by
calc
_ = ∑ E ∈ (Finset.univ : Finset (Fin M)).powersetCard k, (1 : ℝ) := by
apply Finset.sum_congr rfl
intro E hE
simp [hE]
_ = M.choose k := by simp [Finset.card_powersetCard]
rw [hsum]
field_simp
· rw [if_neg hkt]
have hsum : (∑ E ∈ (Finset.univ : Finset (Fin M)).powersetCard k,
∑ F ∈ (Finset.univ : Finset (Fin M)).powersetCard t,
if E = F then (1 : ℝ) else 0) = 0 := by
apply Finset.sum_eq_zero
intro E hE
apply Finset.sum_eq_zero
intro F hF
have hEF : E ≠ F := by
intro heq
have hEk := (Finset.mem_powersetCard.mp hE).2
have hFt := (Finset.mem_powersetCard.mp hF).2
exact hkt (hEk.symm.trans (heq ▸ hFt))
exact if_neg hEF
rw [hsum, mul_zero]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.TreatedCompletionCounting 8 declarations Splitting the original hidden labels into treated and untreated subtypes gives exact multiplicities of completed row treatment counts.
Treated-label fibers of compatible completions
Splitting the original hidden labels into treated and untreated subtypes gives exact multiplicities of completed row treatment counts.
Row counts on the treated original labels of a finite word.
Definition (Lean source)
parameters
def
treatedRowCount p g ℓ :
ℕ
given by
(Finset.univ.filter (fun j : {j // p j} => g j.1 = ℓ)).card
theorem rowCount_split_treatment
Splitting a row into treated and untreated labels preserves its full cardinality. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
treatedRowCount p g ℓ + treatedRowCount (fun j => ¬ p j) g ℓ
= (Finset.univ.filter (fun j => g j = ℓ)).card
Proof (Lean source)
-- @node: rowCount_split_treatment
lemma rowCount_split_treatment {ι : Type*} [Fintype ι] (p : ι → Prop) [DecidablePred p]
(g : ι → Fin B) (ℓ : Fin B) :
treatedRowCount p g ℓ + treatedRowCount (fun j => ¬ p j) g ℓ =
(Finset.univ.filter (fun j => g j = ℓ)).card := by
let e : {j : ι // g j = ℓ} ≃
{j : {j // p j} // g j.1 = ℓ} ⊕ {j : {j // ¬ p j} // g j.1 = ℓ} := {
toFun := fun j => if hj : p j.1 then inl ⟨⟨j.1,hj⟩,j.2⟩
else inr ⟨⟨j.1,hj⟩,j.2⟩
invFun := fun j => elim (fun a => ⟨a.1.1,a.2⟩) (fun a => ⟨a.1.1,a.2⟩) j
left_inv := by intro j; dsimp only; split_ifs <;> rfl
right_inv := by
intro j
cases j with
| inl j => simp [j.1.2]
| inr j => simp [j.1.2] }
simpa only [Fintype.card_sum, Fintype.card_subtype, treatedRowCount]
using (Fintype.card_congr e).symm
def treatedRowFiberEquiv
A fixed treated-count fiber splits bijectively into words on treated and untreated original labels. No label permutations are identified or discarded.
Definition (Lean source)
parameters
def
treatedRowFiberEquiv p u k hk :
{g : FiniteRowWord ι B u // ∀ ℓ, treatedRowCount p g.1 ℓ = k ℓ} ≃ FiniteRowWord {j // p j} B k × FiniteRowWord {j // ¬ p j} B (fun ℓ => u ℓ - k ℓ)
given by
clause 1
toFun g := (⟨fun j => g.1.1 j.1, g.2⟩, ⟨fun j => g.1.1 j.1, fun ℓ => by have hc := rowCount_split_treatment p g.1.1 ℓ rw [g.2 ℓ, g.1.2 ℓ] at hc change treatedRowCount (fun j => ¬ p j) g.1.1 ℓ = u ℓ - k ℓ omega⟩)
clause 2
invFun g := let f := (Equiv.piEquivPiSubtypeProd p (fun _ => Fin B)).symm (g.1.1,g.2.1) have ht (ℓ) : treatedRowCount p f ℓ
= k ℓ := by unfold treatedRowCount have he (j : {j // p j}) : f j.1 = g.1.1 j := dif_pos j.2 simpa only [he] using g.1.2 ℓ have hn (ℓ) : treatedRowCount (fun j => ¬ p j) f ℓ = u ℓ
- k ℓ := by unfold treatedRowCount have he (j : {j // ¬ p j}) : f j.1
= g.2.1 j := dif_neg j.2 simpa only [he] using g.2.2 ℓ ⟨⟨f, fun ℓ => by have hc := rowCount_split_treatment p f ℓ rw [ht ℓ, hn ℓ] at hc have := hk ℓ omega⟩, ht⟩
clause 3
theorem treatedRowCount_sum
Counts across rows exhaust the treated original labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
∑ ℓ, treatedRowCount p g ℓ = card {j // p j}
Proof (Lean source)
-- @node: treatedRowCount_sum
lemma treatedRowCount_sum {ι : Type*} [Fintype ι] (p : ι → Prop) [DecidablePred p]
(g : ι → Fin B) : ∑ ℓ, treatedRowCount p g ℓ = card {j // p j} := by
have hc := Finset.sum_card_fiberwise_eq_card_filter
(Finset.univ : Finset {j // p j}) (Finset.univ : Finset (Fin B))
(fun j => g j.1)
simpa [treatedRowCount] using hc
theorem treatedRowFiber_card_mul_factorials
The fixed treated-count fiber has the product of the two multinomial counts. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
ι :
p :
ι → Prop
u k :
Fin B → ℕ
hk :
∀ ℓ, k ℓ ≤ u ℓ
hu :
∑ ℓ, u ℓ = card ι
hkt :
∑ ℓ, k ℓ = card {j // p j}
conclusion
card {g : FiniteRowWord ι B u // ∀ ℓ, treatedRowCount p g.1 ℓ = k ℓ} * ((∏ ℓ, (k ℓ).factorial) * (∏ ℓ, (u ℓ - k ℓ).factorial))
Proof (Lean source)
-- @node: treatedRowFiber_card_mul_factorials
lemma treatedRowFiber_card_mul_factorials {ι : Type*} [Fintype ι] [DecidableEq ι]
(p : ι → Prop) [DecidablePred p] (u k : Fin B → ℕ) (hk : ∀ ℓ, k ℓ ≤ u ℓ)
(hu : ∑ ℓ, u ℓ = card ι)
(hkt : ∑ ℓ, k ℓ = card {j // p j}) :
card {g : FiniteRowWord ι B u // ∀ ℓ, treatedRowCount p g.1 ℓ = k ℓ} *
((∏ ℓ, (k ℓ).factorial) * (∏ ℓ, (u ℓ - k ℓ).factorial)) =
(card {j // p j}).factorial * (card {j // ¬ p j}).factorial := by
have hc : card {j // p j} + card {j // ¬ p j} = card ι := by
simpa only [Fintype.card_subtype, Finset.card_univ] using
Finset.card_filter_add_card_filter_not (s := (Finset.univ : Finset ι)) p
have hkn : ∑ ℓ, (u ℓ - k ℓ) = card {j // ¬ p j} := by
have he : (∑ ℓ, k ℓ) + (∑ ℓ, (u ℓ - k ℓ)) = ∑ ℓ, u ℓ := by
rw [← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro ℓ _
have := hk ℓ
omega
omega
rw [Fintype.card_congr (treatedRowFiberEquiv p u k hk), Fintype.card_prod]
calc
_ = (card (FiniteRowWord {j // p j} B k) * ∏ ℓ, (k ℓ).factorial) *
(card (FiniteRowWord {j // ¬ p j} B (fun ℓ => u ℓ - k ℓ)) *
∏ ℓ, (u ℓ - k ℓ).factorial) := by ac_rfl
_ = _ := by rw [finiteRowWord_card_mul_factorial B k hkt,
finiteRowWord_card_mul_factorial B _ hkn]
theorem treatedRowFiber_hypergeometric_weight
The treated-count fiber multiplicity relative to all labeled words is the hypergeometric weight, including zero treated or zero hidden labels. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B :
ℕ
ι :
p :
ι → Prop
u k :
Fin B → ℕ
hk :
∀ ℓ, k ℓ ≤ u ℓ
hu :
∑ ℓ, u ℓ = card ι
hkt :
∑ ℓ, k ℓ = card {j // p j}
conclusion
(card {g : FiniteRowWord ι B u // ∀ ℓ, treatedRowCount p g.1 ℓ = k ℓ} : ℝ≥0∞) / card (FiniteRowWord ι B u)
Proof (Lean source)
-- @node: treatedRowFiber_hypergeometric_weight
lemma treatedRowFiber_hypergeometric_weight {ι : Type*} [Fintype ι] [DecidableEq ι]
(p : ι → Prop) [DecidablePred p] (u k : Fin B → ℕ) (hk : ∀ ℓ, k ℓ ≤ u ℓ)
(hu : ∑ ℓ, u ℓ = card ι)
(hkt : ∑ ℓ, k ℓ = card {j // p j}) :
(card {g : FiniteRowWord ι B u // ∀ ℓ, treatedRowCount p g.1 ℓ = k ℓ} : ℝ≥0∞) /
card (FiniteRowWord ι B u) =
(∏ ℓ, ((u ℓ).choose (k ℓ) : ℝ≥0∞)) /
(card ι).choose (card {j // p j}) := by
let A := card {g : FiniteRowWord ι B u // ∀ ℓ, treatedRowCount p g.1 ℓ = k ℓ}
let W := card (FiniteRowWord ι B u)
let P := (∏ ℓ, (k ℓ).factorial) * (∏ ℓ, (u ℓ - k ℓ).factorial)
let R := ∏ ℓ, (u ℓ).choose (k ℓ)
let C := (card ι).choose (card {j // p j})
have hc : card {j // p j} + card {j // ¬ p j} = card ι := by
simpa only [Fintype.card_subtype, Finset.card_univ] using
Finset.card_filter_add_card_filter_not (s := (Finset.univ : Finset ι)) p
have ha : A * P =
(card {j // p j}).factorial * (card {j // ¬ p j}).factorial :=
treatedRowFiber_card_mul_factorials p u k hk hu hkt
have hw : W * (∏ ℓ, (u ℓ).factorial) = (card ι).factorial :=
finiteRowWord_card_mul_factorial B u hu
have hr : R * P = ∏ ℓ, (u ℓ).factorial := by
dsimp only [R, P]
rw [← Finset.prod_mul_distrib, ← Finset.prod_mul_distrib]
apply Finset.prod_congr rfl
intro ℓ _
simpa only [Nat.mul_assoc] using Nat.choose_mul_factorial_mul_factorial (hk ℓ)
have hchoose : C *
((card {j // p j}).factorial * (card {j // ¬ p j}).factorial) =
(card ι).factorial := by
have he : card ι - card {j // p j} = card {j // ¬ p j} := by omega
simpa only [C, he, Nat.mul_assoc] using
Nat.choose_mul_factorial_mul_factorial
(show card {j // p j} ≤ card ι by omega)
have hp : P ≠ 0 := by
apply Nat.mul_ne_zero <;>
exact Finset.prod_ne_zero_iff.mpr (fun ℓ _ => Nat.factorial_ne_zero _)
have hw0 : W ≠ 0 := by
intro hz
rw [hz, zero_mul] at hw
exact Nat.factorial_ne_zero _ hw.symm
have hc0 : C ≠ 0 := by
intro hz
rw [hz, zero_mul] at hchoose
exact Nat.factorial_ne_zero _ hchoose.symm
have hid : A * C = R * W := by
apply Nat.eq_of_mul_eq_mul_right (Nat.pos_of_ne_zero hp)
calc
A * C * P = C * (A * P) := by ac_rfl
_ = (card ι).factorial := by rw [ha, hchoose]
_ = W * (R * P) := by rw [hr, hw]
_ = R * W * P := by ac_rfl
have hid' : (A : ℝ≥0∞) * C = (R : ℝ≥0∞) * W := by exact_mod_cast hid
change (A : ℝ≥0∞) / W = _
rw [← Nat.cast_prod]
change (A : ℝ≥0∞) / W = (R : ℝ≥0∞) / C
apply (ENNReal.div_eq_div_iff (by exact_mod_cast hc0) (by finiteness)
(by exact_mod_cast hw0) (by finiteness)).mpr
simpa only [mul_comm] using hid'
def finiteRowAllocation
The completed row counts are bounded by their row capacities.
Definition (Lean source)
parameters
def
given by
fun ℓ
=> ⟨treatedRowCount p g.1 ℓ, by have hc := rowCount_split_treatment p g.1 ℓ rw [g.2 ℓ] at hc omega⟩
theorem finiteRowWord_average_eq_treated_counts
Averaging any function of the completed treated counts over labeled words gives exact hypergeometric weights. This is the finite posterior averaging identity. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: finiteRowWord_average_eq_treated_counts
lemma finiteRowWord_average_eq_treated_counts {ι : Type*} [Fintype ι] [DecidableEq ι]
(p : ι → Prop) [DecidablePred p] (u : Fin B → ℕ)
(hu : ∑ ℓ, u ℓ = card ι)
(F : (∀ ℓ, Fin (u ℓ + 1)) → ℝ≥0∞) :
(card (FiniteRowWord ι B u) : ℝ≥0∞)⁻¹ *
(∑ g : FiniteRowWord ι B u, F (finiteRowAllocation p u g)) =
∑ k : ∀ ℓ, Fin (u ℓ + 1),
if card {j // p j} = ∑ ℓ, (k ℓ).val then
((∏ ℓ, ((u ℓ).choose (k ℓ).val : ℝ≥0∞)) /
(card ι).choose (card {j // p j})) * F k else 0 := by
let a := finiteRowAllocation p u
have hsplit := Finset.sum_fiberwise_of_maps_to
(s := (Finset.univ : Finset (FiniteRowWord ι B u)))
(t := (Finset.univ : Finset (∀ ℓ, Fin (u ℓ + 1))))
(g := a) (f := fun g => F (a g)) (fun _ _ => Finset.mem_univ _)
rw [← hsplit, Finset.mul_sum]
apply Finset.sum_congr rfl
intro k _
have he (g : FiniteRowWord ι B u) :
a g = k ↔ ∀ ℓ, treatedRowCount p g.1 ℓ = (k ℓ).val := by
constructor
· intro h ℓ
exact congrArg (fun a => (a ℓ).val) h
· intro h
funext ℓ
exact Fin.ext (h ℓ)
have hf : (∑ g ∈ Finset.univ.filter (fun g => a g = k), F (a g)) =
((Finset.univ.filter (fun g => a g = k)).card : ℝ≥0∞) * F k := by
have heq : (∑ g ∈ Finset.univ.filter (fun g => a g = k), F (a g)) =
∑ g ∈ Finset.univ.filter (fun g => a g = k), F k := by
apply Finset.sum_congr rfl
intro g hg
rw [(Finset.mem_filter.mp hg).2]
rw [heq, Finset.sum_const, nsmul_eq_mul]
rw [hf]
have hcard : (Finset.univ.filter (fun g => a g = k)).card =
card {g : FiniteRowWord ι B u //
∀ ℓ, treatedRowCount p g.1 ℓ = (k ℓ).val} := by
simp only [he, Fintype.card_subtype]
rw [hcard, ← mul_assoc]
by_cases hkt : card {j // p j} = ∑ ℓ, (k ℓ).val
· rw [if_pos hkt]
congr 1
rw [mul_comm, ← div_eq_mul_inv]
exact treatedRowFiber_hypergeometric_weight p u (fun ℓ => (k ℓ).val)
(fun ℓ => le_of_lt_succ (k ℓ).isLt) hu hkt.symm
· rw [if_neg hkt]
have hz : card {g : FiniteRowWord ι B u //
∀ ℓ, treatedRowCount p g.1 ℓ = (k ℓ).val} = 0 := by
rw [Fintype.card_eq_zero_iff]
refine ⟨fun g => hkt ?_⟩
have ht := treatedRowCount_sum p g.1.1
simpa only [g.2] using ht.symm
rw [hz]
simp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.TwoPriorRisk 6 declarations Full original-record two-prior risk reduction
Full original-record two-prior risk reduction
theorem measurable_record_components
Joint measurability of the record channel makes its component laws measurable.
Formal statement
hypotheses
V :
Type u_1
Ξ :
Type u_2
D :
s :
Ξ → Schedule V
hm :
conclusion
Measurable (fun ξ => D.map (recordOf (s ξ)))
Proof (Lean source)
-- @node: measurable_record_components
@[fun_prop] lemma measurable_record_components
(D : Measure (Assign V × Audit V)) [IsProbabilityMeasure D]
(s : Ξ → Schedule V)
(hm : Measurable (fun x : Ξ × (Assign V × Audit V) => recordOf (s x.1) x.2)) :
Measurable (fun ξ => D.map (recordOf (s ξ))) := by
apply Measure.measurable_of_measurable_coe
intro E hE
have hr (ξ : Ξ) : Measurable (recordOf (s ξ)) :=
hm.comp (measurable_const.prodMk measurable_id)
simp_rw [Measure.map_apply (hr _) hE]
exact measurable_measure_prodMk_left (hm hE)
theorem mixtureLaw_probability
A measurable mixture of probability record channels is a probability law. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
Ξ :
Type u_2
D :
π :
s :
Ξ → Schedule V
hm :
conclusion
IsProbabilityMeasure (mixtureLaw D π s)
Proof (Lean source)
-- @node: mixtureLaw_probability
lemma mixtureLaw_probability
(D : Measure (Assign V × Audit V)) [IsProbabilityMeasure D]
(π : Measure Ξ) [IsProbabilityMeasure π] (s : Ξ → Schedule V)
(hm : Measurable (fun x : Ξ × (Assign V × Audit V) => recordOf (s x.1) x.2)) :
IsProbabilityMeasure (mixtureLaw D π s) := by
constructor
unfold mixtureLaw
rw [Measure.bind_apply MeasurableSet.univ (measurable_record_components D s hm).aemeasurable]
have hr (ξ : Ξ) : Measurable (recordOf (s ξ)) :=
hm.comp (measurable_const.prodMk measurable_id)
simp [Measure.map_apply (hr _) MeasurableSet.univ]
theorem mixture_seeded_risk_le_worst
Prior averaging of a constant-target family's seeded squared risk is bounded by the worst fixed-schedule risk. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
Ξ :
Type u_2
D :
π :
s :
Ξ → Schedule V
hm :
d :
ℕ
a :
ℝ
hs :
∀ᵐ ξ ∂π, ScheduleClass (s ξ) d ∧ tte (s ξ) = a
T :
conclusion
≤ worstRisk D d T
Proof (Lean source)
-- @node: mixture_seeded_risk_le_worst
lemma mixture_seeded_risk_le_worst
(D : Measure (Assign V × Audit V)) [IsProbabilityMeasure D]
(π : Measure Ξ) [IsProbabilityMeasure π] (s : Ξ → Schedule V)
(hm : Measurable (fun x : Ξ × (Assign V × Audit V) => recordOf (s x.1) x.2))
(d : ℕ) (a : ℝ) (hs : ∀ᵐ ξ ∂π, ScheduleClass (s ξ) d ∧ tte (s ξ) = a)
(T : Estimator V) :
(∫⁻ x, ofReal ((T.1 x - a) ^ 2)
∂((mixtureLaw D π s).prod seedLaw)) ≤ worstRisk D d T := by
let : IsProbabilityMeasure seedLaw := ⟨by simp [seedLaw, Real.volume_Icc]⟩
let : IsProbabilityMeasure (mixtureLaw D π s) := mixtureLaw_probability D π s hm
have hf : Measurable (fun x : Record V × ℝ => ofReal ((T.1 x - a) ^ 2)) := by
exact ((T.2.sub measurable_const).pow_const 2).ennreal_ofReal
rw [lintegral_prod _ hf.aemeasurable]
unfold mixtureLaw
rw [Measure.lintegral_bind (measurable_record_components D s hm).aemeasurable
hf.lintegral_prod_right'.aemeasurable]
calc
_ ≤ ∫⁻ _ : Ξ, worstRisk D d T ∂π := by
apply lintegral_mono_ae
filter_upwards [hs] with ξ hξ
have hr : Measurable (recordOf (s ξ)) :=
hm.comp (measurable_const.prodMk measurable_id)
rw [lintegral_map hf.lintegral_prod_right' hr]
have heq : (∫⁻ ω, ∫⁻ u, ofReal ((T.1 (recordOf (s ξ) ω, u) - a) ^ 2)
∂seedLaw ∂D) = sqLoss D (s ξ) T := by
unfold sqLoss recordLaw
rw [hξ.2]
have hmseed : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(recordOf (s ξ) ω.1, ω.2)) :=
(hr.comp measurable_fst).prodMk measurable_snd
rw [lintegral_map hf hmseed]
exact (lintegral_prod _ (hf.comp hmseed).aemeasurable).symm
rw [heq]
exact le_iSup (fun θ : {θ : Schedule V // ScheduleClass θ d} => sqLoss D θ.1 T)
⟨s ξ, hξ.1⟩
_ = worstRisk D d T := by simp
Each sign error costs at least the squared target amplitude, also for infinite risk. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: threshold_squared_error_le_risk
lemma threshold_squared_error_le_risk {X : Type*} [MeasurableSpace X]
(P : Measure X) (t : X → ℝ) (ht : Measurable t) (a : ℝ) (ha : 0 ≤ a)
(σ : Bool) :
ofReal (a ^ 2) * P (if σ then {x | t x < 0} else {x | t x < 0}ᶜ) ≤
∫⁻ x, ofReal ((t x - signOf σ * a) ^ 2) ∂P := by
have hE : MeasurableSet {x | t x < 0} := measurableSet_lt ht measurable_const
have hmeas : MeasurableSet (if σ then {x | t x < 0} else {x | t x < 0}ᶜ) := by
cases σ <;> simp only [Bool.false_eq_true, ↓reduceIte] <;> first | exact hE | exact hE.compl
rw [← lintegral_indicator_const hmeas]
apply lintegral_mono
intro x
by_cases hx : x ∈ (if σ then {x | t x < 0} else {x | t x < 0}ᶜ)
· rw [Set.indicator_of_mem hx]
apply ENNReal.ofReal_le_ofReal
cases σ
· have hx' : 0 ≤ t x := by simpa using hx
simp only [signOf, Bool.false_eq_true, ↓reduceIte]
nlinarith [sq_nonneg (t x)]
· have hx' : t x < 0 := hx
simp only [signOf, ↓reduceIte]
nlinarith [sq_nonneg (t x)]
· rw [Set.indicator_of_notMem hx]
exact bot_le
theorem two_prior_squared_testing
Threshold testing of two opposite targets bounds any common upper bound on their risks. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: two_prior_squared_testing
lemma two_prior_squared_testing {X : Type*} [MeasurableSpace X]
(P M : Measure X) [IsProbabilityMeasure P] [IsProbabilityMeasure M]
(t : X → ℝ) (ht : Measurable t) (a : ℝ) (ha : 0 < a) (R : ℝ≥0∞)
(hP : (∫⁻ x, ofReal ((t x - a) ^ 2) ∂P) ≤ R)
(hM : (∫⁻ x, ofReal ((t x - -a) ^ 2) ∂M) ≤ R) :
ofReal (a ^ 2 / 2 * (1 - tvDist P M)) ≤ R := by
have hE : MeasurableSet {x | t x < 0} := measurableSet_lt ht measurable_const
have htest := ENNReal.ofReal_le_ofReal
(one_sub_tvDist_le_test (μ := P) (ν := M) hE)
rw [ENNReal.ofReal_add (measureReal_nonneg) (measureReal_nonneg),
ofReal_measureReal, ofReal_measureReal] at htest
have hplus := threshold_squared_error_le_risk P t ht a ha.le true
have hminus := threshold_squared_error_le_risk M t ht a ha.le false
simp only [signOf, Bool.false_eq_true, ↓reduceIte, one_mul, neg_one_mul] at hplus hminus
have hsum : ofReal (a ^ 2) * ofReal (1 - tvDist P M) ≤
2 * R := by
calc
_ ≤ ofReal (a ^ 2) * (P {x | t x < 0} + M {x | t x < 0}ᶜ) :=
mul_le_mul le_rfl htest zero_le zero_le
_ ≤ R + R := by rw [mul_add]; exact add_le_add (hplus.trans hP) (hminus.trans hM)
_ = 2 * R := (two_mul R).symm
have hconst : ofReal (a ^ 2) * ofReal (1 - tvDist P M) =
2 * ofReal (a ^ 2 / 2 * (1 - tvDist P M)) := by
rw [← ENNReal.ofReal_mul (sq_nonneg a), ← ENNReal.ofReal_ofNat 2,
← ENNReal.ofReal_mul (show (0 : ℝ) ≤ 2 by norm_num)]
congr 1
ring
rw [hconst] at hsum
exact (ENNReal.mul_le_mul_iff_right (by norm_num) (by norm_num)).mp hsum
theorem full_record_two_prior_risk
Opposite constant-target priors give the full-record lower bound for every measurable seeded estimator. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
ΞPlus :
Type u_3
ΞMinus :
Type u_4
n d :
ℕ
q :
ℝ
hn :
4 ≤ n
hd :
1 ≤ d
hdu :
d ≤ n - 1
hq :
q ∈ Icc 0 1
πPlus :
Measure ΞPlus
πMinus :
Measure ΞMinus
IsProbabilityMeasure πPlus
IsProbabilityMeasure πMinus
sPlus :
ΞPlus → Schedule (Fin n)
sMinus :
ΞMinus → Schedule (Fin n)
hmMinus :
Δ :
ℝ
hΔ :
0 < Δ
hPlus :
∀ᵐ ξ ∂πPlus, ScheduleClass (sPlus ξ) d ∧ tte (sPlus ξ) = Δ
hMinus :
∀ᵐ ξ ∂πMinus, ScheduleClass (sMinus ξ) d ∧ tte (sMinus ξ) = -Δ
conclusion
ofReal (Δ ^ 2 / 2 * (1 - tvDist (mixtureLaw (thinnedDesign (Fin n) q) πPlus sPlus) (mixtureLaw (thinnedDesign (Fin n) q) πMinus sMinus)))
Proof (Lean source)
-- @node: lem:full-record-two-prior-risk
lemma full_record_two_prior_risk (n d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) (hq : q ∈ Icc 0 1)
(πPlus : Measure ΞPlus) (πMinus : Measure ΞMinus)
[IsProbabilityMeasure πPlus] [IsProbabilityMeasure πMinus]
(sPlus : ΞPlus → Schedule (Fin n)) (sMinus : ΞMinus → Schedule (Fin n))
(hmPlus : Measurable (fun x : ΞPlus × (Assign (Fin n) × Audit (Fin n)) =>
recordOf (sPlus x.1) x.2))
(hmMinus : Measurable (fun x : ΞMinus × (Assign (Fin n) × Audit (Fin n)) =>
recordOf (sMinus x.1) x.2))
(Δ : ℝ) (hΔ : 0 < Δ)
(hPlus : ∀ᵐ ξ ∂πPlus, ScheduleClass (sPlus ξ) d ∧ tte (sPlus ξ) = Δ)
(hMinus : ∀ᵐ ξ ∂πMinus, ScheduleClass (sMinus ξ) d ∧ tte (sMinus ξ) = -Δ) :
ofReal (Δ ^ 2 / 2 * (1 - tvDist
(mixtureLaw (thinnedDesign (Fin n) q) πPlus sPlus)
(mixtureLaw (thinnedDesign (Fin n) q) πMinus sMinus))) ≤
minimaxRisk (thinnedDesign (Fin n) q) d := by
have _population_regime := hn
have _degree_lower := hd
have _degree_upper := hdu
let D := thinnedDesign (Fin n) q
let : IsProbabilityMeasure D := thinnedDesign_probabilityDesign q hq
let : IsProbabilityMeasure seedLaw := ⟨by simp [seedLaw, Real.volume_Icc]⟩
let P := mixtureLaw D πPlus sPlus
let M := mixtureLaw D πMinus sMinus
let : IsProbabilityMeasure P := mixtureLaw_probability D πPlus sPlus hmPlus
let : IsProbabilityMeasure M := mixtureLaw_probability D πMinus sMinus hmMinus
have htv : tvDist (P.prod seedLaw) (M.prod seedLaw) =
tvDist P M := by
simpa using tvDist_compProd_eq P M (const _ seedLaw)
change _ ≤ ⨅ T : Estimator (Fin n), worstRisk D d T
apply le_iInf
intro T
have hP := mixture_seeded_risk_le_worst D πPlus sPlus hmPlus d Δ hPlus T
have hM := mixture_seeded_risk_le_worst D πMinus sMinus hmMinus d (-Δ) hMinus T
have htest := two_prior_squared_testing (P.prod seedLaw) (M.prod seedLaw)
T.1 T.2 Δ hΔ (worstRisk D d T) hP hM
rw [htv] at htest
exact htest
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.UniformHiddenSelection 7 declarations Average selected hidden slots over original-label permutations, keeping the capacity-constrained posterior rather than independent row allocations.
Uniform hidden-subset averaging
Average selected hidden slots over original-label permutations, keeping the capacity-constrained posterior rather than independent row allocations.
Averaging a finite transitive group orbit gives the uniform average on its target. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(Fintype.card G : ℝ)⁻¹ * ∑ g : G, f (g • a) = (Fintype.card X : ℝ)⁻¹ * ∑ x : X, f x
Proof (Lean source)
-- @node: finite_transitive_orbit_average
lemma finite_transitive_orbit_average {G X : Type*} [Group G] [Fintype G]
[Fintype X] [MulAction G X] [IsPretransitive G X]
(a : X) (f : X → ℝ) :
(Fintype.card G : ℝ)⁻¹ * ∑ g : G, f (g • a) =
(Fintype.card X : ℝ)⁻¹ * ∑ x : X, f x := by
classical
letI : Nonempty X := ⟨a⟩
have hcG : (Fintype.card G : ℝ) ≠ 0 := by exact_mod_cast Fintype.card_ne_zero
have hcX : (Fintype.card X : ℝ) ≠ 0 := by exact_mod_cast Fintype.card_ne_zero
have heq (x : X) : (∑ g : G, f (g • x)) = ∑ g : G, f (g • a) := by
obtain ⟨b, rfl⟩ := MulAction.exists_smul_eq G a x
rw [show (∑ g : G, f (g • b • a)) = ∑ g : G, f ((Equiv.mulRight b g) • a) by
apply Finset.sum_congr rfl; intro g _; exact congrArg f (mul_smul g b a).symm]
exact Equiv.sum_comp (Equiv.mulRight b) (fun g : G => f (g • a))
have hs : (Fintype.card X : ℝ) * (∑ g : G, f (g • a)) =
(Fintype.card G : ℝ) * (∑ x : X, f x) := by
calc
_ = ∑ x : X, ∑ g : G, f (g • x) := by simp_rw [heq]; simp
_ = ∑ g : G, ∑ x : X, f (g • x) := Finset.sum_comm
_ = ∑ g : G, ∑ x : X, f x := by
apply Finset.sum_congr rfl
intro g _
exact Equiv.sum_comp (MulAction.toPerm g) f
_ = _ := by simp
field_simp
nlinarith [hs]
A selected set transported by a uniform label permutation is a uniform subset of its original cardinality, including the empty-set endpoint. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: permutation_monomial_average
lemma permutation_monomial_average {ι : Type*} [Fintype ι] [DecidableEq ι]
(E : Finset ι) (x : ι → ℝ) :
(card (Perm ι) : ℝ)⁻¹ *
(∑ e : Perm ι, ∏ j ∈ E, x (e j)) =
(card ι |>.choose E.card : ℝ)⁻¹ *
∑ F ∈ Finset.univ.powersetCard E.card, ∏ j ∈ F, x j := by
classical
letI : IsPretransitive (Perm ι) (powersetCard ι E.card) :=
Set.powersetCard.isPretransitive
have h := finite_transitive_orbit_average (G := Perm ι)
(⟨E, rfl⟩ : powersetCard ι E.card) (fun F => ∏ j ∈ F.val, x j)
have hm (e : Perm ι) :
(∏ j ∈ ((e • (⟨E, rfl⟩ : powersetCard ι E.card)).val), x j) =
∏ j ∈ E, x (e j) := by
rw [Set.powersetCard.coe_smul, Finset.smul_finset_def]
exact Finset.prod_image (fun i _ j _ hij => e.injective hij)
simp_rw [hm] at h
have hc : card (powersetCard ι E.card) = (card ι).choose E.card := by
simpa only [Nat.card_eq_fintype_card] using (Set.powersetCard.card (α := ι) (n := E.card))
rw [hc] at h
convert h using 1
congr 1
exact Finset.sum_subtype (Finset.univ.powersetCard E.card)
(fun F => by simp [powersetCard, Finset.mem_powersetCard]) (fun F => ∏ j ∈ F, x j)
theorem fixed_word_selection_permutation_average Lemma fixed_word_selection_permutation_average in the paper ↗
Uniformly permuting the labels of a fixed row allocation averages every selected slot family to the uniform subset with the same total degree. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: fixed_word_selection_permutation_average
lemma fixed_word_selection_permutation_average {ι : Type*} [Fintype ι] [DecidableEq ι]
(B : ℕ) (g : ι → Fin B) (k : Fin B → ℕ) (x : ι → ℝ) :
(card (Perm ι) : ℝ)⁻¹ *
(∑ e : Perm ι, ∏ ℓ, ∑ E ∈
(Finset.univ.filter (fun j => g j = ℓ)).powersetCard (k ℓ), ∏ j ∈ E, x (e j)) =
(∏ ℓ, ((Finset.univ.filter (fun j => g j = ℓ)).card.choose (k ℓ) : ℝ)) *
((card ι |>.choose (∑ ℓ, k ℓ) : ℝ)⁻¹ *
∑ E ∈ Finset.univ.powersetCard (∑ ℓ, k ℓ), ∏ j ∈ E, x j) := by
classical
let t (ℓ : Fin B) := (Finset.univ.filter (fun j => g j = ℓ)).powersetCard (k ℓ)
have hmono (E : Fin B → Finset ι) (hE : E ∈ piFinset t) :
(card (Perm ι) : ℝ)⁻¹ *
(∑ e : Perm ι, ∏ ℓ, ∏ j ∈ E ℓ, x (e j)) =
(card ι |>.choose (∑ ℓ, k ℓ) : ℝ)⁻¹ *
∑ F ∈ Finset.univ.powersetCard (∑ ℓ, k ℓ), ∏ j ∈ F, x j := by
have hsub (ℓ : Fin B) : E ℓ ⊆ Finset.univ.filter (fun j => g j = ℓ) :=
(Finset.mem_powersetCard.mp ((Fintype.mem_piFinset.mp hE) ℓ)).1
have hcard (ℓ : Fin B) : (E ℓ).card = k ℓ :=
(Finset.mem_powersetCard.mp ((Fintype.mem_piFinset.mp hE) ℓ)).2
have hdis : (↑(Finset.univ : Finset (Fin B)) : Set (Fin B)).PairwiseDisjoint E := by
intro a _ b _ hab
apply Finset.disjoint_left.mpr
intro j hja hjb
exact hab ((Finset.mem_filter.mp (hsub a hja)).2.symm.trans
(Finset.mem_filter.mp (hsub b hjb)).2)
have htotal : (Finset.univ.biUnion E).card = ∑ ℓ, k ℓ := by
rw [Finset.card_biUnion hdis]
simp_rw [hcard]
simp_rw [← Finset.prod_biUnion hdis]
simpa only [htotal] using permutation_monomial_average (Finset.univ.biUnion E) x
simp_rw [Finset.prod_univ_sum]
rw [Finset.sum_comm, Finset.mul_sum]
calc
_ = ∑ E ∈ piFinset t,
(card ι |>.choose (∑ ℓ, k ℓ) : ℝ)⁻¹ *
∑ F ∈ Finset.univ.powersetCard (∑ ℓ, k ℓ), ∏ j ∈ F, x j := by
exact Finset.sum_congr rfl hmono
_ = _ := by
rw [Finset.sum_const, nsmul_eq_mul, Fintype.card_piFinset]
simp only [t, Finset.card_powersetCard, Nat.cast_prod]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.WalshChannelBasics 40 declarations Walsh expansion and channel energy estimates
Walsh expansion and channel energy estimates
def rowDensity
A single-row shifted baseline density for a fixed Boolean sign vector.
Definition (Lean source)
def refDensity
The uniform average of all sign-vector row densities.
def walshCoeff
The permutation-symmetric Walsh likelihood coefficient, zero when the reference density vanishes.
Definition (Lean source)
parameters
d :
ℕ
h :
ℝ
s :
ℕ
w :
ℝ
def
walshCoeff d h s w :
ℝ
given by
if refDensity d h w
= 0 then 0 else ((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w * ∏ j ∈ Finset.univ.filter (fun j : Fin d => j.val < s), signOf (ε j)
def gamma
The squared Walsh coefficient integrated against its reference density.
Definition (Lean source)
parameters
d :
ℕ
h :
ℝ
s :
ℕ
def
gamma d h s :
ℝ
given by
∫ w, (walshCoeff d h s w) ^ 2 * refDensity d h w
def eta
The total nonconstant Walsh channel energy.
def etaOne
The order-weighted nonconstant Walsh channel energy.
Each sign-conditioned response density is a normalized translated baseline. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: rowDensity_integrable_normalized
lemma rowDensity_integrable_normalized (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
Integrable (rowDensity d h ε) ∧ (∫ w, rowDensity d h ε w) = 1 := by
exact translated_cosSqDensity_integrable_normalized _
All sign-conditioned densities and their uniform average are nonnegative. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: refDensity_nonneg
lemma refDensity_nonneg (d : ℕ) (h w : ℝ) : 0 ≤ refDensity d h w := by
apply mul_nonneg (by positivity)
exact Finset.sum_nonneg (fun ε _ => cosSqDensity_nonneg _)
The reference density integrates to one, since there are exactly two-to-d sign vectors. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
Proof (Lean source)
-- @node: refDensity_integrable_normalized
lemma refDensity_integrable_normalized (d : ℕ) (h : ℝ) :
Integrable (refDensity d h) ∧ (∫ w, refDensity d h w) = 1 := by
have hi : ∀ ε : Fin d → Bool, Integrable (rowDensity d h ε) :=
fun ε => (rowDensity_integrable_normalized d h ε).1
constructor
· exact (integrable_finsetSum _ (fun ε _ => hi ε)).const_mul _
· unfold refDensity
rw [integral_const_mul, integral_finsetSum _ (fun ε _ => hi ε)]
simp only [(rowDensity_integrable_normalized d h _).2, Finset.sum_const,
Finset.card_univ, Fintype.card_fun, Fintype.card_bool, Fintype.card_fin,
nsmul_eq_mul, mul_one, Nat.cast_pow, Nat.cast_ofNat]
exact inv_mul_cancel₀ (by positivity)
theorem rowDensity_eq_zero_of_refDensity_eq_zero Lemma rowDensity_eq_zero_of_refDensity_eq_zero in the paper ↗
At a zero of the reference density every component density also vanishes. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
rowDensity d h ε w = 0
Proof (Lean source)
-- @node: rowDensity_eq_zero_of_refDensity_eq_zero
lemma rowDensity_eq_zero_of_refDensity_eq_zero (d : ℕ) (h w : ℝ)
(hw : refDensity d h w = 0) (ε : Fin d → Bool) : rowDensity d h ε w = 0 := by
have hs : (∑ ε : Fin d → Bool, rowDensity d h ε w) = 0 := by
exact (mul_eq_zero.mp hw).resolve_left (inv_ne_zero (by positivity))
exact (Finset.sum_eq_zero_iff_of_nonneg
(fun ε _ => cosSqDensity_nonneg _)).mp hs ε (Finset.mem_univ _)
The degree-zero coefficient equals one wherever the reference density is positive. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: walshCoeff_zero
lemma walshCoeff_zero (d : ℕ) (h w : ℝ) (hw : 0 < refDensity d h w) :
walshCoeff d h 0 w = 1 := by
rw [walshCoeff, if_neg (ne_of_gt hw)]
simp only [Nat.not_lt_zero, Finset.filter_false, Finset.prod_empty, mul_one]
rw [← Finset.sum_div, ← mul_div_assoc]
change refDensity d h w / refDensity d h w = 1
exact div_self (ne_of_gt hw)
Summing a nonempty sign character over the full Boolean cube gives zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sign_character_sum_eq_zero
lemma sign_character_sum_eq_zero (d : ℕ) (E : Finset (Fin d)) (hE : E.Nonempty) :
(∑ ε : Fin d → Bool, ∏ j ∈ E, signOf (ε j)) = 0 := by
have hp (ε : Fin d → Bool) :
(∏ j ∈ E, signOf (ε j)) = ∏ j : Fin d, if j ∈ E then signOf (ε j) else 1 := by
simp
simp_rw [hp]
rw [← Fintype.prod_sum (fun (j : Fin d) (b : Bool) => if j ∈ E then signOf b else (1 : ℝ))]
obtain ⟨j, hj⟩ := hE
apply Finset.prod_eq_zero (Finset.mem_univ j)
simp [hj, signOf]
Multiplication by the reference density cancels the coefficient's denominator, including at zero-density points where all component densities vanish. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: walshCoeff_mul_refDensity
lemma walshCoeff_mul_refDensity (d : ℕ) (h : ℝ) (s : ℕ) (w : ℝ) :
walshCoeff d h s w * refDensity d h w =
((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
rowDensity d h ε w *
∏ j ∈ Finset.univ.filter (fun j : Fin d => j.val < s), signOf (ε j) := by
by_cases hw : refDensity d h w = 0
· simp [hw, rowDensity_eq_zero_of_refDensity_eq_zero d h w hw]
· rw [walshCoeff, if_neg hw, mul_assoc, Finset.sum_mul]
congr 1
apply Finset.sum_congr rfl
intro ε _
rw [mul_right_comm, div_mul_cancel₀ _ hw]
Every nonconstant Walsh coefficient has reference-weighted integral zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
hd :
1 ≤ d
s :
ℕ
hs :
1 ≤ s
conclusion
(∫ w, walshCoeff d h s w * refDensity d h w) = 0
Proof (Lean source)
-- @node: walshCoeff_centered
lemma walshCoeff_centered (d : ℕ) (h : ℝ) (hd : 1 ≤ d) (s : ℕ) (hs : 1 ≤ s) :
(∫ w, walshCoeff d h s w * refDensity d h w) = 0 := by
simp_rw [walshCoeff_mul_refDensity]
rw [integral_const_mul, integral_finsetSum _ (fun ε _ =>
(rowDensity_integrable_normalized d h ε).1.mul_const _)]
simp_rw [integral_mul_const, (rowDensity_integrable_normalized d h _).2, one_mul]
rw [sign_character_sum_eq_zero, mul_zero]
refine ⟨⟨0, by omega⟩, Finset.mem_filter.mpr ⟨Finset.mem_univ _, ?_⟩⟩
simpa using (show 0 < s by omega)
Squared Walsh coefficients have nonnegative reference-weighted energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: gamma_nonneg
lemma gamma_nonneg (d : ℕ) (h : ℝ) (s : ℕ) : 0 ≤ gamma d h s := by
exact integral_nonneg (fun w => mul_nonneg (sq_nonneg _) (refDensity_nonneg d h w))
Weighting each nonconstant degree by its positive order increases the energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: eta_nonneg_le_etaOne
lemma eta_nonneg_le_etaOne (d : ℕ) (h : ℝ) :
0 ≤ eta d h ∧ eta d h ≤ etaOne d h := by
constructor
· exact Finset.sum_nonneg (fun s _ => mul_nonneg (Nat.cast_nonneg _) (gamma_nonneg d h s))
· apply Finset.sum_le_sum
intro s hs
have h1 : (1 : ℝ) ≤ s := by exact_mod_cast (Finset.mem_Icc.mp hs).1
have hg := mul_nonneg (Nat.cast_nonneg (d.choose s) : (0 : ℝ) ≤ d.choose s)
(gamma_nonneg d h s)
calc
(d.choose s : ℝ) * gamma d h s ≤ s * ((d.choose s : ℝ) * gamma d h s) :=
le_mul_of_one_le_left hg h1
_ = s * (d.choose s : ℝ) * gamma d h s := by ring
Relabeling coordinates preserves the row likelihood and transports its sign character. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w * ∏ j ∈ E.map p.toEmbedding, signOf (ε j))
Proof (Lean source)
-- @node: rowDensity_character_relabel
lemma rowDensity_character_relabel (d : ℕ) (h w : ℝ) (p : Perm (Fin d))
(E : Finset (Fin d)) :
(∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w *
∏ j ∈ E.map p.toEmbedding, signOf (ε j)) =
∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w *
∏ j ∈ E, signOf (ε j) := by
let e := (Equiv.piCongrLeft (fun _ : Fin d => Bool) p).symm
have hr (ε : Fin d → Bool) : rowDensity d h ε w = rowDensity d h (e ε) w := by
unfold rowDensity
congr 2
congr 1
exact (Equiv.sum_comp p (fun j => signOf (ε j))).symm
have hc (ε : Fin d → Bool) :
(∏ j ∈ E.map p.toEmbedding, signOf (ε j)) = ∏ j ∈ E, signOf (e ε j) := by
simp [e, Finset.prod_map]
exact Fintype.sum_equiv e _ _ (fun ε => by rw [hr ε, hc ε])
Equal-size subsets have the same Walsh coefficient, by extending a subset bijection into a coordinate permutation. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: rowDensity_character_card_symmetry
lemma rowDensity_character_card_symmetry (d : ℕ) (h w : ℝ)
(E T : Finset (Fin d)) (hcard : E.card = T.card) :
(∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w *
∏ j ∈ E, signOf (ε j)) =
∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w *
∏ j ∈ T, signOf (ε j) := by
let e : E ≃ T := Fintype.equivOfCardEq (by simpa using hcard)
let p : Perm (Fin d) := e.extendSubtype
have hm : E.map p.toEmbedding = T := by
ext j
constructor
· intro hj
obtain ⟨i, hi, rfl⟩ := Finset.mem_map.mp hj
exact e.extendSubtype_mem i hi
· intro hj
let i := e.symm ⟨j, hj⟩
apply Finset.mem_map.mpr
refine ⟨i.val, i.property, ?_⟩
change e.extendSubtype i.val = j
rw [e.extendSubtype_apply_of_mem i.val i.property]
exact congrArg val (e.apply_symm_apply ⟨j, hj⟩)
rw [← hm]
exact (rowDensity_character_relabel d h w p E).symm
The canonical first-s coordinates have cardinality s for every s at most d. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: initial_coordinates_card
lemma initial_coordinates_card (d s : ℕ) (hs : s ≤ d) :
(Finset.univ.filter (fun j : Fin d => j.val < s)).card = s := by
by_cases he : s = d
· subst s
simp
· have hsd : s < d := lt_of_le_of_ne hs he
have hf : Finset.univ.filter (fun j : Fin d => j.val < s) = Finset.Iio ⟨s, hsd⟩ := by
ext j
simp [Fin.lt_def]
rw [hf, Fin.card_Iio]
On positive reference density, the symmetric coefficient agrees with the coefficient of any subset of the specified size. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
walshCoeff d h E.card w
= ((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w * ∏ j ∈ E, signOf (ε j)
Proof (Lean source)
-- @node: walshCoeff_eq_subset_coefficient
lemma walshCoeff_eq_subset_coefficient (d : ℕ) (h w : ℝ) (E : Finset (Fin d))
(hw : 0 < refDensity d h w) :
walshCoeff d h E.card w =
((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
rowDensity d h ε w / refDensity d h w * ∏ j ∈ E, signOf (ε j) := by
rw [walshCoeff, if_neg (ne_of_gt hw)]
congr 1
apply rowDensity_character_card_symmetry
apply initial_coordinates_card
exact (Finset.card_le_card (Finset.subset_univ E)).trans_eq (by simp)
The likelihood has its finite permutation-symmetric Walsh expansion. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
rowDensity d h ε w / refDensity d h w
Proof (Lean source)
-- @node: rowDensity_walsh_expansion
lemma rowDensity_walsh_expansion (d : ℕ) (h w : ℝ) (hw : 0 < refDensity d h w)
(ε : Fin d → Bool) :
rowDensity d h ε w / refDensity d h w =
∑ E : Finset (Fin d), walshCoeff d h E.card w * ∏ j ∈ E, signOf (ε j) := by
simp_rw [walshCoeff_eq_subset_coefficient d h w _ hw]
exact finite_walsh_inversion d (fun δ => rowDensity d h δ w / refDensity d h w) ε
The global square-root Lipschitz bound controls the squared difference of baseline densities, including points crossing the boundary of their support. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: cosSqDensity_difference_sq_le
lemma cosSqDensity_difference_sq_le (x y : ℝ) :
(cosSqDensity x - cosSqDensity y) ^ 2 ≤
32 * pi ^ 2 * (x - y) ^ 2 * (cosSqDensity x + cosSqDensity y) := by
have hl := baseline_translation_affinity.1.dist_le_mul x y
simp only [Real.dist_eq, Real.coe_toNNReal (4 * pi) (by positivity : 0 ≤ 4 * pi)] at hl
have hdiff : (sqrt (cosSqDensity x) - sqrt (cosSqDensity y)) ^ 2 ≤
(4 * pi) ^ 2 * (x - y) ^ 2 := by
have hsq := mul_self_le_mul_self (abs_nonneg _) hl
simpa only [← sq, mul_pow, sq_abs] using hsq
have hx := Real.sq_sqrt (cosSqDensity_nonneg x)
have hy := Real.sq_sqrt (cosSqDensity_nonneg y)
have hsum : (sqrt (cosSqDensity x) + sqrt (cosSqDensity y)) ^ 2 ≤
2 * (cosSqDensity x + cosSqDensity y) := by
nlinarith [sq_nonneg (sqrt (cosSqDensity x) - sqrt (cosSqDensity y))]
calc
(cosSqDensity x - cosSqDensity y) ^ 2 =
(sqrt (cosSqDensity x) - sqrt (cosSqDensity y)) ^ 2 *
(sqrt (cosSqDensity x) + sqrt (cosSqDensity y)) ^ 2 := by
calc
_ = (sqrt (cosSqDensity x) ^ 2 - sqrt (cosSqDensity y) ^ 2) ^ 2 := by rw [hx, hy]
_ = _ := by ring
_ ≤ ((4 * pi) ^ 2 * (x - y) ^ 2) *
(2 * (cosSqDensity x + cosSqDensity y)) :=
mul_le_mul hdiff hsum (sq_nonneg _) (by positivity)
_ = _ := by ring
Reversing one Boolean sign changes the row's sign sum by twice that sign. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: row_sign_sum_flip
lemma row_sign_sum_flip (d : ℕ) (ε : Fin d → Bool) (j : Fin d) :
(∑ i, signOf (update ε j (!(ε j)) i)) =
(∑ i, signOf (ε i)) - 2 * signOf (ε j) := by
have hs : signOf (!(ε j)) = -signOf (ε j) := by
cases ε j <;> norm_num [signOf]
rw [← Finset.add_sum_erase _ _ (Finset.mem_univ j),
← Finset.add_sum_erase _ _ (Finset.mem_univ j)]
simp only [Function.update_self, hs]
have he : (∑ i ∈ Finset.univ.erase j, signOf (update ε j (!(ε j)) i)) =
∑ i ∈ Finset.univ.erase j, signOf (ε i) := by
apply Finset.sum_congr rfl
intro i hi
rw [Function.update_of_ne (Finset.mem_erase.mp hi).1]
rw [he]
ring
The single-sign flip estimate used in the order-weighted Walsh energy proof. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowDensity_flip_difference_sq_le
lemma rowDensity_flip_difference_sq_le (d : ℕ) (h w : ℝ) (hd : 1 ≤ d)
(ε : Fin d → Bool) (j : Fin d) :
(rowDensity d h ε w - rowDensity d h (update ε j (!(ε j))) w) ^ 2 ≤
(32 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) *
(rowDensity d h ε w + rowDensity d h (update ε j (!(ε j))) w) := by
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
have he : ((w - h / (2 * d) * ∑ i, signOf (ε i)) -
(w - h / (2 * d) * ∑ i, signOf (update ε j (!(ε j)) i))) ^ 2 =
h ^ 2 / (d : ℝ) ^ 2 := by
rw [row_sign_sum_flip]
have hs : signOf (ε j) ^ 2 = 1 := by cases ε j <;> norm_num [signOf]
field_simp
nlinarith [hs]
have hb := cosSqDensity_difference_sq_le
(w - h / (2 * d) * ∑ i, signOf (ε i))
(w - h / (2 * d) * ∑ i, signOf (update ε j (!(ε j)) i))
rw [he] at hb
simpa only [rowDensity, mul_div_assoc] using hb
Reversing one coordinate permutes the Boolean cube, so it preserves the density sum. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowDensity_sum_flip
lemma rowDensity_sum_flip (d : ℕ) (h w : ℝ) (j : Fin d) :
(∑ ε : Fin d → Bool, rowDensity d h (update ε j (!(ε j))) w) =
∑ ε : Fin d → Bool, rowDensity d h ε w := by
let flip := fun ε : Fin d → Bool => update ε j (!(ε j))
have hinv : Involutive flip := by
intro ε
funext i
by_cases hi : i = j
· subst i; simp [flip]
· simp [flip, Function.update_of_ne hi]
let e : (Fin d → Bool) ≃ (Fin d → Bool) :=
⟨flip, flip, hinv, hinv⟩
exact e.sum_comp (fun ε => rowDensity d h ε w)
Averaging the single-coordinate differences cancels the reference denominator. At zero reference density every summand is zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowDensity_flip_average_le
lemma rowDensity_flip_average_le (d : ℕ) (h w : ℝ) (hd : 1 ≤ d) (j : Fin d) :
((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
(rowDensity d h ε w - rowDensity d h (update ε j (!(ε j))) w) ^ 2 /
refDensity d h w ≤ 64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2 := by
by_cases hw : refDensity d h w = 0
· simp only [hw, div_zero, Finset.sum_const_zero, mul_zero]
positivity
· have hp := lt_of_le_of_ne (refDensity_nonneg d h w) (Ne.symm hw)
have hb : (∑ ε : Fin d → Bool,
(rowDensity d h ε w - rowDensity d h (update ε j (!(ε j))) w) ^ 2) ≤
(32 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) *
(2 * ∑ ε : Fin d → Bool, rowDensity d h ε w) := by
calc
_ ≤ ∑ ε : Fin d → Bool, (32 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) *
(rowDensity d h ε w + rowDensity d h (update ε j (!(ε j))) w) :=
Finset.sum_le_sum (fun ε _ => rowDensity_flip_difference_sq_le d h w hd ε j)
_ = _ := by rw [← Finset.mul_sum, Finset.sum_add_distrib, rowDensity_sum_flip]; ring
rw [← Finset.sum_div, ← mul_div_assoc]
apply (div_le_iff₀ hp).2
have hc := mul_le_mul_of_nonneg_left hb (by positivity : 0 ≤ ((2 : ℝ) ^ d)⁻¹)
calc
_ ≤ _ := hc
_ = (64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) * refDensity d h w := by
unfold refDensity; ring
The row's centered sign sum is bounded by its number of coordinates. For the stated data and conditions, the stated conclusion holds.
Proof (Lean source)
-- @node: row_sign_sum_abs_le
lemma row_sign_sum_abs_le (d : ℕ) (ε : Fin d → Bool) :
|∑ j, signOf (ε j)| ≤ (d : ℝ) := by
calc
_ ≤ ∑ j, |signOf (ε j)| := Finset.abs_sum_le_sum_abs _ _
_ = _ := by
have hs (j : Fin d) : |signOf (ε j)| = 1 := by cases ε j <;> norm_num [signOf]
simp_rw [hs]
simp
All shifted row densities vanish outside the common enlarged baseline support. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: rowDensity_eq_zero_outside
lemma rowDensity_eq_zero_outside (d : ℕ) (h w : ℝ) (hd : 1 ≤ d) (hh : 0 ≤ h)
(ε : Fin d → Bool) (hw : 1 / 4 + h / 2 < |w|) : rowDensity d h ε w = 0 := by
have hdpos : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
have hshift : |h / (2 * d) * ∑ j, signOf (ε j)| ≤ h / 2 := by
rw [abs_mul, abs_of_nonneg (by positivity : 0 ≤ h / (2 * (d : ℝ)))]
calc
_ ≤ h / (2 * d) * d := mul_le_mul_of_nonneg_left (row_sign_sum_abs_le d ε) (by positivity)
_ = _ := by field_simp
have hout : ¬ |w - h / (2 * d) * ∑ j, signOf (ε j)| ≤ 1 / 4 := by
intro hin
have ht := abs_add_le (w - h / (2 * d) * ∑ j, signOf (ε j))
(h / (2 * d) * ∑ j, signOf (ε j))
rw [sub_add_cancel] at ht
linarith
exact if_neg hout
The reference density has the same common compact support as its shifted components. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h w :
ℝ
hd :
1 ≤ d
hh :
0 ≤ h
hw :
1 / 4 + h / 2 < |w|
conclusion
refDensity d h w = 0
Proof (Lean source)
-- @node: refDensity_eq_zero_outside
lemma refDensity_eq_zero_outside (d : ℕ) (h w : ℝ) (hd : 1 ≤ d) (hh : 0 ≤ h)
(hw : 1 / 4 + h / 2 < |w|) : refDensity d h w = 0 := by
simp [refDensity, rowDensity_eq_zero_outside d h w hd hh _ hw]
The averaged squared density difference under a single coordinate reversal.
Definition (Lean source)
parameters
def
flipEnergy d h j w :
ℝ
given by
((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, (rowDensity d h ε w - rowDensity d h (update ε j (!(ε j))) w) ^ 2 / refDensity d h w
theorem flipEnergy_nonneg
The averaged flip energy is nonnegative, including at zero reference density. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: flipEnergy_nonneg
lemma flipEnergy_nonneg (d : ℕ) (h w : ℝ) (j : Fin d) :
0 ≤ flipEnergy d h j w := by
unfold flipEnergy
exact mul_nonneg (by positivity) (Finset.sum_nonneg (fun ε _ =>
div_nonneg (sq_nonneg _) (refDensity_nonneg d h w)))
A translated baseline row is measurable across the support boundary.
Formal statement
Proof (Lean source)
-- @node: rowDensity_measurable
@[fun_prop]
lemma rowDensity_measurable (d : ℕ) (h : ℝ) (ε : Fin d → Bool) :
Measurable (rowDensity d h ε) := by
unfold rowDensity cosSqDensity
have hs : MeasurableSet {w : ℝ | |w - h / (2 * d) * ∑ j, signOf (ε j)| ≤ 1 / 4} :=
measurableSet_le (by fun_prop) measurable_const
exact Measurable.ite hs (by fun_prop) measurable_const
A finite average of the measurable row densities is measurable.
Formal statement
Proof (Lean source)
-- @node: refDensity_measurable
@[fun_prop]
lemma refDensity_measurable (d : ℕ) (h : ℝ) : Measurable (refDensity d h) := by
unfold refDensity
fun_prop
theorem flipEnergy_measurable
The averaged flip energy is measurable despite division at reference-density zeros.
Formal statement
Proof (Lean source)
-- @node: flipEnergy_measurable
@[fun_prop]
lemma flipEnergy_measurable (d : ℕ) (h : ℝ) (j : Fin d) :
Measurable (flipEnergy d h j) := by
unfold flipEnergy
fun_prop
theorem flipEnergy_le_indicator
The pointwise flip estimate is dominated by a constant on the common support interval. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
flipEnergy d h j w
Proof (Lean source)
-- @node: flipEnergy_le_indicator
lemma flipEnergy_le_indicator (d : ℕ) (h : ℝ) (hd : 1 ≤ d) (hh : 0 ≤ h)
(j : Fin d) (w : ℝ) :
flipEnergy d h j w ≤
(Icc (-(1 / 4 + h / 2)) (1 / 4 + h / 2)).indicator
(fun _ => 64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) w := by
by_cases hw : w ∈ Icc (-(1 / 4 + h / 2)) (1 / 4 + h / 2)
· rw [Set.indicator_of_mem hw]
exact rowDensity_flip_average_le d h w hd j
· rw [Set.indicator_of_notMem hw]
have hout : 1 / 4 + h / 2 < |w| := lt_of_not_ge (by
simpa only [abs_le, Set.mem_Icc] using hw)
have hz := refDensity_eq_zero_outside d h w hd hh hout
simp [flipEnergy, hz]
theorem flipEnergy_integrable
Compact support and the flip estimate supply integrability without an extra premise. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: flipEnergy_integrable
lemma flipEnergy_integrable (d : ℕ) (h : ℝ) (hd : 1 ≤ d) (hh : 0 ≤ h)
(j : Fin d) : Integrable (flipEnergy d h j) := by
have hi : Integrable ((Icc (-(1 / 4 + h / 2)) (1 / 4 + h / 2)).indicator
(fun _ : ℝ => 64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2)) :=
(integrable_indicator_iff measurableSet_Icc).mpr (integrableOn_const measure_Icc_lt_top.ne)
apply hi.mono' (by fun_prop)
exact Filter.Eventually.of_forall (fun w => by
rw [Real.norm_eq_abs, abs_of_nonneg (flipEnergy_nonneg d h w j)]
exact flipEnergy_le_indicator d h hd hh j w)
theorem integral_flipEnergy_le
Integrating the flip bound costs only the support length, at most three quarters. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
(∫ w, flipEnergy d h j w) ≤ 48 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2
Proof (Lean source)
-- @node: integral_flipEnergy_le
lemma integral_flipEnergy_le (d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(hh : h ∈ Icc 0 (1 / 4)) (j : Fin d) :
(∫ w, flipEnergy d h j w) ≤ 48 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2 := by
have hi : Integrable ((Icc (-(1 / 4 + h / 2)) (1 / 4 + h / 2)).indicator
(fun _ : ℝ => 64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2)) :=
(integrable_indicator_iff measurableSet_Icc).mpr (integrableOn_const measure_Icc_lt_top.ne)
calc
_ ≤ ∫ w, (Icc (-(1 / 4 + h / 2)) (1 / 4 + h / 2)).indicator
(fun _ : ℝ => 64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) w :=
integral_mono (flipEnergy_integrable d h hd hh.1 j) hi
(flipEnergy_le_indicator d h hd hh.1 j)
_ = (1 / 2 + h) * (64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2) := by
rw [integral_indicator_const _ measurableSet_Icc,
Real.volume_real_Icc_of_le (by linarith [hh.1])]
simp only [smul_eq_mul]
ring
_ ≤ _ := by
have hc : 0 ≤ 64 * pi ^ 2 * h ^ 2 / (d : ℝ) ^ 2 := by positivity
have hb := mul_le_mul_of_nonneg_right (show 1 / 2 + h ≤ (3 / 4 : ℝ) by
linarith [hh.2]) hc
convert hb using 1 <;> first | rfl | ring
theorem integrated_flip_sum_le
The sum of the integrated coordinate flips has the desired weighted-energy bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: integrated_flip_sum_le
lemma integrated_flip_sum_le (d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(hh : h ∈ Icc 0 (1 / 4)) :
(1 / 4 : ℝ) * ∑ j : Fin d, (∫ w, flipEnergy d h j w) ≤
12 * pi ^ 2 * h ^ 2 / d := by
have hb := Finset.sum_le_sum (s := univ)
(fun (j : Fin d) _ => integral_flipEnergy_le d h hd hh j)
have hc := mul_le_mul_of_nonneg_left hb (by norm_num : (0 : ℝ) ≤ 1 / 4)
simp only [Finset.sum_const, Finset.card_univ, Fintype.card_fin, nsmul_eq_mul] at hc
convert hc using 1 <;> first | rfl | skip
have hd0 : (d : ℝ) ≠ 0 := by exact_mod_cast (show d ≠ 0 by omega)
field_simp
ring
At zero amplitude the reference density is the unshifted baseline. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: refDensity_zero_amplitude
lemma refDensity_zero_amplitude (d : ℕ) (w : ℝ) : refDensity d 0 w = cosSqDensity w := by
simp only [refDensity, rowDensity, zero_div, zero_mul, sub_zero, Finset.sum_const,
Finset.card_univ, Fintype.card_fun, Fintype.card_bool, Fintype.card_fin,
nsmul_eq_mul, Nat.cast_pow, Nat.cast_ofNat]
rw [← mul_assoc, inv_mul_cancel₀ (by positivity), one_mul]
At zero amplitude all nonconstant Walsh coefficients vanish pointwise. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: walshCoeff_zero_amplitude
lemma walshCoeff_zero_amplitude (d s : ℕ) (w : ℝ) (hd : 1 ≤ d) (hs : 1 ≤ s) :
walshCoeff d 0 s w = 0 := by
by_cases hw : refDensity d 0 w = 0
· simp [walshCoeff, hw]
· rw [walshCoeff, if_neg hw]
simp only [rowDensity, zero_div, zero_mul, sub_zero]
rw [← Finset.mul_sum, sign_character_sum_eq_zero, mul_zero, mul_zero]
refine ⟨⟨0, by omega⟩, Finset.mem_filter.mpr ⟨Finset.mem_univ _, ?_⟩⟩
simpa using (show 0 < s by omega)
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.WalshChannelEnergy 7 declarations Assembles finite Parseval, coefficient integrability, and the integrated flip bound.
Weighted Walsh channel energy
Assembles finite Parseval, coefficient integrability, and the integrated flip bound.
theorem walsh_weighted_density_parseval
The order-weighted Walsh energy density equals one quarter of the total flip energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: walsh_weighted_density_parseval
lemma walsh_weighted_density_parseval (d : ℕ) (h w : ℝ) :
(∑ s ∈ range (d + 1),
(d.choose s : ℝ) * (s : ℝ) * (walshCoeff d h s w) ^ 2 * refDensity d h w) =
(1 / 4 : ℝ) * ∑ j : Fin d, flipEnergy d h j w := by
by_cases hz : refDensity d h w = 0
· simp [hz, flipEnergy]
have hg : 0 < refDensity d h w := lt_of_le_of_ne (refDensity_nonneg d h w) (Ne.symm hz)
have hp := finite_walsh_weighted_parseval d (fun ε => rowDensity d h ε w / refDensity d h w)
simp_rw [← walshCoeff_eq_subset_coefficient d h w _ hg] at hp
have hc := Finset.sum_powerset_apply_card
(fun s => (s : ℝ) * (walshCoeff d h s w) ^ 2)
(x := (Finset.univ : Finset (Fin d)))
simp only [Finset.powerset_univ, Fintype.card_fin, Finset.card_univ, nsmul_eq_mul] at hc
rw [hc] at hp
have hdif (ε : Fin d → Bool) (j : Fin d) :
(rowDensity d h ε w / refDensity d h w -
rowDensity d h (update ε j (!(ε j))) w / refDensity d h w) ^ 2 *
refDensity d h w =
(rowDensity d h ε w - rowDensity d h (update ε j (!(ε j))) w) ^ 2 /
refDensity d h w := by field_simp
have hm := congrArg (fun t : ℝ => t * refDensity d h w) hp
simp only [Finset.sum_mul, mul_assoc] at hm
simp_rw [← mul_assoc _ _ (refDensity d h w), hdif] at hm
simpa only [flipEnergy, mul_assoc] using hm
A Walsh likelihood coefficient has absolute value at most one. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: walshCoeff_abs_le_one
lemma walshCoeff_abs_le_one (d : ℕ) (h w : ℝ) (s : ℕ) :
|walshCoeff d h s w| ≤ 1 := by
by_cases hz : refDensity d h w = 0
· simp [walshCoeff, hz]
have hc (ε : Fin d → Bool) :
|rowDensity d h ε w / refDensity d h w *
∏ j ∈ Finset.univ.filter (fun j : Fin d => j.val < s), signOf (ε j)| =
rowDensity d h ε w / refDensity d h w := by
rw [abs_mul, Finset.abs_prod]
have hs (j : Fin d) : |signOf (ε j)| = 1 := by cases ε j <;> norm_num [signOf]
simp only [hs, Finset.prod_const_one, mul_one]
exact abs_of_nonneg (div_nonneg (cosSqDensity_nonneg _) (refDensity_nonneg d h w))
rw [walshCoeff, if_neg hz, abs_mul, abs_of_nonneg (by positivity : 0 ≤ ((2 : ℝ) ^ d)⁻¹)]
calc
_ ≤ ((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w := by
gcongr
simpa only [hc] using Finset.abs_sum_le_sum_abs
(fun ε : Fin d → Bool => rowDensity d h ε w / refDensity d h w *
∏ j ∈ Finset.univ.filter (fun j : Fin d => j.val < s), signOf (ε j)) univ
_ = 1 := by
rw [← Finset.sum_div, ← mul_div_assoc]
exact div_self hz
The coefficient convention at zero reference density preserves measurability.
Formal statement
Proof (Lean source)
-- @node: walshCoeff_measurable
@[fun_prop]
lemma walshCoeff_measurable (d : ℕ) (h : ℝ) (s : ℕ) : Measurable (walshCoeff d h s) := by
unfold walshCoeff
apply Measurable.ite
(measurableSet_eq_fun (refDensity_measurable d h) measurable_const)
· fun_prop
· fun_prop
Squared coefficient densities are integrable, dominated by the normalized reference density. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
d :
ℕ
h :
ℝ
s :
ℕ
conclusion
Proof (Lean source)
-- @node: walshCoeff_sq_density_integrable
lemma walshCoeff_sq_density_integrable (d : ℕ) (h : ℝ) (s : ℕ) :
Integrable (fun w => (walshCoeff d h s w) ^ 2 * refDensity d h w) := by
have hm : Measurable (fun w => (walshCoeff d h s w) ^ 2 * refDensity d h w) := by
fun_prop
apply (refDensity_integrable_normalized d h).1.mono' hm.aestronglyMeasurable
filter_upwards [] with w
rw [Real.norm_eq_abs, abs_of_nonneg (mul_nonneg (sq_nonneg _) (refDensity_nonneg d h w))]
have hs : (walshCoeff d h s w) ^ 2 ≤ 1 :=
(sq_le_one_iff_abs_le_one _).mpr (walshCoeff_abs_le_one d h w s)
simpa using mul_le_mul_of_nonneg_right hs (refDensity_nonneg d h w)
theorem walsh_weighted_integral_parseval
Integrating the finite weighted Parseval formula gives the channel's weighted energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: walsh_weighted_integral_parseval
lemma walsh_weighted_integral_parseval (d : ℕ) (h : ℝ) (hd : 1 ≤ d)
(hh : h ∈ Icc 0 (1 / 4)) :
etaOne d h = (1 / 4 : ℝ) * ∑ j : Fin d, (∫ w, flipEnergy d h j w) := by
have hp := congrArg (fun f : ℝ → ℝ => ∫ w, f w)
(funext (walsh_weighted_density_parseval d h))
rw [integral_finsetSum] at hp
· simp_rw [mul_assoc, integral_const_mul] at hp
rw [integral_finsetSum univ
(by intro j _; exact flipEnergy_integrable d h hd hh.1 j)] at hp
simp_rw [← mul_assoc] at hp
change (∑ s ∈ range (d + 1),
(d.choose s : ℝ) * (s : ℝ) * gamma d h s) = _ at hp
have hr : range (d + 1) = insert 0 (Icc 1 d) := by
ext s
simp only [Finset.mem_range, Finset.mem_insert, Finset.mem_Icc]
omega
rw [hr, Finset.sum_insert (by simp)] at hp
simpa [etaOne, mul_comm (d.choose _ : ℝ)] using hp
· intro s _
simpa only [mul_assoc] using
(walshCoeff_sq_density_integrable d h s).const_mul ((d.choose s : ℝ) * s)
At zero amplitude the order-weighted channel energy vanishes. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: etaOne_zero_amplitude
lemma etaOne_zero_amplitude (d : ℕ) (hd : 1 ≤ d) : etaOne d 0 = 0 := by
apply Finset.sum_eq_zero
intro s hs
have hz : gamma d 0 s = 0 := by
simp [gamma, walshCoeff_zero_amplitude d s _ hd (Finset.mem_Icc.mp hs).1]
rw [hz, mul_zero]
Finite permutation-symmetric Walsh expansion, centered coefficients, and weighted channel energy. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
conclusion 3
0 ≤ eta d h
Proof (Lean source)
-- @node: lem:walsh-channel-energy
lemma walsh_channel_energy (d : ℕ) (h : ℝ) (hd : 1 ≤ d) (hh : h ∈ Icc 0 (1 / 4)) :
(∀ w, 0 < refDensity d h w → walshCoeff d h 0 w = 1 ∧
∀ ε : Fin d → Bool, rowDensity d h ε w / refDensity d h w =
∑ E : Finset (Fin d), walshCoeff d h E.card w * ∏ j ∈ E, signOf (ε j)) ∧
(∀ s, 1 ≤ s → ∫ w, walshCoeff d h s w * refDensity d h w = 0) ∧
0 ≤ eta d h ∧ eta d h ≤ etaOne d h ∧
etaOne d h ≤ 12 * pi ^ 2 * h ^ 2 / d := by
refine ⟨?_, ?_, (eta_nonneg_le_etaOne d h).1, (eta_nonneg_le_etaOne d h).2, ?_⟩
· intro w hw
refine ⟨walshCoeff_zero d h w hw, ?_⟩
exact rowDensity_walsh_expansion d h w hw
· exact fun s hs => walshCoeff_centered d h hd s hs
· by_cases hhzero : h = 0
· subst h
rw [etaOne_zero_amplitude d hd]
simp
· rw [walsh_weighted_integral_parseval d h hd hh]
exact integrated_flip_sum_le d h hd hh
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.Helpers.WalshParseval 7 declarations The Boolean delta kernel, inversion, and squared coefficient norm used in the channel energy proof.
Finite Walsh Parseval identities
The Boolean delta kernel, inversion, and squared coefficient norm used in the channel energy proof.
The sum of products of two sign characters is the Boolean-cube delta kernel. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sign_character_delta_kernel
lemma sign_character_delta_kernel (d : ℕ) (ε δ : Fin d → Bool) :
(∑ E : Finset (Fin d), (∏ j ∈ E, signOf (δ j)) *
∏ j ∈ E, signOf (ε j)) = if δ = ε then (2 : ℝ) ^ d else 0 := by
have hk : (∑ E : Finset (Fin d), (∏ j ∈ E, signOf (δ j)) *
∏ j ∈ E, signOf (ε j)) = ∏ j : Fin d, (signOf (δ j) * signOf (ε j) + 1) := by
simpa only [Finset.prod_mul_distrib, Finset.prod_const_one, mul_one] using
(Fintype.prod_add (fun j : Fin d => signOf (δ j) * signOf (ε j)) (fun _ => (1 : ℝ))).symm
rw [hk]
by_cases he : δ = ε
· subst δ
rw [if_pos rfl]
have hs (j : Fin d) : signOf (ε j) * signOf (ε j) + 1 = 2 := by
cases ε j <;> norm_num [signOf]
simp_rw [hs]
simp
· rw [if_neg he]
obtain ⟨j, hj⟩ : ∃ j, δ j ≠ ε j := by
by_contra h
apply he
funext j
by_contra hj
exact h ⟨j, hj⟩
apply Finset.prod_eq_zero (Finset.mem_univ j)
cases hd : δ j <;> cases he : ε j <;> simp_all [signOf]
Finite Walsh inversion follows by summing the Boolean delta kernel. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: finite_walsh_inversion
lemma finite_walsh_inversion (d : ℕ) (F : (Fin d → Bool) → ℝ) (ε : Fin d → Bool) :
F ε = ∑ E : Finset (Fin d),
(((2 : ℝ) ^ d)⁻¹ * ∑ δ : Fin d → Bool, F δ * ∏ j ∈ E, signOf (δ j)) *
∏ j ∈ E, signOf (ε j) := by
simp_rw [mul_assoc, Finset.sum_mul, ← Finset.mul_sum]
rw [Finset.sum_comm]
simp_rw [mul_assoc, ← Finset.mul_sum, sign_character_delta_kernel]
simp only [mul_ite, mul_zero, Finset.sum_ite_eq', Finset.mem_univ, if_true]
rw [mul_comm (F ε), ← mul_assoc, inv_mul_cancel₀ (by positivity), one_mul]
The squared Walsh coefficients sum to the uniformly averaged squared response. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: finite_walsh_parseval
lemma finite_walsh_parseval (d : ℕ) (F : (Fin d → Bool) → ℝ) :
(∑ E : Finset (Fin d),
(((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
F ε * ∏ j ∈ E, signOf (ε j)) ^ 2) =
((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, (F ε) ^ 2 := by
let c : ℝ := ((2 : ℝ) ^ d)⁻¹
have he (E : Finset (Fin d)) :
(c * ∑ ε : Fin d → Bool, F ε * ∏ j ∈ E, signOf (ε j)) ^ 2 =
c ^ 2 * ∑ ε : Fin d → Bool, ∑ δ : Fin d → Bool,
(F ε * F δ) * ((∏ j ∈ E, signOf (ε j)) * ∏ j ∈ E, signOf (δ j)) := by
rw [mul_pow]
congr 1
rw [pow_two, Finset.sum_mul]
simp_rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro ε _
apply Finset.sum_congr rfl
intro δ _
ring
change (∑ E : Finset (Fin d), (c * _) ^ 2) = c * _
simp_rw [he]
rw [← Finset.mul_sum]
have hk : (∑ E : Finset (Fin d), ∑ ε : Fin d → Bool, ∑ δ : Fin d → Bool,
(F ε * F δ) * ((∏ j ∈ E, signOf (ε j)) * ∏ j ∈ E, signOf (δ j))) =
(2 : ℝ) ^ d * ∑ ε : Fin d → Bool, (F ε) ^ 2 := by
rw [Finset.sum_comm]
conv_lhs =>
arg 2
ext ε
rw [Finset.sum_comm]
simp_rw [← Finset.mul_sum, sign_character_delta_kernel]
simp only [mul_ite, mul_zero, Finset.sum_ite_eq, Finset.mem_univ, if_true]
simp_rw [← pow_two, mul_comm _ ((2 : ℝ) ^ d)]
rw [Finset.mul_sum]
rw [hk]
dsimp [c]
have hc : ((2 : ℝ) ^ d) ≠ 0 := by positivity
field_simp
Flipping one Boolean coordinate negates exactly the characters that contain it. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sign_character_flip
lemma sign_character_flip (d : ℕ) (E : Finset (Fin d)) (ε : Fin d → Bool)
(j : Fin d) :
(∏ i ∈ E, signOf (update ε j (!(ε j)) i)) =
(if j ∈ E then -1 else 1) * ∏ i ∈ E, signOf (ε i) := by
by_cases hj : j ∈ E
· rw [if_pos hj, ← Finset.prod_erase_mul _ _ hj,
← Finset.prod_erase_mul _ _ hj]
have he : (∏ i ∈ E.erase j, signOf (update ε j (!(ε j)) i)) =
∏ i ∈ E.erase j, signOf (ε i) := by
apply Finset.prod_congr rfl
intro i hi
rw [Function.update_of_ne (Finset.mem_erase.mp hi).1]
rw [he, Function.update_self]
have hn : signOf (!(ε j)) = -signOf (ε j) := by cases ε j <;> norm_num [signOf]
rw [hn]
ring
· rw [if_neg hj, one_mul]
apply Finset.prod_congr rfl
intro i hi
rw [Function.update_of_ne (by intro he; subst i; exact hj hi)]
A coordinate flip multiplies each finite Walsh coefficient by its character sign. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: finite_walsh_coefficient_flip
lemma finite_walsh_coefficient_flip (d : ℕ) (F : (Fin d → Bool) → ℝ)
(E : Finset (Fin d)) (j : Fin d) :
(∑ ε : Fin d → Bool, F (update ε j (!(ε j))) * ∏ i ∈ E, signOf (ε i)) =
(if j ∈ E then -1 else 1) * ∑ ε : Fin d → Bool, F ε * ∏ i ∈ E, signOf (ε i) := by
let flip := fun ε : Fin d → Bool => update ε j (!(ε j))
have hinv : Involutive flip := by
intro ε
funext i
by_cases hi : i = j
· subst i; simp [flip]
· simp [flip, Function.update_of_ne hi]
let e := hinv.toPerm
have hc (ε : Fin d → Bool) :
F (flip ε) * (∏ i ∈ E, signOf (ε i)) =
(if j ∈ E then -1 else 1) *
(F (e ε) * ∏ i ∈ E, signOf (e ε i)) := by
change F (flip ε) * _ = _ * (F (flip ε) * _)
change F (flip ε) * _ = _ * (F (flip ε) *
∏ i ∈ E, signOf (update ε j (!(ε j)) i))
rw [sign_character_flip]
split_ifs <;> ring
change (∑ ε : Fin d → Bool, F (flip ε) * _) = _
simp_rw [hc]
rw [← Finset.mul_sum]
congr 1
exact Equiv.sum_comp e (fun ε => F ε * ∏ i ∈ E, signOf (ε i))
Parseval for a coordinate difference retains only Walsh subsets containing that coordinate. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: finite_walsh_flip_parseval
lemma finite_walsh_flip_parseval (d : ℕ) (F : (Fin d → Bool) → ℝ) (j : Fin d) :
(∑ E : Finset (Fin d), if j ∈ E then
(((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
F ε * ∏ i ∈ E, signOf (ε i)) ^ 2 else 0) =
(1 / 4 : ℝ) * ((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
(F ε - F (update ε j (!(ε j)))) ^ 2 := by
have hp := finite_walsh_parseval d (fun ε => F ε - F (update ε j (!(ε j))))
have hc (E : Finset (Fin d)) :
(((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
(F ε - F (update ε j (!(ε j)))) * ∏ i ∈ E, signOf (ε i)) ^ 2 =
4 * (if j ∈ E then
(((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, F ε * ∏ i ∈ E, signOf (ε i)) ^ 2 else 0) := by
simp_rw [sub_mul]
rw [Finset.sum_sub_distrib, finite_walsh_coefficient_flip]
split_ifs <;> ring
simp_rw [hc] at hp
rw [← Finset.mul_sum] at hp
linarith
Summing coordinate differences weights each squared coefficient by its subset size. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: finite_walsh_weighted_parseval
lemma finite_walsh_weighted_parseval (d : ℕ) (F : (Fin d → Bool) → ℝ) :
(∑ E : Finset (Fin d), (E.card : ℝ) *
(((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool, F ε * ∏ i ∈ E, signOf (ε i)) ^ 2) =
(1 / 4 : ℝ) * ∑ j : Fin d,
((2 : ℝ) ^ d)⁻¹ * ∑ ε : Fin d → Bool,
(F ε - F (update ε j (!(ε j)))) ^ 2 := by
have hc (E : Finset (Fin d)) (a : ℝ) :
(E.card : ℝ) * a = ∑ j : Fin d, if j ∈ E then a else 0 := by simp
simp_rw [hc]
rw [Finset.sum_comm]
simp_rw [finite_walsh_flip_parseval]
simp only [Finset.mul_sum, mul_assoc]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.TBlockTestingScale 11 declarations Uniform nonlocal original-record testing scale
Uniform nonlocal original-record testing scale
theorem testingScale_pos_le_one
The nonlocal testing scale is positive and at most one. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
q :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
Proof (Lean source)
-- @node: testingScale_pos_le_one
lemma testingScale_pos_le_one (B d : ℕ) (q : ℝ) (hB : 1 ≤ B) (hd : 1 ≤ d) :
0 < testingScale B d q ∧ testingScale B d q ≤ 1 := by
have hm : (0 : ℝ) < (B * d : ℕ) := by exact_mod_cast (show 0 < B * d by positivity)
have hd' : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
unfold testingScale
split_ifs with hp
· exact ⟨lt_min (by norm_num) (div_pos hd' (Real.sqrt_pos.2 (mul_pos hm hp))), min_le_left _ _⟩
· norm_num
theorem testing_small_amplitude_mem
The chosen testing amplitude is in the baseline channel's declared domain. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
h ∈ Icc 0 (1 / 4)
Proof (Lean source)
-- @node: testing_small_amplitude_mem
lemma testing_small_amplitude_mem (B d : ℕ) (q h : ℝ) (hB : 1 ≤ B) (hd : 1 ≤ d)
(hh : 0 ≤ h) (hsmall : h ≤ (100 * pi)⁻¹ * testingScale B d q) :
h ∈ Icc 0 (1 / 4) := by
have hs := testingScale_pos_le_one B d q hB hd
have hku : (100 * pi)⁻¹ ≤ (1 / 4 : ℝ) := by
rw [inv_eq_one_div]
apply (div_le_iff₀ (by positivity : 0 < 100 * pi)).2
nlinarith [Real.pi_gt_three]
exact ⟨hh, hsmall.trans ((mul_le_of_le_one_right (by positivity) hs.2).trans hku)⟩
theorem testing_small_retention_energy
The testing scale bounds the source-retention energy without dividing by zero. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
q h :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hq :
q ∈ Icc 0 1
hh :
0 ≤ h
hsmall :
h ≤ (100 * pi)⁻¹ * testingScale B d q
conclusion
h ^ 2 * (B * d : ℕ) * retentionP d q ≤ (100 * pi)⁻¹ ^ 2 * (d : ℝ) ^ 2
Proof (Lean source)
-- @node: testing_small_retention_energy
lemma testing_small_retention_energy (B d : ℕ) (q h : ℝ) (hB : 1 ≤ B) (hd : 1 ≤ d)
(hq : q ∈ Icc 0 1) (hh : 0 ≤ h)
(hsmall : h ≤ (100 * pi)⁻¹ * testingScale B d q) :
h ^ 2 * (B * d : ℕ) * retentionP d q ≤ (100 * pi)⁻¹ ^ 2 * (d : ℝ) ^ 2 := by
have hp := retentionP_mem_Icc d q hq
by_cases hpzero : retentionP d q = 0
· simp only [hpzero, mul_zero]; positivity
have hppos : 0 < retentionP d q := lt_of_le_of_ne hp.1 (Ne.symm hpzero)
have hm : (0 : ℝ) < (B * d : ℕ) := by exact_mod_cast (show 0 < B * d by positivity)
have hr : 0 < sqrt ((B * d : ℕ) * retentionP d q) := Real.sqrt_pos.2 (mul_pos hm hppos)
have hscale : testingScale B d q ≤ (d : ℝ) / sqrt ((B * d : ℕ) * retentionP d q) := by
rw [testingScale, if_pos hppos]
exact min_le_right _ _
have hmul : h * sqrt ((B * d : ℕ) * retentionP d q) ≤ (100 * pi)⁻¹ * d := by
apply (le_div_iff₀ hr).mp
exact hsmall.trans (by simpa only [mul_div_assoc] using
mul_le_mul_of_nonneg_left hscale (by positivity : 0 ≤ (100 * pi)⁻¹))
have hsq := Real.sq_sqrt (le_of_lt (mul_pos hm hppos))
have hsquares := mul_self_le_mul_self (by positivity : 0 ≤ h * sqrt ((B * d : ℕ) * retentionP d q)) hmul
nlinarith
theorem testing_small_channel_energy
Walsh energy at the testing amplitude is small, both per row and after retention weighting. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
q h :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hq :
q ∈ Icc 0 1
hh :
0 ≤ h
hsmall :
h ≤ (100 * pi)⁻¹ * testingScale B d q
conclusion
conclusion 1
eta d h ≤ (3 / 2500 : ℝ)
conclusion 2
(B : ℝ) * retentionP d q * etaOne d h ≤ 3 / 2500
Proof (Lean source)
-- @node: testing_small_channel_energy
lemma testing_small_channel_energy (B d : ℕ) (q h : ℝ) (hB : 1 ≤ B) (hd : 1 ≤ d)
(hq : q ∈ Icc 0 1) (hh : 0 ≤ h)
(hsmall : h ≤ (100 * pi)⁻¹ * testingScale B d q) :
eta d h ≤ (3 / 2500 : ℝ) ∧ (B : ℝ) * retentionP d q * etaOne d h ≤ 3 / 2500 := by
have hw := walsh_channel_energy d h hd (testing_small_amplitude_mem B d q h hB hd hh hsmall)
have he := testing_small_retention_energy B d q h hB hd hq hh hsmall
have hdreal : (1 : ℝ) ≤ d := by exact_mod_cast hd
have hdpos : (0 : ℝ) < d := by positivity
have hp := retentionP_mem_Icc d q hq
have hhu : h ≤ (100 * pi)⁻¹ := hsmall.trans
(mul_le_of_le_one_right (by positivity) (testingScale_pos_le_one B d q hB hd).2)
have hsq := mul_self_le_mul_self hh hhu
have hk : pi ^ 2 * ((100 * pi)⁻¹) ^ 2 = (1 / 10000 : ℝ) := by
field_simp <;> ring
have hrow : etaOne d h ≤ 3 / 2500 := by
apply hw.2.2.2.2.trans
apply (div_le_iff₀ hdpos).2
have hm := mul_le_mul_of_nonneg_left hsq (by positivity : 0 ≤ 12 * pi ^ 2)
nlinarith only [hm, hk, hdreal]
refine ⟨hw.2.2.2.1.trans hrow, ?_⟩
have hweight := mul_le_mul_of_nonneg_left hw.2.2.2.2
(mul_nonneg (Nat.cast_nonneg B) hp.1)
have hid : (B : ℝ) * retentionP d q * (12 * pi ^ 2 * h ^ 2 / d) =
12 * pi ^ 2 * (h ^ 2 * (B * d : ℕ) * retentionP d q) / (d : ℝ) ^ 2 := by
push_cast
field_simp <;> ring
rw [hid] at hweight
apply hweight.trans
apply (div_le_iff₀ (sq_pos_of_pos hdpos)).2
have hem := mul_le_mul_of_nonneg_left he (by positivity : 0 ≤ 12 * pi ^ 2)
nlinarith only [hem, hk]
The numerical exponential and geometric factors leave a chi-squared budget below one sixteenth. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: testing_contraction_rhs_small
lemma testing_contraction_rhs_small (a b : ℝ) (ha : 0 ≤ a) (hb : 0 ≤ b)
(hau : a ≤ 3 / 2500) (hbu : b ≤ 3 / 2500) :
4 * exp 1 * a < 1 ∧
exp (exp 1 * b) / (1 - 4 * exp 1 * a) - 1 ≤ 1 / 16 := by
have he : exp 1 < 3 := Real.exp_one_lt_three
have hepos : 0 < exp 1 := Real.exp_pos _
have hden : 4 * exp 1 * a < 1 / 50 := by
nlinarith [mul_le_mul_of_nonneg_left hau hepos.le]
have hexpArg : exp 1 * b ≤ 1 / 100 := by
nlinarith [mul_le_mul_of_nonneg_left hbu hepos.le]
have hexp : exp (exp 1 * b) ≤ 51 / 50 := by
apply (Real.exp_le_exp.mpr hexpArg).trans
have h := Real.exp_bound_div_one_sub_of_interval
(by norm_num : (0 : ℝ) ≤ 1 / 100) (by norm_num : (1 / 100 : ℝ) < 1)
norm_num at h
linarith
refine ⟨by linarith, ?_⟩
have hd : 0 < 1 - 4 * exp 1 * a := by linarith
have hquot : exp (exp 1 * b) / (1 - 4 * exp 1 * a) ≤ 51 / 49 := by
apply (div_le_iff₀ hd).2
linarith
linarith
The hidden-allocation contraction theorem, with its numerical premise discharged by Walsh energy, bounds the good-event chi-squared integral at the chosen amplitude. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
q h :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hq :
q ∈ Icc 0 1
ha :
hw :
AuditLaw D q
hi :
hh :
0 ≤ h
hsmall :
h ≤ (100 * pi)⁻¹ * testingScale B d q
conclusion
(∫ H, indicator {H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H} (hiddenChiSq n B d D h) H ∂(retainedGraphMarginal n B d D))
≤ 1 / 16
Proof (Lean source)
lemma testing_small_good_chiSq_bound (n B d : ℕ) (q h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D)
(hh : 0 ≤ h) (hsmall : h ≤ (100 * pi)⁻¹ * testingScale B d q) :
(∫ H, indicator {H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H}
(hiddenChiSq n B d D h) H ∂(retainedGraphMarginal n B d D)) ≤ 1 / 16 := by
have he := testing_small_channel_energy B d q h hB hd hq hh hsmall
have hetaNonneg := (eta_nonneg_le_etaOne d h).1
have hone : 0 ≤ etaOne d h := hetaNonneg.trans (eta_nonneg_le_etaOne d h).2
have hp := retentionP_mem_Icc d q hq
have hc := testing_contraction_rhs_small (eta d h) ((B : ℝ) * retentionP d q * etaOne d h)
hetaNonneg (mul_nonneg (mul_nonneg (Nat.cast_nonneg B) hp.1) hone) he.1 he.2
rw [design_eq_thinnedDesign D q ha hw hi]
have hcontract := hidden_allocation_contraction n B d h q (halfBernoulli (Fin n))
hn hB hd hfit (testing_small_amplitude_mem B d q h hB hd hh hsmall) hq (thinnedDesign_assignment q hq) hc.1
exact hcontract.trans (by simpa only [mul_assoc] using hc.2)
theorem testing_small_high_retention
A genie bound closes the high-retention regime of the testing proof. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
q h :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hq :
q ∈ Icc 0 1
ha :
hw :
AuditLaw D q
hi :
hh :
0 ≤ h
hsmall :
h ≤ (100 * pi)⁻¹ * testingScale B d q
hp :
(1 / 2 : ℝ) ≤ retentionP d q
conclusion
≤ 1 / 2
Proof (Lean source)
-- @node: testing_small_high_retention
lemma testing_small_high_retention (n B d : ℕ) (q h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D)
(hh : 0 ≤ h) (hsmall : h ≤ (100 * pi)⁻¹ * testingScale B d q)
(hp : (1 / 2 : ℝ) ≤ retentionP d q) :
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤ 1 / 2 := by
have hgenie := genie_partition_tv_bound n B d h q D hB hd hfit
(testing_small_amplitude_mem B d q h hB hd hh hsmall) hq ha hw hi
have he := testing_small_retention_energy B d q h hB hd hq hh hsmall
have hdpos : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
have hroot := Real.sq_sqrt (by positivity : 0 ≤ (B : ℝ) / d)
have hnat : (B * d : ℕ) = (B : ℝ) * d := by push_cast; rfl
rw [hnat] at he
have henergy : h ^ 2 * ((B : ℝ) / d) ≤ 2 * (100 * pi)⁻¹ ^ 2 := by
have hbd : (B : ℝ) / d * (d : ℝ) ^ 2 = (B : ℝ) * d := by
field_simp <;> ring
have hhalf : h ^ 2 * ((B : ℝ) * d) / 2 ≤ h ^ 2 * ((B : ℝ) * d) * retentionP d q := by
nlinarith [mul_nonneg (sq_nonneg h) (by positivity : 0 ≤ (B : ℝ) * d)]
rw [← hbd] at he hhalf
nlinarith [sq_pos_of_pos hdpos]
have hk : pi ^ 2 * ((100 * pi)⁻¹) ^ 2 = (1 / 10000 : ℝ) := by
field_simp <;> ring
have hbound : 2 * pi * h * sqrt ((B : ℝ) / d) ≤ 1 / 2 := by
have he' := mul_le_mul_of_nonneg_left henergy (by positivity : 0 ≤ 4 * pi ^ 2)
have hid : (2 * pi * h * sqrt ((B : ℝ) / d)) ^ 2 =
4 * pi ^ 2 * (h ^ 2 * ((B : ℝ) / d)) := by
rw [mul_pow, mul_pow, mul_pow, hroot]
ring
have hnum : 4 * pi ^ 2 * (2 * (100 * pi)⁻¹ ^ 2) = (8 / 10000 : ℝ) := by
nlinarith only [hk]
rw [← hid, hnum] at he'
nlinarith only [he', sq_nonneg (2 * pi * h * sqrt ((B : ℝ) / d) - 1 / 2)]
exact hgenie.trans hbound
theorem testing_small_few_sources
The same genie bound closes every retention level with fewer than thirty-two sources. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
q h :
ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hq :
q ∈ Icc 0 1
ha :
hw :
AuditLaw D q
hi :
hh :
0 ≤ h
hsmall :
h ≤ (100 * pi)⁻¹ * testingScale B d q
hm :
B * d < 32
conclusion
≤ 1 / 2
Proof (Lean source)
-- @node: testing_small_few_sources
lemma testing_small_few_sources (n B d : ℕ) (q h : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D)
(hh : 0 ≤ h) (hsmall : h ≤ (100 * pi)⁻¹ * testingScale B d q)
(hm : B * d < 32) :
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤ 1 / 2 := by
have hgenie := genie_partition_tv_bound n B d h q D hB hd hfit
(testing_small_amplitude_mem B d q h hB hd hh hsmall) hq ha hw hi
have hhu : h ≤ (100 * pi)⁻¹ := hsmall.trans
(mul_le_of_le_one_right (by positivity) (testingScale_pos_le_one B d q hB hd).2)
have hdreal : (1 : ℝ) ≤ d := by exact_mod_cast hd
have hmreal : (B : ℝ) * d < 32 := by exact_mod_cast hm
have hbd : (B : ℝ) / d ≤ 32 := by
apply (div_le_iff₀ (by positivity : (0 : ℝ) < d)).2
nlinarith [Nat.cast_nonneg (α := ℝ) B]
have hroot := Real.sq_sqrt (by positivity : 0 ≤ (B : ℝ) / d)
have hsq := mul_self_le_mul_self hh hhu
have hk : pi ^ 2 * ((100 * pi)⁻¹) ^ 2 = (1 / 10000 : ℝ) := by
field_simp <;> ring
have hb : h ^ 2 * ((B : ℝ) / d) ≤ 32 * (100 * pi)⁻¹ ^ 2 :=
(mul_le_mul_of_nonneg_left hbd (sq_nonneg h)).trans
(by nlinarith)
have hb' := mul_le_mul_of_nonneg_left hb (by positivity : 0 ≤ 4 * pi ^ 2)
have hbound : 2 * pi * h * sqrt ((B : ℝ) / d) ≤ 1 / 2 := by
have hid : (2 * pi * h * sqrt ((B : ℝ) / d)) ^ 2 =
4 * pi ^ 2 * (h ^ 2 * ((B : ℝ) / d)) := by
rw [mul_pow, mul_pow, mul_pow, hroot]
ring
have hnum : 4 * pi ^ 2 * (32 * (100 * pi)⁻¹ ^ 2) = (128 / 10000 : ℝ) := by
nlinarith only [hk]
rw [← hid, hnum] at hb'
nlinarith only [hb', sq_nonneg (2 * pi * h * sqrt ((B : ℝ) / d) - 1 / 2)]
exact hgenie.trans hbound
theorem testing_radius_member_le
Above twice the unclipped testing scale, the reverse bound excludes an amplitude from the set whose total variation is at most one half. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
q :
ℝ
TV :
ℝ → ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hreverse :
∀ h : ℝ
if
0 < h
and
h ≤ 1 / 4
and
0 < retentionP d q
then
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤ TV h
h :
ℝ
hh :
h ∈ Icc 0 (1 / 4)
hTV :
TV h ≤ 1 / 2
conclusion
h ≤ 2 * testingScale B d q
Proof (Lean source)
-- @node: testing_radius_member_le
lemma testing_radius_member_le (B d : ℕ) (q : ℝ) (TV : ℝ → ℝ)
(hB : 1 ≤ B) (hd : 1 ≤ d)
(hreverse : ∀ h : ℝ, 0 < h → h ≤ 1 / 4 → 0 < retentionP d q →
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤ TV h)
(h : ℝ) (hh : h ∈ Icc 0 (1 / 4)) (hTV : TV h ≤ 1 / 2) :
h ≤ 2 * testingScale B d q := by
have hs := testingScale_pos_le_one B d q hB hd
by_cases hsone : testingScale B d q = 1
· rw [hsone]
linarith [hh.2]
· have hp : 0 < retentionP d q := by
by_contra hp
exact hsone (by simp [testingScale, hp])
have hscale : testingScale B d q =
(d : ℝ) / sqrt ((B * d : ℕ) * retentionP d q) := by
rw [testingScale, if_pos hp] at hsone ⊢
exact min_eq_right (le_of_not_ge (fun hle => hsone (min_eq_left hle)))
by_contra hle
have hlarge : 2 * testingScale B d q < h := lt_of_not_ge hle
have hhpos : 0 < h := lt_trans (mul_pos (by norm_num) hs.1) hlarge
have hm : (0 : ℝ) < (B * d : ℕ) := by
exact_mod_cast (show 0 < B * d by positivity)
have hx : 0 < (B * d : ℕ) * retentionP d q := mul_pos hm hp
have hr : 0 < sqrt ((B * d : ℕ) * retentionP d q) := Real.sqrt_pos.2 hx
have hsq := Real.sq_sqrt (le_of_lt hx)
have hdpos : (0 : ℝ) < d := by exact_mod_cast (show 0 < d by omega)
have hmul : 2 * (d : ℝ) < h * sqrt ((B * d : ℕ) * retentionP d q) := by
rw [hscale, ← mul_div_assoc] at hlarge
exact (div_lt_iff₀ hr).mp hlarge
have henergy : 4 * (d : ℝ) ^ 2 < h ^ 2 * ((B * d : ℕ) * retentionP d q) := by
nlinarith [sq_nonneg (h * sqrt ((B * d : ℕ) * retentionP d q) - 2 * d)]
have hden : 0 < 8 * h ^ 2 * (B * d : ℕ) * retentionP d q := by positivity
have hfraction : 7 * (d : ℝ) ^ 2 /
(8 * h ^ 2 * (B * d : ℕ) * retentionP d q) < 1 / 2 := by
apply (div_lt_iff₀ hden).2
nlinarith [sq_pos_of_pos hdpos]
have hb := hreverse h hhpos hh.2 hp
linarith
theorem testing_radius_of_bounds
Small-distance membership and the reverse bound give both endpoints of the testing radius; no monotonicity of total variation in amplitude is required. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
B d :
ℕ
q :
ℝ
TV :
ℝ → ℝ
hB :
1 ≤ B
hd :
1 ≤ d
hsmall :
∀ h : ℝ
hreverse :
∀ h : ℝ
if
0 < h
and
h ≤ 1 / 4
and
0 < retentionP d q
then
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤ TV h
Proof (Lean source)
-- @node: testing_radius_of_bounds
lemma testing_radius_of_bounds (B d : ℕ) (q : ℝ) (TV : ℝ → ℝ)
(hB : 1 ≤ B) (hd : 1 ≤ d)
(hsmall : ∀ h : ℝ, 0 ≤ h → h ≤ (100 * pi)⁻¹ * testingScale B d q → TV h ≤ 1 / 2)
(hreverse : ∀ h : ℝ, 0 < h → h ≤ 1 / 4 → 0 < retentionP d q →
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤ TV h) :
(100 * pi)⁻¹ * testingScale B d q ≤
sSup {h : ℝ | h ∈ Icc 0 (1 / 4) ∧ TV h ≤ 1 / 2} ∧
sSup {h : ℝ | h ∈ Icc 0 (1 / 4) ∧ TV h ≤ 1 / 2} ≤ 2 * testingScale B d q := by
have hs := testingScale_pos_le_one B d q hB hd
have hkpos : 0 < (100 * pi)⁻¹ := by positivity
have hku : (100 * pi)⁻¹ ≤ (1 / 4 : ℝ) := by
rw [inv_eq_one_div]
apply (div_le_iff₀ (by positivity : 0 < 100 * pi)).2
nlinarith [Real.pi_gt_three]
have hapos : 0 ≤ (100 * pi)⁻¹ * testingScale B d q :=
mul_nonneg (le_of_lt hkpos) (le_of_lt hs.1)
have hau : (100 * pi)⁻¹ * testingScale B d q ≤ (1 / 4 : ℝ) :=
(mul_le_of_le_one_right (le_of_lt hkpos) hs.2).trans hku
have hmem : (100 * pi)⁻¹ * testingScale B d q ∈
{h : ℝ | h ∈ Icc 0 (1 / 4) ∧ TV h ≤ 1 / 2} :=
⟨⟨hapos, hau⟩, hsmall _ hapos le_rfl⟩
constructor
· exact le_csSup ⟨1 / 4, fun h hh => hh.1.2⟩ hmem
· apply csSup_le ⟨_, hmem⟩
intro h hh
exact testing_radius_member_le B d q TV hB hd hreverse h hh.1 hh.2
theorem block_testing_scale
The all-degree complete-record testing bound, reverse bound, and two-sided testing radius. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
n B d :
ℕ
q :
ℝ
hn :
4 ≤ n
hB :
1 ≤ B
hd :
1 ≤ d
hfit :
2 * (B * d) ≤ n
hq :
q ∈ Icc 0 1
ha :
hw :
AuditLaw D q
hi :
conclusion
h :
ℝ
0 ≤ h
h ≤ κ * s
TV h ≤ 1 / 2
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤ TV h
Proof (Lean source)
-- @node: thm:block-testing-scale
theorem block_testing_scale (n B d : ℕ) (q : ℝ)
(D : Measure (Assign (Fin n) × Audit (Fin n)))
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1)
(ha : AssignmentLaw D) (hw : AuditLaw D q) (hi : DesignIndependent D) :
let s := testingScale B d q
let κ := (100 * pi)⁻¹
let TV := fun h => tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h)
(∀ h : ℝ, 0 ≤ h → h ≤ κ * s → TV h ≤ 1 / 2) ∧
(∀ h : ℝ, 0 < h → h ≤ 1 / 4 → 0 < retentionP d q →
1 - 7 * (d : ℝ) ^ 2 / (8 * h ^ 2 * (B * d : ℕ) * retentionP d q) ≤ TV h) ∧
(κ * s ≤ sSup {h : ℝ | h ∈ Icc 0 (1 / 4) ∧ TV h ≤ 1 / 2} ∧
sSup {h : ℝ | h ∈ Icc 0 (1 / 4) ∧ TV h ≤ 1 / 2} ≤ 2 * s) := by
dsimp only
have hsmall : ∀ h : ℝ, 0 ≤ h → h ≤ (100 * pi)⁻¹ * testingScale B d q →
tvDist (blockMixtureLawOf n B d D true h)
(blockMixtureLawOf n B d D false h) ≤ 1 / 2 := by
intro h hh hhu
by_cases hp : (1 / 2 : ℝ) ≤ retentionP d q
· exact testing_small_high_retention n B d q h D hB hd hfit hq ha hw hi hh hhu hp
by_cases hm : B * d < 32
· exact testing_small_few_sources n B d q h D hB hd hfit hq ha hw hi hh hhu hm
have htail := hidden_count_good_event_of_retention_half n B d q D hB hd hfit hq
(le_of_lt (lt_of_not_ge hp)) (Nat.le_of_not_gt hm) ha hw hi
have hchi := testing_small_good_chiSq_bound n B d q h D hn hB hd hfit hq ha hw hi hh hhu
let := design_isProbabilityMeasure D q ha hw hi
have htv := blockMixture_tvDist_good_event_chiSq_le n B d hd hfit D h
{H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H}
have hcompl : {H : OffDiag (Fin n) → Bool |
(B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H}ᶜ =
{H | (undiscovered n B d H : ℝ) < (B * d : ℕ) / 4} := by
ext H
simp only [Set.mem_compl_iff, Set.mem_setOf_eq, not_le]
rw [hcompl] at htv
have hsqrt : sqrt (∫ H,
indicator {H | (B * d : ℕ) / (4 : ℝ) ≤ undiscovered n B d H}
(hiddenChiSq n B d D h) H ∂(retainedGraphMarginal n B d D)) ≤ 1 / 4 := by
exact (Real.sqrt_le_sqrt hchi).trans_eq
((Real.sqrt_eq_iff_eq_sq (by norm_num) (by norm_num)).2 (by norm_num))
exact htv.trans (by linarith only [htail, hsqrt])
have hreverse := fun h hh hhu hp =>
reverse_test_tv_bound n B d h q D hB hd hfit hh hhu hq hp ha hw hi
exact ⟨hsmall, hreverse, testing_radius_of_bounds B d q _ hB hd hsmall hreverse⟩
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.TFrontierAnswer 1 declarations Explicit precision frontier answer
Explicit precision frontier answer
theorem frontier_answer
Explicit-constant precision answer and the necessary and sufficient consistency criteria. For the stated data and conditions, the stated conclusion holds.
Formal statement
conclusion
conclusion 1
conclusion 2
conclusion 3
frontierRule (V := V) n d q = upperRule n d q
conclusion 4
n d :
ℕ
4 ≤ n
1 ≤ d
d ≤ n - 1
frontierScale n d 0 = 1
frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n)
q :
ℝ
0 < q
q ≤ 1
(1 / 2 : ℝ) * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q
Proof (Lean source)
-- @node: thm:frontier-answer
theorem frontier_answer :
(∀ n : ℕ, 4 ≤ n → -- @realizes n(population size n ≥ 4)
∀ d : ℕ, 1 ≤ d → d ≤ n - 1 → -- @realizes d(1 ≤ d ≤ n-1)
∀ q : ℝ, q ∈ Icc 0 1 → -- @realizes q(known retention in [0,1])
ofReal (frontierScale n d q / (2560000 * pi ^ 2)) ≤ R (Fin n) d q ∧
R (Fin n) d q ≤ worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ∧
worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ≤
ofReal (24 * frontierScale n d q))
∧
(∀ (dseq : ℕ → ℕ) (qseq : ℕ → ℝ), AdmissibleSequences dseq qseq →
((∃ T : ∀ n, Estimator (Fin n),
Tendsto (fun n => worstRisk (thinnedDesign (Fin n) (qseq n)) (dseq n) (T n))
atTop (nhds 0)) ↔
(Tendsto (fun n => (dseq n : ℝ) ^ 2 / n) atTop (nhds 0) ∧
Tendsto (fun n => if qseq n = 0 then (⊤ : ℝ≥0∞)
else ofReal (dseq n / (n * qseq n))) atTop (nhds 0))))
∧
(∀ (V : Type) [Fintype V] [DecidableEq V] (n d : ℕ) (q : ℝ),
frontierRule (V := V) n d q = upperRule n d q) ∧
(∀ n d : ℕ, 4 ≤ n → 1 ≤ d → d ≤ n - 1 →
frontierScale n d 0 = 1 ∧ frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n) ∧
∀ q : ℝ, 0 < q → q ≤ 1 →
(1 / 2 : ℝ) * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q ∧
frontierScale n d q ≤ (1 - exp (-1))⁻¹ * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)))
:= by
rcases precision_frontier_explicit with ⟨hrisk, hconsistency, helbow⟩
refine ⟨hrisk, hconsistency, ?_, helbow⟩
intro V _ _ n d q
rfl
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.TNonlocalTestingAnswer 1 declarations Nonlocal testing answer
Nonlocal testing answer
theorem nonlocal_testing_answer
The original block family attains its nonlocal radius, with the explicit degree-two scale. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
s :
testingScale B d q
TV :
fun h
Proof (Lean source)
-- @node: thm:nonlocal-testing-answer
theorem nonlocal_testing_answer (n B d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hB : 1 ≤ B) (hd : 1 ≤ d) (hfit : 2 * (B * d) ≤ n) (hq : q ∈ Icc 0 1) :
let s := testingScale B d q
let TV := fun h => tvDist (blockMixtureLaw n B d q true h)
(blockMixtureLaw n B d q false h)
let hbar := sSup {h : ℝ | h ∈ Icc 0 (1 / 4) ∧ TV h ≤ 1 / 2}
(100 * pi)⁻¹ * s ≤ hbar ∧ hbar ≤ 2 * s ∧
(∀ h : ℝ, 0 ≤ h → h ≤ (100 * pi)⁻¹ * s → TV h ≤ 1 / 2) ∧
(d = 2 → s = if 0 < q then min 1 (sqrt (2 / (B * (2 * q - q ^ 2)))) else 1) := by
dsimp only
have htest := block_testing_scale n B d q (thinnedDesign (Fin n) q) hn hB hd hfit hq
(thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq)
change _ ∧ _ ∧ _ ∧ _
refine ⟨htest.2.2.1, htest.2.2.2, htest.1, ?_⟩
intro hd2
subst d
have hp : retentionP 2 q = 2 * q - q ^ 2 := by
unfold retentionP
ring
by_cases hqpos : 0 < q
· have hppos : 0 < retentionP 2 q := by
rw [hp]
nlinarith [hq.2]
rw [hp] at hppos
simp only [testingScale, hp, if_pos hppos, if_pos hqpos, Nat.cast_mul,
Nat.cast_ofNat]
congr 1
have hden : 0 < (B : ℝ) * 2 * (2 * q - q ^ 2) := by
have hBpos : (0 : ℝ) < B := by exact_mod_cast (show 0 < B by omega)
positivity
have hsqrt : 0 < sqrt ((B : ℝ) * 2 * (2 * q - q ^ 2)) :=
Real.sqrt_pos.2 hden
symm
apply (Real.sqrt_eq_iff_eq_sq (by positivity) (by positivity)).2
rw [div_pow, Real.sq_sqrt hden.le]
field_simp
· have hqzero : q = 0 := le_antisymm (le_of_not_gt hqpos) hq.1
subst q
simp [testingScale, retentionP]
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.TObservableUpper 4 declarations Observable total upper bound
Observable total upper bound
theorem sqLoss_seedless
Ignoring the independent seed reduces squared loss to the original design integral. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: sqLoss_seedless
lemma sqLoss_seedless (D : Measure (Assign V × Audit V)) (θ : Schedule V)
(f : Record V → ℝ) (hf : Measurable f) :
sqLoss D θ ⟨fun x => f x.1, hf.comp measurable_fst⟩ =
∫⁻ ω, ofReal ((f (recordOf θ ω) - tte θ) ^ 2) ∂D := by
let : IsProbabilityMeasure seedLaw := ⟨by simp [seedLaw, Real.volume_Icc]⟩
have hr : Measurable (recordOf θ) := by
fun_prop (disch := exact Measurable.of_discrete)
have hm : Measurable (fun ω : (Assign V × Audit V) × ℝ =>
(recordOf θ ω.1, ω.2)) := (hr.comp measurable_fst).prodMk measurable_snd
unfold sqLoss recordLaw
dsimp only
rw [lintegral_map (by fun_prop) hm,
lintegral_prod _ (by fun_prop)]
simp
Projection onto the target interval cannot increase squared error. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: clipped_sq_error_le
lemma clipped_sq_error_le (v s : ℝ) (hs : |s| ≤ 1) :
(max (-1) (min 1 v) - s) ^ 2 ≤ (v - s) ^ 2 := by
obtain ⟨hslo, hshi⟩ := abs_le.mp hs
by_cases hlo : v < -1
· rw [min_eq_right (by linarith), max_eq_left hlo.le]
nlinarith
· by_cases hhi : 1 < v
· rw [min_eq_left hhi.le, max_eq_right (by norm_num : (-1 : ℝ) ≤ 1)]
nlinarith
· rw [min_eq_right (le_of_not_gt hhi), max_eq_right (le_of_not_gt hlo)]
theorem auditScore_squared_error_le
The audit-score moments and bounded outcomes give the unclipped squared-risk envelope. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
V :
Type u_1
θ :
Schedule V
d :
ℕ
q :
ℝ
hn :
4 ≤ card V
hd :
1 ≤ d
hdu :
d ≤ card V - 1
hclass :
ScheduleClass θ d
hq :
q ∈ Icc 0 1
hqpos :
0 < q
conclusion
Proof (Lean source)
-- @node: auditScore_squared_error_le
lemma auditScore_squared_error_le (θ : Schedule V) (d : ℕ) (q : ℝ)
(hn : 4 ≤ Fintype.card V) (hd : 1 ≤ d) (hdu : d ≤ Fintype.card V - 1)
(hclass : ScheduleClass θ d) (hq : q ∈ Icc 0 1) (hqpos : 0 < q) :
(∫ ω, (auditScore q (recordOf θ ω) - tte θ) ^ 2 ∂thinnedDesign V q) ≤
5 * ((d : ℝ) + 1) ^ 2 / Fintype.card V +
4 * d * (1 - q) / (Fintype.card V * q) := by
let : IsProbabilityMeasure (thinnedDesign V q) := thinnedDesign_probabilityDesign q hq
let : IsProbabilityMeasure (halfBernoulli V) := by
let : IsProbabilityMeasure (bernoulliLaw (1 / 2)) :=
bernoulliLaw_probability _ (by constructor <;> norm_num)
unfold halfBernoulli
infer_instance
obtain ⟨hunbiased, hvariance, horacle⟩ := score_audit (thinnedDesign V q) θ d q
hn hd hdu hclass (thinnedDesign_assignment q hq) (thinnedDesign_audit q hq)
(thinnedDesign_independent q hq) hqpos
have hm : Measurable (fun ω => auditScore q (recordOf θ ω)) := by
fun_prop (disch := exact Measurable.of_discrete)
rw [← hunbiased, ← ProbabilityTheory.variance_eq_integral hm.aemeasurable, hvariance]
have hy (i : V) : (∫ z, (potentialOutcome θ i z) ^ 2 ∂halfBernoulli V) ≤ 1 := by
calc
_ ≤ ∫ _z, (1 : ℝ) ∂halfBernoulli V := by
apply integral_mono Integrable.of_finite Integrable.of_finite
intro z
exact (sq_le_one_iff_abs_le_one _).mpr (abs_potentialOutcome_le_one hclass i z)
_ = 1 := by simp
have hsum : (∑ i, (inNbhd θ i).card *
∫ z, (potentialOutcome θ i z) ^ 2 ∂halfBernoulli V) ≤ Fintype.card V * (d : ℝ) := by
calc
_ ≤ ∑ _i : V, (d : ℝ) := by
apply Finset.sum_le_sum
intro i _
calc
_ ≤ (inNbhd θ i).card * (1 : ℝ) :=
mul_le_mul_of_nonneg_left (hy i) (Nat.cast_nonneg _)
_ ≤ d := by simpa using (show ((inNbhd θ i).card : ℝ) ≤ d by
exact_mod_cast hclass.inDegree i)
_ = _ := by simp
have hc : 0 ≤ 4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) :=
div_nonneg (mul_nonneg (by norm_num) (sub_nonneg.mpr hq.2))
(mul_nonneg (sq_nonneg _) hqpos.le)
have hnpos : (0 : ℝ) < Fintype.card V := by exact_mod_cast (show 0 < Fintype.card V by omega)
calc
_ ≤ 5 * ((d : ℝ) + 1) ^ 2 / Fintype.card V +
4 * (1 - q) / ((Fintype.card V : ℝ) ^ 2 * q) * (Fintype.card V * (d : ℝ)) :=
add_le_add horacle (mul_le_mul_of_nonneg_left hsum hc)
_ = _ := by field_simp
theorem observable_upper
The total observable upper rule attains the envelope, including zero retention. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: thm:observable-upper
theorem observable_upper (d : ℕ) (q : ℝ) (hn : 4 ≤ card V)
(hd : 1 ≤ d) (hdu : d ≤ card V - 1) (hq : q ∈ Icc 0 1) :
minimaxRisk (thinnedDesign V q) d ≤
worstRisk (thinnedDesign V q) d (upperEstimator (card V) d q) ∧
worstRisk (thinnedDesign V q) d (upperEstimator (card V) d q) ≤
min 1 (upperEnvelope (card V) d q) ∧
Measurable (upperRule (V := V) (card V) d q) := by
let : IsProbabilityMeasure (thinnedDesign V q) := thinnedDesign_probabilityDesign q hq
refine ⟨Causalean.Stat.minimaxValueENNReal_le_worstCaseRisk _, ?_,
upperRule_measurable _ _ _⟩
apply worstCaseRiskENNReal_le
intro θ
rw [show upperEstimator (V := V) (card V) d q =
⟨fun x => upperRule (card V) d q x.1,
(upperRule_measurable _ _ _).comp measurable_fst⟩ from rfl,
sqLoss_seedless _ _ _ (upperRule_measurable _ _ _)]
by_cases hbranch : 0 < q ∧ upperEnvelope (card V) d q < 1
· have hrisk := auditScore_squared_error_le θ.1 d q hn hd hdu θ.2 hq hbranch.1
rw [min_eq_right hbranch.2.le, upperEnvelope, if_pos hbranch.1]
calc
_ ≤ ∫⁻ ω, ofReal ((auditScore q (recordOf θ.1 ω) - tte θ.1) ^ 2)
∂thinnedDesign V q := by
apply lintegral_mono
intro ω
apply ENNReal.ofReal_le_ofReal
simpa only [upperRule, if_pos hbranch] using
clipped_sq_error_le (auditScore q (recordOf θ.1 ω)) (tte θ.1) (abs_tte_le_one θ.2)
_ = ofReal (∫ ω, (auditScore q (recordOf θ.1 ω) - tte θ.1) ^ 2
∂thinnedDesign V q) :=
(ofReal_integral_eq_lintegral_ofReal Integrable.of_finite
(Filter.Eventually.of_forall (fun _ => sq_nonneg _))).symm
_ ≤ _ := ENNReal.ofReal_le_ofReal hrisk
· have henv : 1 ≤ upperEnvelope (card V) d q := by
by_cases hpos : 0 < q
· exact le_of_not_gt (fun h => hbranch ⟨hpos, h⟩)
· simp [upperEnvelope, hpos]
rw [min_eq_left henv]
calc
_ ≤ ∫⁻ _ω, (1 : ℝ≥0∞) ∂thinnedDesign V q := by
apply lintegral_mono
intro ω
have ht := (sq_le_one_iff_abs_le_one (tte θ.1)).mpr (abs_tte_le_one θ.2)
simpa [upperRule, hbranch] using ENNReal.ofReal_le_ofReal ht
_ = 1 := by simp
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.TObservedRecordPrecisionFrontier 3 declarations Complete original-record precision frontier
Complete original-record precision frontier
theorem frontierEstimator_risk_upper
The total observable frontier estimator attains the explicit upper constant through the audit-score envelope and the reveal-probability comparisons. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
Proof (Lean source)
-- @node: frontierEstimator_risk_upper
lemma frontierEstimator_risk_upper (n d : ℕ) (q : ℝ)
(hn : 4 ≤ n) (hd : 1 ≤ d) (hdu : d ≤ n - 1) (hq : q ∈ Icc 0 1) :
R (Fin n) d q ≤ worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ∧
worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ≤
ofReal (24 * frontierScale n d q) := by
obtain ⟨hminimax, hupper, _⟩ := observable_upper (V := Fin n) d q
(by simpa using hn) hd (by simpa using hdu) hq
refine ⟨?_, ?_⟩
· simpa only [R, frontierEstimator, Fintype.card_fin] using hminimax
· have hupper' : worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ≤
min 1 (upperEnvelope n d q) := by
simpa only [frontierEstimator, Fintype.card_fin] using hupper
exact hupper'.trans (upperEnvelope_le_frontierScale n d q hn hd hq)
theorem precision_frontier_explicit
The displayed explicit constants establish the risk bounds, consistency criterion, endpoint identities, and interior-retention comparison.
Formal statement
conclusion
conclusion 1
conclusion 2
conclusion 3
n d :
ℕ
4 ≤ n
1 ≤ d
d ≤ n - 1
frontierScale n d 0 = 1
frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n)
q :
ℝ
0 < q
q ≤ 1
(1 / 2 : ℝ) * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q
Proof (Lean source)
-- @node: precision_frontier_explicit
lemma precision_frontier_explicit :
(∀ n : ℕ, 4 ≤ n → -- @realizes n(population size n ≥ 4)
∀ d : ℕ, 1 ≤ d → d ≤ n - 1 → -- @realizes d(1 ≤ d ≤ n-1)
∀ q : ℝ, q ∈ Icc 0 1 → -- @realizes q(known retention in [0,1])
ofReal (frontierScale n d q / (2560000 * pi ^ 2)) ≤ R (Fin n) d q ∧
R (Fin n) d q ≤ worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ∧
worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ≤
ofReal (24 * frontierScale n d q))
∧
(∀ (dseq : ℕ → ℕ) (qseq : ℕ → ℝ), AdmissibleSequences dseq qseq →
((∃ T : ∀ n, Estimator (Fin n),
Tendsto (fun n => worstRisk (thinnedDesign (Fin n) (qseq n)) (dseq n) (T n))
atTop (nhds 0)) ↔
(Tendsto (fun n => (dseq n : ℝ) ^ 2 / n) atTop (nhds 0) ∧
Tendsto (fun n => if qseq n = 0 then (⊤ : ℝ≥0∞)
else ofReal (dseq n / (n * qseq n))) atTop (nhds 0))))
∧
(∀ n d : ℕ, 4 ≤ n → 1 ≤ d → d ≤ n - 1 →
frontierScale n d 0 = 1 ∧ frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n) ∧
∀ q : ℝ, 0 < q → q ≤ 1 →
(1 / 2 : ℝ) * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q ∧
frontierScale n d q ≤ (1 - exp (-1))⁻¹ * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)))
:= by
refine ⟨?_, ?_, ?_⟩
· intro n hn d hd hdu q hq
obtain ⟨hminimax, hupper⟩ := frontierEstimator_risk_upper n d q hn hd hdu hq
refine ⟨?_, hminimax, hupper⟩
exact frontier_minimax_lower n d q hn hd hdu hq
· intro dseq qseq ha
have hadm : ∀ᶠ n in atTop, 4 ≤ n := Filter.eventually_ge_atTop 4
constructor
· rintro ⟨T, hT⟩
let C : ℝ := 2560000 * pi ^ 2
have hC : 0 < C := mul_pos (by norm_num) (sq_pos_of_pos Real.pi_pos)
have hlower : ∀ᶠ n in atTop,
ofReal (frontierScale n (dseq n) (qseq n) / C) ≤
worstRisk (thinnedDesign (Fin n) (qseq n)) (dseq n) (T n) := by
filter_upwards [hadm] with n hn
obtain ⟨hd, hdu, hq⟩ := ha n hn
exact (frontier_minimax_lower n (dseq n) (qseq n) hn hd hdu hq).trans
(minimaxValueENNReal_le_worstCaseRisk (T n))
have hscaled : Tendsto
(fun n => ofReal (frontierScale n (dseq n) (qseq n) / C))
atTop (nhds 0) :=
tendsto_of_tendsto_of_tendsto_of_le_of_le' tendsto_const_nhds hT
(Filter.Eventually.of_forall (fun _ => bot_le)) hlower
have hreal := (ENNReal.tendsto_toReal ENNReal.zero_ne_top).comp hscaled
simp only [ENNReal.toReal_zero] at hreal
have hdiv : Tendsto (fun n => frontierScale n (dseq n) (qseq n) / C)
atTop (nhds 0) := by
apply hreal.congr'
filter_upwards [hadm] with n hn
exact ENNReal.toReal_ofReal (div_nonneg
(frontierScale_nonneg n (dseq n) (qseq n) (ha n hn).2.2) hC.le)
have hF : Tendsto (fun n => frontierScale n (dseq n) (qseq n))
atTop (nhds 0) := by
simpa only [div_mul_cancel₀ _ (ne_of_gt hC), zero_mul] using hdiv.mul_const C
exact (frontierScale_tendsto_iff dseq qseq ha).mp hF
· intro hratios
have hF := (frontierScale_tendsto_iff dseq qseq ha).mpr hratios
refine ⟨fun n => frontierEstimator n (dseq n) (qseq n), ?_⟩
have hupper : ∀ᶠ n in atTop,
worstRisk (thinnedDesign (Fin n) (qseq n)) (dseq n)
(frontierEstimator n (dseq n) (qseq n)) ≤
ofReal (24 * frontierScale n (dseq n) (qseq n)) := by
filter_upwards [hadm] with n hn
obtain ⟨hd, hdu, hq⟩ := ha n hn
exact (frontierEstimator_risk_upper n (dseq n) (qseq n) hn hd hdu hq).2
have hlimit := ENNReal.tendsto_ofReal (hF.const_mul 24)
simp only [mul_zero, ENNReal.ofReal_zero] at hlimit
exact tendsto_of_tendsto_of_tendsto_of_le_of_le' tendsto_const_nhds hlimit
(Filter.Eventually.of_forall (fun _ => bot_le)) hupper
· exact frontier_elbow
theorem observed_record_precision_frontier
There exist positive universal constants c and C0 giving the stated risk sandwich; consistent estimators exist exactly when both asymptotic scale conditions hold; the frontier scale equals 1 at zero retention and min 1 (d²/n) at full retention; and positive universal comparison constants c1 and C1, independent of n, d, and q, bound the frontier by the stated interior-retention scale.
Formal statement
conclusion
∃ c C0 : ℝ,
0 < c
0 < C0
conclusion 1
conclusion 2
conclusion 3
n d :
ℕ
4 ≤ n
1 ≤ d
d ≤ n - 1
frontierScale n d 0 = 1
frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n)
conclusion 4
∃ c1 C1 : ℝ,
0 < c1
0 < C1
n d :
ℕ
4 ≤ n
1 ≤ d
d ≤ n - 1
q :
ℝ
0 < q
q ≤ 1
c1 * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q
frontierScale n d q ≤ C1 * min 1 ((d : ℝ) ^ 2 / n + d / (n * q))
Proof (Lean source)
-- @node: thm:observed-record-precision-frontier
theorem observed_record_precision_frontier :
∃ c C0 : ℝ, 0 < c ∧ 0 < C0 ∧
-- @realizes c(positive universal lower constant)
-- @realizes C0(positive finite universal upper constant)
(∀ n : ℕ, 4 ≤ n → -- @realizes n(population size n ≥ 4)
∀ d : ℕ, 1 ≤ d → d ≤ n - 1 → -- @realizes d(1 ≤ d ≤ n-1)
∀ q : ℝ, q ∈ Icc 0 1 → -- @realizes q(known retention in [0,1])
ofReal (c * frontierScale n d q) ≤ R (Fin n) d q ∧
R (Fin n) d q ≤ worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ∧
worstRisk (thinnedDesign (Fin n) q) d (frontierEstimator n d q) ≤
ofReal (C0 * frontierScale n d q)) ∧
(∀ (dseq : ℕ → ℕ) (qseq : ℕ → ℝ), AdmissibleSequences dseq qseq →
((∃ T : ∀ n, Estimator (Fin n),
Tendsto (fun n => worstRisk (thinnedDesign (Fin n) (qseq n)) (dseq n) (T n))
atTop (nhds 0)) ↔
(Tendsto (fun n => (dseq n : ℝ) ^ 2 / n) atTop (nhds 0) ∧
Tendsto (fun n => if qseq n = 0 then (⊤ : ℝ≥0∞)
else ofReal (dseq n / (n * qseq n))) atTop (nhds 0))))
∧
(∀ n d : ℕ, 4 ≤ n → 1 ≤ d → d ≤ n - 1 →
frontierScale n d 0 = 1 ∧ frontierScale n d 1 = min 1 ((d : ℝ) ^ 2 / n)) ∧
(∃ c1 C1 : ℝ, 0 < c1 ∧ 0 < C1 ∧
∀ n d : ℕ, 4 ≤ n → 1 ≤ d → d ≤ n - 1 →
∀ q : ℝ, 0 < q → q ≤ 1 →
c1 * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)) ≤ frontierScale n d q ∧
frontierScale n d q ≤ C1 * min 1 ((d : ℝ) ^ 2 / n + d / (n * q)))
:= by
rcases precision_frontier_explicit with ⟨hrisk, hconsistency, helbow⟩
refine ⟨1 / (2560000 * pi ^ 2), 24, ?_, by norm_num, ?_,
hconsistency, ?_, ?_⟩
· exact div_pos (by norm_num) (mul_pos (by norm_num) (sq_pos_of_pos Real.pi_pos))
· intro n hn d hd hdu q hq
simpa only [one_div, div_eq_mul_inv, one_mul, mul_comm] using hrisk n hn d hd hdu q hq
· intro n d hn hd hdu
exact ⟨(helbow n d hn hd hdu).1, (helbow n d hn hd hdu).2.1⟩
· refine ⟨1 / 2, (1 - exp (-1))⁻¹, by norm_num, ?_, ?_⟩
· exact inv_pos.mpr (sub_pos.mpr
(Real.exp_lt_one_iff.mpr (by norm_num : (-1 : ℝ) < 0)))
· intro n d hn hd hdu q hq hq1
exact (helbow n d hn hd hdu).2.2 q hq hq1
CausalSmith.Experimentation.EXP_ThinnedgraphAdditiveRiskFrontier_Research.TSupportedBoundaries 4 declarations Supported boundary regimes
Supported boundary regimes
theorem minimaxRisk_mono_degree
Enlarging the schedule degree class cannot reduce minimax risk. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
minimaxRisk D d ≤ minimaxRisk D d'
Proof (Lean source)
-- @node: minimaxRisk_mono_degree
lemma minimaxRisk_mono_degree {V : Type*} [Fintype V] [DecidableEq V]
(D : Measure (Assign V × Audit V)) {d d' : ℕ} (hdd : d ≤ d') :
minimaxRisk D d ≤ minimaxRisk D d' := by
apply minimaxValueENNReal_mono_class
(fun θ => ⟨θ.1, θ.2.mono hdd⟩)
intro T θ
exact le_rfl
theorem degree_one_frontier_reciprocal
The degree-one testing frontier dominates the reciprocal audit sample size. For the stated data and conditions, the stated conclusion holds.
Formal statement
Proof (Lean source)
-- @node: degree_one_frontier_reciprocal
lemma degree_one_frontier_reciprocal (n : ℕ) (hn : 4 ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) :
1 / (1 + n * q) ≤ frontierScale n 1 q := by
have hn' : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hx : 0 ≤ (n : ℝ) * q := mul_nonneg hn'.le hq.1
have hden : 0 < 1 + (n : ℝ) * q := by linarith
by_cases hp : 0 < q
· simp only [frontierScale, if_pos hp, Nat.cast_one, one_pow,
pow_one, sub_sub_cancel]
apply le_min
· exact (div_le_one hden).2 (by linarith)
· exact div_le_div_of_nonneg_left (by norm_num) (mul_pos hn' hp) (by linarith)
· have hz : q = 0 := by linarith [hq.1]
subst q
simp [frontierScale]
theorem degree_one_envelope_upper
The degree-one observable envelope gives the reciprocal audit upper bound. For the stated data and conditions, the stated conclusion holds.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @node: degree_one_envelope_upper
lemma degree_one_envelope_upper (n : ℕ) (hn : 4 ≤ n)
(q : ℝ) (hq : q ∈ Icc 0 1) :
min 1 (upperEnvelope n 1 q) ≤ ofReal (21 / (1 + n * q)) := by
have hn' : (0 : ℝ) < n := by exact_mod_cast (show 0 < n by omega)
have hx : 0 ≤ (n : ℝ) * q := mul_nonneg hn'.le hq.1
have hden : 0 < 1 + (n : ℝ) * q := by linarith
by_cases hp : 0 < q
· have hnp : 0 < (n : ℝ) * q := mul_pos hn' hp
have hu : 5 * ((1 : ℝ) + 1) ^ 2 / n + 4 * 1 * (1 - q) / (n * q) ≤
20 / (n * q) := by
apply (le_div_iff₀ hnp).2
field_simp
nlinarith [hq.2]
have he : min 1 (upperEnvelope n 1 q) ≤ ofReal (min 1 (20 / (n * q))) := by
rw [upperEnvelope, if_pos hp, ← ENNReal.ofReal_one, ← ENNReal.ofReal_min]
exact ENNReal.ofReal_le_ofReal (min_le_min_left _ (by simpa using hu))
apply he.trans
apply ENNReal.ofReal_le_ofReal
by_cases hs : (n : ℝ) * q ≤ 20
· exact (min_le_left _ _).trans ((le_div_iff₀ hden).2 (by linarith))
· apply (min_le_right _ _).trans
apply (div_le_div_iff₀ hnp hden).2
nlinarith
· have hz : q = 0 := by linarith [hq.1]
subst q
simp [upperEnvelope]
theorem supported_boundaries
Universal lower bound and the zero-retention, known-graph, and degree-one boundary sandwiches. the stated conclusion holds.
Formal statement
conclusion
∃ c C0 : ℝ,
0 < c
0 < C0
conclusion 1
Proof (Lean source)
-- @node: thm:supported-boundaries
theorem supported_boundaries :
∃ c C0 : ℝ, 0 < c ∧ 0 < C0 ∧
-- @realizes c(positive universal constant, outside every regime binder)
-- @realizes C0(positive finite universal constant, outside every regime binder)
∀ n : ℕ, 4 ≤ n → -- @realizes n(population size n ≥ 4)
∀ d : ℕ, 1 ≤ d → d ≤ n - 1 → -- @realizes d(1 ≤ d ≤ n-1)
∀ q : ℝ, q ∈ Icc 0 1 → -- @realizes q(known retention in [0,1])
∀ D : Measure (Assign (Fin n) × Audit (Fin n)),
AssignmentLaw D → AuditLaw D q → DesignIndependent D →
ofReal (c * max (min 1 (((d : ℝ) + 1) ^ 2 / n)) (1 / (1 + n * q))) ≤ minimaxRisk D d ∧
(q = 0 → minimaxRisk D d ≤ 1) ∧
(q = 1 → minimaxRisk D d ≤ ofReal (C0 * min 1 (((d : ℝ) + 1) ^ 2 / n))) ∧
(d = 1 → minimaxRisk D d ≤ ofReal (C0 / (1 + n * q))) := by
-- Reuse the proved degree-one block frontier with its conservative universal constant.
let c : ℝ := (2560000 * pi ^ 2)⁻¹
have hc : 0 < c := by dsimp [c]; positivity
refine ⟨c, 21, hc, by norm_num, ?_⟩
intro n hn d hd hdu q hq D ha hw hi
have hD := design_eq_thinnedDesign D q ha hw hi
subst D
have hupper := observable_upper (V := Fin n) d q
(by simpa using hn) hd (by simpa using hdu) hq
have hu : minimaxRisk (thinnedDesign (Fin n) q) d ≤ min 1 (upperEnvelope n d q) := by
simpa only [Fintype.card_fin] using hupper.1.trans hupper.2.1
have hsup := supplied_block_record_lower n d q (thinnedDesign (Fin n) q)
hn hd hdu hq ha hw hi
have hfirst : ofReal (c * min 1 (((d : ℝ) + 1) ^ 2 / n)) ≤
minimaxRisk (thinnedDesign (Fin n) q) d := by
apply le_trans (ENNReal.ofReal_le_ofReal ?_) hsup
apply mul_le_mul_of_nonneg_right _ (le_min (by norm_num) (by positivity))
dsimp [c]
exact (inv_le_inv₀ (by positivity) (by positivity)).2
(by nlinarith [sq_pos_of_pos Real.pi_pos])
have hsecond : ofReal (c * (1 / (1 + n * q))) ≤
minimaxRisk (thinnedDesign (Fin n) q) d := by
have hl := frontier_minimax_lower n 1 q hn (by omega) (by omega) hq
apply le_trans (ENNReal.ofReal_le_ofReal ?_)
(hl.trans (minimaxRisk_mono_degree _ hd))
simpa [c, div_eq_mul_inv, mul_comm] using
mul_le_mul_of_nonneg_left (degree_one_frontier_reciprocal n hn q hq) hc.le
refine ⟨?_, ?_, ?_, ?_⟩
· rw [mul_max_of_nonneg _ _ hc.le, ENNReal.ofReal_max]
exact max_le hfirst hsecond
· intro _
exact hu.trans (min_le_left _ _)
· intro hqone
subst q
have he : min 1 (upperEnvelope n d 1) ≤
ofReal (21 * min 1 (((d : ℝ) + 1) ^ 2 / n)) := by
simp only [upperEnvelope, zero_lt_one, ↓reduceIte, sub_self, mul_zero,
zero_div, add_zero]
rw [← ENNReal.ofReal_one, ← ENNReal.ofReal_min]
apply ENNReal.ofReal_le_ofReal
by_cases hs : ((d : ℝ) + 1) ^ 2 / n ≤ 1
· rw [min_eq_right hs]
have heq : 5 * ((d : ℝ) + 1) ^ 2 / n = 5 * (((d : ℝ) + 1) ^ 2 / n) := by ring
rw [heq]
exact (min_le_right _ _).trans (by nlinarith [show 0 ≤ ((d : ℝ) + 1) ^ 2 / n by positivity])
· rw [min_eq_left (le_of_not_ge hs)]
exact (min_le_left _ _).trans (by norm_num)
exact hu.trans he
· intro hdOne
subst d
exact hu.trans
(degree_one_envelope_upper n hn q hq)