CausalSmith — CSWP-2026-041 · ← back to the paper · arXiv · GitHub

AI referee report

on “Testing Treatment Effect Homogeneity under Finite Conditional Moments”

Overall score 7.0/10 7 findings — 2 major · 4 minor · 1 nit

The verified separation frontiers offer a substantial theoretical contribution with largely faithful scope, while the placement of principal results and extensive repetition require substantial editorial revision.

Referee report of 4 Oct 2026. It was read against the current draft of the paper. One automated referee’s estimate — not peer review. Produced by a single automated referee in the pipeline's review stage. It can be wrong, and it is not peer review.

Summary

The paper characterizes minimax separation for conditional mean-effect homogeneity under conditional moment and independent smoothness restrictions, constructs calibrated tests, and establishes matched outcome-envelope and supplied-propensity comparisons. Taking the verification contract as authoritative, the central conclusions are strong and the prose generally preserves their conditions, including saturation and nonempty-power gates. Publication requires a clearer main-results structure, considerable compression, and more focused explanation of the methodological and econometric significance.

Strengths

Findings

major Separation rates and the cost of outcome tails; Supplied propensity scores and assignment information

The principal frontier statements appear near the end of the supplied-propensity appendix, although the main text repeatedly announces an immediately forthcoming result. For example, 'The next result gives matching bounds for the broader-class critical radius' is followed by interpretation rather than the theorem. The same problem affects the heavy-tail comparison, the shared-class oracle frontier, and the preservation boundary. Readers must traverse extensive construction details and appendices to locate the formal answers to the paper's main questions.

Suggested fix

Move the existing heavy-tail comparison environment into the separation-rates section, and move the broader-class frontier, shared-class frontier, and preservation-boundary environments into their corresponding main-text subsections. Preserve their anchors, hypotheses, constants, and power gates. Place interpretation immediately after each statement and retain supporting calculations and proofs in the appendices.

major global

Repeated definitions and repeated explanatory summaries obscure the logical progression. The heterogeneity distance, model suprema, public domains, experiment, and rejection expectation are reintroduced across numerous environments and their surrounding paragraphs. The calibration and mean handles also restate much of the already specified algorithm, while several paragraphs repeat the same size, power-gate, and capping interpretations.

Suggested fix

Establish a single standing-notation passage and use cross-references in surrounding prose. Remove duplicate formula displays immediately preceding environments that contain the same formulas, consolidate repeated interpretations, and streamline the appendix introductions. Retain the complete verified statements where needed, while giving each surrounding paragraph a distinct explanatory purpose.

minor Related work; Supplied propensity scores and assignment information; Discussion and extensions

Several ordinary-prose passages violate the affirmative-scope contract. These include 'it does not replace the variance by a fold-specific proxy or assume a learned first stage,' 'Requirements for a particular estimated-propensity procedure are outside this experiment comparison,' and the statement that practical calibration requires implementation and simulation 'beyond the present analysis.' The last two identify genuine extension or non-coverage content outside the expressly titled limitations subsection.

Suggested fix

Describe the delivered target affirmatively, for example: 'The criterion assesses finite-sample testing risk for the full population variance over the stipulated primitive classes.' Describe the assignment comparison as comparing experiments receiving records alone or records together with the true propensity. Move the estimated-propensity and practical-calibration extension statements into 'Limitations and future work.'

minor Separation rates and the cost of outcome tails

The explanation that 'propensity regularity enters twice through treatment weighting and projection correction, whereas the baseline appears once and is amplified by the tail factor' assigns a precise mechanism to individual denominator terms without displaying the scalar balance that supports that interpretation. The verified statements establish the exponent and covariance bounds; readers need a clearer bridge between those bounds and this explanation.

Suggested fix

Replace the term-by-term verbal attribution with a short tuning argument showing the interaction bias K^{-S}, the relevant coarse-rank factor, and the correction covariance contribution after clipping allocation. Explain how their balance produces the displayed denominator, using the existing bounds and preserving the distinction between representation selection and the statistical phase boundary.

minor Related work

The higher-order CATE paragraph identifies the methodological connection and different loss criterion, but its comparison remains qualitative despite the cited work's explicit minimax phase transition. This is particularly relevant to the paper's independent propensity and baseline exponents and its quadratic-testing mechanism.

Suggested fix

Add a concise, source-scoped comparison reporting the cited pointwise estimation exponents, their smoothness threshold, and the stated attainment conditions. Explain how the present global testing frontier relates to that estimation benchmark while preserving the distinct targets and model restrictions. Verify the theorem and section locators for the version cited.

minor Introduction; Discussion and extensions

The scientific interpretation centers on the standard deviation of the conditional effect, but the economic reason to care about the two comparisons remains abstract. The introduction gives extensive technical detail before developing a concrete use of a scalar effect modifier or explaining when supplied assignment information corresponds to an economically meaningful design.

Suggested fix

Add one concise motivating example of homogeneity across a scalar covariate under the stipulated design. Explain how the target measures variation in conditional mean effects and how records-only versus supplied-propensity experiments inform observational and known-assignment settings. Keep the public-exponent and known-design conditions explicit.

nit Bias, covariance, profiling, and attainment

The sentence 'The same calculation applies under the zero-based convention after translating the block indices' introduces an unexplained alternative convention after the manuscript has consistently adopted one-based blocks. Terms such as 'retained,' 'handle,' and 'ledger' also recur where standard descriptions would communicate the mathematical role more directly.

Suggested fix

Delete the alternative-indexing sentence and use the established block convention throughout. In running prose, prefer 'score decomposition,' 'bias bound,' and 'covariance bound' where these describe the object accurately, retaining formal environment identifiers and notation.

Questions for the authors

Review history

Round Date Score Findings
1 4 Oct 2026 6.8 9
2 4 Oct 2026 7.2 7
3 · current 4 Oct 2026 7.0 7

Every referee draw is kept: scores are noisy, so the history is shown rather than only the last or the best pass.

Read against paper.tex sha256 23054621b508.