Random Groups and Finite-Population Variance

The number of realized groups diverges: $$G_n \to \infty .$$

The treated-group fraction $p_n$ converges to an interior limit: $$p_n \to p \in (0,1).$$

The grouped-unit fraction $N_n/n$ converges: $$\frac{N_n}{n} \to \rho \in [0,1].$$

There is a finite uniform outcome envelope $B$ such that $$\sup_{n,i,z,A}\left|Y_{i,n}(z,A)\right| \le B < \infty .$$

Define the finite-population partition-average marginal effect by $$\tau_n=\mathbb E_n\!\left[h_{1,n}-h_{0,n}\right].$$ The difference-in-means estimator across treated and control groups is $$\widehat\tau_n = \frac{1}{G_{1n}}\sum_g Z_{ng}h_{1,n}(A_{ng}) - \frac{1}{G_{0n}}\sum_g (1-Z_{ng})h_{0,n}(A_{ng}).$$

For each arm $z$, let $s_{z,n}^2$ be the within-arm sample variance of the observed group means, computed with denominator $G_{zn}-1$. Define the scalar equal-group CR2 variance statistic by $$\widehat V_{\mathrm{CR2},n} = \frac{s_{1,n}^2}{G_{1n}} + \frac{s_{0,n}^2}{G_{0n}}.$$

Let $M,n\in\mathbb N$ satisfy $2\le M$ and $2M\le n$. Let $Y_{i,n}(z,A)$ be a deterministic potential-outcome schedule for units $i\in A$, arms $z\in\mathcal Z$, and groups $A\in\Omega_{n,M}$, with group-level arm tables $h_{z,n}$ as in Definition P-2. Suppose the following conditions hold.

• (Johnson projections.) For each $k=0,\ldots,M$, $\Pi_{n,k}$ is a linear projection on functions $f:\Omega_{n,M}\to\mathbb R$ whose range is the degree-$k$ Johnson harmonic subspace $\mathcal H_{n,k}$, which fixes every element of $\mathcal H_{n,k}$, and whose residual $f-\Pi_{n,k}f$ is orthogonal to $\mathcal H_{n,k}$ under the uniform-slice inner product $\langle u,v\rangle_n=\mathbb E_n[u(A)v(A)]$.
• (Centered orthogonal decomposition.) For every $f:\Omega_{n,M}\to\mathbb R$ and $A\in\Omega_{n,M}$, $$\sum_{k=1}^{M}\Pi_{n,k}f(A)=f(A)-\mathbb E_n f(A),$$ and the projected components are mutually orthogonal: if $k\ne \ell$, then $$\langle \Pi_{n,k}f,\Pi_{n,\ell}g\rangle_n=0$$ for all $f,g:\Omega_{n,M}\to\mathbb R$.
• (Kneser spectrum.) For every $k=0,\ldots,M$ and every $f\in\mathcal H_{n,k}$, the unnormalized Kneser adjacency operator has eigenvalue $$(-1)^k {n-M-k\choose M-k}.$$

Then the normalized Kneser disjointness operator $\mathsf K_n$ acts on each projected component by $$\mathsf K_n(\Pi_{n,k}f)(A)=\lambda_{n,k}\,\Pi_{n,k}f(A), \qquad \lambda_{n,k}=(-1)^k\frac{(M)_k}{(n-M)_k},$$ for every $k=0,\ldots,M$, every $f:\Omega_{n,M}\to\mathbb R$, and every $A\in\Omega_{n,M}$. Moreover, if $f,g:\Omega_{n,M}\to\mathbb R$ are centered under the uniform $M$-slice, then for a uniform ordered disjoint pair $(A,C)$, $$\operatorname{Cov}\{f(A),g(C)\} = \sum_{k=1}^{M}\lambda_{n,k} \langle \Pi_{n,k}f,\Pi_{n,k}g\rangle_n .$$ Finally, the ordered-disjoint covariance of the arm-difference table satisfies $$C_{\tau\tau,n} = \sum_{k=1}^{M}\lambda_{n,k} \bigl\|\Pi_{n,k}(h_{1,n}-h_{0,n})\bigr\|_n^2 .$$

Let $n,M,G_n,G_{1n}$ be natural numbers and let $G_{0n}=G_n-G_{1n}$. Suppose that:

• (Feasible partition.) The two-stage random partition design in Definition P-1 is feasible: $M G_n\le n$.
• (Group sizes.) $2\le M$, $2\le G_{1n}$, and $2\le G_{0n}$.
• (Potential outcomes.) $Y_{i,n}(z,A)$ is a deterministic potential-outcome schedule, defined for every $A\in\Omega_{n,M}$, every $i\in A$, and every $z\in\mathcal Z$.

Set $$V_{z,n} = \operatorname{Var}_{A\sim\mathrm{Unif}(\Omega_{n,M})}\{h_{z,n}(A)\}, \qquad C_{ab,n} = \mathbb E_{A}\mathbb E_{C\mid C\cap A=\varnothing} \bigl[h_{a,n}^{\circ}(A)h_{b,n}^{\circ}(C)\bigr],$$ and $$C_{\tau\tau,n}=C_{11,n}+C_{00,n}-2C_{10,n}, \qquad R_n=\frac{V_{1,n}}{G_{1n}/G_n}+\frac{V_{0,n}}{1-G_{1n}/G_n}.$$ For the difference-in-means estimator $\widehat\tau_n$, the finite-population PAME $\tau_n$, the exact design variance $\sigma_n^2=\operatorname{Var}(\widehat\tau_n)$, and the scalar equal-group CR2 statistic $\widehat V_{\mathrm{CR2},n}$ in Definition P-7, $$\mathbb E[\widehat\tau_n]=\tau_n,$$ $$\sigma_n^2 = \frac{R_n}{G_n} + C_{\tau\tau,n} - \frac{C_{11,n}}{G_{1n}} - \frac{C_{00,n}}{G_{0n}},$$ $$\mathbb E[\widehat V_{\mathrm{CR2},n}] = \frac{V_{1,n}-C_{11,n}}{G_{1n}} + \frac{V_{0,n}-C_{00,n}}{G_{0n}},$$ and $$\mathbb E[\widehat V_{\mathrm{CR2},n}]-\sigma_n^2=-C_{\tau\tau,n}.$$

Fix a common group size $M\ge 2$ and real numbers $p$, $\rho$, and $B$. There is a constant $C>0$ such that, for every feasible triangular schedule array with common group size $M$ and every rowwise choice of Johnson projections $\Pi_{n,k}$ on $L^2(\Omega_{n,M})$, the following conditions imply the displayed limits below:

• (Group growth.) The array satisfies Assumption A-1.
• (Treatment fractions.) The array satisfies Assumption A-2 with limiting treated-group fraction $p$.
• (Sampling fractions.) The array satisfies Assumption A-3 with limiting grouped-unit fraction $\rho$.
• (Bounded outcomes.) The array satisfies Assumption A-4 with uniform outcome envelope $B$.
• (Johnson decomposition.) In every row with $2M\le n$, the projections give the centered orthogonal Johnson decomposition $$\sum_{k=1}^{M}\Pi_{n,k}f(A)=f(A)-\mathbb E_n f$$ for every $f$ and every $A\in\Omega_{n,M}$, and distinct Johnson degrees are orthogonal under $\langle\cdot,\cdot\rangle_n$.
• (Kneser spectrum.) In every row with $2M\le n$, every $f\in\mathcal H_{n,k}$ is an eigenfunction of the unnormalized Kneser adjacency operator with eigenvalue $$(-1)^k {\,n-M-k\choose M-k\,}.$$

Then $$G_n\sigma_n^2-\bigl(R_n-\rho E_{\tau,1,n}\bigr)\to 0,$$ $$G_nC_{\tau\tau,n}+\rho E_{\tau,1,n}\to 0,$$ and $$\frac{C_{11,n}}{p_n}+\frac{C_{00,n}}{1-p_n}\to 0.$$ Moreover, for every row $n$, the pooled contribution of Johnson degrees $k\ge 2$, $$G_n\sum_{k=2}^{M}\lambda_{n,k} \left\|\Pi_{n,k}\bigl(h_{1,n}-h_{0,n}\bigr)\right\|_{n}^{2},$$ has absolute value at most $C/n$.

Let $M\ge 2$, and let $A$ be a triangular array of feasible random-group experiment rows with population sizes $n$, group counts $G_n>0$, treated-group counts $0<G_{1n}<G_n$, grouped-unit counts $N_n=MG_n\le n$, and deterministic potential-outcome schedules. Fix constants $p,\rho,B,c_\sigma\in\mathbb R$. Suppose that:

• (Dense array.) The array $A$ belongs to the dense bounded schedule class $\mathcal C_{\mathrm{dense}}$ in Definition P-8, with parameters $p,\rho,B,c_\sigma$.
• (Johnson projections.) For every row $n$, the spaces $L^2(\Omega_{n,M})$ are equipped with linear projections $\Pi_{n,k}$, $0\le k\le M$, whose ranges are the degree-$k$ Johnson harmonic subspaces $\mathcal H_{n,k}$, which fix functions in their ranges and have residuals orthogonal to $\mathcal H_{n,k}$ under $\langle\cdot,\cdot\rangle_n$.
• (Orthogonal decomposition.) For every row with $2M\le n$, every $f\in L^2(\Omega_{n,M})$, and every $A\in\Omega_{n,M}$, $$\sum_{k=1}^{M}\Pi_{n,k}f(A)=f(A)-\mathbb E_n f,$$ and distinct Johnson components are orthogonal: $$\langle \Pi_{n,k}f,\Pi_{n,\ell}g\rangle_n=0 \qquad\text{whenever } k\ne \ell .$$
• (Kneser spectrum.) For every row with $2M\le n$, every $0\le k\le M$, and every $f\in\mathcal H_{n,k}$, the unnormalized Kneser adjacency operator satisfies $$\sum_{C\cap A=\varnothing} f(C) = (-1)^k {\,n-M-k\choose M-k\,} f(A).$$

Then, as $n\to\infty$, $$G_n\widehat V_{\mathrm{CR2},n}-R_n \xrightarrow{p}0$$ and $$G_n\bigl(\widehat V_{\mathrm{CR2},n}-\sigma_n^2\bigr) -\rho E_{\tau,1,n} \xrightarrow{p}0 .$$ Moreover, for every $\varepsilon>0$, $$\Pr\!\left\{ \widehat V_{\mathrm{CR2},n}/\sigma_n^2<1-\varepsilon \right\}\longrightarrow 0 ,$$ and the ratio $\widehat V_{\mathrm{CR2},n}/\sigma_n^2$ converges in probability to $1$ if and only if $$\frac{\rho E_{\tau,1,n}}{R_n-\rho E_{\tau,1,n}}\longrightarrow 0 .$$

Fix an integer $M\ge 2$. Then the following statements hold.

• (Sparse dense arrays.) For every feasible triangular schedule array with common group size $M$, if the array belongs to $\mathcal C_{\mathrm{dense}}$ in Definition P-8 with limiting grouped-unit fraction $\rho=0$, then $$\frac{\widehat V_{\mathrm{CR2},n}}{\sigma_n^2}\xrightarrow{p}1 .$$
• (Birthday counts.) With $$N_n^{\mathrm{bd}}=M\Big\lfloor \frac{n^{3/4}}{M}\Big\rfloor ,$$ the benchmark counts satisfy $N_n^{\mathrm{bd}}\le n$ for all sufficiently large $n$, $$\frac{N_n^{\mathrm{bd}}}{n}\to 0, \qquad \frac{(N_n^{\mathrm{bd}})^2}{n}\to\infty .$$
• (Birthday-aligned arrays.) For every feasible triangular schedule array with common group size $M$, if its grouped-unit count satisfies $$N_n=M\Big\lfloor \frac{n^{3/4}}{M}\Big\rfloor$$ row by row and the array satisfies Assumption A-1, Assumption A-2, Assumption A-4, Assumption A-5 for some $p$, $B$, and $c_\sigma$, then $$\frac{\widehat V_{\mathrm{CR2},n}}{\sigma_n^2}\xrightarrow{p}1 .$$

Fix a common group size $M$ with $2\le M$, and let $A$ be a triangular schedule array with population sizes $n$, group counts $G_n>0$, treated-group counts $0<G_{1n}<G_n$, $M G_n\le n$, and deterministic potential-outcome rows. Let $p,\rho,B,c_\sigma\in\mathbb R$. Suppose:

• (Envelope and fractions.) $0<\rho$, $1\le B$, and the array satisfies Assumption A-1, Assumption A-2, Assumption A-3 with limiting treated-group fraction $p$ and limiting grouped-unit fraction $\rho$.
• (Johnson decomposition input.) For every row $n$, there are orthogonal projections $\Pi_{n,k}$ onto the degree-$k$ Johnson harmonic spaces $\mathcal H_{n,k}$, $k=0,\ldots,M$, whose positive-degree components give the centered orthogonal decomposition of every $f\in L^2(\Omega_{n,M})$: $$\sum_{k=1}^{M}\Pi_{n,k}f = f-\mathbb E_n f, \qquad \langle \Pi_{n,k}f,\Pi_{n,\ell}g\rangle_n=0 \quad\text{for }k\ne \ell .$$
• (Kneser spectrum input.) For every row $n$ with $2M\le n$, the Kneser adjacency operator acts on $f\in\mathcal H_{n,k}$ by the eigenvalue $$(-1)^k {\,n-M-k\,\choose\, M-k\,}, \qquad k=0,\ldots,M .$$
• (Variance lower bound.) With $$d_{\mathrm{same}}=\frac{1}{M p(1-p)}, \qquad d_{\mathrm{ind}}=\frac{1}{M p(1-p)}-\frac{2\rho}{M},$$ the constant $c_\sigma$ satisfies $$0<c_\sigma<d_{\mathrm{ind}} .$$

Then there exist support sets $\Gamma_n^{\mathrm{same}}$ for the common-arm Rademacher prior $\mathbb H_n^{\mathrm{same}}$ and $\Gamma_n^{\mathrm{ind}}$ for the independent-arm Rademacher prior $\mathbb H_n^{\mathrm{ind}}$ such that both prior probabilities tend to one, every diagonal selection from either support induces an array in $\mathcal C_{\mathrm{dense}}$ with parameters $(p,\rho,B,c_\sigma)$, the conditioned one-realization laws satisfy $$\operatorname{TV}\!\left( \mathbb H_n^{\mathrm{same}}(\mathcal O_n\in\cdot\mid \Gamma_n^{\mathrm{same}}), \mathbb H_n^{\mathrm{ind}}(\mathcal O_n\in\cdot\mid \Gamma_n^{\mathrm{ind}}) \right)\to 0,$$ and the scaled exact variances converge uniformly on the respective supports: $$\sup_{u\in\Gamma_n^{\mathrm{same}}} \left|G_n\sigma_n^2(u)-d_{\mathrm{same}}\right|\to 0, \qquad \sup_{u\in\Gamma_n^{\mathrm{ind}}} \left|G_n\sigma_n^2(u)-d_{\mathrm{ind}}\right|\to 0 .$$ Moreover, there is an $\varepsilon>0$ such that every measurable one-realization variance statistic $\widehat S_n(\mathcal O_n)$ satisfies $$\frac12 \le \liminf_{n\to\infty} \sup_{Y\in\mathcal C_{\mathrm{dense}}} \Pr_Y\!\left( \left| \frac{\widehat S_n(\mathcal O_n)}{\sigma_n^2}-1 \right|>\varepsilon \right).$$