Basu's Theorem

Why This Matters

Proving that two statistics are independent usually requires computing their joint distribution and factoring it. This can be painful. Basu's theorem gives a shortcut: if one statistic is complete sufficient and the other is ancillary, they are independent. No joint distribution computation needed.

The classic application: in a normal sample, the sample mean $\bar{X}$ is independent of the sample variance $S^2$. This fact underpins the derivation of the t-test. Basu's theorem proves it in two lines.

Formal Setup

Definition

Ancillary Statistic

A statistic $A(X_1, \ldots, X_n)$ is ancillary for a parameter $\theta$ if and only if its distribution does not depend on $\theta$. It carries no information about $\theta$ by itself, but may carry information in combination with other statistics.

Definition

Complete Statistic

A statistic $T(X_1, \ldots, X_n)$ is complete if and only if for every measurable function $g$:

$\mathbb{E}_\theta[g(T)] = 0 \text{ for all } \theta \implies P_\theta(g(T) = 0) = 1 \text{ for all } \theta$

Completeness means there are no nontrivial unbiased estimators of zero based on $T$. Informally, $T$ contains no "wasted" information.

Main Theorems

Theorem

Basu's Theorem

Statement

If $T$ is a complete sufficient statistic for $\theta$ and $A$ is ancillary for $\theta$, then $T$ and $A$ are independent (under every $P_\theta$).

Intuition

Sufficiency means that the conditional distribution of the data given $T$ does not depend on $\theta$. Ancillarity means the marginal distribution of $A$ does not depend on $\theta$. Completeness forces these two facts to combine into independence: the conditional distribution of $A$ given $T$ must equal the marginal distribution of $A$.

Proof Sketch

Let $B$ be any measurable set. Define $g(t) = P(A \in B \mid T = t) - P(A \in B)$. By sufficiency, $P(A \in B \mid T = t)$ does not depend on $\theta$. By ancillarity, $P(A \in B)$ does not depend on $\theta$. So $g(t)$ does not depend on $\theta$, and $\mathbb{E}_\theta[g(T)] = P_\theta(A \in B) - P(A \in B) = 0$ for all $\theta$ (using ancillarity again). By completeness, $g(T) = 0$ a.s., meaning $P(A \in B \mid T) = P(A \in B)$ a.s. This is independence.

Why It Matters

Without this theorem, proving independence of $\bar{X}$ and $S^2$ in normal sampling requires computing the joint density via a change of variables. With Basu's theorem, you only need three facts: (1) $\bar{X}$ is complete sufficient for $\mu$ when $\sigma^2$ is known, (2) $S^2/\sigma^2$ is ancillary for $\mu$, (3) apply the theorem. This pattern extends to many other settings.

Failure Mode

If the sufficient statistic is not complete, the theorem fails. For example, in a uniform distribution on $[\theta - 1, \theta + 1]$, the order statistics $(X_{(1)}, X_{(n)})$ are sufficient but not complete. The range $X_{(n)} - X_{(1)}$ is ancillary but not independent of the midrange $(X_{(1)} + X_{(n)})/2$.

report a correction →

Canonical Examples

Example

Normal sampling: mean and variance independence

Let $X_1, \ldots, X_n \sim N(\mu, \sigma^2)$ with $\sigma^2$ known. The sample mean $\bar{X}$ is complete sufficient for $\mu$ (this follows from the normal distribution being an exponential family). The statistic $S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X})^2$ has a distribution that depends only on $\sigma^2$, not on $\mu$. So $S^2$ is ancillary for $\mu$. By Basu's theorem, $\bar{X}$ and $S^2$ are independent. This is the fact that makes the t-statistic $(\bar{X} - \mu)/(S/\sqrt{n})$ have a t-distribution.

Example

Exponential distribution: mean and coefficient of variation

Let $X_1, \ldots, X_n \sim \text{Exp}(\lambda)$. The sample sum $T = \sum X_i$ is complete sufficient for $\lambda$. The vector of ratios $(X_1/T, \ldots, X_n/T)$ is ancillary (its distribution is uniform on the simplex, independent of $\lambda$). By Basu's theorem, $T$ is independent of all the ratios $X_i/T$.

Example

Building the t-statistic pivot

Let $X_1, \ldots, X_n \sim N(\mu, \sigma^2)$. Treat $\sigma^2$ as a fixed but arbitrary nuisance and apply Basu within the model where $\mu$ is the parameter of interest at $\sigma^2$ held fixed: $\bar{X}$ is complete sufficient for $\mu$, and any function of the residuals $X_i - \bar{X}$ has a distribution that does not depend on $\mu$ (shift invariance). In particular $S^2$, and so $S/\sigma$, is ancillary for $\mu$. By Basu, $\bar{X} \perp S$ under $P_{\mu, \sigma^2}$ for every $\sigma^2$. The studentized statistic $(\bar{X} - \mu)/(S/\sqrt{n})$ is the ratio of $\sqrt{n}(\bar{X} - \mu)/\sigma \sim N(0, 1)$ and $S/\sigma$, which are independent by Basu and have distributions free of $\mu$ and $\sigma$. The ratio is therefore $t_{n-1}$ distributed for every $(\mu, \sigma^2)$, which is exactly what makes $\bar{X} \pm t_{n-1, 1-\alpha/2} \cdot S/\sqrt{n}$ a valid confidence interval when $\sigma$ is unknown — the cancellation of $\sigma$ in the ratio is what removes the nuisance. Without the independence step, the $t_{n-1}$ distribution of the ratio does not follow from Cochran's theorem alone.

Common Confusions

Watch Out

Ancillary does not mean useless

An ancillary statistic carries no information about $\theta$ by itself. But conditionally, given the ancillary, the precision of estimation can change. This is the basis of conditional inference. Basu's theorem says: if you have a complete sufficient statistic, you cannot improve estimation by conditioning on the ancillary.

Watch Out

Completeness is doing the heavy lifting

Sufficiency alone does not imply independence from ancillary statistics. Completeness is the key condition. Think of completeness as saying the sufficient statistic has no redundancy: there is no function of $T$ that is itself ancillary.

Conditionality and Ancillary Statistics

The conditionality principle (Cox, 1958) says that if $A$ is ancillary for $\theta$, inference about $\theta$ should be carried out in the conditional distribution of the data given $A = a$, not the marginal distribution. The motivation is that $A$ indexes which experiment effectively took place, so reporting unconditional frequency properties averages over experiments the data did not come from.

A textbook case is Cox's two-instrument example. Flip a fair coin: heads uses a precise instrument, tails uses a noisy one. The outcome of the flip is ancillary for the unknown mean. The unconditional variance of the estimator averages both instruments. The conditional variance, given which instrument was used, is what the experimenter actually faces. Most statisticians find the conditional report more honest.

Basu's theorem interacts cleanly with this principle. If $T$ is complete sufficient and $A$ is ancillary, Basu gives $T \perp A$, so conditioning on $A$ leaves the distribution of $T$ unchanged. In that case conditional and unconditional inference about $\theta$ based on $T$ agree. When multiple ancillaries exist or no complete sufficient statistic is available (for example, curved exponential families), conditioning on different ancillaries can yield different answers, and the choice becomes a genuine modeling decision. Fisher pushed for conditioning on a maximal ancillary; uniqueness is not guaranteed.

Watch Out

Conditionality is not universally accepted

The conditionality principle, the sufficiency principle, and the weak likelihood principle together imply the strong likelihood principle (Birnbaum, 1962), which frequentists who prize coverage guarantees often reject. The debate is real. Treat the principle as a defensible default when a natural ancillary exists, not as a theorem.

Summary

• Complete sufficient + ancillary implies independent
• The proof uses completeness to upgrade "same expectation" to "equal a.s."
• The main application is proving independence without computing joint distributions
• Fails without completeness: sufficiency alone is not enough

Exercises

ExerciseCore

Problem

Let $X_1, \ldots, X_n \sim N(\mu, 1)$. Identify a complete sufficient statistic and an ancillary statistic. State what Basu's theorem tells you.

Hint

Reveal Solution

ExerciseAdvanced

Problem

Give an example where a sufficient statistic $T$ and an ancillary statistic $A$ are not independent. What condition of Basu's theorem fails?

Hint

Reveal Solution

References

Primary source:

• Basu, D. "On Statistics Independent of a Complete Sufficient Statistic," Sankhyā 15 (1955), 377-380. The original statement and proof.

Canonical textbooks:

• Casella & Berger, Statistical Inference (2nd ed., 2002), Section 6.2.4 "Sufficiency, ancillarity, and completeness" and Theorem 6.2.24 (Basu's theorem), pp. 287-289. Example 6.2.25 gives the normal mean/variance independence.
• Lehmann & Casella, Theory of Point Estimation (2nd ed., 1998), Section 1.6 "Complete sufficient statistics," Theorem 1.6.22 (Basu), and Example 1.6.23. Chapter 2.1 connects completeness to UMVUE via Lehmann-Scheffé.
• Lehmann & Romano, Testing Statistical Hypotheses (3rd ed., 2005), Sections 4.4-4.5. Canonical treatment of ancillarity, Basu's theorem, and applications to similar tests.
• Cox, D. R. "Some Problems Connected with Statistical Inference," Annals of Mathematical Statistics 29 (1958), 357-372. The conditionality principle and Cox's two-instrument example.

Additional references:

• Keener, Theoretical Statistics: Topics for a Core Course (2010), Chapter 3 (sufficiency, completeness, Basu).
• Schervish, Theory of Statistics (1995), Section 2.1.3 (completeness) and Section 2.2 (ancillarity and conditioning).
• Birnbaum, A. "On the Foundations of Statistical Inference," JASA 57 (1962), 269-306. Derives the likelihood principle from sufficiency and conditionality.

Next Topics

• Fisher information: the natural next step in estimation theory