Gerhard Tintner, A Note on the Relation Between Mahalanobis Distance and Weighted Regression (1965)

Gerhard Tintner’s contribution to an edited statistical Festschrift is a short mathematical note connecting a measure of multivariate separation with the estimation of linear relations among variables observed with error. Its central move is to recognize the same inverse-covariance quadratic form in Mahalanobis distance and in the residual criterion of weighted regression. From that connection, Tintner proceeds to a generalized eigenvalue problem, approximate tests for the number of underlying relations, and an information-theoretic interpretation of the resulting estimates. The note synthesizes established results and earlier work rather than supplying a lengthy proof of each step.

Tintner begins by specifying the population setting:

Let us consider two normal p-dimensional populations. Let δ be the vector of the differences of the population means and Σ the population variance-covariance matrix.

He defines squared Mahalanobis distance as (\Delta^2=\delta'\Sigma^{-1}\delta/p). The inverse covariance matrix makes separation depend on the joint variability of the observations, rather than on unweighted differences between coordinates. This provides the conceptual basis for the subsequent comparison: a displacement is assessed relative to the covariance structure within which it occurs. Tintner introduces sample estimates of the mean differences and covariance matrix and notes the connection between the empirical distance and Hotelling’s statistic. The supplied text’s equation (2), however, retains (\delta) as its final vector after introducing (d) as the sample replacement. That notation should be checked rather than silently treated as an unambiguous sample formula.

The regression model shifts attention from differences between population means to differences between observations and their systematic components. For each time (t), both the observed vector (x_t) and the systematic vector (\mu_t) are expressed as deviations from their arithmetic means, with (Ex_t=\mu_t). Tintner assumes that the systematic components satisfy (\rho) independent homogeneous linear relations, (\kappa'_v\mu_t=0). The covariance matrix (\Sigma) now describes observation errors. Thus the problem concerns relations among underlying components of several error-bearing variables, rather than simply fitting an error-free explanatory variable to a dependent variable.

The estimation principle brings the two subjects together:

Assuming normality, the method of maximum likelihood is identical with the method of least squares if we want to estimate $\rho$ and the vectors $\kappa_v$.

Tintner minimizes a sum of quadratic residual terms, each of the form ((x_t-m_t)'\Sigma^{-1}(x_t-m_t)), where (m_t) estimates the systematic component. Apart from the normalization by (p) in the distance definition, this has the same structure as the Mahalanobis expression. Weighted regression therefore evaluates discrepancies from fitted systematic vectors through the same covariance-sensitive operation used to evaluate differences between population means. When the error covariance is unknown, Tintner substitutes an estimate (V), mentioning the variate difference method as one possible means of obtaining it. The relevant weights derive from the covariance of errors, not merely from the covariance of the observations.

The next stage turns this residual minimization into a spectral estimation procedure. Estimated coefficient vectors (k_v) are normalized and made mutually orthogonal through (k'vVk_w=\delta{vw}). Tintner then expresses each minimized residual contribution as a sum of squared projected observations, (\sum_v(k'_vx_t)^2). With additional orthogonality conditions on these projections, he obtains ((A-\lambda V)k_v=0), where (A) is the sample covariance matrix of the observations. Nontrivial coefficient vectors require (|A-\lambda V|=0). The resulting roots compare observed variation with error variation; the smallest roots supply the candidate linear relations. This is the operational payoff of the initial analogy: the covariance-weighted measure leads to a method for estimating both the relations and their number.

To determine that number, Tintner orders the roots and forms (\Lambda_r=(N-1)(\lambda_1+\cdots+\lambda_r)). He reports an approximate large-sample chi-square distribution with ((N-p-1+r)r) degrees of freedom, and also gives Anderson’s normal approximation for the standardized statistic. His selection rule is explicitly sequential:

Now an approximate estimate of $\rho$ can be derived as follows: We test successively $\Lambda_1, \Lambda_2 \dots$. If $\Lambda_R$ is not significant at a chosen level of significance, but $\Lambda_{R+1}$ is significant, we have $R$ as an estimate of $\rho$.

The procedure retains a set of candidate relations until adding the next produces a significant statistic. Tintner then uses the (R) smallest roots to recover the corresponding coefficient vectors. The note presents this as approximate inference, dependent on a chosen significance level and the stated distributional approximations. A further cross-reference needs checking: the closing estimation paragraph calls equation (12) a linear system, although the displayed linear system is equation (11) and equation (12) is its determinant condition.

The final paragraph broadens the interpretation:

In a recent paper (Tintner, 1960) it has also been shown that our estimates are optimal in the sense of information theory as proposed by Kullback (1959); they discriminate in this sense most successfully between the population of the observations and the errors.

This optimality claim is attributed to earlier work, not demonstrated within the note. It nevertheless clarifies the conceptual trajectory: covariance-weighted distance, estimation of systematic relations, and discrimination between observations and errors become connected aspects of one procedure. The contribution’s relevance lies in this compact bridge between multivariate distance and error-sensitive regression, with generalized eigenvalues providing the link from a geometric criterion to statistical estimation and testing.