Harold Hotelling Research Paper

1. Early Life

Born on September 29, 1895 in Fulda, Minnesota, Harold Hotelling, whose parents were Clair Hotelling and Lucy Rawson, was the oldest of six children. Having moved with his family to Seattle at age nine, Hotelling studied journalism at the University of Washington, taking time from his studies to work on a weekly paper. In 1919, he received his BA in Journalism, graduating Phi Beta Kappa. Encouraged, in part, by the mathematician and well-known historian, Eric Temple Bell, he went on to graduate study in mathematics at the University of Washington, receiving an MA in 1921. It was during this time that he married Floy Tracy in 1920. He received his Ph.D. in Mathematics in 1924 from Princeton University, doing dissertation research on dynamic topology with Oswald Veblen, although his original intent was to study mathematical economics. His interest in economics took root while growing up and in his belief in ‘the importance … of a study of economics [in] that [it] might possibly impress some people to the point of making some changes in the institutions with which they had become familiar [and] which I [Hotelling] was already prepared to condemn vigorously.’ Alternatively, he ‘couldn’t escape the lure of mathematics’ and notes that he ‘hoped and [it] appeared that the mathematics would be extremely prolific of new economic theorems and ideas if followed out’ (Hotelling 1963).

2. Beginning At Stanford University

Invited to join the Food Research Institute at Stanford University, Hotelling was a research associate there from 1924 to 1927, following which he was an associate professor in Stanford’s Department of Mathematics (1927–31). An early indication of the breadth of his intellectual interests is illustrated by the fact that in 1927 he taught courses on mathematical statistics, differential geometry, and analysis situs (a forerunner of topology), and in that year published papers on the application of the latter to statistics, and on differential equations subject to error. While working in the Food Research Institute, he was introduced to the work of R. A. Fisher (five years his elder), and through their mutual correspondence they became friends. In fact, Hotelling wrote the book review in the Journal of the American Statistical Association (Hotelling 1927) of the first edition of Fisher’s Statistical Methods for Research Workers (and also did subsequent reviews there of the Second through Seventh Edns.). After the planned visit of Fisher to Stanford in the summer of 1928 was abandoned, Hotelling visited Fisher at the Rothamsted Experimental Station during the second half of 1929. Their connections continued and at the outbreak of World War II, Hotelling, along with three others, wrote to Fisher volunteering to care for Fishers’ six daughters in the United States during the war. However, a last minute cancellation kept the Fisher children in Britain (Box 1978, p. 377).

While at Stanford, Hotelling wrote a number of path-breaking papers in statistics and in mathematical economics. The most notable among these statistical results was his generalization of Student’s t-ratio to handle multivariate response data (Hotelling 1931 ). This generalization remains known as Hotelling T . His paper with Holbrook Working (Working and Hotelling 1929) on modeling and linear prediction first demonstrated that the standard error of the estimate of a linear regression depended on the independent variable’s value, thereby giving rise to the well-known Working–Hotelling curved simultaneous confidence regions for a linear regression. His research contributions to mathematical economics were equally notable. His 1929 paper provided a game-theoretic solution given by two differential equations to a problem of price and location competition between two entities in a spatial setting. His 1931 paper on the economics of exhaustible resources is recognized as the seminal paper in the field of natural resource economics. In that paper, Hotelling demonstrates that in an equilibrium setting, prices of exhaustible natural resources will tend to rise over time at a percentage rate equaling the interest rate.

3. Columbia University And The Statistical Research Group

Hotelling moved to Columbia University in 1931 as a Professor of Economics, having been recruited to further the development of mathematical economics, and also to initiate mathematical statistics. Throughout his career at Columbia and later at the University of North Carolina at Chapel Hill, Hotelling was superlative at both attracting outstanding researchers and mentoring young researchers destined for stardom. The Nobel-prize winning economist, Milton Friedman, was among the earliest mentored by Hotelling, spending 1933–4 at Columbia learning mathematical economics from him. Later mentoring Kenneth Arrow, another Nobel-prize winning economist, Hotelling was also instrumental in attracting Abraham Wald, W. Allen Wallis, and Jacob Wolfowitz to the statistics facility at Columbia. As a teacher, Hotelling was perceived to be organized and comprehensive, interweaving theory with application. Importantly, an appreciation of research was imparted to his students, and he was able at an interpersonal level to instill in his students their own confidence to be successful at research (Madow 1960).

Hotelling was one of three original voting fellows of the Institute of Mathematical Statistics on September 12, 1935, and was President of that Institute in 1941. He began his association with the Cowles Commission, participating in its first Summer conference in 1935, and served as President of the Econometric Society during 1936–7. In 1939–40, he was a Visiting Lecturer at the Indian Statistical Institute and was President of the Indian Statistical Congress (1940).

Along with Wallis and Wolfowitz, Hotelling was a charter member of the Statistical Research Group (SRG), at Columbia, for which Wallis was appointed Director of Research. The SRG, founded on July 1, 1942, and in existence until September 30, 1945, was an extraordinary research group of statisticians whose goals were to provide research support and analyses for the armed forces in furthering the quality and accuracy of their war efforts. The origins of sequential analysis stem from Wald’s work for the SRG, and among Hotelling’s contributions were techniques for control charts for multivariate data (Kruskal 1980). (A detailed history of the SRG is provided by Wallis 1980).

Early in Hotelling’s career at Columbia, his wife, Floy, died in 1932, leaving behind two small children. In 1934, Hotelling married Susanna Edmundson and together they had six children, five sons and a daughter (who died in her infancy).

Hotelling, during his prewar years at Columbia, continued to produce ground-breaking research in statistics and economics. His 1933 and 1936 papers introduced and lay the groundwork for principal component analysis as a multivariate statistical technique for simplifying the structure of large numbers of correlated measures (Hotelling 1933, 1936a). In 1936, he extended notions of multiple correlation to canonical correlations in order to measure and study the relationships between two dependent sets of correlated responses (Hotelling 1936b). In the field of economics, his 1935 paper provided a consistent theory uniting demand and utility by means of a constrained optimization problem and his presidential address to the Econometric Society in 1938 (Hotelling 1938) introduced a welfare equilibrium proposition which leads in certain societal settings to the argument that goods be priced at their marginal costs. In 1940, his innovative and insightful paper on the teaching of statistics in universities was published (Hotelling 1940). In this prescient paper, Hotelling observes the inevitability of the increased demand for statisticians, and argues that the teaching of statistics in universities would be best done solely by statisticians in separate departments of statistics (a quite novel idea in 1940!). With regard to this research paper, Jerzy Neyman (1960) later wrote ‘in my opinion the establishment of the excellent departments of statistics in such universities as Chicago, Columbia, Harvard, North Carolina, and Stanford is due to a considerable extent to the publication of Hotelling’s article.’

4. Career At University Of North Carolina

After the war in 1946, Hotelling was recruited by Frank Graham, President of the University of North Carolina at Chapel Hill to start the Department of Mathematical Statistics there upon his arrival. In addition to being appointed professor of mathematical statistics and professor of economics, he was named Associate Director of the Institute of Statistics (directed by Gertrude Cox), which was constituted with Chapel Hill’s department and the Department of Experimental Statistics at North Carolina State, as well as related groups in psychometrics, social sciences and biostatistics. It is notable that when Hotelling was planning to go to Chapel Hill, he proposed that Wald join him there. In order to entice Wald to remain at Columbia, a Department of Mathematical Statistics was promptly proposed and officially approved in December 1946 (Hoxie 1955). As at Columbia, Hotelling attracted outstanding statisticians to the Chapel Hill faculty including R. C. Bose, Wassily Hoeffding, P. L. Hsu (who had been at Columbia with Hotelling), William Madow, Herbert Robbins, S. N. Roy, and Walter Smith. Hotelling relinquished the chairman- ship to George Nicholson in 1952, and devoted the remainder of his career there to his research and his students.

His research at North Carolina involved, on the whole, elaborations and in-depth further study of some of his earlier research. Noteworthy among these is his work in 1947 and 1951 on hypothesis tests related to multivariate analysis of variance, which lead to the testing criterion referred to as the Lawley–Hotelling trace. He formally retired in 1966, but continued to be active in the department. He was elected to the National Academy of Sciences in 1970, and received a number of other academic honors during this period. Subsequent to a serious stroke in mid-1972, Harold Hotelling died in Chapel Hill, North Carolina on December 26, 1973.

5. Hotelling’s Contribution To Multivariate Statistical Analysis

One of the origins of statistical hypothesis testing is a test that the mean µ of a normal distribution has a specified value µ₀. The resultant Student’s t-test, obtained in 1908, remains as one of the most used statistical procedures. The t-statistic is

where x and s, respectively, represent the mean and standard deviation of a sample of size N. Hotelling recognized that many experiments will have correlated measures x₁, …, x_p, which arise from a multivariate normal distribution, and he provided the Hotelling’s T² test to handle the means of multiple measurements (Hotelling 1931). For a sample of size N let x=(x₁, …, x_p) denote the sample mean vector and S=(s_ij) represent the sample covariance matrix.

The T²-statistic is

where µ₀= (µ₁, …, µ_p ) is the specified mean vector in the hypothesis. When the hypothesis is true, (N-p)T²/ p(N-1) has an F-distribution with p and N-p degrees of freedom.

Analogous to Student’s t-test, the T -statistic can also be used to test for the equality of the mean vectors of two normal distributions that have the same unknown covariance matrix but unequal means, µ and ν. Now the statistic becomes

where x and y are mean vectors from samples of size N₁ and N₂, respectively, and S represents the pooled sample covariance matrix. Although there are many parameters in this model, under the null hypothesis the distribution of (N₁+ N₂-p-1)T²/p has an F-distribution w ith p and N₁+ N₂– p -1 d.f.

The T²-statistic additionally arises as a linear discriminant, as noted by Fisher, and is the sample analog of the Mahalanobis squared distance.

Quite remarkably, H otelling noted that for the two- sample problem, the T² statistic is invariant under the transformation x→xA, y→yA, where A is any nonsingular matrix. This anticipated a general theory of statistical inva riance developed much later. In particular, the T² -test has a number of optimal properties among invariant tests: (a) the T² -statistic is the only invariant function of the sufficient statistics x and S; (b) as a test statistic it is the uniformly most powerful test among all invariant tests; and (c) the noncentral distribution depends on the scalar parameter ( µ-ν) Σ⁻¹( µ-ν) , which is the only invariant of the parameters.

Large data sets either in exploratory data analysis, data mining, or model building are accompanied by a large number of variables. The reduction of numbers of variables is often a first step in trying to uncover patterns in the data. Because multivariate measures x₁, …, x_p are related, there is the need to disentangle the correlational structure. To accomplish this, Hotelling (1933, 1936a) introduced the p principal components z₁, …, z_p, which are linear combinations of the underlying measures, namely z_k =Σ^p_1-1a_k1x₁, k=1 , …, p, where the coefficients are normalized by ∑^p_1-1 a_ki²=1. The first set of coefficients a_l, …, a_1p is chosen so as to maximize the variance of z₁. The second set of coefficients a_2l… , a_2p is chosen to maximize the variance of z subject to the constraint that z₁ and z₂ are uncorrelated. At each step the variance is maximized, but with the additional constraints that each z_k be uncorrelated with z₁, …, z_k−1. Their ordered variances Var (z₁)≥…≥Var (z_p) turn out to be the characteristic roots of the population covariance matrix.

In effect, principal components provide transformed coordinates that are uncorrelated, thereby disentangling the measures. Because the principal components are ordered according to their variances, this permits the researcher to focus on a subset of principal components that contributes substantially to the total variance. In multiple regression, the predictor variables often are closely related, and the problem of multicollinearity results. Principal components is one important procedure that introduces orthogonality and reduces dimensionality, thus removing multicollinearity.

To build a hierarchy of correlational measures one can start with the ordinary Pearson product moment correlation between x and y. With multiple measures x₁, …, x_p and y one generates the multiple correlation of y and the linear combination y* =Σa_ix_i, which is the maximum correlation between y and y*. Suppose now there are multiple measures x₁, …, x_p and y₁, …, y_q. To analyze the relation between these two sets of variates, Hotelling (1936b) defined canonical correlations between u_k and v_k, which are the correlations of linear combinations u_k = Σa_kj x_j and v_k = Σb_kj y_j where k =1, 2, …, K min(p, q). The coefficients are normalized so that the variances of u_k and v_k are 1, and are chosen so as to maximize the correlations, and such that u₁, …, u_k are mutually uncorrelated and v₁, …,v_k are mutually uncorrelated k=1,…, K. The new variables u_k and v_k, k =1, …, k are called the canonical variables; furthermore, it turns out that u_i and v_j(i =j) are uncorrelated. The discovery of the corresponding matrix factorization is quite remarkable. A special case of this factorization leads to the singular value decomposition, which is a very important representation in numerical linear algebra.

As a result of this decomposition, one can transform the original variables to new coordinates (u₁,v₁), …, (u_K, v_K), that exhibit a simple structure: (a) the pairs are uncorrelated; (b) the pairs are ordered decreasingly in accordance to their correlations (which are non-negative); that is, (u₁, v₁) are maximally correlated; and (c) the variances of the uk’s and k’s are normalized to be one. In a 1935 paper, Hotelling had earlier focused on the first pair (u₁,v₁), and he called u₁ the most predictable criterion and the best predictor.

Principal components and canonical correlations can be viewed in terms of roots of a determinantal equation and estimated from their sample analogues. It was this context that opened an entire area of statistical research on the distribution of the roots of random determinantal equations. In 1939, a number of papers appeared simultaneously in the statistical literature in which the central distribution of the roots were obtained. From then until the present, there is a continuing flow of theory concerning the noncentral distributions of roots of sample determinantal equations.

Hotelling’s training as a mathematician provided him with a warehouse of tools that he applied to statistical problems. Among his first papers was a presentation in 1926 to the American Mathematical Society on the use of topological principles in the derivation of distributions. Many of his later results were obtained with geometrical arguments. He was able to exploit asymptotic theory to obtain normal approximations and limiting results. Hotelling, in 1930, was the first to attempt to state precise conditions for the consistency and asymptotic normality of the maximum likelihood estimator. With Lester Frankel in 1938, he obtained expansions for the distribution of Student’s t-statistic, thereby leading to a useful normal approximation.

With his interest in relations between variables, Hotelling recognized that the required normality assumptions may be unrealistic. This led him to study the nonparametric version of the product moment correlation. Together with Margaret Pabst, Hotellingstudied the Spearman rank correlation coefficient and they showed that for large samples it approximately follows a normal distribution. In 1953, Hotelling continued his analysis of correlations and obtained expansions of Fisher’s z-transformation of Pearson’s product moment correlation. This work remains among the definitive results in this area.

Hotelling wrote on matrix calculations and developed methods that still stand. His work on weighing designs led to a number of combinatorial generalizations. Although Hotelling had a mathematical focus for his research, he was not averse to working on applied problems. The range of his applied papers include results concerning the physical state of protoplasm, duration of pregnancy, causes of birth rate fluctuations, anthropomorphic measures of Southwest Indians, and effects of pregnancy on dental caries.

Harold Hotelling’s originality and imagination is seen in both the genius of his research that produced innovative methods that stimulated whole areas of modern statistics and in his prescient leadership in the discipline of statistics. Further details concerning Hotelling, including his complete Bibliography:, can be found in Olkin et al. (1960), in Pfouts (1960), and in three articles appearing as ‘Three papers in honor of Harold Hotelling at 65’ in The American Statistician (14), published in 1960 (pp. 15–25).

Bibliography:

1. Box J F 1978 R. A. Fisher: The Life of a Scientist. Wiley, New York
2. Hotelling H 1927 Review of ‘Statistical methods for research workers. ’Journal of the American Statistical Association 22: 411–12
3. Hotelling H 1931 The generalization of Student’s ratio. Annals of Mathematical Statistics 2: 360–78
4. Hotelling H 1933 Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24: 417–41 and 498–520
5. Hotelling H 1936a Simplified calculation of principal components. Psychometrika 1: 27–35
6. Hotelling H 1936b Relations between two sets of variates. Biometrika 27: 321–77
7. Hotelling H 1938 The general welfare in relation to problems of taxation and of railway and utility rates. (Presidential address to the Econometric Society). Econometrica 6: 242–69
8. Hotelling H 1940 The teaching of statistics. Annals of Mathematical Statistics 11: 457–70
9. Hotelling H 1963 ‘Remarks by Harold Hotelling at a Seminar in which Honorary Degree Recipients were asked to give brief intellectual biographies.’ As transcribed from a tape recording — not corrected or edited. University of Rochester. Private Communication
10. Hoxie R G 1955 The Department of Mathematical Statistics. In: A History of the Faculty of Political Science, Columbia University. Columbia University Press, New York, pp. 250–5
11. Kruskal W H 1980 Comment: First interaction with Harold Hotelling; testing the Norden bombsight. Journal of the American Statistical Association 75: 331–33
12. Madow W G 1960 Harold Hotelling as a teacher. American Statistician 14: 15–17
13. Neyman J 1960 Harold Hotelling: A leader in mathematical statistics. In: Olkin I, Ghurye S G, Hoeffding W, Madow W G, Mann H B (eds.) Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling. Stanford University Press, Stanford, CA, pp. 6–10
14. Olkin I, Ghurye S G, Hoeffding W, Madow W G, Mann H B 1960 Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling. Stanford University Press, Stanford, CA
15. Pfouts R W 1960 Essays in Economics and Econometrics: A Volume in Honor of Harold Hotelling. University of North Carolina Press, Chapel Hill, NC
16. Wallis W A 1980 The Statistical Research Group, 1942–45. Journal of the American Statistical Association 75: 320–30
17. Working H, Hotelling H 1929 Applications of the theory of error to the interpretation of trends. Journal of the American Statistical Association 24(March supp.): 73–85