History Of Sample Surveys Research Paper

This research paper considers the development of methods for the sampling of human populations. We begin by describing two antecedents of modern survey methods—the development of government statistics and censuses and the general development of statistical methodology, especially for ratio estimation. Then we deal with the movement from censuses to sample surveys, examining the debate over representative methods and the rise of probability sampling, including the seminal 1934 paper of Jerzy Neyman. We describe the impact in the United States of the results obtained by Neyman on sample surveys and the contributions of statisticians working in US government statistical agencies. We end with a brief description of the subsequent establishment of random sampling and survey methodology more broadly and examine some history of the movement from the study of the design of sampling schemes to the study of other issues arising in the surveying of human populations.

1. Antecedents Of Modern Sample Survey Methods

In order to determine facts about their subjects or citizens, and more often about their number, governments have long conducted censuses. Thus, one set of roots of sample survey methodology is intertwined with the history of methods for census-taking. The origins of the modern census are found in biblical censuses described in the Old Testament (e.g., see the discussions in Duncan (1984) and Madansky (1986) as well as in censuses carried out by the ancient Egyptians, Greeks, Japanese, Persians, and Romans (Taeuber (1978)). For most practical purposes we can skip from biblical times to the end of the eighteenth century and the initiation of census activities in the United States of America, even though there is some debate as to whether Canada, Sweden, or the United States should be credited with originating the modern census (Willcox 1930).

Another antecedent of sampling lies in the problem of estimating the size of a population when it is difficult or impossible to conduct a census or complete enumeration. Attempts to solve this problem even in the absence of formal sampling methods were the inspiration of what we now know as ratio estimation. These ideas emerged as early as the seventeenth century. For example, John Graunt used the technique to estimate the population of England from the number of births and some assumptions about the birth rate and the size of families. Graunt’s research, later dubbed political arithmetic, was based on administrative records (especially parish records) and personal observation, as in his seminal work, Natural and Political Observations Made Upon the Bills of Mortality, published in 1662. To accomplish what Graunt did by assumption, others looked to hard data from subsets of a population. Thus, Sir Frederick Morton Eden, for example, found the average number of people per house in selected districts of Great Britain in 1800. Using the total number of households in Great Britain from tax rolls (and allowing for those houses missing from the rolls), Eden estimated the population of Great Britain as nine million, a figure confirmed by the first British census of 1801 (Stephan (1948).

Even earlier, in 1765 and 1778, civil servants published population estimates for France using an enumeration of the population in selected districts and counts of births, deaths, and marriages, for the country as a whole (see Stephan (1948). It was Pierre Simon de Laplace, however, who first formally described the method of ratio estimation during the 1780s, and he then employed it in connection with a sample survey he initiated to estimate the population of France as of September 22, 1802. He arranged for the government to take a sample of administrative units (communes), in which the total population, y, and the number of registered births in the preceding year, x, were measured. Laplace then estimated the total population of France by Y = Xy/x, where X was the total number of registered births. Laplace was the first to estimate the asymptotic bias and variance of Y, through the use of a superpopulation model for the ratio p = y/x (see Laplace (1814)). Cochran (1978) describes this derivation and links it to results which followed 150 years later. Quetelet later applied the ratio estimation approach in Belgium in the 1820s but abandoned it following an attack by others on some of the underlying assumptions (see Stigler 1986, pp. 162–6).

2. From Censuses To Surveys

The move from censuses to sample surveys to measure the characteristics of a nation’s population was slow and laborious. Bellhouse (1988), Kruskal and Mosteller (1980), and Seng (1951) trace some of this movement, especially as it was reflected in the discussions regarding surveys that took place at the congresses of the International Statistical Institute (ISI). The move succeeded when investigators combined random selection methods with the structuring of the population and developed a theory for relating estimates from samples, however structured, to the population itself.

As early as the 1895 ISI meeting, Kiaer (1896) argued for a ‘representative method’ or ‘partial investigation,’ in which the investigator would first choose districts, cities, etc., and then units (individuals) within those primary choices. The choosing at each level was to be done purposively, with an eye to the inclusion of all types of units. That coverage tenet, together with the large sample sizes recommended at all levels of sampling, was what was judged to make the selection representative. Thus the sample was, approximately, a ‘miniature’ of the population. Kiaer used systematic sampling in his 1895 survey of workers as a means of facilitating special tabulations from census schedules for a study of family data in the 1900 Norwegian census. Kiaer actually introduced the notion of random selection in his description of this work by noting that at the lowest level of the sample structure ‘the sample should be selected in a haphazard and random way, so that a sample selected in this manner would turn out in the same way as would have been the case had the sample been selected through the drawing of lots … ’ (Kiaer 1897, p. 39)

The idea of less than a complete enumeration was opposed widely, and Kiaer presented arguments for sampling at ISI meetings in 1897, 1901, and 1903. Lucien March, in a discussion of Kiaer’s paper at the 1903 meeting, formally introduced the concepts of simple random sampling without replacement and simple cluster sampling, although not using these names (Bellhouse 1988).

Arthur Lyon Bowley did a number of empirical studies on the validity of random sampling, motivated at least in part by a 1912 paper by Franicis Ysidro Edgeworth (1912). Bowley used large sample normal approximations (but did not conceptualize what he was doing as sampling from a finite population) to actually test the notion of representativeness. Then, he carried out a 1912 sample survey on poverty in Reading, in which he drew the respondents at random (see Bowley 1913), although he appears to have equated simple random sampling with systematic sampling (Bellhouse 1988).

At about the same time, in what appears to be an independent line of discovery, Tchouproff and others were overseeing the implementation of sample survey methods in Russia, especially during the First World War and the period immediately following it. This work was partially documented after the Russian Revolution in Vestnik Statistiki, the official publication of the Central Statistical Administration (Zarkovic 1956, 1962). Even though there is clear evidence that Tchouproff had developed a good deal of the statistical theory for random sampling from finite populations during this period (see Seneta 1985), whether he actually implemented forms of random selection in these early surveys is unclear. He later published the formulae for the behavior of sample estimates under simple random sampling and stratified random sampling from finite populations in Tchouproff (1918a, 1918b, 1923a, 1923b).

In a seemingly independent development of these basic ideas for sampling in the context of his work on agricultural experiments, Neyman in Splawa-Neyman (1923) described them in the form of the drawing of balls without replacement from an urn. In the resulting 1925 paper, Splawa-Neyman (1925) gave the basic elements of the theory for sampling from finite populations and its relationship with sampling from infinte populations. These results clearly overlapped those of Tchouproff, and two years later following his death, his partisans took Neyman to task for his lack of citation to the Russian work as well as to earlier work of others (see Fienberg and Tanur 1995, 1996) for a discussion of the controversy).

By 1925, the record of the ISI suggests that the representative method was taken for granted, and the discussions centered around how to accomplish representativeness and how to measure the precision of sample-based estimates, with the key presentations being made by Bowley (1926) and in the same year by the Danish statistician Adolph Jensen. Notions of clustering and stratification were put forward, and Bowley presented a theory of proportionate stratified sampling as well as the concept of a frame, but purposive sampling was still the method of choice. It was not until Gini and Galvani made a purposive choice of which returns of an Italian census to preserve, and found that districts chosen to represent the country’s average on seven variables were, in that sense, unrepresentative on other variables, that purposive sampling was definitively discredited (Gini 1928, Gini and Galvani 1929).

3. Neyman’s 1934 Paper On Sampling

In their work on the Italian census, Gini and Galvani seemed to call into question the accuracy of sampling. Neyman took up their challenge in his classic 1934 paper presented before the Royal Statistical Society, ‘On the two different aspects of the representative method.’ In it he compared purposive and random sampling, and concluded that it wasn’t sampling that was problematic but rather Gini and Galvani’s purposive selection.

Elements of synthesis were prominent in the paper as well. Neyman explicitly uncoupled clustering and purposive sampling, saying, ‘In fact the circumstance that the elements of sampling are not human individuals, but groups of these individuals, does not necessarily involve a negation of the randomness of the sampling’ (1952, p. 571). He calls this procedure ‘random sampling by groups’ and points out that, although Bowley did not consider it theoretically, he used it in practice in London, as did O. Anderson in Bulgaria. Neyman also combined stratification with clustering to form ‘random stratified sampling by groups,’ and he provided a method for deciding how best to allocate samples across strata (optimal allocation). The immediate effect of Neyman’s paper was to establish the primacy of the method of stratified random sampling over the method of purposive selection, something that was left in doubt by the 1925

ISI presentations by Jensen and Bowley. But the paper’s longer-term importance for sampling was the consequence of Neyman’s wisdom in rescuing clustering from the clutches of those who were the advocates of purposive sampling and integrating it with stratification in a synthesis that laid the groundwork for modern-day multistage probability sampling.

Surprisingly, for many statisticians the memorable parts of Neyman’s paper were not these innovations in sampling methodology but Neyman’s introduction of general statistical theory for point and interval estimation, especially the method of confidence intervals.

As pathbreaking as Neyman’s paper was, a number of its results had appeared in an earlier work by Tschuprow (1923a 1923b), in particular the result on optimal allocation in stratified sampling (see Fienberg and Tanur 1995, 1996 for discussions of this point). The method was derived much earlier, by the Danish mathematician Gram (1883) in a paper dealing with calculations for the cover of a forest based on a sample of trees. Gram’s work has only recently been rediscovered and seems not to have been accessible to either Tchouproff or Neyman.

Neyman provided the recipe for others to follow and he continued to explain its use in convincing detail to those who were eager to make random sampling a standard diet for practical consumption (e.g., see Neyman 1952 for a description based on his 1937 lectures on the topic at the US Department of Agriculture Graduate School, as discussed below).

4. The Development Of Random Sampling In The United States

One might have thought that the resolution of the controversy over the representative method and the articulation of the basic elements of a theory of sample surveys by Neyman would have triggered extensive application of random sampling throughout the world. Surprisingly, this was not the case. With some notable exceptions, for example, in England (see Cochran 1939 and Yates 1946) and India, the primary application occurred in the United States and this led to a spate of new and important methodological developments.

As late as 1932, however, there were few examples of probability sampling anywhere in the US federal government (Duncan and Shelton (1978) and the federal statistical agencies had difficulty responding to the demand for statistics to monitor the effects of the programs of President Franklin Roosevelt’s New Deal. In 1933, the American Statistical Association (ASA) set up an advisory committee that grew into the Committee on Government Statistics and Information Services (COGSIS), sponsored jointly by ASA and the Social Science Research Council. COGSIS helped to stimulate the use of probability sampling methods in various parts of the Federal government, and it encouraged employees of statistical agencies to carry out research on sampling theory. For example, to establish a technical basis for unemployment estimates, COGSIS, and the Central Statistical Board which it helped to establish, organized an experimental Trial Census of Unemployment as a Civil Works Administration project in three cities, using probability sampling, and carried out in late 1933 and early 1934. The positive results from this study led in 1940 to the establishment of the first large-scale, ongoing sample survey on employment and unemployment using probability sampling methods. This survey later became known as the Current Population Survey and it continues to the present day.

Another somewhat indirect outcome of the COGSIS emphasis on probability sampling took place at the Department of Agriculture Graduate School where Deming, who recognized the importance of Neyman’s 1934 paper, invited Neyman to present a series of lectures in 1937 on sampling and other statistical methods (Neyman (1952). These lectures had a profound impact on the further development of sampling theory not simply in agriculture, but across the government as well as in universities.

Among those who worked on the probability sampling-based trial Census of Unemployment at the Bureau of the Census was Hansen, who was then assigned with a few others to explore the field of sampling for other possible uses at the Bureau, and went on to work on the 1937 sample Unemployment Census. After working on the sample component of the 1940 decennial census (under the direction of Deming), Hansen worked with others to redesign the unemployment survey based on new ideas on multistage probability samples and cluster sampling (Hansen and Hurwitz 1942, 1943). They expanded and applied their approach in various Bureau surveys, often in collaboration and interaction with others, and this effort culminated in 1953 with the publication of a two-volume compendium of theory and methodology (Hansen et al. 1953a, 1953b).

Cochran’s (1939) paper, written in England and independently of the US developments, is especially notable because of its use of the analysis of variance in sampling settings and the introduction of superpopulation and modeling approaches to the analysis of survey data. In the 1940s, as results from these two separate schools appeared in various statistical journals, we see some convergence of ideas and results. The theory of estimation in samples with unequal probabilities of selection also emerged around this time (see Horvitz and Thompson 1952, Hansen et al. 1985).

Statisticians have continued to develop the theoretical basis of alternative methods of probability sampling and statistical inference from sampling data over the past fifty years (see, e.g., Rao and Bellhouse 1990, Sarndal et al. 1992). The issue of statistical inference for models from survey data remains controversial, at least for those trained from a traditional finite sampling perspective.

5. Market Research And Polling

The 1930s, and especially the period after World War II, however, saw a flowering of survey methodology in market research and polling, as well as in the social sciences more broadly. Initial stimulation came from a number of committees at the Social Science Research Council and social scientists such as Hadley Cantril, Paul Lazarsfeld, Rensis Likert, William Ogburn, and Samuel Stouffer (e.g., see Stephan 1948, Converse 1987 for a discussion). Stouffer actually spent several months in England in 1931–32, and learned about sampling and other statistical ideas from Bowley, Karl and Egon Pearson, and R. A. Fisher. He then participated in the Census of Unemployment at the Bureau of the Census.

Market research and polling trace their own pre-history to election straw votes collected by news- papers, dating back at least to the beginning of the nineteenth century. Converse (1987) points out, how- ever, a more serious journalistic base; election polls were taken and published by such reputable magazines as the Literary Digest (which had gained a reputation for accuracy before the 1936 fiasco). Then, as now, election forecasting was taken as the acid test of survey validity. A reputation for accuracy in ‘calling’ elections was thought to spill over to a presumption of accuracy in other, less verifiable areas.

There was a parallel tradition in market research, dating back to just before the turn of the twentieth century, attempting to measure consumers’ product preferences and the effectiveness of advertising. It was seen as only a short step from measuring the opinions of potential consumers about products to measuring the opinions of the general public about other objects, either material or conceptual. By the mid-1930s there were several well-established market research firms. Many of them conducted election polls in 1936 and achieved much greater accuracy than did the Literary Digest. It was the principals of these firms (e.g., Archibald Crossley, George Gallup, and Elmo Roper) who put polling—election, public opinion, and consumer—on the map in the immediate pre-World War II period.

The polling and market research surveys of Crossley, Gallup, Roper, and others were based on a sampling method involving ‘quota controls,’ but did not involve random sampling. Stephan (1948) observed the close link between their work and the method of purposive sampling that had gained currency much earlier in government and academic research circles, but which by the 1930s had been supplanted by random sampling techniques.

6. From Sampling Theory To The Study Of Nonsampling Error

The 1940s saw a rapid spread of probability sampling methods to a broad array of government agencies. It was, however, only after the fiasco of the 1948 presidential pre-election poll predictions (Mosteller et al. 1949) that market research firms and others shifted towards probability sampling. Even today many organizations use a version of probability sampling with quotas (Sudman 1967).

Amidst the flurry of activity on the theory and practice of probability sampling during the 1940s, attention was being focused on issues of nonresponse and other forms of nonsampling error (e.g., see Deming 1944) such as difficulty in understanding questions, or remembering answers, etc.). A milestone in this effort to understand and model nonresponse errors was the development of an integrated model for sampling and nonsampling error in censuses and surveys, in connection with planning for and evaluation of the 1950 census (Hansen et al. (1951). This analysis-of-variance-like model, or variants of it, has served as the basis of much of the work on nonsampling error since.

New developments in sampling since the mid-twentieth century have to do less with the design of samples and more to do with the structuring of the survey instrument and interview. Thus, there has been a move from face-to-face interviewing to telephone interviewing (with the attendant problems connected with random-digit dialing), and then to the use of computers to assist in interviewing. Nonsampling errors continue to be studied broadly, often under the rubric of cognitive aspects of survey design.

Bibliography:

1. Bellhouse D 1988 A brief history of random sampling methods. In: Krishnaiah P, Rao C (eds.) Handbook of Statistics, Vol. 6. North Holland, Amsterdam, pp. 1–14
2. Bowley A 1913 Working-class households in Reading. Journal of the Royal Statistical Society 76: 672–701
3. Bowley A 1926 Measurement of the precision attained in sampling. Bulletin of the International Statistical Institute 22(1): 6–62
4. Cochran W 1939 The use of the analysis of variance in enumeration by sampling. Journal of the American Statistical Association 128: 124–35
5. Cochran W G 1978 Laplace’s ratio estimator. In: David H (ed.) Contributions to Survey Sampling and Applied Statistics: Papers in Honor of H. O. Hartley. Academic Press, New York, pp. 3–10
6. Converse J 1987 Survey Research in the United States: Roots and Emergence 1890–1960. University of California Press, Berkeley, CA
7. Deming W 1944 On errors in surveys. American Sociological Review 19: 359–69
8. Duncan J, Shelton W 1978 Revolution in United States Government Statistics, 1926–1976. US Government Printing Office, Washington, DC
9. Duncan O 1984 Notes on Social Measurement. Sage, New York
10. Edgeworth F Y 1912 On the use of the theory of probabilities in statistics relating to society. Journal of the Royal Statistical Society 76: 165–93
11. Fienberg S, Tanur J 1995 Reconsidering Neyman on experimentation and sampling: Controversies and fundamental contributions. Probability and Mathematical Statistics 15: 47–60
12. Fienberg S, Tanur J 1996 Reconsidering the fundamental contributions of Fisher and Neyman on experimentation and sampling. International Statistical Review 64: 237–53
13. Gini C 1928 Une application de la methode representative aux materiaux du dernier recensement de la population italienne (1er decembre 1921). Bulletin of the International Statistical Institute 23(2): 198–215
14. Gini C, Galvani L 1929 Di una applicazione del metodo rappresentative all’ultimo censimento italiano della popolazione (1 Decembri 1921). Annali di Statistica 6(4): 1–107
15. Gram J 1883 About calculation of the mass of a forest cover by means of test trees (in Danish). Tidsskrft for Sko brug 6: 137–98
16. Hansen M, Dalenius T, Tepping B 1985 The development of sample surveys of finite populations. In: Atkinson A, Fienberg S (eds.) A Celebration of Statistics: The ISI Centenary Volume. Springer Verlag, New York, pp. 327–54
17. Hansen M, Hurwitz W 1942 Relative efficiencies of various sampling units in population inquiries. Journal of the American Statistical Association 37: 89–94
18. Hansen M, Hurwitz W 1943 On the theory of sampling from finite populations. Annals of Mathematical Statistics 14: 333–62
19. Hansen M, Hurwitz W, Madow W 1953a Sample Survey Methods and Theory, Vol. 1. Wiley, New York
20. Hansen M, Hurwitz W, Madow W 1953b Sample Survey Methods and Theory, Vol. 2. Wiley, New York
21. Hansen M, Hurwitz W, Marks E, Mauldin W 1951 Response errors in surveys. Journal of the American Statistical Association 46: 147–90
22. Horvitz D, Thompson D 1952 A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association 47: 663–85
23. Jenson A 1926 Report on the representative method in statistics. Bulletin of the International Statistical Institute 22: 359–80
24. Kiaer A 1895 1896 Observation et experiences concernant des denombrements representatifs. Bulletin of the International Statistical Institute 9(2): 176–83
25. Kiaer A 1897 The representative method of statistical surveys (in Norwegian). Christiana Videnskabsselskabets Skrifter. II Hisoriskfilosofiske 4 [reprinted with an English trans. by Central Bureau of Statistics of Norway, Oslo, 1976)]
26. Kruskal W, Mosteller F 1980 Representative sampling, iv: The history of the concept in statistics, 1895–1939. International Statistical Review 48: 169–95
27. Laplace P 1814 Essai philosophique sur les probabilites. Dover, New York [trans. of 1840 6th edn. appeared in 1951 as A Philosophical Essay on Probabilities, trans. Truscott F W, Emory F L]
28. Madansky A 1986 On biblical censuses. Journal of Official Statistics 2: 561–69
29. Mahalanobis P 1944 On large-scale sample surveys. Philosophical Transactions of the Royal Society of London 231(B): 329–451
30. Mahalanobis P 1946 Recent experiments in statistical sampling in the Indian Statistical Institute. Journal of the Royal Statistical Society 109: 325–78
31. Mosteller F, Hyman H, McCarthy P J, Marks E S, Truman D B 1949 The Pre-election Polls of 1948. Social Science Research Council, New York
32. Neyman J 1934 On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society 97: 558–625
33. Neyman J 1952 Lectures and Conferences on Mathematical Statistics and Probability. US Department of Agriculture, Washington, DC [The 1952 edition is an expanded and revised version of the original 1938 mimeographed edition.]
34. Rao J, Bellhouse D 1990 The history and development of the theoretical foundations of survey based estimation and statistical analysis. Survey Methodology 16: 3–29
35. Sarndal C, Swensson B, Wretman J 1992 Model Assisted Survey Sampling. Springer Verlag, New York
36. Seneta E 1985 A sketch of the history of survey sampling in Russia. Journal of the Royal Statistical Society Series A 148: 118–25
37. Seng Y 1951 Historical survey of the development of sampling theories and practice. Journal of the Royal Statistical Society, Series A 114: 214–31
38. Splawa-Neyman J 1923 On the application of probability theory to agricultural experiments. Essay on principles (in Polish). Roczniki Nauk Rolniczych Tom (Annals of Agricultural Sciences) X: 1–51 [Sect. 9, translated and edited by D. M. Dabrowska and T. P. Speed, appeared with discussion in Statistical Science (1990) 5: 463–80]
39. Splawa-Neyman J 1925 Contributions of the theory of small samples drawn from a finite population. Biometrika 17: 472–9 [The note on this republication reads ‘These results with others were originally published in La Revue Mensuelle de Statistique, publ. Par l’Office Central de Statistique de la Republique Polonaise 1923 tom. vi, 1–29]
40. Stephan F 1948 History of the use of modern sampling procedures. Journal of the American Statistical Association 43: 12–39
41. Stigler S M 1986 The History of Statistics: The Measurement of Uncertainty before 1900. Harvard University Press, Cambridge, MA
42. Sudman S 1967 Reducing the Costs of Surveys. Aldine, Chicago Taeuber C 1978 Census. In: Kruskal W H, Tanur J M (eds.) International Encyclopedia of Statistics. Macmillan and the Free Press, New York, pp. 42–6
43. Tchouproff A A 1918a On the mathematical expectation of the moments of frequency distributions (Chap. 1 and 2). Biometrika 12: 140–69
44. Tchouproff A A 1918b On the mathematical expectation of the moments of frequency distributions (Chap. 3 and 4). Biometrika 12: 185–210
45. Tschuprow A A 1923a On the mathematical expectation of the moments of frequency distributions in the case of correlated observations (Chaps. i–iii). Metron 2: 461–93
46. Tschuprow A A 1923b On the mathematical expectation of the moments of frequency distributions in the case of correlated observations (Chaps. iv–vi). Metron 2: 646–80
47. Willcox W 1930 Census. In: Seligman E R A, Johnson A (eds.) Encyclopedia of Social Sciences, Vol. 3. Macmillan, New York pp. 295–300
48. Yates F 1946 A review of recent developments in sampling and sample surveys (with discussion). Journal of the Royal Statistical Society 109: 12–42
49. Zarkovic S S 1956 Note on the history of sampling methods in Russia. Journal of the Royal Statistical Society, Series A 119: 336–8
50. Zarkovich S S 1962 A supplement to ‘note on the history of sampling methods in Russia. ’Journal of the Royal Statistical Society, Series A 125: 580–2