History of Social Survey Research Paper

The origins of the investigation of the condition of the working or laboring classes in modern times may be traced to social investigations in the UK in the late eighteenth and early nineteenth century. Such social investigations were not surveys, but are part of the immediate prehistory of the social survey. The social survey as a tool of scientific inquiry is not much more than 110 years old.

1. The Survey Defined

Several specific characteristics distinguish the social survey from the modes of social investigation that preceded it. A social survey involved field work, the collection of data at first hand by a social investigator rather than reliance upon reports by others or on preexisting data. Surveys attempted to achieve comprehensive rather than haphazard coverage, albeit initially within a local rather than a national area. The data in surveys related to individuals, families, and households rather than aggregates, and were analyzed accordingly. Survey research involved the attempt, however primitive, at counting and quantifying the phenomena with which it was concerned. And the social survey developed in close relationship with public policy and social reform.

Throughout the nineteenth century there were investigations into public health, housing, family life, and employment, particularly as these affected the working classes. Some, such as the studies by Frederic Le Play of working-class family budgets, had wider scientific aspirations. The investigations were carried out by private individuals (many of them, in the later nineteenth century, women of some social position), certain professions (associated with, in particular, medicine), members of voluntary associations concerned with social welfare, a few journalists and (by the end of the century) one or two academic scholars. Although the methods of inquiry varied, there was some common ground. Indeed, for a period in the USA what became known as the Social Survey Movement flourished, engaging significant numbers of private, government, and academic researchers. These inquiries signified increasing upper-class and middle-class interest in the condition of the working classes as well as a desire to intervene—a desire both to remedy want and disease through voluntary or state action and to achieve a greater degree of social control through the use of scientific expertise.

There was thus a dual involvement of both social scientists and social reformers in the history of the social survey, and an interplay between them. It is important to stress that before 1940 the social survey was not associated particularly closely with academic social science, which was in any case itself quite small in scale. The modem scientific sample survey, established by academics such as Rensis Likert and Paul Lazarsfeld, government statisticians such as P. C. Mahalanobis and Louis Moss, and market researchers such as Henry Durant and Mark Abrams, has developed since that time. Before World War II, a variety of individuals and groups, only a minority of them academics, contributed to the establishment of the social survey. Occasionally surveys have originated in an abstract desire for more knowledge about the structure and workings of society; more frequently, however, they have been carried out as an indispensable first step in measuring the dimensions of a social problem, ascertaining its causes, and then deciding upon remedial action.

2. Studies Of Local Social Conditions

The history of the social survey can be conveniently schematized into two periods, the early studies of local social conditions—particularly poverty—and the post-1940 probability sample survey. The former were mainly local, the latter frequently national and more recently international in scope and design. However, both types of survey shared features in common. A fundamental change in the concepts guiding the acquisition of social knowledge accompanied the rise of the social survey. A comparison between Henry Mayhew’s examination of the London poor in the middle of the nineteenth century and Charles Booth’s surveys three decades later is instructive. Booth conceptualized the problem differently, conducted a larger-scale inquiry, attempted to measure the phenomena with which he was concerned, and collected data from multiple observers rather than relying upon a single observer. The emergence of the interview, however, was a gradual process.

The history of the social survey is usually dated in Britain to the 1880s when, as economic depression heightened social tensions, social and political groups that had competed in prosperity descended into conflict. One manifestation of these changing circumstances was sporadic social unrest. One particular question which exercised opinion in the middle and upper classes was the extent of poverty. In the 1880s, with more trying economic conditions, the problem of poverty became more insistent and the discrepancy between the numbers in receipt of poor relief and the proportion of the population having inadequate incomes due to underemployment, unemployment, family breakdown, ill-health, old age, and other misfortunes was more apparent. But how great was the difference and who would provide an answer to the question?

The absence of an adequate institutional base from which to undertake such inquiries was a major obstacle. This base was not provided by the political parties. Government departments did little at this period to gather information on social conditions. With the exception of one or two positions in economics, social science was not institutionalized in universities.

Existing methods of collecting social data about poverty were patchy and often lacking in reliability. Henry Mayhew’s observational approach emphasized vivid insight and detailed description over extensive enumeration. Statistics compiled by central and local government, such as they were, related to administrative function and were neither designed nor collected by people with professional training. The interrogatory methods of parliamentary committee and Royal Commissions as devices for investigating problems were unlikely to lead to a satisfactory overall picture of social conditions, producing impressionistic results which varied between individuals. There was a marked lack of clarity in defining the phenomenon.

3. Charles Booth

The pivotal figure of Charles Booth looms large in the history of poverty surveys. Charles Booth was a wealthy Liverpool ship-owner who was annoyed by the lack of a factual basis for the accounts of urban poverty of the time and lack of clarity about three questions to which he sought answers: How many were poor, why were they poor, and what should be done to alleviate poverty? Through careful empirical study Booth sought answers for the East End of London to the first two questions. The Life and Labour of the People in London, published in 17 volumes between 1889 and 1903, is the first great empirical study in the social survey tradition. The publication of the first volume of Booth’s study of poverty in London in 1889 created a sensation.

Booth’s originality lay in his methods of inquiry, not just in how he collected and analyzed his data, but in the way he conceptualized the problem of poverty. Booth attempted to introduce precision to the concept of the ‘poverty line’ where hitherto there had been vagueness, a characteristic which justifies including him as one of the founding fathers of social science. Booth sought to measure the numbers of the population above and below the poverty line with some precision, even though the classification categories that he used lacked adequate justification. This too differentiated his work from previous studies, and foreshadowed the later use of the social survey for estimating population values with some exactitude. Furthermore, although Booth’s study, like most of the early social surveys, was a local study, its coverage was comprehensive—he aimed to gather information on as large a proportion of the population as was possible and especially on all families with children in the parts of London covered by the survey.

Booth was also unusual in using a team of investigators. First-hand data about the household circumstances of poor families were not collected directly by his research team but by what one of them, Beatrice Webb, called the method of the ‘wholesale interview.’ School board visitors, who entered the homes of children attending London schools, were interviewed by Booth to gather information about the circumstances of each family. Booth had printed data collection notebooks that categorized the information gained in interviews with the visitors. Great care was taken to achieve uniformity in the data collection between the different interviewers, and the information obtained was checked against other sources in an attempt to increase its reliability. The quantitative and qualitative data from the notebooks were analyzed by Booth and his staff. The published results were aggregations of the measurable and distillations of the unquantifiable. The findings had direct policy implications and contributed to movements toward social reform and old age pensions in particular.

Booth’s research had important effects on the development of the social survey, independent of any influence his findings may have had, and popularized survey research. Several early US studies were derivative of Booth. And Booth’s work served as a template for a large number of early social researchers.

4. Seebohm Rowntree And A. L. Bowley

The next major survey in Britain, Seebohm Rowntree’s study of York, was conducted in 1899 and published in 1901. This further refined both the conceptualization and measurement of the phenomenon of poverty. Most important, perhaps, it used ‘retail’ rather than ‘wholesale’ interviewing; that is to say, people were interviewed directly in their own homes by members of the research team, not indirectly through reports by middle-class professionals with knowledge of individual families. This transition from reliance upon informants who could report on the circumstances of working-class people to reliance upon the response of the people themselves was a most important one. Rowntree also refined the conceptual understanding of poverty by introducing the concept of the ‘life cycle,’ and by showing how poverty varied between different phases of the life cycle. There were a considerable number of other local poverty studies in Britain prior to 1940, but the original studies by Booth and Rowntree, and restudies conducted in York by Rowntree in 1936 and 1951 were the most influential.

The first social surveyor to use sampling methods was Arthur Lyon Bowley in his studies of poverty in five towns, published as Livelihood and Poverty in 1915. This innovation, which was slow in being adopted, eventually undercut the local social survey, replacing it with the modem probability sample survey. The use of sampling in practice only emerged slowly and was first seriously taken up by US market researchers between the wars. It was used in social survey research for government and academic research only during and after World War II.

5. Early Surveys In The United States

Developments in the USA shared many features of this British experience. In both societies the social survey was shaped by the salience of voluntary effort, of liberal political traditions that restricted government initiatives, and of values promoting ‘responsible individualism.’ Thus we can speak of an Anglo-American tradition of social survey movements that shared many fundamental features. In both countries the settlement house movement provided a base for fostering social investigation. The influence of their ideas and the publicity they gave to the existence of social conditions created a more thoughtful climate for the reception of social surveys.

At the same time, the US context contained challenges not found in the British milieu. More isolated from political power, unable to draw on traditional criticisms of industrialization, and less likely to work closely with organized labor, reformers had to make the most of their middle-class resources. They also stood in a different relationship to emerging social science. By the end of the century, there was a different relationship between professional expertise and the developing skills of social investigation than in the UK, one which also had repercussions for conceptions of objectivity and of what constituted social science. In these circumstances, the social survey became even more important to middle-class reformers. For the objective data provided by the surveys were uniquely effective in mobilizing ‘public opinion’—that amorphous combination of middle-class and working-class attitudes that activated politicians and labor leaders alike within the US political environment. Surveys could get action (especially when linked with sensationalist press reports) even when ‘monster’ meetings and petitions could not.

The social survey in the USA in the period between 1880 and 1940, while resembling and differing from its British counterpart, directly imitated Booth’s work in certain respects. Florence Kelley’s work on the Hull- House Maps and Papers was one of the earliest US surveys and very much in the Booth tradition. Other studies followed. The role of the settlement house movement in publicizing social conditions was not confined to poverty. Indeed issues such as child labor, the treatment of juveniles in the court system, sweated labor and poor housing conditions loomed as large if not larger in their concerns. At the state level some notable innovations were effected, for example, the establishment of the Illinois Juvenile Court in 1899. W. E. B. DuBois’ study of The Philadelphia Negro (1899) by a young black sociologist was notable for its thoroughness and detachment.

6. The Social Survey Movement

The social survey movement, which was best expressed in the Pittsburgh Survey of 1906–9, was different. It emerged from the charity movement, and the belief among significant numbers of social workers that the roots of social problems were to be found in society rather than in the individual. The Pittsburgh Survey, funded by the newly created Russell Sage Foundation, was more in the nature of investigative journalism, with much less emphasis upon quantification. More emphasis was put on influencing public opinion. When published, a great effort was made to feed back the results of the research to the locality in which it had been carried out, by means of publicity and exhibits. This gave surveys of this type a completely different character, for they became exercises in community self-study more than scientific examinations of social questions. Indeed, its success was followed by the establishment of the Department of Survey and Exhibits at the Russell Sage Foundation, which between 1912 and 1931 supported a very large number of local surveys. The US social survey movement had run out of steam by 1930 and disappeared shortly thereafter. There were other developments, such as ‘crime surveys’ as an aid to administrative reform, and the country life movement and the church survey movement, but these were of little lasting methodological significance.

The geographically local social survey between 1880 and 1940 in the UK and the USA led to distinctive changes in social investigation. What distinguished Booth from Henry Mayhew, W. E. B.DuBois from Ray Stannard Baker, A. L. Bowley from R. H. Tawney, was the urge to present systematic and defensible numerical statements about the problem studied based on a large-scale data collection exercise derived from questioning individuals. Such an appeal to ‘objectivity’ and ‘science’ is one which became increasingly prominent as the twentieth century advanced. The local survey was closer to social reform and amelioration, to political action and to nonacademic institutions.

7. The Probability Sample Survey

After 1920, the social survey developed more rapidly and achieved more outstanding results in the USA than in the UK, and at the same time underwent a change in form. It is true that at the theoretical level, Britain maintained its preeminence for longer. The traffic to sit at the feet of statisticians Karl Pearson, R. A. Fisher, G. Udny Yule, and A. L. Bowley in the University of London continued until well into the 1930s. Leading US survey researchers such as Samuel Stouffer learned advanced inferential statistics from them and returned to apply those methods to good effect in the USA. At a practical level, however, the main implementation of probability sampling methods which came gradually to be used, giving the survey as a method very much greater power, took place in the USA. After Kiaer and Bowley’s pioneering work, the application of sampling in national studies to provide a representative national picture may be traced to market research and opinion polling in the 1930s. From 1928, George Gallup had been successfully using probability sampling for marketing and political research in the USA. In 1936 Henry Durant established the British counterpart of Gallup’s organization, the British Institute of Public Opinion (BIPO). Gallup’s triumph over the Literary Digest in 1936 in correctly predicting Roosevelt’s victory from a much smaller sample was a clear marker of the future potential of the method.

At the same time, efforts to improve social measurement were being made. During the 1920s, political scientists and psychologists, such as Harold Gosnell and L. L. Thurstone, made advances in the systematic measurement of voting and of social attitudes. Research units in government departments, one of them led by Rensis Likert, were established. Paul Lazarsfeld came to the USA from Austria as a Rockefeller Fellow, and remained there to commence survey research on mass media audiences in the mid-1930s. By the late 1930s, the basis for the development of the modern probability social survey was in place. With the outbreak of World War II and in particular the entry of the USA into the war in 1941, significant activity took place in government. The American Soldier study directed by Stouffer was important in its own right theoretically and methodologically, and for the stimulus it gave to postwar research at Harvard and Columbia. Likert’s group in the Department of Agriculture became the nucleus of the postwar development of the Survey Research Center at the University of Michigan. Lazarsfeld independently widened his interest in the media to voting and the effects of mass communications in elections.

With the increases in federal government support for social science post-1945, survey research took off, becoming established in the academic world as well as in bureaucratic social survey organizations outside it. There was particularly strong cross-fertilization at centers at the National Opinion Research Center at the University of Chicago, the Survey Research Center at the University of Michigan and the Bureau of Applied Social Research at Columbia. In the UK, government was important, with the Government Social Survey remaining the main center for survey research. Other major survey research organizations and centers in Western Europe followed later. Since the mid-twentieth century, the intellectual leadership in modem sample survey research has lain in the USA, which has tended to remain in the forefront of intellectual fertility, analytic inventiveness, and methodological innovation.

Bibliography:

1. Bulmer M, Sklar K K, Bales K (eds.) 1991 The Social Survey in Historical Perspective 1880–1940. Cambridge University Press, Cambridge, UK
2. Converse J 1987 Survey Research in the United States: Roots and Emergence, 1890–1960. University of California Press, Berkeley, CA