View sample Statistics Research Paper. Browse other research paper examples and check the list of research paper topics for more inspiration. If you need a religion research paper written according to all the academic standards, you can always turn to our experienced writers for help. This is how your paper can get an A! Feel free to contact our custom writing services for professional assistance. We offer high-quality assignments for reasonable rates.
Statistics is a term used to refer to both a ﬁeld of scientiﬁc inquiry and a body of quantitative methods. The ﬁeld of statistics has a 350-year intellectual history rooted in the origins of probability and the rudimentary tools of political arithmetic of the seventeenth century. Statistics came of age as a separate discipline with the development of formal inferential theories in the twentieth century. This research paper brieﬂy traces some of this historical development and discusses current methodological and inferential approaches as well as some cross-cutting themes in the development of new statistical methods.
More Statistics Research Papers:
Statistics is a body of quantitative methods associated with empirical observation. A primary goal of these methods is coping with uncertainty. Most formal statistical methods rely on probability theory to express this uncertainty and to provide a formal mathematical basis for data description and for analysis. The notion of variability associated with data, expressed through probability, plays a fundamental role in this theory. As a consequence, much statistical eﬀort is focused on how to control and measure variability and/or how to assign it to its sources.
Almost all characterizations of statistics as a ﬁeld include the following elements:
(a) Designing experiments, surveys, and other systematic forms of empirical study.
(b) Summarizing and extracting information from data.
(c) Drawing formal inferences from empirical data through the use of probability.
(d) Communicating the results of statistical investigations to others, including scientists, policy makers, and the public.
This research paper describes a number of these elements, and the historical context out of which they grew. It provides a broad overview of the ﬁeld, that can serve as a starting point to many of the other statistical entries in this encyclopedia.
2. The Origins Of The Field of Statistics
The word ‘statistics’ is related to the word ‘state’ and the original activity that was labeled as statistics was social in nature and related to elements of society through the organization of economic, demographic, and political facts. Paralleling this work to some extent was the development of the probability calculus and the theory of errors, typically associated with the physical sciences. These traditions came together in the nineteenth century and led to the notion of statistics as a collection of methods for the analysis of scientiﬁc data and the drawing of inferences therefrom.
As Hacking (1990) has noted: ‘By the end of the century chance had attained the respectability of a Victorian valet, ready to be the logical servant of the natural, biological and social sciences’ ( p. 2). At the beginning of the twentieth century, we see the emergence of statistics as a ﬁeld under the leadership of Karl Pearson, George Udny Yule, Francis Y. Edgeworth, and others of the ‘English’ statistical school. As Stigler (1986) suggests:
Before 1900 we see many scientists of diﬀerent ﬁelds developing and using techniques we now recognize as belonging to modern statistics. After 1900 we begin to see identiﬁable statisticians developing such techniques into a uniﬁed logic of empirical science that goes far beyond its component parts. There was no sharp moment of birth; but with Pearson and Yule and the growing number of students in Pearson’s laboratory, the infant discipline may be said to have arrived. (p. 361)
Pearson’s laboratory at University College, London quickly became the ﬁrst statistics department in the world and it was to inﬂuence subsequent developments in a profound fashion for the next three decades. Pearson and his colleagues founded the ﬁrst methodologically-oriented statistics journal, Biometrika, and they stimulated the development of new approaches to statistical methods. What remained before statistics could legitimately take on the mantle of a ﬁeld of inquiry, separate from mathematics or the use of statistical approaches in other ﬁelds, was the development of the formal foundations of theories of inference from observations, rooted in an axiomatic theory of probability.
Beginning at least with the Rev. Thomas Bayes and Pierre Simon Laplace in the eighteenth century, most early eﬀorts at statistical inference used what was known as the method of inverse probability to update a prior probability using the observed data in what we now refer to as Bayes’ Theorem. (For a discussion of who really invented Bayes’ Theorem, see Stigler 1999, Chap. 15). Inverse probability came under challenge in the nineteenth century, but viable alternative approaches gained little currency. It was only with the work of R. A. Fisher on statistical models, estimation, and signiﬁcance tests, and Jerzy Neyman and Egon Pearson, in the 1920s and 1930s, on tests of hypotheses, that alternative approaches were fully articulated and given a formal foundation. Neyman’s advocacy of the role of probability in the structuring of a frequency-based approach to sample surveys in 1934 and his development of conﬁdence intervals further consolidated this eﬀort at the development of a foundation for inference (cf. Statistical Methods, History of: Post- 1900 and the discussion of ‘The inference experts’ in Gigerenzer et al. 1989).
At about the same time Kolmogorov presented his famous axiomatic treatment of probability, and thus by the end of the 1930s, all of the requisite elements were ﬁnally in place for the identiﬁcation of statistics as a ﬁeld. Not coincidentally, the ﬁrst statistical society devoted to the mathematical underpinnings of the ﬁeld, The Institute of Mathematical Statistics, was created in the United States in the mid-1930s. It was during this same period that departments of statistics and statistical laboratories and groups were ﬁrst formed in universities in the United States.
3. Emergence Of Statistics As A Field
3.1 The Role Of World War II
Perhaps the greatest catalysts to the emergence of statistics as a ﬁeld were two major social events: the Great Depression of the 1930s and World War II. In the United States, one of the responses to the depression was the development of large-scale probability-based surveys to measure employment and unemployment. This was followed by the institutionalization of sampling as part of the 1940 US decennial census. But with World War II raging in Europe and in Asia, mathematicians and statisticians were drawn into the war eﬀort, and as a consequence they turned their attention to a broad array of new problems. In particular, multiple statistical groups were established in both England and the US speciﬁcally to develop new methods and to provide consulting. (See Wallis 1980, on statistical groups in the US; Barnard and Plackett 1985, for related eﬀorts in the United Kingdom; and Fienberg 1985). These groups not only created imaginative new techniques such as sequential analysis and statistical decision theory, but they also developed a shared research agenda. That agenda led to a blossoming of statistics after the war, and in the 1950s and 1960s to the creation of departments of statistics at universities—from coast to coast in the US, and to a lesser extent in England and elsewhere.
3.2 The Neo-Bayesian Revival
Although inverse probability came under challenge in the 1920s and 1930s, it was not totally abandoned. John Maynard Keynes (1921) wrote A Treatise on Probability that was rooted in this tradition, and Frank Ramsey (1926) provided an early eﬀort at justifying the subjective nature of prior distributions and suggested the importance of utility functions as an adjunct to statistical inference. Bruno de Finetti provided further development of these ideas in the 1930s, while Harold Jeﬀreys (1938) created a separate ‘objective’ development of these and other statistical ideas on inverse probability.
Yet as statistics ﬂourished in the post-World War II era, it was largely based on the developments of Fisher, Neyman and Pearson, as well as the decision theory methods of Abraham Wald (1950). L. J. Savage revived interest in the inverse probability approach with The Foundations of Statistics (1954) in which he attempted to provide the axiomatic foundation from the subjective perspective. In an essentially independent eﬀort, Raiﬀa and Schlaifer (1961) attempted to provide inverse probability counterparts to many of the then existing frequentist tools, referring to these alternatives as ‘Bayesian.’ By 1960, the term ‘Bayesian inference’ had become standard usage in the statistical literature, the theoretical interest in the development of Bayesian approaches began to take hold, and the neo-Bayesian revival was underway. But the movement from Bayesian theory to statistical practice was slow, in large part because the computations associated with posterior distributions were an overwhelming stumbling block for those who were interested in the methods. Only in the 1980s and 1990s did new computational approaches revolutionize both Bayesian methods, and the interest in them, in a broad array of areas of application.
3.3 The Role Of Computation In Statistics
From the days of Pearson and Fisher, computation played a crucial role in the development and application of statistics. Pearson’s laboratory employed dozens of women who used mechanical devices to carry out the careful and painstaking calculations required to tabulate values from various probability distributions. This eﬀort ultimately led to the creation of the Biometrika Tables for Statisticians that were so widely used by others applying tools such as chisquare tests and the like. Similarly, Fisher also developed his own set of statistical tables with Frank Yates when he worked at Rothamsted Experiment Station in the 1920s and 1930s. One of the most famous pictures of Fisher shows him seated at Whittingehame Lodge, working at his desk calculator (see Box 1978).
The development of the modern computer revolutionized statistical calculation and practice, beginning with the creation of the ﬁrst statistical packages in the 1960s—such as the BMDP package for biological and medical applications, and Datatext for statistical work in the social sciences. Other packages soon followed—such as SAS and SPSS for both data management and production-like statistical analyses, and MINITAB for the teaching of statistics. In 2001, in the era of the desktop personal computer, almost everyone has easy access to interactive statistical programs that can implement complex statistical procedures and produce publication-quality graphics. And there is a new generation of statistical tools that rely upon statistical simulation such as the bootstrap and Markov Chain Monte Carlo methods. Complementing the traditional production-like packages for statistical analysis are more methodologically oriented languages such as S and S-PLUS, and symbolic and algebraic calculation packages. Statistical journals and those in various ﬁelds of application devote considerable space to descriptions of such tools.
4. Statistics At The End Of The Twentieth Century
It is widely recognized that any statistical analysis can only be as good as the underlying data. Consequently, statisticians take great care in the the design of methods for data collection and in their actual implementation. Some of the most important modes of statistical data collection include censuses, experiments, observational studies, and sample Surveys, all of which are discussed elsewhere in this encyclopedia. Statistical experiments gain their strength and validity both through the random assignment of treatments to units and through the control of nontreatment variables. Similarly sample surveys gain their validity for generalization through the careful design of survey questionnaires and probability methods used for the selection of the sample units. Approaches to cope with the failure to fully implement randomization in experiments or random selection in sample surveys are discussed in Experimental Design: Compliance and Nonsampling Errors.
Data in some statistical studies are collected essentially at a single point in time (cross-sectional studies), while in others they are collected repeatedly at several time points or even continuously, while in yet others observations are collected sequentially, until suﬃcient information is available for inferential purposes. Diﬀerent entries discuss these options and their strengths and weaknesses.
After a century of formal development, statistics as a ﬁeld has developed a number of diﬀerent approaches that rely on probability theory as a mathematical basis for description, analysis, and statistical inference. We provide an overview of some of these in the remainder of this section and provide some links to other entries in this encyclopedia.
4.1 Data Analysis
The least formal approach to inference is often the ﬁrst employed. Its name stems from a famous article by John Tukey (1962), but it is rooted in the more traditional forms of descriptive statistical methods used for centuries.
Today, data analysis relies heavily on graphical methods and there are diﬀerent traditions, such as those associated with
(a) The ‘exploratory data analysis’ methods suggested by Tukey and others.
(b) The more stylized correspondence analysis techniques of Benzecri and the French school.
(c) The alphabet soup of computer-based multivariate methods that have emerged over the past decade such as ACE, MARS, CART, etc.
No matter which ‘school’ of data analysis someone adheres to, the spirit of the methods is typically to encourage the data to ‘speak for themselves.’ While no theory of data analysis has emerged, and perhaps none is to be expected, the ﬂexibility of thought and method embodied in the data analytic ideas have inﬂuenced all of the other approaches.
The name of this group of methods refers to a hypothetical inﬁnite sequence of data sets generated as was the data set in question. Inferences are to be made with respect to this hypothetical inﬁnite sequence. (For details, see Frequentist Inference).
One of the leading frequentist methods is signiﬁcance testing, formalized initially by R. A. Fisher (1925) and subsequently elaborated upon and extended by Neyman and Pearson and others (see below). Here a null hypothesis is chosen, for example, that the mean, µ, of a normally distributed set of observations is 0. Fisher suggested the choice of a test statistic, e.g., based on the sample mean, x, and the calculation of the likelihood of observing an outcome as or more extreme as x is from µ 0, a quantity usually labeled as the p-value. When p is small (e.g., less than 5 percent), either a rare event has occurred or the null hypothesis is false. Within this theory, no probability can be given for which of these two conclusions is the case.
A related set of methods is testing hypotheses, as proposed by Neyman and Pearson (1928, 1932). In this approach, procedures are sought having the property that, for an inﬁnite sequence of such sets, in only (say) 5 percent for would the null hypothesis be rejected if the null hypothesis were true. Often the inﬁnite sequence is restricted to sets having the same sample size, but this is unnecessary. Here, in addition to the null hypothesis, an alternative hypothesis is speciﬁed. This permits the deﬁnition of a power curve, reﬂecting the frequency of rejecting the null hypothesis when the speciﬁed alternative is the case. But, as with the Fisherian approach, no probability can be given to either the null or the alternative hypotheses.
The construction of conﬁdence intervals, following the proposal of Neyman (1934), is intimately related to testing hypotheses; indeed a 95 percent conﬁdence interval may be regarded as the set of null hypotheses which, had they been tested at the 5 percent level of signiﬁcance, would not have been rejected. A conﬁdence interval is a random interval, having the property that the speciﬁed proportion (say 95 percent) of the inﬁnite sequence, of random intervals would have covered the true value. For example, an interval that 95 percent of the time (by auxiliary randomization) is the whole real line, and 5 percent of the time is the empty set, is a valid 95 percent conﬁdence interval.
Estimation of parameters—i.e., choosing a single value of the parameters that is in some sense best—is also an important frequentist method. Many methods have been proposed, both for particular models and as general approaches regardless of model, and their frequentist properties explored. These methods usually extended to intervals of values through inversion of test statistics or via other related devices. The resulting conﬁdence intervals share many of the frequentist theoretical properties of the corresponding test procedures.
Frequentist statisticians have explored a number of general properties thought to be desirable in a procedure, such as invariance, unbiasedness, suﬃciency, conditioning on ancillary statistics, etc. While each of these properties has examples in which it appears to produce satisfactory recommendations, there are others in which it does not. Additionally, these properties can conﬂict with each other. No general frequentist theory has emerged that proposes a hierarchy of desirable properties, leaving a frequentist without guidance in facing a new problem.
4.3 Likelihood Methods
The likelihood function (ﬁrst studied systematically by R. A. Fisher) is the probability density of the data, viewed as a function of the parameters. It occupies an interesting middle ground in the philosophical debate, as it is used both by frequentists (as in maximum likelihood estimation) and by Bayesians in the transition from prior distributions to posterior distributions. A small group of scholars (among them G. A. Barnard, A. W. F. Edwards, R. Royall, D. Sprott) have proposed the likelihood function as an independent basis for inference. The issue of nuisance parameters has perplexed this group, since maximization, as would be consistent with maximum likelihood estimation, leads to different results in general than does integration, which would be consistent with Bayesian ideas.
4.4 Bayesian Methods
Both frequentists and Bayesians accept Bayes’ Theorem as correct, but Bayesians use it far more heavily. Bayesian analysis proceeds from the idea that probability is personal or subjective, reﬂecting the views of a particular person at a particular point in time. These views are summarized in the prior distribution over the parameter space. Together the prior distribution and the likelihood function deﬁne the joint distribution of the parameters and the data. This joint distribution can alternatively be factored as the product of the posterior distribution of the parameter given the data times the predictive distribution of the data.
In the past, Bayesian methods were deemed to be controversial because of the avowedly subjective nature of the prior distribution. But the controversy surrounding their use has lessened as recognition of the subjective nature of the likelihood has spread. Unlike frequentist methods, Bayesian methods are, in principle, free of the paradoxes and counterexamples that make classical statistics so perplexing. The development of hierarchical modeling and Markov Chain Monte Carlo (MCMC) methods have further added to the current popularity of the Bayesian approach, as they allow analyses of models that would otherwise be intractable.
Bayesian decision theory, which interacts closely with Bayesian statistical methods, is a useful way of modeling and addressing decision problems of experimental designs and data analysis and inference. It introduces the notion of utilities and the optimum decision combines probabilities of events with utilities by the calculation of expected utility and maximizing the latter (e.g., see the discussion in Lindley 2000).
Current research is attempting to use the Bayesian approach to hypothesis testing to provide tests and pvalues with good frequentist properties (see Bayarri and Berger 2000).
4.5 Broad Models: Nonparametrics And Semiparametrics
These models include parameter spaces of inﬁnite dimensions, whether addressed in a frequentist or Bayesian manner. In a sense, these models put more inferential weight on the assumption of conditional independence than does an ordinary parametric model.
4.6 Some Cross-Cutting Themes
Often diﬀerent ﬁelds of application of statistics need to address similar issues. For example, dimensionality of the parameter space is often a problem. As more parameters are added, the model will in general ﬁt better (at least no worse). Is the apparent gain in accuracy worth the reduction in parsimony? There are many diﬀerent ways to address this question in the various applied areas of statistics.
Another common theme, in some sense the obverse of the previous one, is the question of model selection and goodness of ﬁt. In what sense can one say that a set of observations is well-approximated by a particular distribution? (cf. Goodness of Fit: Overview). All statistical theory relies at some level on the use of formal models, and the appropriateness of those models and their detailed speciﬁcation are of concern to users of statistical methods, no matter which school of statistical inference they choose to work within.
5. Statistics In The Twenty-ﬁrst Century
5.1 Adapting And Generalizing Methodology
Statistics as a ﬁeld provides scientists with the basis for dealing with uncertainty, and, among other things, for generalizing from a sample to a population. There is a parallel sense in which statistics provides a basis for generalization: when similar tools are developed within speciﬁc substantive ﬁelds, such as experimental design methodology in agriculture and medicine, and sample surveys in economics and sociology. Statisticians have long recognized the common elements of such methodologies and have sought to develop generalized tools and theories to deal with these separate approaches (see e.g., Fienberg and Tanur 1989).
One hallmark of modern statistical science is the development of general frameworks that unify methodology. Thus the tools of Generalized Linear Models draw together methods for linear regression and analysis of various models with normal errors and those log-linear and logistic models for categorical data, in a broader and richer framework. Similarly, graphical models developed in the 1970s and 1980s use concepts of independence to integrate work in covariance section, decomposable log-linear models, and Markov random ﬁeld models, and produce new methodology as a consequence. And the latent variable approaches from psychometrics and sociology have been tied with simultaneous equation and measurement error models from econometrics into a broader theory of covariance analysis and structural equations models.
Another hallmark of modern statistical science is the borrowing of methods in one ﬁeld for application in another. One example is provided by Markov Chain Monte Carlo methods, now used widely in Bayesian statistics, which were ﬁrst used in physics. Survival analysis, used in biostatistics to model the disease-free time or time-to-mortality of medical patients, and analyzed as reliability in quality control studies, are now used in econometrics to measure the time until an unemployed person gets a job. We anticipate that this trend of methodological borrowing will continue across ﬁelds of application.
5.2 Where Will New Statistical Developments Be Focused ?
In the issues of its year 2000 volume, the Journal of the American Statistical Association explored both the state of the art of statistics in diverse areas of application, and that of theory and methods, through a series of vignettes or short articles. These essays provide an excellent supplement to the entries of this encyclopedia on a wide range of topics, not only presenting a snapshot of the current state of play in selected areas of the ﬁeld but also aﬀecting some speculation on the next generation of developments. In an afterword to the last set of these vignettes, Casella (2000) summarizes ﬁve overarching themes that he observed in reading through the entire collection:
(a) Large datasets.
(b) High-dimensional/nonparametric models.
(c) Accessible computing.
(d) Bayes/frequentist/who cares?
(e) Theory/applied/why diﬀerentiate?
Not surprisingly, these themes ﬁt well those that one can read into the statistical entries in this encyclopedia. The coming together of Bayesian and frequentist methods, for example, is illustrated by the movement of frequentists towards the use of hierarchical models and the regular consideration of frequentist properties of Bayesian procedures (e.g., Bayarri and Berger 2000). Similarly, MCMC methods are being widely used in non-Bayesian settings and, because they focus on long-run sequences of dependent draws from multivariate probability distributions, there are frequentist elements that are brought to bear in the study of the convergence of MCMC procedures. Thus the oft-made distinction between the diﬀerent schools of statistical inference (suggested in the preceding section) is not always clear in the context of real applications.
5.3 The Growing Importance Of Statistics Across The Social And Behavioral Sciences
Statistics touches on an increasing number of ﬁelds of application, in the social sciences as in other areas of scholarship. Historically, the closest links have been with economics; together these ﬁelds share parentage of econometrics. There are now vigorous interactions with political science, law, sociology, psychology, anthropology, archeology, history, and many others.
In some ﬁelds, the development of statistical methods has not been universally welcomed. Using these methods well and knowledgeably requires an understanding both of the substantive ﬁeld and of statistical methods. Sometimes this combination of skills has been diﬃcult to develop.
Statistical methods are having increasing success in addressing questions throughout the social and behavioral sciences. Data are being collected and analyzed on an increasing variety of subjects, and the analyses are becoming increasingly sharply focused on the issues of interest.
We do not anticipate, nor would we ﬁnd desirable, a future in which only statistical evidence was accepted in the social and behavioral sciences. There is room for, and need for, many diﬀerent approaches. Nonetheless, we expect the excellent progress made in statistical methods in the social and behavioral sciences in recent decades to continue and intensify.
- Barnard G A, Plackett R L 1985 Statistics in the United Kingdom, 1939–1945. In: Atkinson A C, Fienberg S E (eds.) A Celebration of Statistics: The ISI Centennial Volume. Springer-Verlag, New York, pp. 31–55
- Bayarri M J, Berger J O 2000 P values for composite null models (with discussion). Journal of the American Statistical Association 95: 1127–72
- Box J 1978 R. A. Fisher, The Life of a Scientist. Wiley, New York
- Casella G 2000 Afterword. Journal of the American Statistical Association 95: 1388
- Fienberg S E 1985 Statistical developments in World War II: An international perspective. In: Anthony C, Atkinson A C, Fienberg S E (eds.) A Celebration of Statistics: The ISI Centennial Volume. Springer-Verlag, New York, pp. 25–30
- Fienberg S E, Tanur J M 1989 Combining cognitive and statistical approaches to survey design. Science 243: 1017–22
- Fisher R A 1925 Statistical Methods for Research Workers. Oliver and Boyd, London
- Gigerenzer G, Swijtink Z, Porter T, Daston L, Beatty J, Kruger L 1989 The Empire of Chance. Cambridge University Press, Cambridge, UK
- Hacking I 1990 The Taming of Chance. Cambridge University Press, Cambridge, UK
- Jeﬀreys H 1938 Theory of Probability, 2nd edn. Clarendon Press, Oxford, UK
- Keynes J 1921 A Treatise on Probability. Macmillan, London
- Lindley D V 2000/1932 The philosophy of statistics (with discussion). The Statistician 49: 293–337
- Neyman J 1934 On the two diﬀerent aspects of the representative method: the method of stratiﬁed sampling and the method of purposive selection (with discussion). Journal of the Royal Statistical Society 97: 558–625
- Neyman J, Pearson E S 1928 On the use and interpretation of certain test criteria for purposes of statistical inference. Part I. Biometrika 20A: 175–240
- Neyman J, Pearson E S 1932 On the problem of the most eﬃcient tests of statistical hypotheses. Philosophical Transactions of the Royal Society, Series. A 231: 289–337
- Raiﬀa H, Schlaifer R 1961 Applied Statistical Decision Theory. Harvard Business School, Boston
- Ramsey F P 1926 Truth and probability. In: The Foundations of Mathematics and Other Logical Essays. Kegan Paul, London, pp.
- Savage L J 1954 The Foundations of Statistics. Wiley, New York
- Stigler S M 1986 The History of Statistics: The Measurement of Uncertainty Before 1900. Harvard University Press, Cambridge, MA
- Stigler S M 1999 Statistics on the Table: The History of Statistical Concepts and Methods. Harvard University Press, Cambridge, MA
- Tukey John W 1962 The future of data analysis. Annals of Mathematical Statistics 33: 1–67
- Wald A 1950 Statistical Decision Functions. Wiley, New York
- Wallis W 1980 The Statistical Research Group, 1942–1945 (with discussion). Journal of the American Statistical Association 75: 320–35