Unobtrusive Measures Research Paper

View sample Unobtrusive Measures Research Paper. Browse other  research paper examples and check the list of research paper topics for more inspiration. If you need a religion research paper written according to all the academic standards, you can always turn to our experienced writers for help. This is how your paper can get an A! Feel free to contact our custom writing service for professional assistance. We offer high-quality assignments for reasonable rates.

1. Introduction

All measures must be considered to be subject to error. We would have no way of knowing if, perchance, a measure turned out to be absolutely accurate. For example, the National Bureau of Standards has a 10gram weight which, because of a manufacturing error, weighs just less than 10 grams by about 400 micrograms (the weight of a grain or two of salt) (Freedman et al. 1991). Despite the most careful weighing, done on a weekly basis, the values actually obtained for the standard weight vary by about 15 micrograms (one microgram is about the weight of a speck of dust) either way from the mean. Presumably the mean of a long series of measures is the best estimate of the true weight of the standard, but there is no way of being sure of the true weight. Obviously, the way to deal with such errors, usually thought of as random errors, is to do the same measure several times. That is why careful carpenters measure their boards more than once before cutting them—especially if the wood is expensive. Random errors are associated with the concept of reliability—the expectation that if a measurement is performed twice under exactly the same conditions, very close agreement should be obtained if the measurement instrument and procedures for using it are dependable.

Another type of measurement error, though, is bias, i.e., consistent deviation from the ‘true’ value of the object or phenomenon of interest. For example, a recent news story recounted speculation that, in order to enhance the prospects of players for careers in professional football, some collegiate coaches might have arranged for the running tracks on which the players’ speed is tested to be slightly downhill. If that is true, then the running speeds of players from some universities may be biased toward the fast end of the scale. If mothers are asked to estimate the intelligence (or good looks!) of their children, we might expect some bias to be evident, i.e., higher estimates than would be expected from other means of assessment. The bias is not reduced by replicating the measurement. Downhill is still downhill, and mother’s love is constant.

The problem of bias has to be dealt with either by knowing the degree of bias so one can allow for it, or by using multiple measures that do not share sources of error, i.e., bias. A football scout who knows that a player has been tested on a track with a downhill slant can ‘allow’ for that in interpreting reported running speed. We may expect that mothers will exaggerate a bit in describing their children’s skills or other virtues and discount the glowing adjectives in the descriptions by some amount. But an even better way to deal with bias is to use other measures that are less biased, or not biased at all in the same way as the original measure. Of course, if a scout knows that a track has a downhill bias, that scout may simply retest the athlete on a track known to be quite level and disregard the first, presumably biased, report. A scout might also use game films to arrive at a judgment of the speed of the player.

In social and behavioral sciences we usually have no absolute standard that can be applied, and we often should not assume that any one measure is less biased than another or unbiased altogether. The best solution is to use multiple measures of constructs and pick or devise those measures in such a way that they have minimally overlapping sources of error. A sports scout may not have any way of knowing that other running tracks are level, but if an athlete is tested on three different tracks, the average speed of those three runs is likely to be less biased (as well as having less random error) than any one of the runs. And, if all three tracks have a downhill slant, then maybe most tracks do, and the estimate of the athlete’s speed is not biased with respect to estimates for other athletes.

2. Problems Associated With Single And Reactive Measures

Social science measures are susceptible to many sources of bias, but a few of those sources of bias are particularly important because they are common and may involve relatively large degrees of bias. Different sources of bias are inherent in every measurement procedure. The biases can pose threats to the valid interpretation of a measure, in a manner akin to plausible rival hypotheses as threats to validity of experiments (Campbell and Stanley 1963).

3. Reactive Measurement Effects

A problem in measurement stems from the fact that the very act of measurement may produce a reaction on the part of the measured object that changes that object, whether temporarily or permanently. One cannot determine the tensile strength of a wire without breaking it, and one cannot determine whether a cake is as delicious as it looks without spoiling it to some degree. Asking to measure a person’s height usually results in that person assuming a very erect posture that produces an estimate of height that is taller than their ‘walking around’ height. Similarly, giving a group of people a ‘bigotry’ scale is likely to result in an underestimate of the mean level of bigotry in the group, even if the scale is not explicitly labeled ‘Bigotry Scale.’ Some reactive measurement effects may be of limited duration or generalizability. The effect of bigotry measurement is unlikely to persist, but others may be long lasting, e.g., as when asking a person his or her opinion about something results in reflection leading to crystallization of an opinion that may have previously been ephemeral.

Scientists and others involved in measurement in social sciences often attempt to reduce reactivity in various ways, including telling respondents that their responses will be confidential, or even anonymous, or that their responses will be used only for scientific purposes. Often such generic efforts to reduce reactivity are not enough, or at least not enough to be completely reassuring, in which case the response of the investigator may be to try to identify and employ a nonreactive, or at least less reactive, measure. Although all measures must be considered reactive to some degree, they are not all reactive to the same measurement issues or to the same degree. Other measures may be thought to be relatively low in reactivity because the data on which they are based were collected in spatially and temporally remote ways that should have freed them from contemporary biases. One example would be the use by historians of personal letters in order to diagnose the state of mind of the letter writer at some earlier period in his or her life.

The most obvious and attractive way of reducing reactivity is to carry out measurement activities under conditions that do not require the subject to know that he or she is being measured. Such measures have come to be known as unobtrusive measures (Webb et al. 1966). A young businesswoman might, for example, be invited out for lunch without knowing that she was actually being ‘sized up’ for a promotion. Or, computers in a school might be programmed to record sign-ons by individual students so as to provide a check on students’ reports of computer use and study time. Measures may be unobtrusive because they are embedded in or derived from ongoing activities that do not make measurement a salient feature of the activity or because they are concealed in some way. The term Unobtrusive has come to be used generically for nonreactive measures, but it would be better used to refer specifically to measures obtained without the necessity for persons being measured to be aware of the fact of being measured. It is for that reason that the original publication, Unobtrusive Measures: NonReactive Measures in the Social Sciences (Webb et al. 1966) was retitled NonReactive Measures in the Social Sciences (Webb et al. 1981) in the revised version of the book. Some measures, e.g., many physiological measures, are not highly reactive (susceptible to bias) because they are not under voluntary control; other measures are not particularly reactive because the person being measured is mistaken about the purpose for which they are being measured; and still others because the measurement context is conducive to unbiased responding.

When people are aware of being measured, a frequent consequence is bias, even if the participant is cooperative and well intentioned. People being ‘tested’ often, very naturally, want to make a good impression, although in some specific instances, a person being measured may want to make what would ordinarily seem to be a bad impression, e.g., malingering. People being measured may adopt specialized roles that reflect their ideas about how they ought to behave, and those roles may not be characteristic of them when they are in other situations. Individuals will differ, of course, in the degree to which their characteristic responses are affected by knowledge of being measured. Bias may be limited in some instances because people believe their responses are completely appropriate. Criminals, for example, often believe that their behaviors are quite justified and therefore feel no need to disguise their statements about them.

Biases can be reduced with the implementation of multiple measures that do not share the same sources of bias. Unobtrusive measures are not free from their own sources of error, but they may not reflect the same type or degree of bias or reactivity as other measures. Unobtrusive measures, because they reflect different approaches to measurement, can often get around the limitations of reactive measures.

4. The Varieties Of Unobtrusive Measures

Webb et al. (1966, 1981) provide an extensive review of unobtrusive or nonreactive measures in social science research and many fine examples can be found throughout the literature. Generally speaking, unobtrusive measures can be usefully categorized as follows:

(a) Simple observations. Many interesting behaviors can be directly observed, often without the necessity of the observed actor being aware of the fact. Observations may be made of individuals and groups. Targets of observation may include objects and events as well as persons. For example, the nature of ceremonial events may be of great interest. Also included here are observations of physical location and clustering of people, expressive movements, language behavior in the media or as overheard in public areas, the amount of time individuals spend gazing at public displays, or time sampling of observations to determine whether certain occurrences are linked temporally. A recent news story reported an observation that military dictators who begin to detect resistance and who desire to establish their legitimacy tend to abandon their military uniforms and don civilian attire, with recent appearances of Saddam Hussain of Iraq in regular business suits cited as an interesting example.

(b) Contrived observations. Simple observation can be tedious as one may need to wait a long while for behavior of interest to occur. Under such circumstances, researchers may contrive situations likely to produce relevant responses. The TV show ‘You’re on Candid Camera!’ was based on responses of people to contrived situations. Social psychologists make extensive use of confederates in contrived situations in which the subject is unaware that he or she is a participant in an experiment. Also included in this category of contrived situations are the experiments in which unsuspecting individuals are deceived into doing things and studied in the process. Real estate companies are routinely tested for racially discriminatory practices by persons pretending to be clients. Bochner (1979) has made ‘wrong number’ phone calls and staged fake collapses on trains to assess individual’s helpfulness in common situations. These types of investigations can be classified as nonreactive or unobtrusive because the participants are unaware that they are participating in an experiment. This lack of awareness implies that their reactions will be natural. Contrivances may include the use of hidden hardware such as audio-and videotapes.

(c) Physical traces. These include both erosion and accretion measures. Measures of erosion include measurements of floor tile or carpet wear in front of various museum exhibits to determine their popularity, or the wear and tear of library books to examine selective reader interest. Measures of accretion include indicators of inscriptions in public restrooms, the amount of debris left from a ticker-tape parade, or the number of cigarette packs that were thrown out with the trash in selected residential areas. Detectives regularly rely on physical traces of responses they have never seen to solve crimes.

(d) Archives. These include actuarial records. Materials supplied by the mass media, industrial and institutional records, sales records, and private written documents are archival records. Archival records are regularly used to track behaviors that might not otherwise be detected with reactive measures. Episodic personal and private records are in heavy use in hearings involving activities in the White House to check on the veracity of self-reports by those whose actions are in question.

It is important that unobtrusive measures not be regarded as substitutes for other kinds of reactive measures. That is the antithesis of the rationale behind unobtrusive measures. Unobtrusive measures are properly thought of as complementary to other measures such as questionnaires and interviews.

5. The Uses Of Unobtrusive Measures

The countering of bias in measurement is of focal concern in considering the usefulness of unobtrusive measures, but such measures may actually be adopted for other purposes. That is in part because an investigator may follow different strategies for dealing with bias: verification, adjustment, and avoidance. An investigator may believe that measures being used are not greatly biased, but he or she may also believe that caution requires sensitivity to possible bias. That investigator may elect to use one or more nonreactive measures in order to probe for bias in case it exists. For example, in a workplace study of attempts to induce exercise, a researcher may ask respondents how often they walk up stairs rather than take an elevator, and reports of workers might be regarded as generally unbiased. Nonetheless, the investigator might still elect to probe the accuracy of those reports by using occasional observers to determine whether the proportion of people climbing stairs is consistent with self-reports. If it should appear that workers tend to exaggerate their reported use of stairs, the investigator might change the questionnaire to try to elicit better data or abandon self-reports in favor of reliance on observers. A prudent investigator would, at the very least, be cautious in interpreting self-report data. Probing might not require large numbers of observations or observations of all subjects of interest. A second strategy is to collect sufficient data by alternative means to make it possible to estimate by how much the numbers obtained by a primary measure (likely to be a questionnaire) are out so that estimates can be appropriately discounted. Observational data might indicate that reported use of stairs is exaggerated by 25 percent, in which case the researcher might deflate his estimates of total physical activity by an appropriate amount. Obviously, unobtrusive measures may be used in conjunction with other measures to arrive at a summary assessment that is likely to be less biased than would have been the case for a single measure. If trainees in a program claim to be spending 10 hours per week in a computer laboratory, but actual counts of people working in the laboratory are not consistent with the claimed level of use, a researcher might adjust downward estimates of exposure to the exercises involved.

It is not always easy, however, to obtain unobtrusive measures that can be used directly and exactly in order to adjust for biases in other measures. One major problem is that more common measures such as questionnaires and unobtrusive or other nonreactive measures are not in the same metrics, so that combining them is not easy. What would one do, for example, with questionnaire data indicating that people claim to eat four servings of vegetables every day and observational data indicating that many people fail to finish eating and discard portions of vegetables served to them? Such discrepant data may lead one to question the accuracy of one set of data or the other, but the discrepancies may not translate easily into quantitative adjustments.

Which data should one trust? Certainly our inclination would likely be to distrust questionnaire and other self-report data. Campbell (1969) suggested as much in discussing the role of qualitative data in program evaluation. Emily Dickinson once wrote the line in a poem ‘I like a look of anguish because I know it’s true.’ That is likely to strike all of us as obviously true. Try to imagine the alternative statement ‘I like a self-report of anguish because I know it’s true.’ And yet, there certainly must be times when self-reports are better assessments of underlying dispositions than overt behaviors. Think, for example, of the pressures toward conformity that lead people who may not be at all religious to bow their heads when other people pray, or the politeness that may prevail between politicians who dislike each other. The use of multiple and different measures cannot guarantee anything.

Under some circumstances, nonreactive measures may be available and be used alone. That may be because they are sometimes compelling, but also because they are sometimes inexpensive. In particular, archival records may be exploited, often at fairly low cost. If records are kept for reasons unrelated to any particular policy use, or at least for reasons unrelated to the purposes for which they are used in research, they may be of great value and characterized by very little bias. The identification by Dr. John Snow of a contaminated water supply as the cause of a cholera outbreak in London in the 1850s was facilitated by records kept of the location of individual cases of the infection, making it possible for Snow to map those cases onto alternate water systems (Freedman 1991). Barthel and Holmes (1968) were able to use information in high school yearbooks to show that persons who later became schizophrenic had low levels of social activity before the onset of their illness.

6. Advantages Of Unobtrusive Measures

Aside from the possibility of reducing bias in measurement, several other advantages are often associated with unobtrusive measures (Rathje 1979, Babbie 1989). Typically, unobtrusive measures require little, if any, effort on the part of persons being assessed, and often no physical contact with, or even close proximity to participants is required. Unobtrusive measures tend to focus on behaviors and obviate the problems that may stem from inaccurate reporting on the part of respondents. Yet another advantage is that the employment of nonreactive measurement procedures is often relatively inexpensive, e.g., simple observations, physical traces, and archival records. They can be of great value in longitudinal studies.

7. Limitations Of Unobtrusive Measures

Unobtrusive measures, like all other measures, have limitations. In the first place, it is not always easy to identify unobtrusive measures. Unobtrusive methods are typically limited in certain areas that can be open to interrogation with more reactive methods such as interviews and questionnaires. Methods that utilize verbal communication, such as structured and unstructured interviews, have ‘an ability to reach into all content areas’ (Webb et al. 1966, 1981). Webb et al. (1981) provided a ‘generative taxonomy’ for nonreactive measures in order to facilitate thinking about them, but very often coming up with good ones is more an act of creative thinking than of straightforward science. No rules govern the process, and hence, no limitations exist either. That means, however, that one cannot guarantee that an unobtrusive measure will be accepted by reviewers or readers of one’s work. Unobtrusive measures are often novel and have no history of use in a field by means of which the case for them can be buttressed. Moreover, many unobtrusive measures do not readily fit the requirements of conventional psychometric analyses so that the usual indicators of difficulty level, variance, and so on are not available, and reliability cannot be directly computed. In the end, unobtrusive measures must have face validity, i.e., readers must see immediately and intuitively that they make sense. That is not always likely.

Unobtrusive measures may sometimes raise troubling ethical questions, for they may be obtained under conditions that at least appear to violate usual expectations about informed consent, confidentiality, and so on. Records may have been assembled with no expectation that they would ever be used for anything other than their original purposes, and even if confidentiality is protected at the public level, some people might feel that their personal confidence is breached when researchers gain access to their records. People may feel that even public behavior is in some sense private if they have no expectation of being systematically observed. In fact, some courts have ruled that people riding in automobiles have expectations of privacy that should protect them against at least some types of observation. Ethical concerns may not rule out many unobtrusive measures, but at least they require careful consideration by researchers.

Furthermore, unobtrusive measures are not always inexpensive. Observers, for example, are expensive to train and support in the field, and unless they can produce information that is markedly better than what can be derived from questionnaires, they may not be affordable. Archives may be readily available, but the cost of mining them for usable data may be quite high. Many other types of unobtrusive measures may turn out to be surprisingly expensive. Questionnaires and interviews may be performed by mail, phone, or web sites, and the expenses associated with them, especially if multiple sites are involved, can prohibit them from being favorite in the array of methods potentially available. As noted earlier, however, unobtrusive measures may still be useful at modest cost when used as probes into the quality and bias of more traditional measures.

Bibliography:

  1. Babbie E R 1989 The Practice of Social Research, 5th edn. Wadsworth Publishing, Belmont, CA
  2. Barthel C N, Holmes D S 1968 High school yearbooks: A nonreactive measure of social isolation in graduates who later become schizophrenic. Journal of Abnormal Psychology 73: 313–16
  3. Bochner S 1979 Designing unobtrusive field experiments in social psychology. In: Sechrest L (ed.) Unobtrusive Measurement Today. Jossey-Bass, San Francisco, CA
  4. Campbell D T 1969 Reforms as experiments. American Psychologist 24: 409–29
  5. Campbell D T, Stanley J C 1963 Experimental and quasiexperimental designs for research on teaching. In: Gage N L (ed.) Handbook of Research on Teaching: A Project of the American Education Research Association. Rand McNally, Chicago, IL
  6. Freedman D A 1991 Statistical models and shoe leather. In: Marsden P V (ed.). Sociological Methodology 21: 291–313
  7. Freedman D, Pisani R, Purves R 1991 Statistics, 2nd edn. Norton, New York
  8. Rathje W L 1979 Trace measures. In: Sechrest L (ed.) Unobtrusive Measurement Today. Jossey-Bass, San Francisco, CA
  9. Sechrest L (ed.) 1979 Unobtrusive Measurement Today. JosseyBass, San Francisco, CA
  10. Sechrest L, Phillips M 1979 Unobtrusive measures: An overview. In: Sechrest L (ed.) Unobtrusive Measurement Today. JosseyBass, San Francisco, CA
  11. Webb E J, Campbell D, Schwartz R, Sechrest L 1966 Unobtrusive Measures: NonReactive Measures in the Social Sciences. Rand McNally, Chicago, IL
  12. Webb E J, Campbell D, Schwartz R, Sechrest L, Grove J 1981 Non-Reactive Measures in the Social Sciences, 2nd edn. Houghton Mifflin, Boston, MA
Urban Activity Patterns Research Paper
History Of Universities Research Paper

ORDER HIGH QUALITY CUSTOM PAPER


Always on-time

Plagiarism-Free

100% Confidentiality
Special offer! Get discount 10% for the first order. Promo code: cd1a428655