School Outcomes Research Paper

Classroom learning and school achievement broadly defined continue to grow in important ways (see, for example, Biddle et al. 1997). However, much of the recent scholarship on classroom processes has been conceptual in nature, and the associated empirical research (if any) typically involves only a few classrooms. There is considerable advocacy for certain types of classroom environments—classroom environments that are notably different from traditional instruction—but solid empirical support to show the effects of new instructional models is largely absent.

Here we discuss the need for new school-based research that addresses curriculum and instructional issues in order to advance theoretical understanding of student learning in school settings. We argue that it is no longer possible to discuss ‘successful schooling’ on the basis of subject-matter outcomes alone. Effective schools research and policy discussions must also include the measurement of non-subject-matter outcomes of schooling.

1. Status Of Achievement In Schools

The state of achievement in US schools is widely debated and incorporates at least three major camps. The first contends that current school performance is lower than that of US youth from the mid-1970s and that of international contemporaries. Indeed, several publications supported by the federal government have corroborated the assertion that US schools are failing, including Nation at Risk (National Commission for Excellence in Education 1983) and Prisoners of Time (National Education Commission on Time and Learning 1994).

A second camp argues that students’ performance today is as good as it was in the mid-1970s and that international comparisons are uninterpretable (Berliner and Biddle 1995). These and other researchers note that the stability of achievement has been maintained at a time when US schools are increasingly diverse and serving more minority students and those from poor homes than ever before. This camp also: (a) notes that reports of average school achievement are often highly misleading, because students’ performance varies markedly between schools; and (b) has concerns about the performance of minority students.

Importantly, there is documentable evidence not only of the stability of student achievement over time, but also of real progress. For example, in less than a decade, the number of students, including minority students, taking advanced placement tests in high schools for college credit doubled such that one-fifth of US students in 1996 entered college with credits earned from testing (Good and Braden 2000).

A third camp agrees with the second position, but asserts that traditional achievement standards are no longer applicable to a changing society—schools must address different, higher subject-matter standards. Although we accept the premise that some curriculum changes are probably necessary to accommodate the knowledge needs of a changing society, we have two major problems with advocates for new, higher standards. First, there has been no specification of what these new skills are and, typically, if examples are even provided, the rhetoric of higher standards essentially translates into moving content normally taught later (college calculus) to appear sooner (in high school). This position is specious, because it lacks sound theoretical and empirical support. Second, extant data suggest that achievement of students in states with ‘higher standards’ is lower than that of students in states with purported lower standards (Camilli and Firestone 1999)!

We align our arguments with advocates of the second camp, and we note that there is extensive evidence to support this position—student achievement scores are stable in the face of growing student diversity (Berliner and Biddle 1995, Good and Braden 2000). However, despite strong empirical data to show that student performance has not declined, we believe there is much room for improvement. For example, schools in urban settings need to improve student subject-matter performance, and all schools need to further enhance student non-subject-matter achievement.

The academic accomplishments of students in today’s schools are difficult to understand for various reasons. Here, we review several reasons why the achievement of US students is problematic.

2. Which Test Is Appropriate?

For one example of the difficulty of using extant assessment instruments to compare students within the USA or across cultures, one only need note the discrepancy between the measures. For example, as Bracey and Resnick (1998) report, in 1996 the National Assessment of Educational Progress (NAEP) mathematics performance indicated that only 18 percent of US fourth graders were proficient, only 2 percent were advanced, and 32 percent were below basic. In contrast, US fourth graders performed well above the average scores of the 26 countries that participated in the international TIMSS study at the fourth-grade level. Which test is the best descriptor? Are US students above average in fourth-grade mathematics, or not? This issue of ‘which test’ is embedded in US testing, as well. For example, in New York state there has been a raging debate about the appropriateness of a state-mandated test (i.e., in relation to other measures, such as advanced placement tests).

3. Problems With Curriculum, Theory, And Test Alignment

Other factors also limit efforts to relate the implemented curriculum with student achievement. For example, in mathematics, a current ‘theoretical’ recommendation is to avoid instruction that is a mile wide and an inch deep (i.e., in terms of the breadth and depth of curriculum concepts covered). Those who advise teachers about this problem apparently do not worry about the opposite problem—creating a curriculum that is four inches wide and a mile deep! Aligning standardized achievement tests with curriculum implementation is vital for meaningful interpretation of student scores. Appropriately, achievement tests should assess the philosophical orientation of the curriculum. Then, other tests and research can be used to assess the utility of a curriculum for various purposes (i.e., do various curricula have different effects on long-term memory of the curriculum taught or on students’ ability to transfer ideas learned in a curriculum to solve new problems?).

Finally, curriculum writers often do not understand the learning theory that undergirds particular instructional practices or goals. Some curriculum theorists do not appear to understand that behavioral models are powerful when dealing with factual and conceptual knowledge (and constructivist models are exceedingly weak in this area). Also, they often fail to understand that information-processing models (cognitive science) are more powerful than behavioral or constructivist principles at addressing wide horizontal transfer of academic concepts and authentic problem solving. In addition, in applying constructivist principles, many curriculum experts do not even recognize the differences among constructivist theories.

Despite the difficulties associated with assessment, instruments, and the curricula, and instructional alignment with achievement outcomes, there are data to suggest that teachers and schools can make a difference in student achievement.

4. Effective Teachers

The question that was problematic is the mid- 1970s—‘Do teachers make a difference in student learning?’—can now be answered with a definite ‘yes’ (Weinert and Helmke 1995). There are clear data to illustrate that some teachers notably outperform other teachers in helping similar students to learn the type of material historically measured on standardized achievement tests. It is not only possible to identify effective teachers on this useful but limited measure of student learning (e.g., see Good et al. 1983), but also research has illustrated that the practices of ‘successful’ teachers could be taught to other teachers.

As one instance of this research base, the Missouri Mathematics Program was successful in improving students’ performance in mathematics. However, a series of research studies in this program found that teachers who were involved in the training program implemented the intended program unevenly. Also, not surprisingly, higher levels of program implementation were associated with higher levels of student mathematical performance. Importantly, some of the ‘resistance’ to implementing the Missouri Mathematics Program was due to teachers’ beliefs about mathematics and how it should be taught (Good et al. 1983). Teachers’ beliefs are a key, but often overlooked, point in school reform. And, as Randi and Corno (1997) found, failure to take teachers’ ideas into account lowers the impact of many reform interventions.

Many aspects of teaching and learning are still problematic. Although there are numerous plausible case examples of teachers who impact students’ ability to think and to apply knowledge, there is still debate about how teachers obtain these ‘higher order’ outcomes and whether other teachers can be educated in ways that allow them to achieve comparable effects on their students. Unfortunately, research on teachers’ roles in helping students to develop thinking and problem-solving skills has in recent times been limited to case-study research. Further, research that helps teachers to expand—and assess—their capacity for impacting students’ thinking abilities has been inert.

5. Effective Schools

There is some evidence that schools (serving similar populations of students) have more effect on student achievement (as measured by conventional standardized achievement tests) than do other schools (e.g., Good and Weinstein 1986). However, unlike the correlational and experimental research on teacher effects, the early research base describing ‘effective schools’ was sparse and questionable. Teddlie and Stringfield’s (1993) longitudinal study in Louisiana has provided strong evidence that schools make a difference. They found that schools serving similar populations of students had different ‘climates’ and that these school differences were associated with differential student achievement. In general, the study confirmed many of the arguments that school effects researchers had made previously. Interestingly, in their eight-year study, about one-half of the schools retained their relative effectiveness during the time period (stability rates were similar for schools that had initially been defined as effective or ineffective). More work on this issue is warranted.

There still have not been successful and replicated studies to show that factors associated with ‘effective’ schools can be implemented in other schools in ways that enhance student achievement. Although there are notable examples that schools can be transformed, there is comparatively little research to see if this knowledge can be used to improve other schools.

6. Comprehensive School Programs

In recent years, intervention using components of previous teacher and school effects research and emerging ideas (e.g., cooperative student groups) have been combined in ‘comprehensive school programs’ designed to transform all aspects of schooling at the same time (including governance, structure, instruction, home schooling, communication, curriculum, and evaluation!). Although these broad interventions offer potential, it should be noted that the theoretical assumptions and congruence of these programs have not been assessed. Some programs appear to have a serious misalignment between program components. For example, as McCaslin and Good (1992) noted, some schools emphasize a behavioral approach to classroom management (‘do as you are told’) and a constructivist approach to the curriculum (‘think and argue your conception’).

Recent school intervention efforts have focused upon the impact of broad programs on student achievement. Typically, research includes no observational data, and thus there is no evidence of what parts of the program are implemented and/or whether the program parts that are presented represent a theoretically integrated program or a ‘patchwork quilt’ of separate and conflicting program parts. Further, the more complex and comprehensive the intervention model, the more likely teachers will alter program parts (often permanently). These ‘teachable’ moments go unnoticed by reformers. Perhaps unsurprisingly, there is growing consensus that such programs have not had consistent, demonstrable effects on students’ achievement (Slavin 1999).

7. Little Support For New Research

In the past decade, policy leaders have tended to agree with advocates of camp one about the effectiveness of schools. Those in camp one (and who believe that schools are failing radically) have been able to convince policymakers to invest widely in charter and voucher plans, even in the absence of any convincing pilot data. To these advocates, both the problem and solution are evident. Those in the second camp (stable student achievement over time) have been ‘represented’ by researchers attempting to implement comprehensive school reform. Unfortunately, as noted above, such research attempts fail to show the effects of program implementation. This group of researchers does not question the knowledge base sufficiently and hence fails to develop research designs that could improve its conceptions of ‘good practice.’

Those who fall into the third camp (new societal requirements mandating new processes and outcomes of schooling) are prepared to reject extant theory and research on the basis that it is outdated and irrelevant. Indeed, it is not only that monolithic state-mandated standardized tests have gotten in the way of scientific research, but also that there is a growing willingness of professional groups to advocate desirable teaching processes on theoretical grounds alone. Prestigious groups, such as the National Council of Teachers of Mathematics (NCTM), have encouraged curriculum development without advocacy for research on instruction and have encouraged teachers to teach less and to use more student group work. Although it is perhaps the case that at an aggregated level, teachers as a group should teach less, the real issue is about when teachers should teach and when students should explore alone or with other students in small or large groups.

8. Controllable And Uncontrollable Influences On Student Learning

It has become better understood in the last 25 years of the twentieth century that schools can influence students’ learning independent of home background. However, it is also better understood that the effects of schools are mediated by other factors. Bracey and Resnick (1998) argue that many factors impacting students are within the control of the school district, including: aligning textbooks with standards; aligning the broader curriculum with standards; employing high-quality teachers and providing them with appropriate professional development; and having modern textbooks and laboratory equipment. Although these factors are controllable in theory, it should be noted that some school districts have considerably fewer resources to work with than do other districts. It seems unconscionable that all schools do not receive these controllable resources, which are known to impact achievement.

Bracey and Resnick (1998) have argued that there are various out-of-school factors that can influence student achievement, including: teenage pregnancy rates; percentage of female-headed households; poverty rate; parental and maternal educational levels; the percentage of students for whom English is not their native language; number of violent incidents per year; and number of annual policy visits disciplinary actions that occur at particular schools. These opportunity-to-learn variables are seen as conditions that impact student achievement but are largely beyond the direct influence of the school system. Accordingly, various groups have noted that achievement problems of students from low-income families need to be addressed by other social agencies. Clearly, poor achievement in schools is often outside the control of individual schools, but not outside the influence of the broader society. If a society chooses to hold individual schools ‘accountable’ for achievement, it is incumbent on that society to spread its wealth in ways that address factors that individual schools cannot address.

9. Improving Performance: Aligning Theory And Research

Massive investments of public funds in student testing have occurred over the last two decades of the twentieth century and, as noted, some policymakers have called for even more testing and for raising extant standards. The research for higher standards has led to the development of tests in some states that are designed, absurdly, so that most students fail them. Oddly, despite this massive investment in testing, no information has been obtained to describe adequately what students learn in schools and how conditions can be changed to further enhance student learning. We believe that school reform efforts need to bring theory and empirical research together. It is time to stop advocacy from prestigious groups that do not support their reform calls in the form of theory and data.

There is a growing sense of the importance of instructional balance in order to scaffold the learning needs of students in complex classroom settings (Good and Brophy 2000, Grouws and Cebulla 2000). However, these assertions demand more research examination in various contexts. In contrast, this advice is currently not echoed by professional groups. These groups continue to overemphasize (in our opinion) problem solving and application without concomitant attention to the development of conceptual knowledge. The ability to solve problems also requires the ability to recognize and find problems as well as the computational and conceptual skills necessary for addressing problems. Simply put, there can be too little or too much attention to concepts, facts, or problem solving.

10. Non-Subject-Matter Outcomes Of Schooling

Americans have always argued that their students should not simply be ‘academic nerds,’ but that students should also be participants in broader society (have jobs, participate in drama or sports, etc.). Increasingly, parents and citizens have expressed interest in schools being more responsive to the nonsubject-matter needs of youth. Often, this argument is expressed only in terms of improving achievement—if children feel secure, they will achieve better. But increasingly, the assertion is made that some outcomes of schooling are important whether or not they impact subject-matter learning.

Despite the considerable advocacy of policy leaders for higher standards, many citizens are pushing hard for students to learn responsibility. For example, when asked what first comes to their minds when Americans think about today’s teenagers, 67 percent described them as ‘rude, irresponsible, and wild,’ and 41 percent reported that they did not think teens had enough to do and that they were not being taught the value of hard work (schools are not doing their ‘job’) (Farkas et al. 1997). This idea is echoed in the 1999 Phi Delta /Kappan Gallup poll, where 43 percent of parents reported that the main emphasis of schools should be on students’ academic skills, and 47 percent of parents said they believe that the main emphasis of schools should be helping students to learn to take responsibility. And earlier in a 1996 Phi Delta/Kappan Gallup poll, parents indicated they would prefer their children to receive bad grades and be active in school activities rather than making an excellent grade and not participating in extracurricular activities (60 percent to 20 percent). It is clear that parents and citizens, reacting to a perception of adolescents’ moral decline, believe that schools should play a greater role in teaching non-subject-matter content, such as responsibility and civic behavior.

Although citizens want schools to provide students with more non-subject-matter knowledge, there are problems with the systematic addition of this ‘content’ to the curriculum. One problematic issue is the difficulty in finding consensus on what a moral curriculum would look like. Subsequently, what assessment criteria should be used to measure the curriculum’s effectiveness? Is the curriculum successful if more students vote? Or is it effective if students are able to negotiate conflicts calmly? As we have argued, achievement tests do not always represent adequately the curricula they are supposed to measure. Therefore, it is reasonable to think that appropriate accountability and assessment tools would be difficult to implement for non-subject-matter content—at least without adequate empirical research.

Policymakers, however, are not only supporting past emphases of the impact of schools on achievement, but are pressuring for higher and higher standards. In some instances, the standards have been pushed so high (and artificially high) that in Virginia and in Arizona over 90 percent of students failed exams that presumably represented efforts by state government to improve standards. At some level, increasing standards may be a laudable goal; however, developing standards that only 10 percent of our students pass seems to be a ridiculous and self-defeating effort. Given the exaggerated, negative views of youth, and testing policies designed to guarantee youth’s failure, it seems important to explore motivation. Are such practices designed to suggest that youth, especially minority youth, do not deserve resources in their schools?

11. Educators On Noncognitive Factors

Rothstein (2000) has noted that, in part, achievement measures are reified in making decisions about the effectiveness of schooling, because there are many of these measures, but few ways to measure noncognitive outcomes. He noted that there are 10 core outcomes that motivate citizens to invest in schools so that students can contribute to society and lead productive adult lives. The dimensions he identifies are literacy, competencies in mathematics, science, and technology, problem solving, foreign languages, history knowledge, responsible democratic citizenship, interest in creative arts, sound wellness practices, teamwork and social ethics, and equity (narrowing the gap between minority and white students in the other nine areas).

Wanlass (2000) has drawn attention to the numerous problems in contemporary society, including the effects of a global economy, increased ethnic diversity, advanced technology, and increasing risk for various problems such as intolerance (hate crimes), overpopulation, and a dramatic depletion of natural resources. In this context, she argued a need to help youth develop the affective dispositions required for the modern world (leadership, tolerance, philanthropic concern). She suggested several strategies, capitalizing on the unique strengths and talents of more students and the need to better develop the unique abilities of nonacademic areas as well as academic concerns (performing arts, social interaction, leadership, civic-mindedness, social advocacy, etc.).

Goodenow (1992) has argued that students who feel more connected to their schools are more likely to be academically and socially successful. Students who felt more personally valued and invested in were more likely to place higher value and have higher expectations for classroom success than were students who did not feel valued. This finding was replicated in a large-scale study (Blum and Rinehart 1996), which surveyed more than 90,000 students nationwide and found that those who felt more connected to their schools were less likely to engage in risky behavior, including: smoking, doing drugs, engaging in sexual activity, and repeated truancy, and they were less likely to drop out of school. In this study, school connectedness was defined as students who: (a) felt they were treated fairly, (b) reported they got along with other teachers and students, and (c) felt close to people at school. Connectedness was also explored in terms of how schools were perceived by their teachers and administrators. A school that encouraged more connectedness exhibited less student prejudice and had higher daily attendance rates, lower drop out rates, and more teachers employed with masters degrees. Although these data are correlational in nature, it suggests that some factors thought to be uncontrollable, may in fact be controllable.

Researchers have also investigated the role of nonsubject-matter variables as they influence students’ motivation to achieve. For example, Nichols (1997) examined the nature of the relationship between students’ perceptions of fairness and their reported levels of motivation to achieve. Of the four dimensions of fairness identified through factor analytic techniques (personal, rules, punishment, teacher), a personal sense of fairness more strongly mediated motivational variables than did other dimensions. Similar to Goodenow (1992), this data set indicated that students who felt more personally valued were more likely to be motivated to achieve in school settings. However, even if students’ personal sense of fairness was not linked directly to an achievement motivation, it would seem that this is a desirable outcome of schooling. Students should feel respected and safe, whether or not these variables influence achievement.

12. Conclusion

Clearly, more public debate is called for, if more focused judgments are to be reached concerning those aspects of schools that are defined as most critical. However, given the growing evidence that citizens are concerned about non-subject-matter outcomes of schooling, it seems incumbent on policymakers to ‘proxy’ these concerns by collecting data on some of the many outcomes that could be measured. There is a need for research and arguments focused on nonsubject-matter variables, as well. For example, it seems that high subject-matter specialization and excessively high levels of student obedience (as opposed to student initiatives) are not a prescription for forging a professional community well prepared for engaging in creative work and problem solving.

A recent poll of teenagers found that the three greatest pressures they experience are to get good grades (44 percent), to get into college (32 percent), and to fit in socially. In contrast, fewer teens report feeling pressure to use drugs or alcohol (19 percent) or to be sexually active (13 percent). What modern youth are concerned about varies markedly from the view presented by the media. Unfortunately, policymakers at present seem more intent on blaming youth and holding them accountable than on understanding them. The collection of reliable data on various non-subject-matter outcomes of schooling might help to assure that youth are developing the pro- active skills necessary for life in a democracy (e.g., civility, a willingness to examine ideas critically, and ambition).

The historical concern for evidence about subjectmatter growth should continue—particularly if these measures are collected in ways that provide a basis for curriculum improvement. However, it is important to recognize that US public schools are about much more than subject-matter acquisition. More research on students and how they mediate social, emotional, and academic contexts might enable educators to design better school programs that recognize students both as academic learners and as social beings (McCaslin and Good 1992).

Bibliography:

1. Berliner D, Biddle B 1995 The Manufactured Crisis: Myth, Fraud and the Attack on America’s Public Schools. Addison-Wesley, New York
2. Biddle B, Good T, Goodson I 1997 Teachers: The International Handbook of Teachers and Teaching. Kluwer, Dordrecht, The Netherlands, Vols. 1 and 2
3. Blum R W, Rinehart P M 1996 Reducing the Risk: Connections That Make a Difference in the Li es of Youth. University of Minnesota, Division of General Pediatrics and Adolescent Health, Minneapolis, MN
4. Bracey G W, Resnick M A 1998 Raising the Bar: A School Board Primer on Student Achievement. National School Boards Association, Alexandria, VA
5. Camilli G, Firestone W 1999 Values and state ratings: An examination of the state-by-state indicators in quality counts. Educational Measurement: Issues and Practice 35: 17–25
6. Farkas S, Johnson J, Duffett A, Bers A 1997 Kids These Days: What Americans Really Think About The Next Generation. Public Agenda, New York
7. Good T L, Braden J 2000 The Great School Debate: Choice, Vouchers, and Charters. Erlbaum, Mahwah, NJ
8. Good T L, Brophy J 2000 Looking in Classrooms, 8th edn. Longman, New York
9. Good T L, Weinstein R 1986 Schools make a difference: Evidence, criticisms, and new directions. American Psychologist 41: 1090–7
10. Good T L, Grouws D A, Ebmeier H 1983 Active Mathematics Teaching. Longman, New York
11. Goodenow C 1992 School motivation, engagement, and sense of belonging among urban adolescent students. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA
12. Grouws D, Cebulla K 2000 Elementary and middle school mathematics at the crossroads. In: Good T (ed.) American Education: Yesterday, Today, and Tomorrow. Ninety-Ninth Yearbook of the National Society for the Study of Education. University of Chicago Press, Chicago, Chapter 5, pp. 209–55
13. McCaslin M, Good T 1992 Compliant cognition: The misalliance of management and instructional goals and current school reform. Educational Researcher 21: 4–17
14. National Commission for Excellence in Education 1983 A Nation At Risk: The Imperatives for Education Reform. US Department of Education, National Commission for Excellence in Education, Washington, DC
15. National Education Commission on Time and Learning 1994 Prisoners of Time. US Government Printing Office, Washington, DC
16. Nichols S 1997 Students in the Classroom: Engagement and Perceptions of Fairness. Master’s thesis, University of Arizona
17. Randi J, Corno L 1997 Teachers as innovators. In: Biddle B, Good T, Goodson I (eds.) International Handbook of Teachers and Teaching. Kluwer, Dordrecht, The Netherlands, Vol. 2, Chap. 12, pp. 1163–221
18. Rothstein R 2000 Toward a composite index of school performance. Elementary School Journal 100(5): 409–42
19. Slavin R 1999 How Title I can become the engine of reform in America’s schools. In: Orfield G, Debray E (eds.) Hard Work for Good Schools: Facts Not Fads In Title I Reform. The Civil Rights Project, Harvard University, Cambridge, MA, Chap. 7, pp. 86–101
20. Teddlie C, Stringfield S 1993 Schools Make a Difference: Lessons Learned From a 10-Year Study of School Eff Teachers College Press, New York
21. Wanlass Y 2000 Broadening the concept of learning and school competence. Elementary School Journal 100(5): 513–28
22. Weinert L, Helmke A 1995 Interclassroom differences in instructional quality and interindividual differences in cognitive development. Educational Psychologist 30: 15–20