Bullying and Crime Research Paper

This research paper reports on an updated systematic review and meta-analysis that was undertaken under the aegis of the Swedish National Council for Crime Prevention and further supported by the British Academy and conducted by the current authors. Only longitudinal prospective studies were included in the review, which aimed to examine to what extent school bullying predicts later offending and violence. Significant effect sizes were found even after controlling for other major childhood risk factors. Being a bully increased the likelihood of being an offender by more than half and increased the likelihood of being violent by two thirds. These results either reflect the persistence of an underlying aggressive or antisocial tendency or a facilitating effect of school bullying on later offending and violence (or both).

The implication is that high quality bullying prevention programs (and possibly multiple component programs which also target aggression) should be promoted. They could be viewed as an early form of crime prevention. They can potentially have long-term effects by improving the future psychosocial adjustment of school bullies and reducing the associated health, welfare, education, and other costs.

Introduction

School bullying has recently become a topic of major public concern and has attracted a lot of media attention, with articles in major newspapers and magazines reporting cases of children who committed (or attempted) suicide because of their victimization at school and parents suing school authorities for their failure to protect their offspring from continued bullying victimization (e.g., Ttofi and Farrington 2012). There is, nevertheless, a number of “skeptics” who still perceive school bullying as being part of a normal developmental process, or as one of those school experiences that prepare children for the grown-up world. Scientific evidence regarding possible detrimental effects of school bullying on children’s mental health and future psychosocial adjustment can only be provided through a systematic review and meta-analysis, providing an unbiased standardized effect size and defining the magnitude of the effect.

Background Research on Bullying and Crime

School bullying is a special category of aggressive behavior involving repeated unprovoked acts against less powerful (emotionally or physically) individuals (Farrington 1993; Olweus 1993). Of course, schools, like other institutions, will always be a place in which the basic human motive of aggression will be demonstrated. However, school bullying should not be confused with more or less normal aggressive interactions such as rough and tumble play.

Scientific interest in the problem of bullying and its negative short-term and long-term effects emerged after the well-publicized suicides of three Norwegian boys in 1982, which were attributed to severe peer bullying (Olweus 1993). School bullying has gradually become a topic of major public concern via “bullying awareness days,” national initiatives in various (European) countries (Smith and Brain 2000), and anti-bullying research networks across the world (e.g., Anti-Bullying Alliance; BRNET; International Observatory for Violence in Schools; PREVNet).

Any suggestion regarding the short-term negative impact of peer aggression and victimization seems reasonable even to the lay mind. Establishing, on the other hand, the long-term effects of school bullying and arguing that children involved in peer aggression are more likely to follow an antisocial path (compared with noninvolved students) is more challenging. Some early longitudinal studies did provide evidence of the long-term impact of school bullying and, notably, established the intergenerational transmission of school bullying. In the Cambridge Study in Delinquent Development, for example, boys who were bullies at age fourteen tended, at age thirty-two, to have children who were bullies (Farrington 1993). As another example, in his follow-up study of over 700 Stockholm boys, Olweus (1993) reported that 36 % of bullies at ages thirteen to sixteen were convicted three or more times between ages sixteen and twenty-four, compared with 10 % of the remainder.

There have been surprisingly few recently published longitudinal studies on the developmental pathways of children involved in school bullying since the seminal work of Olweus in Scandinavia and some other European examples. Two special issues in peer-reviewed journals have recently been published in an attempt to address this gap in research literature (Farrington et al. 2011; Ttofi et al. 2011a). Both issues presented new findings on the long-term negative consequences of school bullying based on major prospective longitudinal studies from around the world. Longitudinal investigators of twenty-nine studies conducted analyses for a more comprehensive British Academy funded project, which examined the long-term association of school bullying with both internalizing (such as anxiety, self-esteem, and stress) and externalizing (such as aggression, alcohol, and drug use) problems (see Farrington et al. 2012, Table 4, for a list of all contributors).

The special issue of Criminal Behaviour and Mental Health focused on the association between bullying perpetration at school and offending later in life. A systematic review and meta-analysis on the topic was carried out (Ttofi et al. 2011c). The special issue of the Journal of Aggression, Conflict and Peace Research focused on the association between bullying victimization (i.e., being bullied) and internalizing problems later in life, such as anxiety and depression. A systematic review and meta-analysis was again carried out examining the extent to which bullying victimization at school predicted depression (Ttofi et al. 2011b), showing that the probability of being depressed up to seven years later in life (M = 7.13 years; SD = 8.79) was significantly higher for victims of school bullying than for control students, i.e., children not involved in school bullying.

Building upon the above-mentioned research activities, an effort was made to update the relevant systematic reviews (Farrington et al. 2012) and to study further outcomes, such as violence (Ttofi et al. 2012). This research paper presents results from the updated systematic review on the association of school bullying with offending later in life. Additional analyses are presented on the long-term link of bullying with violence.

Bullying and Crime Research Methods

The main objectives of the systematic review were two-fold. Firstly, to assess whether bullying at school (perpetration and victimization) was a significant risk factor predicting offending and violence later in life (unadjusted effect sizes). Secondly, to assess whether these associations were still significant after controlling for other major childhood risk factors, measured at the baseline period (adjusted effect sizes). Results on offending and violence were carefully treated in separate analyses and the outcome measures under each category generally did not overlap. However, it is possible that in some studies, outcome measures such as “police arrests” would include violence. “Offending” included outcome measures such as police or court contact, property offending, criminal convictions, property theft, vandalism, shoplifting, vehicle theft, etc. “Violence” included outcome measures such as forced sexual contact, criminal violence, physical fights, violent convictions, violent offending, weapon carrying, assault, etc.

Further analyses were conducted to investigate moderators that might explain variability in effect sizes, such as the age at which bullying was measured (Time 1), the age at which the outcome measures were taken (Time 2), the number of covariates controlled for in the adjusted effect sizes, the length of the follow-up period (measured in years), and the way in which the outcomes were measured (i.e., official data versus self-reports).

Stringent inclusion and exclusion criteria were set in advance. For example, reports were included only if they were based on prospective longitudinal data. The predictor must have been a measure of school bullying (and not other more general forms of peer aggression/victimization) and must have preceded the outcome (i.e., offending or violence). A clear measurement of offending and/or violence must have been included in the report as an outcome measure. Studies were included if participants were school-aged children in the community and exposure to bullying (perpetration and victimization) specified the school years. Published and unpublished reports of the literature were included in order to minimize the possibility of publication bias in the results.

Reports were excluded if the character of the data was qualitative in nature (e.g., qualitative data based on interviews) and did not allow calculation of an effect size. This did not apply if a qualitative method (e.g., interviews or observation studies) was used to obtain a quantitative measure. If the outcome measure (offending or violence) was part of a wider theoretical construct (e.g., externalizing or antisocial behavior), then the relevant report was again excluded. Reports based on clinic samples or incarcerated youth were also excluded.

Extensive searches were carried out and a detailed description of them can be found in the Swedish report (Farrington et al. 2012) and the most recent work focusing on violent outcomes (Ttofi et al. 2012). In total, the same searching strategies were repeated in 19 electronic databases, and the full volumes of 63 journals were searched either online or in print. In the Swedish report, readers can also find detailed tables of the key features of each report, such as the sample size, the country where the study took place, the exact confounds controlled for at the baseline period, etc.

Bullying and Crime Research Results

In total, 661 reports concerned with the association of school bullying with internalizing (e.g., anxiety, depression, self-esteem, etc.) and externalizing (e.g., aggressive behavior, conduct problems, offending, etc.) problems were located. All reports were screened in line with the inclusion and exclusion criteria and classified in five different categories (see Farrington et al. 2012, Table 5). Further to a detailed screening of all manuscripts, a total number of 48 reports, corresponding to 29 longitudinal studies, presented data on the long-term association of school bullying (perpetration and victimization) with offending in adolescence or young adulthood (see Farrington et al. 2012, Table 6). A total number of 51 reports from 28 longitudinal studies were included in the systematic review on the association of school bullying (perpetration and victimization) with violence in adolescence or young adulthood (see Ttofi et al. 2012, Table 1).

When different manuscripts relating to the same longitudinal study reported different effect sizes (because of differences, e.g., in the sample size or in the follow-up period that the authors have used), the combination of effect sizes across reports is not straightforward as these effect sizes are based on dependent samples. These dependencies were taken into account, as ignoring them would result in standard errors that were too small, often by a large degree. Advice from leading experts in the field was sought on this matter (Wilson 2010). Clear rules were set in advance for computing effect sizes across reports from the same longitudinal study (see Farrington et al. 2012; Ttofi et al. 2012).

Bullying Perpetration At School And Offending Later In Life

Eighteen studies provided an effect size for bullying perpetration versus offending. The summary unadjusted effect size across the 18 studies was OR = 2.64 (95 % CI: 2.17–3.20; z = 9.83) for the random-effects model. The random-effects model was used since the heterogeneity test, Q, of 84.89 was highly significant at p = 0.0001. When the three studies with only unadjusted effect sizes were excluded, the summary effect size for the remaining 15 studies – for the random-effects model – was O= 2.54 (95 % CI: 2.05–3.14, z=8.52). Again, there was significant variability in effect sizes across these studies (Q =76.03, p ¼ 0.0001).

After controlling for covariates, the adjusted summary effect size was reduced to OR=1.89 (95 % CI: 1.60–2.23; z =7.49) but this was still highly significant (see Farrington et al. 2012; Figs. 3 and 4). This OR indicates quite a strong relationship between bullying perpetration and later offending. For example, if a quarter of children were bullies and a quarter were offenders, this value of the OR would correspond to 34.5 % of bullies becoming offenders, compared with 21.8 % of non-bullies. Thus, being a bully increases the risk of being an offender (even after controlling for other childhood risk factors) by more than half.

For the adjusted summary effect size, various moderators were investigated to explain the heterogeneity in effect sizes across studies, which was significant (Q= 36.82, p=0.001). These included the number of covariates controlled for at baseline (range: 1–20; M= 7.00; SD=5.22), the age at which school bullying was measured (range: 6.23–15.54; M =11.26; SD=2.68), the age of participants when outcome measures were taken (range: 10.00–24.64; M = 17.10; SD = 4.91), and the length of the follow-up period, measured in years (range: 0.42–16.50; M = 5.84; SD = 4.56).

The age at which bullying was measured was positively associated with the effect size, but the regression coefficient was not statistically significant (B = 0.019, SE = 0.024, p= 0.428). The length of the follow-up period was significantly negatively associated with the effect size (B =- 0.027, SE = 0.012, p = 0.018). As expected, the age of the study participants when outcome measures were taken was significantly negatively related to the effect size (B = -0.025, SE = 0.012, p = 0.039). The above two negative relationships suggest that bullying perpetration has a stronger effect in the short term. The relationship between the number of covariates controlled for and the effect size was in the expected negative direction and also significant (B = – 0.027, SE = 0.013, p = 0.037). Therefore, the adjusted effect size decreased as the number of covariates controlled for increased.

Other moderators that may explain variability in effect sizes include the type of longitudinal studies (i.e., prospective versus retrospective) and the way in which the outcomes were measured (i.e., official data versus self-reports). In the Farrington et al. (2012) report, the reader can obtain information about these moderators (see their Table 6). Only three studies out of fifteen presented outcome measures based on official records for offending, making a moderator analysis inappropriate (due to uneven study numbers). Finally, only one study presented results based on a retrospective measure of bullying victimization, so any analyses on this matter would be meaningless.

If the studies included in a meta-analysis are a biased sample of all relevant studies, then the mean effect computed will reflect this bias (Borenstein et al. 2009, p. 277). It is clear from our thorough searching strategies that every precaution was taken to ensure that all eligible studies would be represented in the meta-analysis. In order to further increase the validity of the meta-analysis findings, a number of publication bias analyses were carried out.

Firstly, the Duval and Tweedie’s Trim-andFill procedure was used. This technique displays the differences in effect sizes that could be attributable to bias by imputing effect sizes until the error distribution more closely approximates normality, offering the best estimate of the unbiased effect size (Borenstein et al. 2009, p. 286). No imputed effect sizes appeared on the relevant funnel plot (they would have been presented as solid black dots; see Farrington et al. 2012, Fig. 6), indicating no publication bias. The imputed summary effect size (represented by a solid black diamond) had not shifted at all.

Indeed, under the fixed effect model, the point estimate and 95 % confidence interval for the combined studies was 1.86 (95 % CI: 1.71–2.03). Using Trim-and-Fill procedure, these values remained unchanged. Under the random-effects model, the point estimate and 95 % confidence interval for the combined studies was 1.89 (95 % CI: 1.60–2.23). Using Trim-and-Fill procedure, these values were again unchanged.

Furthermore, Rosenthal’s Fail-Safe N test (Rosenthal 1979) was conducted. One concern of publication bias is that some nonsignificant studies are missing from a given analysis and that these studies, if included, would nullify the observed effect. Rosenthal suggested that, rather than simply speculate about the impact of the missing studies, we compute the number of nonsignificant studies that would be required to nullify the effect. If this number is small, then there is reason for concern because some nonsignificant studies may have been never communicated to the scientific community (e.g., due to “publication bias”). However, if this number is large, one can be confident that the treatment effect, while possibly inflated by the exclusion of some studies, is nevertheless not zero.

Bullying Perpetration At School And Violence Later In Life

A total number of 15 studies were concerned with the association of bullying perpetration with aggression and violence later in life. The unadjusted summary effect size across these studies was OR = 3.09 (95 % CI: 2.35–4.07; z = 8.10). For one study, only an unadjusted effect size was available. The unadjusted effect size for the remaining 14 studies was OR = 2.97 (95 % CI: 2.25–3.92; z = 7.71; Q = 151.81, p =0.0001; I2 = 91.44). All individual studies yielded a significant effect size (see Ttofi et al. 2012, Fig. 1). After controlling for covariates, the adjusted summary effect size was reduced to OR = 2.04 (95 % CI: 1.69–2.45; z = 7.53) but this was still highly significant (see Ttofi et al. 2012, Fig. 2). This OR indicates quite a strong relationship between bullying perpetration and later violence. For example, if a quarter of children were bullies and a quarter were violent, this value of OR would correspond to 35.8 % of bullies becoming violent, compared with 21.4 % of non-bullies. Thus, being a bully increases the risk of being violent (even after controlling for other childhood risk factors) by two thirds.

Although all individual studies yielded an effect size supporting the link between school bullying and aggression/violence later in life, the magnitude and the significance of the effect varied across these studies. Various moderator analyses were conducted in order to explain this variability (Q = 75.801, p = 0.0001, I2 = 82.85). These included the number of covariates controlled for at baseline (range: 2–20; M = 6.93; SD = 5.25), the age at which school bullying was measured (range: 8.00–15.54; M = 12.04; SD = 2.35), the age of participants when outcome measures were taken (range: 10.00–24.64; M = 17.65; SD = 4.83), and the length of the follow-up period, measured in years (range: 0.42–16.50; M = 5.61; SD = 4.88).

The age of participants when bullying was measured was significantly negatively correlated with the effect size (B = 0.065; SE = 0.021; p=0.002), suggesting that the younger the children were when they exhibited this form of problem behavior, the more likely it was that they would be violent later in life. The age of participants when outcome measures were taken was also significantly negatively related to the effect size (B = – 0.033; SE = 0.009; p = 0.0005). In other words, the lower the age of the participants when aggression or violence was measured, the larger the effect, possibly because this was associated with a shorter follow-up period. This is consistent with the significant negative association between the length of follow-up period and the magnitude of the effect size (B = – 0.017; SE = 0.009; p = 0.051). As expected, the magnitude of the effect size decreased as the number of confounds controlled for increased (B = – 0.013; Intercept = 0.668; SE = 0.010; p = 0.185), but the relevant regression coefficient was not significant.

As with the previous meta-analysis, a number of sensitivity analyses were conducted. Firstly, the Duval and Tweedie’s Trim-and-Fill procedure was performed. Three imputed effect sizes appeared on the relevant funnel plot (see Ttofi et al. 2012, Fig. 3) and the imputed summary effect size (represented by a solid black diamond) had shifted slightly, suggesting a trivial overestimation of the summary effect size.

As already mentioned, the difference was very small. Under the fixed effect model, the point estimate and 95 % confidence interval for the combined studies was 1.83 (95 % CI: 1.71–1.95). Using Trim-and-Fill procedure, the imputed point estimate was 1.76 (95% CI: 1.65–1.88). Under the random-effects model, the point estimate and 95 % confidence interval for the combined studies was 2.04 (95% CI: 1.69–2.45). Using Trim-and-Fill procedure, the imputed point estimate was 1.77 (95 % CI: 1.45–2.16).

Finally, the Rosenthal’s Fail-Safe N test was performed. This meta-analysis incorporated data from 14 studies, which yielded a z-value of 17.12216 and corresponding 2-tailed p-value of 0.000001. The fail-safe N is 1055. This means that one would need to locate and include 1055 “null” studies in order for the combined 2-tailed p-value to exceed 0.050. Put another way, 75.4 missing studies would be needed for every observed study for the effect to be nullified. It is impossible that such a large number of studies were conducted but not published or not included in our analysis.

Further Findings

Further analyses were performed to examine the association of bullying victimization with later offending before (Unadjusted OR =1.32; 95 % CI: 1.13–1.55, z = 3.40) and after controlling for other major childhood risk factors (Adjusted OR = 1.14; 95 % CI: 0.997–1.310, z=1.91) and relevant forest plots are shown in the Farrington et al. (2012) report (see Figs. 11 and 12). This was a very weak relationship. Moderator analyses and publication bias analyses similar to those presented in the current entry were also presented in that report.

Finally, analyses were performed to examine the association of bullying victimization with later violence before (Unadjusted OR = 1.65; 95 % CI: 1.42–1.92; z = 6.48) and after (Adjusted OR = 1.42; 95 % CI: 1.248–1.6172; z = 5.3117) controlling for covariates (see Ttofi et al. 2012, Figs. 4 and 5). Again, moderator analyses and publication bias analyses similar to those presented in the current entry are also presented in that report.

Sensitivity analyses were performed for these two sets of meta-analyses and the results showed in general no evidence of publication bias (see Farrington et al. 2012; Ttofi et al. 2012).

Possible Controversies In The Literature

The results of these systematic reviews and meta-analyses suggest that there are long-term detrimental effects of school bullying on later offending and violence. This was even the case when confounded variables that are risk factors for bullying or victimization as well as the undesirable outcomes were controlled for. Therefore, one can conclude that school bullying is an independent predictor of the later psychosocial development of perpetrators as well as of victims. It is the first time that this conclusion is not only based on a few selected primary studies and narrative reviews, but, instead, on comprehensive meta-analyses of prospective longitudinal studies that included new data from a substantial body of yet unpublished research. The findings remained robust in sensitivity analyses testing potential publication biases, of which there was no sign.

The relation of bullying perpetration with later offending and violence might reflect the persistence of an underlying disposition for antisocial behavior that has different manifestations over time (Farrington 1993; Lo¨ sel and Bliesener 2003). However, as the relation remained after controlling for other childhood risk factors, bullying perpetration may also increase the likelihood of later offending and violence.

Of course, one should acknowledge that any direct mention of causality should be carefully treated. Although most studies use bullying as the predictor of later outcomes, implying in this way a specific temporal sequence, alternative models have been suggested. Very few bullying studies have examined alternative models on whether bullying is a cause or a consequence of psychopathologic behavior (e.g., Boulton et al. 2010; Kim et al. 2006). This is not a trivial matter and it would shed more light on the temporal sequence and the causal ordering between bullying and other internalizing or externalizing behaviors. The substantial adjusted effect size for victimization versus later depression found in a previous meta-analysis (Farrington et al. 2012; Ttofi et al. 2011b), for example, suggests in a way that the frequent internalizing symptoms of victims are not only a trigger for being bullied, but a psychological consequence.

Systematic reviews on risk factors are important as they can advance theory and also help to develop effective prevention programs (Murray et al. 2009). For example, it would be interesting to examine whether victims of bullying suffer from low self-esteem or whether school bullies lack cognitive or affective empathy. Such findings, based on relevant systematic reviews, could guide future intervention initiatives, while also refining theory about the causes of bullying perpetration and victimization.

Open Questions And Future Research Directions

In the current meta-analysis, studies were included and analyzed based on “level analyses.” Levels of bullying perpetration were compared with later levels of offending and violence. It would also have been interesting to complete a systematic review on “change analyses,” examining whether changes in bullying from Time 1 to Time 2 are followed by changes in an outcome from Time 2 to Time 3. However, there are hardly any studies on this matter, since such analyses would require relevant data from multiple waves. Such analyses would allow, to an extent, making safer inferences about causality, although change data are subject to greater variability than level data. Systematic reviews of longitudinal studies which control for confounded variables can give some hints on whether variables are simple correlational risk factors, risk markers, or causal risk factors (Kraemer et al. 2005).

Future research should also examine possible gender-specific and ethnic-specific effects of bullying on later violent behavior and offending. Such information was hardly ever available in the current literature. To investigate and disentangle the impact of these and other variables on the relation between bullying and later outcomes, more longitudinal studies with a sound control for childhood risk factors are needed. The results of meta-regression analyses were not always as expected in the meta-analyses for the British Academy project on “Health and Criminal Outcomes of School Bullying” because of the large differences in the type of covariates researchers controlled for. However, one should note that the lack of a sufficient number of studies with consistent patterns of characteristics is a typical problem in meta-analyses (Lipsey 2003).

Future research should also examine mediators or possible causal mechanisms between school bullying and the various outcomes. The underlying mechanisms, for example, may be the reinforcement obtained by dominating others and the development of an identity as a “bully” that goes beyond the school context.

Conclusions

This is the first time that research has provided an unbiased standardized effect size regarding the predictive efficiency of school bullying in relation to violence and offending later in life. The significant summary effect sizes have important implications for policy and practice as they give a stronger voice to anti-bullying agencies and reestablish the moral imperative of school communities to create an appropriate violence-free school climate.

High quality bullying prevention programs should be promoted (Farrington and Ttofi 2009; Ttofi and Farrington 2011). They could be viewed as an early form of crime prevention. These programs can potentially have long-term effects by improving the future psychosocial adjustment of school bullies and reducing the associated health, welfare, education, and other costs. The effectiveness of other school-based programs for the prevention of problem behaviors has been examined through thorough systematic reviews (e.g., Wilson et al. 2001) and it is possible that such programs, or other general multicomponent programs, might have positive effects in reducing aggression and bullying behavior.

Previous research has provided strong evidence about the monetary value of saving high-risk youth (Cohen and Piquero 2009). Children involved in school bullying are undoubtedly youth at risk, with significantly higher probabilities of following an antisocial path. What remains unanswered is the identification of protective factors that interrupt the continuity from school bullying to later adverse outcomes and confer resiliency on this special category of high-risk youth (Ttofi and Farrington 2012).

Bibliography:

1. Borenstein M, Hedges LV, Julian PTH, Rothstein HR (2009) Introduction to meta-analysis. Willey, Chichester
2. Boulton MJ, Smith PK, Cowie H (2010) Short-term longitudinal relationships between children’s peer victimization/bullying experiences and self-perceptions: evidence for reciprocity. School Psychol Int 31(3):296–311
3. Cohen MA, Piquero AR (2009) New evidence on the monetary value of saving a high risk youth. J Quant Criminol 25:25–49
4. Farrington DP (1993) Understanding and preventing bullying. In: Tonry M (ed) Crime and justice, vol 17. University of Chicago Press, Chicago
5. Farrington DP, Ttofi MM (2009) School-based programs to reduce bullying and victimization. Campbell Systematic Reviews 6
6. Farrington DP, Ttofi MM, Losel F (2011) Editorial: school bullying and later offending. Crim Behav Mental Health 21(2):77–79
7. Farrington DP, Losel F, Ttofi MM, Theodorakis N (2012) School bullying, depression and offending behaviour later in life: an updated systematic review of longitudinal studies. Swedish National Council for Crime Prevention, Stockholm
8. Kim YS, Leventhal BL, Koh Y-J, Hubbard A, Boyce TW (2006) School bullying and youth violence: causes or consequences of psychopathologic behavior? Arch Gen Psychiat 63:1035–1041
9. Kraemer HC, Lowe KK, Kupfer DJ (2005) To your health: how to understand what research tells us about risks. Oxford University Press, New York
10. Lipsey MW (2003) Those confounded moderators in meta-analysis: good, bad, and ugly. Ann Am Acad Polit Soc Sci 587(1):69–81
11. Losel F, Bliesener T (2003) Aggression und delinquenz unter jugendlichen: Untersuchungen von kognitiven und sozialen Bedingunger. Luchterhand, Hermann
12. Murray J, Farrington DP, Eisner MP (2009) Drawing conclusions about causes from systematic reviews of risk factors: the Cambridge quality checklists. J Exp Criminol 5(1):1–23
13. Olweus D (1993) Bullying at school: what we know and what we can do. Blackwell, Oxford
14. Rosenthal R (1979) The “File drawer problem” and tolerance for null results. Psychological Bulletin 86:638– 641.
15. Smith PK, Brain P (2000) Bullying in schools: lessons from two decates of research. Aggressive Behavior 26:1–9
16. Ttofi MM, Farrington DP (2011) Effectiveness of school-based programs to reduce bullying: a systematic and meta-analytic review. J Exp Criminol 7(1):27–56
17. Ttofi MM, Farrington DP (2012) Risk and protective factors, longitudinal research and bullying prevention. New Direct Youth Develop 133:85–98
18. Ttofi MM, Farrington DP, Losel F (2011a) Editorial: health consequences of school bullying. J Aggress Conflict Peace Res 3(2):60–62
19. Ttofi MM, Farrington DP, Losel F, Loeber R (2011b) Do the victims of school bullies tend to become depressed later in life? A systematic review and meta-analysis of longitudinal studies. J Aggress Conflict Peace Res 3(2):63–73
20. Ttofi MM, Farrington DP, Losel F, Loeber R (2011c) The predictive efficiency of school bullying versus later offending: a systematic/meta-analytic review of longitudinal studies. Crim Behav Mental Health 21(2):80–89
21. Ttofi MM, Farrington DP, Losel F (2012) School bullying as a risk marker for aggression and violence later in life: a systematic review and meta-analysis of prospective longitudinal studies. Aggression and Violent Behavior (Accepted, in press)
22. Wilson DB, Gottfredson DC, Najaka SS (2001) School-based prevention of problem behaviors: a meta-analysis. J Quant Criminol 17(3):247–272

More Bullying Research Paper Examples:

• Bullying Research Paper
  • Bullying Prevention Research Paper
  • Bullying and Crime Research Paper
  • Bullying in School Research Paper