PROTOCOL: Effectiveness of Programs to Prevent School Bullying

The definition of school bullying includes several key elements: physical, verbal, or psychological attack or intimidation that is intended to cause fear, distress, or harm to the victim; an imbalance of power (psychological or physical), with more powerful child (or children) oppressing less powerful ones; and repeated incidents between the same children over a prolonged period (Farrington, 1993; Olweus, 1993; Roland, 1989). School bullying can occur in school or on the way to or from school. It is not bullying when two persons of the same strength (physical, psychological, or verbal) victimize each other. Bullying is different from aggression or violence; not all aggression/violence involves bullying, and not all bullying involves aggression/violence. For example, bullying includes being called nasty names, being rejected, ostracized or excluded from activities, having rumors spread about you, having belongings taken away, teasing and threatening (Baldry & Farrington, 1999). Our aim is to review programs that are specifically intended to prevent or reduce school bullying, not programs that are intended to prevent or reduce school aggression or violence. School bullying is perceived to be an important social problem in many different countries. The nature and extent of the problem, and research on it, in 21 different countries, is reviewed in Smith et al. (1999). Special methods are needed to study bullying in different countries because of problems of translating the term “bullying” into different languages. Smith et al. (2002) have reviewed the meaning of bullying in 14 different countries. In studying bullying, researchers often ask about specific components such as “hit him/her on the face” or “excluded him/her from games” rather than use the actual word “bullying” in their interviews and questionnaires (Pateraki & Houndoumadi, 2001). However, they still use the word “bullying” in their reports. Many school-based intervention programs have been devised and implemented in an attempt to reduce school bullying. These have been targeted on bullies, victims, peers, teachers, or on the school in general. Many programs seem to have been based on commensense ideas about what might reduce bullying rather than on empirically-supported theories of why children bully, why children become victims, or why bullying events occur. The first large-scale anti-bullying program was implemented in Norway in 1983. A more intensive version of the national program was evaluated in Bergen by Olweus (1991). This program aimed to increase awareness and knowledge of teachers, parents, and students about bullying and to dispel myths about it. Booklets were distributed to schools, folders were distributed to families, and videos about bullying were shown and discussed in schools. Also, schools received information about the nature and extent of bullying (based on questionnaires completed by students), teachers were encouraged to develop explicit rules about bullying, and monitoring and supervision of children in the playground was improved. The evaluation by Olweus (1991) showed a dramatic decrease in victimization of about half after the program. Since then at least 15 other large-scale anti-bullying programs, some inspired by Olweus and some based on other principles, have been implemented and evaluated in at least 10 other countries. Baldry and Farrington (2007) reviewed 16 major evaluations in 11 different countries and concluded that 8 produced desirable results, 2 produced mixed results, 4 produced small or negligible effects, and 2 produced undesirable results. Most programs were quite complex, and the effectiveness of different components of programs was not clear. These 16 evaluations are listed after the References. The best single source of reports of anti-bullying programs is the book edited by Smith, Pepler, and Rigby (2004), which contains descriptions of 13 programs in 11 different countries. The most relevant existing reviews are by Smith et al. (2004), who summarized effect sizes in 14 whole-school anti-bullying programs, and by Vreeman and Carroll (2007), who reviewed 26 school-based programs. These two prior reviews are of high quality. However, neither carried out a full meta-analysis measuring weighted mean effect sizes and correlations between study features and effect sizes. The Smith et al. (2004) review covered only 14 evaluations up to 2002, 6 of which were uncontrolled. The Vreeman and Carroll (2007) review covered 26 evaluations up to 2004, and was restricted to studies published in the English language. We hope to go beyond these previous reviews by (a) searching for evaluations up to the end of 2007, (b) searching for international evaluations, and (c) carrying out more extensive meta-analyses, as specified in this protocol. American research is generally targeted on school violence or peer victimization rather than bullying. There are a number of existing reviews of school violence programs and school-based interventions for aggressive behavior (e.g. Howard et al., 1999; Mytton et al., 2006; Wilson et al., 2003; Wilson & Lipsey, 2007). We will consult these, but we must emphasize that our research aims to review anti-bullying programs specifically. The main objective is to assess the effectiveness of school-based anti-bullying programs in reducing school bullying. Our aim is to locate and summarize all the major evaluations of programs in developed countries. Bullying has been studied in Australia, Austria, Belgium, Canada, Cyprus, Denmark, England and Wales, Finland, France, Germany, Greece, Iceland, Ireland, Israel, Italy, Luxembourg, Japan, Malta, New Zealand, Northern Ireland, Norway, Portugal, Scotland, Spain, Sweden, Switzerland, The Netherlands, and the United States (Smith et al., 1999). We aim (potentially) to include research in all these countries. We aim to measure effect sizes in each evaluation and to investigate which features (e.g. of programs, students, and schools) are related to effect sizes. We hope to make recommendations about which components of programs are most effective in which circumstances, and hence about how future anti-bullying programs might be improved. We also hope to make recommendations about how the design and analysis of evaluations of anti-bullying programs might be improved in future. We propose the following criteria for inclusion of studies in our systematic review: There is an additional criterion for inclusion in our meta-analysis (but not in our systematic review): However, the reviewers of our protocol argued that useful information was contained in smaller studies, that the minimum sample size of 200 was arbitrary, that methodological quality and publication bias should be addressed in other ways, that attrition is equally important in smaller and larger studies, and that larger studies do not necessarily have higher external validity. Based on Vreeman and Carroll (2007), we estimate that about half of all evaluations have an initial sample size of 200 or more, and conversely half have an initial sample size less than this. Therefore, not setting a minimum sample size doubles the number of studies included in our systematic review. We plan to analyze larger and smaller studies separately to investigate whether the results obtained with the two types of studies are concordant or discordant. We propose to search as many as possible of the following electronic databases using the following keywords: We will not include Violence or Aggression as key words along with Bully/Bullies/Anti-Bullying because we know that this will identify many studies that are not relevant to the present review. We have considered whether to experiment with key words such as Intimidation/Harrassment/Teasing/Victim* but, as mentioned above, our aim is to review studies of interventions designed specifically to reduce school bullying. List of Databases Australian Criminology Database (CINCH) Cochrane Controlled Trials Register C2-SPECTR Criminal Justice Abstracts Database of Abstracts of Reviews of Effectiveness (DARE) Dissertation Abstracts Educational Resources Information Clearinghouse (ERIC) EMBASE Google Scholar MEDLINE National Criminal Justice Reference Service (NCJRS) PsychInfo/Psychlit Sociological Abstracts Social Sciences Citation Index (SSCI) Our experience is that electronic searches are very time-consuming and identify few previously unknown studies. We propose to handsearch the following key journals since 1983: Aggression and Violent Behavior Aggressive Behavior Archives of Pediatrics and Adolescent Medicine British Journal of Educational psychology Child Development Criminal Justice and Behavior Developmental Psychology Educational Psychology International Journal on Violence and Schools Journal of Adolescence Journal of Educational Psychology Journal of Interpersonal Violence Journal of School Health Journal of School Psychology Journal of School Violence Journal of Emotional Abuse School Psychology International School Psychology Review Victims and Offenders Violence and Victims Very few studies randomly assign students to experimental or control groups. A few studies randomly assign schools or school classes (e.g. Cross et al., 2004). Some studies have before and after measures of bullying in experimental and control schools or school classes (e.g. Alsaker & Valkenover, 2001). Some studies compare students of a particular age before the intervention with students of the same age in the same schools after the intervention (e.g. Olweus, 2005). Many studies have before and after measures of bullying in experimental schools but no control schools or school classes (e.g. Pitts & Smith, 1995). These latter studies will be excluded. Most studies measure bullying and victimization using student self-report questionnaires. Information is usually collected separately about whether students bully and whether students are bullied. Some studies also use teacher ratings, peer ratings, or (more rarely) playground observations or school records. We propose to calculate effect size measures of bullying and victimization, based on self-report questionnaires, in as many studies as possible. Our main meta-analysis will be based on this measure. Where self-report questionnaires are not available, we will calculate effect size measures based on other sources. Where short-term and long-term follow-up measures are available, we will calculate effect sizes separately for the shortest and longest follow-up periods. We propose to cover the following topics at least: The Appendix contains a draft coding schedule. A random sample of 20 studies will be coded by two persons in order to measure reliability. (see Lipsey & Wilson, 2001, p. 202). Where before and after information is provided, the OR and d will be adjusted for the before data. Effect size measures will be calculated for bullying and victimization separately. Weighted mean effect size measures will be calculated using the procedures described in Lipsey and Wilson (2001). Where appropriate, fixed effects or random effects models will be used (depending on the heterogeneity Q). Correlations between features of studies and effect sizes will be investigated, in an attempt to assess which components of programs might be the most effective in which circumstances. Meta-analytic regressions will also be carried out to investigate independent influences of program components, methodological quality, features of participants, and design features. As mentioned, we hope to make recommendations about how to make anti-bullying programs more effective and about how to improve the design and evaluation of such programs, including recommendations about analytic techniques that the researchers should use. Ideally, we could produce a “CONSORT” statement for reports of anti-bullying programs that specifies what information should always be reported. Statistical problems arise when the unit of allocation (e.g. schools or school classes) is different from the unit of analysis (e.g. students). This problem has rarely been addressed in the bullying literature. We will address it using formulae to correct test statistics for clustering developed by Cathleen McHugh based on Hedges (2007). We will carry out a sensitivity analysis assuming intraclass correlations between .10 and .20, as used by the What Works Clearinghouse of the U.S. Department of Education (Lipsey, 2008). We will also address the possibility of publication bias by using techniques described by Rothstein, Sutton, and Borenstein (2005), such as the funnel plot. The review will be focussed on quantitative studies but will include qualitative information (e.g. on the degree of implementation of interventions) where this is helpful in discussing explanations for findings or conflicting results. Our tentative time schedule is as follows. We have already completed most of the searches. February, 2008: Develop/test coding scheme March-April: Code included studies, analyze results May-June 2008: Complete first draft of review and submit it to Campbell Collaboration Later in 2008: Receive feedback from Campbell Collaboration. Submit final draft of review for electronic publication on the Campbell website. The review will be updated every 3 years. The lead reviewer will take the lead in arranging this. We are very grateful to David Wilson, Jeff Valentine and two anonymous reviewers for helpful comments. The only possible (minor) conflict of interest is that one of the evaluations of anti-bullying programs was conducted by Baldry and Farrington (2004). Otherwise, none of us has any interest in the conclusions of the review and none of us stands to benefit in any way from any source that has any interest in the conclusions of the review. We are not aware of any personal, political, academic, or financial factors that might bias our judgment. Use one coding sheet for each distinct research project that is included in the systematic review. Throughout, zero = not known, 1 = Yes, 2 = No Details of primary report ________________________________________________________________________ ________________________________________________________________________ Publication type: Details of other reports ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ The research was conducted in one of the following countries since 1983: What measures were used? Where available, effect size measures will be based on self-report questionnaires. If these are not available, code in the order: teacher ratings, peer ratings, systematic observation, school records. Scales: Type and components of intervention: (Yes = 1, No = 2) What measures were used? (Yes = 1, No = 2) Scales: Prevalence of Bullying Short: Odds Ratio ORBS______ Confidence Interval CIBS______ Long: Odds Ratio ORBL_______ Confidence Interval CIBL______ Prevalence of Victimization Short: Odds Ratio ORVS______ Confidence Interval CIVS______ Long: Odds Ratio ORVL_______ Confidence Interval CIVL______ Bullying scores Short: dBS______ SEBS______ Long: dBL_______ SEBL______ Victimization Scores Short: dVS______ SEVS _____ Long: dVL_______ SEVL______

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.