Protocol for a Systematic Review: The Tools of the Mind Curriculum for Improving Self‐Regulation in Early Childhood: A Systematic Review

Self-regulation, defined as volitional control of attention, behavior, and executive functions for the purposes of goal-directed action (Blair & Ursache, 2011), is associated with multiple school-related outcomes (Calkins, Howse, & Philippot, 2004; Diamond & Lee, 2011; McClelland & Tominey, 2011). For example, children with robust self-regulation have been shown (Fisher, Hirsh-Pasek, Newcombe, & Golinkoff, 2013; Ramani, 2012) to more cooperatively participate in classroom activities, sustain focus on tasks (Bierman, Nix, & Greenberg, 2008; Drake, Belsky, & Fearon, 2014), and exhibit reduced behavioral issues (Feng et al., 2008; Ponitz, McClelland, Matthews, & Morrison, 2009). Conversely, lower levels of self-regulation skills are associated with externalizing behaviors (Flouri, Midouhas, & Joshi, 2014; Olson & Lunkenheimer, 2009), diminished attention (Raver et al., 2011; Tough, 2012), and lower academic achievement (Kim, Nordling, Yoon, & Kochanska, 2014; Nota, Soresi, & Zimmerman, 2004; Soares, Vannest, & Harrison, 2009). In addition to academic outcomes, children with poor self-regulatory competencies are more likely to have worse health and financial outcomes in adulthood (Moffitt, Arseneault, & Caspi, 2011; Schlam, Wilson, Shoda, & Mischel, 2013). Given the role of self-regulation in promoting both child and adult outcomes, early intervention in preschool contexts holds considerable promise for improving a child's development trajectory. As Heckman noted, early “skill begets skill; learning begets learning” (Heckman & Masterov, 2007, p. 3). Consequently, small self-regulatory differences in early childhood can be magnified to progressively larger differences over time (Alexander, Entwisle, & Kabbani, 2001; O'Shaughnessy, Lane, Gresham, & Beebe-Frankenberger, 2003). Thus, early childhood emerges as an especially critical period in which to intervene. Previous research about the current state of young children's self-regulation development further underscores the need for early intervention. In the United States, a nationally representative survey indicated that 46% of American kindergarten teachers reported at least half of their students to be routinely struggling with self-control (Rimm-Kaufman, Pianta, & Cox, 2000). In fact, American preschool students are three times more likely to be expelled for unmanageable behavior than primary and secondary students (Gilliam, 2005). Certain subpopulations of children exhibit diminished self-regulation from a young age. For example, children growing up in poverty are more likely to experience self-regulatory problems (Raver, Blair, & Willoughby, 2013; Raver, 2012), which make low-income children susceptible to disciplinary action inside and outside of school (Alloway, Lawrence, & Rodger, 2013; Miller, Nevado-Montenegro, & Hinshaw, 2012). For example, a Washington DC report (Office of the State Superintendent of Education, 2013) revealed that students aged three and four received 181 suspensions during the 2012-2013 school year, most of which went to students in low-income schools. Moreover, many children in the United States and worldwide have chronic regulatory deficits such as Attention Deficit Hyperactivity Disorder (ADHD) and conduct disorder (CD). In 2013, 11% of American children between the ages 4 of 17 had been diagnosed with ADHD, which reflects a 41% increase over a single decade (Center for Disease Control, 2013). In the UK, 7% of British boys and 3% of British girls aged 5-10 meet the diagnostic criteria for conduct disorder (NICE, 2013), which presents challenges to the educators responsible for student learning (Webster-Stratton, Reid, & Stoolmiller, 2009). Other research indicates that educators across Europe and Asia (Ben-Ari, 1995; Kwon, 2003) have reported rising levels of chronic self-regulatory deficits among young children. Thus, the current state of students' self-regulation has emerged as a pressing problem confronting education systems in countries around the world. By contrast, the body of evidence on curricula and interventions that significantly improve mainstream students' self-regulation is sparse. For example, the U.S. Department of Education's Institute of Educational Sciences (IES) funded a randomized controlled trial (RCT) that assessed 14 preschool curricula; the results indicated that none of the curricula significantly improved children's self-regulation skills beyond traditional comparator curricula (Preschool Curriculum Evaluation Research Consortium, 2008). Moreover, none of the 14 programs identified self-regulation development as their primary curricular focus, despite abundant research indicating the benefits of self-regulation for young children. To the best of our knowledge, only one early childhood curriculum emphasizes self-regulation cultivation as its paramount aim: Tools of the Mind (Tools). Since its development in 1993, Tools has been adopted in parts of the United States, Canada, and South America. Twenty U.S. states now have at least one Tools school; in certain areas such as Washington DC, Tools has been implemented in the majority of local preschools (Tools of the Mind, 2015). In the face of the program's proliferation, it is important to establish evidence of Tools' effectiveness on hypothesized outcomes. That is, does Tools enhance children's self-regulation and academic outcomes as compared with traditional ‘business-as-usual’ or other program curricula? This review aims to be the first to address this question. Tools derives its inspiration from the work of psychologist Lev Vygotsky. In his book Thought and Language (1962), Vygotsky develops the concept of ‘mental tools,‘ which extend mental faculties in the way that physical tools extend physical faculties. For example, although young children typically struggle with task focus, they can be taught to use private speech (e.g., self-talk meant to guide one's actions as opposed to communicate with others) in order to maintain concentration amid distractions. In this case, private speech serves as a mental tool that enables children to focus beyond their baseline abilities (Vygotsky, 1962). According to the curricular developers, Tools is informed by “neuropsychological research on the development of self-regulation/executive functions in children” (Bodrova & Leong, 2015, Tools' website home page). Unlike several self-regulation interventions, which often involve individualized plans for specific children (Gulchak, 2008; Soares et al., 2009) or a set of exercises to supplement an existing curriculum (Bierman, Domitrovich, Blair, Nelson, & Gill, 2008; Domitrovich, Cortes, & Greenberg, 2007), Tools is intended to be a comprehensive curriculum delivered to all students in a mainstream classroom. Tools operates through integrating self-regulation oriented activities within academic instruction (Bodrova, Leong, & Akhutina, 2011, p. 18). That is, each Tools activity contains both a target academic skill (e.g., reading a book with a classmate) and a self-regulatory skill (e.g., waiting one's turn to read the book). Overall, Tools includes 61 activities that simultaneously target students' self-regulation as well as their foundational academic skills. Two activities, Buddy Reading and Make Believe Play, are emblematic of Tools' approach. Buddy Reading involves two students who cooperatively read a book. One child receives a picture of a mouth, which designates him or her as the reader; the other child receives a picture of an ear, which designates him or her as the listener. The reader then reads the story while the other child actively listens and checks for decoding errors. The children then switch roles after the first reader completes the story (Leong & Bodrova, 2011). Given proper execution, Buddy Reading simultaneously targets literacy and self-regulation. Since self-regulation is defined as the ability to autonomously control attention and behavior, Buddy Reading should theoretically hone children's self-regulation because children must 1) use working memory to remember and act out their roles, 2) demonstrate attentional flexibility by switching across roles, and 3) exhibit inhibitory control to suppress desires to switch roles at inappropriate times (e.g., the listener should not attempt to become the reader before his or her turn). The second activity emblematic of the Tools approach is Make Believe Play, which is meant to occur every day in Tools classrooms (Bodrova & Leong, 2013). Tools' focus on play originated with Vygotsk/s assertion that pretend play scenarios directly cultivate self-regulation skills: “At every step the child is faced with a conflict between the rule of the game and what he would do if he could suddenly act spontaneously. In the game he acts counter to what he wants … [achieving] the maximum display of willpower” (1933, p. 14). Pretend play thus requires children to focus on a role (e.g., a grocer), enact that role (e.g., help a ‘customer’ bag groceries), and inhibit the impulse to switch roles (e.g., become the grocery store manager instead of the grocer) even when the child wishes to act spontaneously. Vygotsky (1933) argued that effective play scenarios require three elements: children must 1) determine an imaginary scenario, 2) negotiate roles for themselves and one another, and 3) act out those roles with fidelity (i.e., not switch or cease a role simply because one has lost interest in it). In order to achieve such structured play scenarios, Tools teachers work with students to create play plans as depicted in figure 1. Sample play plan from a Tools classroom As observed in figure 1, the play plan includes both textual and pictorial elements. According to the Tools manual (Leong & Bodrova, 2011), play planning involves multiple steps. First, the teacher convenes a group of students who collectively determine a play scenario. Second, students negotiate roles for each child to assume throughout the play block. For example, in figure 1, the children have decided to enact a scenario involving a princess and prince. Each child then creates a play plan that includes his or her name, a picture of the child acting out that role, and a textual description of the play plan. The plan from figure 1 indicates that the student will pretend to be Sleeping Beauty and marry a prince. Thus, make-believe play planning simultaneously involves writing practice, drawing practice, and goal-oriented thinking to guide the child's subsequent behavior. If students forget their roles, then the teacher and/or other students should reference the play plan (Leong & Bodrova, 2011); in a sense, the play plan constitutes a contract that commits the child to a specific role. This play-planning process precedes the actual play scenario, which is where Vygotsky (1933) argues children's willpower is directly taxed. In sum, whether children are engaged in literacy, mathematics, or play scenarios, each Tools activity aims to target self-regulation. Tools is designed to be implemented with fidelity by classroom teachers throughout a full academic year (Leong & Bodrova, 2011). This immersive component of Tools differentiates it from other self-regulation programs and emerges as a key mechanism of its purported efficacy. Tools' theory of change contains three stages: 1) students are regulated by their teacher, 2) students regulate one another, and 3) students self-regulate (Bodrova & Leong, 2007). When students first arrive in a classroom, Vygotsky wrote that they are “slaves to their environment,” whereas education's aim must be to transform them into “masters of their own behavior” (L. S. Vygotsky, 1962, p. 147). Bodrova and Leong, the Tools curricular developers, attempted to capture Vygotsky's philosophy through the teacher-regulated, other-regulated, and self-regulated theory of change model depicted in figure 2. Tools of the Mind Theory of Change (Adapted from Bodrova & Leong, 2007) In a quote that encapsulates this theory of change, Vygotsky wrote that “inner regulation of purposeful activity originates in external regulation” (Vygotsky & Luria, 1994, p. 164). That is, before regulating him or herself, a child's thoughts and actions must first be regulated by someone outside of the child (i.e., an adult or more competent peer). In one of his earlier writings, Vygotsky explains that adults must use their superior cognitive control to lead children toward constant improvement of their cognitive control (Vygotsky, 1994, p. 366). Along a similar line of reasoning, Bruner coined the term ‘scaffolding’ (Wood, Bruner, & Ross, 1976), which refers to a teacher's provision of the minimum support necessary to propel children toward their learning goals. As a child become more competent, the teacher gradually removes support until the child can work independently. The Tools developers refer to both Vygotsky's and Bruner's theories when explaining Tools' theory of change (Bodrova et al., 2011). Vygotsky's research also informs the second component of Tools' theory of change: Children's co-regulation of one another. Vygotsky argued that children's higher mental function development originates in social interactions before becoming internalized later on (Vygotsky, 1978). The Tools developers explain that the roots of self-regulation originate in other-regulation, which “implies that children act both as subjects of another person's regulatory behaviors and as actors regulating another person's behaviors” (Leong & Bodrova, 2011, p. 73). In sum, the Tools theory of change specifies children's self-regulation as the ultimate goal and then identifies teacher-regulation and other-regulation as the steps toward that end. Given self-regulation's role in promoting a multitude of desirable life outcomes, it is critical to identify educational practices that improve self-regulation skills. The Tools developers claim that the program effectively promotes children's self-regulation, and Tools has already been implemented in the U.S., Canada, and parts of South America. Although Tools' proliferation has been consistent in recent years, the findings from Tools evaluation studies have been inconsistent. For example, in a sampling of five randomized Tools evaluation studies, three found small to moderately positive effects on students' self-regulation (Barnett et al., 2008; Blair & Raver, 2014; Diamond, Barnett, Thomas, & Munro, 2007), one found no effect (Lonigan & Phillips, 2012), and one found no effect during pre-kindergarten but negative effects at kindergarten and first grade follow-ups (Farran & Wilson, 2014). Thus, despite relatively similar research designs with considerable methodological overlap, these five studies arrived at substantially different conclusions regarding Tools' effectiveness. These mixed findings have thus far precluded any authoritative conclusion regarding the curriculum's effectiveness. The literature lacks a systematic review to assess: 1) Tools' overall effect, 2) the heterogeneity of curricular effectiveness across study-level and child-level characteristics, and 3) the quality of the Tools research base as a whole. By investigating those three issues, the present review aims to provide education policymakers and practitioners with useful information regarding whether to implement Tools. Our central objective is to identify, appraise, and synthesize the available evidence regarding Tools in order to evaluate Tools' effectiveness as compared with other curricula, including business-as-usual and other programs. We will conduct this review in accordance with the Campbell Collaboration's systematic review guidelines, which can be found at www.campbellcollaboration.org. Included studies should have experimental, quasi-experimental, or non-experimental designs that have adequate statistical mechanisms to control for potential confounds. At the least, studies must have pre- and post-tests on the outcome measures of interest. Although we would ideally restrict included studies to randomized trials, randomization is difficult in education research given ethical concerns and school district policies. Thus, this review will accept the quasi-experimental designs described above in order to include as many studies as possible in the review. Nonetheless, we will present results separately for randomized and non-randomized trials. Students of any age, gender, ethnicity, special education status, language learning status, and socio-economic status will be included in this review. We will include any study that analyzes Tools' effect in comparison to one or more “business-as-usual” curricula. Business-as-usual curricula are those that the school had used before the intervention study began. We will also include studies where Tools was implemented alongside another program or intervention that is new for the school. We will include data from any follow-up periods included in the original studies. The follow-up data will be classified into three categories: short-term (i.e., data taken between the end of the Tools intervention year to five months following the intervention), medium-term (i.e., data taken between six and 11 months after the end of the Tools intervention), and long-term (i.e., data taken at 12 months or more after the end of the Tools intervention). We will include studies from any setting where Tools was implemented. Because Tools is a school-based curriculum, we expect that our search will yield only school-based studies. Nonetheless, no a priori setting-based exclusion criteria will be imposed. The included primary studies will likely involve either experimental or quasi-experimental research designs. Some studies will involve random assignment of students to either Tools or business-as-usual classrooms; however, other studies may include samples where random assignment was not possible. In instances where district- or school-level policies precluded random assignment, the primary studies should have adequate statistical mechanisms (e.g., pre- and post-test comparisons) to control for potential confounds. The studies will likely involve academic and self-regulation assessments at the beginning and end of the school year. The academic achievement data may derive from schools' existing tests or from alternative standardized instruments provided by researchers. The self-regulation assessments will likely be provided and administered by researchers, who sample either a group of students in a school/classroom or the entire classroom/school population. Finally, some primary studies may compare Tools with ‘business-as-usual’ classrooms, whereas others may compare Tools with another intervention group. In this review, all ‘business-as-usual’ and other intervention groups will be considered as the comparator condition, while Tools serves as the treatment condition. We will also run sensitivity analyses to determine whether different effects are observed for Tools versus ‘business-as-usual’ comparisons and Tools versus other intervention comparisons. We will presume that any effect sizes from the same study will be statistically dependent regardless of the observed intra-class correlation. Moreover, we will also assume that any instances of multiple reports from the same study will yield data dependency issues. In order to address data dependency, we will use the robust variance estimation SPSS macro described in Tanner-Smith & Tipton (2014). We will also use the metafor package (Viechtbauer, 2010) in R to perform multilevel meta-analysis as a robustness check. Two researchers (Baron and Melendez-Torres) will independently conduct eligibility screening on all retrieved studies. Specifically, both researchers will screen titles, abstracts, and (where appropriate) full texts in order to determine whether studies are suitable for inclusion in the review. All disagreements of inclusion versus exclusion will be resolved through discussion and consensus. As for coding, we have developed a data extraction form for this review (see Appendix A). Baron and Melendez-Torres will independently code the studies selected for In instances of or we will study for the for all studies, we will all disagreements through discussion and consensus. In the each of will be reported a of studies with of across multiple will be in the and sensitivity analyses will be to determine results change with and the studies. We will use the robust variance estimation SPSS macro described in Tanner-Smith & to address data dependency issues. Specifically, some studies will include multiple measures of self-regulation and/or academic In an attempt to use all available we will all effect sizes from each study on both self-regulation and academic achievement while for dependency in effect sizes from the same Moreover, we will use the metafor package (Viechtbauer, 2010) in R to perform meta-analysis with random effects on effect as a robustness on the As for effect we will use the standardized for outcomes and the for outcomes. we have effect sizes across all studies, we will all included effect sizes into the most (e.g., To do we will use available effect effect which can effect data on both and We will effect sizes for each of the comparison (e.g., other no For example, if a study Tools with another intervention as well as a business-as-usual curriculum, then each of the two comparison will have its own effect for this review. We will heterogeneity across studies the and the For the we will use the to determine whether the indicates heterogeneity among the effect For the the the of variance to which results in a from to If we identify a of studies (i.e., or then we will conduct analyses to determine whether the intervention effect significantly across study-level or We will analyses by the in each if not differences across If a of studies than are then will be a We do not plan to include information on outcomes of (e.g., all outcomes will be from the study and will be in an for effect (e.g., where the original outcome data to each effect review The lead is the who develops and the review and roles for of the review with the base and for the of the review. description of and methodological within the review The review includes at least one on the review who has at least one who has methodological and at least one who has statistical is also to have one with information The first has through the United and Research The have no of interest. We aim to a review by Baron will be responsible for the review which will occur every three following the in By this accept for and the review in accordance with Campbell The Campbell will provide as support as possible to with the of the review. review must be to the within two of If are not before the or if we are to for an the has the to the or the to alternative The also has the to or the if it does not meet the of the and/or the Campbell accept for the review in of new and and other and the review at least every five years, if for the review to others as with the The support of the in review is to the review, and subsequent in the Campbell The Campbell no on of the findings of a Campbell systematic review in a more form as a either before or after the of the in Campbell Some however, have that of findings that have or will reported and in such a should be of possible conflict with of the in Campbell in a after or in status in Campbell should the Campbell and include a to that systematic in Campbell and with the may have or for accept for any the to a Campbell review, and to in the Campbell on of the Baron

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.