Development and internal validation of a search filter meant to identify Swiss graduate theses focusing on children’s homes written in French

Abstract Background and rationale: Swiss social work students from higher education universities (HETS Lausanne, HETS Fribourg, HETS Valais and HETS Genève) have published over 4000 theses from 2008-2018, many of which report primary qualitative research (mainly interview studies). The current number of social work theses (thousands), variation in indexing, terminology and keywords used by students (foyer, maison, home, lieu de vie, institution, etc.) (or used in different universities) and database limitations (no abstracts available online for almost all theses) has made it difficult for to find Swiss works focusing on this topic. Children’s home research not identified by end-users and stakeholders are unlikely to be read, could be unnecessarily undertaken again and may therefore contribute to research waste. Neglecting children’s home research found in graduate theses also means that children in care and care workers would have given their time to contribute to research that may end up having little to no impact. A well-developed topic search filter meant to help identify children’s home research could prove to be a useful tool to limit research waste in this context. Objective: To develop and validate search filters meant to retrieve Swiss social work graduate theses focusing on children’s homes (topic search filter) written in French and indexed in Swissbib. Study design: Search filter development study and internal validation of the filters on one (1) dataset of relevant records and one (1) dataset of irrelevant records Methods: First, consecutive theses published by HETS Lausanne social work students from 1960-2015 were manually sought and assessed until 10 children’s home theses were found for each decade. Based on a brief assessment of their contents, 73 theses were deemed to focus on children’s homes and 123 to focus on other topics (“irrelevant theses”). The titles of the 73 included theses constituted the development set. The development set was analyzed using the Systematic Review Accelerator word frequency analyzer tool and the most commonly found 1 or 2 words keywords were extracted to an Excel document. Each keyword’s ability to correctly identify relevant theses (sensitivity) and correctly exclude irrelevant theses (specificity) was then calculated by emulating Swissbib’s exact keyword search and wildcard searches functions with imbricated COUNTIF().Two search filters were iteratively developed by subjectively selecting and combining keywords with the highest specificity and/or the highest sensitivity. The best “OR” keyword combinations were then sought through trial and error using the dynamic Excel search emulator, trying as best as possible to select keywords leading to inclusion of relevant records only. For all relevant theses titles which could not be found I tried to identify which missing keyword(s) combination(s) could have been used to include them without also including too many irrelevant titles too. Results: The highly sensitive search filter achieved 79.45% sensitivity and 95.12% specificity. This search filter found 58 out of 73 (79.45%) relevant theses and adequately excluded 117 out of 123 (95.12%) irrelevant records. The most effective highly specific search filter achieved 69.86% sensitivity and 98.37% specificity. This search filter found 51 out of 73 (69.86%) relevant theses and adequately excluded 121 out of 123 (98.37%) irrelevant records. It is meant to adequately exclude as many non-children’s homes theses as possible. Although a few additional relevant theses with uninformative titles could have been found with children’s homes non-specific keywords this was deemed to be inadequate for this would likely negatively affect the filter performances when/if used in a different dataset or database. Limitations: Search filter performances could therefore be slightly different in Swissbib. These search filter performances in other databases including French literature is currently unknown. The performance of a search filter is bounded by the limitations of the data (titles and abstracts) on which it is based; in the current context it is difficult to achieve higher sensitivities when a substantial portion of titles provide little information on the study focus and no abstracts are available. These search filter performances may vary over time if a) keywords in use change or b) graduate theses titles become more or less informative. These search filters are meant to identify children’s homes theses published from 1960 to 2015.A number of French keywords specific to children’s homes such as “foyer de jeunesse” (Canada), “home d’enfants”, “établissement dépositaire”, “Serix” and “maison d’éducation à caractère social (MECS)” (France) were not identified within children’s home theses, which suggests relevant titles can still be missed by these search filters and performances will vary if they are used to search non-Swiss theses. A sample of n=73 children’s homes theses was not enough to identify the full breadth of children’s homes keywords in use in Switzerland from 1960 to 2015. Identifying theses whose focus were children’s homes involved a level of subjectivity, particularly for institutions with mixed populations (children and adults) and other authors may differ with some of the judgements made. Discussion: The performances of the two search filters developed in this study should be tested (external validation) in other Swiss databases such as RERO Explore or Renouvaud prior to a more widespread use. HETS Lausanne’s librarians may encourage social work students to use these search filters to identify Swiss children’s homes theses and the Excel search simulator could also be used (and improved) to teach about developing effective search strategies. HETS students and higher education students could be given more or better guidance on how to make their theses titles more informative to future potential readers if they wish for others to benefits from insights emerging from their work. Swiss social work higher education schools should consider also including theses abstracts in electronic databases to help users identify works they may be interested in. PDF versions of graduate theses would also massively increase their accessibility and both national and international visibility. Funding: No funding was received for this work. Registration: Not done, previous report versions nevertheless available at https://osf.io/dc6xy/files . Data and materials: See https://osf.io/dc6xy/files. All other data is otherwise included within this manuscript. Keywords: Children’s homes, search filter, sensitivity and specificity, indexing, literature searching

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.