The use of indicators in French Universities
Notice bibliographique
Résumé
The use of indicators in the management of French universities is becoming more and more prevalent and advanced, at least as far as HSS (humanities and social sciences) are concerned. In the present chapter, we will provide evidence of the general use of indicators and of differences between disciplinary fields. In order to put these results in context, we will first provide some information on the French system and how the recent reforms favored the development of indicators. We will then describe what we have learnt from the qualitative study on the attitudes of the humanities and the sciences to indicators. We will then present some lessons drawn from a quantitative study in which we were able to compare universities mainly specialized in humanities with universities mainly specialized in the sciences. In doing so, we will start out by looking at the use of indicators. This issue has been largely studied in the management sciences, and different authors have suggested different uses. Simons for instance distinguished between diagnostic use of indicators (indicators are used to produce an evaluation of performance) and interactive use of indicators (indicators are used to reveal strengths and weaknesses and to learn about them). Cavalluzzo and Ittner also distinguish between reporting (i.e. providing information about activities), and steering or making decisions (using indicators in order to introduce change). Drawing on these two typologies, we first look at cases where indicators are used to legitimize what has been done and to account for it. Indicators are produced in order to show that a level of performance is achieved, to provide data required by external actors, describing current achievements. We will also consider cases where data are produced in order to compare units or teams and thus to evaluate their activity. Finally, we look at cases where data and indicators are used in order to make decisions or choices and to take action. The legitimation, evaluation, discussion and decision uses of indicators will be studied for data on teaching, on research and on budgets in order to see whether different issues lead to different uses. A second issue addressed by the present chapter deals with disciplinary differences. In France, there are some 'complete universities' (with or without medicine), but also many universities specialized in law and economic sciences, universities with a strong orientation in the NS (natural sciences), and universities that are specialized in the HSS. This allows us to compare the uses of indicators in the humanities and the science‑dominated institutions (HSS institutions and NS institutions) in the following: the former represent approximately 15% of the French universities and the latter 14%.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,054 | 0,079 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,000 |
| Méta-épidémiologie (sens large) | 0,001 | 0,001 |
| Bibliométrie | 0,014 | 0,022 |
| Études des sciences et des technologies | 0,004 | 0,007 |
| Communication savante | 0,012 | 0,004 |
| Science ouverte | 0,001 | 0,005 |
| Intégrité de la recherche | 0,002 | 0,002 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,001 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».