AN EXPERIMENTAL APPROACH TO THE CONSTRUCTION OF BINARY DECISION CLASSES FROM CARD SORT DATA
Notice bibliographique
Résumé
This thesis presents work done towards understanding the data collected from a card sorting study of facial photographs. In that study, 25 participants sorted 356 photos (178 Caucasian and 178 First Nations) into piles based on similarity. Photos placed in the same pile are deemed to be similar, photos in different piles are deemed to be dissimilar. Looking to establish binary decision classes is reasonable because the underlying question that participants answered was “Are these photos similar or not?”. There may also be more than two decision classes to describe all the behaviours. For example, an initial split into decision classes may be thought of as “doing something” and “not doing something”. The latter could be split into two, and the whole process repeated. Differences amongst the sorting behaviours of participants are evident, but the reason for these differences is difficult to determine. An early hypothesis was that perceived race was being used a criterion for some participants but not for others. An analysis that looked at the ratio of Caucasian and First Nations photos in each pile was used determine a pair of decision classes from which accurate classifiers could be built. Open questions from that earlier work include the basis for participants making those decisions and whether the behaviour supported by a small amount of carefully chosen data would be supported by all the data. There are several million possible decision class pairs that could be used to split those 25 participants into 2 groups. This work applies a knowledge discovery approach to find other candidate decision classes for this data, for which accurate classifiers can also be built. Each participant made a relatively small number of direct comparisons and a large number of indirect comparisons to determine whether each pair of photos (63,190 in all) was similar or dissimilar. For each pair, a binary vector was used to record the judgement of each participant (0 if the participant thought the pair was similar, 1 if dissimilar). These vectors were used as the basis for the present study. Each pair of photos can be said to have a certain power to discern between participants. If all participants gave the same judgement for a pair, the pair has no power to discern between participants. Conversely, a pair which 12 or 13 participants had rated similar (or dissimilar) has the highest power to discern between participants, because for this pair there will be the most disagreement when considering pairs of participants. This work focuses on those pairs with maximum discernibility. To generalize earlier work on a heuristic for evaluating candidate decision classes, it is hypothesized that a t-test could be used to give a better indication about the suitability of a decision class pair. To this end, some experimental analysis of the card sorting study data was undertaken and the rough set attribute reduction methodology was used to evaluate the findings from the computational experiment. ii
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,020 | 0,112 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,001 | 0,001 |
| Bibliométrie | 0,001 | 0,002 |
| Études des sciences et des technologies | 0,002 | 0,003 |
| Communication savante | 0,004 | 0,003 |
| Science ouverte | 0,002 | 0,002 |
| Intégrité de la recherche | 0,001 | 0,002 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,012 | 0,002 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».