An Exploratory Study of Student Use and Perceived Accuracy of Faculty Evaluations on RateMyProfessors.Com
Notice bibliographique
Résumé
EXECUTIVE SUMMARY The website RateMyProfessors.com, established in 1999, has provided students with a means of providing publically available feedback on courses and faculty at institutions of higher education in the United States, Canada, England, Scotland and Wales (www.RateMyProfessors.com, 2008a). Due to the increased popularity of the website, academicians have investigated various issues of the online rating site. However, these studies did not answer several basic questions. In the present study, the authors seek to address the following five areas: student awareness / usage of the site, how they use the site, who do they rate, basis for their ratings, and perceived accuracy of the ratings on the site. The results are based on a sample of 152 undergraduate business students at a small regional university in the Southeast United States. Keywords: Evaluations; Faculty Evaluations; Student Evaluations; RateMyProfessor; Faculty Ratings INTRODUCTION Beginning in the late 1990s, several websites were established to provide individuals with public access to evaluations of faculty members in institutions of higher education. To date, there are five such sites operating with the most popular being RateMyProfessors.Com (hereafter referred to as RMP). According to information contained on the RMP website, over 6.8 million ratings have been submitted by students; over 1 million faculty members have been rated from over 6000 institutions (www.RateMyProfessors.com, 2008a). Anyone can access the site, search for ratings (or postings) of a faculty member and read the ratings. Individuals may search ratings by course title or subject, by institution and/or by faculty. Individuals can post a rating for a faculty member by becoming a registered user or anonymously (as a guest). Individuals rate a faculty member on a 1 - 5 point Likert scale in 4 categories: Easiness (difficulty of professor or course material); Helpfulness (professor's helpfulness and approachability); Clarity (professors' organization, time management skills and how well they teach the course material); and Rater Interest (class recommended to take before graduation) (www.RateMyProfessors.com, 2008b). An overall rating (Good Quality, Average Quality, Poor Quality) is provided based on the average scores on Helpfulness and Clarity. Additionally, individuals may elect to post comments and indicate how hot a faculty member is as indicated by a 'chili pepper' rating (www.RateMyProfessors.com, 2008b). Students have both formal and informal networks whereby they disseminate information about a faculty member to their peers; however, some students (particularly those who are new at the institution) may not have access to these networks. Most faculty evaluations are confidential, are not published by the institution, and are not available to students for review. The popularity of this site appears to be driven by the free access to faculty evaluations. However, several question the reliability and validity of the information contained on the site (cf., Jaschik, 2008; Silva, Silva, Quinn, Draper, Cover & Munoff, 2008; Otto, Sanford & Ross, 2008; Lawson & Stephenson, 2005). For example, anyone can post an evaluation of a faculty member - whether he took the course or not. Also, a person could post multiple evaluations for the same faculty member to bias his overall rating. For that matter, a faculty member with low evaluations could post evaluations to improve his overall rating (Montell, 2006). Finally the criteria used to create die ratings (two items very broadly defined) has been questioned. LITERATURE REVIEW The literature reviewed pertains to those articles which directly relate to the RMP website and/or the other sites offering a similar service (PassCollege.com, ProfessorPerformance.com, RatingsOnline.com and Reviewum.com). The authors did not include other literature as it relates to student evaluations - whether students are able to objectively evaluate faculty members, items used to evaluate faculty teaching effectiveness, etc. …
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,008 | 0,041 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,000 |
| Bibliométrie | 0,001 | 0,001 |
| Études des sciences et des technologies | 0,001 | 0,000 |
| Communication savante | 0,002 | 0,001 |
| Science ouverte | 0,000 | 0,001 |
| Intégrité de la recherche | 0,000 | 0,001 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,003 | 0,001 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».