An Exploratory Study of Student Use and Perceived Accuracy of Faculty Evaluations on RateMyProfessors.Com
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
EXECUTIVE SUMMARY The website RateMyProfessors.com, established in 1999, has provided students with a means of providing publically available feedback on courses and faculty at institutions of higher education in the United States, Canada, England, Scotland and Wales (www.RateMyProfessors.com, 2008a). Due to the increased popularity of the website, academicians have investigated various issues of the online rating site. However, these studies did not answer several basic questions. In the present study, the authors seek to address the following five areas: student awareness / usage of the site, how they use the site, who do they rate, basis for their ratings, and perceived accuracy of the ratings on the site. The results are based on a sample of 152 undergraduate business students at a small regional university in the Southeast United States. Keywords: Evaluations; Faculty Evaluations; Student Evaluations; RateMyProfessor; Faculty Ratings INTRODUCTION Beginning in the late 1990s, several websites were established to provide individuals with public access to evaluations of faculty members in institutions of higher education. To date, there are five such sites operating with the most popular being RateMyProfessors.Com (hereafter referred to as RMP). According to information contained on the RMP website, over 6.8 million ratings have been submitted by students; over 1 million faculty members have been rated from over 6000 institutions (www.RateMyProfessors.com, 2008a). Anyone can access the site, search for ratings (or postings) of a faculty member and read the ratings. Individuals may search ratings by course title or subject, by institution and/or by faculty. Individuals can post a rating for a faculty member by becoming a registered user or anonymously (as a guest). Individuals rate a faculty member on a 1 - 5 point Likert scale in 4 categories: Easiness (difficulty of professor or course material); Helpfulness (professor's helpfulness and approachability); Clarity (professors' organization, time management skills and how well they teach the course material); and Rater Interest (class recommended to take before graduation) (www.RateMyProfessors.com, 2008b). An overall rating (Good Quality, Average Quality, Poor Quality) is provided based on the average scores on Helpfulness and Clarity. Additionally, individuals may elect to post comments and indicate how hot a faculty member is as indicated by a 'chili pepper' rating (www.RateMyProfessors.com, 2008b). Students have both formal and informal networks whereby they disseminate information about a faculty member to their peers; however, some students (particularly those who are new at the institution) may not have access to these networks. Most faculty evaluations are confidential, are not published by the institution, and are not available to students for review. The popularity of this site appears to be driven by the free access to faculty evaluations. However, several question the reliability and validity of the information contained on the site (cf., Jaschik, 2008; Silva, Silva, Quinn, Draper, Cover & Munoff, 2008; Otto, Sanford & Ross, 2008; Lawson & Stephenson, 2005). For example, anyone can post an evaluation of a faculty member - whether he took the course or not. Also, a person could post multiple evaluations for the same faculty member to bias his overall rating. For that matter, a faculty member with low evaluations could post evaluations to improve his overall rating (Montell, 2006). Finally the criteria used to create die ratings (two items very broadly defined) has been questioned. LITERATURE REVIEW The literature reviewed pertains to those articles which directly relate to the RMP website and/or the other sites offering a similar service (PassCollege.com, ProfessorPerformance.com, RatingsOnline.com and Reviewum.com). The authors did not include other literature as it relates to student evaluations - whether students are able to objectively evaluate faculty members, items used to evaluate faculty teaching effectiveness, etc. …
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it