Book review of "Using quality benchmarks for assessing and developing undergraduate programs"
Bibliographic record
Abstract
Dunn, D. S., McCarthy, M. A., Baker, S.C., & Halonen, J. S. (2011). Using quality benchmarks for assessing and developing undergraduate programs. San Francisco, CA: Jossey-Bass. Pages: 384. Price: $54.00 CDN (hardcover). ISBN 978-0-470-40556-7Although higher education has been grappling with accountability and assessment for decades, few topics produce such strong reactions among faculty and administrators as the need for a formal assessment structure and (p.46).With the recent push on quality assurance, this book explores in a very timely fashion the aims, strengths and barriers of program assessment and extends to the examination of key factors to consider in the context of such an assessment. To make the task of assessing an entire program of study more feasible, the authors divide the seemingly overwhelming task into eight components, making the assessment process more manageable, efficient and relevant. This is important if faculty members are to adopt a continuous and progressive outlook to assessment, rather than simply see the process as a sporadic exercise, undertaken solely out of necessity.The book is structured in two parts: the first part outlines a framework of eight key program domains which characterize the health of academic programs; the second part examines the practice of assessing the key program domains in different contexts and disciplines.Each of the key program domains in the first part of the book is the feature of its own chapter and speaks to the most engaging issues in the field. The first few chapters outline the role and impact of program leadership, overcoming faculty resistance to assessment and developing a sustainable assessment culture, program level learning outcomes as a blueprint for curriculum and useful criteria to evaluate effective curriculum design. The last few chapters describe the role of student development beyond intellectual training, the evaluation of faculty characteristics and contributions, how to use program assessment results to make a case for additional resources and a framework for characterizing administrative support.Each chapter contains an easy to read benchmark table with a list of attributes for which four progressive performance descriptors are outlined (underdeveloped, developing, effective, and distinguished). Descriptors for each attribute are subsequently explained and, often colourfully, illustrated by examples pulled from the considerable collective program assessment and development experience of the authors. This certainly aids in grounding the attributes and making more evident the spectrum of potential program realities. Authors are cautious to mention that benchmarks are not one size fits all. Seemingly designed with a larger research intensive university in mind, the benchmark tables are flexible and adaptable based on context. Each chapter concludes with a series of guiding questions which serve as catalyst for reflection on performance patterns in the context of each key program domain. The chapters in the second part of the book examine the use of the benchmarks in disciplines with largely differing outlooks on program assessment. Chapter 10 looks at benchmarking quality in the arts, humanities and interdisciplinary programs and chapter 11 investigates the same theme in the natural sciences. Chapters 12 and 13 conclude with an in-depth explanation of how to conduct a program self-study and practical tips.A valuable feature that makes this book a practical resource is that most chapters can easily stand alone. This is useful for a potential faculty member or reviewer who might be interested in a specific component of a review, be it program level learning outcomes or the examination of program resources, instead of having to read the book in its entirety.Other notable strengths include a description of assessment as having a formative purpose. The book emphasizes that the program assessment exercise is truly about program enhancement. …
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.005 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.045 | 0.029 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".