Bibliographic record
Abstract
A s charter members of the American Evaluation Association (AEA) and being active in its predecessor organizations, we have an extensive history of discussing, reviewing, debating, and wondering about accreditation, certification, and credentialing (ACC).We are longterm collaborators (over 34 years) who have been involved continuously in presenting and writing about the preparation of evaluators and certification.In the early 1990s as an AEA board member, Engle recruited (cajoled, actually) Altschuld to lead a new directory of evaluation preparation programs.This effort led to a New Directions in Program Evaluation (now New Directions for Evaluation [NDE]) issue (No. 62) published in 1994.We note this earlier result because it relates to the current issue.The first coeditor was communicating with the NDE Editor-in-Chief about an issue on needs assessment (Engle is a contributor to it).In that conversation, it was mentioned that she and I were recommended for an update to the 1994 publication focusing on the interconnected ideas of ACC.Wow! What an opportunity to reconsider a topic that was done two decades ago, especially given the vastly changing context of the practice of evaluation, the theoretical understandings of it, and what has happened and is happening with professional organizations.We were delighted to be thought of for the task and the rest of the story is, as they say, "history."This issue is not an update of the 1994 effort, nor is it intended to be: its focus is on interconnecting and dissecting ideas relative to ACC.We wanted to take a fresh look at them and we wanted our chapter authors to do the same thing.To that end, we assembled a diverse group of individuals with unique viewpoints to be our writers.We debated how to start the current journey and felt that looking into ACC necessitated placing the three components in the historical context of how evaluation has evolved in the United States since the 1960s.We are not advocating for ACC; rather, we illuminate what their nature is and how past and current conditions have influenced thinking about them.We see the three parts of ACC as interrelated, and as critically important to any discussion of our field.We start the discussion of the historical context by creating a timeline from the 1960s to the present (although we are sure that some events are omitted, making this a personal expression of history, not an actual one).This is the core content in Chapter 1 ("The Inexorable Historical Press of the Developing Evaluation Profession") written by us.In what ways do the past and the present inform and guide us relative to ACC?The
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.070 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.005 | 0.004 |
| Science and technology studies | 0.004 | 0.001 |
| Scholarly communication | 0.009 | 0.006 |
| Open science | 0.006 | 0.004 |
| Research integrity | 0.006 | 0.007 |
| Insufficient payload (model declined to judge) | 0.407 | 0.323 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".