Assessment of Training Effectiveness Adjusted for Learning (ATEAL) Part II: Practical Application
Bibliographic record
Abstract
Safety training programs are a popular method, in industry globally, to increase awareness of risks to employees and employers and plays a critical part in reducing safety incidents. The most frequently used method to assess the effectiveness of the training is to have the participants answer Multiple Choice Question (MCQ) and True/False (T/F) questions after the training. The metrics used to report the outcome of the assessments have drawbacks that make it difficult for the trainer and organization to easily identify the concepts that need more focus and those that do not. The goal of this research study is to compare how the methods used to measure training effectiveness of concepts in Level 2 post training assessment differ in how they assess training effectiveness using actual training results. Pre- and Post-training assessments were administered to the participants in 3 different utility industries and were analyzed for training effectiveness using the traditional metrics as well as using ATEAL method. The results were then compared and detailed recommendations of the best and least learned concepts by industry are presented based on these comparative analyses. The ATEAL method is further used to quantify the opportunities for improvement in the training programs based on the participant prior knowledge and any negative training impact observed. Results of the comparison of the various methods show that the proposed ATEAL method provides a quick, accurate and easy way to assesses the effectiveness of the training of concepts and the method identified that for 40% of the concepts trained a higher percentage of participants exhibited more prior knowledge than positive learning and for 6% of the concepts a higher percentage exhibited negative training. These results also provide a directional guide on the improvements that can be made to improve the training effectiveness of the programs. Additionally, it also shows that the ATEAL method can be used in any learning environment where there is a pre-/post-test evaluation of the change and is not limited in application to MCQ and T/F questions.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.018 | 0.075 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.009 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".