The Experimental Study of the Effect of Functional-Variational Factors on the Results of Linguistic Testing
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
A feature of the presented study is a comprehensive approach to studying the reliability problem of linguistic testing results due to the several functional and variable factors impact. Contradictions and ambiguous views of scientists on the researched issues determine the relevance of this study. The article highlights the problem of equivalence of diagnostic potential and the complexity of closed and open test tasks. The issues of polymorphism and monomorphism of tests, as well as variability of tests on the sequence of different question forms and their ratio in one test, are revealed. The article authors substantiate the need for in-depth empirical research to further improve the basic quality design principles and improve tests for final testing of high school students. The main purpose of the study was to examine the controlled and uncontrolled factors that affect test results. This article represents a study conducted among the students of the 8th and 9th grades (N=332) and in the subject field of Ukrainian (as a native language). In the first phase of the study the criteria-oriented author's tests and questionnaires were used by the authors as research tools. Data analysis was performed using factor and analysis of variance (and other mathematical and statistical methods), which allowed proving the direct influence of the factor "form of test tasks" on the test results (impact power is up to 35%). The methodology of the study allowed us to determine the combination of forms of tasks that have the lower impact on test results and, accordingly, are characterized by the highest diagnostic reliability. In the second (refining) phase of the study, taking into account the previous results, it was found that the ratio of test questions, as well as their sequence, can also affect the test results and are detected depending on the gender characteristics of the test participants. According to the results of the research, some ideal externally-organized design of Ukrainian language tests for in-school control and classroom-assessment is proposed. It is provided by a combination of the multiple-choice questions and constructed-response questions in equal proportions in descending order of operational complexity of tasks (taking into account gender peculiarities of testing participants). The prospects for continuation and deepening of the research, connected with further study of the optimal empirically valid test structure, are substantiated. In the second (refining) phase of the study, taking into account the previous results, it was found that the ratio of test questions, as well as their sequence, can also affect the test results and are detected depending on the gender characteristics of the test participants. According to the results of the research, some ideal externally-organized design of Ukrainian language tests for in-school control and classroom-assessment is proposed. It is provided by a combination of the multiple-choice questions and constructed-response questions in equal proportions in descending order of operational complexity of tasks.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.014 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it