Automated Question Generation for Electronics Engineering Exams Using Retrieval-Augmented Generation
Bibliographic record
Abstract
This paper presents an Automated Question Generation System designed to help lecturers in making high quality exams for their reviewees in preparation for the Electronics Engineering Licensure Exam in the Philippines. The system integrates OpenAI’s GPT-4 for natural language processing, LangChain for document management, and Streamlit for an interactive web interface. Educational materials such as textbooks and lecture notes are processed by splitting them into chunks and storing them in a Chroma vector database. Users can specify exam topics and the desired number of questions through the web interface, prompting the system to retrieve relevant content and generate multiple-choice questions using a fine-tuned language model with Retrieval-Augmented Generation (RAG). The system provides detailed outputs, including questions, answer keys, token usage, and source references, ensuring transparency and reliability.The system’s performance was evaluated using the BLEU and ROUGE metrics, which originate from machine translation and summarization tasks, respectively, to measure the quality and relevance of generated questions. BLEU assesses n-gram precision by comparing generated text with reference text, while ROUGE evaluates recall by measuring the overlap of n-grams, such as unigrams (ROUGE-1), bigrams (ROUGE-2), and the longest common subsequence (ROUGE-L), between the generated and reference texts. In addition to these automated metrics, human experts reviewed the generated questions for accuracy, relevance, and clarity, while large language models such as ChatGPT and Gemini analyzed grammatical correctness and logical consistency. These combined evaluations demonstrated strong alignment with expert standards, confirming the system’s capability to generate accurate, relevant, and grammatically correct exam questions. These results highlight the system’s potential as a comprehensive tool for exam preparation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".