Authors

A model for automatic generation of educational tests from unstructured text based on domain-adaptive retraining of large language models

This article proposes a model for automatically generating educational tests based on domain-adaptive retraining of large language models (LLM). Traditional methods for developing test items are time-consuming and limited to narrow subject specialization. An analysis of existing approaches to generating test items using solely prompting of pre-trained LLMs revealed key limitations: unstable results, insufficient control over question structure, and the need for post-editing, which justifies the need to develop specialized solutions. A pipeline for retraining the T-Lite 1.0 language model (7 billion parameters) using the LoRA technique on a dataset of 4000 validated test tasks in higher education disciplines is proposed. A distinctive feature of the proposed method is the use of chain-of-thought to structure the task generation process by decomposing it into components: topic, goal, format, prerequisite knowledge, wording and expected response. An inference pipeline for the task generation system was developed, integrating multimodal processing of input data (text/image), automatic content segmentation, and multi-stage task generation using specialized Qwen2-VL and Gemma-2-27b-it models. The model was tested and implemented into the independent assessment ecosystem of the i-exam.ru portal. This expanded the functionality of existing online testing services by providing additional capacity to expand the uncompromised database of test-based assignments, which is particularly important for ensuring the reliability of online testing procedures, such as those of the FEPO and FIEB. This implementation confirms the model’s ability to generate high-quality educational tests that meet psychometric requirements and are suitable for use in the educational process. Read more...