Framework multicritério para seleção de modelos de linguagem em contextos educacionais: uma análise de desempenho, custo e eficiência energética

Abstract

The adoption of artificial intelligence models in Virtual Learning Environments (VLEs) requires a balance among response quality, latency, computational cost, and energy consumption, especially in continuous and large-scale usage scenarios. This study proposes a multicriteria decision framework for selecting between Small Language Models (SLMs) and Large Language Models (LLMs) in educational contexts, modeling the choice as a Profile Matching problem based on the Euclidean distance between the scenario’s target profile and the normalized empirical performance of each model. Following the logic of Design Science Research (DSR), 21 models (13 base models and 8 fine-tuned models) were evaluated, ranging from 1 to 70 billion parameters, across 240 multiple-choice questions (120 original and 120 from ENEM), distributed among eight subjects and three difficulty levels. The results indicate that LLMs achieve higher accuracy, with a clear advantage on difficult questions, whereas untuned SLMs are substantially faster (approximately 8.56 times lower latency) and more energy-efficient (approximately 4.83 times less energy). Fine-tuning, under the conditions evaluated, increased operational costs without a consistent gain in accuracy. Across the ten simulated scenarios, the framework selected eight distinct models, demonstrating that the best choice depends on the context and that profile-based selection outperforms the maximization of isolated metrics.