Comparative Analysis of TF-IDF and LLM-Based Text Corpus Generation for Course Recommendation Systems

Abstract

Course recommendation systems have become increasingly critical in addressing the complex educational challenges of personalized learning pathways. This research intro- duces a novel approach to text corpus generation that leverages advanced natural language processing techniques to enhance course recommendation accuracy. Traditional keyword extraction methods, particularly TF-IDF, often struggle to capture nuanced semantic relationships within academic content. Our study proposes an innovative methodology that combines traditional statistical methods for large language model (LLM) keyword extraction to generate a comprehensive, multidimensional course corpus. The proposed framework segments college databases across semesters and subjects, generating a three-dimensional keyword representation that captures the intricate relationships between academic content. By comparing traditional TF-IDF keyword extraction with LLM-based semantic keyword generation, we demonstrate significant improvements in recommendation relevance and precision. Experimental results reveal that LLM-based approaches substantially outperform traditional statistical methods in capturing contextual and semantic nuances of academic content.