Application of Large Language Models for the Analysis of Value-Patriotic Discourse of Russian-Speaking Users

The article explores the potential of using large language models (LLMs) for the automated analysis of value-laden patriotic discourse among Russian-speaking social media users. Drawing on a corpus of messages from VK, Odnoklassniki, and Telegram (2023–2025), it investigates the degree of alignment between automated coding results and expert annotations based on a specially developed categorical scheme. The codebook includes eight dimensions: Sh. Schwartz's basic values; R. Inglehart’s two axes (traditionalism/secularism and survival/self-expression); A. Maslow’s hierarchy of needs; types of patriotism (constructive/aggressive), drawing on the concepts of K.D. Ushinsky and V.S. Solovyov; dominant speech act types per J. Austin; and binary indicators for explicit patriotism and civic identity. The experiment was conducted on the Pride and Patriotism message cluster (N = 456), where the density of value markers is highest; the comparison was implemented through error matrices, accuracy, macro/weighted F1, and Cohen's κ coefficient. It was shown that while the LLM reliably identifies explicit patriotic themes, its agreement with experts is significantly lower in multi-class and fine-grained value classification (Schwartz, Maslow, Inglehart scales, types of patriotism, Austin's speech acts). The model demonstrated systematic biases and a tendency to over-diagnose certain categories. It is concluded that LLMs in their current configuration can serve as auxiliary tools for preliminary markup and hypothesis generation but cannot function as an autonomous substitute for expert-led content analysis of value discourse.