The method of synthetic focus groups in the context of digital transformation of sociological research

Cover Page

Cite item

Full Text

Abstract

The subject of the research is the use of synthetic focus groups (SFG), created using large language models (LLM), within the framework of the digital transformation of sociological analysis. The object of the research is innovative methods for collecting and interpreting qualitative data, aimed at simulating group discussions without the participation of real respondents. The article discusses the heuristic potential of SFG, the possibilities of applying the method in conditions of limited access to respondents, and its relevance for studying attitudes toward artificial intelligence in the field of higher education. Special attention is paid to the ways of organizing interaction with language models through a system of prompts, the formation of participant roles, and the analysis of the obtained discursive positions. The article also addresses the ethical and methodological challenges arising from the use of synthetic participants for research purposes. The methodology combines theoretical analysis of scientific publications with practical modeling of SFG, implemented through sequential prompts to language models and subsequent interpretation of the generated data. The scientific novelty of the work lies in the testing of the synthetic focus group (SFG) method as an innovative tool for collecting qualitative information using generative language models (LLM) such as Gemini, Qwen, Llama, Deepseek, and Mistral. The study pays particular attention to modeling SFG based on user prompts aimed at exploring the attitudes of students and faculty toward artificial intelligence in the higher education system. All models used emphasized the methodological limitations of SFG: the possibility of data distortion, the necessity of clearly indicating the synthetic nature of the participants, and the need to complement such studies with traditional methods. Such methods can be useful in situations requiring rapid hypothesis generation, preliminary testing of research scenarios, as well as in educational and expert practice. At the same time, the necessity of a critical approach to the use of SFG is emphasized, especially in the context of the validity and representativeness of the obtained data.

References

  1. Zhang T., Zhang X., Cools R., Simeone A. L. Focus Agent: LLM-Powered Virtual Focus Group // ACM International Conference on Intelligent Virtual Agents (IVA '24). Glasgow, United Kingdom, September 16-19, 2024. New York: ACM, 2024. 10 p. doi: 10.1145/3652988.3673918.
  2. Salminen J., Liu C., Pian W., et al. Deus Ex Machina and Personas from Large Language Models: Investigating the Composition of AI-Generated Persona Descriptions // Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24). Honolulu, HI, USA, May 11-16, 2024. New York: ACM, 2024. 20 p. URL: http://dx.doi.org/10.1016/j.cpr.2009.10.006.
  3. Shin J., Hedderich M. A., Rey B., et al. Understanding Human-AI Workflows for Generating Personas // Proceedings of the 2024 ACM Designing Interactive Systems Conference (DIS 2024) / ed. by A. Vallgarda et al. 2024. P. 757-781. URL: https://doi.org/10.1145/3643834.3660729.
  4. Chuang Y.-S., Goyal A., Harlalka N., Suresh S. Simulating Opinion Dynamics with Networks of LLM-based Agents // Findings of the Association for Computational Linguistics: NAACL 2024. doi: 10.18653/v1/2024.findings-naacl.211.
  5. Gu H., Chandrasegaran S., Lloyd P. Synthetic Users: Insights from Designers' Interactions with Persona-Based Chatbots // Artificial Intelligence for Engineering Design, Analysis and Manufacturing. 2025. Vol. 39. Art. e2. P. 1-17. doi: 10.1017/S089006042400002X.
  6. Holzinger A., Kargl M., Kipperer B., et al. Personas for Artificial Intelligence (AI) an Open Source Toolbox // IEEE Access. 2022. Vol. 10. P. 23732-23747. doi: 10.1109/ACCESS.2022.3154776. EDN: HBQBBR.
  7. Argyle L. P., Busby E., Wingate D., [и др.] Out of One, Many: Using Language Models to Simulate Human Samples [Электронный ресурс] // Political Analysis. 2022. URL: https://www.researchgate.net/publication/363584667_ Out_of_One_Many_Using_Language_Models_to_Simulate_Human_Samples (дата обращения 12.05.2025).
  8. Epstein J. M. Generative Social Science: Studies in Agent-Based Computational Models [Электронный ресурс]. 2006. Режим доступа: https://www.researchgate.net/publication/267118857_Generative_Social_Science_Studies_in_Agent-Based_Computational_Models (дата обращения: 10.05.2025).
  9. Lorenz J., Neumann M. Individual attitude change and societal dynamics: Computational experiments with psychological theories // OSF Preprints. 2020. URL: https://doi.org/10.31234/osf.io/ebfvr (дата обращения 12.05.2025).
  10. Sattele V., Ortiz J. C. Generating User Personas with AI: Reflecting on its Implications for Design // DRS2024: Boston / ed. by C. Gray et al. Boston, USA, June 23-28, 2024. URL: http://dx.doi.org/10.21606/drs.2024.1024.
  11. Татарова Г. Г., Чеховский И. В. Метод фокус-групп: экспертные оценки "факторов неуспешности" // Социология: методология, методы, математическое моделирование. 2012. № 34. С. 5-31. EDN: PEOKEH.
  12. Goodman-Deane J., Waller S., Demin D., González-de-Heredia A., Bradley M., Clarkson J. P. Evaluating Inclusivity using Quantitative Personas // Proceedings of the Design Research Society Conference 2018. Limerick, Ireland, June 25-28, 2018. Limerick: DRS, 2018. URL: https://doi.org/10.21606/drs.2018.400.
  13. Chapman C., Love E., Milham R. P., ElRif P., Alford J. L. Quantitative Evaluation of Personas as Information // Proceedings of the Human Factors and Ergonomics Society Annual Meeting. September 1, 2008. Vol. 52, No. 16. P. 1107-1111. URL: https://doi.org/10.1177/154193120805201602.
  14. Paoli S. Writing User Personas with Large Language Models: Testing Phase 6 of a Thematic Analysis of Semi-Structured Interviews. arXiv preprint arXiv:2305.18099. 2023. 30 p. URL: https://arxiv.org/abs/2305.18099.
  15. Феникс Дж., Тейлор М. Промт-инжиниринг для GenAI: паттерны надёжных запросов для качественных результатов. Астана: Спринт Бук, 2025. 432 с.
  16. Brown T., Mann B., Ryder N., et al. Language Models are Few-Shot Learners // Advances in Neural Information Processing Systems (NeurIPS 2020). Vancouver, Canada, December 6-12, 2020. Red Hook, NY: Curran Associates, Inc., 2020. 23 p. URL: https://arxiv.org/abs/2005.14165.
  17. Wei J., Wang X., Schuurmans D., et al. Chain of Thought Prompting Elicits Reasoning in Large Language Models // Advances in Neural Information Processing Systems (NeurIPS 2022). New Orleans, USA, November 28 - December 9, 2022. Red Hook, NY: Curran Associates, Inc., 2022. 21 p. URL: https://arxiv.org/abs/2201.11903.
  18. Белановский С. А. Глубокое интервью и фокус-группы. Москва: ИНФРА-М, 2019. 352 с. EDN: WONHVD.
  19. Мельникова О. Т., Нестерова Е. М. Особенности групповой дискуссии в очных и онлайн фокус-группах // Вестник Московского университета. Сер. 14. Психология. 2024. Т. 47. № 1. С. 184-206. doi: 10.11621/LPJ-24-08. EDN: RMUJCC.

Supplementary files

Supplementary Files
Action
1. JATS XML

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).