ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research - Prandner, Dimitri; Wetzelhütter, Daniela; Hese, Sönke (Frontiers in Education, 2025)
Social scientists are faced with the challenge of designing complex studies and analyzing collected data via various programs such as R, Stata, SPSS, or Python. This often requires the use of analytical procedures and specific software packages that are beyond an individual’s established skillsets and technical knowledge. To address these challenges, generative artificial intelligence, such as ChatGPT, can now be employed as ‘assistants’—with both associated risks and benefits. Accordingly, this paper explores the potential and pitfalls of using a tool like ChatGPT as an assistant in quantitative data analysis. We investigate the practical use of ChatGPT-3.5 by replicating analyses and findings in everyday scientific research. Unlike previous studies, which have primarily focused optimizing the use of chatbots for code generation, our approach examines an amateur level use of AI tools to support and reference regular research activities, with an emphasis on minimal technical expertise. While we overall conducted three experiments, with the goal to replicate academic papers, the article’s focus is on the methodologically most complex one, by De Wet et al. from 2020. In this case AI is used for the step-by-step replication of the two-dimensional model of value types proposed by Schwartz (2012). The results of this experiment highlight the challenges of using ChatGPT 3.5 for specific, detailed tasks in academic research, as a tendency for responses to repeat in loops when solutions were not readily available emerged at several stages. Thus, we concluded that there are severe limitations in the AI’s ability to provide accurate and comprehensive solutions for complex tasks and emphasize the need for caution and verification when using AI powered tools for complex research procedures.