The rising interest in ChatGPT from both the public and the media has been noteworthy, signaling a burgeoning field of research that has seen a near-exponential surge (Farhat et al., 2024), particularly within the realms of business and management (Rane, 2023; Elbanna and Armstrong, 2024). Numerous studies have explored the potential applications of ChatGPT-4, aiming to utilize its capabilities to enhance the efficiency of diverse business processes. Examples include explorations into ChatGPT-4’s utility in streamlining customer service, decision-making, higher education, marketing and operations (Dwivedi et al., 2023; Loos et al., 2023), as well as its competency in tackling examination questions across various business disciplines (Terwiesch, 2023; Wood et al., 2023).

However, there exists a notable gap in the literature: the scarcity of critical and reflective examinations of ChatGPT-4’s outputs from a management history perspective. Given the significance of understanding the origins and evolution of management thought (Wren and Bedeian, 2023), an analysis of how Artificial Intelligence (AI) chatbots like ChatGPT-4 interpret and respond to inquiries about fundamental management concepts and ideas could provide valuable insights. At the outset, we had a hunch that ChatGPT-4 could potentially struggle with recounting the origins and history of SWOT analysis since its history is murky, even in the peer-reviewed academic literature (Helms and Nixon, 2010; Benzaghta et al., 2021). Research has found that there is considerable confusion and misunderstanding of SWOT’s origins even among scholars publishing on the topic (Madsen, 2016; Puyt et al., 2023).

It is important to understand the ability of AI models like ChatGPT to retrieve valid knowledge, for example about management history. This is especially the case in the age of ChatGPT and similar chatbots since many students and practitioners turn to these technological tools and apps for answers and guidance about tools like SWOT analysis. An exploration of ChatGPT-4’s ability to provide valid answers on advanced and less clear-cut topics could provide useful insight that could be of use to those developing Large Language Models (LLM). Studies like ours could play an important role in informing the ongoing development and refinement of AI chatbots, ensuring they align more closely with the nuanced trajectories of management theories and practices. Our study also has implications for management history literature and management research more broadly, as it enriches our comprehension of AI’s potential impact on the field and the benefits and limitations of using these technologies in management education and scholarship.

In our study, we assessed ChatGPT-4’s ability to provide accurate information about the origins and evolution of SWOT analysis. SWOT analysis is arguably a very good case since studies have shown that it is a very popular tool in business practice and research (Qehaja and Kutllovci, 2020; King et al., 2023). Therefore, it is of great interest to find out the extent to which ChatGPT-4 is providing accurate information about the history and origins of SWOT analysis. Specifically, we compared the responses of ChatGPT-4 to a series of assertions in the literature that we know are inaccurate (but pervasive). For example: 1. The origins of SWOT analysis can be traced back to the Harvard Business School in the 1960s, where it emerged from discussions in the classroom (Chermack and Kasshanna, 2007; Leih and de los Reyes, 2023), 2. SWOT analysis was initially introduced in 1965 in the book Business Policy: Text and Cases (Learned et al., 1965), a case book written by Harvard Business School Professors in the 1960s (Barney, 1995; Jarzabkowski and Kaplan, 2015) and 3. SWOT analysis is developed in the 1960s by Albert S. Humphrey (Jain, 2015; Ojala, 2017; Teoli and An, 2019; Myllylä and Kaivo-oja, 2024).

To conduct an evaluation, we developed a Python script to systematically prompt ChatGPT-4 API, using a series of questions, designed to probe its knowledge base and assess its proficiency in recounting the historical background and conceptual contributions to SWOT analysis. We manipulated the settings of the two main parameters that control the creativity of the responses in ChatGPT-4 (Temperate and Top_p) to find out to what extent they affect the differences in the quality of the responses to our prompts. The responses were recorded in an Excel spreadsheet and the accuracy of the responses was rated.

The rest of the paper is structured in the following way. In Section 2 we provide a brief overview of SWOT Analysis’ origins and significance. In Section 3 we describe the methodology. Section 4 presents the results. In Section 5 we discuss the results considering existing literature. Finally, in Section 6 we conclude the paper and suggest propositions for follow-up studies.

In this section, we provide an overview of the origins and significance of SWOT analysis. We start by briefly explaining the basics. SWOT is an acronym for Strengths, Weaknesses, Opportunities and Threats (Hansler, 1988). The present-day SWOT analysis is depicted as a 2 × 2 matrix used for brainstorming (Piercy and Giles, 1989; Valentin, 2001). It is widely used in businesses and organizations to map out their strengths (what they excel at), weaknesses (areas needing improvement), opportunities (chances in the environment to grow or excel), and threats (external challenges to overcome). Its application can be found in business strategy (Pickton and Wright, 1998), marketing planning (Fahy and Smithee, 1999), change management (Heusinkveld and Benders, 2012) as well as personal development (Bailey, 1999). SWOT analysis can be called the evergreen strategy tool for environmental scanning. Its popularity is evident through mainstream appeal, exemplified by the tool’s feature on the TV show Silicon Valley. Moreover, another characteristic of SWOT is its simplicity and the fact that it has been transformed into a verb (‘Let us SWOT it’).

The original SWOT analysis, denoted as the SOFT approach (Stewart et al., 1965), along with its modern rendition featuring a 2 × 2 matrix in a cruciform shape (Argenti, 1974) are both notably underrepresented in the literature. Variations like Threat/Opportunity analysis (Ansoff, 1975), WOTS-UP analysis (Steiner and Miner, 1977) and the TOWS matrix (Weihrich, 1982) are widely recognized in the literature; however, they lack theoretical linkage to the original SOFT approach.

In the 1980s, a paradigm shift from business policy to strategic management led to the adaptation of economists’ language and models (Bower, 1982). SWOT analysis went out of fashion in the scholarly literature and has undergone several significant changes since its introduction. As a consequence, a disconnect in theory development within the strategic management literature has emerged (Hofer and Schendel, 1978; Hambrick and Chen, 2008). Since the 1990’s, the dominant logic in the field of strategic management has been the resource-based view of the firm, a perspective inspired by economic theory (Wernerfelt, 1984, 1995; Barney, 1991). This perspective is rooted in the LCAG framework (Christensen et al., 1978; Andrews, 1980). Through revisionism and reification, the LCAG framework or Andrew’s strategy framework (De Los Reyes, 2011; Bell and Rochford, 2016) is often misattributed as the origins of SWOT analysis (Mintzberg, 1990; Barney, 1995; Argyres and McGahan, 2002; Bower, 2008). Due to this, there exists scant consensus and documentation regarding the tool’s history and development (Bell and Rochford, 2016).

During the turn of the millennium, notable innovations in SWOT analysis emerged from other models and techniques. This period marked the rise of hybrid SWOT models such as A’WOT (Kurttila et al., 2000), Dual-perspective SWOT (Novicevic et al., 2004) and Meta-SWOT (Agarwal et al., 2012).

Over the past 30 years, but probably longer, SWOT analysis has been reported as the number one strategy tool in practice (Webster et al., 1989; Rigby, 2001; Stenfors et al., 2007; Bellamy et al., 2019). Despite its popularity and widespread use among practitioners, strategy scholars criticize SWOT analysis for being a traditional or even a simplistic classificatory device (Haberberg, 2000; Arnaud et al., 2016; David et al., 2021) and even repeat the assertion that it has ‘little intellectual content’ (Jarzabkowski and Kaplan, 2015, p. 542).

In our view, the enormous popularity and widespread diffusion of SWOT in business practice and business school education (Farrokhnia et al., 2023), as well as the relatively broad and mainstream appeal of the tool, makes it all the much more important to ensure that the history and origins are correctly understood and recounted. This is especially the case in the age of ChatGPT and cognoscenti (Borges, 2023), since many students and practitioners turn to this technology for answers and guidance about tools like SWOT.

We prepared for the evaluation by manipulating two key parameters, Temperature and Top_p, which control the level of creativity or randomness in the responses generated by the ChatGPT-4 API from OpenAI.com. The Temperature setting ranges from 0.0 to 2.0, while the Top_p setting ranges from 0.0 to 1.0. Notably, only one variable can be adjusted at a time when prompting the API with a Python script, necessitating a comparison between default settings (Temperature 1.0; Top_p 0.5) and stricter settings (Temperature 0.2; Top_p 0.1). Our aim was to assess the response quality across different settings, with an initial hypothesis that stricter settings would produce more reliable outcomes. However, at the outset, there was uncertainty related to determining which variable setting yields the most trustworthy results.

To carry out the evaluation, we followed a systematic approach to data gathering and analysis. Initially, we selected 50 relevant publications on SWOT analysis spanning from 1965 to 2019. Subsequently, we extracted all pertinent data necessary for grading ChatGPT 4 responses - such as Author(s), Outlet, Pages, Title, and Year, and compiled them into a reference list. We formulated standard questions based on the first 50 authors and developed and tested a Python script with appropriate parameters. We then prompted the API using these standard questions loaded from an Excel file. The resulting responses were recorded in a separate Excel file. We then graded these responses on a binary scale for both default and strict settings, with particular attention paid to identifying hallucinations and traces of training data sources. Further analysis included investigating the availability of scholarly publications online, constructing descriptive statistical analyses with normal distributions, generating descriptive tables, and finally, formulating propositions for future research.

Our findings present a nuanced view of ChatGPT’s capabilities in the field of management history. We observe that while ChatGPT-4 demonstrates a high level of proficiency in describing and explaining the general concept of SWOT analysis and in terms of carrying out relatively straight-forward analyses using the framework, it performs worse on advanced tasks, such as detailing the concept’s history and origins. We found that there are notable discrepancies when it comes to detailing the origins of SWOT analysis. These inaccuracies range from minor factual errors to more significant hallucinations (Ahmad et al., 2023; Giuffrè et al., 2023; Moritz, 2024), where the AI fabricates elements of the strategy tool’s history. In addition, there might be another explanation for the accuracy of some results. Apart from harvesting training data from a few topical websites, it is also possible that references to scholarly articles are harvested from Google Scholar (scholar.google.com). Scholar Google indexes pre printer servers like ArXiv, SSRN or SocArXiv and provides direct links to PDF files.

Our research allows us to situate these findings within the expansive realm of Large Language Models (LLMs), with a specific emphasis on ChatGPT. During our investigation, we noted that ChatGPT’s responses to prompts on the conceptual contributions on SWOT analysis were somewhat lacking in accuracy. This inaccuracy could be attributed, in part, to the inherently complex and obscure history of SWOT Analysis, where there is little consensus even among experts in the area. Although exploring these inaccuracies across other management theories was beyond the purview of our study, it is plausible that similar discrepancies could be identified in ChatGPT’s treatment of other management concepts and ideas, such as PEST analysis, stakeholder analysis or scenario planning. For training the generative training data, access to open-access publications could be a good start. Supervised finetuning could be done by panels of topical experts, similar to a peer review process.

In conclusion, this study contributes to the ongoing debate about the reliability and usefulness of AI language models in academic and professional settings. Our findings not only add to our understanding of ChatGPT’s capabilities and limitations but also paves the way for future research on improving AI’s role in supporting strategic management education and practice.

The implications of our study are three-fold. Firstly, it highlights the potential of AI language models like ChatGPT as valuable resources for educational and professional purposes, offering instant access to a wealth of knowledge. It is, however, key that the information provided by the chatbots is accurate and that hallucinations are minimized. If not, the chatbots will perpetuate falsehood and contribute to the longevity of academic urban legends (McGee, 1985; Rekdal, 2014). Secondly, it underscores the critical need for users to approach AI-generated information cautiously, especially when dealing with complex historical data where accuracy is paramount.

Thirdly, our findings underscore the necessity for expert involvement in the training of AI models to enhance their proficiency in specialized fields such as business and management. This opens the door to the potential creation of chatbots specifically designed for business and management applications, offering tailored insights and analyses.

The data presented in the study can be found in the Supplementary material.

RP: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing. DM: Conceptualization, Validation, Writing – original draft, Writing – review & editing.