Recently, artificial intelligence (AI) has gained popularity as a source of information due to its ability to provide human-like responses to prompts and questions. ChatGPT (Open AI) (1), Gemini (Google) (2), and Copilot (Microsoft) (3) are examples of such engines that have gained popularity. Patients will undoubtedly continue to turn towards AI for inquiries regarding their medical conditions, treatments, and related advice, given its ease of use (4, 5).

The novel use of AI chatbots by orthopaedic patients has already commenced. AI has demonstrated its ability to bridge gaps in patient understanding and assist in patient medical decision-making regarding informed consent for orthopaedic surgical procedures, an area often associated with shortcomings (6). In total hip arthroplasty, AI chatbot ChatGPT has demonstrated the capability of providing adequate responses to most frequently asked questions regarding the indications for surgery, the surgery itself, and the postoperative recovery (7), proving to be an asset to surgeons looking to maximize efficiency while simultaneously keeping their patients well-informed. Similar assessments have been done for treatments of elbow ulnar collateral ligament (8) and anterior cruciate ligament injuries (9). ChatGPT responses were also assessed in foot and ankle surgery regarding treatment for common conditions (10).

While multiple surgical treatment options exist for acute Achilles tendon ruptures (AATR), there is lack of consensus which operative option is the most optimal. There is also evidence for effective nonoperative treatment, leading patients to feel overwhelmed in their search for answers. Consequently, patients may seek answers from AI chatbots. Ultimately, the variety of treatment options for AATR and indications for them, opens an opportunity for AI chatbots to provide information to patients and streamline the patient-surgeon discussion. Although previous research has highlighted the potential of AI chatbots to provide adequate responses to other orthopaedic procedures, no study has been done in foot and ankle surgery regarding AATR. This study, therefore, aims to assess and compare the responses to some of the most frequently asked questions about AATR provided by ChatGPT (3.5; Open AI), Gemini (Google), and Copilot (Microsoft), and ultimately determine their utility as an adjunct educational tool in clinical practice.

In order to develop a clinically relevant and patient-centered list of ten (10) frequently asked questions regarding the diagnosis and management of AATR (Table 1), a multi-stage process was utilized. Initially, a comprehensive survey of patient information sections on prominent orthopaedic and general medical websites—including WebMD, Healthline, Johns Hopkins Medicine, Hospital for Special Surgery, Campbell Clinic, and Massachusetts General Hospital (all accessed via Google search) was conducted. The aim was to identify questions about AATR that recurred frequently across these reputable patient-facing resources. This pool of commonly encountered online questions was then reviewed and discussed by the senior clinical authors, drawing upon their collective daily experience and expertise in treating patients with AATRs in their foot and ankle and sports medicine practices. The final selection of ten questions was aimed to reflect genuine and common inquiries posed by patients during clinical consultations, ensuring the study's practical relevance (Table 1). Three free online AI forums [ChatGPT 3.5 (1), Google Gemini (2), and Microsoft Copilot (3)] were last accessed on October 5, 2024.

For each AI chatbot, and for each of the ten questions, a new and entirely separate chat session was opened to prevent any influence from previous interactions, ensuring that chat history was cleared and not carried over. The responses to the 10 questions from each chatbot were recorded. For brevity and conciseness, each question ended with “be concise” to appropriately prompt the chatbots. A questionnaire was then generated with the three chatbots' responses to each question. Four board-certified orthopaedic surgeons, consisting of two sports surgeons and two foot and ankle surgeons, were chosen as the reviewers to the AI chatbot responses considering their expertise and primary role in surgical treatment of AATRs. The reviewers were blinded to the names of the AI chatbots and source of each response. A rating system, modeled from a previously described scale (7) was used to assess the responses. The system was divided into a four-point scale (Table 2). The chatbot responses were graded based on the need for clarification and overall satisfaction. Satisfactory responses provided enough information that would not require further clarification for the patient. Responses requiring minimal clarification were acceptable but not detailed enough, and responses requiring substantial clarification did not provide enough evidence-based information. Finally, unsatisfactory responses did not provide accurate information. Four board-certified subspecialty-trained orthopaedic surgeons (JK and RT in foot and ankle, KF and RR in sports medicine) were given the questionnaire and asked to assess the value of the three responses for the ten questions using the four-point scale. A mean score for each chatbot response was calculated from the four scores of the orthopaedic surgeons.

The data for the responses to the 10 questions were reported as means ± standard deviation (Table 3). A Kruskal–Wallis test was used to compare the responses between the three AI systems using the scores assigned by the surgeons. The level of significance was set at p ≤ 0.05. Statistical analyses were completed using IBM SPSS Statistics for Windows, Version 29.0.2.0 Armonk, NY: IBM Corp. No institutional review board approval was required by this study.

The full response to each question can be found in Supplement 1. Samples of the questionnaire including the responses to the questions by each AI chatbot can be found in Supplement 2.

Online AI chatbots are free to use by the public and their versatile use can be translated to application in healthcare. As chatbots become more refined and frequently used by patients, it is important to continue to assess them and utilize them appropriately in the clinical setting.

Treatment of AATRs poses a clinical challenge to orthopaedic surgeons due to the multifactorial approach to treatment. Given the variable indications for surgical management, different options for operative treatment, as well as the influx of evidence for nonoperative management in specific clinical scenarios, the shared decision-making process between the surgeon and patient can be extensive. This study's goal was to determine the value of responses to frequently asked questions produced by popular free chatbots as patients continue to turn to AI for information to guide their decisions when undergoing medical care. Given the use of AI has previously been shown to be a valuable aide in shared medical decision making without adversely affecting clinical efficiency (52) adding it as a reinforcing tool when treating patients with AATRs could improve the patient and surgeon experience.

The use of ChatGPT as an adjunct educational tool has been previously studied in orthopaedic surgery. In total hip arthroplasty (THA), ChatGPT provided easy-to-understand answers to frequently asked questions about indications for surgery, various surgical techniques, and outcomes (7). Utilizing a similar mode of assessment, Johns et al. (8) analyzed ChatGPT's responses to common inquiries in elbow ulnar collateral ligament (UCL) reconstruction and concluded that 60% of the responses were satisfactory or excellent, while 40% of the responses were unsatisfactory, needing further explanation. Still, they demonstrated the potential of a free online chatbot to improve patients' basic knowledge regarding management of UCL reconstruction. Employing a multi-metric approach that included the validated DISCERN instrument and the AIRM scale, Anastasio et al. (10) assessed ChatGPT's responses to common foot and ankle surgery questions, finding variable quality in the information provided. In their discussion, they also highlighted the need for future research to directly compare AI-generated responses against information from traditional internet search tools to better contextualize their value. Finally, Li at al (9). followed the same pattern and assessed frequently asked questions regarding anterior cruciate ligament (ACL) reconstruction. They concluded that ChatGPT was able to adequately respond to background questions but noted that any treatment-specific questions would be better addressed by the treating orthopaedic surgeon.

To our knowledge, there are no studies comparing the most popular AI chatbots and their accuracy in responding to questions about the evaluation and management of AATR. The authors chose AATR management due to controversy surrounding the various forms of surgical repair techniques, postoperative protocols, and nonoperative treatment options dependent on surgeon and patient factors. The responses from the three AI chatbots analyzed in our study were largely uniform and satisfactory with almost all responses needing minimal clarification. One of the engines, Microsoft Copilot, scored comparatively lower on three of the ten questions. The four physicians reviewing these responses deemed the answer to those three questions inferior compared to ChatGPT and Google Gemini. Nonetheless, the overall value of these responses concludes that these forums can be beneficial for patient use and could provide additional reinforcements to the conversations between patients and their surgeons. However, while our findings suggest AI's promising role in AATR patient education, these results must be interpreted with an understanding of current AI's broader challenges. Notably, its crucial general limitation of producing “hallucinations', where responses may appear credible but are, in fact, incorrect or not grounded in evidence. While no such inaccuracies were noted by our reviewers in the answers provided for this study, this inherent potential for error underscores the critical importance of patients verifying any medical advice obtained from AI with healthcare professionals. This highlights the ongoing need for vigilance and robust verification mechanisms as AI integrates into healthcare. With continued development, AI engines will likely become valuable supportive tools in orthopaedic patient education. Building on this study's insights, future research can further explore AI's efficacy in orthopaedic patient education. Key directions include evaluating responses for a wider range of orthopaedic conditions, such as other traumatic injuries and chronic diseases, assessing AI chatbot performance across diverse languages to ensure equitable patient access, and reproducibility among patients. Importantly, incorporating direct feedback from patients or non-medical individuals on the clarity, understandability, and trustworthiness of AI-generated information will be vital for gauging its real-world applicability and impact.