Gender medicine investigates the influence of sex/gender on the pathophysiology, prevention and treatment of disease, and the social and psychological aspects of illness (1, 2). Although medical research has been performed dominantly on men both in preclinical and clinical studies (3), there have been continuous efforts to overcome this gender bias (4). Since Healy B proposed gender differences in clinical outcomes (5), the subject has been discussed extensively, including in medical fields such as cardiovascular and gastrointestinal disease and oncology (6–9).

Artificial intelligence (AI) is a branch of computer science in which machines are developed to mimic human intelligence, including cognition, perception, and problem-solving (10, 11). This field has developed quickly and been applied to many areas, including medicine (11, 12). With its sophisticated algorithms, AI assists doctors and health professionals with data management, image-based diagnostics, robotic surgery, prediction models, and decision-making support (13, 14).

The widespread application of AI has promoted research in related fields, supporting the implementation of AI technologies in health care (14). Guo et al. found that publications on health care related to AI increased an average of 17% per year since 1995, with a steep increase of 45% between 2014 and 2019 (15). Along with the increased number of publications in medical AI, gender differences are important in other research areas. As bibliometric analysis quantitatively analyzes scientific publications, it can provide researchers and stakeholders with a macroscopic overview of research trends and help develop further research direction and policy. This study aims to assess the research activity on medical AI related to sex/gender and explore the global research trends over the past 20 years.

We extracted bibliographic data on gender-related medical AI articles from Web of Science (WoS) Core Collection. WoS Core Collection, which contains over 20,000 peer-reviewed, high-quality journals published worldwide covering various fields (16), is one of the most well-established and commonly used databases for bibliometric analysis (17, 18). Articles from 2001 to 2020 were collected using the following search terms: {TS=(“artificial intelligence” OR “machine intelligence” OR “artificial neural network^*” OR “machine learning” OR “deep learn^*” OR “natural language process^*” OR “robotic^*” OR “thinking computer system” OR “fuzzy expert system^*” OR “evolutionary computation” OR “hybrid intelligent system^*”)} AND {TS=(disease^* OR illness OR health-related OR medic^* OR “medical diagnosis” OR treatment OR health^* OR wellness OR well-being OR prescription OR drug)} AND {TS=(gender OR sex OR male OR female)}.

The inclusion criteria were: (i) articles, review articles, and editorial materials; (ii) publications from 2001 to 2020; and (iii) full texts published in English. Articles were excluded if they were a proceeding paper, meeting abstract, book review, book chapter, or correction.

For bibliometric analysis, we extracted the title, abstract, year of publication, journal name with impact factor, authors, institution, country, WoS subject category, keywords, and number of citations. We determined the annual publication growth, the relative research interest (RRI), and percentage of gender-related articles in the medical AI area. Four 5-year periods (2001–2005, 2006–2010, 2011–2015, and 2016–2020) were used to compare the progress of each country. Two bibliometric indicators, Specialization Index (SI) and Category Normalized Citation Impact (CNCI), were computed by InCites with the following equation (19, 20):

We also analyzed the publication distributions by institution, author, WoS subject category, and journal. We used VOSviewer (Leiden University, Leiden, The Netherlands; version 1.6.11) to draw network visualization maps and performed a citation analysis to identify the most cited articles.

Our bibliometric analysis of the gender-related articles in medical AI revealed major changes over the last 20 years. The number of publications and percentage of gender-related articles in medical AI fields continuously increased from 2001 to 2020, with a steep increase in the past 5 years. This change can be explained by both increased interest of AI and awareness of gender medicine. Due to the technological development including computing power and data storage, AI has been developed (24), leading to advances in researches and collaborative works in medical AI fields (15, 25). In addition, there have been continuous efforts to overcome this gender bias (4), although women used to be underrepresented in clinical research (26).

After the National Institute of Health (NIH) Revitalization Act of 1993 mandated the enrollment of women and ethnic minorities in clinical research in the USA (27), funding agencies such as the Canadian Institutes of Health Research (28), European Commission (29), and NIH (30) required consideration of sex and gender in study design, analysis, and reporting for grant applications. In addition, several editorial guidelines included gender-specific work [e.g., Animal Research: Reporting In Vivo Experiments (ARRIVE) (31), Sex and Gender Equity in Research (SAGER) (32), and International Committee of Medical Journal Editors (ICMJE) recommendations (33)].

Both the number of publications and RRI on gender-related medical AI have steadily increased for 20 years, showing the increase of research interests in related fields. The percentage of gender-related articles in medical AI doubled in the last 20 years to 6.5%, although this figure remains small. According to Sugimoto et al., in 2016, two-thirds of articles were gender-related reporting articles of clinical medicine and public health research, whereas one-third of such articles were for biomedical research (34). Geller et al. showed that 26% of NIH-funded randomized control trials in 2018 included sex as a covariate (35). Compared to other fields, medical AI had a low percentage of gender-related articles. This requires further study.

As the number of publications can only provide volumetric information, our analysis showed SI and CNCI across countries and over time. These two parameters can provide different perspectives on research trends (36). SI, the ratio of the percentage of publications related to the specific area in a given country to those worldwide, evaluates specialization. CNCI, which is the ratio of the observed to the expected number of citations in the same WoS category, shows the citation impact. For example, although Canada and Peoples R China had the highest percentage increase in the number of publications over the previous 20 years, Canada showed overspecialization and citation impact specifically in gender-related medicine AI research compared to the worldwide figures, whereas Peoples R China did not.

The USA had the most publications on gender-related medical AI between 2001 and 2020, with overall high CNCIs and SIs. As expected, the top 10 institutions and high-ranked authors were from the USA. According to the network visualization plot, most of the top 10 institutions were also well-connected through research networks. According to the bibliographic analysis of authors, it was possible to understand the relationships between authors. Author co-citation analysis visualized the intellectual structure of the scientific knowledge domain by calculating how often the author's work is cited with other authors (37), whereas bibliographic coupling showed the similarity relationships by calculating how often two papers are cited together (38). In addition, co-authorship analysis showed the cooperative and interactive relationships between authors, indicating the authors' willingness to write a paper together (39).

Surgery and Urology and Nephrology was the most common WoS subject category in our analysis. Similarly, Surgical Endoscopy and Other Interventional Techniques, Asian Journal of Surgery, Urology, and Journal of Urology were the journals that ranked high in the number of gender-related publications in medical AI. Surgery is one of the most developed areas in medical AI. AI can be applied pre-, intra-, and post-surgery, such as for preoperative risk prediction, imaging, 3D reconstruction, and robotic intervention (40, 41). As several studies reported the sex differences in prognosis after surgery (42–44), sex should be considered in AI surgery research. Urology is another area of interest in gender-related medical AI. There are anatomical, physiological, and pathophysiological urological differences between men and women (45). Hormones and metabolisms differ by sex, thereby affecting medical conditions (46). Furthermore, environmental and occupational exposures may differ by gender, which should be considered in gender medicine (47).

The network visualization map of keywords across the time periods showed that research topics have continuously expanded and changed over past two decades. In the first period (2001–2005), there was only two clusters; one was disease and the other was artificial neural networks and cancer. In the last period (2016–2020), there was 6 clusters including machine learning, risk, and surgery. These results can be used to guide future studies by listing the trending topics.

The citation analysis showed that gender-related medical AI had a high influence, with an average of 15 citations. The topics covered in the top 9 articles with the highest citations were surgery, imaging, and prediction models. The most cited article was the study of Wynants et al. (21), which systematically reviewed and critically evaluated all 232 predictive models for diagnosis and prognosis of COVID-19 including 169 studies. This study showed that gender is one of the frequent prognostic factors of COVID-19. As the COVID-19 pandemic has posed a threat to the global economic and health systems with high morbidity and mortality (48), COVID-19-related articles have recently dominated medical publishing in the last 2 years (49). The study of Poplin et al., the second most cited article, developed deep learning models using retinal fundus images to predict multiple cardiovascular risk factors including age and gender (22). The third most cited article was the article by Aarts et al. (23), which reviewed the effectiveness and safety of four types of hysterectomy surgeries in women with benign gynecological diseases. Interestingly, most of the top 9 articles were published in journals with an IF < 10. This demonstrates an increased interest in this field.

This bibliometric study has some limitations. First, like other bibliometric studies, the results can be affected by the search term and databases used. As we only used the WoS, we could not include publications in other electronic databases (e.g., PubMed or Embase). However, we selected the WoS covering a broad range of articles (50) and applied search strategies with high sensitivity. Second, there was a possibility of the inclusion of studies that had little to do with our topics. As we focused on showing macroscopic tendencies, studies were identified through search if they had AI-, medication-, and gender-related terms in titles, abstracts, or keywords, regardless of their topics. For example, the article by Roberts et al. (51), the originally identified as the second most cited article, suggested the structural topic models for surveys in political sciences; it was not presented in Table 6 for qualitative interpretation. Despite this, including gender-related words is meaningful because it covers gender in any way. Third, the number of citations can be biased by self-citations and time elapsed since publication. Lastly, non-English publications were not included.

To the best of our knowledge, this is the first bibliometric study to investigate the worldwide research output of gender-related medical AI by bibliometric analysis. This study concluded that gender-related research in medical AI increased over the past 20 years. Despite increased interest, gender-related research is still low in medical AI field and further research is needed.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

JY and HG contributed to designing the study. HY contributed to acquisition and analysis of data. HY, JY, and HG contributed to interpretation of data. HY and HL contributed to drafting of the manuscript. JY and HG contributed to critical revision of the manuscript. All authors contributed to the conception and design and read and approved the final manuscript.

The research leading to these results received funding from Ministry of Science and ICT under WISET202103GI01.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.