<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Archiving and Interchange DTD v2.3 20070202//EN" "archivearticle.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="systematic-review">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Educ.</journal-id>
<journal-title>Frontiers in Education</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Educ.</abbrev-journal-title>
<issn pub-type="epub">2504-284X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/feduc.2023.1240962</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Education</subject>
<subj-group>
<subject>Systematic Review</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Dynamics of automatized measures of creativity: mapping the landscape to quantify creative ideation</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Ul Haq</surname> <given-names>Ijaz</given-names></name>
<uri xlink:href="http://loop.frontiersin.org/people/2035287/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Pifarr&#x000E9;</surname> <given-names>Manoli</given-names></name>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/1387373/overview"/>
</contrib>
</contrib-group>
<aff><institution>Faculty of Education, Psychology and Social Work, University of Lleida</institution>, <addr-line>Lleida</addr-line>, <country>Spain</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Mohammad Khalil, University of Bergen, Norway</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Chen-Yao Kao, National University of Tainan, Taiwan; Faisal Saeed, Kyungpook National University, Republic of Korea</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Manoli Pifarr&#x000E9; <email>manoli.pifarre&#x00040;udl.cat</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>12</day>
<month>10</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="collection">
<year>2023</year>
</pub-date>
<volume>8</volume>
<elocation-id>1240962</elocation-id>
<history>
<date date-type="received">
<day>15</day>
<month>06</month>
<year>2023</year>
</date>
<date date-type="accepted">
<day>18</day>
<month>09</month>
<year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2023 Ul Haq and Pifarr&#x000E9;.</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Ul Haq and Pifarr&#x000E9;</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract>
<p>The growing body of creativity research involves Artificial Intelligence (AI) and Machine learning (ML) approaches to automatically evaluating creative solutions. However, numerous challenges persist in evaluating the creativity dimensions and the methodologies employed for automatic evaluation. This paper contributes to this research gap with a scoping review that maps the Natural Language Processing (NLP) approaches to computations of different creativity dimensions. The review has two research objectives to cover the scope of automatic creativity evaluation: to identify different computational approaches and techniques in creativity evaluation and, to analyze the automatic evaluation of different creativity dimensions. As a first result, the scoping review provides a categorization of the automatic creativity research in the reviewed papers into three NLP approaches, namely: text similarity, text classification, and text mining. This categorization and further compilation of computational techniques used in these NLP approaches help ameliorate their application scenarios, research gaps, research limitations, and alternative solutions. As a second result, the thorough analysis of the automatic evaluation of different creativity dimensions differentiated the evaluation of 25 different creativity dimensions. Attending similarities in definitions and computations, we characterized seven core creativity dimensions, namely: novelty, value, flexibility, elaboration, fluency, feasibility, and others related to playful aspects of creativity. We hope this scoping review could provide valuable insights for researchers from psychology, education, AI, and others to make evidence-based decisions when developing automated creativity evaluation.</p></abstract>
<kwd-group>
<kwd>review</kwd>
<kwd>creativity process</kwd>
<kwd>ideation</kwd>
<kwd>evaluation</kwd>
<kwd>artificial intelligence</kwd>
</kwd-group>
<contract-sponsor id="cn001">Ministerio de Ciencia, Innovaci&#x000F3;n y Universidades<named-content content-type="fundref-id">10.13039/100014440</named-content></contract-sponsor>
<counts>
<fig-count count="4"/>
<table-count count="3"/>
<equation-count count="0"/>
<ref-count count="86"/>
<page-count count="16"/>
<word-count count="11362"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Educational Psychology</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec id="s1">
<title>1. Introduction</title>
<p>Creativity as a 21<sup>st</sup> century skill is increasingly becoming an explicit part of educational policy initiatives and curricula (Plucker et al., <xref ref-type="bibr" rid="B59">2023</xref>). Creativity is a multifaceted concept, and research in this area has made remarkable progress in understanding the different components embedded in creativity phenomena, such as idea generation through collaborative creative (co-creative) processes (Sawyer, <xref ref-type="bibr" rid="B66">2011</xref>, <xref ref-type="bibr" rid="B67">2022</xref>). Furthermore, research also revealed the significance of another important component of creativity: creativity evaluation (Guo et al., <xref ref-type="bibr" rid="B31">2023</xref>), which is the ability to accurately identify creative ideas, solutions, or characteristics among individuals to understand their creative strengths and potential (Kim et al., <xref ref-type="bibr" rid="B42">2019</xref>). In the educational context, creativity evaluation is an essential step for teachers and students because it is helpful to monitor, refine, and implement creative ideas, which could improve students&#x00027; creative performance in the creative process (Rominger et al., <xref ref-type="bibr" rid="B64">2022</xref>).</p>
<p>Creativity evaluation poses a challenging problem in creativity research. Creativity evaluation mainly involves four dimensions: fluency (number of meaningful ideas), flexibility (number of different categories), elaboration (detailed ideas), and novelty (uniqueness of ideas) (Bozkurt Altan and Tan, <xref ref-type="bibr" rid="B9">2021</xref>). To evaluate these creativity dimensions, various manual creativity evaluations (paper-based) and psychological tests have been commonly used (Rafner et al., <xref ref-type="bibr" rid="B62">2022</xref>). Examples are the Torrance Tests of Creative Thinking (Torrance, <xref ref-type="bibr" rid="B75">2008</xref>), Creativity Assessment Packet (CAP) (Williams, <xref ref-type="bibr" rid="B83">1980</xref>), and Divergent Production abilities (DP), (Guilford, <xref ref-type="bibr" rid="B30">1967</xref>). Other ways to evaluate creativity include a rating scale (Gong and Zhang, <xref ref-type="bibr" rid="B29">2017</xref>; Birkey and Hausserman, <xref ref-type="bibr" rid="B6">2019</xref>), a survey and questionnaire (De Stobbeleir et al., <xref ref-type="bibr" rid="B17">2011</xref>; Gong et al., <xref ref-type="bibr" rid="B28">2019</xref>), using a grading rubric (Vo and Asojo, <xref ref-type="bibr" rid="B79">2018</xref>), and subjective scoring of creativity dimensions (George and Wiley, <xref ref-type="bibr" rid="B26">2020</xref>). However, these manual creativity evaluations face some challenges, e.g., being error-prone (experts&#x00027; ratings do not always agree on what is creative) and time-consuming (Said-Metwaly et al., <xref ref-type="bibr" rid="B65">2017</xref>; Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref>). These challenges can be tackled using automated creativity evaluation supported by AI techniques which can also enrich co-creation by providing real-time feedback to guide students to develop novel solutions (George and Wiley, <xref ref-type="bibr" rid="B26">2020</xref>; Kenworthy et al., <xref ref-type="bibr" rid="B41">2023</xref>).</p>
<p>Artificial intelligence (AI) focuses on enabling machines to perform tasks that typically demand human intelligence. Within AI, machine learning (ML) algorithms learn from data to make predictions. Notably, computer vision is used for analyzing figural data, and NLP is used for analyzing textual data. Given our focus on textual ideas, NLP enables machines to comprehend, interpret, analyze, and generate human language (Braun et al., <xref ref-type="bibr" rid="B10">2017</xref>). NLP contains a variety of approaches and techniques such as text similarity, text classification, topic modeling, information extraction, and text generation, each with its computational techniques spanning from statistical methods to predictive and deep learning models. NLP provides different opportunities to compute variables related to creativity dimensions. Among these, the following five variables could be computed in the vector space provided by NLP: (1) Contextual and semantic similarity are applied to measure the uniqueness of ideas and originality (Hass, <xref ref-type="bibr" rid="B32">2017</xref>; Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref>); (2) text clustering could identify different categories in the text; (3) text classification is used to compute novelty (Simpson et al., <xref ref-type="bibr" rid="B70">2019</xref>); (4) keyword searching is mainly used to compute elaboration (Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref>); and (5) information retrieval could be applied to score the level of idea elaboration (Vartanian et al., <xref ref-type="bibr" rid="B77">2020</xref>). These implications of NLP in co-creative processes can be used to automatically evaluate creativity and support co-creation by providing feedback (Bae et al., <xref ref-type="bibr" rid="B4">2020</xref>; Kang et al., <xref ref-type="bibr" rid="B38">2021</xref>; Kovalkov et al., <xref ref-type="bibr" rid="B44">2021</xref>).</p>
<p>Considering the above implications of NLP, current research focuses on studying how different computational techniques can measure creativity dimensions (Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref>). Research on this topic has been very productive and has designed other computational techniques to measure creativity dimensions, e.g., (1) novelty is measured by keyword similarity (Prasch et al., <xref ref-type="bibr" rid="B60">2020</xref>), part of speech tagging (Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref>; Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref>), and different ML classifiers, such as Bayesian classifiers, random tree, and Support Vector Machine (SVM) (Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>; Simpson et al., <xref ref-type="bibr" rid="B70">2019</xref>; Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref>); (2) originality dimension is measured by Latent Semantic Analysis (LSA) (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>), Global Vectors for word representation (GloVe) (Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref>), and part of speech tagging (Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref>); (3) fluency dimension is measured by LSA (Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref>; LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref>); (4) elaboration dimension is measured by parts of speech tagging (Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref>); and (5) level of details dimension is measured by text-mining methods (Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref>).</p>
<p>This study aims to tackle the following four main challenges that current research faces when designing computational techniques to measure creativity: (1) a range of computational techniques evaluating various creativity dimensions; (2) there is no consensus about the use of a specific technique for computing a specific creativity dimension; (3) some of the studies do not expose and argue the rationale that supports the use of a specific technique to compute a specific creativity dimension, e.g., evaluation of the category switch dimension of creativity using LSA (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>); and (4) the need to consider the limitations of computational techniques that could affect the evaluation of creativity dimensions (Olivares-Rodr&#x000ED;guez et al., <xref ref-type="bibr" rid="B55">2017</xref>; Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref>). Considering these challenges, as per our knowledge, no existing literature review addresses the above four challenges. Therefore, this exploration led us to two research questions: (1) What NLP approaches and techniques are used to automatically measure creativity? and (2) What creativity dimensions are computed automatically, and how? These research questions enable us to address the previous four challenges in automatic creativity evaluation. Furthermore, these research questions help to understand the concept of NLP approaches and creativity dimensions, their applications in evaluating creativity dimensions, identify research gaps and limitations, and propose alternative solutions for advancing the evaluation and promotion of creativity. Therefore, we chose a scoping review because it helps to understand key concepts and identify knowledge gaps (Munn et al., <xref ref-type="bibr" rid="B54">2018</xref>) to inspire innovation and improve the education of future generations through advanced technologies.</p>
</sec>
<sec id="s2">
<title>2. Research objectives</title>
<p>This scoping review aims to meet the following two objectives.</p>
<list list-type="order">
<list-item><p>To identify and categorize different ML approaches used in automatic creativity evaluation, highlighting their application scenarios and limitations of computational approaches and techniques. This categorization could contribute to a deeper understanding of the contribution that different ML approaches can make to automatic creativity.</p></list-item>
<list-item><p>To analyze the definition and computation of different creativity dimensions used in automatic creativity evaluation research. This analysis can help establish a joint agreement on creativity dimensions and their computation, which will pave the way for advancements in automatic creativity evaluation.</p></list-item>
</list>
</sec>
<sec id="s3">
<title>3. Method</title>
<p>This section describes the sampling method we used to collect and compile the state-of-the-art approaches to automatic creativity evaluation. Our methodological framework follows the PRISMA technique (Dickson and Yeung, <xref ref-type="bibr" rid="B19">2022</xref>) by conducting a scoping review to find relevant and significant research papers by identifying the following four core concepts.</p>
<list list-type="order">
<list-item><p>Creativity: The articles must be related to creativity, especially the creative process (Sawyer, <xref ref-type="bibr" rid="B66">2011</xref>).</p></list-item>
<list-item><p>Measurement/evaluation/assessment of creativity dimensions.</p></list-item>
<list-item><p>Technology: We selected those studies that are assisted or evaluated with technology support. This core concept aims to review the technological support for creativity evaluation and explore future research in the creative process.</p></list-item>
<list-item><p>Domain: We focused on the creativity process applicable in the educational sector that helps to enhance students&#x00027; creativity. Other fields such as medicine, finance, and business were excluded from the search query.</p></list-item>
</list>
<p>Exploring the current literature considering the above four core concepts, peer-reviewed journals and conference papers are included in this mapping study. Regarding the time span, we searched from 2005 to 2021, although interestingly, according to our inclusion&#x02013;exclusion criteria, the oldest study included is from 2009, and most are from recent past years. It indicates that automatic creativity evaluation has recently grabbed researchers&#x00027; attention and is still an open and active research problem.</p>
<p>We excluded articles focused on the person&#x00027;s or organization&#x00027;s creativity evaluation. We excluded domains other than education, e.g., medicine and finance. Articles in other languages apart from English published before 2005 and articles with no technological role and creativity were also excluded.</p>
<p>For this mapping study, we extracted articles published in Scopus with the search query: [(creativ<sup>&#x0002A;</sup> OR &#x0201C;Creative Process&#x0201D; OR &#x0201C;Novelty&#x0201D; OR &#x0201C;Flexibility&#x0201D; OR &#x0201C;Fluency&#x0201D; OR &#x0201C;Elaboration&#x0201D; OR &#x0201C;Originality&#x0201D;) AND (Measur<sup>&#x0002A;</sup>OR Evaluat<sup>&#x0002A;</sup> OR Asses<sup>&#x0002A;</sup> OR Calcul<sup>&#x0002A;</sup> OR Analys<sup>&#x0002A;</sup> OR Scor<sup>&#x0002A;</sup> OR Qunat<sup>&#x0002A;</sup>) AND (Automat<sup>&#x0002A;</sup> OR Comput<sup>&#x0002A;</sup> OR Machin<sup>&#x0002A;</sup> OR Natural<sup>&#x0002A;</sup> OR Artificial<sup>&#x0002A;</sup> OR Deep learning OR Mathemat<sup>&#x0002A;</sup> OR Mining) AND (E-learning OR educa<sup>&#x0002A;</sup> OR Learn<sup>&#x0002A;</sup> OR School OR students<sup>&#x0002A;</sup>)].</p>
<p>The search query resulted in 364 research articles. By applying the inclusion and exclusion criteria while reading the title, abstract, keywords, and conclusion, the search is filtered to 65 articles. Furthermore, the authors read, checked, and discussed the selected articles and conducted all the screening stages to answer the two research questions. The consensus among the authors developed by solving discrepancies since member checking is a well-established procedure to build up &#x0201C;trustworthiness&#x0201D; in qualitative research (Toma, <xref ref-type="bibr" rid="B74">2011</xref>). After this process, a total of 26 articles were finally included in this scoping review. The overall article selection procedure through the PRISMA technique is depicted in <xref ref-type="fig" rid="F1">Figure 1</xref>.</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Screening procedure of the articles using the PRISMA technique.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="feduc-08-1240962-g0001.tif"/>
</fig></sec>
<sec id="s4">
<title>4. Results</title>
<sec>
<title>4.1. Approaches and techniques used in automatic creativity evaluation (RQ1)</title>
<p>The compilation of computational approaches and techniques in automatic creativity evaluation research to answer the first research question gives the following three results;</p>
<p>The first result reveals that creativity evaluation research spreads over three different NLP approaches, namely, (1) text similarity, which measures the relatedness and closeness among words, sentences, or paragraphs presented in a numerical space; (2) text classification, which is a supervised learning approach (needs data training) that requires ML algorithms [such as the K-nearest neighbor (KNN) algorithm and random forest] to analyze text automatically and then to assign a set of predefined tags or categories; and (3) text mining that uses NLP to examine and transform extensive unstructured text data to discover new information and patterns. These three NLP approaches and their computational techniques identified in the studies included in this review are displayed in <xref ref-type="fig" rid="F2">Figure 2</xref>.</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>Different NLP approaches in creativity evaluation.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="feduc-08-1240962-g0002.tif"/>
</fig>
<p>As a second result, the scoping review shows that text similarity is the most common approach (69% of the reviewed studies), followed by text classification (27%), and text mining is less commonly used (only 4% of the studies), as shown in <xref ref-type="fig" rid="F2">Figure 2</xref>.</p>
<p>As a third result, our scoping review has identified and categorized the computation techniques used in the three NLP approaches (text similarity, text classification, and text mining) and the creativity dimensions that were evaluated automatically. In the following sections, we present the mapping that we have built after a thorough analysis of all the studies included in the scoping review.</p>
<p><bold>Regarding the text similarity</bold> approach, NLP converts textual ideas into a numerical vector space. To do this conversion, the studies revised the use of a wide range of techniques that could be classified into the next three categories: string-based similarity, corpus-based similarity, and knowledge-based similarity. These three categories and their computational techniques identified in the reviewed studies are shown in <xref ref-type="fig" rid="F3">Figure 3</xref>, and <xref ref-type="table" rid="T1">Table 1</xref> maps automatic creativity evaluation studies into the three categories and techniques used.</p>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p>Text similarity approaches, categories, sub-categories, and their computational techniques.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="feduc-08-1240962-g0003.tif"/>
</fig>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Categorizing of review studies in text similarity approaches and percentages of studies included in the review that use each approach.</p></caption> 
<table frame="box" rules="all">
<thead>
<tr style="background-color:&#x00023;919498;color:&#x00023;ffffff">
<th valign="top" align="left"><bold>Text similarity categories</bold></th>
<th valign="top" align="left"><bold>Sub-categories</bold></th>
<th valign="top" align="left"><bold>Vectorization techniques</bold></th>
<th valign="top" align="left"><bold>Dimensions</bold></th>
<th valign="top" align="left"><bold>Studies</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">String-based 6%</td>
<td/>
<td valign="top" align="left">Keyword matching</td>
<td valign="top" align="left">Novelty, usefulness</td>
<td valign="top" align="left">Prasch et al., <xref ref-type="bibr" rid="B60">2020</xref></td>
</tr>
<tr>
<td valign="top" align="left">Knowledge-based<break/> 22%</td>
<td/>
<td valign="top" align="left">Part of speech tagging</td>
<td valign="top" align="left">Novelty, level of details</td>
<td valign="top" align="left">Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Part of speech tagging</td>
<td valign="top" align="left">Originality, value, overall value, feasibility</td>
<td valign="top" align="left">Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Clustering in the knowledge graph</td>
<td valign="top" align="left">Novelty, surprise, rarity, recreational effort</td>
<td valign="top" align="left">Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Semantic network</td>
<td valign="top" align="left">Flexibility</td>
<td valign="top" align="left">Cosgrove et al., <xref ref-type="bibr" rid="B16">2021</xref></td>
</tr>
<tr>
<td valign="top" align="left">Corpus-based 72%</td>
<td valign="top" align="left">Statical based</td>
<td valign="top" align="left">LSA</td>
<td valign="top" align="left">Category switch, variety, original, prune originality, common use</td>
<td valign="top" align="left">Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">LSA</td>
<td valign="top" align="left">Fluency, originality</td>
<td valign="top" align="left">Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">LSA</td>
<td valign="top" align="left">Similarity, fluency</td>
<td valign="top" align="left">LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Vectorization of linguistic features</td>
<td valign="top" align="left">Similarity</td>
<td valign="top" align="left">Zu&#x000F1;iga et al., <xref ref-type="bibr" rid="B86">2017</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Deep learning</td>
<td valign="top" align="left">Word2Vec</td>
<td valign="top" align="left">Originality, flexibility, fluency</td>
<td valign="top" align="left">Sung et al., <xref ref-type="bibr" rid="B73">2022</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">GloVe</td>
<td valign="top" align="left">Originality</td>
<td valign="top" align="left">Acar et al., <xref ref-type="bibr" rid="B1">2021</xref>; Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>; Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">GloVe</td>
<td valign="top" align="left">Similarity of text</td>
<td valign="top" align="left">Olson et al., <xref ref-type="bibr" rid="B56">2021</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">GloVe</td>
<td valign="top" align="left">Diversity (Novelty)</td>
<td valign="top" align="left">Johnson and Hass, <xref ref-type="bibr" rid="B37">2022</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Universal sentence encoder</td>
<td valign="top" align="left">Novelty</td>
<td valign="top" align="left">Kenworthy et al., <xref ref-type="bibr" rid="B41">2023</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">GAN</td>
<td valign="top" align="left">Novelty, value, surprise</td>
<td valign="top" align="left">Franceschelli and Musolesi, <xref ref-type="bibr" rid="B25">2022</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">LSTM</td>
<td valign="top" align="left">originality</td>
<td valign="top" align="left">Marrone et al., <xref ref-type="bibr" rid="B50">2022</xref></td>
</tr>
</tbody>
</table>
</table-wrap>
<p>In the first category, string-based similarity (6% of the text similarity approach of reviewed studies) matches exact keywords or alphabet strings, e.g., Longest Common Substring (LCS) or N-gram (a subsequence of n items from a given sequence of text). The string similarity of ideas with the existing ideas in the database is computed by using keyword matching (Prasch et al., <xref ref-type="bibr" rid="B60">2020</xref>).</p>
<p>In the second category, corpus-based similarity is mostly used (72% of textual similarity), and the results are presented in <xref ref-type="table" rid="T1">Table 1</xref>. The corpus-based similarity is classified into two sub-categories: On the one hand, the statistical-based models, e.g., LSA, present corpus in the word-document matrix as words in row vectors and each document as a column vector, and weighting schemes and dimension reduction schemes are applied before calculating the cosine similarity among word vectors (Martin and Berry, <xref ref-type="bibr" rid="B51">2007</xref>; Wagire et al., <xref ref-type="bibr" rid="B80">2020</xref>). On the other hand, the deep learning-based models (both word and sentence embeddings) use supervised (which need to be trained on data), semi-supervised, or unsupervised methods (no prior training) that are trained on a large corpus, e.g., Wikipedia and common crawl dataset. Deep learning models such as Word2Vec (Mikolov et al., <xref ref-type="bibr" rid="B53">2013</xref>) or GloVe (Pennington et al., <xref ref-type="bibr" rid="B58">2014</xref>) use knowledge from large datasets, encode the data, and find similarities in words or sentences. The GloVe model showed reliable results as compared with the experts&#x00027; scores, especially for single-word creativity tasks (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>; Johnson and Hass, <xref ref-type="bibr" rid="B37">2022</xref>).</p>
<p>In the third category, knowledge-based similarity (used in 22% of text similarity approaches in reviewed studies, as presented in <xref ref-type="table" rid="T1">Table 1</xref>) using the knowledge of ontologies represents the textual data on a semantic network graph consisting of nodes representing semantic memory and lines. Ontologies are the dictionaries of millions of words and are lexically associated, e.g., WordNet, Wikipedia, and DBpedia.</p>
<p><bold>Text classification</bold> is the second NLP approach used by 27% of the reviewed studies in automatic creativity evaluation depicted in <xref ref-type="fig" rid="F1">Figure 1</xref>. Classification is an ML technique that categorizes text into predefined categories. The classification consists of four main steps: (1) data collection, pre-processing (data acquisition, cleaning, and labeling), and data presentation (feature selection, dividing into training and testing datasets); (2) applying classifier models; (3) evaluation of classifiers; and (4) prediction (output of the testing data). These four steps are influential factors when applying text classification in automatic creativity evaluation. <xref ref-type="table" rid="T2">Table 2</xref> gives an overview of the classification approach, the datasets, classifiers, evaluations, and creativity dimensions in creativity evaluation research.</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Text classification-based creativity evaluation studies.</p></caption> 
<table frame="box" rules="all">
<thead>
<tr style="background-color:&#x00023;919498;color:&#x00023;ffffff">
<th valign="top" align="left"><bold>Datasets</bold></th>
<th valign="top" align="left"><bold>Classifiers</bold></th>
<th valign="top" align="left"><bold>Evaluation</bold></th>
<th valign="top" align="left"><bold>Dimensions</bold></th>
<th valign="top" align="left"><bold>Studies</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">In 4,099,877 solutions from Project Euler Website</td>
<td valign="top" align="left">Linear regression and SVM</td>
<td valign="top" align="left">Comparison with expert rating</td>
<td valign="top" align="left">Novelty, usefulness, quality</td>
<td valign="top" align="left">Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref></td>
</tr>
<tr>
<td valign="top" align="left">Two datasets were used: 1. ideas: 1,480; 2, domain dataset: a collection of 1,144 sports datasets from Wikipedia</td>
<td valign="top" align="left">SVM, neural networks (NN), logistic regression, decision trees, KNN, and Naive Bayes</td>
<td valign="top" align="left">F-measure is a measure of a test&#x00027;s accuracy. Precision and recall are calculated</td>
<td valign="top" align="left">Novelty</td>
<td valign="top" align="left">Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref></td>
</tr>
<tr>
<td valign="top" align="left">Semeval-2017 jokes, 4,030 short texts, and VU Amsterdam Metaphor Corpus</td>
<td valign="top" align="left">Bayesian approach</td>
<td valign="top" align="left">Bayesian approach is compared to the best-worst scaling method</td>
<td valign="top" align="left">Novelty, humor</td>
<td valign="top" align="left">Simpson et al., <xref ref-type="bibr" rid="B70">2019</xref></td>
</tr>
<tr>
<td valign="top" align="left">Internet movies database and Rotten Tomatoes dataset contained textual, image, and numerical attributes</td>
<td valign="top" align="left">SVM, random forest, ridge regression, Bayesian regression, and K-nearest regression</td>
<td valign="top" align="left">Correlation analysis</td>
<td valign="top" align="left">Novelty, value, influence, unexpectedness</td>
<td valign="top" align="left">Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref></td>
</tr>
<tr>
<td valign="top" align="left">User queries Wikipedia as knowledge source</td>
<td valign="top" align="left">Random trees</td>
<td valign="top" align="left">Sensitivity, Specificity</td>
<td valign="top" align="left">Diversity</td>
<td valign="top" align="left">Olivares-Rodr&#x000ED;guez et al., <xref ref-type="bibr" rid="B55">2017</xref></td>
</tr>
<tr>
<td valign="top" align="left">203 responses present in the multiplex lexical network</td>
<td valign="top" align="left">Logistic regression, random forest, and SVM classifiers</td>
<td valign="top" align="left">Entropy</td>
<td valign="top" align="left">Fluency</td>
<td valign="top" align="left">Stella and Kenett, <xref ref-type="bibr" rid="B72">2019</xref></td>
</tr>
<tr>
<td valign="top" align="left">1,214 recipes, 2,130 ingredients, and 235 cooking techniques</td>
<td valign="top" align="left">K-neighbor classifier, SVM, multi-layer perceptron classifier, and the random forest</td>
<td valign="top" align="left">The scoring function of classifiers, random forest, has the best results. No other evaluation</td>
<td valign="top" align="left">Novelty, adaptiveness, style, transcendence, realization</td>
<td valign="top" align="left">Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref></td>
</tr></tbody>
</table>
</table-wrap>
<p><bold>Text mining</bold> is the third approach in automatic creativity evaluation, which is the practice of analyzing a vast collection of textual data to capture key concepts, trends, patterns, and hidden relationships. In the scoping review, text mining is used (Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref>). The studies used four mining techniques, e.g., all words count, stop list inclusion (defined terms that are not meaningful), counting part of speech, and applying inverse document frequency (a technique to extract rare and important documents).</p>
</sec>
<sec>
<title>4.2. Creativity dimensions are computed automatically (RQ2)</title>
<p>In the studies included in this scoping review of automatic creativity evaluation, we differentiated 25 different creativity dimensions. These 25 dimensions of creativity are displayed in the second column (Manifestation) of <xref ref-type="table" rid="T3">Table 3</xref>. We analyze the similarities in the conceptual definition and computational approach employed in various studies that consider different dimensions for assessing creativity. This analysis allows us to categorize these 25 manifestations of creativity into seven core creativity dimensions, namely, novelty, value, flexibility, elaboration, fluency, feasibility, and others related to playful aspects of creativity such as humor or recreational efforts, which are displayed in the first column of <xref ref-type="table" rid="T3">Table 3</xref> (Core Dimension).</p>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Characterization of 25 creativity dimensions into seven core creativity dimensions (first column) and creativity dimensions manifested (second column) based on similarities in definitions (third column) and computation (fourth column).</p></caption> 
<table frame="box" rules="all">
<thead>
<tr style="background-color:&#x00023;919498;color:&#x00023;ffffff">
<th valign="top" align="left"><bold>Core dimension</bold></th>
<th valign="top" align="left"><bold>Dimension manifestation</bold></th>
<th valign="top" align="left"><bold>Dimension definition</bold></th>
<th valign="top" align="left"><bold>Dimension computation</bold></th>
<th valign="top" align="left"><bold>Study</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Novelty</td>
<td valign="top" align="left">Novelty</td>
<td valign="top" align="left">Novelty is an idea with respect to prior ideas or deviation from existing solutions</td>
<td valign="top" align="left">Textual similarity of a given solution to all existing or previous solutions</td>
<td valign="top" align="left">Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>; Prasch et al., <xref ref-type="bibr" rid="B60">2020</xref>; Kenworthy et al., <xref ref-type="bibr" rid="B41">2023</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">A measure of how unique a concept is relative to others</td>
<td valign="top" align="left">Span (Path length) is the sum of distances of each entity or unique words from the central entity or topic (e.g., predefined hierarchical topical categories of Wikipedia).</td>
<td valign="top" align="left">Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">The deviation from existing knowledge/experience</td>
<td valign="top" align="left">Average semantic distance between the dominant terms included in the textual representation of the story, compared to the average semantic distance of the dominant terms in all stories</td>
<td valign="top" align="left">Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">-</td>
<td valign="top" align="left">Pairwise text similarity using linguistic features</td>
<td valign="top" align="left">Simpson et al., <xref ref-type="bibr" rid="B70">2019</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Novelty is defined as a unique solution</td>
<td valign="top" align="left">From surprise and relevance score surprise term is computed from document term frequency in idea data, and relevance term is calculated from domain dataset (sports was collected from Wikipedia)</td>
<td valign="top" align="left">Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">How an artifact is different from others</td>
<td valign="top" align="left">Calculation of the distance between a given artifact and the other artifacts in a descriptive space</td>
<td valign="top" align="left">Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>; Franceschelli and Musolesi, <xref ref-type="bibr" rid="B25">2022</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Novelty to originality score and defines that the creative method led to more innovative products</td>
<td valign="top" align="left">The classifier models learn from ingredients and techniques and classify them as novel or not novel in the case study of culinary products</td>
<td valign="top" align="left">Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Originality</td>
<td valign="top" align="left">Similarity to existing ideas</td>
<td valign="top" align="left">Semantic distance between the responses</td>
<td valign="top" align="left">Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>; Song et al., <xref ref-type="bibr" rid="B71">2020</xref>; Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Originality is referred as a novelty</td>
<td valign="top" align="left">The semantic distance among ideas</td>
<td valign="top" align="left">Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Statistically infrequent responses</td>
<td valign="top" align="left">Semantic distance between the responses</td>
<td valign="top" align="left">Acar et al., <xref ref-type="bibr" rid="B1">2021</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">A response that is more unusual within a given context would be more Original.</td>
<td valign="top" align="left">Semantic distance between a given responses</td>
<td valign="top" align="left">Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Similarity</td>
<td valign="top" align="left">The similarity of meaning between multiple texts</td>
<td valign="top" align="left">The similarity of the new response was measured with topic clusters, rubrics, and example responses</td>
<td valign="top" align="left">LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">The similarity of the original poem to the translated poem</td>
<td valign="top" align="left">Similarity distance is calculated between original (English language) and translated poems (Spanish)</td>
<td valign="top" align="left">Zu&#x000F1;iga et al., <xref ref-type="bibr" rid="B86">2017</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">Similar contexts have smaller distances</td>
<td valign="top" align="left">Semantic distance between different words</td>
<td valign="top" align="left">Olson et al., <xref ref-type="bibr" rid="B56">2021</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Diversity</td>
<td valign="top" align="left">Semantic distance among user queries</td>
<td valign="top" align="left">Semantic similarity is estimated of each user-issued query to the k most relevant concepts for the challenge using distance formulas</td>
<td valign="top" align="left">Olivares-Rodr&#x000ED;guez et al., <xref ref-type="bibr" rid="B55">2017</xref></td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="left">The degree to which participants engaged in semantic context search</td>
<td valign="top" align="left">Semantic diversity refers to the degree to which the contexts surrounding words vary in their meanings</td>
<td valign="top" align="left">Johnson and Hass, <xref ref-type="bibr" rid="B37">2022</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Rarity</td>
<td valign="top" align="left">A rare combination of properties</td>
<td valign="top" align="left">The sum of weights on the min-weight closure of the cluster graph is compared to the maximum sum of weights in the story</td>
<td valign="top" align="left">Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Common use</td>
<td valign="top" align="left">Common uses of objects in Object use tasks</td>
<td valign="top" align="left">Each response was compared to the most common use of the corresponding object (collected previously from Common Use Judges)</td>
<td valign="top" align="left">Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Surprise or unexpectedness</td>
<td valign="top" align="left">Unexpectedness or surprise defines how different the artifact or some of its attributes are from expected behavior</td>
<td valign="top" align="left">The similarity of a given artifact with other artifacts</td>
<td valign="top" align="left">Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>; Franceschelli and Musolesi, <xref ref-type="bibr" rid="B25">2022</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Influence</td>
<td valign="top" align="left">How impactful or inspiring it has been</td>
<td valign="top" align="left">The similarity of an artifact with other artifacts occurs later</td>
<td valign="top" align="left">Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref></td>
</tr>
<tr>
<td valign="top" align="left">Value</td>
<td valign="top" align="left">Value</td>
<td valign="top" align="left">A measure of how artifact is valued by domain experts for artifact</td>
<td valign="top" align="left">Datapoint is highly valuable if its combination of correlated dimensions leads to a better rating prediction.</td>
<td valign="top" align="left">Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>; Franceschelli and Musolesi, <xref ref-type="bibr" rid="B25">2022</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Overall value</td>
<td valign="top" align="left">Overall value of the outcome of the designs from design ideation</td>
<td valign="top" align="left">Semantic analysis of verbalizations can be promising to measure the semantic value.</td>
<td valign="top" align="left">Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Quality</td>
<td valign="top" align="left">Quality is related to reliability, maintainability, extensibility, and adaptability.</td>
<td valign="top" align="left">Quality and Usefulness are computed from two metrics. Static Code Metrics: Line of codes Dynamic Code Metrics: number of visited lines</td>
<td valign="top" align="left">Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Usefulness</td>
<td valign="top" align="left">The correct solutions to programming tasks</td>
<td/>
<td valign="top" align="left">Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Adaptiveness and Style</td>
<td valign="top" align="left">Adaptiveness is the solution to solve a problem. Style is elegance and other aesthetic qualities</td>
<td valign="top" align="left">Adaptiveness as useful solutions style as quality</td>
<td valign="top" align="left">Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref></td>
</tr>
<tr>
<td valign="top" align="left">Elaboration</td>
<td valign="top" align="left">Elaboration</td>
<td valign="top" align="left">The degree to which they explain and embellish their responses</td>
<td valign="top" align="left">Counting based on: (1) Unweighted Word, (2) Stop listed Inclusion, (3) Part of Speech Inclusion, and (4) Inverse Frequency Weighting.</td>
<td valign="top" align="left">Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Level of details</td>
<td valign="top" align="left">Level of details of the ideas</td>
<td valign="top" align="left">Count of named entities. Examples of entities are person, place, things, money, etc</td>
<td valign="top" align="left">Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref></td>
</tr>
<tr>
<td valign="top" align="left">Flexibility</td>
<td valign="top" align="left">Flexibility</td>
<td valign="top" align="left">Semantic memory structure</td>
<td valign="top" align="left">1. Cosine similarity to estimate the edges between nodes in semantic network. 2. Number of similar clusters</td>
<td valign="top" align="left">Sung et al., <xref ref-type="bibr" rid="B73">2022</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Category switch</td>
<td valign="top" align="left">Number of changes in the category of use between responses</td>
<td valign="top" align="left">The similarity scores between successive response pairs were averaged for each object</td>
<td valign="top" align="left">Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Variety</td>
<td valign="top" align="left">Measure the variety of responses produced by each person</td>
<td valign="top" align="left">The similarity scores between every single pair of responses for an object were also averaged as a measure of the variety of responses produced by each person</td>
<td valign="top" align="left">Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref></td>
</tr>
<tr>
<td valign="top" align="left">Fluency</td>
<td valign="top" align="left">Fluency</td>
<td valign="top" align="left">Number of ideas</td>
<td valign="top" align="left">Counting the number of ideas</td>
<td valign="top" align="left">Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref>; Stella and Kenett, <xref ref-type="bibr" rid="B72">2019</xref>; Sung et al., <xref ref-type="bibr" rid="B73">2022</xref></td>
</tr>
<tr>
<td valign="top" align="left">Feasibility</td>
<td valign="top" align="left">Feasibility</td>
<td valign="top" align="left">Feasibility can be materialized or achieved in real practice</td>
<td valign="top" align="left">Polysemy, abstraction, and IC are highly correlated to the feasibility score</td>
<td valign="top" align="left">Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Transcendence and realism</td>
<td valign="top" align="left">Transforming into reality</td>
<td valign="top" align="left">Development of the product and its communication with the other products</td>
<td valign="top" align="left">Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref></td>
</tr>
<tr>
<td valign="top" align="left">Other</td>
<td valign="top" align="left">Humor</td>
<td valign="top" align="left">Humor is funniness</td>
<td valign="top" align="left">Pairwise comparison of text</td>
<td valign="top" align="left">Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref></td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Recreational effort</td>
<td valign="top" align="left">Difficult to achieve</td>
<td valign="top" align="left">The number of different clusters that each story contains as compared to the maximum number of clusters in a story of the whole group</td>
<td valign="top" align="left">Simpson et al., <xref ref-type="bibr" rid="B70">2019</xref></td>
</tr></tbody>
</table>
</table-wrap>
<p>Furthermore, the results obtained to answer research question two are illustrated in <xref ref-type="fig" rid="F4">Figure 4</xref>, which displays the percentage of the seven core creativity dimensions identified in this review. These results show that novelty is the most evaluated dimension in the studies compiled in this scoping review.</p>
<fig id="F4" position="float">
<label>Figure 4</label>
<caption><p>Percentage distribution of each core creativity dimension in the reviewed studies.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="feduc-08-1240962-g0004.tif"/>
</fig></sec>
</sec>
<sec id="s5">
<title>5. Discussion</title>
<sec>
<title>5.1. Approaches and techniques used in automatic creativity evaluation</title>
<p>The scoping review identified three main NLP approaches used in automatic creativity evaluation, namely, (1) text similarity, (2) text classification, and (3) text mining. In the next sections, we discuss the contribution of each computational approach to automatic creativity evaluation, argue their applications, discuss their limitations, identify research gaps, and make further recommendations for automatic creativity evaluations.</p>
<p><bold>Regarding the text similarity approach</bold>, the scoping review revealed that it is used in 69% of the studies, which helps understand creative thinking (Li et al., <xref ref-type="bibr" rid="B47">2023</xref>). Our analysis concluded that the widespread use of textual similarity in automatic creativity evaluation is because automatic creativity evaluation is more focused on evaluating originality, novelty, similarity, or diversity dimensions of creativity. The computations of these dimensions involve assessing the similarity of an idea with the existing ideas. The text similarity approach provides a variety of computational techniques to measure the similarity of ideas, as shown in <xref ref-type="fig" rid="F3">Figure 3</xref>.</p>
<p>Concerning the three categories of text similarity, namely, string similarity, corpus-based similarity, and knowledge-based similarity as set out in <xref ref-type="table" rid="T3">Table 3</xref>, the scoping review shows differences in the process of similarity computation that have an impact on how they are applied. On the one hand, string-based and knowledge-based similarities have limited application in automatic creativity evaluation because string-based only considers syntactic similarity (not semantic) and knowledge-based only extracts from text-specific entities, such as a person&#x00027;s name, place, and money (Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref>). During ideation, the knowledge-based approach might focus on entities rather than technical terms or scientific jargon within the sentence used by sentences solving a scientific challenge. For example, when brainstorming about renewable energy solutions, the knowledge-based approach might not capture specific terms such as &#x0201C;photovoltaics&#x0201D; or &#x0201C;wind turbines.&#x0201D; On the other hand, corpus-based techniques are widely used, so in the following, we elaborate on corpus-based techniques.</p>
<p>Regarding corpus-based similarity, it has been commonly used in automatic evaluation because it provides a wide range of computational techniques, from simple statistical to deep learning models, as shown in <xref ref-type="fig" rid="F2">Figure 2</xref>. Considering that a statistical model such as LSA is applied to examine semantic similarity, memory, and creativity (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>), it has shown a more reliable scoring technique of originality on divergent thinking tasks than human ratters (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>; Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref>; LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref>), as shown in <xref ref-type="table" rid="T1">Table 1</xref>. We argue that LSA uses statistical techniques, including Probabilistic Latent Semantic Analysis (Hofmann, <xref ref-type="bibr" rid="B34">1999</xref>), Latent Dirichlet Allocation (Blei et al., <xref ref-type="bibr" rid="B7">2003</xref>), and Non-Negative Matrix Factorization (Lee and Seung, <xref ref-type="bibr" rid="B46">1999</xref>), which limit its implication because these consider words statistics (e.g., co-occurrence of words) instead of word contextual and semantic meaning. These limitations are addressed by deep learning models, which we discuss below.</p>
<p>Recently, drastic changes in NLP research with the development of deep learning models based on deep neural architectures have unlocked ways to model text with more nuance and complexity. This advancement started with the development of word embedding models such as GloVe or Word2Vec pre-trained, including Wikipedia, news articles, and web pages. These predictive models use a neural network with one or more hidden layers to learn the vector representations of words. The GloVe showed comparable results to human experts&#x00027; scores in single-word creativity tasks (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>; Olson et al., <xref ref-type="bibr" rid="B56">2021</xref>). However, word embedding models do not differentiate between a list of keywords and a meaningful sentence; hence, they cannot capture the semantic and contextual meaning of the whole sentence (idea) in the vector space. The vectorization of the whole sentence is one major innovation in text modeling: The transformer architecture generally outperforms word embedding models on standard tasks, and often by large margins (Wang et al., <xref ref-type="bibr" rid="B82">2018</xref>, <xref ref-type="bibr" rid="B81">2019</xref>), which utilizes a concept called attention (Vaswani et al., <xref ref-type="bibr" rid="B78">2017</xref>). Attention makes it computationally tractable for a transformer model to consider a long sequence of text by selecting the most important parts of the sequence. Attention allows the training of large models on words and the complex contexts in which those words occur. This development resulted mainly in two kinds of categories, pre-trained sentence embedding models and text generation models which are discussed below.</p>
<p>Sentence embedding models vectorize the whole sentence into a vector space that keeps the semantic and contextual meaning of the entire sentence. The sentence embedding models are unsupervised techniques that do not require external data, e.g., Unsupervised Smooth Inverse Frequency (uSIF) (Ethayarajh, <xref ref-type="bibr" rid="B24">2018</xref>) and Geometric Sentence embedding (GEM) (Yang et al., <xref ref-type="bibr" rid="B84">2018</xref>). Some transformers allowed the tuning of parameters or training on their datasets to improve performance (if a large dataset is available), e.g., Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al., <xref ref-type="bibr" rid="B18">2018</xref>), Sentence Transformer (Reimers and Gurevych, <xref ref-type="bibr" rid="B63">2019</xref>), MPNet (Song et al., <xref ref-type="bibr" rid="B71">2020</xref>), Skip-Thought (ST) (Kiros et al., <xref ref-type="bibr" rid="B43">2015</xref>), InferSent (Conneau et al., <xref ref-type="bibr" rid="B15">2017</xref>), and Universal Sentence Encoder (USE) (Cer et al., <xref ref-type="bibr" rid="B13">2018</xref>). In creativity research, the USE model is used to evaluate the novelty of ideas (Kenworthy et al., <xref ref-type="bibr" rid="B41">2023</xref>). We argue that more exploration is needed to apply different, or combinations of sentence embedding models to evaluate creative ideas in an open-ended co-creation.</p>
<p>Text generation models generate new text that is similar to a given text prompt, such as Generative Pre-trained Transformer (GPT-3) (Brown et al., <xref ref-type="bibr" rid="B11">2020</xref>), Text-to-Text Transfer Transformer (T5) (Raffel et al., <xref ref-type="bibr" rid="B61">2020</xref>), and Long Short-Term Memory (LSTM) (Huang et al., <xref ref-type="bibr" rid="B35">2022</xref>). In creativity research, one of the text-generated models, the Generative Adversarial Network (GAN) (Aggarwal et al., <xref ref-type="bibr" rid="B3">2021</xref>), is used by Franceschelli and Musolesi (<xref ref-type="bibr" rid="B25">2022</xref>) to evaluate novelty, surprise, and relevance. We present two criticisms regarding using text generation models for evaluating open-ended ideas. First, text generation is specialized to generate text from a given text that could be useful for dialog generation, machine translation, chatbots, and prompt-based learning (Liu et al., <xref ref-type="bibr" rid="B48">2023</xref>). Second, as the model becomes better at generating text with an improved understanding of language, it is more likely to generate text that closely resembles the input data rather than producing more novel or creative outputs. However, we argue that text generation models are not tested on a larger scale in creativity research, so future investigations could help understand these limits.</p>
<p>Finally, two conclusions are drawn from the above discussion. First, for single-word tasks in creativity research, word embedding models can be used, especially the GloVe embedding model, which is widely used. Word embedding models represent words in a high-dimensional vector space, enabling the computation of their contextual and semantic similarity with other words. Second, for open-ended co-creation resulting in ideas of sentence structure, sentence embedding models can be useful in three ways: (a) In open-ended ideation, mostly the ideas are in sentence structure, so these sentence models present the whole sentence in a vector space, capturing the semantic and contextual meaning of the whole sentence; (b) sentence embedding models outperform the word embedding models for textual similarity tasks; and (c) sentence embedding models can also be applied to small datasets and open-ended problems because these models are pre-trained over large corpora. Finally, we recommend not only validating sentence embedding models but also applying text generation models within a broader context of co-creation.</p>
<p>We concluded that sentence embedding models offer a powerful measure that can be used alongside statistical (Acar et al., <xref ref-type="bibr" rid="B1">2021</xref>), word embedding models (Organisciak et al., <xref ref-type="bibr" rid="B57">2023</xref>), and standard subjective scoring methods of the creative process and its output (Kenett, <xref ref-type="bibr" rid="B40">2019</xref>).</p>
<p><bold>Text classification approach</bold> refers to the automated categorization or labeling of textual data into predetermined classes or categories using machine learning classifiers. A large dataset is used for text classification, which is divided into training and testing (the usual ratio is 70% training and 30% testing datasets). An ML classifier learns from the training dataset and then uses the knowledge learned during training to categorize the testing dataset. Therefore, integrating text classification into automatic creativity evaluation depends on four key factors: the dataset, the selection of appropriate machine learning classifiers, the accuracy of the ML classifier, and the creativity dimensions being evaluated. These factors in the reviewed studies using the text classification approach are highlighted in <xref ref-type="table" rid="T2">Table 2</xref>.</p>
<p>Using text classification, it is essential to consider the dataset factor for three reasons: First, the datasets used for classification need pre-processing and labeling. Pre-processing includes removing noisy or irrelevant information, and labeling includes giving a class label to each idea. Second, a large dataset is required to train the ML classifiers. The prediction capability of ML classifiers increases with an increase in the amount of data used for training. All studies reviewed in <xref ref-type="table" rid="T2">Table 2</xref> except Stella and Kenett (<xref ref-type="bibr" rid="B72">2019</xref>) use more than a thousand ideas or solutions for the classification problem. A smaller dataset may need better or more balanced results. Third, ML classifiers trained on one type of data cannot be applied to another kind of data. For example, classifiers trained on datasets from the linguistic domain cannot be used to test data from the scientific domain.</p>
<p>Furthermore, classifier selection and accuracy are also critical. Regarding classifier selection, the working methods of ML classifiers are different and dependent on the nature of the dataset, e.g., SVM works well for multiclass classification, and random forest excels in scenarios involving numerical and categorical features. Similarly, logistic regression works on linear problems; the K-neighbor classifier is best for text, and SVM can also work for multiclass dataset classification. The Bayesian approach is a simple and fast algorithm. The reviewed studies lack arguments for using a specific classifier in their studies. Regarding the accuracy of ML, there is a risk of not getting high accuracy. Different automatic evaluators are used to evaluate model accuracy, such as confusion matrix, entropy, and sensitivity, as shown in <xref ref-type="table" rid="T2">Table 2</xref>. It is suggested to apply several classifiers, and one with high accuracy can be used for prediction in a similar domain.</p>
<p>Finally, the text classification approach can be applied to evaluate different dimensions of creativity; however, it requires a large, labeled dataset, which limits its application in creativity research. We also argue that the dataset&#x00027;s preparation and labeling might be expensive, which mitigates the advantages of automatic evaluation over manual creativity evaluation, e.g., accuracy, cost, and time. Furthermore, the text classification problems are domain-dependent. So, for creativity tasks, such as object use tasks and alternate use tasks, some public datasets are available that could apply to similar tasks. However, it is not useful for small and open-ended creative tasks because it is not enough to train an ML classifier and is domain-independent. In short, large dataset preparation, labeling, and domain dependence make the text classification approach less reliable and expensive than manual creativity evaluation.</p>
<p><bold>Text mining</bold> employs NLP statistical computation to discover new information and patterns. It uses statistical indicators such as the frequency of words, word patterns, and correlation within words. Dumas et al. (<xref ref-type="bibr" rid="B22">2021</xref>) implemented four text-mining techniques and measured the elaboration score in Alternate Use Tasks (AUT). Elaboration was computed in four different ways: (1) unweighted word count method: count the number of words; (2) stop listed inclusion: a preliminary agreed list of stop words; (3) parts of speech include verbs, nouns, adjectives, and adverbs; and (4) inverse frequency weighting: commonness of a word in an initial corpus of text.</p>
<p>The above text-mining techniques are the basic statistical operations in NLP. Text mining holds the potential to handle a massive amount of data to discover new information, patterns, trends, relationships, etc., that could be useful in creativity research. Text-mining applications include search engines, product suggestion analysis, social media analytics, and trend analysis.</p>
</sec>
<sec>
<title>5.2. Automatically computed creativity dimensions</title>
<p>The scoping review noted 25 creativity dimensions computed automatically. However, our analysis reveals that these creativity dimensions are not sufficiently based on previous creativity research and theory. Therefore, we have found some theoretical and methodological inconsistencies that should be tackled in future research. In this line of argument, first, we highlight that some of the creativity dimensions studied in the scoping review are defined and computed, building links with the challenges or the creativity tasks designed for the experiment but not with a strong theoretical framework. For example, a category switch is defined as the similarity difference between two successive responses in object use tasks (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>). Another example is the creativity dimensions of quality (reusability) and usefulness (Degree of completion) that are defined and computed in the context of programming problems (Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>). Second, another reason for the inconsistency among the dimensions of creativity is the variation in manifestations employed across the reviewed articles. Specifically, it has been observed that dimensions such as novelty (Prasch et al., <xref ref-type="bibr" rid="B60">2020</xref>), similarity (LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref>), and originality (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>) are defined in a similar manner, a strong focus on the similarity between ideas or solutions. Moreover, these dimensions are often measured using semantic textual similarity, although different computational techniques are performed.</p>
<p>To mitigate these shortcomings, this scoping review has thoroughly analyzed the conceptual and computational framework used in each study and contributed to the emergence of seven core creativity dimensions that could be automatically evaluated and bring more consistency to this research area. These seven core creativity dimensions are novelty, elaboration, flexibility, value, feasibility, fluency, and others related to playful aspects of creativity, such as humor and recreational efforts. Following, we discuss each core creativity dimension identified and highlight the key aspects of its conceptual definition and computational approach.</p>
<p><bold>Novelty is the first core dimension</bold> in automatic creativity research that is most evaluated in 59% of the reviewed studies. Despite this high interest, our revision indicates multifariousness in defining and measuring novelty. As a consequence of that, the reviewed studies refer to novelty using the following different words or manifestations, namely, (1) uniqueness: the uniqueness of a concept related to the other concepts (Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref>); (2) originality: how different the outcome is from standard/other solutions (Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref>) or semantic distance among ideas (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>); (3) similarity: the similarity of meaning between multiple texts (LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref>) or similarity distance between the texts (Olson et al., <xref ref-type="bibr" rid="B56">2021</xref>); (4) diversity: the diversity of users&#x00027; entered queries; (5) rarity: the rare combination or rare ideas (Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref>) or unique solution (Doboli et al., <xref ref-type="bibr" rid="B20">2020</xref>); (6) common use: the difference between common and uncommon solutions; (7) surprise: that how much an artifact is different from existing attributes (Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>); and (8) influence or the comparison of an artifact with other artifacts (Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>).</p>
<p>Nonetheless, the diversity in labeling and defining the novelty dimension, our analysis identified the next six characteristics that could be included in defining novelty and assisting its automatic evaluation: (1) deviation from the standard, routine way of solving a given problem (Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>); (2) semantic distance between ideas (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>); (3) similarity of meaning between multiple texts (LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref>); (4) Semantic similarity of the user query to the concepts in the challenge; (5) combination of properties (Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref>); and (6) surprise and unexpected ideas (Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>). These six characteristics involved in the definition of novelty in the studies reviewed give an account of the complexity of defining the novelty dimension and acknowledge the challenges in developing automatic measures for novelty.</p>
<p>Despite these challenges, the scoping review has highlighted some common computing approaches and techniques to measure novelty as a core dimension and they can be synthesized in the next five characteristics: (1) distance of the new solution to the existing solution (Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>); (2) semantic distance among ideas (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>; Olson et al., <xref ref-type="bibr" rid="B56">2021</xref>); (3) semantic similarity of user queries and relevant concepts in Wikipedia; (4) semantic distance between the clusters in a story; and (5) semantic distance between the consecutive fragments of the story (Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref>). It concludes that when developing an automatic evaluation of novelty, the semantic distance of a solution to existing solutions should be considered.</p>
<p><bold>Value is the second core dimension</bold> identified in automatic creativity evaluation. The scoping review identified the next four concepts related to value (Shrivastava et al., <xref ref-type="bibr" rid="B69">2017</xref>; Franceschelli and Musolesi, <xref ref-type="bibr" rid="B25">2022</xref>): (1) overall value, which relates how an artifact is perceived by society (Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref>); (2) quality, this concept is mainly used for programming solutions when they embody specific attributes such as reliability, characterized by error-free operation; maintainability, denoting ease of maintenance; extensibility, encompassing scalability and simplified modification; and, adaptability, reflecting the flexibility to integrate new technologies seamlessly (Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>); (3) the concept of usefulness which is linked to the notion of correctness; and (4) the concept of adaptiveness, it pertains to useful solutions that effectively address specific problems (Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref>). In sum, these four concepts share a common meaning of usefulness and quality that could be considered the value dimension of creativity. Furthermore, from a computer science perspective, value, quality, usefulness, adaptiveness, and style are the non-functional characteristics related to quality attributes. These quality attributes have different computations depending on the nature of the task, e.g., quality and useful programming solutions are the reusability and scalability of computer programs (Manske and Hoppe, <xref ref-type="bibr" rid="B49">2014</xref>), and usefulness is the degree of completing the task (Prasch et al., <xref ref-type="bibr" rid="B60">2020</xref>). Therefore, the value dimension needs clear definitions and computation metrics like other dimensions.</p>
<p><bold>The third core dimension</bold> used in automatic creativity evaluation is flexibility. Flexibility refers to one of the key executive functions of creative thinking (Boot et al., <xref ref-type="bibr" rid="B8">2017</xref>), which drives individuals to follow diverse directions, dimensions, and pathways (Acar et al., <xref ref-type="bibr" rid="B1">2021</xref>), more likely to produce highly creative ideas (Zhang et al., <xref ref-type="bibr" rid="B85">2020</xref>). Creativity research defines flexibility in two distinct ways. First, it involves category switching (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>; Acar et al., <xref ref-type="bibr" rid="B2">2019</xref>; Mastria et al., <xref ref-type="bibr" rid="B52">2021</xref>), which refers to the ability to transition from one semantic concept to another. Second, flexibility is also measured by the number of semantic categories, varieties (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>), or topics generated during the creative process. Owing to variations in the definition of flexibility across creativity research, different computational approaches are employed to compute this dimension. On one side, flexibility as a category switch is a measure of the similarity of one idea to all existing ideas. Therefore, semantic similarity approaches are used to evaluate flexibility, such as LSA (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>), network graphs (Cosgrove et al., <xref ref-type="bibr" rid="B16">2021</xref>), and sentence embedding models. On the other side, flexibility identifies semantic categories, varieties, or topics that can be evaluated using text clustering (Sung et al., <xref ref-type="bibr" rid="B73">2022</xref>) or topic modeling techniques [e.g., Latent Dirichlet Allocation (LDA); Chauhan and Shah, <xref ref-type="bibr" rid="B14">2021</xref>] to categorize or extract different topics from the textual ideas. We argue that flexibility as a category switch could be the easiest way to compute because it acquires simple text similarities rather than identifying categories in the text, which involve more variables and algorithms.</p>
<p><bold>Regarding elaboration</bold> as a core creativity dimension in automatic creativity evaluation, it is defined as the degree of elaboration to which the participants embellish their responses (Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref>; Dumas et al., <xref ref-type="bibr" rid="B22">2021</xref>) or which gives further details on adding reasoning or cause to an idea. Automatic creativity evaluation captures the level of detail of an idea by counting the number of words used in the idea (Camburn et al., <xref ref-type="bibr" rid="B12">2019</xref>). The scoping review has identified four different methods for evaluating the level of idea elaboration: (1) counting all words in an idea (Counting unweighted measures); (2) counting stop words (words that do not have semantic meanings); (3) counting nouns, verbs, and adverbs; and (4) specifying and counting adjectives (parts of speech inclusion) and uncommon words with high weight (inverse frequency weighting). An idea with more words is considered an elaborated idea. We argue that the above-adopted computation of elaboration may not capture conjunctions (Tuzcu, <xref ref-type="bibr" rid="B76">2021</xref>) or reasoning words (Sedova et al., <xref ref-type="bibr" rid="B68">2019</xref>; Hennessy et al., <xref ref-type="bibr" rid="B33">2020</xref>), adding more explanation to the ideas. Therefore, we suggest the semantic search to specify the words that cause reasoning or words that give reason to the idea, such as because, therefore, and since.</p>
<p><bold>Fluency</bold> is defined as the number of ideas generated during an ideation process. This scoping review showed that fluency is one of the core dimensions that finds consensus on its conceptual definition (number of ideas) and computational approach (counting ideas) (Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref>; Stella and Kenett, <xref ref-type="bibr" rid="B72">2019</xref>). Creativity research claims that when there are more ideas, there is a greater chance of producing original ideas or products (Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref>). Fluency measurement is easy to implement and is independent of other ideas such as elaboration. Compared to novelty and flexibility, which require comparison with different ideas, fluency can be easily computed for each idea.</p>
<p><bold>Feasibility</bold> is defined as the solution that is achievable in real practice (Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref>). The scoping review found transcendence and realization have been used as manifestations of feasibility as they refer to the achievement in real practice or transforming into reality (Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref>). These dimensions share the same characteristic of transforming an idea or solution into real practice, which is significant in creativity research. The creativity research highlights the significance of putting ideas into practice; however, the automatic computation of feasibility (Georgiev and Casakin, <xref ref-type="bibr" rid="B27">2019</xref>), transcendence, and realization (Jimenez-Mavillard and Suarez, <xref ref-type="bibr" rid="B36">2022</xref>) does not provide any rationale from the creativity research. Feasibility is mostly a product-oriented dimension and is mostly used in the ideation process, but finding transformable ideas into real practice is still a challenge to address. Therefore, it is a dimension that needs further research to automatically measure feasible, transcendent, and realistic ideas.</p>
<p><bold>Finally, other dimensions</bold> associated with the playful aspects of creativity, such as humor (Simpson et al., <xref ref-type="bibr" rid="B70">2019</xref>) and recreational effort (Karampiperis et al., <xref ref-type="bibr" rid="B39">2014</xref>), were identified in the reviewed articles. Humor, representing the funniness of ideas, is typically measured through pairwise text comparison techniques. At the same time, recreational effort is defined as a solution that is difficult to achieve and is measured using clustering methods. These dimensions contribute to the playful nature of creativity, so it is essential to establish clear definitions and develop suitable computational approaches from both psychological and computer science perspectives.</p>
</sec>
</sec>
<sec id="s6">
<title>6. Conclusion</title>
<p>This article has the objective of conducting a scoping review of automatic creativity evaluation from creativity and computer science perspectives. To meet this objective, we defined two research questions: The first identifies the NLP approaches and techniques used in automatic creativity, and the second analyzes which and how different creativity dimensions are computed.</p>
<p>The first research question&#x00027;s contributions are multi-fold: (1) identifying the existing ML approaches and techniques in automatic creativity evaluation; (2) categorizing the approaches into different groups for deep compilation, e.g., text similarity, text classification, and text mining. Among these, text similarity is commonly used; (3) classifying creativity evaluation studies into different techniques accordingly, e.g., classifying studies in text similarity approaches using various techniques such as string similarity, corpus-based similarity, and knowledge-based similarity. Our results showed that corpus-based methods are widely used for automatic creativity evaluation. Corpus-based techniques, LSA (Dunbar and Forster, <xref ref-type="bibr" rid="B23">2009</xref>; Dumas and Dunbar, <xref ref-type="bibr" rid="B21">2014</xref>; LaVoie et al., <xref ref-type="bibr" rid="B45">2020</xref>) and GloVe algorithm (Beaty and Johnson, <xref ref-type="bibr" rid="B5">2021</xref>; Olson et al., <xref ref-type="bibr" rid="B56">2021</xref>), have shown a positive correlation with human experts&#x00027; similarity scores; (4) identifying the limitations of the critical challenge and identifying alternative techniques, for example, statistical and word embedding techniques are generally used, but they cannot capture the semantic and contextual meaning of a whole sentence; and (5) providing a broad overview of all existing automatic creativity to give a deeper understanding of all the approaches. We concluded that word embedding models, especially GloVe, work better for single-word tasks, and for open-ended ideas in sentence structure, sentence embedding models could provide promising results.</p>
<p>The second research question&#x00027;s contributions are also multi-fold: first, we have examined what creativity dimensions are automatically evaluated in the different articles analyzed in this scoping review. In contrast to creativity research, which has standardized tests that evaluate four specific dimensions, 25 different creativity dimensions are found in automatic creativity evaluation. Second, the scoping review has analyzed how these dimensions are defined and measured in automatic creativity evaluation. We found similarities in the definitions and computations of different creativity dimensions. Finally, based on a thorough analysis of the definitions and computations used in the studies, we characterized the 25 dimensions into seven core dimensions. This analysis helps elaborate a coherent and consistent framework about core creativity dimensions and their computation.</p>
<p>The overall contributions of this scoping review bridge the realms of computer science and education. For computer scientists, this review provides insights to refine existing NLP approaches and provides opportunities for developing more novel NLP methods for evaluating and promoting creativity. Meanwhile, educators can use these automatic evaluations as pedagogical tools in real-world classroom practices. The implications of automatic creativity evaluation could help assess and nurture creativity, which is becoming an explicit part of educational policy initiatives and curricula. Ultimately, this scoping review leverages AI as a valuable tool in evaluating and enhancing creativity capable of equipping future citizens with the necessary competencies to generate innovative solutions to the world&#x00027;s complex economic, environmental, and social challenges.</p>
<sec>
<title>6.1. Limitations and future work</title>
<p>This scoping review has two limitations, which may have conditioned our results. The first limitation could be the search keyword strategy, which may be insufficient to include key articles in our field of study. Second, the exclusion and inclusion criteria may suffer from the omission of relevant studies that could have answered our research questions. We tried to mitigate this risk by carefully constructing an inclusive search string and providing explicit inclusion and exclusion criteria with co-authors&#x00027; consensus.</p>
<p>In future, concluding from this scoping review, we intend to design experimental research to evaluate the reliability of deep learning models such as sentence embedding models to measure the novelty of ideas in an open-ended co-creative process. Furthermore, we also suggest using text generation models to recommend diverse hints to improve divergent thinking in the creative process. Regarding the automatic evaluation of creativity dimensions, our review highlighted that there is still a research gap in studies that fully automate the main core dimensions of creativity. So, we plan to simultaneously measure different core creativity dimensions by evaluating idea datasets with ML techniques. Finally, the development of reliable and automatic evaluation of the different dimensions of creativity could be the seed for the design and the delivery of real-time recommendations during the creative process that could trigger students&#x00027; creativity.</p>
</sec>
</sec>
<sec sec-type="data-availability" id="s7">
<title>Data availability statement</title>
<p>The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.</p>
</sec>
<sec sec-type="author-contributions" id="s8">
<title>Author contributions</title>
<p>IU has contributed in the conceptualization of the paper, methodology and investigation; he has participated in writing the original manuscript, revision and edition. MP is the principal investigator of the research project and she has designed the project, she has also contributed in the conceptualization of the paper, methodology and investigation; she has participated in writing the manuscript, revision and edition. Both authors contributed to the article and approved the submitted version.</p>
</sec>
</body>
<back>
<sec sec-type="funding-information" id="s9">
<title>Funding</title>
<p>This research has been funded by the Ministry of Science and Innovation of the Government of Spain under Grants EDU2019-107399RB-I00 and PID2022-139060OB-I00.</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s10">
<title>Publisher&#x00027;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Acar</surname> <given-names>S.</given-names></name> <name><surname>Berthiaume</surname> <given-names>K.</given-names></name> <name><surname>Grajzel</surname> <given-names>K.</given-names></name> <name><surname>Dumas</surname> <given-names>D.</given-names></name> <name><surname>Flemister</surname> <given-names>C.</given-names></name> <name><surname>Organisciak</surname> <given-names>P.</given-names></name></person-group> (<year>2021</year>). <article-title>Applying automated originality scoring to the verbal form of torrance tests of creative thinking</article-title>. <source>Gifted Child Quart.</source> <volume>67</volume>, <fpage>3</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.1177/00169862211061874</pub-id></citation>
</ref>
<ref id="B2">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Acar</surname> <given-names>S.</given-names></name> <name><surname>Runco</surname> <given-names>M. A.</given-names></name> <name><surname>Ogurlu</surname> <given-names>U.</given-names></name></person-group> (<year>2019</year>). <article-title>The moderating influence of idea sequence: A re-analysis of the relationship between category switch and latency</article-title>. <source>Person. Indiv. Differ.</source> <volume>142</volume>, <fpage>214</fpage>&#x02013;<lpage>217</lpage>. <pub-id pub-id-type="doi">10.1016/j.paid.2018.06.013</pub-id></citation>
</ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aggarwal</surname> <given-names>A.</given-names></name> <name><surname>Mittal</surname> <given-names>M.</given-names></name> <name><surname>Battineni</surname> <given-names>G.</given-names></name></person-group> (<year>2021</year>). <article-title>Generative adversarial network: An overview ofvtheory and applications</article-title>. <source>Int. J. Inform. Manage. Data Insights</source> <volume>1</volume>, <fpage>100004</fpage>. <pub-id pub-id-type="doi">10.1016/j.jjimei.2020.100004</pub-id></citation>
</ref>
<ref id="B4">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bae</surname> <given-names>S. S.</given-names></name> <name><surname>Kwon</surname> <given-names>O.-H.</given-names></name> <name><surname>Chandrasegaran</surname> <given-names>S.</given-names></name> <name><surname>Ma</surname> <given-names>K.-L.</given-names></name></person-group> (<year>2020</year>). <article-title>&#x0201C;Spinneret: aiding creative ideationvthrough non-obvious concept associations,&#x0201D;</article-title> in <source>Proceedings of the 2020 CHI Conference on HumanvFactors in Computing Systems</source> <fpage>1</fpage>&#x02013;<lpage>13</lpage>. <pub-id pub-id-type="doi">10.1145/3313831.3376746</pub-id></citation>
</ref>
<ref id="B5">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Beaty</surname> <given-names>R. E.</given-names></name> <name><surname>Johnson</surname> <given-names>D. R.</given-names></name></person-group> (<year>2021</year>). <article-title>Automating creativity assessment with semdis: An open platformvfor computing semantic distance</article-title>. <source>Behav. Res. Methods</source> <volume>53</volume>, <fpage>757</fpage>&#x02013;<lpage>780</lpage>. <pub-id pub-id-type="doi">10.3758/s13428-020-01453-w</pub-id><pub-id pub-id-type="pmid">32869137</pub-id></citation></ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Birkey</surname> <given-names>R.</given-names></name> <name><surname>Hausserman</surname> <given-names>C.</given-names></name></person-group> (<year>2019</year>). <article-title>&#x0201C;Inducing creativity in accountants&#x00027; task performance: The effects of background, environment, and feedback,&#x0201D;</article-title> in <source>Advances in Accounting Education: Teaching and Curriculum Innovations</source> (Emerald Publishing Limited) <fpage>109</fpage>&#x02013;<lpage>133</lpage>. <pub-id pub-id-type="doi">10.1108/S1085-462220190000022006</pub-id></citation>
</ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Blei</surname> <given-names>D. M.</given-names></name> <name><surname>Ng</surname> <given-names>A. Y.</given-names></name> <name><surname>Jordan</surname> <given-names>M. I.</given-names></name></person-group> (<year>2003</year>). <article-title>Latent dirichlet allocation</article-title>. <source>J. Mach. Learn. Res.</source> <volume>3</volume>, <fpage>993</fpage>&#x02013;<lpage>1022</lpage>. <pub-id pub-id-type="doi">10.5555/944919.944937</pub-id></citation>
</ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Boot</surname> <given-names>N.</given-names></name> <name><surname>Baas</surname> <given-names>M.</given-names></name> <name><surname>M&#x000FC;hlfeld</surname> <given-names>E.</given-names></name> <name><surname>de Dreu</surname> <given-names>C. K.</given-names></name> <name><surname>van Gaal</surname> <given-names>S.</given-names></name></person-group> (<year>2017</year>). <article-title>Widespread neural oscillations in the delta band dissociate rule convergence from rule divergence during creative idea generation</article-title>. <source>Neuropsychologia</source> <volume>104</volume>, <fpage>8</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuropsychologia.2017.07.033</pub-id><pub-id pub-id-type="pmid">28774832</pub-id></citation></ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bozkurt Altan</surname> <given-names>E.</given-names></name> <name><surname>Tan</surname> <given-names>S.</given-names></name></person-group> (<year>2021</year>). <article-title>Concepts of creativity in design-based learning in STEM education</article-title>. <source>Int. J. Technol. Design Educ.</source> <volume>31</volume>, <fpage>503</fpage>&#x02013;<lpage>529</lpage>. <pub-id pub-id-type="doi">10.1007/s10798-020-09569-y</pub-id></citation>
</ref>
<ref id="B10">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Braun</surname> <given-names>D.</given-names></name> <name><surname>Hernandez Mendez</surname> <given-names>A.</given-names></name> <name><surname>Matthes</surname> <given-names>F.</given-names></name> <name><surname>Langen</surname> <given-names>M.</given-names></name></person-group> (<year>2017</year>). <article-title>&#x0201C;Evaluating natural language understanding services for conversational question answering systems,&#x0201D;</article-title> in <source>Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue</source> (<publisher-loc>Saarbrucken, Germany</publisher-loc>: <publisher-name>Association for Computational Linguistics</publisher-name>) <fpage>174</fpage>&#x02013;<lpage>185</lpage>. <pub-id pub-id-type="doi">10.18653/v1/W17-5522</pub-id></citation>
</ref>
<ref id="B11">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brown</surname> <given-names>T.</given-names></name> <name><surname>Mann</surname> <given-names>B.</given-names></name> <name><surname>Ryder</surname> <given-names>N.</given-names></name> <name><surname>Subbiah</surname> <given-names>M.</given-names></name> <name><surname>Kaplan</surname> <given-names>J. D.</given-names></name> <name><surname>Dhariwal</surname> <given-names>P.</given-names></name> <etal/></person-group>. (<year>2020</year>). <article-title>Language models are few-shot learners</article-title>. <source>Adv. Neural Inf. Proc. Syst.</source> <volume>33</volume>, <fpage>1877</fpage>&#x02013;<lpage>1901</lpage>. <pub-id pub-id-type="doi">10.48550/arXiv.2005.14165</pub-id></citation>
</ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Camburn</surname> <given-names>B.</given-names></name> <name><surname>He</surname> <given-names>Y.</given-names></name> <name><surname>Raviselvam</surname> <given-names>S.</given-names></name> <name><surname>Luo</surname> <given-names>J.</given-names></name> <name><surname>Wood</surname> <given-names>K.</given-names></name></person-group> (<year>2019</year>). <article-title>&#x0201C;Evaluating crowdsourced design concepts with machine learning,&#x0201D;</article-title> in <source>International Design Engineering Technical Conferences and Computers and Information in Engineering Conference</source> 7. <pub-id pub-id-type="doi">10.1115/DETC2019-97285</pub-id></citation>
</ref>
<ref id="B13">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cer</surname> <given-names>D.</given-names></name> <name><surname>Yang</surname> <given-names>Y.</given-names></name> <name><surname>Kong</surname> <given-names>S.-,y.</given-names></name> <name><surname>Hua</surname> <given-names>N.</given-names></name> <name><surname>Limtiaco</surname> <given-names>N.</given-names></name> <name><surname>John</surname> <given-names>R. S.</given-names></name> <etal/></person-group>. (<year>2018</year>). <article-title>Universal sentence encoder</article-title>. <source>arXiv preprint arXiv:1803.11175</source>.</citation>
</ref>
<ref id="B14">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chauhan</surname> <given-names>U.</given-names></name> <name><surname>Shah</surname> <given-names>A.</given-names></name></person-group> (<year>2021</year>). <article-title>Topic modeling using latent dirichlet allocation: A survey</article-title>. <source>ACM Comput. Surv.</source> <volume>54</volume>, <fpage>1</fpage>&#x02013;<lpage>35</lpage>. <pub-id pub-id-type="doi">10.1145/3462478</pub-id></citation>
</ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Conneau</surname> <given-names>A.</given-names></name> <name><surname>Kiela</surname> <given-names>D.</given-names></name> <name><surname>Schwenk</surname> <given-names>H.</given-names></name> <name><surname>Barrault</surname> <given-names>L.</given-names></name> <name><surname>Bordes</surname> <given-names>A.</given-names></name></person-group> (<year>2017</year>). <article-title>Supervised learning of universal sentence representations from natural language inference data</article-title>. <source>arXiv preprint arXiv:1705.02364</source>.</citation>
</ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cosgrove</surname> <given-names>A. L.</given-names></name> <name><surname>Kenett</surname> <given-names>Y. N.</given-names></name> <name><surname>Beaty</surname> <given-names>R. E.</given-names></name> <name><surname>Diaz</surname> <given-names>M. T.</given-names></name></person-group> (<year>2021</year>). <article-title>Quantifying flexibility in thought: The resiliency of semantic networks differs across the lifespan</article-title>. <source>Cognition</source> <volume>211</volume>, <fpage>104631</fpage>. <pub-id pub-id-type="doi">10.1016/j.cognition.2021.104631</pub-id><pub-id pub-id-type="pmid">33639378</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>De Stobbeleir</surname> <given-names>K. E.</given-names></name> <name><surname>Ashford</surname> <given-names>S. J.</given-names></name> <name><surname>Buyens</surname> <given-names>D.</given-names></name></person-group> (<year>2011</year>). <article-title>Self-regulation of creativity at work: The role of feedback-seeking behavior in creative performance</article-title>. <source>Acad. Manage. J.</source> <volume>54</volume>, <fpage>811</fpage>&#x02013;<lpage>831</lpage>. <pub-id pub-id-type="doi">10.5465/amj.2011.64870144</pub-id></citation>
</ref>
<ref id="B18">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Devlin</surname> <given-names>J.</given-names></name> <name><surname>Chang</surname> <given-names>M.-W.</given-names></name> <name><surname>Lee</surname> <given-names>K.</given-names></name> <name><surname>Toutanova</surname> <given-names>K.</given-names></name></person-group> (<year>2018</year>). <article-title>Bert: Pre-training of deep bidirectional transformers for language understanding</article-title>. <source>arXiv preprint arXiv:1810.04805</source>.</citation>
</ref>
<ref id="B19">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dickson</surname> <given-names>K.</given-names></name> <name><surname>Yeung</surname> <given-names>C. A.</given-names></name></person-group> (<year>2022</year>). <article-title>PRISMA 2020 updated guideline</article-title>. <source>Br. Dental J.</source> <volume>232</volume>, <fpage>760</fpage>&#x02013;<lpage>761</lpage>. <pub-id pub-id-type="doi">10.1038/s41415-022-4359-7</pub-id><pub-id pub-id-type="pmid">35689040</pub-id></citation></ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Doboli</surname> <given-names>S.</given-names></name> <name><surname>Kenworthy</surname> <given-names>J.</given-names></name> <name><surname>Paulus</surname> <given-names>P.</given-names></name> <name><surname>Minai</surname> <given-names>A.</given-names></name> <name><surname>Doboli</surname> <given-names>A.</given-names></name></person-group> (<year>2020</year>). <article-title>&#x0201C;A cognitive inspired method for assessing novelty of short-text ideas,&#x0201D;</article-title> in <source>2020 International Joint Conference on Neural Networks (IJCNN)</source> (IEEE), <fpage>1</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1109/IJCNN48605.2020.9206788</pub-id></citation>
</ref>
<ref id="B21">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dumas</surname> <given-names>D.</given-names></name> <name><surname>Dunbar</surname> <given-names>K. N.</given-names></name></person-group> (<year>2014</year>). <article-title>Understanding fluency and originality: A latent variable perspective</article-title>. <source>Think. Skills Creat.</source> <volume>14</volume>, <fpage>56</fpage>&#x02013;<lpage>67</lpage>. <pub-id pub-id-type="doi">10.1016/j.tsc.2014.09.003</pub-id></citation>
</ref>
<ref id="B22">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dumas</surname> <given-names>D.</given-names></name> <name><surname>Organisciak</surname> <given-names>P.</given-names></name> <name><surname>Maio</surname> <given-names>S.</given-names></name> <name><surname>Doherty</surname> <given-names>M.</given-names></name></person-group> (<year>2021</year>). <article-title>Four text-mining methods for measuring elaboration</article-title>. <source>J. Creat. Behav.</source> <volume>55</volume>, <fpage>517</fpage>&#x02013;<lpage>531</lpage>. <pub-id pub-id-type="doi">10.1002/jocb.471</pub-id></citation>
</ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dunbar</surname> <given-names>K.</given-names></name> <name><surname>Forster</surname> <given-names>E.</given-names></name></person-group> (<year>2009</year>). <article-title>&#x0201C;Creativity evaluation through latent semantic analysis,&#x0201D;</article-title> in <source>Proceedings of the Annual Meeting of the Cognitive Science Society</source>, 31.</citation>
</ref>
<ref id="B24">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ethayarajh</surname> <given-names>K.</given-names></name></person-group> (<year>2018</year>). <article-title>&#x0201C;Unsupervised random walk sentence embeddings: A strong but simple baseline,&#x0201D;</article-title> in <source>Proceedings of The Third Workshop on Representation Learning for NLP</source> <fpage>91</fpage>&#x02013;<lpage>100</lpage>. <pub-id pub-id-type="doi">10.18653/v1/W18-3012</pub-id></citation>
</ref>
<ref id="B25">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Franceschelli</surname> <given-names>G.</given-names></name> <name><surname>Musolesi</surname> <given-names>M.</given-names></name></person-group> (<year>2022</year>). <article-title>Deepcreativity: measuring creativity with deep learning techniques</article-title>. <source>Intell. Artif.</source> <volume>16</volume>, <fpage>151</fpage>&#x02013;<lpage>163</lpage>. <pub-id pub-id-type="doi">10.3233/IA-220136</pub-id></citation>
</ref>
<ref id="B26">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>George</surname> <given-names>T.</given-names></name> <name><surname>Wiley</surname> <given-names>J.</given-names></name></person-group> (<year>2020</year>). <article-title>Need something different? Here&#x00027;s what&#x00027;s been done: Effects of examples and task instructions on creative idea generation</article-title>. <source>Memory Cogn.</source> <volume>48</volume>, <fpage>226</fpage>&#x02013;<lpage>243</lpage>. <pub-id pub-id-type="doi">10.3758/s13421-019-01005-4</pub-id><pub-id pub-id-type="pmid">31907862</pub-id></citation></ref>
<ref id="B27">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Georgiev</surname> <given-names>G. V.</given-names></name> <name><surname>Casakin</surname> <given-names>H.</given-names></name></person-group> (<year>2019</year>). <article-title>&#x0201C;Semantic measures for enhancing creativity in design education,&#x0201D;</article-title> in <source>Proceedings of the Design Society: International Conference on Engineering Design</source> (<publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>), <fpage>369</fpage>&#x02013;<lpage>378</lpage>. <pub-id pub-id-type="doi">10.1017/dsi.2019.40</pub-id></citation>
</ref>
<ref id="B28">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gong</surname> <given-names>Z.</given-names></name> <name><surname>Shan</surname> <given-names>C.</given-names></name> <name><surname>Yu</surname> <given-names>H.</given-names></name></person-group> (<year>2019</year>). <article-title>The relationship between the feedback environment and creativity: a self-motives perspective</article-title>. <source>Psychol. Res Behav. Manag.</source> <volume>12</volume>, <fpage>825</fpage>&#x02013;<lpage>837</lpage>. <pub-id pub-id-type="doi">10.2147/PRBM.S221670</pub-id><pub-id pub-id-type="pmid">31572030</pub-id></citation></ref>
<ref id="B29">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gong</surname> <given-names>Z.</given-names></name> <name><surname>Zhang</surname> <given-names>N.</given-names></name></person-group> (<year>2017</year>). <article-title>Using a feedback environment to improve creative performance: a dynamic affect perspective</article-title>. <source>Front. Psychol.</source> <volume>8</volume>, <fpage>1398</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2017.01398</pub-id><pub-id pub-id-type="pmid">28861025</pub-id></citation></ref>
<ref id="B30">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Guilford</surname> <given-names>J. P.</given-names></name></person-group> (<year>1967</year>). <article-title>Creativity: Yesterday, today and tomorrow</article-title>. <source>J. Creat. Behav.</source> <volume>1</volume>, <fpage>3</fpage>&#x02013;<lpage>14</lpage>. <pub-id pub-id-type="doi">10.1002/j.2162-6057.1967.tb00002.x</pub-id></citation>
</ref>
<ref id="B31">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Guo</surname> <given-names>Y.</given-names></name> <name><surname>Lin</surname> <given-names>S.</given-names></name> <name><surname>Williams</surname> <given-names>Z. J.</given-names></name> <name><surname>Zeng</surname> <given-names>Y.</given-names></name> <name><surname>Clark</surname> <given-names>L. Q. C.</given-names></name></person-group> (<year>2023</year>). <article-title>Evaluative skill in the creativeprocess: A cross-cultural study</article-title>. <source>Think. Skills Creativ.</source> <volume>47</volume>, <fpage>101240</fpage>. <pub-id pub-id-type="doi">10.1016/j.tsc.2023.101240</pub-id></citation>
</ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hass</surname> <given-names>R. W.</given-names></name></person-group> (<year>2017</year>). <article-title>Tracking the dynamics of divergent thinking via semantic distance: Analytic methods and theoretical implications</article-title>. <source>Memory Cogn.</source> <volume>45</volume>, <fpage>233</fpage>&#x02013;<lpage>244</lpage>. <pub-id pub-id-type="doi">10.3758/s13421-016-0659-y</pub-id><pub-id pub-id-type="pmid">27752960</pub-id></citation></ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hennessy</surname> <given-names>S.</given-names></name> <name><surname>Howe</surname> <given-names>C.</given-names></name> <name><surname>Mercer</surname> <given-names>N.</given-names></name> <name><surname>Vrikki</surname> <given-names>M.</given-names></name></person-group> (<year>2020</year>). <article-title>Coding classroom dialogue: Methodological considerations for researchers</article-title>. <source>Learning, Cult. Soc. Interact.</source> <volume>25</volume>, <fpage>100404</fpage>. <pub-id pub-id-type="doi">10.1016/j.lcsi.2020.100404</pub-id></citation>
</ref>
<ref id="B34">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Hofmann</surname> <given-names>T.</given-names></name></person-group> (<year>1999</year>). <article-title>&#x0201C;Probabilistic latent semantic indexing,&#x0201D;</article-title> in <source>Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR&#x00027;99</source> (<publisher-loc>New York, NY, USA</publisher-loc>: <publisher-name>Association for Computing Machinery</publisher-name>), <fpage>50</fpage>&#x02013;<lpage>57</lpage>. <pub-id pub-id-type="doi">10.1145/312624.312649</pub-id></citation>
</ref>
<ref id="B35">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Huang</surname> <given-names>R.</given-names></name> <name><surname>Wei</surname> <given-names>C.</given-names></name> <name><surname>Wang</surname> <given-names>B.</given-names></name> <name><surname>Yang</surname> <given-names>J.</given-names></name> <name><surname>Xu</surname> <given-names>X.</given-names></name> <name><surname>Wu</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Well performance prediction based on long short-term memory (lstm) neural network</article-title>. <source>J. Petroleum Sci. Eng.</source> <volume>208</volume>, <fpage>109686</fpage>. <pub-id pub-id-type="doi">10.1016/j.petrol.2021.109686</pub-id></citation>
</ref>
<ref id="B36">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jimenez-Mavillard</surname> <given-names>A.</given-names></name> <name><surname>Suarez</surname> <given-names>J. L.</given-names></name></person-group> (<year>2022</year>). <article-title>A computational approach for creativity assessment of culinary products: the case of elbulli</article-title>. <source>AI Soc.</source> <volume>37</volume>, <fpage>331</fpage>&#x02013;<lpage>353</lpage>. <pub-id pub-id-type="doi">10.1007/s00146-021-01183-3</pub-id></citation>
</ref>
<ref id="B37">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Johnson</surname> <given-names>D. R.</given-names></name> <name><surname>Hass</surname> <given-names>R. W.</given-names></name></person-group> (<year>2022</year>). <article-title>Semantic context search in creative idea generation</article-title>. <source>J. Creat. Behav.</source> <volume>56</volume>, <fpage>362</fpage>&#x02013;<lpage>381</lpage>. <pub-id pub-id-type="doi">10.1002/jocb.534</pub-id></citation>
</ref>
<ref id="B38">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kang</surname> <given-names>Y.</given-names></name> <name><surname>Sun</surname> <given-names>Z.</given-names></name> <name><surname>Wang</surname> <given-names>S.</given-names></name> <name><surname>Huang</surname> <given-names>Z.</given-names></name> <name><surname>Wu</surname> <given-names>Z.</given-names></name> <name><surname>Ma</surname> <given-names>X.</given-names></name></person-group> (<year>2021</year>). <article-title>&#x0201C;Metamap: Supporting visual metaphor ideation through multi-dimensional example-based exploration,&#x0201D;</article-title> in <source>Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems</source> <fpage>1</fpage>&#x02013;<lpage>15</lpage>. <pub-id pub-id-type="doi">10.1145/3411764.3445325</pub-id></citation>
</ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Karampiperis</surname> <given-names>P.</given-names></name> <name><surname>Koukourikos</surname> <given-names>A.</given-names></name> <name><surname>Koliopoulou</surname> <given-names>E.</given-names></name></person-group> (<year>2014</year>). <article-title>&#x0201C;Towards machines for measuring creativity: The use of computational tools in storytelling activities,&#x0201D;</article-title> in <source>2014 IEEE 14th International Conference on Advanced Learning Technologies</source> <fpage>508</fpage>&#x02013;<lpage>512</lpage>. <pub-id pub-id-type="doi">10.1109/ICALT.2014.150</pub-id></citation>
</ref>
<ref id="B40">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kenett</surname> <given-names>Y. N.</given-names></name></person-group> (<year>2019</year>). <article-title>What can quantitative measures of semantic distance tell us about creativity?</article-title> <source>Curr. Opin. Behav. Sci.</source> <volume>27</volume>, <fpage>11</fpage>&#x02013;<lpage>16</lpage>. <pub-id pub-id-type="doi">10.1016/j.cobeha.2018.08.010</pub-id></citation>
</ref>
<ref id="B41">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kenworthy</surname> <given-names>J. B.</given-names></name> <name><surname>Doboli</surname> <given-names>S.</given-names></name> <name><surname>Alsayed</surname> <given-names>O.</given-names></name> <name><surname>Choudhary</surname> <given-names>R.</given-names></name> <name><surname>Jaed</surname> <given-names>A.</given-names></name> <name><surname>Minai</surname> <given-names>A. A.</given-names></name> <etal/></person-group>. (<year>2023</year>). <article-title>Toward the development of a computer-assisted, real-time assessment of ideational dynamics in collaborative creative groups</article-title>. <source>Creativ. Res. J.</source> <volume>35</volume>, <fpage>396</fpage>&#x02013;<lpage>411</lpage>. <pub-id pub-id-type="doi">10.1080/10400419.2022.2157589</pub-id></citation>
</ref>
<ref id="B42">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname> <given-names>S.</given-names></name> <name><surname>Choe</surname> <given-names>I.</given-names></name> <name><surname>Kaufman</surname> <given-names>J. C.</given-names></name></person-group> (<year>2019</year>). <article-title>The development and evaluation of the effect of creative problem-solving program on young children&#x00027;s creativity and character</article-title>. <source>Think. Skills Creativ.</source> <volume>33</volume>, <fpage>100590</fpage>. <pub-id pub-id-type="doi">10.1016/j.tsc.2019.100590</pub-id></citation>
</ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kiros</surname> <given-names>R.</given-names></name> <name><surname>Zhu</surname> <given-names>Y.</given-names></name> <name><surname>Salakhutdinov</surname> <given-names>R. R.</given-names></name> <name><surname>Zemel</surname> <given-names>R.</given-names></name> <name><surname>Urtasun</surname> <given-names>R.</given-names></name> <name><surname>Torralba</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2015</year>). <article-title>&#x0201C;Skip-thought vectors,&#x0201D;</article-title> in <source>Advances in Neural Information Processing Systems</source> 28.</citation>
</ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kovalkov</surname> <given-names>A.</given-names></name> <name><surname>Paa&#x000DF;en</surname> <given-names>B.</given-names></name> <name><surname>Segal</surname> <given-names>A.</given-names></name> <name><surname>Pinkwart</surname> <given-names>N.</given-names></name> <name><surname>Gal</surname> <given-names>K.</given-names></name></person-group> (<year>2021</year>). <article-title>Automatic creativity measurement in scratch programs across modalities</article-title>. <source>IEEE Trans. Learn. Technol.</source> <volume>14</volume>, <fpage>740</fpage>&#x02013;<lpage>753</lpage>. <pub-id pub-id-type="doi">10.1109/TLT.2022.3144442</pub-id></citation>
</ref>
<ref id="B45">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>LaVoie</surname> <given-names>N.</given-names></name> <name><surname>Parker</surname> <given-names>J.</given-names></name> <name><surname>Legree</surname> <given-names>P. J.</given-names></name> <name><surname>Ardison</surname> <given-names>S.</given-names></name> <name><surname>Kilcullen</surname> <given-names>R. N.</given-names></name></person-group> (<year>2020</year>). <article-title>Using latent semantic analysis to score short answer constructed responses: Automated scoring of the consequences test</article-title>. <source>Educ. Psychol. Measur.</source> <volume>80</volume>, <fpage>399</fpage>&#x02013;<lpage>414</lpage>. <pub-id pub-id-type="doi">10.1177/0013164419860575</pub-id><pub-id pub-id-type="pmid">32158028</pub-id></citation></ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>D. D.</given-names></name> <name><surname>Seung</surname> <given-names>H. S.</given-names></name></person-group> (<year>1999</year>). <article-title>Learning the parts of objects by non-negative matrix factorization</article-title>. <source>Nature</source> <volume>401</volume>, <fpage>788</fpage>&#x02013;<lpage>791</lpage>. <pub-id pub-id-type="doi">10.1038/44565</pub-id><pub-id pub-id-type="pmid">10548103</pub-id></citation></ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Li</surname> <given-names>Y.</given-names></name> <name><surname>Du Ying</surname> <given-names>X. I. E.</given-names></name> <name><surname>Liu</surname> <given-names>C.</given-names></name> <name><surname>Yang</surname> <given-names>Y.</given-names></name> <name><surname>Li</surname> <given-names>Y.</given-names></name> <name><surname>Qiu</surname> <given-names>J.</given-names></name></person-group> (<year>2023</year>). <article-title>A meta-analysis of the relationship 649 between semantic distance and creative thinking</article-title>. <source>Adv. Psychol. Sci.</source> <volume>31</volume>, <fpage>519</fpage>. <pub-id pub-id-type="doi">10.3724/SP.J.1042.2023.00519</pub-id></citation>
</ref>
<ref id="B48">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liu</surname> <given-names>P.</given-names></name> <name><surname>Yuan</surname> <given-names>W.</given-names></name> <name><surname>Fu</surname> <given-names>J.</given-names></name> <name><surname>Jiang</surname> <given-names>Z.</given-names></name> <name><surname>Hayashi</surname> <given-names>H.</given-names></name> <name><surname>Neubig</surname> <given-names>G.</given-names></name></person-group> (<year>2023</year>). <article-title>Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing</article-title>. <source>ACM Comput. Surv.</source> <volume>55</volume>, <fpage>1</fpage>&#x02013;<lpage>35</lpage>. <pub-id pub-id-type="doi">10.1145/3560815</pub-id></citation>
</ref>
<ref id="B49">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Manske</surname> <given-names>S.</given-names></name> <name><surname>Hoppe</surname> <given-names>H. U.</given-names></name></person-group> (<year>2014</year>). <article-title>&#x0201C;Automated indicators to assess the creativity of solutions to programming exercises,&#x0201D;</article-title> in <source>2014 IEEE 14th International Conference on Advanced Learning Technologies</source> <fpage>497</fpage>&#x02013;<lpage>501</lpage>. <pub-id pub-id-type="doi">10.1109/ICALT.2014.147</pub-id></citation>
</ref>
<ref id="B50">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marrone</surname> <given-names>R.</given-names></name> <name><surname>Cropley</surname> <given-names>D. H.</given-names></name> <name><surname>Wang</surname> <given-names>Z.</given-names></name></person-group> (<year>2022</year>). <article-title>Automatic assessment of mathematical creativity using natural language processing</article-title>. <source>Creat. Res. J.</source> <volume>2022</volume>, <fpage>1</fpage>&#x02013;<lpage>16</lpage>. <pub-id pub-id-type="doi">10.1080/10400419.2022.2131209</pub-id></citation>
</ref>
<ref id="B51">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Martin</surname> <given-names>D. I.</given-names></name> <name><surname>Berry</surname> <given-names>M. W.</given-names></name></person-group> (<year>2007</year>). <article-title>&#x0201C;Mathematical foundations behind latent semantic analysis,&#x0201D;</article-title> in <source>Handbook of Latent Semantic Analysis</source> <fpage>35</fpage>&#x02013;<lpage>56</lpage>.</citation>
</ref>
<ref id="B52">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mastria</surname> <given-names>S.</given-names></name> <name><surname>Agnoli</surname> <given-names>S.</given-names></name> <name><surname>Zanon</surname> <given-names>M.</given-names></name> <name><surname>Acar</surname> <given-names>S.</given-names></name> <name><surname>Runco</surname> <given-names>M. A.</given-names></name> <name><surname>Corazza</surname> <given-names>G. E.</given-names></name></person-group> (<year>2021</year>). <article-title>Clustering and switching in divergent thinking: Neurophysiological correlates underlying flexibility during idea generation</article-title>. <source>Neuropsychologia</source> <volume>158</volume>, <fpage>107890</fpage>. <pub-id pub-id-type="doi">10.1016/j.neuropsychologia.2021.107890</pub-id><pub-id pub-id-type="pmid">34010602</pub-id></citation></ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mikolov</surname> <given-names>T.</given-names></name> <name><surname>Chen</surname> <given-names>K.</given-names></name> <name><surname>Corrado</surname> <given-names>G.</given-names></name> <name><surname>Dean</surname> <given-names>J.</given-names></name></person-group> (<year>2013</year>). <article-title>Efficient estimation of word representations in vector space</article-title>. <source>arXiv preprint arXiv:1301.3781.</source></citation>
</ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Munn</surname> <given-names>Z.</given-names></name> <name><surname>Peters</surname> <given-names>M. D.</given-names></name> <name><surname>Stern</surname> <given-names>C.</given-names></name> <name><surname>Tufanaru</surname> <given-names>C.</given-names></name> <name><surname>McArthur</surname> <given-names>A.</given-names></name> <name><surname>Aromataris</surname> <given-names>E.</given-names></name></person-group> (<year>2018</year>). <article-title>Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach</article-title>. <source>BMC Med. Res. Methodol.</source> <volume>18</volume>, <fpage>1</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1186/s12874-018-0611-x</pub-id><pub-id pub-id-type="pmid">30453902</pub-id></citation></ref>
<ref id="B55">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Olivares-Rodr&#x000ED;guez</surname> <given-names>C.</given-names></name> <name><surname>Guenaga</surname> <given-names>M.</given-names></name> <name><surname>Garaizar</surname> <given-names>P.</given-names></name></person-group> (<year>2017</year>). <article-title>Automatic assessment of creativity in heuristic problem-solving based on query diversity</article-title>. <source>DYNA</source> <volume>92</volume>, <fpage>449</fpage>&#x02013;<lpage>455</lpage>. <pub-id pub-id-type="doi">10.6036/8243</pub-id></citation>
</ref>
<ref id="B56">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Olson</surname> <given-names>J. A.</given-names></name> <name><surname>Nahas</surname> <given-names>J.</given-names></name> <name><surname>Chmoulevitch</surname> <given-names>D.</given-names></name> <name><surname>Cropper</surname> <given-names>S. J.</given-names></name> <name><surname>Webb</surname> <given-names>M. E.</given-names></name></person-group> (<year>2021</year>). <article-title>Naming unrelated words predicts creativity</article-title>. <source>Proc. Nat. Acad. Sci.</source> <volume>118</volume>, <fpage>e2022340118</fpage>. <pub-id pub-id-type="doi">10.1073/pnas.2022340118</pub-id><pub-id pub-id-type="pmid">34140408</pub-id></citation></ref>
<ref id="B57">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Organisciak</surname> <given-names>P.</given-names></name> <name><surname>Newman</surname> <given-names>M.</given-names></name> <name><surname>Eby</surname> <given-names>D.</given-names></name> <name><surname>Acar</surname> <given-names>S.</given-names></name> <name><surname>Dumas</surname> <given-names>D.</given-names></name></person-group> (<year>2023</year>). <article-title>How do the kids speak? Improving educational use of text mining with child-directed language models</article-title>. <source>Inf. Learn. Sci.</source> <volume>124</volume>, <fpage>25</fpage>&#x02013;<lpage>47</lpage>. <pub-id pub-id-type="doi">10.1108/ILS-06-2022-0082</pub-id></citation>
</ref>
<ref id="B58">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pennington</surname> <given-names>J.</given-names></name> <name><surname>Socher</surname> <given-names>R.</given-names></name> <name><surname>Manning</surname> <given-names>C. D.</given-names></name></person-group> (<year>2014</year>). <article-title>&#x0201C;Glove: global vectors for word representation,&#x0201D;</article-title> in <source>Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing</source> (EMNLP), <fpage>1532</fpage>&#x02013;<lpage>1543</lpage>. <pub-id pub-id-type="doi">10.3115/v1/D14-1162</pub-id></citation>
</ref>
<ref id="B59">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Plucker</surname> <given-names>J. A.</given-names></name> <name><surname>Meyer</surname> <given-names>M. S.</given-names></name> <name><surname>Karami</surname> <given-names>S.</given-names></name> <name><surname>Ghahremani</surname> <given-names>M.</given-names></name></person-group> (<year>2023</year>). <article-title>&#x0201C;Room to run: Using technology to move creativity into the classroom,&#x0201D;</article-title> in <source>Creative Provocations: Speculations on the Future of Creativity, Technology and Learning</source> (<publisher-loc>Springer</publisher-loc>) <fpage>65</fpage>&#x02013;<lpage>80</lpage>. <pub-id pub-id-type="doi">10.1007/978-3-031-14549-0_5</pub-id></citation>
</ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Prasch</surname> <given-names>L.</given-names></name> <name><surname>Maruhn</surname> <given-names>P.</given-names></name> <name><surname>Br&#x000FC;nn</surname> <given-names>M.</given-names></name> <name><surname>Bengler</surname> <given-names>K.</given-names></name></person-group> (<year>2020</year>). <article-title>&#x0201C;Creativity assessment via novelty and usefulness (canu) &#x02013; approach to an easy to use objective test tool,&#x0201D;</article-title> in <source>Proceedings of the Sixth International Conference on Design Creativity (ICDC)</source> <fpage>019</fpage>&#x02013;<lpage>026</lpage>. <pub-id pub-id-type="doi">10.35199/ICDC.2020.03</pub-id></citation>
</ref>
<ref id="B61">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Raffel</surname> <given-names>C.</given-names></name> <name><surname>Shazeer</surname> <given-names>N.</given-names></name> <name><surname>Roberts</surname> <given-names>A.</given-names></name> <name><surname>Lee</surname> <given-names>K.</given-names></name> <name><surname>Narang</surname> <given-names>S.</given-names></name> <name><surname>Matena</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2020</year>). <article-title>Exploring the limits of transfer learning with a unified text-to-text transformer</article-title>. <source>J. Mach. Learn. Res.</source> <volume>21</volume>, <fpage>687</fpage> <fpage>5485</fpage>&#x02013;<lpage>5551</lpage>. <pub-id pub-id-type="doi">10.48550/arXiv.1910.10683</pub-id></citation>
</ref>
<ref id="B62">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rafner</surname> <given-names>J.</given-names></name> <name><surname>Biskj&#x000E6;r</surname> <given-names>M. M.</given-names></name> <name><surname>Zana</surname> <given-names>B.</given-names></name> <name><surname>Langsford</surname> <given-names>S.</given-names></name> <name><surname>Bergenholtz</surname> <given-names>C.</given-names></name> <name><surname>Rahimi</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Digital games for creativity assessment: strengths, weaknesses and opportunities</article-title>. <source>Creat. Res. J.</source> <volume>34</volume>, <fpage>28</fpage>&#x02013;<lpage>54</lpage>. <pub-id pub-id-type="doi">10.1080/10400419.2021.1971447</pub-id></citation>
</ref>
<ref id="B63">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Reimers</surname> <given-names>N.</given-names></name> <name><surname>Gurevych</surname> <given-names>I.</given-names></name></person-group> (<year>2019</year>). <article-title>Sentence-bert: Sentence embeddings using siamese bert-networks</article-title>. <source>arXiv preprint arXiv:1908.10084.</source></citation>
</ref>
<ref id="B64">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rominger</surname> <given-names>C.</given-names></name> <name><surname>Benedek</surname> <given-names>M.</given-names></name> <name><surname>Lebuda</surname> <given-names>I.</given-names></name> <name><surname>Perchtold-Stefan</surname> <given-names>C. M.</given-names></name> <name><surname>Schwerdtfeger</surname> <given-names>A. R.</given-names></name> <name><surname>Papousek</surname> <given-names>I.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Functional brain activation patterns of creative metacognitive monitoring</article-title>. <source>Neuropsychologia</source> <volume>177</volume>, <fpage>108416</fpage>. <pub-id pub-id-type="doi">10.1016/j.neuropsychologia.2022.108416</pub-id><pub-id pub-id-type="pmid">36343705</pub-id></citation></ref>
<ref id="B65">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Said-Metwaly</surname> <given-names>S.</given-names></name> <name><surname>Van den Noortgate</surname> <given-names>W.</given-names></name> <name><surname>Kyndt</surname> <given-names>E.</given-names></name></person-group> (<year>2017</year>). <article-title>Approaches to measuring creativity: A systematic literature review</article-title>. <source>Creativity.</source> <volume>4</volume>, <fpage>238</fpage>&#x02013;<lpage>275</lpage>. <pub-id pub-id-type="doi">10.1515/ctra-2017-0013</pub-id></citation>
</ref>
<ref id="B66">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sawyer</surname> <given-names>R. K.</given-names></name></person-group> (<year>2011</year>). <article-title><italic>Explaining creativity: The science of human innovation</italic> (Oxford university press) Sawyer R. K. (2021). The iterative and improvisational nature of the creative process</article-title>. <source>J. Creat.</source> <volume>31</volume>, <fpage>100002</fpage>. <pub-id pub-id-type="doi">10.1016/j.yjoc.2021.100002</pub-id></citation>
</ref>
<ref id="B67">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sawyer</surname> <given-names>R. K.</given-names></name></person-group> (<year>2022</year>). <article-title>The dialogue of creativity: Teaching the creative process by animating student work as a collaborating creative agent</article-title>. <source>Cogn. Instruct.</source> <volume>40</volume>, <fpage>459</fpage>&#x02013;<lpage>487</lpage>. <pub-id pub-id-type="doi">10.1080/07370008.2021.1958219</pub-id></citation>
</ref>
<ref id="B68">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sedova</surname> <given-names>K.</given-names></name> <name><surname>Sedlacek</surname> <given-names>M.</given-names></name> <name><surname>Svaricek</surname> <given-names>R.</given-names></name> <name><surname>Majcik</surname> <given-names>M.</given-names></name> <name><surname>Navratilova</surname> <given-names>J.</given-names></name> <name><surname>Drexlerova</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2019</year>). <article-title>Do those who talk more learn more? the relationship between student classroom talk and student achievement</article-title>. <source>Learn. Instruct.</source> <volume>63</volume>, <fpage>101217</fpage>. <pub-id pub-id-type="doi">10.1016/j.learninstruc.2019.101217</pub-id></citation>
</ref>
<ref id="B69">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shrivastava</surname> <given-names>D.</given-names></name> <name><surname>Ahmed</surname> <given-names>C. G. S.</given-names></name> <name><surname>Laha</surname> <given-names>A.</given-names></name> <name><surname>Sankaranarayanan</surname> <given-names>K.</given-names></name></person-group> (<year>2017</year>). <article-title>A machine learning approach for evaluating creative artifacts</article-title>. <source>ArXiv abs/1707.05499</source>.</citation>
</ref>
<ref id="B70">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Simpson</surname> <given-names>E.</given-names></name> <name><surname>Do Dinh</surname> <given-names>E.-L.</given-names></name> <name><surname>Miller</surname> <given-names>T.</given-names></name> <name><surname>Gurevych</surname> <given-names>I.</given-names></name></person-group> (<year>2019</year>). <article-title>&#x0201C;Predicting humorousness and metaphor novelty with gaussian process preference learning,&#x0201D;</article-title> in <source>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</source> <fpage>5716</fpage>&#x02013;<lpage>5728</lpage>. <pub-id pub-id-type="doi">10.18653/v1/P19-1572</pub-id></citation>
</ref>
<ref id="B71">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Song</surname> <given-names>K.</given-names></name> <name><surname>Tan</surname> <given-names>X.</given-names></name> <name><surname>Qin</surname> <given-names>T.</given-names></name> <name><surname>Lu</surname> <given-names>J.</given-names></name> <name><surname>Liu</surname> <given-names>T.-Y.</given-names></name></person-group> (<year>2020</year>). <article-title>Mpnet: Masked and permuted pre-training for language understanding</article-title>. <source>Adv. Neural Inf. Proc. Syst.</source> <volume>33</volume>, <fpage>16857</fpage>&#x02013;<lpage>16867</lpage>. <pub-id pub-id-type="doi">10.48550/arXiv.2004.09297</pub-id></citation>
</ref>
<ref id="B72">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stella</surname> <given-names>M.</given-names></name> <name><surname>Kenett</surname> <given-names>Y. N.</given-names></name></person-group> (<year>2019</year>). <article-title>Viability in multiplex lexical networks and machine learning characterizes human creativity</article-title>. <source>Big Data Cogn. Comput.</source> <volume>3</volume>, <fpage>45</fpage>. <pub-id pub-id-type="doi">10.3390/bdcc3030045</pub-id></citation>
</ref>
<ref id="B73">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sung</surname> <given-names>Y.-T.</given-names></name> <name><surname>Cheng</surname> <given-names>H.-H.</given-names></name> <name><surname>Tseng</surname> <given-names>H.-C.</given-names></name> <name><surname>Chang</surname> <given-names>K.-E.</given-names></name> <name><surname>Lin</surname> <given-names>S.-Y.</given-names></name></person-group> (<year>2022</year>). <article-title>&#x0201C;Construction and validation of a computerized creativity assessment tool with automated scoring based on deep-learning techniques,&#x0201D;</article-title> in <source>Psychology of Aesthetics, Creativity, and the Arts.</source> <pub-id pub-id-type="doi">10.1037/aca0000450</pub-id></citation>
</ref>
<ref id="B74">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Toma</surname> <given-names>J. D.</given-names></name></person-group> (<year>2011</year>). <article-title>&#x0201C;Approaching rigor in applied qualitative,&#x0201D;</article-title> in <source>The SAGE Handbook for Research in Education: Pursuing Ideas as the Keystone of Exemplary Inquiry</source> <fpage>263</fpage>&#x02013;<lpage>281</lpage>. <pub-id pub-id-type="doi">10.4135/9781483351377.n17</pub-id></citation>
</ref>
<ref id="B75">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Torrance</surname> <given-names>E. P.</given-names></name></person-group> (<year>2008</year>). <source>The Torrance Tests of Creative Thinking Norms&#x02014;Technical Manual Figural (Streamlined) Forms a and b. 1998.</source> <publisher-loc>Bensenville, IL</publisher-loc>: <publisher-name>Scholastic Testing Service</publisher-name>.</citation>
</ref>
<ref id="B76">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tuzcu</surname> <given-names>A.</given-names></name></person-group> (<year>2021</year>). <article-title>The impact of google translate on creativity in writing activities</article-title>. <source>Lang. Educ. Technol.</source> <volume>1</volume>, <fpage>40</fpage>&#x02013;<lpage>52</lpage>.</citation>
</ref>
<ref id="B77">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vartanian</surname> <given-names>O.</given-names></name> <name><surname>Smith</surname> <given-names>I.</given-names></name> <name><surname>Lam</surname> <given-names>T. K.</given-names></name> <name><surname>King</surname> <given-names>K.</given-names></name> <name><surname>Lam</surname> <given-names>Q.</given-names></name> <name><surname>Beatty</surname> <given-names>E. L.</given-names></name></person-group> (<year>2020</year>). <article-title>The relationship between methods of scoring the alternate uses task and the neural correlates of divergent thinking: Evidence from voxel-based morphometry</article-title>. <source>NeuroImage</source> <volume>223</volume>, <fpage>117325</fpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2020.117325</pub-id><pub-id pub-id-type="pmid">32882380</pub-id></citation></ref>
<ref id="B78">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vaswani</surname> <given-names>A.</given-names></name> <name><surname>Shazeer</surname> <given-names>N.</given-names></name> <name><surname>Parmar</surname> <given-names>N.</given-names></name> <name><surname>Uszkoreit</surname> <given-names>J.</given-names></name> <name><surname>Jones</surname> <given-names>L.</given-names></name> <name><surname>Gomez</surname> <given-names>A. N.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>&#x0201C;Attention is all you need,&#x0201D;</article-title> in <source>Advances in Neural Information Processing Systems</source> 30.</citation>
</ref>
<ref id="B79">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vo</surname> <given-names>H.</given-names></name> <name><surname>Asojo</surname> <given-names>A.</given-names></name></person-group> (<year>2018</year>). <article-title>Feedback responsiveness and students&#x00027; creativity</article-title>. <source>Acad. Exch. Quart.</source> <volume>1</volume>, <fpage>53</fpage>&#x02013;<lpage>57</lpage>.</citation>
</ref>
<ref id="B80">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wagire</surname> <given-names>A. A.</given-names></name> <name><surname>Rathore</surname> <given-names>A.</given-names></name> <name><surname>Jain</surname> <given-names>R.</given-names></name></person-group> (<year>2020</year>). <article-title>Analysis and synthesis of industry 4.0 research landscape: Using latent semantic analysis approach</article-title>. <source>J. Manuf. Technol. Manag.</source> <volume>31</volume>, <fpage>31</fpage>&#x02013;<lpage>51</lpage>. <pub-id pub-id-type="doi">10.1108/JMTM-10-2018-0349</pub-id></citation>
</ref>
<ref id="B81">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>A.</given-names></name> <name><surname>Pruksachatkun</surname> <given-names>Y.</given-names></name> <name><surname>Nangia</surname> <given-names>N.</given-names></name> <name><surname>Singh</surname> <given-names>A.</given-names></name> <name><surname>Michael</surname> <given-names>J.</given-names></name> <name><surname>Hill</surname> <given-names>F.</given-names></name> <etal/></person-group>. (<year>2019</year>). <article-title>&#x0201C;Superglue: A stickier benchmark for general-purpose language understanding systems,&#x0201D;</article-title> in <source>Advances in neural Information Processing Systems</source> 32.</citation>
</ref>
<ref id="B82">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>A.</given-names></name> <name><surname>Singh</surname> <given-names>A.</given-names></name> <name><surname>Michael</surname> <given-names>J.</given-names></name> <name><surname>Hill</surname> <given-names>F.</given-names></name> <name><surname>Levy</surname> <given-names>O.</given-names></name> <name><surname>Bowman</surname> <given-names>S. R.</given-names></name></person-group> (<year>2018</year>). <article-title>Glue: A multi-task benchmark and analysis platform for natural language understanding</article-title>. <source>arXiv preprint arXiv:1804.07461.</source></citation>
</ref>
<ref id="B83">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Williams</surname> <given-names>F.</given-names></name></person-group> (<year>1980</year>). <source>Creativity Assessment Packet (CAP)</source>. <publisher-loc>Buffalo, NY</publisher-loc>: <publisher-name>D. O. K. Publishers Inc</publisher-name>.</citation>
</ref>
<ref id="B84">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yang</surname> <given-names>Z.</given-names></name> <name><surname>Zhu</surname> <given-names>C.</given-names></name> <name><surname>Chen</surname> <given-names>W.</given-names></name></person-group> (<year>2018</year>). <article-title>Parameter-free sentence embedding via orthogonal basis</article-title>. <source>arXiv preprint arXiv:1810.00438.</source></citation>
</ref>
<ref id="B85">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>W.</given-names></name> <name><surname>Sjoerds</surname> <given-names>Z.</given-names></name> <name><surname>Hommel</surname> <given-names>B.</given-names></name></person-group> (<year>2020</year>). <article-title>Metacontrol of human creativity: The neurocognitive mechanisms of convergent and divergent thinking</article-title>. <source>NeuroImage</source> <volume>210</volume>, <fpage>116572</fpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2020.116572</pub-id><pub-id pub-id-type="pmid">31972282</pub-id></citation></ref>
<ref id="B86">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Zu&#x000F1;iga</surname> <given-names>D.</given-names></name> <name><surname>Amido</surname> <given-names>T.</given-names></name> <name><surname>Camargo</surname> <given-names>J.</given-names></name></person-group> (<year>2017</year>). <article-title>&#x0201C;Communications in computer and information science,&#x0201D;</article-title> in <source>Colombian Conference on Computing</source> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>).</citation>
</ref>
</ref-list>
</back>
</article>