<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="brief-report" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Psychiatry</journal-id>
<journal-title>Frontiers in Psychiatry</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Psychiatry</abbrev-journal-title>
<issn pub-type="epub">1664-0640</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fpsyt.2023.1137792</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Psychiatry</subject>
<subj-group>
<subject>Perspective</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Testamentary capacity assessment in dementia using artificial intelligence: prospects and challenges</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes"><name>
<surname>Economou</surname>
<given-names>Alexandra</given-names>
</name><xref rid="aff1" ref-type="aff"><sup>1</sup></xref>
<xref rid="c001" ref-type="corresp"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/1457482/overview"/>
</contrib>
<contrib contrib-type="author"><name>
<surname>Kontos</surname>
<given-names>John</given-names>
</name><xref rid="aff2" ref-type="aff"><sup>2</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2313153/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Psychology, School of Philosophy, National and Kapodistrian University of Athens</institution>, <addr-line>Athens</addr-line>, <country>Greece</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of History and Philosophy of Science, National and Kapodistrian University of Athens</institution>, <addr-line>Athens</addr-line>, <country>Greece</country></aff>
<author-notes>
<fn id="fn0001" fn-type="edited-by">
<p>Edited by: Thomas Nilsson, University of Gothenburg, Sweden</p>
</fn>
<fn id="fn0002" fn-type="edited-by">
<p>Reviewed by: Stefano Zago, IRCCS Ca 'Granda Foundation Maggiore Policlinico Hospital, Italy</p>
</fn>
<corresp id="c001">&#x002A;Correspondence: Alexandra Economou, <email>aoikono@psych.uoa.gr</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>31</day>
<month>05</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="collection">
<year>2023</year>
</pub-date>
<volume>14</volume>
<elocation-id>1137792</elocation-id>
<history>
<date date-type="received">
<day>04</day>
<month>01</month>
<year>2023</year>
</date>
<date date-type="accepted">
<day>15</day>
<month>05</month>
<year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2023 Economou and Kontos.</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Economou and Kontos</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>Testamentary capacity (TC), a set of capacities involved in making a valid Will, has become prominent in capacity evaluations due to the demographic increase in older persons and associated increase in cognitive impairment. The assessment of contemporaneous TC follows the criteria derived from the Banks v Goodfellow case, which do not bind capacity solely on the basis of presence of a cognitive disorder. Although effort is being made for establishing more objective criteria for TC judgment, variations in situational complexity call for incorporating the different circumstances of the testator in capacity assessment. Artificial intelligence (AI) technologies such as statistical machine learning have been used in forensic psychiatry mainly for the prediction of aggressive behavior and recidivism but little has been done in the area of capacity assessment. However, the statistical machine learning model responses are difficult to interpret and explain, which presents problems with regard to the new General Data Protection Regulation (GDPR) of the European Union. In this Perspective we present a framework for an AI decision support tool for TC assessment. The framework is based on AI decision support and explainable AI (XAI) technology.</p>
</abstract>
<kwd-group>
<kwd>testamentary capacity</kwd>
<kwd>capacity assessment</kwd>
<kwd>dementia</kwd>
<kwd>cognitive impairment</kwd>
<kwd>explainable artificial intelligence</kwd>
<kwd>decision support systems</kwd>
</kwd-group>
<counts>
<fig-count count="0"/>
<table-count count="1"/>
<equation-count count="0"/>
<ref-count count="53"/>
<page-count count="6"/>
<word-count count="5426"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Forensic Psychiatry</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec id="sec1" sec-type="intro">
<title>Introduction</title>
<p>Capacity is a legal and clinical term that refers to the ability to perform certain functions or to make autonomous decisions. It is a time-limited and task-specific ability that a person is presumed to have unless proven otherwise (<xref ref-type="bibr" rid="ref1">1</xref>). Testamentary capacity (TC), a specific civil competency, describes a complex set of capacities involving the decision to make a valid Will. Like all mental capacities, it is based on the understanding of relevant facts and information, the appreciation of the nature of the situation and the reasonably foreseeable consequences of such a decision (<xref ref-type="bibr" rid="ref2">2</xref>).</p>
<p>TC has become the focus of capacity evaluation in recent years due to the demographic increase in older persons, with the concomitant increase in dementia and cognitive impairment (<xref ref-type="bibr" rid="ref3">3</xref>), as well as societal changes involving the rise in different forms of family unions, divorce, and remarriage rates (<xref ref-type="bibr" rid="ref2">2</xref>).</p>
</sec>
<sec id="sec2">
<title>Criteria for testamentary capacity</title>
<p>The ascertainment of contemporaneous TC requires understanding of (a) the nature of the act of making a Will and its consequences; (b) the extent of one&#x2019;s assets; (c) the claims of those who might benefit from the will; (d) the impact of the distribution of the assets; and additionally, (e) that the testator is free of mental disorders or delusions that influence the disposition of assets (<xref ref-type="bibr" rid="ref4">4</xref>). These criteria, derived from the seminal Banks v Goodfellow case on TC in England in 1870, were ground-breaking because of the notion that capacity is not bound by a diagnosis; the real question according to the Chief Justice was whether the testator was of sound mind to make a Will and not whether he suffered from &#x201C;general insanity.&#x201D; Therefore, the presence of a cognitive disorder does not in itself preclude capacity although it may support the clinical assessment. Changes in the environmental context of TC since that time have led to a proposal for an updated test for TC that explicitly incorporates situational complexity, as well as greater clarity regarding the cognitive underpinnings of &#x201C;understanding&#x201D; (<xref ref-type="bibr" rid="ref5">5</xref>).</p>
<p>Whereas the Banks v Goodfellow case was the result of a case of psychosis, dementia is far more prevalent in recent times (<xref ref-type="bibr" rid="ref5">5</xref>). The challenge in assessing TC in dementia is its progressive nature impacting on cognitive ability and the fact that different types of dementia are characterized by differing profiles of cognitive loss. In accordance with the Banks v Goodfellow case, a diagnosis of dementia does not automatically necessitate loss of TC. The question that therefore arises is what cognitive abilities are required to satisfy the criteria for establishing TC in dementia (<xref ref-type="bibr" rid="ref3">3</xref>).</p>
</sec>
<sec id="sec3">
<title>The assessment of testamentary capacity</title>
<p>Analysis of TC in legal cases can be either contemporaneous with a Will execution (the main focus of this paper) or retrospective. Undue influence, a conceptually separate but related to TC issue refers to wrongful persuasion that overpowers the will of a person and induces the person under the will of another. It includes the criterion of the testator&#x2019;s susceptibility due to a psychiatric or medical condition, which renders the testator vulnerable to the influence of others and can invalidate a Will (<xref ref-type="bibr" rid="ref6">6</xref>, <xref ref-type="bibr" rid="ref7">7</xref>).</p>
<p>The assessment of TC may be informed by expert medical or psychological assessment. The expert selected must have the appropriate knowledge and skill to provide <italic>an evidence-based assessment and the reasoning for their opinion</italic> with the appropriate legal framework (<xref ref-type="bibr" rid="ref8">8</xref>). Resulting clinical classification of TC can be high, low, or borderline (<xref ref-type="bibr" rid="ref6">6</xref>) but ultimately it is the court that makes the final decision (<xref ref-type="bibr" rid="ref9">9</xref>). Because TC varies as a function of complexity within a testator&#x2019;s environment and of the Will itself, a certain level of cognitive impairment may not interfere with TC in one individual with few assets in a simple context but may do so in a more complex or conflictual context (<xref ref-type="bibr" rid="ref4">4</xref>, <xref ref-type="bibr" rid="ref5">5</xref>).</p>
<sec id="sec4">
<title>Forensic assessment instruments</title>
<p>TC has been typically assessed by semi-structured interviews, such as the Contemporaneous Assessment Instrument (CAI), which follows the six criteria established by Applebaum and Grisso that were pertinent to the four legal requirements for decision-making (understanding the nature of the act and its effect; the extent of the property the testator is disposing; the claims which the testator should give effect; to be free of a disorder of mind preventing the disposing of one&#x2019;s property). These are that the functions assessed should be conceptually close to the standards of competence; the content of the instrument should be pertinent to the distribution of one&#x2019;s estate; the instrument should be meaningful to the person assessed; the instrument should be sufficiently standardized to enable comparisons within and across research groups; the measures must have objective and reliable scoring criteria; and the instrument should be practical for use. By definition, some of the criteria must be specific to the person being assessed (<xref ref-type="bibr" rid="ref2">2</xref>).</p>
<p>The Testament Definition Scale (TDS) is a short rating scale designed to evaluate and quantify the individual&#x2019;s ability to give an exhaustive definition of what a testament is, thus addressing the first criterion derived from the Banks v Goodfellow case (<xref ref-type="bibr" rid="ref10">10</xref>).</p>
<p>The Legal Capacity Questionnaire (LCQ), a screening instrument for attorneys, examines three of the four legal requirements of TC using true/false, multiple-choice, and open-ended questions. On the basis of limited normative data for older adults and correlational analyses with mental status and behavioral instruments, cut scores have been created for high, borderline, and low capacity. The LCQ is geared toward older clients with cognitive disabilities, is administered in a standardized manner and is norm-referenced but lacks reliability and validity information, the criterion of understanding what a Will is, and the investigation of psychotic symptomatology and undue influence (<xref ref-type="bibr" rid="ref6">6</xref>).</p>
<p>The Testamentary Capacity Assessment Tool (TCAT) (<xref ref-type="bibr" rid="ref11">11</xref>) is a brief instrument that has been validated in older adults with dementia by comparing the scores with the opinion of a forensic psychiatrist regarding the TC of the patient, and the judgment of a senior Judge. It has been standardized in an Italian neurologically healthy sample (<xref ref-type="bibr" rid="ref12">12</xref>). The TCAT examines orientation, autobiographical memory, knowledge of financial parameters, intention and judgment. Behavior pertaining to presence of acute psychopathology is also documented. The instrument can be used to screen the clearly competent testator or as corroborative evidence of an expert&#x2019;s judgment. Because it is based only on the interview with the patient independent of collateral information it is impartial to potential conflict within the patient&#x2019;s family.</p>
<p>The more recent Testamentary Capacity Instrument (TCI), Martin et al. (<xref ref-type="bibr" rid="ref13">13</xref>) is an interview-based instrument that assesses the aforementioned four criteria of TC in a standardized interview-based format, utilizing a variety of question formats depending on the section and provides objective scoring criteria. Preliminary data showed that it was a valid measure of TC with good inter-rater reliability and internal reliability for the criteria for which such measures could be calculated. No cut scores were proposed for determining capacity, because the TCI was intended to serve as a source of additional information for making clinical judgment.</p>
</sec>
<sec id="sec5">
<title>Neuropsychological assessment</title>
<p>Neuropsychological assessment is not a source of direct evidence for TC, although it can contribute to the clinical and diagnostic context for its assessment (<xref ref-type="bibr" rid="ref14">14</xref>) and can provide important evidence where incapacity is alleged to invalidate a Will (<xref ref-type="bibr" rid="ref3">3</xref>).</p>
<p>Cognitive screening measures such as the MMSE are not reliable and valid measures of TC and are not sensitive to impairments affecting judgment and reasoning despite their wide use in capacity evaluations. The cognitive domains of complex attention, executive function, learning and memory, language, and social cognition may affect TC in different ways, although there is limited information on how impairments in these domains impact on the Banks v Goodfellow criteria (<xref ref-type="bibr" rid="ref3">3</xref>). A recent study that correlated performance on a neuropsychological battery to the TCI in a small number of cognitively healthy persons and persons with AD showed that measures of executive function, verbal memory, and semantic knowledge accounted for over 70% of shared variance with TCI performance and each measure correlated with each of the four TCI elements, suggesting a similar multidimensional cognitive basis for each TCI component (<xref ref-type="bibr" rid="ref15">15</xref>).</p>
</sec>
<sec id="sec6">
<title>Retrospective assessment</title>
<p>Although a contemporaneous assessment is desirable, many situations require a retrospective assessment of TC. A proposed methodology along with pitfalls of conducting a retrospective neuropsychological assessment are described in Zago and Bolognini (<xref ref-type="bibr" rid="ref16">16</xref>). Of interest is the inclusion of handwriting analysis, evaluated longitudinally, as a complementary tool for the evaluation of cognitive status, both with respects to formal aspects of the writing skills (spelling, grammar, layout) and logical consistency of the written text. The specifics of conducting such an analysis and deriving a &#x201C;writing score&#x201D; in the posthumous challenge of a will are provided in Balestrino et al. (<xref ref-type="bibr" rid="ref17">17</xref>). Preliminary evidence points to the association of a low writing score with abnormal performance on mental status tests but the establishment of the validity of the method requires further research (<xref ref-type="bibr" rid="ref16">16</xref>, <xref ref-type="bibr" rid="ref17">17</xref>). An assessment of medical history should precede such assessment to exclude testators with neurological conditions impacting on graphomotor skills, such as Amyotrophic Lateral Sclerosis or Parkinson&#x2019;s disease, or other medical conditions such as rheumatoid arthritis.</p>
</sec>
<sec id="sec7">
<title>Challenges in testamentary capacity assessment</title>
<p>Providing evidence-based assessment and reasoning for a TC opinion requires training that the person conducting the assessment may not have. Moreover, while effort is being made for establishing more objective criteria for TC judgment, assessment also needs to be tailored to the individual situation of the testator.</p>
<p>Evaluation of TC in aphasia and in neurodegenerative disorders characterized by progressive language impairment are a challenge to TC evaluation, as the testator&#x2019;s ability to verbally express the components of TC and explain decisions is essential for establishing TC [e.g., case by (<xref ref-type="bibr" rid="ref18">18</xref>)]. The assessment of TC in persons with severe aphasia is therefore problematic or even impossible, and techniques and formats that may facilitate the evaluation, such as prompted recall or multiple-choice formats for recognition memory, are not without drawbacks (<xref ref-type="bibr" rid="ref13">13</xref>). It is proposed that persons with aphasia should have greater access to the courts through the facilitative support of speech-language pathologists (<xref ref-type="bibr" rid="ref19">19</xref>, <xref ref-type="bibr" rid="ref20">20</xref>), which is in line with the supported decision-making trend of disability rights law (<xref ref-type="bibr" rid="ref21">21</xref>).</p>
<p>In cases of severe aphasia, nonverbal neuropsychological assessment could serve as a proxy measure although it cannot replace the TC criteria. Linguistic deficits notwithstanding, other cognitive functions may be affected in aphasia (<xref ref-type="bibr" rid="ref22">22</xref>), which may be independently impaired and may impact on mental capacity. To the best of our knowledge, there are no studies on the association of nonverbal assessment with neuropsychological tests of cognitive function that can be used in TC assessment. The establishment of such an association would facilitate the clinical and diagnostic context for TC assessment in aphasia and some types of dementia.</p>
</sec>
</sec>
<sec id="sec8">
<title>Forensic psychiatry and artificial intelligence</title>
<p>Artificial intelligence (AI) technologies and applications are ubiquitous but are not a monolithic technology. AI technologies can involve rule bases, language models, and machine learning (ML); applications can be trained using supervised ML and reinforcement learning or can look for patterns using unsupervised ML (<xref ref-type="bibr" rid="ref23">23</xref>). ML programs are generally not designed to offer an explanation for their predictions, and different ML algorithms vary in degree of transparency in decision-making. Therefore, when a model fails to make a prediction in a different sample it is difficult to analyze the reason (<xref ref-type="bibr" rid="ref24">24</xref>).</p>
<p>The computer generation of explanations dates back to the 80s with the implementation of the MYCIN expert system based on the tracing of rule-based reasoning for treatment advice for patient symptoms (<xref ref-type="bibr" rid="ref25">25</xref>). More recently, a system was presented that generated explanations based on the tracing of text-based reasoning (<xref ref-type="bibr" rid="ref26">26</xref>). Recent initiatives like the DARPA explainable artificial intelligence (XAI) program (<xref ref-type="bibr" rid="ref27">27</xref>) have brought explainability back into focus (<xref ref-type="bibr" rid="ref28">28</xref>), although the basic aspect of XAI by means of knowledge representations of a set of concepts within a domain, and statistical, formal or causal models remains mostly unexplored. The source of this neglect is the extreme popularity of statistical methods that generate &#x201C;black boxes,&#x201D; which do not lend themselves to human understanding of their operations. Such models are constructed by &#x201C;blindly&#x201D; fitting parametric functions to the data in a curve-fitting process by automatically adjusting the values of thousands or even millions of parameters that have no meaning and no causal relations to the problems under study (<xref ref-type="bibr" rid="ref29">29</xref>).</p>
<p>The role of AI in forensic psychiatry has received increased attention in recent years, with the majority of studies coming from the area of predicting violence risk in patients in psychiatric institutions or in people held in custody, e.g., [(<xref ref-type="bibr" rid="ref30 ref31 ref32 ref33 ref34">30&#x2013;34</xref>); review by (<xref ref-type="bibr" rid="ref35">35</xref>)], and a review on the controversial topic of use of brain reading AI for neuroprediction of violence by Tortora et al. (<xref ref-type="bibr" rid="ref36">36</xref>), and the prediction of future offenses and dangerousness in persons with psychiatric disorders, e.g., [(<xref ref-type="bibr" rid="ref37">37</xref>, <xref ref-type="bibr" rid="ref38">38</xref>); review by (<xref ref-type="bibr" rid="ref39">39</xref>)]. The advantage of ML is that it can combine large numbers of data and investigate large numbers of predictors in nonlinear and interactive ways. ML models are particularly useful for criterion-referenced assessment, designed to predict specific behaviors and outcomes, rather than for norm-referenced assessment, designed to measure constructs such as personality and mental status (<xref ref-type="bibr" rid="ref24">24</xref>). Not surprisingly, most studies in forensic psychiatry involve criterion-referenced prediction rules.</p>
<sec id="sec9">
<title>Ethical and legal issues in the use of AI</title>
<p>The new General Data Protection Regulation (GDPR) of the European Union enacted in 2018 prohibits automated decision-making, that is, making a decision solely by automated means without any human involvement, if a decision produces legal or similar consequences (<xref ref-type="bibr" rid="ref23">23</xref>). The GDPR has also created a &#x201C;right to explanation&#x201D; of how an AI system works and how it reached a decision, which is especially pertinent for ML models that are inherently challenging to interpret (<xref ref-type="bibr" rid="ref23">23</xref>, <xref ref-type="bibr" rid="ref40">40</xref>) and raise ethical issues, particularly in legal settings (<xref ref-type="bibr" rid="ref24">24</xref>). It has been argued that the explanations that derive from correlational computations can lead to spurious correlations rather than cause-effect relationships, resulting in erroneous or biased explanations (<xref ref-type="bibr" rid="ref41">41</xref>). International guidelines for trustworthy AI place explainability in the center of trustworthy systems and regulatory guidelines (<xref ref-type="bibr" rid="ref42">42</xref>, <xref ref-type="bibr" rid="ref43">43</xref>). Therefore, there is a growing need for the design and development of transparent and interpretable AI models and explanations that reflect the needs of the users (<xref ref-type="bibr" rid="ref44">44</xref>, <xref ref-type="bibr" rid="ref45">45</xref>).</p>
</sec>
<sec id="sec10">
<title>Testamentary capacity and artificial intelligence</title>
<p>Little attention has been paid to the potential contribution of AI to capacity assessment, although a theoretical paper has been written on the broader civil competency of financial capacity in cognitive disorders that includes AI (<xref ref-type="bibr" rid="ref46">46</xref>). According to one perspective, capacity assessment does not lend itself to formalization; in mental health the notion of incapacity is a complex legal, clinical, ethical, and social construct. Testing for it varies according to the specific circumstance for making a decision, making it difficult to formalize mathematically as required by AI (<xref ref-type="bibr" rid="ref47">47</xref>).</p>
<p>On the other hand, the use of ontologies for enhancing human understandability of global post-hoc explanations of black-box models, as presented in Confalonieri et al. (<xref ref-type="bibr" rid="ref48">48</xref>) may be developed for machine analysis of human explanations. Using this reverse reasoning, it is the <italic>human explanations</italic> regarding the evaluation of TC that could be subjected to machine analysis. Thus, instead of humans evaluating machine-generated explanations as to their correctness, AI would be called upon for the evaluation of human-generated explanations of a person&#x2019;s behavior, to facilitate the decision of the expert. The decision of the AI system can be explained using XAI methods in order to increase the trust of the human evaluator in it. The explanation that a person provides for a certain behavior or decision could be processed by a natural language programming (NLP) system in order to draw conclusions for the assessment of one&#x2019;s TC.</p>
<p>Using NLP methods proposed for assessing artificial agents&#x2019; responses in similar situations (<xref ref-type="bibr" rid="ref49">49</xref>), &#x201C;what&#x201D; questions may be used for the understanding of the nature of the act of making a Will and its consequences as well as knowledge of the extent of one&#x2019;s assets, and &#x201C;why&#x201D; questions may help disclose the testator&#x2019;s understanding of the claims of those who might benefit from the Will and the impact of the distribution of the testator&#x2019;s assets. The computer analysis of the human explanations mentioned above may contribute to offering advice to the expert making the assessment as to their correctness. This analysis could be supported by an explanatory model of the kind proposed in Pfeiff (<xref ref-type="bibr" rid="ref50">50</xref>) of explanatory practices in psychotherapy intended to make the patients assume responsibility for their symptoms. Two different notions of rationality are used: theoretical rationality, the adoption of beliefs that cohere with one&#x2019;s relevant background knowledge; and pragmatic rationality, the adoption of beliefs that are consistent with one&#x2019;s goals. The notion of dysfunctionality is proposed in which a decision is dysfunctional if it does not satisfy one&#x2019;s needs and may even cause harm. By analogy, the notions could be adapted to the assessment of TC.</p>
<p>For a retrospective analysis of handwritten or written texts AI offers some tools for their automatic analysis that takes into account both the formal aspects of the writing skills (lexical, syntactic, semantic and discourse level) and the spatial orientation in cases of handwritten texts. There is a large body of research that presents methods for evaluating these aspects of natural language in texts using AI [e.g., (<xref ref-type="bibr" rid="ref51">51</xref>, <xref ref-type="bibr" rid="ref52">52</xref>)]. The identification of emotional state through the analysis of handwriting features is a new area in AI (<xref ref-type="bibr" rid="ref53">53</xref>) that may be extended to the identification of handwriting features that reflect cognitive state.</p>
<p>To develop and test such an XAI based decision support tool would require a corpus or dataset of explanations provided by persons of varying TC, together with expert assessment of the state of their TC. However, the presence of a language disorder would invalidate the method proposed for assessing reasoning capacity because an AI system analysis of the verbal explanations of the decisions of the testator presupposes lack of language disorder of the testator. It is almost impossible to construct an AI system assessing human explanations that do not exhibit correctness in all the linguistic aspects to be tested, as listed in <xref rid="tab1" ref-type="table">Table 1</xref>. Therefore, a challenge for the NLP system supporting the experts in their assessment of explanations would be to detect a language disorder as distinct from problems with reasoning and filter out the aberrant explanations before they are submitted to the XAI decision support system (DSS). <xref rid="tab1" ref-type="table">Table 1</xref> lists the basic tests to be performed by the NLP system during the filtering process.</p>
<table-wrap position="float" id="tab1">
<label>Table 1</label>
<caption>
<p>Human explanations as a tool for TC assessment.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">NLP analysis of human explanations</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Lexical correctness, i.e., correct choice of words</p>
</list-item>
</list>
</td>
</tr>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Ontological correctness, i.e., correct grouping of objects, events, and concepts</p>
</list-item>
</list>
</td>
</tr>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Syntactic correctness of the sentences</p>
</list-item>
</list>
</td>
</tr>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Selection restrictions correctness, i.e., correct choice of verb arguments</p>
</list-item>
</list>
</td>
</tr>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Semantic correctness of the sentences, i.e., constructing sentences with semantically consistent combination of words</p>
</list-item>
</list>
</td>
</tr>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Logical correctness of conclusions drawn from the facts</p>
</list-item>
</list>
</td>
</tr>
<tr>
<td align="left" valign="top">
<list list-type="order">
<list-item>
<p>Decision correctness, e.g., decisions consistent with the relation of the beneficiaries to the testator</p>
</list-item>
</list>
</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="sec11">
<title>Objectives of an XAI system for TC assessment</title>
<p>The design and implementation of an XAI DSS for TC assessment would have the following characteristics: (a) be a truly mixed human-AI system (b) be trustworthy, (c) utilize tacit knowledge of area ontology and simulation models, (d) be tested for its adaptation to the users regardless of their age, race, gender, or capabilities, and e) adapt its explainable decision support and explainable question answering &#x201C;behavior&#x201D; to the needs of the users. A mostly unexplored area is the variation of the explanation generation of the testator spanning several axes, such as from special to general ontology levels. This variation of the style of explanations would depend on the educational level and background of the testator. The explanation analysis software must be capable of processing a variety of styles of explanations, which is an open research question (see <xref rid="sec16" ref-type="sec">Appendix</xref> in <xref ref-type="supplementary-material" rid="SM1">Supplementary material</xref> for some possible variations to guide XAI experts).</p>
<p>Area under the curve (AUC) could be used as a test statistic to describe the ability of the tool to discriminate between capacity and incapacity and to compare the performance of different tools and methods, after Garb and Wood (<xref ref-type="bibr" rid="ref24">24</xref>).</p>
</sec>
</sec>
<sec id="sec12" sec-type="conclusions">
<title>Conclusion</title>
<p>The concepts proposed in this paper are preliminary and aim to address some of the challenges of TC evaluation. An AI decision support tool for TC assessment could be a hybrid system based on expert knowledge (rule-based) using causal relationships as well as data-driven knowledge (statistical ML). Such a system would support rather than replace the clinical decision of the expert, which can be high, low or borderline TC depending on the integration of evidence. An explainable, question answering DSS is proposed for the evaluation of the reasoning of the expert making a TC assessment. The XAI system would use natural language and would be developed and tested on a corpus or dataset of explanations provided by persons of varying TC. Such a system would be adaptable to the circumstances of the testator and would utilize tacit knowledge for human empowerment.</p>
</sec>
<sec id="sec13" sec-type="data-availability">
<title>Data availability statement</title>
<p>The original contributions presented in the study are included in the article/<xref ref-type="supplementary-material" rid="SM1">Supplementary material</xref>, further inquiries can be directed to the corresponding author.</p>
</sec>
<sec id="sec14">
<title>Author contributions</title>
<p>AE conceptualized the general idea and drafted the manuscript. AE and JK conceptualized and wrote the specific ideas presented. All authors contributed to the article and approved the submitted version.</p>
</sec>
<sec id="sec41" sec-type="funding-information">
<title>Funding</title>
<p>Open access funds were provided by the Special Account for Research Grants of the National and Kapodistrian University of Athens.</p>
</sec>
<sec id="conf1" sec-type="COI-statement">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec id="sec100" sec-type="disclaimer">
<title>Publisher&#x2019;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
</body>
<back>
<sec id="sec16" sec-type="supplementary-material">
<title>Supplementary material</title>
<p>The Supplementary material for this article can be found online at: <ext-link xlink:href="https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1137792/full#supplementary-material" ext-link-type="uri">https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1137792/full#supplementary-material</ext-link></p>
<supplementary-material xlink:href="Data_Sheet_1.PDF" id="SM1" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</sec>
<ref-list>
<title>References</title>
<ref id="ref1"><label>1.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Arboleda-Fl&#x00F3;rez</surname> <given-names>J</given-names></name></person-group>. <article-title>Forensic psychiatry: contemporary scope, challenges and controversies</article-title>. <source>World Psychiatry</source>. (<year>2006</year>) <volume>5</volume>:<fpage>87</fpage>&#x2013;<lpage>91</lpage>. PMID: <pub-id pub-id-type="pmid">16946941</pub-id></citation></ref>
<ref id="ref2"><label>2.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brenkel</surname> <given-names>M</given-names></name> <name><surname>Whaley</surname> <given-names>K</given-names></name> <name><surname>Herrmann</surname> <given-names>N</given-names></name> <name><surname>Crawford</surname> <given-names>K</given-names></name> <name><surname>Hazan</surname> <given-names>E</given-names></name> <name><surname>Cardiff</surname> <given-names>L</given-names></name> <etal/></person-group>. <article-title>A case for the standardized assessment of testamentary capacity</article-title>. <source>Can Geriatr J</source>. (<year>2018</year>) <volume>21</volume>:<fpage>26</fpage>&#x2013;<lpage>31</lpage>. doi: <pub-id pub-id-type="doi">10.5770/cgj.21.283</pub-id>, PMID: <pub-id pub-id-type="pmid">29581818</pub-id></citation></ref>
<ref id="ref3"><label>3.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Purser</surname> <given-names>K</given-names></name> <name><surname>Lonie</surname> <given-names>J</given-names></name></person-group>. <article-title>Mapping dementia and cognitive decline in testamentary capacity</article-title>. <source>Int J Law Psychiatry</source>. (<year>2019</year>) <volume>66</volume>:<fpage>101450</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.ijlp.2019.101450</pub-id>, PMID: <pub-id pub-id-type="pmid">31706409</pub-id></citation></ref>
<ref id="ref4"><label>4.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shulman</surname> <given-names>KI</given-names></name> <name><surname>Peisah</surname> <given-names>C</given-names></name> <name><surname>Jacoby</surname> <given-names>R</given-names></name> <name><surname>Heinik</surname> <given-names>J</given-names></name> <name><surname>Finkel</surname> <given-names>S</given-names></name></person-group>. <article-title>Contemporaneous assessment of testamentary capacity</article-title>. <source>Int Psychogeriatr</source>. (<year>2009</year>) <volume>21</volume>:<fpage>433</fpage>&#x2013;<lpage>9</lpage>. doi: <pub-id pub-id-type="doi">10.1017/S1041610209008874</pub-id>, PMID: <pub-id pub-id-type="pmid">19323871</pub-id></citation></ref>
<ref id="ref5"><label>5.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shulman</surname> <given-names>KI</given-names></name> <name><surname>Himel</surname> <given-names>SG</given-names></name> <name><surname>Hull</surname> <given-names>IM</given-names></name> <name><surname>Peisah</surname> <given-names>C</given-names></name> <name><surname>Amodeo</surname> <given-names>S</given-names></name> <name><surname>Barnes</surname> <given-names>C</given-names></name></person-group>. <article-title>Banks V Goodfellow (1870): time to update the test for testamentary capacity</article-title>. <source>Can B Rev</source>. (<year>2017</year>) <volume>95</volume>:<fpage>251</fpage>.</citation></ref>
<ref id="ref6"><label>6.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marson</surname> <given-names>DC</given-names></name> <name><surname>Huthwaite</surname> <given-names>JS</given-names></name> <name><surname>Herbert</surname> <given-names>K</given-names></name></person-group>. <article-title>Testamentary capacity and undue influence in the elderly: a jurisprudent therapy perspective</article-title>. <source>Law Psychol Rev</source>. (<year>2004</year>) <volume>28</volume>:<fpage>71</fpage>.</citation></ref>
<ref id="ref7"><label>7.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mart</surname> <given-names>EG</given-names></name></person-group>. <article-title>Neuropsychological assessment of testamentary capacity and undue influence</article-title>. <source>Arch Clin Neuropsychol</source>. (<year>2016</year>) <volume>31</volume>:<fpage>554</fpage>&#x2013;<lpage>1</lpage>. doi: <pub-id pub-id-type="doi">10.1093/arclin/acw048</pub-id>, PMID: <pub-id pub-id-type="pmid">27480988</pub-id></citation></ref>
<ref id="ref8"><label>8.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lonie</surname> <given-names>J</given-names></name> <name><surname>Purser</surname> <given-names>K</given-names></name></person-group>. <article-title>Assessing testamentary capacity from the medical perspective</article-title>. <source>Aust Bar Rev</source>. (<year>2017</year>) <volume>44</volume>:<fpage>297</fpage>&#x2013;<lpage>7</lpage>.</citation></ref>
<ref id="ref9"><label>9.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shulman</surname> <given-names>KI</given-names></name> <name><surname>Cohen</surname> <given-names>CA</given-names></name> <name><surname>Kirsh</surname> <given-names>FC</given-names></name> <name><surname>Hull</surname> <given-names>IM</given-names></name> <name><surname>Champine</surname> <given-names>PR</given-names></name></person-group>. <article-title>Assessment of testamentary capacity and vulnerability to undue influence</article-title>. <source>Am J Psychiatry</source>. (<year>2007</year>) <volume>164</volume>:<fpage>722</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1176/ajp.2007.164.5.722</pub-id>, PMID: <pub-id pub-id-type="pmid">17475729</pub-id></citation></ref>
<ref id="ref10"><label>10.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Heinik</surname> <given-names>J</given-names></name> <name><surname>Werner</surname> <given-names>P</given-names></name> <name><surname>Lin</surname> <given-names>R</given-names></name></person-group>. <article-title>How do cognitively impaired elderly patients define &#x201C;statement&#x201D;: reliability and validity of the testament definition scale</article-title>. <source>Isr J Psychiatry Relat Sci</source>. (<year>1999</year>) <volume>36</volume>:<fpage>23</fpage>&#x2013;<lpage>8</lpage>. PMID: <pub-id pub-id-type="pmid">10389360</pub-id></citation></ref>
<ref id="ref11"><label>11.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Papageorgiou</surname> <given-names>SG</given-names></name> <name><surname>Voskou</surname> <given-names>P</given-names></name> <name><surname>Economou</surname> <given-names>A</given-names></name> <name><surname>Beratis</surname> <given-names>I</given-names></name> <name><surname>Douzenis</surname> <given-names>A</given-names></name></person-group>. <article-title>Testamentary capacity assessment tool (TCAT): a brief instrument for patients with dementia</article-title>. <source>J Alzheimers Dis</source>. (<year>2018</year>) <volume>61</volume>:<fpage>985</fpage>&#x2013;<lpage>4</lpage>. doi: <pub-id pub-id-type="doi">10.3233/JAD-170297</pub-id>, PMID: <pub-id pub-id-type="pmid">29254082</pub-id></citation></ref>
<ref id="ref12"><label>12.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bolognini</surname> <given-names>N</given-names></name> <name><surname>Gramegna</surname> <given-names>C</given-names></name> <name><surname>Esposito</surname> <given-names>A</given-names></name> <name><surname>Aiello</surname> <given-names>EN</given-names></name> <name><surname>Difonzo</surname> <given-names>T</given-names></name> <name><surname>Zago</surname> <given-names>S</given-names></name></person-group>. <article-title>The testamentary capacity assessment tool (TCAT): validation and normative data</article-title>. <source>Neurol Sci</source>. (<year>2022</year>) <volume>43</volume>:<fpage>2831</fpage>&#x2013;<lpage>8</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10072-021-05736-8</pub-id></citation></ref>
<ref id="ref13"><label>13.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Martin</surname> <given-names>RC</given-names></name> <name><surname>Gerstenecker</surname> <given-names>A</given-names></name> <name><surname>Hebert</surname> <given-names>K</given-names></name> <name><surname>Triebel</surname> <given-names>K</given-names></name> <name><surname>Marson</surname> <given-names>D</given-names></name></person-group>. <article-title>Assessment of testamentary capacity in older adults: description and initial validation of a standardized interview instrument</article-title>. <source>Arch Clin Neuropsychol</source>. (<year>2022</year>) <volume>37</volume>:<fpage>1133</fpage>&#x2013;<lpage>47</lpage>. doi: <pub-id pub-id-type="doi">10.1093/arclin/acac028</pub-id>, PMID: <pub-id pub-id-type="pmid">35596954</pub-id></citation></ref>
<ref id="ref14"><label>14.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marson</surname> <given-names>D</given-names></name></person-group>. <article-title>Topical issues in contemporaneous assessment of testamentary capacity</article-title>. <source>Psychol Inj and Law</source>. (<year>2022</year>) <volume>15</volume>:<fpage>357</fpage>&#x2013;<lpage>6</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s12207-022-09466-w</pub-id></citation></ref>
<ref id="ref15"><label>15.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gerstenecker</surname> <given-names>A</given-names></name> <name><surname>Martin</surname> <given-names>RC</given-names></name> <name><surname>Hebert</surname> <given-names>K</given-names></name> <name><surname>Triebel</surname> <given-names>K</given-names></name> <name><surname>Marson</surname> <given-names>DC</given-names></name></person-group>. <article-title>Cognitive correlates of impaired testamentary capacity in Alzheimer&#x2019;s dementia</article-title>. <source>Arch Clin Neuropsychol</source>. (<year>2022</year>) <volume>37</volume>:<fpage>1148</fpage>&#x2013;<lpage>57</lpage>. doi: <pub-id pub-id-type="doi">10.1093/arclin/acac034</pub-id>, PMID: <pub-id pub-id-type="pmid">35731016</pub-id></citation></ref>
<ref id="ref16"><label>16.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zago</surname> <given-names>S</given-names></name> <name><surname>Bolognini</surname> <given-names>N</given-names></name></person-group>. <article-title>Neuropsychological autopsy of testamentary capacity: methodology and issues in the elderly</article-title>. <source>Appl Neuropsychol Adult</source>. (<year>2022</year>) <volume>29</volume>:<fpage>570</fpage>&#x2013;<lpage>3</lpage>. doi: <pub-id pub-id-type="doi">10.1080/23279095.2020.1791869</pub-id>, PMID: <pub-id pub-id-type="pmid">32687452</pub-id></citation></ref>
<ref id="ref17"><label>17.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Balestrino</surname> <given-names>M</given-names></name> <name><surname>Fontana</surname> <given-names>P</given-names></name> <name><surname>Terzuoli</surname> <given-names>S</given-names></name> <name><surname>Volpe</surname> <given-names>S</given-names></name> <name><surname>Inglese</surname> <given-names>ML</given-names></name> <name><surname>Cocito</surname> <given-names>L</given-names></name></person-group>. <article-title>Altered handwriting suggests cognitive impairment and may be relevant to posthumous evaluation</article-title>. <source>J Forensic Sci</source>. (<year>2012</year>) <volume>57</volume>:<fpage>1252</fpage>&#x2013;<lpage>8</lpage>. doi: <pub-id pub-id-type="doi">10.1111/j.1556-4029.2012.02131.x</pub-id>, PMID: <pub-id pub-id-type="pmid">22515315</pub-id></citation></ref>
<ref id="ref18"><label>18.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ferguson</surname> <given-names>A</given-names></name> <name><surname>Worrall</surname> <given-names>L</given-names></name> <name><surname>McPhee</surname> <given-names>J</given-names></name> <name><surname>Buskell</surname> <given-names>R</given-names></name> <name><surname>Armstrong</surname> <given-names>E</given-names></name> <name><surname>Togher</surname> <given-names>L</given-names></name></person-group>. <article-title>Case study testamentary capacity and aphasia: a descriptive case report with implications for clinical practice</article-title>. <source>Aphasiology</source>. (<year>2003</year>) <volume>17</volume>:<fpage>965</fpage>&#x2013;<lpage>06</lpage>. doi: <pub-id pub-id-type="doi">10.1080/02687030344000337</pub-id></citation></ref>
<ref id="ref19"><label>19.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sherratt</surname> <given-names>S</given-names></name></person-group>. <article-title>Court access for people with aphasia: a review of case law from 1915 to 2021</article-title>. <source>Perspect ASHA Spec Int Groups</source>. (<year>2022</year>) <volume>7</volume>:<fpage>1698</fpage>&#x2013;<lpage>16</lpage>. doi: <pub-id pub-id-type="doi">10.1044/2022_PERSP-21-00294</pub-id></citation></ref>
<ref id="ref20"><label>20.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Suleman</surname> <given-names>S</given-names></name> <name><surname>Hopper</surname> <given-names>T</given-names></name></person-group>. <article-title>Decision-making capacity and aphasia: speech-language pathologists&#x2019; perspectives</article-title>. <source>Aphasiology</source>. (<year>2016</year>) <volume>30</volume>:<fpage>381</fpage>&#x2013;<lpage>5</lpage>. doi: <pub-id pub-id-type="doi">10.1080/02687038.2015.1065468</pub-id></citation></ref>
<ref id="ref21"><label>21.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Weisbord</surname> <given-names>RK</given-names></name> <name><surname>Horton</surname> <given-names>D</given-names></name></person-group>. <article-title>The future of testamentary capacity</article-title>. <source>Wash Lee L Rev</source>. (<year>2022</year>) <volume>79</volume>:<fpage>609</fpage>.</citation></ref>
<ref id="ref22"><label>22.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fonseca</surname> <given-names>J</given-names></name> <name><surname>Raposo</surname> <given-names>A</given-names></name> <name><surname>Martins</surname> <given-names>IP</given-names></name></person-group>. <article-title>Cognitive functioning in chronic post-stroke aphasia</article-title>. <source>Appl Neuropsychol Adult</source>. (<year>2019</year>) <volume>26</volume>:<fpage>355</fpage>&#x2013;<lpage>4</lpage>. doi: <pub-id pub-id-type="doi">10.1080/23279095.2018.1429442</pub-id></citation></ref>
<ref id="ref23"><label>23.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grimm</surname> <given-names>PW</given-names></name> <name><surname>Grossman</surname> <given-names>MR</given-names></name> <name><surname>Cormack</surname> <given-names>GV</given-names></name></person-group>. <article-title>Artificial intelligence as evidence</article-title>. <source>Nw J Tech Intel Prop</source>. (<year>2021</year>) <volume>19</volume>:<fpage>9</fpage>.</citation></ref>
<ref id="ref24"><label>24.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Garb</surname> <given-names>HN</given-names></name> <name><surname>Wood</surname> <given-names>JM</given-names></name></person-group>. <article-title>Methodological advances in statistical prediction</article-title>. <source>Psychol Assess</source>. (<year>2019</year>) <volume>31</volume>:<fpage>1456</fpage>&#x2013;<lpage>66</lpage>. doi: <pub-id pub-id-type="doi">10.1037/pas0000673</pub-id></citation></ref>
<ref id="ref25"><label>25.</label> <citation citation-type="other"><person-group person-group-type="author"><name><surname>Buchanan</surname> <given-names>BG</given-names></name> <name><surname>Shortliffe</surname> <given-names>EH</given-names></name></person-group>. <article-title>Rule based expert systems: the MYCIN experiments of the Stanford heuristic programming project</article-title>. <comment>The Addison-Wesley series in artificial intelligence</comment> (<year>1984</year>).</citation></ref>
<ref id="ref26"><label>26.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kontos</surname> <given-names>J</given-names></name></person-group>. <article-title>ARISTA knowledge engineering with scientific texts</article-title>. <source>Inf Softw Technol</source>. (<year>1992</year>) <volume>34</volume>:<fpage>611</fpage>&#x2013;<lpage>6</lpage>. doi: <pub-id pub-id-type="doi">10.1016/0950-5849(92)90138-F</pub-id></citation></ref>
<ref id="ref27"><label>27.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll1">Broad Agency Announcement</collab></person-group>. <article-title>Explainable AI (XAI)</article-title> (<year>2016</year>) <comment>Available at: </comment><ext-link xlink:href="https://www.darpa.mil/attachments/DARPA-BAA-16-53.pdf" ext-link-type="uri">https://www.darpa.mil/attachments/DARPA-BAA-16-53.pdf</ext-link> (accessed December 12, 2022).</citation></ref>
<ref id="ref28"><label>28.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Confalonieri</surname> <given-names>R</given-names></name> <name><surname>Coba</surname> <given-names>L</given-names></name> <name><surname>Wagner</surname> <given-names>B</given-names></name> <name><surname>Besold</surname> <given-names>TR</given-names></name></person-group>. <article-title>A historical perspective of explainable artificial intelligence</article-title>. <source>Wiley Interdiscip Rev: Data Min Knowl Discov</source>. (<year>2021a</year>) <volume>11</volume>:<fpage>e1391</fpage>. doi: <pub-id pub-id-type="doi">10.1002/widm.1391</pub-id></citation></ref>
<ref id="ref29"><label>29.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marcus</surname> <given-names>G</given-names></name></person-group>. <article-title>Deep learning: a critical appraisal</article-title>. <source>arXiv preprint arXiv</source>. (<year>2018</year>) <volume>1801</volume>:<fpage>00631</fpage>.</citation></ref>
<ref id="ref30"><label>30.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Constantinou</surname> <given-names>AC</given-names></name> <name><surname>Freestone</surname> <given-names>M</given-names></name> <name><surname>Marsh</surname> <given-names>W</given-names></name> <name><surname>Coid</surname> <given-names>J</given-names></name></person-group>. <article-title>Causal inference for violence risk management and decision support in forensic psychiatry</article-title>. <source>Decis Support Syst</source>. (<year>2015</year>) <volume>80</volume>:<fpage>42</fpage>&#x2013;<lpage>55</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.dss.2015.09.006</pub-id></citation></ref>
<ref id="ref31"><label>31.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gou</surname> <given-names>N</given-names></name> <name><surname>Xiang</surname> <given-names>Y</given-names></name> <name><surname>Zhou</surname> <given-names>J</given-names></name> <name><surname>Zhang</surname> <given-names>S</given-names></name> <name><surname>Zhong</surname> <given-names>S</given-names></name> <name><surname>Lu</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>Identification of violent patients with schizophrenia using a hybrid machine learning approach at the individual level</article-title>. <source>Psychiatry Res</source>. (<year>2021</year>) <volume>306</volume>:<fpage>114294</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.psychres.2021.114294</pub-id>, PMID: <pub-id pub-id-type="pmid">34823086</pub-id></citation></ref>
<ref id="ref32"><label>32.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sonnweber</surname> <given-names>M</given-names></name> <name><surname>Lau</surname> <given-names>S</given-names></name> <name><surname>Kirchebner</surname> <given-names>J</given-names></name></person-group>. <article-title>Violent and non-violent offending in patients with schizophrenia: exploring influences and differences via machine learning</article-title>. <source>Compr Psychiatry</source>. (<year>2021</year>) <volume>107</volume>:<fpage>152238</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.comppsych.2021.152238</pub-id>, PMID: <pub-id pub-id-type="pmid">33721584</pub-id></citation></ref>
<ref id="ref33"><label>33.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hofmann</surname> <given-names>LA</given-names></name> <name><surname>Lau</surname> <given-names>S</given-names></name> <name><surname>Kirchebner</surname> <given-names>J</given-names></name></person-group>. <article-title>Advantages of machine learning in forensic psychiatric research&#x2014;uncovering the complexities of aggressive behavior in schizophrenia</article-title>. <source>Appl Sci</source>. (<year>2022</year>) <volume>12</volume>:<fpage>819</fpage>. doi: <pub-id pub-id-type="doi">10.3390/app12020819</pub-id></citation></ref>
<ref id="ref34"><label>34.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mosteiro</surname> <given-names>P</given-names></name> <name><surname>Rijcken</surname> <given-names>E</given-names></name> <name><surname>Zervanou</surname> <given-names>K</given-names></name> <name><surname>Kaymak</surname> <given-names>U</given-names></name> <name><surname>Scheepers</surname> <given-names>F</given-names></name> <name><surname>Spruit</surname> <given-names>M</given-names></name></person-group>. <article-title>Machine learning for violence risk assessment using Dutch clinical notes</article-title>. <source>arXiv preprint</source>. (<year>2022</year>) <volume>13535</volume>:<fpage>aXiv:2204</fpage>. doi: <pub-id pub-id-type="doi">10.48550/arXiv.2204.13535</pub-id></citation></ref>
<ref id="ref35"><label>35.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tay</surname> <given-names>JL</given-names></name> <name><surname>Li</surname> <given-names>Z</given-names></name> <name><surname>Sim</surname> <given-names>K</given-names></name></person-group>. <article-title>Effectiveness of artificial intelligence methods in personalized aggression risk prediction within inpatient psychiatric treatment settings&#x2014;a systematic review</article-title>. <source>J Pers Med</source>. (<year>2022</year>) <volume>12</volume>:<fpage>1470</fpage>. doi: <pub-id pub-id-type="doi">10.3390/jpm12091470</pub-id>, PMID: <pub-id pub-id-type="pmid">36143255</pub-id></citation></ref>
<ref id="ref36"><label>36.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tortora</surname> <given-names>L</given-names></name> <name><surname>Meynen</surname> <given-names>G</given-names></name> <name><surname>Bijlsma</surname> <given-names>J</given-names></name> <name><surname>Tronci</surname> <given-names>E</given-names></name> <name><surname>Ferracuti</surname> <given-names>S</given-names></name></person-group>. <article-title>Neuroprediction and ai in forensic psychiatry and criminal justice: a neurolaw perspective</article-title>. <source>Front Psychol</source>. (<year>2020</year>) <volume>11</volume>:<fpage>220</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fpsyg.2020.00220</pub-id>, PMID: <pub-id pub-id-type="pmid">32256422</pub-id></citation></ref>
<ref id="ref37"><label>37.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Soini</surname> <given-names>EJ</given-names></name> <name><surname>Rissanen</surname> <given-names>T</given-names></name> <name><surname>Tiihonen</surname> <given-names>J</given-names></name> <name><surname>Hodgins</surname> <given-names>S</given-names></name> <name><surname>Eronen</surname> <given-names>M</given-names></name> <name><surname>Ryyn&#x00E4;nen</surname> <given-names>OP</given-names></name></person-group>. <article-title>Predicting forensic admission among the mentally ill in a multinational setting: a Bayesian modelling approach</article-title>. <source>Data Knowl Eng</source>. (<year>2009</year>) <volume>68</volume>:<fpage>1427</fpage>&#x2013;<lpage>40</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.datak.2009.07.005</pub-id></citation></ref>
<ref id="ref38"><label>38.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Watts</surname> <given-names>D</given-names></name> <name><surname>Moulden</surname> <given-names>H</given-names></name> <name><surname>Mamak</surname> <given-names>M</given-names></name> <name><surname>Upfold</surname> <given-names>C</given-names></name> <name><surname>Chaimowitz</surname> <given-names>G</given-names></name> <name><surname>Kapczinski</surname> <given-names>F</given-names></name></person-group>. <article-title>Predicting offenses among individuals with psychiatric disorders-a machine learning approach</article-title>. <source>J Psychiatr Res</source>. (<year>2021</year>) <volume>138</volume>:<fpage>146</fpage>&#x2013;<lpage>4</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.jpsychires.2021.03.026</pub-id>, PMID: <pub-id pub-id-type="pmid">33857785</pub-id></citation></ref>
<ref id="ref39"><label>39.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Berly</surname> <given-names>A</given-names></name> <name><surname>Manaouil</surname> <given-names>C</given-names></name> <name><surname>Dervaux</surname> <given-names>A</given-names></name></person-group>. <article-title>L&#x2019;intelligence artificielle peut-elle aider &#x00E0; estimer le risque de r&#x00E9;cidive dans les comportements violents?</article-title> <source>M&#x00E9;d et Droit</source>. (<year>2020</year>) <volume>2020</volume>:<fpage>105</fpage>&#x2013;<lpage>9</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.meddro.2020.02.001</pub-id></citation></ref>
<ref id="ref40"><label>40.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Goodman</surname> <given-names>B</given-names></name> <name><surname>Flaxman</surname> <given-names>S</given-names></name></person-group>. <article-title>European Union regulations on algorithmic decision-making and a &#x201C;right to explanation&#x201D;</article-title>. <source>AI Mag</source>. (<year>2017</year>) <volume>38</volume>:<fpage>50</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1609/aimag.v38i3.2741</pub-id></citation></ref>
<ref id="ref41"><label>41.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chou</surname> <given-names>YL</given-names></name> <name><surname>Moreira</surname> <given-names>C</given-names></name> <name><surname>Bruza</surname> <given-names>P</given-names></name> <name><surname>Ouyang</surname> <given-names>C</given-names></name> <name><surname>Jorge</surname> <given-names>J</given-names></name></person-group>. <article-title>Counterfactuals and causability in explainable artificial intelligence: theory, algorithms, and applications</article-title>. <source>Inform Fusion</source>. (<year>2022</year>) <volume>81</volume>:<fpage>59</fpage>&#x2013;<lpage>83</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.inffus.2021.11.003</pub-id></citation></ref>
<ref id="ref42"><label>42.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hagendorff</surname> <given-names>T</given-names></name></person-group>. <article-title>The ethics of AI ethics: an evaluation of guidelines</article-title>. <source>Minds Mach</source>. (<year>2020</year>) <volume>30</volume>:<fpage>99</fpage>&#x2013;<lpage>06</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11023-020-09517-8</pub-id></citation></ref>
<ref id="ref43"><label>43.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Veale</surname> <given-names>M</given-names></name></person-group>. <article-title>A critical take on the policy recommendations of the EU high-level expert group on artificial intelligence</article-title>. <source>Eur J Risk Regul</source>. (<year>2020</year>) <volume>11</volume>:<fpage>1</fpage>&#x2013;<lpage>10</lpage>. doi: <pub-id pub-id-type="doi">10.1017/err.2019.65</pub-id></citation></ref>
<ref id="ref44"><label>44.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liao</surname> <given-names>QV</given-names></name> <name><surname>Varshney</surname> <given-names>KR</given-names></name></person-group>. <article-title>Human-centered explainable ai (xai): from algorithms to user experiences</article-title>. <source>arXiv preprint arXiv</source>. (<year>2021</year>) <volume>2110</volume>:<fpage>10790</fpage>. doi: <pub-id pub-id-type="doi">10.48550/arXiv.2110.10790</pub-id></citation></ref>
<ref id="ref45"><label>45.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schoonderwoerd</surname> <given-names>TA</given-names></name> <name><surname>Jorritsma</surname> <given-names>W</given-names></name> <name><surname>Neerincx</surname> <given-names>MA</given-names></name> <name><surname>Van Den Bosch</surname> <given-names>K</given-names></name></person-group>. <article-title>Human-centered XAI: developing design patterns for explanations of clinical decision support systems</article-title>. <source>Int J Hum Comput Stud</source>. (<year>2021</year>) <volume>154</volume>:<fpage>102684</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.ijhcs.2021.102684</pub-id></citation></ref>
<ref id="ref46"><label>46.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nowrangi</surname> <given-names>MA</given-names></name> <name><surname>Sevinc</surname> <given-names>G</given-names></name> <name><surname>Kamath</surname> <given-names>V</given-names></name></person-group>. <article-title>Synthetic review of financial capacity in cognitive disorders: foundations, interventions, and innovations</article-title>. <source>Curr Geri Rep</source>. (<year>2019</year>) <volume>8</volume>:<fpage>257</fpage>&#x2013;<lpage>4</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s13670-019-00304-7</pub-id>, PMID: <pub-id pub-id-type="pmid">33344109</pub-id></citation></ref>
<ref id="ref47"><label>47.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Markou</surname> <given-names>C</given-names></name> <name><surname>Hands</surname> <given-names>L</given-names></name></person-group>. <article-title>Capacitas ex machina: are computerised assessments of mental capacity a benchmark or 'red line' for artificial intelligence</article-title> In: <person-group person-group-type="editor"><name><surname>Deakin</surname> <given-names>S</given-names></name> <name><surname>Markou</surname> <given-names>C</given-names></name></person-group>, editors. <source>Is law computable? Critical perspectives on law + artificial intelligence</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Hart Publishing</publisher-name> (<year>2020</year>)</citation></ref>
<ref id="ref48"><label>48.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Confalonieri</surname> <given-names>R</given-names></name> <name><surname>Weyde</surname> <given-names>T</given-names></name> <name><surname>Besold</surname> <given-names>TR</given-names></name> <name><surname>del Prado Mart&#x00ED;n</surname> <given-names>FM</given-names></name></person-group>. <article-title>Using ontologies to enhance human understandability of global post-hoc explanations of black-box models</article-title>. <source>Artif Intell</source>. (<year>2021b</year>) <volume>296</volume>:<fpage>103471</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.artint.2021.103471</pub-id></citation></ref>
<ref id="ref49"><label>49.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kontos</surname> <given-names>J</given-names></name></person-group>. <article-title>Human-system Interface with explanation of actions for autonomous anti-UAV systems</article-title>. <source>Int J Artif Intell Appl</source>. (<year>2021</year>) <volume>12</volume>:<fpage>51</fpage>&#x2013;<lpage>8</lpage>. doi: <pub-id pub-id-type="doi">10.5121/ijaia.2021.12404</pub-id></citation></ref>
<ref id="ref50"><label>50.</label> <citation citation-type="gov"><person-group person-group-type="author"><name><surname>Pfeiff</surname> <given-names>JS</given-names></name></person-group>. <article-title>Constructing and applying explanatory models in clinical psychology</article-title> (<comment>Doctoral dissertation</comment>, <publisher-name>Hannover: Institutionelles Repositorium der Leibniz Universit&#x00E4;t Hannover</publisher-name>) (<year>2020</year>). <volume>265</volume>. doi: <pub-id pub-id-type="doi">10.15488/11363</pub-id></citation></ref>
<ref id="ref51"><label>51.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bryant</surname> <given-names>C</given-names></name> <name><surname>Yuan</surname> <given-names>Z</given-names></name> <name><surname>Qorib</surname> <given-names>MR</given-names></name> <name><surname>Cao</surname> <given-names>H</given-names></name> <name><surname>Ng</surname> <given-names>HT</given-names></name> <name><surname>Briscoe</surname> <given-names>T</given-names></name></person-group>. <article-title>Grammatical error correction: a survey of the state of the art</article-title>. <source>arXiv preprint arXiv</source>. (<year>2022</year>) <volume>2211</volume>:<fpage>05166</fpage>. doi: <pub-id pub-id-type="doi">10.48550/arXiv.2211.05166</pub-id></citation></ref>
<ref id="ref52"><label>52.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Kontos</surname> <given-names>J</given-names></name> <name><surname>Kasda</surname> <given-names>P</given-names></name></person-group>. <source>Artificial intelligence professor John Kontos needles poly Kasda&#x2019;s conscious eye</source>. <publisher-loc>Athens, Greece</publisher-loc>: <publisher-name>Notios Anemos</publisher-name> (<year>2015</year>).</citation></ref>
<ref id="ref53"><label>53.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rahman</surname> <given-names>A</given-names></name> <name><surname>Halim</surname> <given-names>Z</given-names></name></person-group>. <article-title>Identifying dominant emotional state using handwriting and drawing samples by fusing features</article-title>. <source>Appl Intell</source>. (<year>2023</year>) <volume>53</volume>:<fpage>2798</fpage>&#x2013;<lpage>814</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10489-022-03552-x</pub-id></citation></ref>
</ref-list>
</back>
</article>