<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Comput. Neurosci.</journal-id>
<journal-title>Frontiers in Computational Neuroscience</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Comput. Neurosci.</abbrev-journal-title>
<issn pub-type="epub">1662-5188</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fncom.2025.1595278</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neuroscience</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Neural correspondence to spectrum of environmental uncertainty in multiple-cue probability judgment system with time delay</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Chang</surname> <given-names>Yoo-Sang</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2223026/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/methodology/"/>
<role content-type="https://credit.niso.org/contributor-roles/conceptualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-original-draft/"/>
<role content-type="https://credit.niso.org/contributor-roles/visualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Seong</surname> <given-names>Younho</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2296987/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/methodology/"/>
<role content-type="https://credit.niso.org/contributor-roles/conceptualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Yi</surname> <given-names>Sun</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Industrial and Systems Engineering, North Carolina A&#x0026;T State University</institution>, <addr-line>Greensboro, NC</addr-line>, <country>United States</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Mechanical Engineering, North Carolina A&#x0026;T State University</institution>, <addr-line>Greensboro, NC</addr-line>, <country>United States</country></aff>
<author-notes>
<fn fn-type="edited-by" id="fn0001">
<p>Edited by: Azzurra Invernizzi, Icahn School of Medicine at Mount Sinai, United States</p>
</fn>
<fn fn-type="edited-by" id="fn0002">
<p>Reviewed by: Daniele Corbo, University of Brescia, Italy</p>
<p>Marialaura De Grazia, University of Pavia, Italy</p>
</fn>
<corresp id="c001">&#x002A;Correspondence: Younho Seong, <email>yseong@ncat.edu</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>17</day>
<month>07</month>
<year>2025</year>
</pub-date>
<pub-date pub-type="collection">
<year>2025</year>
</pub-date>
<volume>19</volume>
<elocation-id>1595278</elocation-id>
<history>
<date date-type="received">
<day>17</day>
<month>03</month>
<year>2025</year>
</date>
<date date-type="accepted">
<day>02</day>
<month>06</month>
<year>2025</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2025 Chang, Seong and Yi.</copyright-statement>
<copyright-year>2025</copyright-year>
<copyright-holder>Chang, Seong and Yi</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>Despite state-of-the-art technologies like artificial intelligence, human judgment is critically essential in cooperative systems, such as the multi-agent system (MAS), which collect information among agents based on multiple-cue judgment. Human agents can prevent impaired situational awareness of automated agents by confirming situations under environmental uncertainty. System error caused by uncertainty can result in an unreliable system environment, and this environment affects the human agent, resulting in non-optimal decision-making in MAS. Thus, it is necessary to know how human behavior is changed to capture system reliability under uncertainty. Another issue affecting MAS is time delay, which can delay agent information transfer, resulting in low performance and instability. However, it is difficult to find studies on the influence of time delay on human agents. This study is about understanding the human decision-making process under a specific system reliability environment by uncertainty with time delay. We used concepts of expected and unexpected uncertainty to implement reliability of the system usage environment with three types of time delay conditions: no time delay, regular time delay, and irregular time delay conditions. We used electroencephalogram (EEG) for human cognitive neural mechanisms in multiple-cue judgment systems to understand human decision-making. In the reliability of system usage environment, the unreliable system environment significantly creates less memory load by less utilization of system rules for decision-making. In terms of time delay, delayed information delivery does not significantly affect memory load for decision-making.</p>
</abstract>
<kwd-group>
<kwd>decision-making</kwd>
<kwd>electroencephalogram</kwd>
<kwd>expected uncertainty</kwd>
<kwd>unexpected uncertainty</kwd>
<kwd>cognitive process</kwd>
<kwd>multi-agent system</kwd>
<kwd>time delay</kwd>
</kwd-group>
<contract-num rid="cn1">DE-NA0004189</contract-num>
<contract-sponsor id="cn1">U.S. Department of Energy<named-content content-type="fundref-id">10.13039/100000015</named-content></contract-sponsor>
<counts>
<fig-count count="10"/>
<table-count count="4"/>
<equation-count count="0"/>
<ref-count count="43"/>
<page-count count="14"/>
<word-count count="8601"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="sec1">
<label>1</label>
<title>Introduction</title>
<p>Humans play an essential role in multi-agent systems (MASs), even though artificial intelligence technologies are being developed. MAS is a cooperative system based on the interaction among agents to solve problems (<xref ref-type="bibr" rid="ref5">Balaji and Srinivasan, 2010</xref>). A representative example of MAS is the defense system with autonomous agents, such as unmanned ground vehicles (UGVs) or unmanned aerial vehicles (UAVs). The MAS-based defense system is operated by a multiple-cue judgment system by human agents based on obtained information (multiple cues) for appropriate decisions in a dynamic environment (<xref ref-type="bibr" rid="ref33">Sokolowski, 2003</xref>).</p>
<p>The human agent is essential to MAS because it makes decisions based on evaluating the current system&#x2019;s performance. The human agent inspects the system performance by comparing information from autonomous agents and real-world results to check whether the system shows valid performance for the goal. The system performance can fluctuate due to system errors such as communication network or sensor errors in a dynamic environment. The fluctuating system performance can cause invalid system performance by receiving and transmitting invalid information under uncertain environments. The human agent makes decisions after examination of system performance, whether it can make valid or invalid system performance by checking to make decisions for aimed output. Based on these experiences of system usage, the human agent tries to avoid system failure due to impaired situational awareness caused by system error of autonomous agents in MAS (<xref ref-type="bibr" rid="ref38">Tweedale et al., 2007</xref>).</p>
<sec id="sec2">
<label>1.1</label>
<title>Human decision-making under environmental uncertainty</title>
<p>Despite inappropriate decision-making avoidance by the human agent, environmental uncertainty affects the MAS performance because of an unreliable system usage environment. MAS operates in dynamic environments with unexpected situations. The unstable environments cause system errors, such as communication network errors among agents or physical sensor errors due to geographic or weather conditions (<xref ref-type="bibr" rid="ref41">Yiu et al., 2022</xref>). The incorrect information due to the unstable environments causes unreliable system usage contexts, resulting in loss by experiences of invalid system performance to human agents.</p>
<p>There are two reasons for the non-optimal choice of uncertainty from the neurological perspective. The first reason is the neural state changes for every decision-making trial, such as the atmosphere under a specific environment. The second reason is the noise of input observation, such as incorrect information reception (<xref ref-type="bibr" rid="ref16">Geng et al., 2020</xref>). For the characteristics of sensitive neural states of humans, the non-optimal decision-making can be worse by the negative effect of unreliable system usage experiences. Thus, studies of the human cognitive process are needed to deal with the impact of environmental uncertainty in MAS with human agents&#x2019; decision-making characteristics for high-level MAS.</p>
</sec>
<sec id="sec3">
<label>1.2</label>
<title>Decision-making under time delay of information delivery</title>
<p>Time delays can influence the performance of MAS. The communication network in MAS is operated based on a distributed network system to control cooperative autonomous agents with information sharing. During information sharing, a time delay occurs when the communication network system has issues, such as loss of packet, uncertain system mode, or unexpected interference among agents in MAS (<xref ref-type="bibr" rid="ref43">Zhang et al., 2018</xref>). The time delay by communication system errors can cause low system quality, resulting in performance instability due to system degradation (<xref ref-type="bibr" rid="ref35">Sumpter et al., 2019</xref>) and serious problems, such as system breakdown (<xref ref-type="bibr" rid="ref43">Zhang et al., 2018</xref>). There are some studies on improving communication network performances of MAS performances considering time delay in terms of mathematical models among autonomous agents (<xref ref-type="bibr" rid="ref24">Li et al., 2012</xref>; <xref ref-type="bibr" rid="ref25">Li et al., 2018</xref>; <xref ref-type="bibr" rid="ref37">Trentelman et al., 2013</xref>). <xref ref-type="bibr" rid="ref35">Sumpter et al. (2019)</xref> considered trust aspects based on topology and time delay for agent communication to build a mathematical MAS model. In contrast, it is difficult to find studies about human cognitive characteristics under time delay in MAS. This study focuses on time delay regarding information reception by the human agent. <xref ref-type="fig" rid="fig1">Figure 1</xref> shows the concept of time delay in information delivery.</p>
<fig position="float" id="fig1">
<label>Figure 1</label>
<caption>
<p>Time delay concept in information delivery.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g001.tif"/>
</fig>
<p>In this study, the time delay is assumed to mean that human agents can receive at the time &#x201C;t&#x201D; if there is no delayed information delivery from each agent. In contrast, they will receive information at time &#x201C;t&#x202F;+&#x202F;d&#x201D; if they receive one information with &#x201C;d&#x201D; time delay after receiving other information at the time &#x201C;t.&#x201D; Based on the experience of time delay, we expect an insight into how the human cognitive process is performed to deal with the delay in information delivery in MAS.</p>
</sec>
<sec id="sec4">
<label>1.3</label>
<title>Expected and unexpected uncertainties</title>
<p>We used concepts of expected and unexpected uncertainties to implement unreliable system usage environments by uncertainty, as suggested by <xref ref-type="bibr" rid="ref42">Yu and Dayan (2005)</xref>. The expected and unexpected uncertainty is made using stimulus&#x2013;response&#x2013;outcome (S-R-O) rules. The agent can get benefits or losses (outcome) through learning about the relationship between stimulus and response. For example, they see different colors of balls, such as green and blue. These balls are stimuli (S) information before decision-making. This decision-making, choosing one color ball, is response (R), which leads to outcomes (O). According to the response, the outcome (O) is getting (positive) or loss (negative) of one dollar. For example, if they choose green ball and the outcome is getting one dollar, it gives positive feedback. Thus, they can learn that green balls are associated with benefits. In contrast, if they lose one dollar by choosing the blue ball, the decision policy of negative feedback is associated with the blue ball. The experience ratio of positive or negative outcomes determines the expected and unexpected uncertainty. The expected uncertainty can occur when they observe a lot of negative feedback, such as invalid outcomes. In contrast, unexpected uncertainty can occur through many observations of positive feedback, such as valid outcomes. So, the expected uncertainty is knowing that the specific judgment system or policy is unstable and unreliable in getting valid results by observing many invalid results (e.g., 50% invalid system performance observations). If the decision-maker is under the expected uncertainty, they will not use the current decision policy to explore another policy for their benefit. In contrast, unexpected uncertainty gives intense violated experiences by less invalid result observation (e.g., 10% invalid system performance observations). Unexpected uncertainty occurs in a reliable decision-making environment, so they will exploit current decision-policy rather than decision-policy under expected uncertainty (<xref ref-type="bibr" rid="ref6">Bland and Schaefer, 2012</xref>). Based on the concept of expected and unexpected uncertainty, we set up the experience of environmental uncertainty by showing intended invalid system performances to build reliable or unreliable system usage performances based on expected and unexpected uncertainty concepts, as shown in <xref ref-type="fig" rid="fig2">Figure 2</xref>.</p>
<fig position="float" id="fig2">
<label>Figure 2</label>
<caption>
<p>The explanation of expected and unexpected uncertainty concepts.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g002.tif"/>
</fig>
</sec>
<sec id="sec5">
<label>1.4</label>
<title>Relationship between trust in system and expected and unexpected uncertainties</title>
<p>The expected and unexpected uncertainties are related to trustworthiness in automated systems with humans. In automation, trust refers to the attitude of an agent to use or not use an automated system (<xref ref-type="bibr" rid="ref23">Lee and See, 2004</xref>). Trust in automation is a critical issue. For example, suppose human agents would not use information from an automated system, such as changing the railroad to avoid a collision with a forward train despite correct information because of distrust of the system. In that case, they will not change the railroad. Then, collisions happen, resulting in system breakdown and loss of lives (<xref ref-type="bibr" rid="ref12">Dzindolet et al., 2003</xref>). Thus, to prevent the disuse of the system in appropriate situations, trustworthiness should be measured by human agents.</p>
<p>There are studies that seek to understand human agents&#x2019; cognitive states regarding the experience of trust or distrust by adjusting invalid results. <xref ref-type="bibr" rid="ref30">Oh et al. (2020)</xref> implemented invalid result stimulus by incorrect action, as invalid system performances. For example, the participants decided to drive to the third lane because the vehicle existed in front of the first and second lanes. In this case, the invalid result is that the vehicle was moved to the first lane, resulting in a collision by invalid system action even though they controlled vehicles driving to the third lane. <xref ref-type="bibr" rid="ref8">Choo and Nam (2022)</xref> suggested a classifier for trust and distrust situations by showing invalid results based on given information to solve the problem with decision-making policy.</p>
<p>In a study of expected and unexpected uncertainty, <xref ref-type="bibr" rid="ref18">Kogler et al. (2017)</xref> experimented with understanding cognitive states in expected or unexpected uncertainty using invalid results. They control the ratio of invalid results to make expected and unexpected uncertainty situations. For example, a cue, such as a star, circle, or triangle, is given for decision-making. The valid result is shown with rewards if they press the buttons related to each shape. For example, the reward (money) is increased by pressing the star shape button when they see the star shape. The invalid result is a loss of points by pressing the star shape when the cue is also a star shape. The expected uncertainty comprised 50% of invalid results, and the unexpected uncertainty situation comprised 20% of the invalid results after decision-making. Like the studies of trust in automation and expected and unexpected uncertainty, trustworthiness or expected and unexpected uncertainty depends on the ratio of invalid results. Therefore, this study focuses on understanding human cognitive states under different reliability levels of system usage environment (decision-making environment) under expected and unexpected uncertainty situations in multiple-cue judgment systems with a time delay of information delivery.</p>
</sec>
<sec id="sec6">
<label>1.5</label>
<title>Electroencephalogram</title>
<p>Electroencephalogram is a measure of minute brain signals from the scalp. There are two types of EEG devices. First is invasive EEG, which measures the signal through implanted devices with brain surgery. The second is a non-invasive EEG device that wears a cap with electrodes to record EEG data. Non-invasive EEG devices show lower EEG data quality than invasive EEG devices, but safety is much better than invasive EEG devices because there is no surgery to record the signal (<xref ref-type="bibr" rid="ref1">Ahmadian et al., 2013</xref>). Non-invasive EEG data records brain signals through electrodes. These electrodes are mainly located on five different lobes, such as frontal (F), central lobe (C), parietal lobe (P), temporal lobe (T), and occipital lobe (O). The electrode layout is up to the study goal, where they want to know how the neural activity operates (<xref ref-type="bibr" rid="ref21">Kumar and Bhuvaneswari, 2012</xref>; <xref ref-type="bibr" rid="ref26">Lim et al., 2018</xref>).</p>
<p>The measured signals are used to understand neural activity by analyzing the amplitude change in the signal. EEG data is interpreted using different analysis domains. The first domain is the time domain based on the amplitude of potential value in time windows. The representative time domain analysis is the event-related potential (ERP), which is an analysis of amplitude peaking when a human experiences an event (stimulus) (<xref ref-type="bibr" rid="ref11">Du et al., 2013</xref>). The second is frequency domain analysis through a transformation from a time window to a frequency window. The frequency domain analysis is used to know brain states, such as sleep, relaxation, or anxiety according to the different frequency bands (<xref ref-type="bibr" rid="ref13">Firoz et al., 2022</xref>). The last analysis domain is the time&#x2013;frequency domain, which is used to understand brain activity through frequency change for each time window (<xref ref-type="bibr" rid="ref2">Al-Fahoum and Al-Fraihat, 2014</xref>). This study used a non-invasive EEG device with ERP analysis as time domain analysis.</p>
</sec>
<sec id="sec7">
<label>1.6</label>
<title>Event-related potential studies in human decision-making</title>
<p>For analysis of neural states under uncertainty in a multiple-cue judgment system, we used the ERP. ERP is a temporal analysis that measures amplitude (potential value) in a given the time window. Generally, neural states are interpreted by positive or negative peaking of amplitude from the onset, which is getting feedback timing from the event. The representative ERP pattern is P300, which shows positive peaking after 300&#x202F;ms from the onset. The P300 patterns have been interpreted in cognitive states. For example, a higher P300 amplitude means the mental state is superior. In contrast, lower P300 amplitude occurs when the cognitive state is relatively weak, such as alcohol, drug, or nicotine dependence (<xref ref-type="bibr" rid="ref36">Sur and Sinha, 2009</xref>).</p>
<p>ERP analysis is used in trust in human decision-making. <xref ref-type="bibr" rid="ref27">Long et al. (2012)</xref> analyzed ERP for trust choice. The ERP pattern was shown for &#x201C;gain,&#x201D; as a trustful choice, and &#x201C;no gain,&#x201D; as a distrustful choice. In this study, feedback-related negativity (FRN) and P300 patterns appeared. FRN is showing a negative peak amplitude from 200 to 300&#x202F;ms after onset. P300 amplitude, as the P300 effect, is increased for trustful and distrustful choices. FRN effect is decreased in distrustful choice. <xref ref-type="bibr" rid="ref10">de Visser et al. (2018)</xref> measured trust in different agent algorithms. The trust or distrust performances were significantly distinguished by observational error-related negativity (oERN), showing negative peaking after about 44&#x202F;ms from the onset, and observational error positivity (oPe), with positive peaking after about 150&#x202F;ms from the onset. The one is shown by detecting unconscious errors, and oPe is shown by recognizing errors. Trust states were measured when humans have conversations with a chatbot in e-commerce by showing P2 (P200), showing positive or negative peaking approximately 200&#x202F;ms after onset, and late positive potential (LPP) showing positive peaking approximately 600&#x202F;ms after onset. P2 is shown under low-level performance of the system, and LPP is shown under conscious control with sustained attention (<xref ref-type="bibr" rid="ref39">Wang et al., 2023</xref>).</p>
<p>ERP studies have been conducted on expected and unexpected uncertainty. <xref ref-type="bibr" rid="ref18">Kogler et al. (2017)</xref> performed an ERP analysis to determine how neural states change when humans are under expected and unexpected, uncertain situations by giving positive and negative feedback in gambling experiments. FRN, showing negative peaking approximately 200&#x2013;300&#x202F;ms after onset, was shown when they saw negative feedback to their decision-making under uncertainties. P300 pattern appeared in positive feedback under the expected and unexpected uncertainty. <xref ref-type="bibr" rid="ref7">Boelaert (2022)</xref> performed ERP studies by decision-making based on multiple pattern cues under expected and unexpected uncertainties. P300 and N2pc, showing negative peaks approximately 180&#x2013;300&#x202F;ms after onset. N2pc is one of the N2 (N200) patterns, and it can appear when humans pay attention to distractors instead of targets.</p>
<p>ERP studies show neural states under trust and distrust, or expected and unexpected uncertainty. However, it is difficult to find ERP studies in the multiple-cue judgment system, such as MAS, with time delay issues for information reception. This study focused on how neural states operate under expected and unexpected uncertainty situations with a time delay in information delivery in the multiple-cue judgment system.</p>
<p>This study is about understanding the cognitive process of the human agent based on decision-making under uncertainty in the multiple-cue judgment system, such as MAS. Based on concepts of expected and unexpected uncertainties, the uncertainty is implemented by showing the frequency of invalid system performance after decision-making. Also, a time delay is added to investigate how this issue affects human decision-making by multiple cues (information) with time delay. An EEG measures neural correspondence after experiencing the multiple-cue judgment system. This study will give insight into how an unstable and unreliable decision-making environment under uncertainty affects the human agent&#x2019;s decision-making process with time-delayed information transmission for developing high-level MAS.</p>
</sec>
</sec>
<sec sec-type="methods" id="sec8">
<label>2</label>
<title>Method</title>
<sec id="sec9">
<label>2.1</label>
<title>Experimental setup for unreliable system performance environment</title>
<p>In this study, the unreliable system usage environment is comprised of a ratio of invalid system performances based on multiple cues and a real-world result in decision-making. The five agents&#x2019; cues have information on where the bulb will be located between left and right. The agent can detect the bulb&#x2019;s location and transmit information to a human agent. The real-world result shows the bulb&#x2019;s location with feedback, such as correct or wrong prediction based on decision-making with cues, as shown in <xref ref-type="fig" rid="fig3">Figure 3</xref>.</p>
<fig position="float" id="fig3">
<label>Figure 3</label>
<caption>
<p>Experiment design for the valid and invalid results according to decision-making.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g003.tif"/>
</fig>
<p>Invalid system performance means that it is not the same between given multiple cues from each agent and real-world results after decision-making in this study. For example, suppose that the real-world result is that the bulb is located on the right side when three out of five arrows point out left in the cue section. In this case, an invalid system performance situation is shown, which can result in the wrong consequences for the bulb location in the real world for the system aim. In the experiment process for each trial in <xref ref-type="fig" rid="fig4">Figure 4</xref>, fixation is shown for 1&#x202F;s to refresh. Then, multiple cues (arrows) are shown. For decision-making based on multiple cues, we let decision-makers know they should press keypad &#x201C;1&#x201D; if they think the bulb will appear on the left side and that they should press keypad &#x201C;2&#x201D; if they predict the bulb will be on the right side.</p>
<fig position="float" id="fig4">
<label>Figure 4</label>
<caption>
<p>Experiment process in one decision-making trial.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g004.tif"/>
</fig>
<p>The expected and unexpected uncertainty situation consists of the different ratios of invalid system performances to build three different system usage environments. The invalid system performance is when multiple cues and real-world results are different. The first is the perfect system environment (PF) with no invalid system performances. Second is the expected uncertainty environment (EX) with a 50% ratio of invalid system performances. Finally, the unexpected uncertainty environment (UX) is a 10% ratio of invalid system performances. There are 60 decision-making trials for each environment. The invalid system performance is manipulated intently for reliable or unreliable system usage environments.</p>
</sec>
<sec id="sec10">
<label>2.2</label>
<title>Experimental design for time delay</title>
<p>The time delay is implemented by showing each arrow with specific interval times with three types of conditions. The first is no time delay (NO), showing all five arrows simultaneously. The second is regular time delay (RE), showing the four arrows with 0.25-s intervals after showing the first arrow. The last is the irregular time delay (IR), showing one arrow after 1.5&#x202F;s, while the other four arrows are shown with a 0.25-s delay. The first arrow is shown after 0.25&#x202F;s from the empty display for all conditions, and the arrows are not shown simultaneously in RE and IR conditions. The order of the arrows shown is random, not shown from top to bottom, as shown in <xref ref-type="fig" rid="fig5">Figure 5</xref>.</p>
<fig position="float" id="fig5">
<label>Figure 5</label>
<caption>
<p>Experiment with the design of time delay conditions by time flow.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g005.tif"/>
</fig>
<p>The experiment is conducted in nine conditions combined with three types of system usage environments and three types of time delay situations, as shown in <xref ref-type="table" rid="tab1">Table 1</xref>. We first experimented, showing NOPF. After the NOPF condition, the other eight conditions were randomly presented to participants. This study used PsychoPy (2023.2.3) as an experiment tool based on Python code.</p>
<table-wrap position="float" id="tab1">
<label>Table 1</label>
<caption>
<p>The number and ratio of invalid system performance to build three different system usage environments (uncertainty situations) and nine experiment conditions with three different time delay issues.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th/>
<th align="left" valign="top">No time delay (NO)</th>
<th align="left" valign="top">Regular time delay (RE)</th>
<th align="left" valign="top">Irregular time delay (IR)</th>
<th align="left" valign="top">The number of invalid system performances (trials)</th>
<th align="center" valign="top">Ratio of invalid system performances (%)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle">Perfect System (PF)</td>
<td align="left" valign="middle">NOPF<sup>1)</sup></td>
<td align="left" valign="middle">REPF<sup>2)</sup></td>
<td align="left" valign="middle">IRPF<sup>3)</sup></td>
<td align="left" valign="middle">0 out of 60 trials</td>
<td align="center" valign="middle">0%</td>
</tr>
<tr>
<td align="left" valign="middle">Expected Uncertainty situations (EX)</td>
<td align="left" valign="middle">NOEX<sup>4)</sup></td>
<td align="left" valign="middle">REEX<sup>5)</sup></td>
<td align="left" valign="middle">IREX<sup>6)</sup></td>
<td align="left" valign="middle">30 out of 60 trials</td>
<td align="center" valign="middle">50%</td>
</tr>
<tr>
<td align="left" valign="middle">Unexpected Uncertainty situations (UX)</td>
<td align="left" valign="middle">NOUX<sup>7)</sup></td>
<td align="left" valign="middle">REUX<sup>8)</sup></td>
<td align="left" valign="middle">IRUX<sup>9)</sup></td>
<td align="left" valign="middle">6 out of 60 trials</td>
<td align="center" valign="middle">10%</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p>NOPF<sup>1)</sup>: no time delay under the perfect system; REPF<sup>2)</sup>: regular time delay under the perfect system; IRPF<sup>3)</sup>: irregular time delay under the perfect system; NOEX<sup>4)</sup>: no time delay under the expected uncertainty situations; REEX<sup>5)</sup>: regular time delay under the expected uncertainty situations; IREX<sup>6)</sup>: irregular time delay under the expected uncertainty situations; NOUX<sup>7)</sup>: no time delay under the unexpected uncertainty situations; REUX<sup>8)</sup>: regular time delay under the uncertainty situations; IRUX<sup>9)</sup>: irregular time delay under the uncertainty situations.</p>
</table-wrap-foot>
</table-wrap>
<p>The perfect system situations, such as no time delay under the perfect system (NOPF), the regular time delay under the perfect system (REPF), and the irregular time delay under the perfect system (IRPF), have a 0% ratio of invalid system performance (i.e., 0 invalid system performance out of 60 decision-making trials) as most reliable system usage environment for decision-making.</p>
<p>The expected uncertainty situations, such as no time delay under the unexpected uncertainty situations (NOEX), the regular time delay under the expected uncertainty (REEX), and the irregular time delay under the expected uncertainty situations (IREX), have a 50% ratio of invalid system performance (i.e., 30 invalid system performances out of 60 decision-making trials) as unreliable system usage environment for decision-making.</p>
<p>The unexpected uncertainty situations, such as no time delay under the unexpected uncertainty situations (NOUX), the regular time delay under the unexpected uncertainty (REUX), and the irregular time delay under the unexpected uncertainty situations (IRUX) have a 10% ratio of invalid system performance (i.e., six invalid system performances out of 60 decision-making trials) as a reliable system usage environment for decision-making.</p>
</sec>
<sec id="sec11">
<label>2.3</label>
<title>Electroencephalogram instrumentation</title>
<p>We used EEG to understand the cognitive process based on decision-making in a multiple-cue judgment system. EEG measures human electrical signals as brain activity from the scalp. The cognitive process is interpreted by changes in electrical signals when humans experience specific stimuli. The electrical signal data is measured and collected by electrodes (channel). We used a 10&#x2013;20 system. The common mode sense (CMS) and driven right leg (DRL) are located, respectively, AFz and FCz, and two mastoid electrodes (TP9 and TP10) are offline re-referenced to improve EEG quality (<xref ref-type="bibr" rid="ref14">Fong et al., 2012</xref>; <xref ref-type="bibr" rid="ref34">Stopczynski et al., 2014</xref>). Thus, 30 electrodes were used (Fp1, Fp2, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, PO9, O1, Oz, O2, and PO10) for EEG data analysis. The EEG device was an Emotiv EpocFlex get kit, and the data sampling size was 128&#x202F;Hz.</p>
</sec>
<sec id="sec12">
<label>2.4</label>
<title>Demographic information of participants</title>
<p>The experiment was conducted with 19 participants. The participants&#x2019; ethnicity was 11 Africans, two African Americans, three Asians, and two Caucasian Americans. The average age is 32.1, and all participants are older than 18. All participants had no physical or psychological disorder to conduct experiments. For every experiment session, we asked whether they needed a rest. The average experiment time is 63.85&#x202F;min. This study proceeded with institutional review board (IRB) approval with received consent from all participants.</p>
</sec>
<sec id="sec13">
<label>2.5</label>
<title>Electroencephalogram data preprocessing</title>
<p>Data preprocessing is necessary for EEG data because of noise and artifacts, such as eye or muscle movement, in raw EEG data. The EEG data were filtered by the higher edge of the frequency at 30&#x202F;Hz and epoched before 200&#x202F;ms to after 1,000&#x202F;ms from the onset as the timing when participants see the feedback after the decision-making. We referenced EEG data based on two mastoids (TP9 and TP10), and the baseline was removed before 200&#x202F;ms to remove noise and normalize EEG data (<xref ref-type="bibr" rid="ref14">Fong et al., 2012</xref>). We used independent component analysis (ICA) to remove artifacts and only retained brain activities after ICA. Finally, we removed EEG epochs when the voltage is higher than 75&#x202F;&#x03BC;V or lower than &#x2212;75&#x202F;&#x03BC;V to remove EEG data having extremely high or low voltage. We used EEGLAB (v2022.1) as an EEG data preprocessing and analysis software tool based on MATLAB for EEG data preprocessing. ERP analysis was performed according to ERP epochs for nine conditions (3 uncertainty situations &#x00D7; 3 time delay conditions), as shown in <xref ref-type="table" rid="tab1">Table 1</xref>.</p>
</sec>
<sec id="sec14">
<label>2.6</label>
<title>Statistical analysis</title>
<p>For significant analysis of the effect of uncertainty situations and time delay issues, we conducted an analysis of variance (ANOVA) by using the SAS program (Version 9.41) for ERP amplitude. The dependent variables are uncertainty situations (EX, PF, and UX) and time delay conditions (IR, NO, and RE). The significance level is 0.05 (&#x03B1;&#x202F;&#x003C;&#x202F;0.05). For the post-hoc test, we used Tukey&#x2019;s test.</p>
</sec>
</sec>
<sec sec-type="results" id="sec15">
<label>3</label>
<title>Results</title>
<sec id="sec16">
<label>3.1</label>
<title>Data visualization of event-related potential analysis</title>
<p>We used the ERP analysis to understand how reliable or unreliable system usage environment and time delay affect human decision-making. ERP is the interpretation of change in recorded electrical signal data based on the time domain from the onset when humans see or experience specific stimuli. The EEG analysis of this study shows a P300 (or P3) pattern showing a positive peak amplitude (potential value) of approximately 300&#x202F;ms. The time range is 350&#x2013;450&#x202F;ms for each of the nine conditions around the parietal lobe part with six channels (Cz, CP1, P3, O1, Pz, Oz, O2, P4, and CP2) based on Pz. Also, P300 patterns are shown in terms of system usage environment by systems usage reliability, as shown in <xref ref-type="fig" rid="fig6">Figure 6</xref>.</p>
<fig position="float" id="fig6">
<label>Figure 6</label>
<caption>
<p>Event-related potential plot for nine conditions and system usage environments. <bold>(A)</bold> 9 conditions; <bold>(B)</bold> Perfect systems (PF); <bold>(C)</bold> Expected uncertainty situations (EX); <bold>(D)</bold> Unexpected uncertainty situations (UX).</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g006.tif"/>
</fig>
<p>For eye inspection of the ERP plot in <xref ref-type="fig" rid="fig6">Figure 6</xref>, the grand averaged ERP plot shows positive peaking approximately 300&#x202F;ms (350&#x202F;~&#x202F;450&#x202F;ms) after onset. In the ERP plot, the expected uncertainty situations (IREX, NOEX, and REEX) seem to have the highest amplitude rather than the unexpected uncertainty situations (IRUX, NOUX, and REUX) and perfect system situations (IRPF, NOPF, and REPF). In the time delay conditions, no delay seems to have the highest amplitude in the expected uncertainty situation and perfect system.</p>
<p>We plotted EEG data as averaged topography to know how different system performance environments change brain activity and time delay conditions by showing averaged potential values. The positivity&#x2019;s tone is warm, and the color becomes thicker when the potential value increases, as shown in <xref ref-type="fig" rid="fig7">Figure 7</xref>. Positivity means having an area of positive amplitude (microvolts).</p>
<fig position="float" id="fig7">
<label>Figure 7</label>
<caption>
<p>The grand averaged topography plot by system usage environment and time delay conditions: <bold>(A)</bold> no time delay under perfect system (NOPF); <bold>(B)</bold> no time delay under the expected uncertainty situations (NOEX); <bold>(C)</bold> no time delay unexpected under the unexpected uncertainty situations (NOUX); <bold>(D)</bold> regular time delay under perfect system (REPF); <bold>(E)</bold> regular time delay under the expected uncertainty situations (REEX); <bold>(F)</bold> regular time delay under the unexpected uncertainty situations (REUX); <bold>(G)</bold> irregular time delay under perfect system (IRPF); <bold>(H)</bold> irregular time delay under the expected uncertainty situations (IREX); <bold>(I)</bold> irregular time delay under the unexpected uncertainty situations (IRUX); <bold>(J)</bold> potential value (amplitude) range.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g007.tif"/>
</fig>
<p>In visual inspection, the positivity of NO, RE, and IR conditions seem to have similar positivity in the expected uncertainty (EX) (see <xref ref-type="fig" rid="fig7">Figures 7B</xref>,<xref ref-type="fig" rid="fig7">E</xref>,<xref ref-type="fig" rid="fig7">H</xref>) and perfect system (PF) (see <xref ref-type="fig" rid="fig7">Figures 7A,D,G</xref>). In the unexpected uncertainty (UX) situation, NO conditions look to have the smallest positivity (see <xref ref-type="fig" rid="fig7">Figure 7C</xref>).</p>
<p><xref ref-type="table" rid="tab2">Table 2</xref> shows the grand averaged amplitude by system usage environment and time delay conditions (nine conditions). <xref ref-type="table" rid="tab2">Table 2</xref> also shows the mean of the grand average by three different system usage environments (uncertainty situations) and three different time delay types.</p>
<table-wrap position="float" id="tab2">
<label>Table 2</label>
<caption>
<p>The grand averaged amplitude is determined by the system usage environment and time delay conditions.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th/>
<th align="center" valign="top" colspan="2">Perfect system (PF)</th>
<th align="center" valign="top" colspan="2">Expected uncertainty (EX)</th>
<th align="center" valign="top" colspan="2">Unexpected uncertainty (UX)</th>
<th align="center" valign="top" colspan="2">Mean of grand averaged amplitude by NO/RE/PF</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">No time delay (NO)</td>
<td align="left" valign="top">NOPF<sup>1)</sup></td>
<td align="center" valign="top">0.6958</td>
<td align="left" valign="top">NOEX<sup>2)</sup></td>
<td align="center" valign="top">3.401</td>
<td align="left" valign="top">NOUX<sup>3)</sup></td>
<td align="center" valign="top">1.059</td>
<td align="left" valign="top">NOPF &#x0026;<break/>NOEX &#x0026;<break/>NOUX</td>
<td align="center" valign="top">1.7186</td>
</tr>
<tr>
<td align="left" valign="top">Regular time delay (RE)</td>
<td align="left" valign="top">REPF<sup>4)</sup></td>
<td align="center" valign="top">0.518</td>
<td align="left" valign="top">REEX<sup>5)</sup></td>
<td align="center" valign="top">2.57</td>
<td align="left" valign="top">REUX<sup>6)</sup></td>
<td align="center" valign="top">1.532</td>
<td align="left" valign="top">REPF &#x0026;<break/>REEX &#x0026;<break/>REUX</td>
<td align="center" valign="top">1.5402</td>
</tr>
<tr>
<td align="left" valign="top">Irregular time delay (IR)</td>
<td align="left" valign="top">IRPF<sup>7)</sup></td>
<td align="center" valign="top">0.6811</td>
<td align="left" valign="top">IREX<sup>8)</sup></td>
<td align="center" valign="top">2.804</td>
<td align="left" valign="top">IRUX<sup>9)</sup></td>
<td align="center" valign="top">1.412</td>
<td align="left" valign="top">IRPF &#x0026;<break/>IREX &#x0026;<break/>IRUX</td>
<td align="center" valign="top">1.6322</td>
</tr>
<tr>
<td align="left" valign="top" rowspan="2">Mean of grand averaged amplitude by EX/PF/UX</td>
<td align="left" valign="top" colspan="2">NOPF &#x0026;<break/>REPF &#x0026;<break/>IRPF</td>
<td align="left" valign="top" colspan="2">NOEX &#x0026;<break/>REEX&#x0026;<break/>IREX</td>
<td align="left" valign="top" colspan="2">NOUX &#x0026;<break/>REUX&#x0026;<break/>IRUX</td>
<td colspan="2" rowspan="2"/>
</tr>
<tr>
<td align="left" valign="top" colspan="2">0.6317<sup>b10)</sup></td>
<td align="left" valign="top" colspan="2">2.9248<sup>a</sup></td>
<td align="left" valign="top" colspan="2">1.3344<sup>b</sup></td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p>Three different system usage environments (uncertainty situations) and three different time delay types (unit: microvolts (&#x03BC;V)). NOPF<sup>1)</sup>: no time delay under the perfect system; NOEX<sup>2)</sup>: no time delay under the expected uncertainty situations; NOUX<sup>3)</sup>: no time delay under the unexpected uncertainty situations; REPF<sup>4)</sup>: regular time delay under the perfect system; REEX<sup>5)</sup>: regular time delay under the expected uncertainty situations; REUX<sup>6)</sup>: regular time delay under the unexpected uncertainty situations; IRPF<sup>7)</sup>: irregular time delay under the perfect system; IREX<sup>8)</sup>: irregular time delay under the expected uncertainty situations; IRUX<sup>9)</sup>: irregular time delay under the unexpected uncertainty situations; <sup>10)</sup> means of the grand averaged amplitude with different letters in a row are different at 5% significance level by Tukey&#x2019;s test by three uncertainty situations of nine conditions.</p>
</table-wrap-foot>
</table-wrap>
<p>In grand averaged amplitude, no time delay under expected uncertainty shows the highest amplitude (3.021&#x202F;&#x03BC;V). We used two-way ANOVA for P300 potential values (amplitude) in nine conditions. The different uncertainty situations significantly affected P300 amplitude differences (<italic>F</italic> =&#x202F;7.76, <italic>p</italic> =&#x202F;0.0006&#x202F;&#x003C;&#x202F;0.05). The grand averaged P300 amplitude was calculated for the parietal lobe part channels (Cz, CP1, P3, O1, Pz, Oz, O2, P4, and CP2) for 350&#x2013;450&#x202F;ms. The means of amplitude by uncertainty situations and time delay conditions is the mean of the grand averaged P300 amplitude value. In Tukey&#x2019;s test for the post-hoc test, the expected uncertainty situation makes a significant amplitude difference from other situations, such as perfect system (<italic>p</italic> =&#x202F;0.0004) and unexpected uncertainty situations (<italic>p</italic> =&#x202F;0.0203). However, there is no significant difference in amplitude change between unexpected uncertainty situations and a perfect system (<italic>p</italic> =&#x202F;0.4568). In time delay conditions, different time delay conditions did not significantly affect P300 amplitude differences (<italic>F</italic> =&#x202F;0.04, <italic>p</italic> =&#x202F;0.9562). Also, there is no interaction effect between uncertainty situations and time delay conditions (<italic>F</italic> =&#x202F;0.22, <italic>p</italic> =&#x202F;0.9297). It means these two dependent variables have no relationship to affect amplitude change.</p>
</sec>
<sec id="sec17">
<label>3.2</label>
<title>Event-related potential analysis by three uncertainty situations</title>
<p><xref ref-type="fig" rid="fig8">Figure 8</xref> shows the grand averaged ERP analysis by three uncertainty situations. There is a difference between the nine-condition analysis and the three uncertainty situations analysis.</p>
<fig position="float" id="fig8">
<label>Figure 8</label>
<caption>
<p>Event-related potential analysis by three time delay conditions with amplitude values: <bold>(A)</bold> grand averaged event-related potential plot by uncertainty situations (&#x2212;200 to 1,000&#x202F;ms); <bold>(B)</bold> grand averaged event-related potential plot uncertainty situations (350&#x2013;450&#x202F;ms); <bold>(C)</bold> topography of the expected uncertainty situations (EX); <bold>(D)</bold> topography of perfect system (PF); <bold>(E)</bold> topography of the unexpected uncertainty situations (UX); <bold>(F)</bold> potential value (amplitude) range.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g008.tif"/>
</fig>
<p>In the nine-condition analysis, the mean of grand averaged amplitude was calculated for the 19 subjects for uncertainty situations. The analysis is visualized in a grand averaged ERP plot before 200 to after 1,000&#x202F;ms from the onset in <xref ref-type="fig" rid="fig8">Figure 8A</xref>, 350&#x2013;450&#x202F;ms from the onset, and topography with grand averaged amplitude in <xref ref-type="fig" rid="fig8">Figure 8B</xref>.</p>
<p>In visual inspection, EX shows the largest positivity (see <xref ref-type="fig" rid="fig8">Figure 8C</xref>), and PF shows the smallest (see <xref ref-type="fig" rid="fig8">Figure 8D</xref>). In grand averaged potential values by system usage environments, such as the grand average of no time delay, regular time delay, and irregular time delay under the uncertainty environment, the expected uncertainty environment (the grand average of NOEX, REEX, and IREX environments) shows the highest potential value (2.903&#x202F;&#x03BC;V). The unexpected uncertainty environment (the average of NOUX, REUX, and IRUX) shows the second-highest amplitude (1.196&#x202F;&#x03BC;V). The perfect system environment (the average of NOPF, REPF, and IRPF) shows the lowest amplitude (0.6439&#x202F;&#x03BC;V), as shown in <xref ref-type="table" rid="tab3">Table 3</xref>.</p>
<table-wrap position="float" id="tab3">
<label>Table 3</label>
<caption>
<p>The grand averaged ERP amplitude for three uncertainty situations from 350 to 450&#x202F;&#x03BC;V.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th/>
<th align="center" valign="top">Expected uncertainty situation (EX)</th>
<th align="center" valign="top">Perfect system situation (PF)</th>
<th align="center" valign="top">Unexpected uncertainty situation (UX)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle">Amplitude (&#x03BC;V)</td>
<td align="center" valign="middle">2.903<sup>a1)</sup></td>
<td align="center" valign="middle">0.6439<sup>b</sup></td>
<td align="center" valign="middle">1.196<sup>ab</sup></td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><sup>1)</sup>Means of the grand averaged amplitude with different letters in a row are different at 5% significance level by Tukey&#x2019;s test by three uncertainty situations of nine conditions.</p>
</table-wrap-foot>
</table-wrap>
<p>We performed a one-way ANOVA to determine the significant differences in the grand averaged P300 amplitude under uncertainty situations. The amplitude size is significantly different in uncertain situations (<italic>F</italic> =&#x202F;3.34, <italic>p</italic> =&#x202F;0.0429). In the Tukey post-hoc test, the perfect system and the expected uncertainty situations have significant amplitude differences (<italic>p</italic> =&#x202F;0.0425). In contrast, expected and unexpected uncertainty situations have no significant amplitude differences (<italic>p</italic> =&#x202F;0.1564). Also, the perfect system and unexpected uncertainty situations have no significant difference (<italic>p</italic> =&#x202F;0.8173).</p>
</sec>
<sec id="sec18">
<label>3.3</label>
<title>Event-related potential analysis by three time delay conditions</title>
<p><xref ref-type="fig" rid="fig9">Figure 9</xref> shows the ERP analysis by three time delay conditions. The analysis is visualized in the grand averaged ERP plot before 200&#x202F;ms to after 1,000&#x202F;ms from the onset, 350&#x2013;450&#x202F;ms from the onset, and topography with grand averaged amplitude.</p>
<fig position="float" id="fig9">
<label>Figure 9</label>
<caption>
<p>Event-related potential analysis by three time delay conditions with amplitude values: <bold>(A)</bold> grand averaged event-related potential plot by time delay (&#x2212;200 to 1,000&#x202F;ms); <bold>(B)</bold> grand averaged event-related potential plot by time delay (350&#x2013;450&#x202F;ms); <bold>(C)</bold> topography of irregular time delay (IR); <bold>(D)</bold> topography of no time delay (NO); <bold>(E)</bold> topography of regular time delay (RE); <bold>(F)</bold> potential value (amplitude) range.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g009.tif"/>
</fig>
<p>In averaged potential values by time delay conditions, such as the grand average of the perfect system, expected uncertainty, and unexpected uncertainty environments with no time delay, the no time delay conditions (the grand average of NOPF, NOEX, and NOUX conditions) show the highest potential value (1.592&#x202F;&#x03BC;V). The grand average of irregular time conditions (the grand average of IRPF, IREX, and IRUX conditions) shows the second-highest potential value (1.568&#x202F;&#x03BC;V). The average of regular time conditions (the grand average of REPF, REEX, and REUX conditions) has the lowest potential value (1.514&#x202F;&#x03BC;V), as shown in <xref ref-type="table" rid="tab4">Table 4</xref>. In one-way ANOVA in time delay conditions, there is no significant amplitude difference by different time delay conditions (<italic>F</italic> =&#x202F;0.00, <italic>p</italic> =&#x202F;0.9958).</p>
<table-wrap position="float" id="tab4">
<label>Table 4</label>
<caption>
<p>The grand averaged ERP amplitude for three time delay conditions from 350 to 450&#x202F;&#x03BC;V.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th/>
<th align="center" valign="top">No time delay (NO)</th>
<th align="center" valign="top">Irregular time delay (IR)</th>
<th align="center" valign="top">Regular time delay (RE)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle">Amplitude (&#x03BC;V)</td>
<td align="center" valign="middle">1.592</td>
<td align="center" valign="middle">1.568</td>
<td align="center" valign="middle">1.514</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
</sec>
<sec sec-type="discussion" id="sec19">
<label>4</label>
<title>Discussion</title>
<p>The P300 pattern can be interpreted as memory load. Memory load (or working memory load) is the cognitive ability to temporarily store and use information to support decision-making (<xref ref-type="bibr" rid="ref4">Baddeley, 2003</xref>; <xref ref-type="bibr" rid="ref9">Cowan, 1999</xref>; <xref ref-type="bibr" rid="ref32">Scharinger et al., 2017</xref>). High memory load makes impairment of cognitive reappraisal of emotional responses for specific situations (<xref ref-type="bibr" rid="ref15">Gan et al., 2017</xref>). In memory load, positive peaking or deflection for 300&#x2013;600&#x202F;ms after onset (<xref ref-type="bibr" rid="ref19">Kok, 1997</xref>; <xref ref-type="bibr" rid="ref20">Kramer et al., 1986</xref>; <xref ref-type="bibr" rid="ref29">Mecklinger et al., 1992</xref>). <xref ref-type="bibr" rid="ref40">Wang et al. (2015)</xref> set up 450&#x2013;550&#x202F;ms in terms of memory load. Thus, 350&#x2013;450&#x202F;ms is a reasonable time window to know the impact of expected and unexpected uncertainty in memory load. In studies on memory load with P300, memory load can be increased when P300 amplitude is decreased (<xref ref-type="bibr" rid="ref40">Wang et al., 2015</xref>).</p>
<sec id="sec20">
<label>4.1</label>
<title>Memory load by nine conditions</title>
<p>The no time delay under the expected uncertainty condition (NOEX) shows the highest amplitude (3.021&#x202F;&#x03BC;V), as shown in <xref ref-type="fig" rid="fig10">Figure 10</xref>.</p>
<fig position="float" id="fig10">
<label>Figure 10</label>
<caption>
<p>Comparison graph for the mean of grand averaged P300 amplitude from 350 to 450&#x202F;ms under nine conditions.</p>
</caption>
<graphic xlink:href="fncom-19-1595278-g010.tif"/>
</fig>
<p>This result can be interpreted as the memory load being reduced in the unreliable system usage environment (expected uncertainty situation) when information delivery has no time delay. In terms of uncertainty situations, the expected uncertainty situations (NOEX, IREX, and REEX) show the highest amplitude values. The unexpected uncertainty situations (REUX, IRUX, and NOUX) show the second-highest amplitude values. Finally, the lowest amplitude is shown in the perfect system situations (NOPF, IRPF, and REPF).</p>
<p>In two-way ANOVA, there is a significant amplitude change by the uncertainty situations (<italic>p</italic> =&#x202F;0.0006). There is also a significant amplitude difference between the expected uncertainty and two other uncertainty situations. The expected uncertainty situations showed a higher mean of grand averaged P300 amplitude (2.9248&#x202F;&#x03BC;V) than two other uncertainty situations, such as the unexpected uncertainty situation (1.3344&#x202F;&#x03BC;V) and perfect system (0.6317&#x202F;&#x03BC;V), as shown in <xref ref-type="table" rid="tab2">Table 2</xref>. This means that an unreliable system usage environment can cause a less memory load that shows a higher P300 amplitude than a reliable system usage environment.</p>
</sec>
<sec id="sec21">
<label>4.2</label>
<title>Memory load under reliable or unreliable system usage environment</title>
<p>The expected uncertainty environment shows the highest amplitude (2.903&#x202F;&#x03BC;V) when compared with the other two system usage environments, such as unexpected uncertainty (1.196&#x202F;&#x03BC;V) and the perfect system (0.6439&#x202F;&#x03BC;V) environments, as shown in <xref ref-type="table" rid="tab3">Table 3</xref>. In Tukey&#x2019;s post-hoc analysis after one-way ANOVA analysis, there is a significant difference in amplitude between the expected uncertainty situation and the perfect system (<italic>p</italic> =&#x202F;0.0425). In contrast, there are no significant differences between expected and unexpected uncertainty situations (<italic>p</italic> =&#x202F;0.1564) or between unexpected uncertainty situations and perfect systems (<italic>p</italic> =&#x202F;0.8173).</p>
<p>In two-way ANOVA for the mean of grand averaged P300 amplitude in nine conditions, the expected uncertainty situations and the other two are significantly different (<italic>p</italic> =&#x202F;0.0004 for PF; <italic>p</italic> =&#x202F;0.0203 for UX). The expected and perfect systems are significantly different in one-way ANOVA for grand averaged P300 amplitude in three uncertainty situations. The expected uncertainty situations (EX) show a higher P300 amplitude for the other two uncertainty situations. This means that expected uncertainty situations lower memory load. Based on this result, we can conclude that the observation of an invalid result, which affects unreliable environment construction by expected or unexpected uncertainty situations, requires less memory load.</p>
<p>The reason for showing high P300 amplitude under unreliable system usage would be that humans will not follow the decision support system by distrusting the unreliable system usage environment. The expected or unexpected uncertainty environment makes an unreliable environment, unlike a perfect system showing no invalid result after decision-making. The experience of invalid performance can make human agents not trust the decision support system and information because of invalid results, despite following system rules for decisions. By distrusting the systems, the human agents will make decisions without considering information from agents. In the perfect system, it shows no invalid result if they follow the decision policy of the multiple-cue judgment system. So, the perfect system shows valid performance when human agents follow system rules. For this reason, they need to remember the rules to get valid results with the memory load increase. Thus, it can be considered that unreliable system environments result in less remembering rules for utilizing the system due to distrustful experiences by expected or unexpected uncertainty, so a low memory load is shown in the multiple-cue judgment environments. However, we need to study the relationship between memory load and human behavior that does not follow the decision policy of the system.</p>
</sec>
<sec id="sec22">
<label>4.3</label>
<title>Memory load under time delay issue</title>
<p>The NO condition shows the highest amplitude (1.592 &#x03BC;V) in the time delay issue. The IR condition shows the second-highest amplitude (1.568 &#x03BC;V), and the RE shows the lowest amplitude (1.514 &#x03BC;V) in <xref ref-type="table" rid="tab4">Table 4</xref>. In two-way ANOVA and one-way ANOVA, there are no significant amplitude changes by time delay conditions (<italic>F</italic> =&#x202F;0.00, <italic>p</italic> =&#x202F;0.9958 in a one-way ANOVA; <italic>F</italic> =&#x202F;0.04, <italic>p</italic> =&#x202F;0.9562 in two-way ANOVA). Thus, it can be concluded that delayed information delivery does not affect memory load.</p>
<p>In the time delay aspect, the error-related negativity (ERN or Ne) patterns were shown in specific electrodes. ERN is shown during a 0&#x2013;100-ms time window after onset. ERN can reflect cognitive states for mismatching expected and real-world results (<xref ref-type="bibr" rid="ref22">Larson et al., 2010</xref>). In this study, there were ERN patterns during 0&#x2013;100&#x202F;ms. One of the ERN studies mentioned that this ERN amplitude is negatively increased when emphasizing the accuracy of matching between result and decision-making rather than emphasizing reaction time, such as notifying &#x201C;your decision is slow&#x201D; (<xref ref-type="bibr" rid="ref28">Mattes et al., 2023</xref>). Based on this study, the time delay aspect can be interpreted as pressure on accuracy or reaction time for decision-making. Also, we will approach different analysis methods, such as frequency domain analysis, to find how the cognitive process operates by delayed information delivery.</p>
</sec>
</sec>
<sec sec-type="conclusions" id="sec23">
<label>5</label>
<title>Conclusion</title>
<p>This study is about understanding the cognitive process under the reliability of system usage environment with environmental uncertainty and time delay issues in multiple-cue judgment systems, such as MASs. We used the EEG for the neurological analysis of this study. The EEG analysis shows that P300 appears, and we interpret this result in terms of memory load. We can conclude that the memory load is significantly reduced under unreliable system usage environments because the decision policy of a system is less considered when making decisions to use the system. In this case, we need to perform a study to understand how memory load changes when humans do not follow the decision policy of the system for decision-making. In the time delay issue in information delivery, we could find that delayed information delivery does not significantly affect memory load for decision-making in multiple-cue judgment systems.</p>
<p>The significant finding of this study is about differences in memory load by unreliable or reliable system usage environment (expected or unexpected uncertainty situations), not the measurement of cognitive states by valid or invalid feedback, as prior studies have been studied. However, the participants of the ethnicity group were imbalanced, so there is a weakness in normalized analysis by specific ethnicity groups. Also, this study proceeded with a theoretical approach to the visual aspect of the user interface. For practical analysis of cognitive states, the practical experiment can be conducted using a practical user interface for a specific automated system. The time delay of information delivery will be conducted with another ERP pattern, such as ERN, to interpret cognitive states. Also, we will perform other domain analyses for EEG data, such as frequency domain analysis, to determine which cognitive process operates due to delayed information delivery.</p>
<p>This study is about understanding cognitive states in an unreliable MAS usage environment with multiple sources of information. This study focuses on an unreliable system usage environment, which is not a valid (good) or invalid (poor) system performance. This unique study aspect can give insight into measuring and monitoring human cognitive states and whether users are trapped in unreliable MAS. Thus, we expect insight into an index of categorization of neural states under a reliable or unreliable decision-making environment to use a multiple-cue judgment system with information delivery delays from this study.</p>
</sec>
</body>
<back>
<sec sec-type="data-availability" id="sec24">
<title>Data availability statement</title>
<p>The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.</p>
</sec>
<sec sec-type="ethics-statement" id="sec25">
<title>Ethics statement</title>
<p>The studies involving humans were approved by Research Compliance and Ethics at North Carolina Agricultural and Technical State University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.</p>
</sec>
<sec sec-type="author-contributions" id="sec26">
<title>Author contributions</title>
<p>Y-SC: Methodology, Conceptualization, Writing &#x2013; original draft, Visualization, Writing &#x2013; review &#x0026; editing. YS: Methodology, Conceptualization, Writing &#x2013; review &#x0026; editing. SY: Writing &#x2013; review &#x0026; editing.</p>
</sec>
<sec sec-type="funding-information" id="sec27">
<title>Funding</title>
<p>The author(s) declare that financial support was received for the research and/or publication of this article. This work was supported by NNSA&#x2019;s TRACS funded by the U.S. Department of Energy: DE-NA0004189.</p>
</sec>
<sec sec-type="COI-statement" id="sec28">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="ai-statement" id="sec29">
<title>Generative AI statement</title>
<p>The author(s) declare that no Gen AI was used in the creation of this manuscript.</p>
</sec>
<sec sec-type="disclaimer" id="sec30">
<title>Publisher&#x2019;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list>
<title>References</title>
<ref id="ref1"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ahmadian</surname> <given-names>P.</given-names></name> <name><surname>Cagnoni</surname> <given-names>S.</given-names></name> <name><surname>Ascari</surname> <given-names>L.</given-names></name></person-group> (<year>2013</year>). <article-title>How capable is non-invasive EEG data of predicting the next movement? A mini review</article-title>. <source>Front. Hum. Neurosci.</source> <volume>7</volume>:<fpage>124</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fnhum.2013.00124</pub-id>, PMID: <pub-id pub-id-type="pmid">23579176</pub-id></citation></ref>
<ref id="ref2"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Al-Fahoum</surname> <given-names>A. S.</given-names></name> <name><surname>Al-Fraihat</surname> <given-names>A. A.</given-names></name></person-group> (<year>2014</year>). <article-title>Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains</article-title>. <source>ISRN Neurosci.</source> <volume>2014</volume>, <fpage>1</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1155/2014/730218</pub-id>, PMID: <pub-id pub-id-type="pmid">24967316</pub-id></citation></ref>
<ref id="ref3"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Azizian</surname> <given-names>A.</given-names></name> <name><surname>Polich</surname> <given-names>J.</given-names></name></person-group> (<year>2007</year>). <article-title>Evidence for attentional gradient in the serial position memory curve from event-related potentials</article-title>. <source>J. Cogn. Neurosci.</source> <volume>19</volume>, <fpage>2071</fpage>&#x2013;<lpage>2081</lpage>. doi: <pub-id pub-id-type="doi">10.1162/jocn.2007.19.12.2071</pub-id>, PMID: <pub-id pub-id-type="pmid">17892393</pub-id></citation></ref>
<ref id="ref4"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Baddeley</surname> <given-names>A.</given-names></name></person-group> (<year>2003</year>). <article-title>Working memory: looking back and looking forward</article-title>. <source>Nat. Rev. Neurosci.</source> <volume>4</volume>, <fpage>829</fpage>&#x2013;<lpage>839</lpage>. doi: <pub-id pub-id-type="doi">10.1038/nrn1201</pub-id>, PMID: <pub-id pub-id-type="pmid">14523382</pub-id></citation></ref>
<ref id="ref5"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Balaji</surname> <given-names>P. G.</given-names></name> <name><surname>Srinivasan</surname> <given-names>D.</given-names></name></person-group> (<year>2010</year>). An introduction to multi-agent systems. Studies in computational intelligence (310).</citation></ref>
<ref id="ref6"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bland</surname> <given-names>A. R.</given-names></name> <name><surname>Schaefer</surname> <given-names>A.</given-names></name></person-group> (<year>2012</year>). <article-title>Different varieties of uncertainty in human decision-making</article-title>. <source>Front. Neurosci.</source> <volume>6</volume>:<fpage>85</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fnins.2012.00085</pub-id>, PMID: <pub-id pub-id-type="pmid">22701401</pub-id></citation></ref>
<ref id="ref7"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Boelaert</surname> <given-names>A.</given-names></name></person-group> (<year>2022</year>). Expected and Unexpected Uncertainty in Visual Search: Exploring P3 and N2pc. 2022. Available online at: <ext-link xlink:href="http://lib.ugent.be/catalog/rug01:003118647" ext-link-type="uri">http://lib.ugent.be/catalog/rug01:003118647</ext-link></citation></ref>
<ref id="ref8"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Choo</surname> <given-names>S.</given-names></name> <name><surname>Nam</surname> <given-names>C.</given-names></name></person-group> (<year>2022</year>). <article-title>Detecting human trust calibration in automation: a convolutional neural network approach</article-title>. <source>IEEE Trans. Hum.-Mach. Syst.</source> <volume>52</volume>:<fpage>7015</fpage>. doi: <pub-id pub-id-type="doi">10.1109/THMS.2021.3137015</pub-id></citation></ref>
<ref id="ref9"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Cowan</surname> <given-names>N.</given-names></name></person-group> (<year>1999</year>). An embedded-processes model of working memory. Models of working memory. Models of Working Memory: Mechanisms of Active Maintenance and Executive Control.</citation></ref>
<ref id="ref10"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>de Visser</surname> <given-names>E. J.</given-names></name> <name><surname>Beatty</surname> <given-names>P. J.</given-names></name> <name><surname>Estepp</surname> <given-names>J. R.</given-names></name> <name><surname>Kohn</surname> <given-names>S.</given-names></name> <name><surname>Abubshait</surname> <given-names>A.</given-names></name> <name><surname>Fedota</surname> <given-names>J. R.</given-names></name> <etal/></person-group>. (<year>2018</year>). <article-title>Learning from the slips of others: neural correlates of Trust in Automated Agents</article-title>. <source>Front. Hum. Neurosci.</source> <volume>12</volume>:<fpage>309</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fnhum.2018.00309</pub-id>, PMID: <pub-id pub-id-type="pmid">30147648</pub-id></citation></ref>
<ref id="ref11"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Du</surname> <given-names>X.</given-names></name> <name><surname>Qin</surname> <given-names>Y.</given-names></name> <name><surname>Tu</surname> <given-names>S.</given-names></name> <name><surname>Yin</surname> <given-names>H.</given-names></name> <name><surname>Wang</surname> <given-names>T.</given-names></name> <name><surname>Yu</surname> <given-names>C.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Differentiation of stages in joke comprehension: evidence from an ERP study</article-title>. <source>Int. J. Psychol.</source> <volume>48</volume>, <fpage>149</fpage>&#x2013;<lpage>157</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00207594.2012.665162</pub-id>, PMID: <pub-id pub-id-type="pmid">22515793</pub-id></citation></ref>
<ref id="ref12"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dzindolet</surname> <given-names>M. T.</given-names></name> <name><surname>Peterson</surname> <given-names>S. A.</given-names></name> <name><surname>Pomranky</surname> <given-names>R. A.</given-names></name> <name><surname>Pierce</surname> <given-names>L. G.</given-names></name> <name><surname>Beck</surname> <given-names>H. P.</given-names></name></person-group> (<year>2003</year>). <article-title>The role of trust in automation reliance</article-title>. <source>Int. J. Hum.-Comput. Stud.</source> <volume>58</volume>:<fpage>697</fpage>. doi: <pub-id pub-id-type="doi">10.1016/S1071-5819(03)00038-7</pub-id></citation></ref>
<ref id="ref13"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Firoz</surname> <given-names>K. F.</given-names></name> <name><surname>Seong</surname> <given-names>Y.</given-names></name> <name><surname>Oh</surname> <given-names>S.</given-names></name></person-group> (<year>2022</year>). &#x201C;A neurological approach to classify trust through EEG signals using machine learning techniques.&#x201D; In Proceedings of the 2022 IEEE international conference on human-machine systems, ICHMS 2022.</citation></ref>
<ref id="ref14"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Fong</surname> <given-names>M. C. M.</given-names></name> <name><surname>Minett</surname> <given-names>J. W.</given-names></name> <name><surname>Blu</surname> <given-names>T.</given-names></name> <name><surname>Wang</surname> <given-names>W. S. Y.</given-names></name></person-group> (<year>2012</year>). &#x201C;<article-title>Brain-computer interface (BCI): is it strictly necessary to use random sequences in visual spellers?</article-title>&#x201D; in <source>APCHI&#x2019;12 - proceedings of the 2012 Asia Pacific conference on computer-human interaction</source>.</citation></ref>
<ref id="ref15"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gan</surname> <given-names>S.</given-names></name> <name><surname>Yang</surname> <given-names>J.</given-names></name> <name><surname>Chen</surname> <given-names>X.</given-names></name> <name><surname>Zhang</surname> <given-names>X.</given-names></name> <name><surname>Yang</surname> <given-names>Y.</given-names></name></person-group> (<year>2017</year>). <article-title>High working memory load impairs the effect of cognitive reappraisal on emotional response: evidence from an event-related potential study</article-title>. <source>Neurosci. Lett.</source> <volume>639</volume>, <fpage>126</fpage>&#x2013;<lpage>131</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.neulet.2016.12.069</pub-id>, PMID: <pub-id pub-id-type="pmid">28041962</pub-id></citation></ref>
<ref id="ref16"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Geng</surname> <given-names>B.</given-names></name> <name><surname>Brahma</surname> <given-names>S.</given-names></name> <name><surname>Wimalajeewa</surname> <given-names>T.</given-names></name> <name><surname>Varshney</surname> <given-names>P. K.</given-names></name> <name><surname>Rangaswamy</surname> <given-names>M.</given-names></name></person-group> (<year>2020</year>). <article-title>Prospect theoretic utility based human decision making in multi-agent systems</article-title>. <source>IEEE Trans. Signal Process.</source> <volume>68</volume>:<fpage>339</fpage>. doi: <pub-id pub-id-type="doi">10.1109/TSP.2020.2970339</pub-id></citation></ref>
<ref id="ref17"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Karis</surname> <given-names>D.</given-names></name> <name><surname>Fabiani</surname> <given-names>M.</given-names></name> <name><surname>Donchin</surname> <given-names>E.</given-names></name></person-group> (<year>1984</year>). <article-title>&#x201C;P300&#x201D; and memory: individual differences in the von Restorff effect</article-title>. <source>Cogn. Psychol.</source> <volume>16</volume>, <fpage>177</fpage>&#x2013;<lpage>216</lpage>. doi: <pub-id pub-id-type="doi">10.1016/0010-0285(84)90007-0</pub-id></citation></ref>
<ref id="ref18"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kogler</surname> <given-names>L.</given-names></name> <name><surname>Sailer</surname> <given-names>U.</given-names></name> <name><surname>Derntl</surname> <given-names>B.</given-names></name> <name><surname>Pfabigan</surname> <given-names>D. M.</given-names></name></person-group> (<year>2017</year>). <article-title>Processing expected and unexpected uncertainty is modulated by fearless-dominance personality traits &#x2013; an exploratory ERP study on feedback processing</article-title>. <source>Physiol. Behav.</source> <volume>168</volume>, <fpage>74</fpage>&#x2013;<lpage>83</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.physbeh.2016.10.016</pub-id>, PMID: <pub-id pub-id-type="pmid">27789252</pub-id></citation></ref>
<ref id="ref19"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kok</surname> <given-names>A.</given-names></name></person-group> (<year>1997</year>). <article-title>Event-relate1d-potential (ERP) reflections of mental resources: a review and synthesis</article-title>. <source>Biol. Psychol.</source> <volume>45</volume>, <fpage>19</fpage>&#x2013;<lpage>56</lpage>. doi: <pub-id pub-id-type="doi">10.1016/S0301-0511(96)05221-0</pub-id>, PMID: <pub-id pub-id-type="pmid">9083643</pub-id></citation></ref>
<ref id="ref20"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kramer</surname> <given-names>A.</given-names></name> <name><surname>Schneider</surname> <given-names>W.</given-names></name> <name><surname>Fisk</surname> <given-names>A.</given-names></name> <name><surname>Donchin</surname> <given-names>E.</given-names></name></person-group> (<year>1986</year>). <article-title>The effects of practice and task structure on components of the event-related brain potential</article-title>. <source>Psychophysiology</source> <volume>23</volume>, <fpage>33</fpage>&#x2013;<lpage>47</lpage>. doi: <pub-id pub-id-type="doi">10.1111/j.1469-8986.1986.tb00590.x</pub-id>, PMID: <pub-id pub-id-type="pmid">3945706</pub-id></citation></ref>
<ref id="ref21"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kumar</surname> <given-names>J. S.</given-names></name> <name><surname>Bhuvaneswari</surname> <given-names>P.</given-names></name></person-group> (<year>2012</year>). <article-title>Analysis of electroencephalography (EEG) signals and its categorization - a study</article-title>. <source>Procedia Eng.</source> <volume>38</volume>:<fpage>298</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.proeng.2012.06.298</pub-id></citation></ref>
<ref id="ref22"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Larson</surname> <given-names>M. J.</given-names></name> <name><surname>Baldwin</surname> <given-names>S. A.</given-names></name> <name><surname>Good</surname> <given-names>D. A.</given-names></name> <name><surname>Fair</surname> <given-names>J. E.</given-names></name></person-group> (<year>2010</year>). <article-title>Temporal stability of the error-related negativity (ERN) and post-error positivity (Pe): the role of number of trials</article-title>. <source>Psychophysiology</source> <volume>47</volume>, <fpage>1167</fpage>&#x2013;<lpage>1171</lpage>. doi: <pub-id pub-id-type="doi">10.1111/j.1469-8986.2010.01022.x</pub-id>, PMID: <pub-id pub-id-type="pmid">20477982</pub-id></citation></ref>
<ref id="ref23"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>J. D.</given-names></name> <name><surname>See</surname> <given-names>K. A.</given-names></name></person-group> (<year>2004</year>). <article-title>Trust in automation: designing for appropriate reliance</article-title>. <source>Hum. Factors</source> <volume>46</volume>:<fpage>30392</fpage>. doi: <pub-id pub-id-type="doi">10.1518/hfes.46.1.50_30392</pub-id>, PMID: <pub-id pub-id-type="pmid">15151155</pub-id></citation></ref>
<ref id="ref24"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Li</surname> <given-names>Z.</given-names></name> <name><surname>Duan</surname> <given-names>Z.</given-names></name> <name><surname>Xie</surname> <given-names>L.</given-names></name> <name><surname>Liu</surname> <given-names>X.</given-names></name></person-group> (<year>2012</year>). <article-title>Distributed robust control of linear multi-agent systems with parameter uncertainties</article-title>. <source>Int. J. Control.</source> <volume>85</volume>:<fpage>4644</fpage>. doi: <pub-id pub-id-type="doi">10.1080/00207179.2012.674644</pub-id></citation></ref>
<ref id="ref25"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Li</surname> <given-names>X.</given-names></name> <name><surname>Soh</surname> <given-names>Y. C.</given-names></name> <name><surname>Xie</surname> <given-names>L.</given-names></name></person-group> (<year>2018</year>). <article-title>Robust consensus of uncertain linear multi-agent systems via dynamic output feedback</article-title>. <source>Automatica</source> <volume>98</volume>:<fpage>20</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.automatica.2018.09.020</pub-id></citation></ref>
<ref id="ref26"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lim</surname> <given-names>S. H.</given-names></name> <name><surname>Nisar</surname> <given-names>H.</given-names></name> <name><surname>Thee</surname> <given-names>K. W.</given-names></name> <name><surname>Yap</surname> <given-names>V. V.</given-names></name></person-group> (<year>2018</year>). <article-title>A novel method for tracking and analysis of EEG activation across brain lobes</article-title>. <source>Biomed. Signal Process. Control.</source> <volume>40</volume>:<fpage>17</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.bspc.2017.06.017</pub-id></citation></ref>
<ref id="ref27"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Long</surname> <given-names>Y.</given-names></name> <name><surname>Jiang</surname> <given-names>X.</given-names></name> <name><surname>Zhou</surname> <given-names>X.</given-names></name></person-group> (<year>2012</year>). <article-title>To believe or not to believe: trust choice modulates brain responses in outcome evaluation</article-title>. <source>Neuroscience</source> <volume>200</volume>, <fpage>50</fpage>&#x2013;<lpage>58</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.neuroscience.2011.10.035</pub-id>, PMID: <pub-id pub-id-type="pmid">22079574</pub-id></citation></ref>
<ref id="ref28"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mattes</surname> <given-names>A.</given-names></name> <name><surname>Porth</surname> <given-names>E.</given-names></name> <name><surname>Niessen</surname> <given-names>E.</given-names></name> <name><surname>Kummer</surname> <given-names>K.</given-names></name> <name><surname>M&#x00FC;ck</surname> <given-names>M.</given-names></name> <name><surname>Stahl</surname> <given-names>J.</given-names></name></person-group> (<year>2023</year>). <article-title>Larger error negativity peak amplitudes for accuracy versus speed instructions may reflect more neuro-cognitive alignment, not more intense error processing</article-title>. <source>Sci. Rep.</source> <volume>13</volume>:<fpage>2259</fpage>. doi: <pub-id pub-id-type="doi">10.1038/s41598-023-29434-x</pub-id>, PMID: <pub-id pub-id-type="pmid">36755038</pub-id></citation></ref>
<ref id="ref29"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mecklinger</surname> <given-names>A.</given-names></name> <name><surname>Kramer</surname> <given-names>A. F.</given-names></name> <name><surname>Strayer</surname> <given-names>D. L.</given-names></name></person-group> (<year>1992</year>). <article-title>Event related potentials and EEG components in a semantic memory search task</article-title>. <source>Psychophysiology</source> <volume>29</volume>, <fpage>104</fpage>&#x2013;<lpage>119</lpage>. doi: <pub-id pub-id-type="doi">10.1111/j.1469-8986.1992.tb02021.x</pub-id>, PMID: <pub-id pub-id-type="pmid">1609022</pub-id></citation></ref>
<ref id="ref30"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Oh</surname> <given-names>S.</given-names></name> <name><surname>Seong</surname> <given-names>Y.</given-names></name> <name><surname>Yi</surname> <given-names>S.</given-names></name> <name><surname>Park</surname> <given-names>S.</given-names></name></person-group> (<year>2020</year>). <article-title>Neurological measurement of human trust in automation using electroencephalogram</article-title>. <source>Int. J. Fuzzy Log. Intell. Syst.</source> <volume>20</volume>:<fpage>261</fpage>. doi: <pub-id pub-id-type="doi">10.5391/IJFIS.2020.20.4.261</pub-id></citation></ref>
<ref id="ref31"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Polich</surname> <given-names>J.</given-names></name></person-group> (<year>2007</year>). <article-title>Updating P300: an integrative theory of P3a and P3b</article-title>. <source>Clin. Neurophysiol.</source> <volume>118</volume>, <fpage>2128</fpage>&#x2013;<lpage>2148</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.clinph.2007.04.019</pub-id>, PMID: <pub-id pub-id-type="pmid">17573239</pub-id></citation></ref>
<ref id="ref32"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Scharinger</surname> <given-names>C.</given-names></name> <name><surname>Soutschek</surname> <given-names>A.</given-names></name> <name><surname>Schubert</surname> <given-names>T.</given-names></name> <name><surname>Gerjets</surname> <given-names>P.</given-names></name></person-group> (<year>2017</year>). <article-title>Comparison of the working memory load in N-back and working memory span tasks by means of EEG frequency band power and P300 amplitude</article-title>. <source>Front. Hum. Neurosci.</source> <volume>11</volume>:<fpage>6</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fnhum.2017.00006</pub-id>, PMID: <pub-id pub-id-type="pmid">28179880</pub-id></citation></ref>
<ref id="ref33"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sokolowski</surname> <given-names>J. A.</given-names></name></person-group> (<year>2003</year>). <article-title>Enhanced decision modeling using multiagent system simulation</article-title>. <source>Simulation</source> <volume>79</volume>:<fpage>8886</fpage>. doi: <pub-id pub-id-type="doi">10.1177/0037549703038886</pub-id></citation></ref>
<ref id="ref34"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stopczynski</surname> <given-names>A.</given-names></name> <name><surname>Stahlhut</surname> <given-names>C.</given-names></name> <name><surname>Petersen</surname> <given-names>M. K.</given-names></name> <name><surname>Larsen</surname> <given-names>J. E.</given-names></name> <name><surname>Jensen</surname> <given-names>C. F.</given-names></name> <name><surname>Ivanova</surname> <given-names>M. G.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>Smartphones as pocketable labs: visions for mobile brain imaging and neurofeedback</article-title>. <source>Int. J. Psychophysiol.</source> <volume>91</volume>:<fpage>7</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.ijpsycho.2013.08.007</pub-id>, PMID: <pub-id pub-id-type="pmid">23994206</pub-id></citation></ref>
<ref id="ref35"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Sumpter</surname> <given-names>J.</given-names></name> <name><surname>Thomas</surname> <given-names>C.</given-names></name> <name><surname>Yi</surname> <given-names>S.</given-names></name> <name><surname>Kruger</surname> <given-names>A.</given-names></name></person-group> (<year>2019</year>). &#x201C;Analysis of the effects of communication trust and delay on consensus of multi-agent systems.&#x201D; In <italic>2019 IEEE 2nd connected and automated vehicles symposium, CAVS 2019 - proceedings</italic>.</citation></ref>
<ref id="ref36"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sur</surname> <given-names>S.</given-names></name> <name><surname>Sinha</surname> <given-names>V.</given-names></name></person-group> (<year>2009</year>). <article-title>Event-related potential: an overview</article-title>. <source>Ind. Psychiatry J.</source> <volume>18</volume>:<fpage>7865</fpage>. doi: <pub-id pub-id-type="doi">10.4103/0972-6748.57865</pub-id>, PMID: <pub-id pub-id-type="pmid">21234168</pub-id></citation></ref>
<ref id="ref37"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Trentelman</surname> <given-names>H. L.</given-names></name> <name><surname>Takaba</surname> <given-names>K.</given-names></name> <name><surname>Monshizadeh</surname> <given-names>N.</given-names></name></person-group> (<year>2013</year>). <article-title>Robust synchronization of uncertain linear multi-agent systems</article-title>. <source>IEEE Trans. Autom. Control</source> <volume>58</volume>:<fpage>9011</fpage>. doi: <pub-id pub-id-type="doi">10.1109/TAC.2013.2239011</pub-id></citation></ref>
<ref id="ref38"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tweedale</surname> <given-names>J.</given-names></name> <name><surname>Ichalkaranje</surname> <given-names>N.</given-names></name> <name><surname>Sioutis</surname> <given-names>C.</given-names></name> <name><surname>Jarvis</surname> <given-names>B.</given-names></name> <name><surname>Consoli</surname> <given-names>A.</given-names></name> <name><surname>Phillips-Wren</surname> <given-names>G.</given-names></name></person-group> (<year>2007</year>). <article-title>Innovations in multi-agent systems</article-title>. <source>J. Netw. Comput. Appl.</source> <volume>30</volume>:<fpage>5</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.jnca.2006.04.005</pub-id></citation></ref>
<ref id="ref39"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>C.</given-names></name> <name><surname>Li</surname> <given-names>Y.</given-names></name> <name><surname>Fu</surname> <given-names>W.</given-names></name> <name><surname>Jin</surname> <given-names>J.</given-names></name></person-group> (<year>2023</year>). <article-title>Whether to trust chatbots: applying the event-related approach to understand consumers&#x2019; emotional experiences in interactions with chatbots in e-commerce</article-title>. <source>J. Retail. Consum. Serv.</source> <volume>73</volume>:<fpage>3325</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.jretconser.2023.103325</pub-id></citation></ref>
<ref id="ref40"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>L.</given-names></name> <name><surname>Zheng</surname> <given-names>J.</given-names></name> <name><surname>Huang</surname> <given-names>S.</given-names></name> <name><surname>Sun</surname> <given-names>H.</given-names></name></person-group> (<year>2015</year>). <article-title>P300 and decision making under risk and ambiguity</article-title>. <source>Comput. Intell. Neurosci.</source> <volume>2015</volume>, <fpage>1</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1155/2015/108417</pub-id>, PMID: <pub-id pub-id-type="pmid">26539213</pub-id></citation></ref>
<ref id="ref41"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yiu</surname> <given-names>C. Y.</given-names></name> <name><surname>Ng</surname> <given-names>K. K. H.</given-names></name> <name><surname>Li</surname> <given-names>X.</given-names></name> <name><surname>Zhang</surname> <given-names>X.</given-names></name> <name><surname>Li</surname> <given-names>Q.</given-names></name> <name><surname>Lam</surname> <given-names>H. S.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Towards safe and collaborative aerodrome operations: assessing shared situational awareness for adverse weather detection with EEG-enabled Bayesian neural networks</article-title>. <source>Adv. Eng. Inform.</source> <volume>53</volume>:<fpage>1698</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.aei.2022.101698</pub-id></citation></ref>
<ref id="ref42"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yu</surname> <given-names>A. J.</given-names></name> <name><surname>Dayan</surname> <given-names>P.</given-names></name></person-group> (<year>2005</year>). <article-title>Uncertainty, neuromodulation, and attention</article-title>. <source>Neuron</source> <volume>46</volume>, <fpage>681</fpage>&#x2013;<lpage>692</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.neuron.2005.04.026</pub-id>, PMID: <pub-id pub-id-type="pmid">15944135</pub-id></citation></ref>
<ref id="ref43"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>J.</given-names></name> <name><surname>Lyu</surname> <given-names>M.</given-names></name> <name><surname>Shen</surname> <given-names>T.</given-names></name> <name><surname>Liu</surname> <given-names>L.</given-names></name> <name><surname>Bo</surname> <given-names>Y.</given-names></name></person-group> (<year>2018</year>). <article-title>Sliding mode control for a class of nonlinear multi-agent system with time delay and uncertainties</article-title>. <source>IEEE Trans. Ind. Electron.</source> <volume>65</volume>:<fpage>1777</fpage>. doi: <pub-id pub-id-type="doi">10.1109/TIE.2017.2701777</pub-id></citation></ref>
</ref-list>
</back>
</article>