<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Neurol.</journal-id>
<journal-title>Frontiers in Neurology</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Neurol.</abbrev-journal-title>
<issn pub-type="epub">1664-2295</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fneur.2019.00888</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neurology</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Agreement Between High-Resolution Ultrasound and Electro-Physiological Examinations for Diagnosis of Carpal Tunnel Syndrome in the Indonesian Population</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Emril</surname> <given-names>Dessy R.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/704145/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Zakaria</surname> <given-names>Iskandar</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
</contrib>
<contrib contrib-type="author">
<name><surname>Amrya</surname> <given-names>Mirza</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Neurology, Faculty of Medicine, Universitas Syiah Kuala, Dr. Zainoel Abidin General Hospital</institution>, <addr-line>Banda Aceh</addr-line>, <country>Indonesia</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Radiology, Faculty of Medicine, Universitas Syiah Kuala, Dr. Zainoel Abidin General Hospital</institution>, <addr-line>Banda Aceh</addr-line>, <country>Indonesia</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Ke-Vin Chang, National Taiwan University Hospital, Taiwan</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Abdallah El-Sayed Allam, Faculty of Medicine Tanta University, Egypt; Jeimylo Cristobal De Castro, Centuria Medical Makati, Philippines</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Dessy R. Emril <email>dessyemril&#x00040;unsyiah.ac.id</email></corresp>
<fn fn-type="other" id="fn001"><p>This article was submitted to Applied Neuroimaging, a section of the journal Frontiers in Neurology</p></fn></author-notes>
<pub-date pub-type="epub">
<day>26</day>
<month>08</month>
<year>2019</year>
</pub-date>
<pub-date pub-type="collection">
<year>2019</year>
</pub-date>
<volume>10</volume>
<elocation-id>888</elocation-id>
<history>
<date date-type="received">
<day>19</day>
<month>03</month>
<year>2019</year>
</date>
<date date-type="accepted">
<day>31</day>
<month>07</month>
<year>2019</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2019 Emril, Zakaria and Amrya.</copyright-statement>
<copyright-year>2019</copyright-year>
<copyright-holder>Emril, Zakaria and Amrya</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract><p>Carpal tunnel syndrome (CTS) is a disorder of the wrist due to narrowing of the carpal tunnel. It can be caused by trauma or tumors in the tunnel resulting in compression of the median nerve. This disorder is often diagnosed with early symptoms such as tingling, numbness, and weakness that subsequently lead to hand muscle atrophy. While ultrasonography (USG) is one of the diagnostic methods of CTS, neurophysiological diagnosis, such as with nerve conduction study (NCS), is standard in clinics where the necessary equipment is available. This cross-sectional study aimed to compare USG diagnostic values with NCS results to determine USG efficacy for diagnosis of CTS. Data on medical history, physical examination, ultrasound results, and NCS examination from patients who had been diagnosed with CTS at a regional general hospital in Indonesia were collected. In total, 46 patients participated in the study and data were compared using 2 &#x000D7; 2 table analyses and the kappa statistic. Results showed USG sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and accuracy values of 88.5, 65, 76.6, 81.25, 2.52, 0.17, and 78.2%, respectively (<italic>p</italic> &#x0003C; 0.005). Comparison between NCS and the USG assessment obtained a kappa coefficient of &#x003BA; &#x0003D; 0.71 and showed high agreement (&#x003BA; &#x0003D; 0.410.60). In conclusion, the diagnostic value of USG compared to NCS is acceptable. Therefore, USG examination is a feasible CTS diagnostic alternative for clinicians who do not have access to an electrodiagnostic facility.</p></abstract>
<kwd-group>
<kwd>carpal tunnel syndrome</kwd>
<kwd>ultrasonography</kwd>
<kwd>sensitivity</kwd>
<kwd>specificity</kwd>
<kwd>electrodiagnostic</kwd>
<kwd>kappa value</kwd>
</kwd-group>
<counts>
<fig-count count="2"/>
<table-count count="5"/>
<equation-count count="0"/>
<ref-count count="23"/>
<page-count count="6"/>
<word-count count="3718"/>
</counts>
</article-meta> 
</front>
<body>
<sec sec-type="intro" id="s1">
<title>Introduction</title>
<p>Carpal tunnel syndrome (CTS) results in a complex set of symptoms due to entrapment of the median nerve by the transverse carpal ligament within the carpal tunnel of the wrist (<xref ref-type="bibr" rid="B1">1</xref>). Initially, symptoms include sensory disorders in the form of paresthesia, numbness, or tingling of the fingers (<xref ref-type="bibr" rid="B2">2</xref>). Motoric symptoms occur in severe conditions, which can lead to atrophy of the thenar muscle or other muscles innervated by the median nerve and can have a negative impact on quality of life (<xref ref-type="bibr" rid="B2">2</xref>, <xref ref-type="bibr" rid="B3">3</xref>). According to the labor statistics agency report, CTS is a disease that often occurs among industrial workers in developed countries. Based on a study conducted by the American Academy of Orthopedic Surgeons in the United States, the incidence of CTS is 1&#x02013;3 cases per 1,000 people per year, with a prevalence of 50 cases per 1,000 people in the general population. However, the incidence can be as high as 150 cases per 1,000 people per year, with a prevalence of 500 per 1,000 people in high-risk groups. The ratio of women to men with CTS is 3&#x02013;10: 1 (<xref ref-type="bibr" rid="B4">4</xref>, <xref ref-type="bibr" rid="B5">5</xref>) and CTS is most prevalent in the 45&#x02013;60 y age group (<xref ref-type="bibr" rid="B6">6</xref>).</p>
<p>In addition to clinical symptoms, the diagnosis of CTS can be strengthened by physical, neurophysiological, radiological, and laboratory examination (<xref ref-type="bibr" rid="B2">2</xref>). Neurophysiological analysis includes electromyography (EMG) nerve conduction study (NCS). According to Aroori, EMG-NCS examination is the standard in diagnosing CTS with sensitivity between 49 and 84% and specificity between 95 and 99% (<xref ref-type="bibr" rid="B7">7</xref>). However, the introduction of needles into the muscle during the examination is invasive and can be painful, causing some patients to be reluctant (<xref ref-type="bibr" rid="B8">8</xref>). In addition, the equipment for the EMG-NCS is relatively expensive and can only be found in certain health service centers.</p>
<p>Another diagnostic method of CTS is ultrasonography (USG) examination. USG allows for visualization of anatomy and nerve form directly with fairly good sensitivity and specificity values of 77.6 and 86.8%, respectively (<xref ref-type="bibr" rid="B9">9</xref>). Examination using USG is more comfortable for patients because the transducer is only in light contact with the skin of the patient&#x00027;s hand (<xref ref-type="bibr" rid="B7">7</xref>, <xref ref-type="bibr" rid="B9">9</xref>). Further, in Indonesia, not all hospitals and health service centers have EMG-NCS equipment, but USG facilities are common.</p>
<p>There has been a lack of studies focused on the efficacy of USG for CTS diagnosis in the Indonesian population. This study aims to find the diagnostic value of USG for CTS in a sample of Indonesian patients by comparing diagnostic results from USG with those from EMG-NCS.</p></sec>
<sec id="s2">
<title>Research Methodology</title>
<sec>
<title>Patient Selection</title>
<p>This was a diagnostic study with an analytic cross-sectional approach conducted from October 1 to December 30, 2017. The population were, all patients diagnosed with CTS at the Neurology Polyclinic located at the Regional General Hospital Dr. Zainoel Abidin in Banda Aceh, Indonesia. Inclusion criteria were patients diagnosed with CTS according to clinical findings of CTS, willingness to participate in the study, and able to follow the entire series of examinations. Exclusion criteria were patients who refused to take part in the examination, pregnant women, and postoperative CTS patients. This study was carried out in accordance with the recommendations of CIOMS 2016 guidelines, Ethical Committee of Medical Faculty, Syiah Kuala University number 66/KE/FK/2017. The protocol was approved by the Ethical Committee of Medical Faculty, Syiah Kuala University. All subjects gave written informed consent in accordance with the Declaration of Helsinki.</p></sec>
<sec>
<title>Nerve Conduction Study</title>
<p>All patients who were suspected of having CTS underwent standard median distal motor nerve latency (DML) studies (Technique H) by a neurologist with a protocol based on a summary statement from the American Academy of Neurology regarding CTS diagnosis (<xref ref-type="bibr" rid="B10">10</xref>). Motor median conduction studies were performed using standard supramaximal stimulation techniques. The DML was measured by an active electrode placed above the muscle belly of the M. abductor pollicis brevis. The nerve was stimulated using bipolar stimulation electrodes with the cathode positioned 2 cm proximal to the wrist fold. The cathode was placed closest to the recording electrode. CTS was positive if the NCS showed prolonged DML (&#x0003E;4.0 ms) (<xref ref-type="bibr" rid="B11">11</xref>).</p></sec>
<sec>
<title>Ultrasonography Scanning Method and Diagnostic Protocol</title>
<p>The radiologist who performed median nerve USG in the forearm was blinded to clinical and NCS data. The examination of the median nerve was performed using a 5&#x02013;12 MHz linear-array transducer of GE&#x000AE; logic 9 US machine. The USG settings such as frequency, depth, and focal zone, were optimized for nerve imaging. The probe was maintained at a perpendicular angle during analysis to prevent anisotropy and median nerve deformation, and additional weight was not permitted when pressure was applied to the skin surface with the probe. Subjects were seated facing the examiner. With extended arms, the wrists rested on a hard surface, with forearms supinated and fingers half-extended. Transverse images f the median nerve with cross-sectional area measurements were obtained at two levels; at the CSAc at the wrist fold (CSAc &#x0003D; carpal tunnel cross-sectional area) and the pronator quadratus level (CSAp &#x0003D; proximal cross-sectional area) (<xref ref-type="fig" rid="F1">Figure 1</xref>) (<xref ref-type="bibr" rid="B12">12</xref>). Result of measurement both arms were examined using the same technique. The value of the difference between CSAc and CSAp (&#x00394; CSA) was calculated for each wrist.</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Transverse images of the median nerve with cross-sectional area measurements at two levels: <bold>(A)</bold> CSAc at the wrist fold and <bold>(B)</bold> CSAp at the pronator quadratus muscle.</p></caption>
<graphic xlink:href="fneur-10-00888-g0001.tif"/>
</fig>
<p>Measuring of the CSA of the median nerve was done by tracing method. The subject was diagnosed has carpal tunnel syndrome if the difference between CSAc and CSAp (&#x00394;CSA) &#x02265;2 mm<sup>2</sup>.</p></sec>
<sec>
<title>Statistical Analysis</title>
<p>The sampling technique used a diagnostic test sample formula with a sample size of 46 samples. The USG and NCS results were compared using 2 &#x000D7; 2 tables and analyzed by the Receiver Operating Characteristic (ROC) curve. Next, the reliability was compared using the kappa statistic.</p></sec></sec>
<sec sec-type="results" id="s3">
<title>Results</title>
<p>The frequency distribution of participant characteristics are presented in <xref ref-type="table" rid="T1">Table 1</xref>.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Frequency distribution of participant characteristics.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Variable</bold></th>
<th valign="top" align="center"><bold>Frequency</bold></th>
<th valign="top" align="center"><bold>Percentage (%)</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="3"><bold>Gender</bold></td>
</tr>
<tr>
<td valign="top" align="left">Male</td>
<td valign="top" align="center">9</td>
<td valign="top" align="center">19.6</td>
</tr>
<tr>
<td valign="top" align="left">Female</td>
<td valign="top" align="center">37</td>
<td valign="top" align="center">80.4</td>
</tr>
<tr>
<td valign="top" align="left" colspan="3"><bold>Age (years)</bold></td>
</tr>
<tr>
<td valign="top" align="left">&#x0003C;40</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">15.2</td>
</tr>
<tr>
<td valign="top" align="left">40&#x02013;60</td>
<td valign="top" align="center">31</td>
<td valign="top" align="center">67.4</td>
</tr>
<tr>
<td valign="top" align="left">&#x0003E;60</td>
<td valign="top" align="center">8</td>
<td valign="top" align="center">17.4</td>
</tr>
<tr>
<td valign="top" align="left" colspan="3"><bold>Location of CTS</bold></td>
</tr>
<tr>
<td valign="top" align="left">Left</td>
<td valign="top" align="center">24</td>
<td valign="top" align="center">52.2</td>
</tr>
<tr>
<td valign="top" align="left">Right</td>
<td valign="top" align="center">22</td>
<td valign="top" align="center">47.8</td>
</tr>
<tr>
<td valign="top" align="left" colspan="3"><bold>Diabetes Mellitus</bold></td>
</tr>
<tr>
<td valign="top" align="left">Yes</td>
<td valign="top" align="center">0</td>
<td valign="top" align="center">0</td>
</tr>
<tr>
<td valign="top" align="left">No</td>
<td valign="top" align="center">46</td>
<td valign="top" align="center">100</td>
</tr>
<tr>
<td valign="top" align="left" colspan="3"><bold>Hypertension</bold></td>
</tr>
<tr>
<td valign="top" align="left">Yes</td>
<td valign="top" align="center">19</td>
<td valign="top" align="center">41.3</td>
</tr>
<tr>
<td valign="top" align="left">No</td>
<td valign="top" align="center">27</td>
<td valign="top" align="center">58.7</td>
</tr>
<tr>
<td valign="top" align="left" colspan="3"><bold>Durations of onset of CTS</bold></td>
</tr>
<tr>
<td valign="top" align="left">&#x0003C;1 year</td>
<td valign="top" align="center">25</td>
<td valign="top" align="center">54.3</td>
</tr>
<tr>
<td valign="top" align="left">&#x02265;1 year</td>
<td valign="top" align="center">21</td>
<td valign="top" align="center">45.7</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>CTS, carpal tunnel syndrome</italic>.</p>
</table-wrap-foot>
</table-wrap>
<p><xref ref-type="table" rid="T1">Table 1</xref> shows that CTS was diagnosed more frequently in women (80.4%) than in men (19.6%) and individuals 40&#x02013;60 years old had the highest prevalence of CTS (67.4%). The affected hand was similar between the left side (52.2%) and right side (47.8%). There were 41.3% of participants with hypertension and no patients with a history of diabetes mellitus. There was almost no difference in duration of onset of CTS from &#x0003C;1 year and &#x02265;1 year.</p>
<sec>
<title>Ultrasonography and Electrodiagnostic Results</title>
<p><xref ref-type="table" rid="T2">Table 2</xref> shows the mean tunnel inlet (CSAc) measured by USG was 14.87 mm<sup>2</sup>, with 12.39 mm<sup>2</sup> for the mean of the pronator quadratus muscle (CSAp) inspection, and 3.06 mm<sup>2</sup> for the mean change (&#x00394; CSAc-CSAp). The mean DML was 4.96 ms &#x000B1; 1.721.</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Ultrasonography and electrodiagnostic results.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th valign="top" align="left"><bold>Parameters</bold></th>
<th valign="top" align="center"><bold>Mean</bold></th>
<th valign="top" align="center"><bold>Standard Deviation (SD)</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Ultrasonography (mm<sup>2</sup>)</td>
<td valign="top" align="left">Tunnel Inlet (CSAc)</td>
<td valign="top" align="center">14.87</td>
<td valign="top" align="center">2.254</td>
</tr>
<tr>
<td/>
<td valign="top" align="left">Pronator quadratus (CSAp)</td>
<td valign="top" align="center">12.39</td>
<td valign="top" align="center">1.638</td>
</tr>
<tr>
<td/>
<td valign="top" align="left">&#x00394; CSAc-CSAp</td>
<td valign="top" align="center">3.06</td>
<td valign="top" align="center">2.100</td>
</tr>
<tr>
<td valign="top" align="left">Electrodiagnostic (ms)</td>
<td valign="top" align="left">Median Distal Motor Latency</td>
<td valign="top" align="center">4.96</td>
<td valign="top" align="center">1.721</td>
</tr>
</tbody>
</table>
</table-wrap></sec>
<sec>
<title>Comparison Analysis Based on Ultrasonography Results</title>
<p>The statistical analysis presented in <xref ref-type="table" rid="T3">Table 3</xref>, shows that there is a significant difference between positive (15.74 mm<sup>2</sup>) and negative (13.26 mm<sup>2</sup>) CTS patients in tunnel inlet (CSAc) mean (<italic>p</italic> &#x0003C; 0.001). In addition, the mean &#x00394; CSAc-CSAp also shows a significant difference between positive (4.23 mm<sup>2</sup>) and negative (0.88 mm<sup>2</sup>) CTS patients (<italic>p</italic> &#x0003C; 0.001).</p>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Ultrasonography result value.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th valign="top" align="center"><bold>Mean CSAc (mm<sup><bold>2</bold></sup>)</bold></th>
<th valign="top" align="center"><bold><italic>P</italic>-value</bold></th>
<th valign="top" align="center"><bold>Mean CSAp (mm<sup><bold>2</bold></sup>)</bold></th>
<th valign="top" align="center"><bold><italic>P</italic>-Value</bold></th>
<th valign="top" align="center"><bold>&#x00394;CSAc-CSAp (mm<sup><bold>2</bold></sup>)</bold></th>
<th valign="top" align="center"><bold><italic>P</italic>-value</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Positive</td>
<td valign="top" align="center">15.74</td>
<td/>
<td valign="top" align="center">11.89</td>
<td/>
<td valign="top" align="center">4.23</td>
<td/>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="center">0.000</td>
<td/>
<td valign="top" align="center">0.007</td>
<td/>
<td valign="top" align="center">0.000</td>
</tr>
<tr>
<td valign="top" align="left">Negative</td>
<td valign="top" align="center">13.26</td>
<td/>
<td valign="top" align="center">13.32</td>
<td/>
<td valign="top" align="center">0.88</td>
<td/>
</tr>
</tbody>
</table>
</table-wrap></sec>
<sec>
<title>Comparison of Ultrasonography and Electrodiagnostic Results</title>
<p>In this study, the results of ultrasound examinations were performed on patients with suspected CTS. The interpretation of the results was grouped by diagnosis into CTS and not CTS. Next, electrodiagnostic results were used to classify patients into CTS and not CTS groups. The resulting groups were analyzed to determine validity of the diagnostic value for USG and are presented in <xref ref-type="table" rid="T4">Table 4</xref>. USG results showed a sensitivity value of 88.5%, specificity of 65%, positive predictive value of 76.6%, negative predictive value of 81.25%, positive likelihood ratio of 2.52, negative likelihood ratio of 0.17, and accuracy of 78.2%. The results of the ROC curve analysis of ultrasound examination with electrodiagnostic can be seen in <xref ref-type="fig" rid="F2">Figure 2</xref>.</p>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Comparison of ultrasonography and electrodiagnostic results.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th/>
<th valign="top" align="center"><bold>Positive</bold></th>
<th valign="top" align="center"><bold>Negative</bold></th>
<th valign="top" align="center"><bold>Total</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="5" style="background-color:#bbbdc0"><bold>ELECTRODIAGNOSTIC RESULT</bold></td>
</tr>
<tr>
<td valign="top" align="left">USG Result</td>
<td valign="top" align="left">Positive</td>
<td valign="top" align="center">23</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">30</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td valign="top" align="left">Negative</td>
<td valign="top" align="center">3</td>
<td valign="top" align="center">13</td>
<td valign="top" align="center">16</td>
</tr> <tr>
<td/>
<td valign="top" align="left">Total</td>
<td valign="top" align="center">26</td>
<td valign="top" align="center">20</td>
<td valign="top" align="center">46</td>
</tr>
</tbody>
</table>
</table-wrap>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>ROC Curve of ultrasonography results.</p></caption>
<graphic xlink:href="fneur-10-00888-g0002.tif"/>
</fig></sec>
<sec>
<title>Ultrasound Results Based on ROC Curve Analysis</title>
<p>The ROC curve analysis shows the ultrasound examination curve above the 50% line, approaching toward 100%. The area under the curve (AUC) value obtained from the ROC method was 81.3% (95% CI: 68.2&#x02013;94.5, <italic>p</italic> &#x0003C; 0.001). Statistically, the AUC value of 81.3 is good. An AUC value of 81.3% with &#x00394; CSAc-CSAp cut-off value of 2.10 mm<sup>2</sup> means that if ultrasound examination is used to diagnose the possibility of CTS in 46 patients, then the right conclusion will be obtained in 37 patients. This study also carried out a comparative analysis between the results of ultrasound examination with NCS for agreement in diagnosing CTS. The comparison between the two examinations is presented in <xref ref-type="table" rid="T5">Table 5</xref>.</p>
<table-wrap position="float" id="T5">
<label>Table 5</label>
<caption><p>Table Kappa 2 &#x000D7; 2.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th/>
<th valign="top" align="center" colspan="3" style="border-bottom: thin solid #000000;"><bold>Electrodiagnostic</bold></th>
</tr>
<tr>
<th/>
<th/>
<th valign="top" align="center"><bold>Negative</bold></th>
<th valign="top" align="center"><bold>Positive</bold></th>
<th valign="top" align="center"><bold>Total</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">USG Result</td>
<td valign="top" align="left">Negative</td>
<td valign="top" align="center">13</td>
<td valign="top" align="center">3</td>
<td valign="top" align="center">16</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td valign="top" align="left">Positive</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">23</td>
<td valign="top" align="center">30</td>
</tr> <tr>
<td/>
<td valign="top" align="left">Total</td>
<td valign="top" align="center">20</td>
<td valign="top" align="center">26</td>
<td valign="top" align="center">46</td>
</tr>
</tbody>
</table>
</table-wrap></sec>
<sec>
<title>Kappa Analysis Between Ultrasonography and Electrodiagnostic</title>
<p>From <xref ref-type="table" rid="T5">Table 5</xref>, the results of the test using kappa statistic is 0.71, which indicates that there is a good match between USG and NCS examination results.</p></sec></sec>
<sec sec-type="discussion" id="s4">
<title>Discussion</title>
<p>This cross-sectional study aimed to compare USG with NCS diagnostic results for CTS to determine USG efficacy in settings without access to NCS. The results of comprehensive statistical analysis show that in terms of sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and accuracy, USG examination has effective diagnostic value when compared to NCS.</p>
<p>When the peripheral nerve is entrapped, US imaging can reveal nerve flattening at the compressed site and swollen nerve fascicles, which are proximal to the level of compression (<xref ref-type="bibr" rid="B13">13</xref>). Changes in the nerve&#x00027;s shape and size are common sonographic findings of entrapment neuropathy and measurement of the nerve&#x00027;s cross-sectional area is the most commonly employed indicator to differentiated between normality and pathology (<xref ref-type="bibr" rid="B14">14</xref>). According to Fowler et al., various results of research examining USG diagnostic testing in CTS patients have varied sensitivity results, ranging from 57 to 98%, while the specificity varies between 51 to 100% (<xref ref-type="bibr" rid="B9">9</xref>). Yazdchi study et al. obtained results of sensitivity between 70 to 89% and specificity of 57 to 97% (<xref ref-type="bibr" rid="B15">15</xref>). The ROC curve also shows that the values found are 81.3%, so it can be interpreted that ultrasound has good diagnostic value in comparison with electrodiagnostic study in CTS patients. In this study, the cut-off value for &#x00394; CSAc-CSAp was 2.10 mm<sup>2</sup>. This result was consistent with previous studies that mention a &#x00394; CSAc-CSAp threshold of 2 mm<sup>2</sup> to yield the greatest sensitivity (99%) and specificity (100%) for the diagnosis of CTS. Based on these results, USG is proven to be useful as an alternative tool in diagnosing CTS (<xref ref-type="bibr" rid="B12">12</xref>).</p>
<p>This study also compared the strength of agreement between USG examination and NCS in establishing a CTS diagnosis. The results of the analysis using the kappa statistic to compare the inter-rater agreement had a good level of agreement. In their study Shahebari et al. obtained a kappa value of 0.71 to 0.78 (good) in assessing the accuracy of USG in establishing a diagnosis of CTS using a median nerve area (<xref ref-type="bibr" rid="B16">16</xref>). In another study, Lee et al. only obtained a kappa coefficient of 0.55 (moderate) in assessing bone size in diagnosing CTS using ultrasound (<xref ref-type="bibr" rid="B17">17</xref>). This study concludes that this examination has a good strength of agreement and USG and NCS examination can be considered comparable diagnostic options for CTS.</p>
<p>The results showed that there were fewer men with CTS than women. Of the 46 people with CTS clinical signs, there were 37 women (80.4%) and 9 men (19.6%) with a ratio of 4:1. This result is similar to the majority of the literature, which states that women are more often affected by CTS than men, but with a ratio of 1.4:1 (<xref ref-type="bibr" rid="B18">18</xref>). Parviizi and Kim report that CTS is also more common in women but with a ratio 3&#x02013;10:1 (<xref ref-type="bibr" rid="B4">4</xref>). According to Duncan et al., this is likely because women have a smaller wrist size than men which causes the size of the carpal tunnel to become smaller in general. In addition, women also experience hormonal changes that affect changes in the tenosynovial tissue in the carpal tunnel (<xref ref-type="bibr" rid="B19">19</xref>). Age distribution ranged from 24 to 63 years. Based on the present cases, most were found in the age groups 46&#x02013;60 and 31&#x02013;45 years. This is in accordance with the literature which states that CTS is often diagnosed in patients aged 45&#x02013;60 years and only 10% of patients younger than 31 years are diagnosed with CTS (<xref ref-type="bibr" rid="B4">4</xref>, <xref ref-type="bibr" rid="B20">20</xref>). Based on the distribution of clinical findings, 24 people (52.2%) experienced symptoms in the left hand and 22 people (47.8%) in the right hand, which aligns with the literature that states CTS often occurs in the right hand (<xref ref-type="bibr" rid="B4">4</xref>, <xref ref-type="bibr" rid="B20">20</xref>).</p>
<p>Based on the distribution, according to concomitant diseases, 19 (41.3%) patients had a history of hypertension, while 27 (58.7%) other patients had no history of hypertension. Based on the literature, hypertension and diabetes mellitus are the most common risk factors for CTS disease, but in this study, there were no patients who had a history of diabetes mellitus (<xref ref-type="bibr" rid="B21">21</xref>, <xref ref-type="bibr" rid="B22">22</xref>). Based on the distribution according to the duration of suffering from CTS, 25 (54.3%) had been diagnosed with CTS for &#x0003C;1 year, while 21 (45.7%) patients were diagnosed with CTS for more than 1 year.</p>
<p>USG examination can be used in centers that do not have NCS as a standard CTS diagnostic tool (<xref ref-type="bibr" rid="B17">17</xref>). High-resolution USG enables clear visualization of nerve form and texture. The use of ultrasound in a clinic setting offers benefits of no radiation, portability, and excellence in visualization of muscles and surrounding connective tissue (<xref ref-type="bibr" rid="B23">23</xref>).</p></sec>
<sec sec-type="conclusions" id="s5">
<title>Conclusion</title>
<p>The results of this study showed that the USG examination had excellent level of agreement in diagnosing CTS compared to NCS. For this reason, ultrasound examination can be used as an additional or alternative diagnostic tool in diagnosing CTS patients. The recommendations for further research are to link the results of USG with NCS toward motoric and sensory quality in CTS patients. For clinicians, the researchers suggest that this study can be used as a reference in establishing a diagnosis of CTS in centers without an electrodiagnostic facility.</p></sec>
<sec sec-type="data-availability" id="s6">
<title>Data Availability</title>
<p>All datasets generated for this study are included in the manuscript and/or the supplementary files.</p></sec>
<sec id="s7">
<title>Ethics Statement</title>
<p>This study was carried out in accordance with the recommendations of CIOMS 2016 guidelines, Ethical Committee of Medical Faculty, Syiah Kuala University number 66/KE/FK/2017. The protocol was approved by the Ethical Committee of Medical Faculty, Syiah Kuala University. All subjects gave written informed consent in accordance with the Declaration of Helsinki.</p></sec>
<sec id="s8">
<title>Author Contributions</title>
<p>DE substantially contributed to the conception and design of the work, as well as the acquisition, analysis, and interpretation of data for the work. IZ had a substantial contribution in the ultrasonography examination and design of the work. MA substantially contributed to the collection and analysis of data.</p>
<sec>
<title>Conflict of Interest Statement</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p></sec></sec>
</body>
<back>
<ref-list>
<title>References</title>
<ref id="B1">
<label>1.</label>
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Salter</surname> <given-names>RB</given-names></name></person-group>. <source>Textbook of Disorders and Injuries of the Musculoskeletal System</source>. <publisher-loc>Baltimore, MD</publisher-loc>: <publisher-name>Livingstone, Ltd</publisher-name> (<year>2009</year>). p. <fpage>9</fpage>&#x02013;<lpage>14</lpage>.</citation></ref>
<ref id="B2">
<label>2.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Keith</surname> <given-names>MW</given-names></name> <name><surname>Masear</surname> <given-names>V</given-names></name> <name><surname>Chung</surname> <given-names>K</given-names></name> <name><surname>Maupin</surname> <given-names>K</given-names></name> <name><surname>Andary</surname> <given-names>MT</given-names></name> <name><surname>Amadio</surname> <given-names>PC</given-names></name> <etal/></person-group>. <article-title>Diagnosis of carpal tunnel syndrome</article-title>. <source>J Am Acad Orthop Surg</source>. (<year>2009</year>) <volume>17</volume>:<fpage>389</fpage>&#x02013;<lpage>96</lpage>. <pub-id pub-id-type="doi">10.5435/00124635-200906000-00007</pub-id><pub-id pub-id-type="pmid">19474448</pub-id></citation></ref>
<ref id="B3">
<label>3.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Katz Jeffrey</surname> <given-names>N</given-names></name></person-group>. <article-title>Carpal tunnel syndrome</article-title>. <source>N Engl J Med</source>. (<year>2011</year>) <volume>346</volume>:<fpage>23</fpage>. <pub-id pub-id-type="doi">10.1056/NEJMcp013018</pub-id><pub-id pub-id-type="pmid">12050342</pub-id></citation></ref>
<ref id="B4">
<label>4.</label>
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Parviizi</surname> <given-names>J</given-names></name> <name><surname>Kim</surname> <given-names>GK</given-names></name></person-group>. <source>High Yileld Orthopedics: Carpal Tunnel Syndrome.</source> <publisher-loc>Philadelphia</publisher-loc>: <publisher-name>Elsevier</publisher-name>. (<year>2010</year>).</citation></ref>
<ref id="B5">
<label>5.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Salawati</surname> <given-names>L</given-names></name></person-group> <article-title>Syahrul. Carpal tunnel syndrome</article-title>. <source>J Kedokt Syiah Kuala</source>. (<year>2014</year>) <volume>14</volume>:<fpage>29</fpage>&#x02013;<lpage>37</lpage>.</citation></ref>
<ref id="B6">
<label>6.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Calfee</surname> <given-names>RP</given-names></name> <name><surname>Dale</surname> <given-names>AM</given-names></name> <name><surname>Ryan</surname> <given-names>D</given-names></name> <name><surname>Descatha</surname> <given-names>A</given-names></name> <name><surname>Franzblau</surname> <given-names>A</given-names></name> <name><surname>Evanoff</surname> <given-names>B</given-names></name></person-group>. <article-title>Performance of simplified scoring systems for hand diagrams in carpal tunnel syndrome screening</article-title>. <source>J Hand Surg Am.</source> (<year>2012</year>) <volume>37</volume>:<fpage>10</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1016/j.jhsa.2011.08.016</pub-id><pub-id pub-id-type="pmid">21975100</pub-id></citation></ref>
<ref id="B7">
<label>7.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aroori</surname> <given-names>S</given-names></name> <name><surname>Spence</surname> <given-names>RA</given-names></name></person-group>. <article-title>Carpal tunnel syndrome</article-title>. <source>Ulster Med J.</source> (<year>2008</year>) <volume>77</volume>:<fpage>6</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.12968/indn.2008.14.3.75647</pub-id><pub-id pub-id-type="pmid">18269111</pub-id></citation></ref>
<ref id="B8">
<label>8.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Basiri</surname> <given-names>K</given-names></name> <name><surname>Katirji</surname> <given-names>B</given-names></name></person-group>. <article-title>Practical approach to electrodiagnosis of the carpal tunnel syndrome: a review</article-title>. <source>Adv Biomed Res</source>. (<year>2015</year>) <volume>4</volume>:<fpage>50</fpage>. <pub-id pub-id-type="doi">10.4103/2277-9175.151552</pub-id><pub-id pub-id-type="pmid">25802819</pub-id></citation></ref>
<ref id="B9">
<label>9.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fowler</surname> <given-names>JR</given-names></name> <name><surname>Gaughan</surname> <given-names>JP</given-names></name> <name><surname>Ilyas</surname> <given-names>AM</given-names></name></person-group>. <article-title>The sensitivity and specificity of ultrasound for the diagnosis of carpal tunnel syndrome: a meta-analysis</article-title>. <source>Clin Orthop Relat Res.</source> (<year>2011</year>) <volume>469</volume>:<fpage>1089</fpage>&#x02013;<lpage>94</lpage>. <pub-id pub-id-type="doi">10.1007/s11999-010-1637-5</pub-id><pub-id pub-id-type="pmid">20963527</pub-id></citation></ref>
<ref id="B10">
<label>10.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jablecki</surname> <given-names>CK</given-names></name> <name><surname>Andary</surname> <given-names>MT</given-names></name> <name><surname>Floeter</surname> <given-names>MK</given-names></name> <name><surname>Miller</surname> <given-names>RG</given-names></name> <name><surname>Quartly</surname> <given-names>CA</given-names></name> <name><surname>Vennix</surname> <given-names>MJ</given-names></name> <etal/></person-group>. <article-title>Practice parameter: electrodiagnostic studies in carpal tunnel syndrome. Report of the American Association of Electrodiagnostic Medicine, American Academy of Neurology, and the American Academy of Physical Medicine and Rehabilitation</article-title>. <source>Neurology</source>. (<year>2002</year>) <volume>58</volume>:<fpage>1589</fpage>&#x02013;<lpage>92</lpage>. <pub-id pub-id-type="doi">10.1212/WNL.58.11.1589</pub-id><pub-id pub-id-type="pmid">12058083</pub-id></citation></ref>
<ref id="B11">
<label>11.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kwon</surname> <given-names>BC</given-names></name> <name><surname>Jung</surname> <given-names>KI</given-names></name> <name><surname>Baek</surname> <given-names>GH</given-names></name></person-group>. <article-title>Comparison of sonography and electrodiagnostic testing in the diagnosis of carpal tunnel syndrome</article-title>. <source>J Hand Surg Am</source>. (<year>2008</year>) <volume>33</volume>:<fpage>65</fpage>&#x02013;<lpage>71</lpage>. <pub-id pub-id-type="doi">10.1016/j.jhsa.2007.10.014</pub-id><pub-id pub-id-type="pmid">18261667</pub-id></citation></ref>
<ref id="B12">
<label>12.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Klauser</surname> <given-names>AS</given-names></name> <name><surname>Halpern</surname> <given-names>EJ</given-names></name> <name><surname>De Zordo</surname> <given-names>T</given-names></name> <name><surname>Feuchtner</surname> <given-names>GM</given-names></name> <name><surname>Arora</surname> <given-names>R</given-names></name> <name><surname>Gruber</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>Carpal tunnel syndrome assessment with US: value of additional cross-sectional area measurements of the median nerve in patients versus healthy volunteers</article-title>. <source>Radiology</source>. (<year>2009</year>) <volume>250</volume>:<fpage>171</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1148/radiol.2501080397</pub-id><pub-id pub-id-type="pmid">19037017</pub-id></citation></ref>
<ref id="B13">
<label>13.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kerasnoudis</surname> <given-names>A</given-names></name> <name><surname>Tsivgoulis</surname> <given-names>G</given-names></name></person-group>. <article-title>Nerve ultrasound in peripheral neuropathies: a review</article-title>. <source>J Neuroimaging</source>. (<year>2015</year>) <volume>25</volume>:<fpage>528</fpage>&#x02013;<lpage>38</lpage>. <pub-id pub-id-type="doi">10.1111/jon.12261</pub-id><pub-id pub-id-type="pmid">25996962</pub-id></citation></ref>
<ref id="B14">
<label>14.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chang</surname> <given-names>KV</given-names></name> <name><surname>Wu</surname> <given-names>WT</given-names></name> <name><surname>Han</surname> <given-names>DS</given-names></name> <name><surname>&#x000D6;z&#x000E7;akar</surname> <given-names>L</given-names></name></person-group>. <article-title>Ulnar nerve cross-sectional area for the diagnosis of cubital tunnel syndrome: a meta-analysis of ultrasonographic measurements</article-title>. <source>Arch Phys Med Rehabil</source>. (<year>2018</year>) <volume>99</volume>:<fpage>743</fpage>&#x02013;<lpage>57</lpage>. <pub-id pub-id-type="doi">10.1016/j.apmr.2017.08.467</pub-id><pub-id pub-id-type="pmid">28888384</pub-id></citation></ref>
<ref id="B15">
<label>15.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yazdchi</surname> <given-names>M</given-names></name> <name><surname>Tarzemani</surname> <given-names>MK</given-names></name> <name><surname>Mikaeili</surname> <given-names>H</given-names></name> <name><surname>Ayromlu</surname> <given-names>H</given-names></name> <name><surname>Ebadi</surname> <given-names>H</given-names></name></person-group>. <article-title>Sensitivity and specificity of median nerve ultrasonography in diagnosis of carpal tunnel syndrome</article-title>. <source>Int J Gen Med.</source> (<year>2012</year>) <volume>5</volume>:<fpage>99</fpage>&#x02013;<lpage>103</lpage>. <pub-id pub-id-type="doi">10.2147/IJGM.S17785</pub-id><pub-id pub-id-type="pmid">22319247</pub-id></citation></ref>
<ref id="B16">
<label>16.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sahebari</surname> <given-names>M</given-names></name> <name><surname>Pezeshki Rad</surname> <given-names>M</given-names></name> <name><surname>Nahayati</surname> <given-names>MA</given-names></name> <name><surname>Saeidi</surname> <given-names>M</given-names></name> <name><surname>Boostani</surname> <given-names>R</given-names></name> <name><surname>Hamidian</surname> <given-names>Y</given-names></name> <etal/></person-group>. <article-title>High resolution ultrasonography of cross sectional area of median nerve compared with electro diagnostic study in carpal tunnel syndrome</article-title>. <source>Rheumatol Res</source>. (<year>2017</year>) <volume>2</volume>:<fpage>127</fpage>&#x02013;<lpage>31</lpage>. <pub-id pub-id-type="doi">10.22631/rr.2017.69997.1031</pub-id></citation></ref>
<ref id="B17">
<label>17.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>KM</given-names></name> <name><surname>Kim</surname> <given-names>HJ</given-names></name></person-group>. <article-title>Relationship between electrodiagnosis and various ultrasonographic findings for diagnosis of carpal tunnel syndrome</article-title>. <source>Ann Rehabil Med</source>. (<year>2016</year>) <volume>40</volume>:<fpage>1040</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.5535/arm.2016.40.6.1040</pub-id><pub-id pub-id-type="pmid">28119834</pub-id></citation></ref>
<ref id="B18">
<label>18.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Atroshi</surname> <given-names>I</given-names></name> <name><surname>Gummesson</surname> <given-names>C</given-names></name> <name><surname>Johnsson</surname> <given-names>R</given-names></name> <name><surname>Ornstein</surname> <given-names>E</given-names></name> <name><surname>Ranstam</surname> <given-names>J</given-names></name> <name><surname>Ros&#x000E9;n</surname> <given-names>I</given-names></name></person-group>. <article-title>Prevalence of carpal tunnel syndrome in a general population</article-title>. <source>JAMA</source>. (<year>1999</year>) <volume>282</volume>:<fpage>153</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1001/jama.282.2.153</pub-id><pub-id pub-id-type="pmid">10411196</pub-id></citation></ref>
<ref id="B19">
<label>19.</label>
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Duncan</surname> <given-names>SF</given-names></name> <name><surname>Kakinoki</surname> <given-names>R</given-names></name></person-group>. <article-title>(eds</article-title>.). <source>Carpal Tunnel Syndrome and Related Median Neuropathies: Challenges and Complications</source>. <publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name> (<year>2017</year>).</citation></ref>
<ref id="B20">
<label>20.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aboonq</surname> <given-names>MS</given-names></name></person-group>. <article-title>Pathophysiology of carpal tunnel syndrome</article-title>. <source>Neurosciences</source>. (<year>2015</year>) <volume>20</volume>:<fpage>4</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="pmid">25630774</pub-id></citation></ref>
<ref id="B21">
<label>21.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Soman</surname> <given-names>D</given-names></name></person-group>. <article-title>Carpal tunnel syndrome - an unusual entrapment neuropathy with hypertensive microvascular insufficiency</article-title>. <source>BMH Med J</source>. (<year>2017</year>) <volume>4</volume>:<fpage>134</fpage>&#x02013;<lpage>8</lpage>.</citation></ref>
<ref id="B22">
<label>22.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Oktayaglu</surname> <given-names>P</given-names></name> <name><surname>Nas</surname> <given-names>K</given-names></name> <name><surname>Kilin&#x000E7;</surname> <given-names>F</given-names></name> <name><surname>Tasdemir</surname> <given-names>N</given-names></name> <name><surname>Bozkurt</surname> <given-names>M</given-names></name> <name><surname>Yildiz</surname> <given-names>I</given-names></name></person-group>. <article-title>Assessment of the preserve of carpal tunnel syndrome in patients with diabetes mellitus, hypothyroidism, and acromegaly</article-title>. <source>J Clin Diagn Res.</source> (<year>2015</year>) <volume>9</volume>:<fpage>OC14</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.7860/JCDR/2015/13149.6101</pub-id></citation></ref>
<ref id="B23">
<label>23.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chang</surname> <given-names>KV</given-names></name> <name><surname>Wu</surname> <given-names>WT</given-names></name> <name><surname>Huang</surname> <given-names>KC</given-names></name> <name><surname>Jan</surname> <given-names>WH</given-names></name> <name><surname>Han</surname> <given-names>DS</given-names></name></person-group>. <article-title>Limb muscle quality and quantity in elderly adults with dynapenia but not sarcopenia: an ultrasound imaging study</article-title>. <source>Exp Gerontol</source>. (<year>2019</year>) <volume>108</volume>:<fpage>54</fpage>&#x02013;<lpage>61</lpage>. <pub-id pub-id-type="doi">10.1016/j.exger.2018.03.019</pub-id></citation></ref>
</ref-list> 
</back>
</article>