Dutch elite athletes ≥16 years old and practicing an outdoor sport discipline on an international level were eligible to participate in our study. Exclusion criteria were based on the use of the ingestible temperature capsule: (I) a bodyweight <36.5 kg, (II) an implanted electro-medical device, (III) a history of obstructive/inflammatory bowel disease or surgery, or (IV) a scheduled MRI scan within 5 days of the experiment. Participant characteristics of the analytical cohort (n = 101) as well as for groups based on sex and sport discipline are presented in Table 1. Participants were active in 13 different types of sport, including n = 27 endurance trained athletes (mountain biking n = 5, open water swimming n = 2, road cycling n = 7, triathlon n = 13), n = 28 mixed trained athletes (3 × 3 basketball n = 5, beach volleyball n = 8, field hockey n = 14, soccer n = 1), n = 11 power trained athletes (BMX n = 11), and n = 35 skill trained athletes (baseball n = 10, sailing n = 3, skateboarding n = 2, softball n = 20). None of the participating athletes conducted a dedicated heat acclimatization program prior to participation, and only four athletes reported having some heat exposure but were not acclimatized. The study was in accordance with the Declaration of Helsinki and was approved by the Medical Ethical Committee of the Radboud university medical center (#2018-4640). All participants gave their written informed consent prior to the testing procedures.

This study is part of the Thermo Tokyo research project, which rationale and design have been described in detail previously (de Korte et al., 2021b). In short, participants were invited to complete a personalized incremental exercise test on a cycling ergometer (Lode ergometer, Lode B.V., Groningen, Netherlands, or Tacx Neo Smart T2800, Tacx B.V., Wassenaar, Netherlands) in simulated hot and humid environmental conditions (ambient temperature 31.6 ± 1.0°C, relative humidity 74 ± 5%, Wet Bulb Globe Temperature 28.6 ± 0.8°C, ambient vapor pressure 3.45 kPa, absolute humidity 0.0245 kg/m³). The specific heat stress environmental conditions were chosen to simulate the heat stress expected for the Tokyo 2020 Olympic Games (Gerrett et al., 2019), which can be categorized as high according to previously described classifications based on the Wet Bulb Globe Temperature and relative humidity (Gonzalez, 1995; American College of Sports Medicine et al., 2007). Participants were instructed to refrain from strenuous exercise (24 h) and consumption of alcohol or caffeine (12 h) prior to the exercise test. Furthermore, all participants were instructed to consume their last meal ≥3 h preceding the experiment and consume 500 ml of water ~2 h before arriving at the laboratory. The exercise tests consisted of a 20 min warm-up at ±70% of the maximal HR, which was obtained from training data or a previously performed maximal exercise test. The warm-up phase was followed by an incremental phase during which the workload, measured in W, was increased by 5% every 3 min until volitional exhaustion. T_c, HR, and power output were measured continuously throughout the protocol.

After data extraction, individual data records were cleaned by manually removing erroneous outlier values for T_c. Thereafter, min averages of T_c and HR were calculated using a customized MATLAB and Statistics Toolbox (2012b, The MathWorks, Inc., Natick, USA) software package. Missing T_c data were linearly interpolated using a customized MATLAB script if gaps were <5 min. Gaps >5 min were visually inspected by two researchers and only interpolated if the interpolated data fitted the T_c curve. In case of discrepancy between the two researchers, the evaluation of a third researcher was decisive. All case-by-case visual data inspections were performed by the same three dedicated and experienced researchers (JK, CB, TE). A total of 106 elite athletes completed the exercise protocol. T_c measurements were missing in 3 participants due to a loss of sensor signal and data records from another 2 participants were excluded based on significant abnormalities in the T_c curve following a case-by-case review. The analytical cohort consisted of 101 athletes.

Bland-Altman plots were generated by plotting the average of T_c−est and T_c values against the difference between the two methods. Bias was computed as the mean of the difference between T_c−est and T_c, and a one-sample t-test was used to determine whether there was a systematic bias. To assess the agreement between T_c−est and T_c, the limits of agreement (LoA) were derived from Bland-Altman plots (Bland and Altman, 1986), modified for multiple non-constant measurements per individual (Bland and Altman, 1999, 2007). The LoA were calculated for all data obtained during the exercise test, as well as for peak T_c data specifically. A bivariate correlation plot was constructed, and the Pearson correlation coefficient was calculated to further assess the agreement between T_c−est and T_c. Furthermore, root mean square error (RMSE) weighted for participant and exercise test duration was computed (Buller et al., 2015). A one-way Analysis of Variance (ANOVA) was used to assess differences in the RMSE across subgroups of sex and sports discipline. Athletes were classified as endurance, mixed, power, or skill trained based on the relative isometric and isotonic components of their exercise training according to the European Society of Cardiology Guidelines (Pelliccia et al., 2021). The accuracy, sensitivity, and specificity of T_c−est to predict whether participants reached a certain core temperature value was examined using T_c−est classification thresholds between 37.0 and 39.75°C for all data and between 39.0 to 39.75°C for peak T_c data. We considered a sensitivity and specificity between 90 and 100% as excellent, 80–89% as good, 70–79% as fair, 60–69% as poor and <60% as failure to predict peak T_c. Statistical analyses were performed using SPSS Statistics v25 (IBM Corp, Armonk, NY) and data were considered significant if p < 0.05. All parameters were visually inspected for normality and all data were presented as mean ± standard deviation (SD) unless indicated otherwise.

The incremental exercise tests had a mean duration of 44 ± 10 min with a peak power output of 192 ± 53 W and a normalized peak power output of 2.6 ± 0.8 W/kg. Baseline HR was 82 ± 14 bpm and increased with 101 ± 15 bpm up to a peak HR value of 182 ± 12 bpm (Figure 1A). The range of HR was 45–207 bpm.

The dataset contained n = 5,025 T_c observations ranging from 35.8 to 40.4°C (Figure 2A). During the incremental exercise test, T_c increased from 37.1 ± 0.4°C at baseline to a peak T_c of 38.9 ± 0.6°C (Figure 1B). The computed T_c−est also consisted of n = 5,025 readouts and ranged from 36.7 to 39.9°C (Figure 2B). T_c−est increased to a peak value of 38.9 ± 0.3°C (Figure 1B).

A strong association was found between T_c−est and T_c (r = 0.86, p < 0.001, Figure 3A). A small yet significant systematic bias of 0.15 ± 0.36°C (p < 0.001) was found for T_c−est, with LoA of ±0.45°C and a RMSE of 0.35 ± 0.18°C (Figure 3C). For peak T_c (range: 37.6–40.4°C), a moderate association between T_c and T_c−est was found (r = 0.57, p < 0.001, Figure 3B). A lower systematic bias (−0.02 ± 0.47°C (p < 0.001)) but greater LoA (±0.92°C) and RMSE (0.47°C) was found for T_c−est at peak T_c (Figure 3D).

The Pearson correlation coefficients between T_c−est and T_c ranged from 0.85 to 0.90 across subgroups based on sex and sport discipline (all p-values < 0.001, Figure 4). No differences were found for RMSEs across subgroups based on sex (p = 0.79) and sport discipline (p = 0.81, Figure 4). The mean bias for T_c−est varied between 0.04 and 0.23°C across subgroups, with the lowest mean bias for the endurance trained athlete group (Figure 4).

The accuracy of T_c−est varied between 84 and 99% (Table 2). The sensitivity was excellent (>90%) between 37.0 and 38.25°C but gradually declined with the lowest sensitivity (<16%) for T_c−est > 39.25°C (Table 2). The specificity of T_c−est demonstrated an opposite pattern and varied between 24 and 100%. The highest specificity (>90%) was observed for T_c−est values >38.75°C and the lowest specificity (<25%) for T_c−est values of 37.0°C (Table 2). The prevalence of false-negative observations was higher compared to true positive observations for all T_c−est thresholds >39.0°C. A comparable pattern for sensitivity and specificity was observed for peak core temperatures (Supplementary Table S1).

We assessed the validity of the ECTemp™ algorithm in a unique and large group of 101 elite athletes exercising in the heat, using a well-controlled exercise protocol. Another strength of our study is the large number of T_c readouts (n = 5,025) allowing validation of the algorithm across a large T_c spectrum (35.8–40.4°C). Outcomes of our study can be of interest to athletes and coaches as the ECTemp™ algorithm may be used in regular training sessions with an expected T_c in the low to mid-range, or while evaluating heat preparedness strategies such as cooling interventions and monitoring physiological adaptations during heat acclimation. A limitation of our study is the use of an ingestible telemetric temperature capsule system as the reference measurement and surrogate marker for T_c. Although the myTemp system has been demonstrated to be valid (Bongers et al., 2018a,b), an intestinal sensor may respond less rapidly compared to the esophageal temperature at the start of exercise or to a change in exercise intensity (Byrne and Lim, 2007), resulting in an underestimation from the actual T_c and possibly even larger underestimation of T_c−est at any given time point. We have only examined the validity of ECTemp™ in a specific setting, namely during a laboratory-based incremental cycling protocol in hot and humid conditions, resulting in a gradual and progressive increase in T_c. As the T_c response may be very different in real-life field conditions (in terms of magnitude and the course over time), future studies are warranted to examine whether the performance of the algorithm is reproducible under field conditions. Such follow-up studies can also determine whether the ECTemp™ validity is reproducible across different exercise modalities (i.e., continuous vs. intermittent exercise) and environmental conditions (i.e., cool vs. moderate vs. hot). Another important consideration for future work is that most of our T_c observations are in the mid-range of the T_c spectrum. Especially higher T_c values are underrepresented. Although our findings are robust, additional research into the performance of the algorithm in the upper end of the T_c spectrum is warranted.