We sought temperature data from contact temperature sensors. Studies using “conventional” contact temperature sensors—those that can be affixed to the skin surface to measure T_skin—were included; other contact sensors (including those temporarily held against the skin) were included only in cases where they provided insight into setup considerations for affixable sensors. Measurements from liquid-crystal thermometers or other colour-change materials and measurements from non-contact methods were excluded. Studies were included if human or other animal T_skin was measured in vivo; surface temperature (T_surface) measurements of a physical model were also included if the model was used in the context of understanding human T_skin measurement and if the temperatures investigated were physiologically relevant for humans. Included studies were required to report the measurement of a suitable comparison temperature, typically another T_skin or T_surface (the comparator determined to what outcome subgroup the data belonged; see data synthesis below). Comparisons performed simply for the purpose of validating or correcting a prototype sensor system were excluded (e.g., Chen et al., 2015). Comparisons with non-contact methods or numerical models, while useful in their own right, were beyond the scope of this article. (The comparison of contact and infra-red devices for measuring T_skin has been reviewed elsewhere; Bach et al., 2015b.) Studies investigating T_skin or T_surface during clinical or rehabilitation treatments/conditions were excluded (e.g., cryotherapy, ultrasound therapy, radiotherapy, other hyperthermia therapy). Studies published from 1960 (until 13.07.2016) were considered for inclusion, which was a modification from the unrestricted date range specified in the protocol (see Supplementary Material Appendix 1).

We sought general information about T_skin sensors and their use in published research (e.g., sensor type, attachments, conditions of sensor use, and use of the T_skin data). Studies included were those that used contact temperature sensors for the measurement of human T_skin in vivo during experimental studies involving sport, exercise, and other physical activity (PA; hereafter for practicality, sport, exercise, occupational and other PA will be collectively termed PA). The sensors had to be affixed to the skin surface in some way and remain in place during the PA. While measurement of T_skin is relevant beyond situations involving PA (e.g., clinical use, passive heat stress, sleep studies, circadian rhythm studies), measurements in these contexts were beyond the scope of this review.

For inclusion, studies needed to report T_skin data recorded during or immediately following PA, or use that data to calculate another variable that was reported in the results. Studies were excluded if they involved only “trivial” PA, such as standing tasks. Studies in which it was clear the T_skin data had been previously published in article form and studies involving daily monitoring were excluded unless that monitoring was part of a specific occupational routine including an expected component of PA. For objective 2, only studies published in the years 2011–2016 (until 13.07.2016) were included to be reflective of current practice at the time of review.

The search yielded 4,299 records (Figure 1). Following removal of duplicates (n = 1,095) and exclusion during screening (n = 3,184), 20 studies remained. An additional study was located from searches of the reference lists. Thus, 21 studies were finally included (Yakovlev and Utekhin, 1965, 1966; Guadagni et al., 1972; Jirak et al., 1975; Flesch et al., 1976; Mahanty and Roemer, 1979a,b; Krause, 1993; Dollberg et al., 1994; Lee et al., 1994; Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Deng and Liu, 2008; Harper Smith et al., 2010; Youhui et al., 2010; Tyler, 2011; James et al., 2014; Psikuta et al., 2014; Bach et al., 2015a; McFarlin et al., 2015; Priego Quesada et al., 2015). From these 21 studies, 31 distinct subsets of comparisons were identified. A subset is considered here as data from a particular study that addresses a distinct aspect of one (or more) of the outcome subgroups. Four subsets (Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; Tyler, 2011) were applicable to two outcome subgroups and one study (Psikuta et al., 2014) was applicable to four, giving 38 subsets in total.

Selected general information from the included studies is given in Supplementary Material Table 4. From the 21 studies included, the relevant outcome data comprised measurements from a physical model in nine studies (Yakovlev and Utekhin, 1965, 1966; Flesch et al., 1976; Mahanty and Roemer, 1979a; Krause, 1993; Lee et al., 1994; James et al., 2014; Psikuta et al., 2014; Priego Quesada et al., 2015) and measurements from the skin of human subjects in 17 studies (Yakovlev and Utekhin, 1965, 1966; Guadagni et al., 1972; Jirak et al., 1975; Mahanty and Roemer, 1979a,b; Dollberg et al., 1994; Lee et al., 1994; Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Deng and Liu, 2008; Harper Smith et al., 2010; Youhui et al., 2010; Tyler, 2011; James et al., 2014; Bach et al., 2015a; McFarlin et al., 2015). In human studies, T_skin data was reported from single body sites in 10 studies (Yakovlev and Utekhin, 1965, 1966; Jirak et al., 1975; Mahanty and Roemer, 1979a,b; Dollberg et al., 1994; Lee et al., 1994; Deng and Liu, 2008; Youhui et al., 2010; McFarlin et al., 2015) and as the mean of multiple sites in seven studies (Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; Youhui et al., 2010; Tyler, 2011; James et al., 2014; Bach et al., 2015a); in two studies it was unclear if the data represented single sites or a mean of multiple sites (Yakovlev and Utekhin, 1966; Mahanty and Roemer, 1979b). The range for the number of skin sites used within a study was 1–14 (3–14 for those that reported mean T_skin).

This second search yielded 313 records (Figure 1). Following removal of duplicates (n = 9) and exclusion during screening (n = 132), 172 studies were retained (see in-text Appendix). These studies included 2,267 participants (2014 male, 213 female, 40 not specified). T_skin data were reported for participants cycling in 84 studies (49% of included studies), running in 40 studies (23%), walking in 40 studies (23%), occupational PA in 7 studies (4%), and other PA in 18 studies (11%). Most studies (90%) also reported resting T_skin data.

A summary of the information about the sensors, attachments, and data use is given in Table 1. The type of contact sensor used (e.g., thermistor, thermocouple) was reported in 84% of studies. Of the studies reporting sensor type, thermistors were the most common (89 studies) followed by thermocouples (30 studies) and iButtons (oscillator-based digital thermometer; 26 studies). Identifying the sensor manufacturer (e.g., Grant Instruments Ltd, Cambridge, UK) and model (e.g., EUS-UU-VL3-0) was not always possible: in 34 studies (20%) no manufacturer or supplier information was reported and in 65 studies (38%) no sensor model information was reported. Details about sensor calibration were rarely reported with 94% of studies providing no or unclear information. Similarly, most studies (89%) reported no error-related information about the sensor (e.g., uncertainty, precision). Over half of the studies reported no information about the method of sensor attachment: of the 73 studies (42%) reporting clear information, 63 studies used tape. It was possible to determine whether the sensors were covered or uncovered in 40% of studies, with the remaining 60% unclear or not reported. Most studies reported mean T_skin (83%) and almost all studies reported absolute temperature values (97%). Irrespective of the number of measurement sites used, 57 studies (33%) reported some data from single measurement sites or single sensor.

The number and location of body sites used are given in Supplementary Material Tables 5 and 6, respectively. The number of body sites per participant ranged from one to sixteen, with four sites most commonly used (46%). Chest (74%), anterior thigh (71%), lower leg (70.3%), and upper arm (56%) were the most common sites used.

Data from one study were identified (Mahanty and Roemer, 1979a) (n = 1 subset, giving 2 comparisons; Figure 3). The surface temperature of a physical model was estimated from a thermocouple 0.4 mm below the surface, without and with a surface temperature sensor directly above (undisturbed and disturbed states, respectively). The undisturbed surface temperature was similar to both the disturbed temperature and the surface thermistor probe acting as the disturbance (mean differences of <0.1°C for the point estimates).

Data from three studies were identified (Mahanty and Roemer, 1979a; Lee et al., 1994; Psikuta et al., 2014) (n = 4 subsets, giving 96 comparisons; Figure 4 and Supplementary Material Figure 3). Temperature of the externally applied surface sensors was typically lower than invasive T_skin in vivo (Lee et al., 1994) and lower than the model reference temperature (Lee et al., 1994; Psikuta et al., 2014), although the magnitude of the mean differences was variable, ranging from <0.5°C (Mahanty and Roemer, 1979a; Lee et al., 1994; Psikuta et al., 2014) to 5.8°C (Psikuta et al., 2014). For surface T_skin sensors and attachments commonly used in non-clinical human research studies (Psikuta et al., 2014), greater differences were observed with a greater surface-environment gradient (see also influence of the environmental conditions). The data from Krause (1993) was not included here because the reference temperature sensors within the physical model are not described and no plate temperature data is reported.

Data from six studies were identified (Dollberg et al., 1994; Buono and Ulrich, 1998; Deng and Liu, 2008; Tyler, 2011; Psikuta et al., 2014; Priego Quesada et al., 2015) (n = 6 subsets, giving 19 comparisons; Figure 5 and Supplementary Material Figure 4). The type of attachment used had mixed effects on the mean differences of T_skin or T_surface with absolute mean differences ranging from 0.1 to 1.4°C. The LoA were typically within ±1.0°C for measurements on human skin, while those on physical models ranged from 0.3 to 1.5°C.

Using an attachment resulted in numerically greater measured temperatures by 0.1–1.3°C compared to the same sensors “uncovered” for human skin (Buono and Ulrich, 1998; Tyler, 2011) or a model surface (Priego Quesada et al., 2015). With increasing environmental temperature, the differences of covered versus “uncovered” sensors became smaller in one study (mean difference of 1.3°C in a 23°C environment, decreasing to 0.4°C in 42°C) (Buono and Ulrich, 1998) but tended to become larger in another study (mean difference of 0.1°C increasing to 0.5°C for one layer of tape, 0.8°C increasing to 0.9°C for two layers of tape plus bandage, environments of 24° and 35°C, respectively; Supplementary Material Figure 4) (Tyler, 2011). Shielding the sensor (Dollberg et al., 1994), increasing the attachment thickness (Deng and Liu, 2008), or increasing the number of attachment layers (Tyler, 2011) tended to increase the measured temperature. One study with pre-term infants inside incubators had a pertinent risk of bias in that the measured T_skin itself influenced the environmental conditions within the incubator (Dollberg et al., 1994). In that study, higher measured temperature caused lower environmental temperatures and so this risk of bias likely mitigated the magnitude of the observed difference: indeed, the net effect was a 0.8°C greater skin-to-environment temperature gradient when the shielding attachment was used. Elsewhere, aluminium tape typically resulted in higher measured temperatures than other common surgical tapes (>1°C for pooled mean differences), although interestingly, these higher T_surface were typically closer to the expected T_surface of the aluminium plate (Psikuta et al., 2014) (see also section Thermal Equilibrium of the Surface Sensor with the Underlying Temperature).

Data from four studies were identified (Yakovlev and Utekhin, 1966; Guadagni et al., 1972; Jirak et al., 1975; Mahanty and Roemer, 1979b) (n = 4 subsets; 2 subsets displayed, giving 43 comparisons; Figure 6 and Supplementary Material Figure 5). Absolute mean differences ranged from 0 to 1.33°C for the pressure comparisons available (Figure 6). The measured temperature tended to increase with increasing pressure over the range of 2–681 mmHg (Jirak et al., 1975; Mahanty and Roemer, 1979b). Halving or doubling the applied pressure resulted in mean differences consistently within ±0.5°C (note that the common comparison pressures were used as presented in the original articles) (Jirak et al., 1975; Mahanty and Roemer, 1979b). For example, for a circular sensor at the forehead the mean differences, versus 136 mmHg, were −0.31°C at 68 mmHg and +0.36°C at 272 mmHg (Jirak et al., 1975; Supplementary Material Figure 5). From the data of one subset (Jirak et al., 1975), the LoA were typically greater when the higher pressures were included (409, 545, and 681 mmHg), although all within ±1.0°C.

One of the subsets not able to be displayed in the plots (Guadagni et al., 1972) was simply summarised in the original article as a statement that the effect of pressure on steady state skin temperature (human participants; sites not reported) was <0.01°C at pressures between 31 and 52 mmHg, and that at pressures beyond this range, the steady state temperature becomes pressure-dependent and increases with increasing pressure. For the other subset not able to be displayed (Yakovlev and Utekhin, 1966), differences of 0.1–0.7°C were reported for the seven sensor types tested on human skin (sites not reported) whereby the pressure was generated by 5 and 70 g weights (sensor surface areas were not reported, therefore the pressures are unknown). Notwithstanding, the authors did state, without the supporting data, that pressure effects can be disregarded when the maximum pressure of the sensor on the skin does not exceed 15–37 mmHg.

Data from six studies were identified (Yakovlev and Utekhin, 1966; Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; Tyler, 2011; Psikuta et al., 2014) (n = 6 subsets; five subsets displayed, giving 38 comparisons; Figure 7 and Supplementary Material Figure 6).

For the subsets in which human participants were used (Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; Tyler, 2011), potential influences of the environment can be examined indirectly via the relative differences for the same two comparators under different environments—here, the different environments cannot be directly compared because the T_skin will also change. (Because these indirect comparisons may be confounded by other variables such as the differences in absolute T_skin, they are interpreted cognizant of this possible but unclear risk of bias.) Irrespective of the underlying cause, the mean difference between the same two comparators were often different when the comparison was made under different environmental temperatures in three subsets (Buono and Ulrich, 1998; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010): for example, relative mean differences of 1.3 and 0.4°C for the same attachment comparison under 23 and 42°C environmental temperatures (Buono and Ulrich, 1998). In other words, the magnitude of mean difference for a given comparison of sensor setups showed some dependence on the environmental temperature. For the influence of air movement during rest, the pattern of relative differences at 10, 20, and 30°C environmental temperatures was similar under both 0.2 and 2.3 m/s air velocities (e.g., relative mean differences of 0.70 and 0.79°C for the two velocities, respectively, in a 20°C environment; Supplementary Material Figure 6) (Harper Smith et al., 2010).

For one subset in which a model was used (Psikuta et al., 2014), data are presented in Figure 7 as pooled estimates for main effects of the environmental variables. In this subset (Psikuta et al., 2014), compared to T_surface measurements in the 35°C environment, the cooler environments resulted in lower, and more variable, measured values despite the plate being maintained at ~36.5°C; the magnitudes of mean differences were often of practical relevance (>0.5°C). Comparing wind velocity within the same environmental temperature indicates that the influence of an increase in wind velocity (from 0.5 to 1.2 m/s) was negligible when the environmental temperature (35°C) was close to the T_surface (36.5°C) but became increasingly relevant as the environmental temperature diverged further from the plate temperature (relative differences of −0.4 and −1.0°C for the point estimates in environments of 26 and 16°C, respectively). Data from the other subset using a physical model are not presented due to uncertainties in interpretation of the environment and the reference T_surface but the authors did state effects of the ambient air temperature being 0.2–6.0°C for seven different sensor types (Yakovlev and Utekhin, 1966).

Data from 14 studies were identified (Yakovlev and Utekhin, 1965, 1966; Jirak et al., 1975; Flesch et al., 1976; Mahanty and Roemer, 1979a,b; Krause, 1993; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; Youhui et al., 2010; James et al., 2014; Psikuta et al., 2014; Bach et al., 2015a; McFarlin et al., 2015) (n = 17 subsets; 14 subsets displayed, giving 87 comparisons; Figure 8 and Supplementary Material Figure 7). The risk of bias for calibration of these sensors used was considered high for four subsets [in vivo (McFarlin et al., 2015) and models (Flesch et al., 1976; Krause, 1993), (James et al., 2014) model uncorrected], unclear for five subsets [in vivo (Yakovlev and Utekhin, 1965, 1966; Mahanty and Roemer, 1979a; Youhui et al., 2010), and model (Yakovlev and Utekhin, 1965)], and low for eight subsets [in vivo (Jirak et al., 1975; Mahanty and Roemer, 1979b; van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; James et al., 2014; Bach et al., 2015a) and models (Psikuta et al., 2014; James et al., 2014) model corrected].

Absolute mean differences for some subsets were almost consistently within (Youhui et al., 2010; James et al., 2014) or beyond 0.5°C (Yakovlev and Utekhin, 1965, 1966; Jirak et al., 1975; Krause, 1993; McFarlin et al., 2015). For four in vivo studies, the LoA were almost consistently within ±1.0°C (van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; James et al., 2014; Bach et al., 2015a) and in two studies consistently greater than ±1.0°C (Yakovlev and Utekhin, 1965; McFarlin et al., 2015).

Data from three subsets were not able to be presented in the plots due to insufficient information [human participants at rest (Mahanty and Roemer, 1979a,b) and a model (Yakovlev and Utekhin, 1965)]. According to the text of the original articles, a thermocouple taped to the skin and a thermistor probe “agreed within ±0.15°C (maximum difference)” on one occasion (Mahanty and Roemer, 1979a) and had a mean difference of 0.03°C on the other occasion (Mahanty and Roemer, 1979b). For the physical model (Yakovlev and Utekhin, 1965), a nickel wire sensor and a “point” sensor (diameter 0.3–1 mm) were compared following the wetting of the suede surface of a model: the temperature from the nickel wire sensor decreased by approximately 4°C over the subsequent 2 min whereas the temperature from the “point” sensor decreased by <1°C over the same period.

This is the first systematic review to comprehensively collate data from comparisons of measurements made using contact T_skin sensors. A strength of this work is the breadth of information included and the ability to examine specific outcomes of interest in detail.

While comparisons of outcome data were typically possible, meta-analyses were not performed due to the appreciable variety of test conditions, sensor setups, and types of temperature data used (e.g., single site versus mean T_skin; multiple measurements within a period versus period grand mean), and the resultant lack of a common reference condition. Further, the lack of a universal gold-standard method for measuring T_skin limited the ability to judge one comparator T_skin sensor setup as being more accurate than the other. For this reason, emphasis was typically placed on the magnitude of effects rather than the direction of the effect.

The often limited reporting of methodological information contributed to the number of “unclear” judgements within the risk of bias assessments and made extraction of study information or data challenging at times. To guide both study design and reporting of study information in future studies comparing temperature measurements, we recommend authors to prospectively think about how their work will be judged during a subsequent risk of bias assessment. Details to consider include—but are not limited to—sequence generation, any blinding, completeness of outcome data (e.g., detachment of sensors, sensor malfunctions), selective reporting (data that was intended to be reported but was not), consistency of test conditions, sensor calibration details, and study support.

We used threshold values to indicate practical significance throughout this review (human skin in vivo, ±0.5°C for mean difference and ±1.0°C for 95% LoA; physical models, ±0.5°C for mean difference and 95% LoA). This was a necessary simplification for feasibility and is intended as a general guide only. It is likely that, in practice, these thresholds will need to be adjusted according to specific measurement contexts and objectives. For example, multiple spot measurements from contact T_skin sensors are often used to calculate a mean T_skin, ostensibly representative of the body as a whole. Temperature differences due to the sensor setup itself may in some cases be within differences due to other experimental decisions such as site selection, replication of sensor placement, or the weightings used (Livingstone et al., 1987; Choi et al., 1997).

It is acknowledged that additional descriptive statistics are also useful for making inferences in method comparisons (e.g., typical error of the estimate, regression and correlation coefficients). Finally, not considered here was any influence of the signal processor/data logger setup (Jutte et al., 2005).

While the skin surface is an accessible site for measurement, it presents difficulties for accurate quantification of temperature in that the surface is an interface between distinct mediums, with each medium having its own thermal properties and temperature gradients. Accordingly, the sensor system is in partial contact with medium of interest (i.e., the skin) and partial contact with the adjacent environment (e.g., microclimate air, liquids, clothing). Accurate knowledge of T_skin requires the sensor temperature to be in equilibrium with the corresponding skin temperature and for the sensor setup itself to have negligible additional effect on heat exchanges between the underlying skin and its adjacent environment(s) (Bedford and Warner, 1934; Hardy, 1934b; Childs, 2001; Taylor et al., 2014). Demonstrable differences among various T_skin sensor setups may reduce, at least in part, to how well each respective sensor setup achieves this balance between thermal equilibrium and temperature modification. Selected sources of error that may be expected during T_skin measurements are given in Figure 9, along with suggestions for how such an error may be minimised (Michalski et al., 2001; Nicholas and White, 2001). Considering sources of error and error minimisation can assist T_skin sensor design or assist sensor selection for those measuring T_skin. Notwithstanding, ideal dimensions or thermal properties of sensors and their attachments cannot govern sensor design or setup selection alone as these devices must also be robust and suitable enough for practical use (Youhui et al., 2010; Webb et al., 2013).

The data identified within this review were limited and incompletely address the question of validity of common T_skin sensors and so future work is warranted. In particular, estimates of uncertainty during T_skin measurements and the magnitude of any effects of the sensor and attachment disturbing the underlying T_skin are unclear. From data of the only study identified, there was no demonstrable effect of thermal disturbance of the site underlying a surface sensor (Mahanty and Roemer, 1979a). However, it is unlikely this finding is generalizable to conventional T_skin sensor types and across environmental conditions as the surface probe used was designed specifically to minimise any such modification of T_surface and the environmental temperature used (~22°C) represented a moderate difference from the surface temperature (29–33°C). Notwithstanding, this finding does illustrate the promise of sensor design based on error minimisation (Figure 9). Key characteristics of this sensor included using a small sensing component (thermistor) housed in a stainless steel disc (19 mm diameter, 0.13 mm thick), which was partially painted matte black (high emissivity, similar to skin) and partially covered with reflective silver coating to compensate for addition heat losses due in part to the surface area of the sensor system being greater than that of the skin itself (Mahanty and Roemer, 1979a).

Numerical modelling has indicated that insulated T_skin sensors can cause the temperature of the underlying skin to be higher than it would otherwise be because insulating the sensor from the adjacent environment also insulates the underlying skin (Boetcher et al., 2009). On the contrary, sensors with high thermal conductivity and no insulating layer can lower the underlying T_skin by inducing a fin effect whereby the total surface area of the sensor exposed to the environment is greater than that of the skin it covers and, in this way, heat conducted from the skin to the sensor can be lost by convection to the adjacent environment at a rate that exceeds that of the undisturbed skin itself (Guadagni et al., 1972; Boetcher et al., 2009). Heat transfer from the sensor to the environment by radiation may be greater or lesser than that of the undisturbed skin because, despite the greater surface area available for radiating heat in the case of the sensor, the emissivity of the skin is itself very high (~0.98; Steketee, 1973).

The skin has a boundary layer of air that mitigates the temperature difference directly adjacent to its surface. This difference under warm-hot conditions (relatively still air, ambient temperature of 28–29°C) may be <0.5°C for air within 1 mm, and <1°C for air up to 6 mm from the skin surface (McGlone and Bazett, 1927). In cooler conditions or with forced air convection (e.g., wind or body movements), this gradient increases and the temperature difference can become appreciable within the height of commonly used T_skin sensors and their attachments. For surface T_skin sensor equilibrium with the underlying temperature, a range of magnitudes was observed from <0.5°C (Mahanty and Roemer, 1979a; Lee et al., 1994; Psikuta et al., 2014) up to 6°C (Psikuta et al., 2014). However, details about the ambient environment were not clear from the two of the studies included here (Mahanty and Roemer, 1979a; Lee et al., 1994).

Psikuta et al. (2014) observed that ambient temperature, and thus the surface-environment gradient, can influence the estimated error: while few sensor-attachment combinations were within 1°C of the plate temperature in the 16°C environment, all were within this range in the 35°C environment. The temperature of the aluminium plate was reported as 36.5±0.03°C in this experiment and, therefore, it is intuitive that errors tend to reduce as the ambient temperature approaches the surface temperature.

Although numerical modelling and non-contact methods can also add insight, it was not feasible to also cover these in detail here and contact versus non-contact methods have been discussed elsewhere (Bach et al., 2015b). As mentioned previously, numerical modelling has indicated that, at a simulated environmental temperature of 22°C and T_skin of ~31.7°C, errors approaching 0.5°C can be expected with insulated contact sensors (Boetcher et al., 2009). Numerically modelling performed by Deng and Liu (2008) indicated that, for the attachment material, decreasing the thickness and increasing the thermal conductivity decreased the measurement error. These results are consistent with experimental observation of aluminium tape tending to better reflect the underlying plate temperature, particularly at low wind velocities (0.5 m/s) (Psikuta et al., 2014). For improving absolute temperature measurement, more work is required to delineate measurement errors in the context of human skin, especially during sweating.

Sensor setup variables can influence the measured values from T_skin sensors. Examples of both small and practically significant effects (less than or greater than 0.5°C absolute mean difference, respectively) can be found for the outcome subgroups of attachment type, sensor-applied pressure, environmental conditions, and sensor type. The 95% LoA were often, though not always, within ±1.0°C for human skin in vivo and ±0.5°C for physical models. An implication of these observations is that equivalent measurements of T_skin cannot be assumed when different components of the sensor setup or different environments are used. Thus, these setup variables (e.g., sensor type, attachment, environmental conditions) should be considered prior to measurement and clearly reported to assist subsequent interpretation of T_skin measurements. Further work could identify or experimentally justify particular setups or conditions in which measurement biases are mitigated (Figure 9).

Relative temperature differences between sensor types can vary with the environmental temperature, activity, and time (van Marken Lichtenbelt et al., 2006; Harper Smith et al., 2010; Bach et al., 2015a), although this is not always the case (James et al., 2014). In other words, two different setups may not simply be a consistent offset from each other, but rather vary in magnitude with other parameters. Physical properties of the sensors and their attachments likely contribute to the consistency, or inconsistency, of relative differences, particularly during transients.

Some patterns in temperature differences during changes in environmental temperatures could reflect an influence of thermal inertia (Supplementary Material Figure 7A): iButtons appeared to either have a reduced response time to a true change in T_skin or, alternatively, they were less influenced by the change in environment and better reflected the true change in T_skin (van Marken Lichtenbelt et al., 2006; Bach et al., 2015a). The practical consequences due to the thermal inertia of the sensor setup will likely depend on the time course of the expected change in T_skin. The response time of the sensor on the skin surface should be a key consideration when relatively fast changes in T_skin are expected (Arunachalam et al., 2008), but may be of lower importance compared to other considerations when a relatively stable T_skin is expected. In research settings it is common—and good practice—for participant instrumentation to occur well before experimental recordings are made, which should mitigate any transient effects of initial sensor temperature before application and the duration of a sensor to reach a thermal steady state once applied (Guadagni et al., 1972).

Surface pressure is known to influence skin perfusion, however, studies here indicated that the measured T_skin temperature almost invariably increased with increasing pressure whereas skin perfusion tends to decrease, even with a little as 5 mmHg (Holloway et al., 1976). Physical effects of the surface sensor pressing into the skin were investigated further by Mahanty and Roemer (1979b) using numerical modelling and the calculated effects agreed closely with their experimental results, suggesting these physical factors associated with depression are a suitable explanation. While increasing the sensor-applied pressure may lead to temperature errors, the thermal contact must also be sufficient and, therefore, there is likely a trade-off between the two. Unstable temperature readings were attributed to insufficient thermal contact at pressures lower than 31 mmHg in one study (Guadagni et al., 1972), but such instability issues were not apparent in another study investigating pressures of 2–20 mmHg (Mahanty and Roemer, 1979b). In the latter study, the authors suggested that 2 mmHg was sufficient for measuring T_skin and used numerical modelling to support this observation (Mahanty and Roemer, 1979b).

Controlling the applied pressure is a challenge for measuring T_skin in vivo. While sensor-applied pressure variations during normal use are not clear, it is notable that mean differences in T_skin due to halving or doubling a given applied pressure were <0.5°C. This finding was consistent across the wide range of pressures investigated (2–681 mmHg). Increasing the sensor surface area can be considered with respect to limiting the applied pressure and associated depression into the skin, but any such increases in area may have consequences for modifying evaporative and non-evaporative heat exchanges between the skin and the environment.

A challenge for measurement comparisons on human skin is that T_skin varies by time and location (Pennes, 1948; Kaufman and Pittman, 1966; Webb, 1992). Possible comparisons on human skin include using identical measurement sites at different times, or simultaneous measurements at different sites. While some authors compare results from single sites, others use multiple measurement sites and compare the resultant mean T_skin. With respect to temporal changes in T_skin of human participants, Guadagni et al. (1972) observed that after 2.5–3 h rest in a room temperature of 24.5°C, changes in T_skin at a given site were <0.01°C within 2 or 3 min periods. A different approach was used by Mahanty and Roemer (1979b) whereby throughout a measurement period involving different experimental sensor-applied pressures, adjacent temperature sensors remained in place to allow minor temperature changes not due to experimental sensor pressure to be compensated for. Irrespective of the approach, care needs to be taken to minimise risks of systematic temperature bias.

With the expanding interest in wearable technology for day-to-day applications, there should be an expectation of acceptable validity for any measurements being taken (Sperlich and Holmberg, 2017). A direct outcome here is that sensor location and the effects of the carrier devices or attachments will need to be taken into account for such applications.

Here we have collected information about how contact T_skin sensors are currently (2011–2016) used and reported in published studies involving sport, exercise and other PA. Thus, this work provides context and a basis for integrating the findings delineated above in more challenging conditions than rest. These findings may, however, only provide limited information on these aspects in other research or clinical conditions.

In human studies, details about T_skin sensor setup details were often poorly reported and, from those reporting setup information, it was evident that the setups often varied in terms of the sensors, attachments, and locations used. Key setup variables need to be considered further and consistently reported. For example, the sensor attachment method was clear in only 42% of the articles sampled yet attachment can bias the measured values (Buono and Ulrich, 1998; Psikuta et al., 2014). Similarly, clear information about sensor calibration was present in only 6% these studies.

Absolute T_skin values are used often in research with 97% of the studies sampled here reporting some form of absolute T_skin and 17% reporting change scores (some reported both, but not necessarily for all the data). While common inferential statistical approaches end up treating absolute values as relative differences (e.g., within-subject differences for repeated measures, difference between group means for independent samples), the point remains that absolute values are of interest to researchers and other users of T_skin information. Thus, clearer reporting of setup variables may also improve awareness about the external validity of published absolute T_skin data, such as for inter-study comparisons or applied use of T_skin (e.g., temperature thresholds during heat stress).

Sensors used in the comparisons of measured temperatures (objective 1) only partially represented the sensors currently used in human studies involving PA. The outcome of some comparisons may be more generally applicable, such as effects of the sensor pressure, whereas the outcomes of other comparisons may be restricted to specific setups or environments, such as sensor type or a particular skin-environment temperature gradient. A reason contributing to specificity of comparison outcomes is that different sensor setups likely vary in variables not taken into account here, such as sensor and attachment composition (and resultant thermal properties), dimensions, and surface contact. Future work will benefit from the identification of setup variables suitable for widespread use.