Mongolian gerbils (Meriones unguiculatus) were housed in standard animal racks (Bio A.S. Vent Light, Ehret Labor- und Pharmatechnik, Emmendingen, Germany) in groups of 2–3 animals per cage with free access to water and food at 20–24°C room temperature under 12/12 h dark/light cycle. The use and care of animals was approved by the state of Bavaria (Regierungspräsidium Mittelfranken, Ansbach, Germany, No. 54-2532.1-02/13). A total of 32 male gerbils aged 10–12 weeks were purchased from Janvier Laboratories Inc. and used in this study after acclimatization in our animal facility.

The pure tone acoustic trauma for tinnitus induction is applied under deep ketamine xylazine anesthesia as described in detail earlier (Ahlf et al., 2012; Walter et al., 2012; Tziridis et al., 2014, 2015; Krauss et al., 2016a). In a nutshell, the anesthetized animals were placed on a regulated heating pad with a temperature of 37°C central in front of a Loudspeaker (Canton Plus X Series 2; Canton, Weilrod, Germany). Using a signal generator (hp 33120A, HP, Böblingen, Germany) connected to an audio amplifier (Amp 75, Thomas Wulf, Frankfurt, Germany), a 2 kHz pure tone was presented at a sound pressure level of 115 dB SPL for 75 min.

For ASR measurements, as described earlier (e.g., Ahlf et al., 2012; Tziridis et al., 2012), animals were placed in a transparent acrylic tube (length 10 cm, inner diameter 4.3 cm) which was positioned at a distance of 10 cm in front of a loudspeaker (Canton Plus X Series 2), on a low-vibration table (TMC, Peabody, MA, USA). The whole setup was placed in an acoustic chamber (Industrial Acoustics Company GmbH, Niederkrüchten, Germany). The startle response was measured by a piezo force sensor (Honeywell FSG15N1A; sensitivity 0.24 mV/g; null shift at 25°C is ±1 mV; force range 0–1,500 g) attached underneath the tube. The front end of the tube was closed with a stainless steel grate (wire mesh, width 0.5 mm) allowing for acoustic stimulation with no detectable distortion within the used stimulation range of 250–8,000 Hz (signal-to-noise ratio at least 70 dB). Sound pressure level was calibrated using a condenser microphone (B and K Type 4190) via a preamplifier (B and K Type 2669) and measuring amplifier (B and K Type 2610). Stimulus generation and data acquisition was performed using custom-made programs (Matlab 2008, MathWorks, Natick, MA, USA). As startle amplitudes tend to be higher for the first few trials, five startle stimuli were presented before the beginning of each measurement to rule out strong habituation effects (Turner et al., 2006; Valsamis and Schmid, 2011).

The standard gap-startle protocol to measure behavioral correlates of tinnitus in rodents consists of several trials using a 20 ms long 115 dB SPL loud noise burst as startle stimulus presented in a continuous background band pass noise with a spectral width of half an octave centered on a given frequency. In half of the trials the “no-gap” condition, i.e., without any silent period within the background noise, was presented. In the other half of the trials—the “gap” condition—the band pass noise was interrupted by a 50 ms interval of silence, presented 100 ms before the startle stimulus. The stimulation was chosen according to the protocol used by Turner et al. (2006), but the stimulus frequency range was adapted to our animal model (cf. below). The response of the animals to the startle pulse was measured with the piezo force sensor described above.

Invalid trials (trials where the animal moved before the startle stimulus) were discarded by thresholding of the signal in the time interval 550 ms before the startle stimulus. The threshold was set to 0.5 mV. As the signal is superposed by high frequency measurement noise a low pass filter (butterworth 6th order, cutoff: 40 Hz was applied). The complete procedure is explained in detail in Supplementary Figure 1.

In a first protocol, we presented 200 trials with and 200 trials without a gap in a background noise centered at 2,000 Hz (± half an octave) to evaluate the distributions of the different responses of the animals to the two different stimulus conditions (in depth analysis of habituation effects is shown in Supplementary Figure 2). In other words all together 400 trials were presented to each animal (two stimulus conditions and 200 repetitions of each stimulus). In a second protocol we analyzed frequency dependent effects of the background noise as used in the standard protocol (cf. Turner et al., 2006): Here, only 15 trials with and without gap for each of 9 different center frequencies were presented (center frequencies: 500, 707, 1,000, 1,414, 2,000, 2,828, 4,000, 5,657, 8,000 Hz, all together 270 stimuli were presented). (For tinnitus testing, this protocol was measured before and after a pure tone acoustic trauma.) In both protocols the inter-stimulus intervals were randomized (10 ± 2 s) to exclude any possible adaptation or habituation of the animals to fixed time intervals (Joober et al., 2002; Ahlf et al., 2012; Krauss et al., 2016a,b).

Typically the startle reflex amplitude (A) is defined as the peak-to-peak amplitude of the reflexive response of the animal. According to Joober et al. (2002) and Jovanovic et al. (2004), the prepulse inhibition is defined as 1 minus the amplitude ratio of A_gap vs. A_nogap:

where A_gap and A_nogap are the peak-to-peak response amplitudes for the gap and no-gap condition, respectively. Hence, the PPI value is always from the interval]−∞,1].

The complete evaluation software including the applied statistical tests is written in Python 2.7 using the Pylab, Numpy and SciPy library, for scientific research (Hunter, 2007; Oliphant, 2007; Millman and Aivazis, 2011; Walt et al., 2011). All calculations were performed on a standard desktop PC. The statistical distributions were fitted using a maximum likelihood estimator provided by the stats library included in SciPy. Bootstrapped data sets were drawn using a self-written Python program based on the Numpy (random) library (Walt et al., 2011).

For any proper selection of statistical tests to be applied to a certain data set, knowledge about the distribution of values within the data set is crucial. Therefore, to obtain a valuable estimation of the distributions of startle reflex amplitudes and the PPI values, data from n = 6 animals were collected, with 200 gap and 200 no-gap condition measurement repetitions each. The stimulation paradigm for these measurements was a narrow band noise centered around 2 kHz with a spectral width of half an octave (cf. section Methods).

For processing the raw data, a fully automated procedure based on a MATLAB program has been applied. This fully automated evaluation of the startle reflex amplitudes provides the advantage that the evaluation is not influenced by any subjective bias. The program applies a low-pass (Butterworth, 6th order) filter with a cutoff frequency of 40 Hz to remove any high frequency background noise. The cutoff frequency was chosen not to distort the startle reflex amplitudes (cf. Supplementary Figure 1). Invalid trials, i.e., trials where the animal moved during the 550 ms time interval before the startle stimulus, were detected using an empirically determined threshold criterion (force > 0.2 mN = 0.5 mV) and discarded from further analysis (Figures 1A,B). The peak-to-peak amplitudes of the valid ASRs were calculated from the 150 ms time interval starting at stimulus onset (Figures 1C,D).

The distributions of the ASR amplitudes for gap (A_gap) stimuli are compared to those of the no-gap condition (A_nogap), as this is critical for any statistical testing of PPI changes. Obviously, the peak-to-peak ASR amplitudes (Figure 1E) were not Gaussian-like distributed, indicating that standard parametric testing procedures, (such as t-testing) cannot be applied to ASR amplitudes, ratios of ASR amplitudes or PPI values. Or in other words, the mean of the peak-to-peak ASR amplitude is highly influenced by outliers and therefore, may not be considered a good statistical measure. This is further demonstrated with 10,000 bootstrapped data sets drawn from the measured no-gap ASR amplitudes (of the shown animal) as shown in Figure 1F, where the distributions of the medians as well as the means are given. The accompanying boxplots provide evidence that the variance of the means is significantly higher than the variance of the medians. Taken together, any statistical analysis of raw ASR data must be based on non-parametrical testing. For example, Mann-Whitney U-statistics could be applied to test if presentation of a gap in noise before the startle stimulus led to a significant PPI (as evident in Figure 1C vs. Figure 1D: p < 0.001) which represents the basis for GPIAS behavioral testing for tinnitus assessment.

As demonstrated in the previous section, ASR amplitudes are broadly distributed and skewed. Therefore, comparing the ASR amplitudes with and without gap cannot simply be achieved by using the mean and standard deviation, but the whole distribution has to be taken into account. From our dataset (200 measurement repetitions at 2 kHz band noise gap and no-gap, respectively), the ratio of all combinations of gap and no-gap ASR amplitudes was calculated (cf. Supplementary Figures 3, 4). The full combinatorial number N_PPI of all possibly ASR amplitude combinations is given by:

Hence, if all measurements are valid, a maximal number of 40,000 PPI values can be calculated. The calculation of all combinatorial PPI values is a valid estimator for the distribution of the compound variable (combination of several variables, Poe et al., 2005).

To compare the histogram of the PPI values to standard stochastic distributions, ASR amplitude ratios (= 1-PPI; cf. Equation 1) were used to shift the value range from]−∞, 1] (Figure 2A2) to [0, ∞[(Figure 2A1). Fitting different distributions to the data using a maximum likelihood estimator revealed that the lognormal distribution provides the highest likelihood and therefore estimates the true data distribution best. Consequently, the calculated logarithm of the ASR amplitude ratios was approximately normally distributed (Figure 2B). The finding that the ratio distribution can be approximated with a lognormal distribution has been reported for standard startle paradigms in humans and mice (Csomor et al., 2008). Additionally, the Akaike information criterion was used to quantify which distribution fits best. This criterion introduces a penalty for the number of used fit parameters preventing overfitting (Akaike, 1974; Saffron et al., 2006). This criterion also leads to the result that the lognormal distribution fits best to the observed data.

Therefore, although parametric statistics may not be applied to raw ASR amplitude ratio data, parametric statistics may be applied to logarithmized ASR amplitude ratio data.

To further examine the underlying statistical distribution, a quantile analysis was performed, allowing for the evaluation of how well a given distribution describes the data (q-q plot and p-p plot, Michael, 1983; Gan and Koehler, 1990; Holmgren, 1995). The cumulated distribution (integral function of probability density) of the fitted distributions and of the data (ratio histogram) were calculated (Figure 2C).

Figures 2D,E show so called p-p and q-q analysis respectively. These plots indicate how well the histogram could be described by the fitted distributions, with a perfect fit resulting in all supporting points lying on the identity line for p-p (Figure 2D) as well as for q-q plots (Figure 2E).

For q-q analysis the quantiles of the data are plotted as a function of the quantiles of the fitted distribution. The q-q analysis emphasizes the edges of the distributions. One scale invariant representation is the so called p-p plot (Holmgren, 1995) comparing the cumulated probabilities (cannot exceed 1). Thus, the upper limit of a p-p plot is always one. This q-q plot shows that the ratio distribution could be nicely described by a lognormal distribution up to the 95% percentile (further animals are shown in Supplementary Figure 5).

As the standard procedure for tinnitus detection in animal models is the analysis of the PPI decrease due to a treatment (in most cases an acoustic trauma, Bauer and Brozoski, 2001; Norena and Eggermont, 2003; Yang et al., 2011; Ahlf et al., 2012; Tziridis et al., 2015; Krauss et al., 2016b) the next section discusses possible measures for PPI change and valid inferential statistical tests.

As demonstrated above, the ASR amplitude ratios (1-PPI) can be described by lognormal distributions up to the 95% quantile, so that parametric statistics may be applied if data are logarithmized. To test if the ASR amplitude ratio distributions changed significantly post trauma relative to pre trauma conditions, the combined standard error of the logarithmized ASR amplitude ratios was calculated. The logarithmized ratio of the ASR amplitudes is given by the difference:

As the logarithmized ASR amplitude ratios are Gaussian-like distributed, and the difference of two Gaussian-like distributed random variables is again Gaussian-like distributed (Eisenberg and Sullivan, 2008; Kersting and Wakolbinger, 2008), one may infer that also the logarithmized ASR amplitude values for both the gap and the no-gap condition could be Gaussian-like distributed (however it is not crucial that they are Gaussian distributed). That this is at least the case for our data set could be confirmed using the Shapiro-Wilk test (Shapiro and Wilk, 1965) (Figure 3, p > 0.1 for both the gap and the no-gap condition). The standard error for the distribution of the logarithmized ASR amplitude ratios can therefore be calculated using error propagation.

(ΔLges : standard error of means of the logarithmized ratios, ΔLgap /ΔLnogap : standard error of means of logarithmized gap and no-gap amplitudes)

Furthermore, the variance of a compound Gaussian-like distributed measure is simply the sum of the variances (Satterthwaite, 1941). The effective number of independent samples of the compound variable can be calculated using the variance and the standard error

where Var(L_ges) is the variance of the compound distribution and ΔL_ges the standard error of the compound distribution. However, this effective n is only an approximation and may also be replaced by a more conservative estimation (cf. Supplementary Figures 7D–F). The information on the standard error of the compound variable (logarithmized ratios) and an effective n makes it possible to test if the pure tone acoustic trauma leads to a significant change of the logarithmized ratios. In other words, the null hypothesis (H₀) can be formulated as follows:

The logarithmized ratios before (Lgespre) and after (Lgespost) the acoustic trauma arise from the same distribution. (Note that this test is done for each stimulus center frequency individually).

Calculating the values Mean(L_ges) and ΔLges2 for pre- and post-trauma conditions allows calculating the T statistics for the comparison between pre- and post-trauma conditions:

The mean of the logarithmized ratios (Mean(Lgespre/post)) is the difference of the means of the logarithmized gap and no-gap amplitudes. However, it can also be regarded as the mean of the full combinatorial difference of logarithmized gap and no-gap amplitudes. It can be shown that these two possibilities are equal (cf. Supplements, Equation 2).

Finally, using the T-statistics, a p-value can be calculated:

Where T refers to the test statistics, Stud to the students T-distribution and df(n) are the degrees of freedom (Monte Carlo simulation used to prove validity of statistics cf. simulation study Supplementary Figures 6, 7).

In summary, the paragraph shows how to calculate the p-value when comparing the logarithmized ASR-ratios before and after an acoustic trauma for one specified stimulus frequency (band noise center frequency) and one animal. The p-value is a measure for the probability that the null hypothesis (pure tone trauma has no effect on ASR amplitude ratios) is falsely rejected. In other words, this p-value indicates if the effect of a change of the average ratio of gap and no-gap amplitudes is by chance (sampling error) or if there exists a real effect. However, this measure only indirectly gives information on the size of the effect. Furthermore, it has to be considered that not significant (p > 0.05) does not mean that the trauma did not lead to any effect (e.g., development of a tinnitus percept) but that the sample size is too small to detect the effect or that there is no effect. The p-value makes no statement about the second order error.

Furthermore, a novel measure for the ΔPPI (until now: ΔPPI = PPI_post - PPI_pre) and hence for the tinnitus percept can be defined using the effect size (Figure 4D). This measure for the PPI change is normalized and not dependent on the dimension of the measured variables. This measure is based on two Gaussian distributions and represents the difference of the means in terms of standard deviations (Figures 4A–C).

As the sample sizes are not equal, the definition by Hedges (Zimmermann et al., 2005; Nakagawa and Cuthill, 2007; Hofmann and Smits, 2008) has to be used.

(s: combined, weighted standard deviation, Lgespre/post: logarithmized ratios for pre and post condition, respectively, n_pre/post sample size same as for Welch-t-test, as variance estimator of Lgespre/post the sum of the variences of logarithmized gap and no-gap amplitudes are used).

Furthermore, the effect size (Hedges g) can be corrected for small sample sizes (Hedges, 1982; Nakagawa and Cuthill, 2007):

In the following the effect size is used as synonym for the sample size-corrected version of Hedges g (g^*).

Figure 4D gives the effect size (corrected Hedges g) of one exemplary animal as a function of the frequency spectrum (center frequency) of the band noise presented. This animal showed a clear effect (ASR amplitude ratio decrease, PPI increase) at a center frequency of 1 kHz and a PPI decrease at a frequency of 5.7 kHz, indicating a potential tinnitus percept there.

Figure 5 summarizes the GPIAS results for all 26 animals to which the standard tinnitus paradigm was applied. It shows the effects size as a function of the median of the classical ΔPPI (calculated from the full combinatorial of all pre and post PPI values, error bars: quartiles, complete procedure of full combinatorial calculation shown in Supplements, cf. Supplementary Figures 3, 4), significant PPI changes are either colored in red (PPI decrease) or green (PPI increase). Trivially, effect size and ΔPPI are highly correlated, but the effect size is normed by the standard deviations of the logarithmized ASR amplitude ratio distributions. Using this new statistical criterion, only three animals show a significant PPI decrease indicating that the number of measured trials for the standard Turner paradigm might be too low to see small effects.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.