The ocular response analyzer (ORA; Reichert, Depew, NY, USA) is a dynamic bidirectional tonometer that uses infrared light reflected from the cornea during deformation by a puff of air to calculate two measures of intraocular pressure (IOP). Although Goldmann-correlated IOP (IOPg) is associated with IOP measured using Goldmann applanation tonometry (1), it can produce an underestimation of less than 1 mmHg (2). Corneal-compensated IOP (IOPcc) uses an empirically derived formula to mitigate the confounding effects of central corneal thickness and corneal stiffness on measured IOP (3). Unlike most other tonometers, the ORA produces additional measurements of the biomechanical parameters of the eye. Corneal hysteresis (CH) is a viscoelastic parameter that represents the ability of the eye to dissipate energy. Low CH is associated with the loss of structural and functional integrity in glaucoma (4, 5), and it predicts conversion from ocular hypertension to glaucoma (6). CH is also reduced in patients with keratoconus, compared to healthy controls, but its diagnostic potential is limited by substantial overlap between the two populations (7).

The ORA produces 38 additional metrics that describe the shape of its twin-peaked applanation waveform (8). These outcomes are not displayed on its clinical interface and must be downloaded from the device. Analysis of waveform parameters has emerging utility for detecting and monitoring ocular disease (9). Aoki and colleagues found that waveform parameters correlate significantly with the progression of visual field deterioration in glaucoma (10). The keratoconus match index (KMI) is a keratoconus-specific set of seven waveform parameters derived from a neural-network analysis of patients included in the internal databases of the device, to represent the agreement between the pressure-applanation waveforms of a given eye and the typical pressure-applanation waveforms of a keratoconic eye. Labris and colleagues used the KMI to differentiate patients with keratoconus, even subclinical disease, from healthy controls with accuracy greater than that with CH alone (11, 12). Thus, biomechanical assessment of the eye using the ORA has the potential to supplement corneal-imaging techniques for the detection of keratoconus.

The initial iteration of the ORA, the G1, first received approval from the United States Food and Drug Administration in 2004. In 2016, the manufacturer introduced a third generation of the device, the G3. The G3 ORA was a substantial departure from the first two generations of the device because its working distance was decreased from 11.3 mm to 8.3 mm and because improvements were made to its air-puff-delivery and applanation-detection systems. These changes attenuated turbulence in the air puff and reduced noise in the infrared signal to optimize repeatability (8). By failing to report which generation of the ORA was used for data collection, most researchers and clinicians assume that the G3 ORA is interchangeable with earlier generations of ORA. It is likely, however, that the aforementioned updates to the G3 ORA altered its pressure-applanation waveforms. Subsequent changes to the outcome metrics of the G3 ORA have not been evaluated against the G1 ORA. Moreover, KMI was not included in the software package of the G3 ORA; thus, its performance has not been tested on the updated device. The purposes of this study are to evaluate the agreement between the outcome metrics of G3 ORA and G1 ORA and to test the ability of KMI to identify patients with keratoconus on the updated device.

This cross-sectional study conformed to the tenets of the Declaration of Helsinki. The Biomedical Institutional Review Board at The Ohio State University (OSU) approved the research (protocol number 2022H0067), and all participants provided written informed consent before enrollment. The study presented here is part of a larger investigation into ocular biomechanics.

The demographics of the keratoconus cohort were reported previously (14). Briefly, 50 participants with keratoconus (mean ± standard deviation age = 39 ± 14 years; 28% female) were enrolled. Of the total participants, 58% self-reported as White, 30% as Black, 6% as Hispanic, and 6% as mixed-race. Of these 100 eyes, measurements could be acquired on 80% of them using the G1 ORA and on 94% of them using the G3 ORA. Thus, a total of 78 eyes (n = 78) were analyzed in the keratoconus cohort. Simulated keratometry values were 47.5 ± 7.85 D in the flat meridian and 51.7 ± 9.81 D in the steep meridian.

A total of 153 participants were enrolled in the healthy cohort (age = 39 ± 17 years; 61% female). Of these, 82% self-reported as White, 5% as Black, 1% as Hispanic, 8% as Asian, and 2% as mixed-race. Race and ethnicity were unknown in 1% of the healthy participants. Both the G1 ORA and the G3 ORA were able to acquire measurements on 99% of the 153 left eyes. Thus, 149 eyes (n = 149) were analyzed in the healthy cohort. Simulated keratometry values were 43.1 ± 1.51 D in the flat meridian and 44.2 ± 1.45 D in the steep meridian.

To the knowledge of the authors, this is the first study to test the hypothesis that the reduced working distance and updated air-puff-delivery and optical-detection systems of the G3 ORA changed its outcome metrics, compared to the G1 ORA. In both cohorts, healthy and keratoconus, there were significant differences between the devices for all the biomechanical outcome metrics, CH, KMI, p1area, p2area, w1, w2, h1, and h2. CH, a viscoelastic measurement of energy dissipation, was higher in the G3 ORA than in the G1 ORA. In general, the peaks of the pressure-applanation waveforms were higher and wider in the G3 ORA than in the G1 ORA, resulting in greater areas under them in the former than in the latter. The exception to this result was w1, which was significantly wider in the G1 ORA than in the G3 ORA. The fact that the waveform score was significantly greater in the G3 ORA than in the G1 ORA in both cohorts suggests that the newer device achieves better alignment with the cornea than the older device. This interpretation is supported by the fact that the G1 ORA could acquire a measurement on only 80% of eyes with keratoconus, but the G3 ORA could acquire a measurement on 94% of them, which approaches the 99% capture rate on healthy eyes.

In aggregate, these results are strong evidence that the updates to the G3 ORA did change its outcome parameters, compared to earlier generations. These changes may have clinical and research significance. For example, although the median (healthy cohort) and mean (keratoconus cohort) differences between the devices for CH were small but statistically significant, the limits of agreement on Bland–Altman analysis were large, and the C_R values were above 3 mmHg in both cohorts. From other works, the average differences in CH between patients with glaucoma and healthy controls and between patients with keratoconus and healthy controls are typically less than 3 mmHg (7, 23, 24). Thus, changing between generations could mask clinically relevant differences in CH.

The interpretations of IOPg and IOPcc are similar to those of CH. There were no significant results between the G1 ORA and G3 ORA for both outcomes when differences in means and medians were assessed using the paired t-test (healthy participants) or the Wilcoxon signed-rank test (keratoconus participants), and there were strong-to-moderate linear associations between the devices. These realities are reflected in a bias near zero on their corresponding Bland–Altman plots. However, inspection of their limits of agreement and the accompanying C_R values suggests substantial variability between the devices, greater than 5 mmHg, even in the healthy cohort. Certainly, 5 mmHg is clinically meaningful when managing patients with open-angle glaucoma, in whom a change of 1 mmHg is associated with a 10% increase or decrease in risk for disease progression (25).

The clinical significance of the difference in waveform parameters between the devices is more difficult to establish than that of CH or IOP, as investigation into the diagnostic potential of the waveform parameters remains in its early stages. Based on the work that has been completed on the topic, it is known that relatively small increases in p2area are associated with worsening visual sensitivity in patients with glaucoma (10). Furthermore, Goebels and colleagues reported a difference in p2area between participants with keratoconus and controls, which falls within the range of variation between the G3 ORA and G1 ORA reported here (26). Additionally, the magnitudes of decreases in p1area, p2area, and h1 after laser refractive surgery reported by Landoulsi and colleagues fall within the limits of agreement of the two ORA generations (27). So too do changes in waveform parameters elicited by orthokeratology contact lenses in children (28). Thus, it is clear that the two devices cannot be used interchangeably. When possible, clinicians should note the generation of the device used and use the same generation device for serial measurements over time. Researchers should report on the generation of ORA used for data collection and should not extrapolate the results of an early-generation ORA to a G3 device.

Coefficient of variation (C_V) was included as an additional metric of device performance. It is not surprising that C_V was higher in the keratoconus cohort than in the healthy cohort, regardless of the generation of the device. Irregularity in the curvature of the ocular surface in eyes with keratoconus likely elicits more scatter of the infrared photons used by the optical-detection system of the ORA than in healthy eyes. Moreover, as mentioned previously, deficits in the alignment of the G1 ORA with the keratoconic eye likely add to measurement variability. The differences in variability between the devices were substantially less than the differences between the devices within each cohort. There were modest trends of decreased variability in the G3 ORA in the healthy cohort and increased variability in the keratoconus cohort. The peaks of the applanation curve are blunted in keratoconus, especially central disease (14). Thus, it is not surprising that an ectatic cornea elicits more variability on the device that produces taller peaks, the G3, than on the one that generates shorter peaks. This study was not designed to determine whether the trend toward increased variability in the G3 waveform parameters hampers its diagnostic accuracy, so future work will have to address this issue. The C_V values for IOPg and IOPcc on the G1 ORA calculated in this study are nearly double those reported by Wang and colleagues (29). The reason for the discrepancy between the two studies lies in the way that C_V was calculated. Wang and colleagues calculated C_V for each parameter using the mean and standard deviation from three measurements on each participant. In this study, population-based C_V was calculated from the measurement with the highest waveform score for each parameter based on the mean and standard deviation of the entire study population. Since variability of a single measurement taken from many participants is larger than the variability of multiple measurements taken on one participant, the C_V values reported in this study cannot be compared directly to those from Wang and colleagues.

Even though their pressure-applanation waveforms differed, the two devices performed similarly when using KMI to differentiate healthy eyes from eyes with keratoconus, as represented by their areas under the ROC curves. The area under the ROC curve value in the G1 ORA lagged behind the values reported by Labiris and colleagues in frank ectasia and subclinical disease, 98 and 94%, respectively (11, 12). Taken together, these results suggest that KMI is due for an update. First, KMI needs to be optimized for the G3 ORA. Its robust pressure-applanation waveforms may yield more and different markers of ectasia than the waveforms of the G1 ORA, on which KMI was first developed. That is, the current iteration of KMI may be suppressing the full potential of the G3 ORA. Second, image analysis of the pressure-applanation waveforms of the ORA is likely well suited for machine learning applications to identify differences between healthy eyes and eyes with keratoconus. Such work is already being done on eyes with myopia (30), and multiple groups have applied machine learning to analyze images of keratoconus corneas generated by Cornea Visualization with Scheimpflug Technology (Corvis ST; OCULUS Optikgeräte GmbH, Wetzlar, Germany) (31, 32), an air-puff tonometer that uses a high-speed camera with Scheimpflug geometry to image the cornea during deformation. A next step in this line of research is to apply similar machine learning techniques to optimize the potential of the ORA, which may be more ubiquitous in primary eye clinics, where keratoconus is often first detected, than the Corvis ST, to detect early keratoconus in conjunction with traditional cornea imaging techniques.

This study has several limitations. First, the stage of keratoconus was not graded; thus, the agreement between the devices could not be assessed as a function of disease severity. Second, this study was cross-sectional in eyes with previously diagnosed keratoconus. The true value of metrics like KMI is to predict disease incidence in healthy eyes and disease progression in eyes with mild disease. That work must be done in a future longitudinal study. Third, it is possible that participants in the healthy cohort had subclinical keratoconus, which could confound the differences found between devices. Fourth, a second-generation (G2) ORA was not included in the study design. It would be expected that the G2 ORA would perform in a similar manner to the G1 ORA, given that they share working distances and air-puff-delivery and optical-detection systems. However, this hypothesis cannot be confirmed in the current study. Finally, we did not exclude participants who wore contact lenses, nor were contact lens wearers without their lenses for a set duration before the start of the study. Exclusion of contact lens wearers was not feasible for this study because rigid gas permeable lenses are the leading refractive error correction for patients with keratoconus (33, 34). Moreover, these lenses likely have little effect on the biomechanical parameters of the eye (35). Soft contact lenses may elicit corneal swelling (36), but they have not been shown to induce consistent changes to the biomechanical parameters of the ORA. Conflicting studies suggest that the use of soft contact lenses has no effect on CH (37), increases CH (38), or decreases CH (39). Such volatile results suggest that soft contact lenses did not have a meaningful effect on our results.

The changes made to the working distance and the air-puff-delivery and optical-detection systems of the G3 ORA altered its pressure-applanation waveforms and thus its biomechanical outcome metrics, compared to the G1 ORA. Additionally, there was meaningful variation between the IOP outcomes generated by the two devices. Thus, the G1 ORA and G3 ORA are not interchangeable devices. Clinicians should note the generation of the device used and then use the same generation for serial measurements over time. Researchers should report the generation of the device used to collect data in scientific communications. Caution must be taken when comparing results from a G1 ORA to those from a G3 ORA. Finally, both generations of the ORA can use KMI to differentiate eyes with keratoconus from healthy eyes, but future studies should update the KMI to maximize the diagnostic potential on the G3 device. Machine learning applications may be particularly well suited to aid in this task.