The science analysis of data in high–energy astrophysics differs from most fields of astronomy in important ways. The data, typically from space-based observatories, consist of energetic photons counted individually as they arrive in a detector. These datasets often can be viewed in tabular form as a sequence of events with four characteristics: arrival time, location in two-dimensions, and energy. The analysis commonly proceeds in stages: sources are identified in the 2-dimensional image, photons are extracted for individual sources or emitting regions, and 1-dimensional analysis proceeds for the energy distribution and arrival times. These univariate distributions are often complicated: multi-component spectral emission processes are convolved with instrumental sensitivity, and temporal processes can depend on unpredictable variations in accretion onto compact objects. Common analysis procedures include:

1) Individual photons are examined, often smoothed with knowledge of the telescope point spread function, in the image plane;

Table 1 summarizes important statistical procedures developed in the high–energy astrophysical community over the past half century. The accomplishments are impressive, but the impact on the research community is mixed. Some methods, such as the Lomb-Scargle periodogram, are widely used, although there may be insufficient appreciation of the challenges of estimating reliable False Alarm Probabilities (VanderPlas, 2018). But other valuable statistical procedures − such as different limits for source existence and flux (Kashyap et al., 2010) and Bayesian estimates of faint-source hardness ratios (Park et al.,2006) − are not commonly used. Many have listened the warning that likelihood ratio tests should not be used near the boundary of parameter values (Protassov et al., 2002), but there is inadequate recognition that likelihood ratios should be penalized by model complexity as with the Bayesian Information Criterion.

There is also a general unawareness within the astronomical community of basic methods that are common in other fields. For example, multiple linear regression for count data (Cameron and Trivedi, 2013) is used extensively in econometrics and other areas, but astronomers often compare a response variable to single covariates in a sequential fashion. Aperiodic stochastic temporal behaviors (that might arise from accretion processes or magnetic activity) are analyzed using Fourier methods designed for periodic time series rather than autoregressive modeling (Box et al., 2015).

These issues motivated the workshop iid2022: Statistical Methods for Event Data-Illuminating the Dynamic Universe workshop, held in Huntsville Alabama on November 15–18, 2022. The spirit of the workshop was to give the participant an opportunity to review and learn about certain statistical methods, and also make presentations based on their own research. Accordingly, the eight sessions had introductory talks by more senior scientists, followed by oral presentations by students and early–career scientists. The National Science Foundation provided support for twenty students and early–career scientists to attend the workshop, via a grant issued to the University of Alabama in Huntsville. Such support was essential to attract students who would not otherwise have had the opportunity to attend.

Table 1 lists presentations made at the workshop. The vast majority of attendees were astronomers, with a few notable exceptions such as Prof. Dale Zimmermann of the University of Iowa, who gave the keynote lecture, and biostatistics graduate student Jesus Vasquez from the University of North Carolina at Chapel Hill.

High–energy astronomy has its roots in the study of cosmic rays on mountaintops during the 1930s and the discovery of X-rays from the solar corona during the 1950s (Rossi 1948; Tousey et al., 1951). The first detection of X-rays outside the Solar System involved a few thousand counts from the Galactic Plane obtained during a brief rocket flight (Giacconi et al., 1962). Early analyses involved simple statistical procedures such as the running mean (Bowyer et al., 1964) or (mathematically incorrect) least squares procedures applied to Poisson distributed data. The first use of the Poisson distribution to derive a cosmic source flux upper limit appears to be by Hearn (1968).

As satellite observatories replaced sounding rockets, more specialized statistical procedures began to emerge and accelerated in the early 21st century. Table 2 lists some of the important milestones classified by the scientific problem addressed. Some methods have had very broad impact with over a thousand citations by later studies. Altogether, the development and promulgation of analysis methods has been substantial and often quite successful.

In addition to procedures developed by practitioners within the field, methods for astronomy have been adopted from the wider arena of statistics. In early years, the textbook Data Reduction and Error Analysis for the Physical Sciences (Bevington, 1969) promoting least squares procedures had the greatest impact, not least because it included convenient Fortran codes that could be typed into IBM cards and used on main frame computers. It was largely supplanted by Numerical Recipes: The Art of Scientific Computing (Press et al., 1992) with editions providing code in Fortran, Pascal, C and C++. Numerical Recipes garnered > 12,000 citations in astronomy and > 120,000 citations in all fields.

Other useful textbooks include Statistical Methods in Experimental Physics (Eadie et al., 1971), Practical Statistics for Astronomers (Wall and Jenkins, 2012), Modern Statistical Methods for Astronomy with R Applications (Feigelson and Babu, 2012), Statistics, Data Mining, and Machine Learning in Astronomy (Ivezić et al., 2019), and Statistics and Analysis of Scientific Data (Bonamente, 2022). Bayesian inference has become an important tool for modeling astronomical data as treated in texts like (Hilbe et al., 2017) and (Bailer-Jones, 2017). However, neither the classic works nor the newer volumes emphasize low-count rate problems as encountered in high–energy astronomy. Some require a basic knowledge of probability and statistics, and this can limit their diffusion among astronomers who are often missing such courses in their undergraduate education.

Table 3 lists a few of the methods discussed in the iid2022 workshop that are directly relevant to high–energy data and science analysis. Software implementation are combined with methodologies to allow quick implementation. In some cases, such as Baddeley’s book for analyzing Poisson images and variability detection procedures discussed by Feigelson, the codes are already available in the general purpose R statistical software environment. In other cases, such as Scargle’s Bayesian Blocks and Xu’s multidimensional change-point analysis, codes are written specifically for use in X-ray and γ-ray astronomy.

Presentations at the iid2022 workshop demonstrate that the development of innovative procedures for analyzing high–energy astronomical data is proceeding in a vibrant fashion. But there are considerable difficulties in promulgation of new methodology in the research communities. We outline here challenges that can be readily identified and suggest directions for improvements for the coming years.