This single-blind, prospective derivation study was conducted over a 15-month period in the ED of a major tertiary care hospital in Singapore. This study followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines for derivation studies.

Participants were recruited between 8:00 a.m. and 6:00 p.m., Monday through Friday, from a high-volume ED serving approximately 200,000 patients annually in Singapore. As an ethnically and linguistically diverse city-state located in Southeast Asia, this setting provided a diverse patient population with varied linguistic and ethnic backgrounds. Singapore has a resident population of 4.04 million, made up of ethically Chinese (74.3%), Malay (13.5%), Indian (9%), and other ethnic groups (3.2%) (10). Over 97% of the population is literate in one (25.7%) or more than one (74.3%) languages. The most common first language spoken by residents is English (48.3%) followed by Mandarin Chinese (29.9%) (10).

Eligible participants were English or Mandarin-speaking patients with mild to moderate symptoms on the Patient Acuity Category Scale (11) who were not in danger of acute collapse or in need of resuscitation, aged at least 21 years (the legal age of consent in Singapore), with a chief complaint of palpitations, chest pain, dizziness or difficulty breathing. These symptoms often overlap with both panic-related anxiety and more acute cardiopulmonary conditions, making them critical for the model’s screening utility.

Patients allocated to the most severe triage category were excluded due to the need for acute management. Police cases, patients who were unwilling or unable to complete study procedures, and those who presented with altered mental status, dementia, or psychosis were also excluded.

Upon triage, clinical research coordinators identified potential participants through chief complaints documented in the electronic medical records. Following an initial assessment by the ED physician to ensure clinical stability, coordinators provided study details to patients and obtained written informed consent prior to initiating study procedures. All procedures were approved by the hospital’s Institutional Review Board.

Following consent, participants underwent a structured diagnostic interview to establish the diagnosis of panic disorder and the presence of panic attacks using a modified version of the Panic Disorder Module the Structured Clinical Interview for DSM Disorders (SCID) (12). The SCID is a well-validated semi-structured interview used to diagnose lifetime and current (past month) Axis I psychiatric disorders. For the present study, the SCID was modified to assess all 13 panic attack symptoms. Instead of ending the interview if the participant answered no to the first screening item (“Have you ever had a panic attack, when you suddenly felt frightened, or anxious or suddenly developed a lot of physical symptoms?”), the interviewer proceeded to assess the presence or absence of each of the panic attack symptoms individually. This modification was made to address the fact that ED patients frequently present early in the course of illness with non-fearful panic attacks (13). Hence they may not be aware that their symptoms panic-related when they first present to the ED.

The SCID Panic Disorder module was administered by trained bilingual clinical interviewers fluent in English and Mandarin following a rigorous forward and back translation process, using standardized translated prompts to ensure conceptual consistency across languages. The interviewers administered the SCID and documented each symptom as either present or absent (scored 1 or 0), in addition to coding a diagnosis of panic attack(s), panic disorder, or no panic-related anxiety. All interviewers were blind to the clinical diagnosis provided by the ED physician, which was obtained from the medical record after the SCID diagnostic interview. Inter-rater reliability was high for panic disorder (κ=0.82) and panic attacks (κ=1.00). Weekly consensus meetings were held throughout the study period. Diagnostic uncertainties were adjudicated by the study psychiatrist (LL).

Participants also completed a the Clinical Report Form (CRF) which captured patient-reported demographics, lifetime medical history, frequency of ED visits and hospital admissions in the past year, recent life events in the past year, physician diagnosis for the current ED visit, and discharge disposition.

A priori sample size calculations indicated that a total sample size of 310 participants would be sufficient to estimate the area under the curve at 0.85 with a 95% confidence interval within a reasonable width of 0.125, assuming that approximately 20% of participants met criteria for panic-related anxiety.

The procedures and results to develop the original model are detailed in Sung et al. (8). To derive our updated predictive model, we used the SCID symptom checklist as the primary predictor set, with presence/absence of panic-related anxiety diagnosis (as diagnosed by the SCID) as the outcome variable. We entered all 13 panic attack symptoms into a stepwise multivariable logistic regression with an entry probability of 0.01 and an exit probability of 0.05 to identify symptoms that independently predicted a panic-related anxiety diagnosis.

Model performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, sensitivity, specificity, positive and negative likelihood ratios, and the Youden index. We used Youden Index for the best balance of sensitivity and specificity. We evaluated various cutoff points for panic symptom counts to identify the optimal threshold for the updated model’s clinical application. All analyses were conducted using Stata, and confidence intervals were calculated for all diagnostic performance metrics.

We recruited 324 eligible participants for the study (See Figure 1). The most common reason for exclusion was lack of proficiency in the two study languages. Three participants withdrew before completing all study procedures, resulting in a final sample of 321 participants. Full details of sample demographics are shown in Table 1. The mean age was 52.3 years (SD = 12.4), with a higher proportion of males (68.9%) than females. The sample was ethnically diverse, with 61.4% Chinese, 13.7% Malay, 19.3% Indian, and 5.6% representing other ethnic groups. The majority of participants presented with a chief complaint of chest pain (92.2%), followed by dizziness (5.0%) and palpitations (2.8%).

Using the SCID, 39% (n=125) of participants met diagnostic criteria for panic-related anxiety, either as isolated panic attacks or as part of a panic disorder diagnosis. Among those diagnosed, the most common symptoms reported included shortness of breath, dizziness, and palpitations, which were frequently co-occurring with chest pain. Fewer than 1% of participants received an anxiety-related diagnosis upon ED discharge, and only 0.6% were referred to outpatient psychiatry for follow-up, highlighting the discrepancy between clinical presentation and discharge diagnosis (see Table 1).

As presented in Table 2, the original model did not perform as well in the external validation sample. When applying the same cutoff score of ≥ 3 in the external validation sample, the original model’s performance declined, with a classification accuracy of 77.9%, reduced sensitivity of 60%, and a Youden index slightly below 50%.

In deriving the updated model, the stepwise logistic regression model identified seven symptoms as significant predictors of panic-related anxiety: shortness of breath, dizziness, palpitations, trembling or shaking, fear of dying, choking, and sweating. When tested at an optimal cutoff score of ≥3 symptoms, the updated model achieved an area under the curve (AUC) of 0.88, with a sensitivity of 78.4% and specificity of 85.7%. The Youden index was 64.1%, indicating good discriminatory ability (Table 2). Positive and negative likelihood ratios were 5.49 and 0.25, respectively, further supporting the model’s utility in distinguishing patients with panic-related anxiety from those without. Shortness of breath, dizziness, and palpitations were significant predictors for panic-related anxiety in both the original and updated models, but the remaining items did not overlap (See Table 2).

Consistent with previous studies (6, 8, 9), nearly 40% of participants in the present study met diagnostic critiera for panic-related anxiety and physician detection was low in the ED setting. Due to the physical nature of panic attack symptoms, it is not uncommon for patients to initially seek care in medical settings (4, 15, 16), which may lead to considerable delays in receiving appropriate diagnosis and treatment. For example, data from a worldwide meta-analysis found a peak age of onset of 15.5 years for first panic attack symptoms, in contrast to a peak age at first diagnosis of panic disorder of 39.5 years (17). More than 60% of patients who visited medical settings following their first panic attack reported that these visits did little to address their concerns and may have served to increase their anxiety about future attacks (16).

When patients with panic-related anxiety do not receive timely information regarding the cause of their physical symptoms, they are more likely to return to the ED when symptoms re-occur (18, 19). This pattern has been observed worldwide and remains a considerable public health problem leading to poor long-term patient outcomes, repeat ED admissions, and unnecessary medical costs (4, 20–22). Systematic screening for panic-related anxiety by ED clinicians is likely to help considerably with early detection and right-siting of these patients earlier in the course of illness. In practice, this brief symptom-based model may support ED screening by providing clinicians with a rapid, structured way to identify patients at higher likelihood of panic-related anxiety, facilitating timely reassurance and appropriate referral after medical causes are excluded. Given its brevity and symptom-based format, the SCID could feasibly be administered by triage nurses, emergency physicians, or other trained ED staff as part of routine assessment.

Strengths of the present study include a relatively large sample size, the use of gold-standard diagnostic interview assessments of panic-related anxiety, and the availability of complete data for all participants. Limitations include recruitment from a single ED, over-representation of Indian participants compared to the Singapore general population, and exclusion of a number of potential participants due to language barriers. Furthermore, screening of participants was restricted to weekday daytime hours, which may have affected the representativeness of the sample. Although the SCID has been widely used in multilingual clinical settings, the Mandarin administration used in this study has not undergone independent psychometric validation within this sample, which may have introduced measurement variability. Furthermore, some panic-like symptoms may have been substance-induced (e.g., related to caffeine, stimulants, or medications) rather than attributable to panic disorder, which could have contributed to misclassification and influenced model performance.