This cross-sectional analysis followed the PRISMA guidelines (15, 16). Our study was not subject to Institutional Review Board (IRB) approval, as it did not involve human subjects.

In accordance with the PRISMA harms group, we used terms and definitions for harms displayed in Figure 1 (17).

An SR librarian developed a search string to search the databases MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and Cochrane Database of Systematic Reviews. The strategies combined controlled vocabulary (e.g., MeSH Naltrexone in MEDLINE; Emtree naltrexone in EMBASE) with text words for the generic name, chemical synonyms (e.g., “naltrexone hydrochloride” and “N-cyclopropylmethylnoroxymorphone”), and brand names (e.g., ReVia, Vivitrol, Depade, Nodict, Trexan, and Vivitrex). Where available, we applied systematic review limits/filters (e.g., PubMed “Systematic Review” filter and database-specific SR limits). Afterward, we uploaded the records obtained to Rayyan (https://rayyan.qcri.org/), an SR screening platform. Two investigators (JS and LP) independently screened records in a masked, duplicate fashion for inclusion and removed all duplicates. Following title and abstract screening, investigators were unmasked, and any disagreements were resolved by a third-party investigator (MG).

The search string was uploaded to the Open Science Framework (OSF) (18).

To be included in our sample, we required the following criteria: 1) the publication must be an SR regardless of having a meta-analysis or not, and 2) the SR must be designated to evaluate naltrexone for both FDA-approved uses (AUD and OUD) and off-label uses. Studies had to be in English and only include human subjects. Studies were excluded if they were not related to naltrexone or were not SRs.

Two investigators (JS and LP) were trained on SRs via the Johns Hopkins Systematic Review course (19). Investigators were instructed on how to extract harms items from SRs in other fields of medicine using a pilot-tested Google Form. Training on A MeaSurement Tool to Assess Systematic Reviews-2 (AMSTAR-2) in video and lecture format was also provided. Data from the AMSTAR-2 tool were compiled and interpreted using a pilot-tested Google Form. Senior author MV—who has published a multitude of studies evaluating the methodology of SRs—led all training (20–23).

Two investigators (JS and LP) extracted study characteristics using a pilot-tested Google Form. The characteristics included title, journal, Rayyan ID, and nine variables to evaluate studies (e.g., whether harms were evaluated as an outcome and whether the SR mentioned adherence to PRISMA guidelines) (16). Using methods similar to those of Mahady and colleagues, the same investigators extracted the data items listed in Table 1 from included SRs, coding each item as “yes” or “no” (24). Using methods similar to those of Qureshi and colleagues, they also extracted the items listed in Table 2 , again coding “yes” or “no” unless free response or multiple choice was required (9, 25, 26). All extraction was performed independently in masked duplicates; disagreements were resolved by discussion, with MG adjudicating as needed.

To quantify how much the included SRs relied on the same primary studies, we calculated the corrected covered area (CCA), which standardizes overlap by accounting for both the number of SRs and the number of unique studies (27). We first constructed a citation matrix listing all included SRs (columns) against all primary studies (rows), marking presence/absence. We then computed CCA = (C − U)/[(U × R) − U], where C is the total number of primary study citations across all SRs (sum of matrix entries), U is the number of unique primary studies, and R is the number of SRs. Higher CCA indicates greater redundancy of evidence across reviews. Following published guidance, we interpreted overlap as minimal (<20%), moderate (20%–50%), or high (>50%). For pairs of SRs with ≥50% overlap (high), we performed targeted, side-by-side comparisons (“dyads”) of harms reporting to evaluate consistency (e.g., whether similar adverse events, definitions, and severities were presented despite drawing on largely the same primary evidence).

The authors performed a quality appraisal of each SR using the AMSTAR-2 instrument (28). Each of the 16 items was scored as “yes”, “partial yes”, or “no” depending on whether all criteria were met, some criteria were met, or the criteria were insufficiently met to warrant “yes” or “partial yes”. Items 11, 12, and 15 pertain to SRs with a meta-analysis; reviews without a meta-analysis were therefore scored out of 13 rather than 16. AMSTAR-2 assigns overall confidence ratings based on the presence of critical and non-critical flaws: reviews with no or only one non-critical weakness were rated high, those with more than one non-critical weakness but no critical flaws were rated moderate, those with one critical flaw (with or without non-critical weaknesses) were rated low, and those with more than one critical flaw (with or without non-critical weaknesses) were rated critically low. Using these criteria, each SR in our sample was classified into a quality category using the AMSTAR-2 quality assessment generator.

The characteristics of included studies, harms data, and AMSTAR-2 data for all included SRs were reported in frequency and percentage. A bivariate analysis was performed to determine if any associations existed between quality rating, general characteristics, and harms reporting. The nature of the data (i.e., statistical assumptions and distributional qualities) influenced the choice of statistical test. A p-value less than or equal to 0.05 was considered significant. For the CCA, the following were reported: the number of primary studies across all SRs, the range of primary studies used by an included SR, and the number of primary studies reported in one, two or more, and five or more included SRs (26). Overall, CCA was calculated across all SRs. Lastly, in all pairs of SRs with a high overlap of primary studies, individual harms and reporting items were compared (27). Stata 16.1 (StataCorp, LLC, College Station, TX) was used for data analysis. Data scrubbing was conducted using Microsoft Excel.

To maximize transparency and reproducibility, all study materials were publicly archived on the OSF (https://osf.io/zae45/) (18). The repository includes the full protocol with prespecified objectives, eligibility criteria, outcomes, and analysis plans; complete database search strategies; the deidentified, raw screening and extraction datasets; the pilot-tested extraction forms used by investigators; and the statistical code used for the corrected covered area analysis. Screening and data extraction were conducted independently in masked duplicates, with disagreements resolved by consensus or third-party adjudication; AMSTAR-2 assessments followed the same process. Version history is preserved in the OSF to document any updates to methods or data, and all materials are available to enable verification, replication, and extension of our analyses.

Our search returned 1,013 articles. After duplicates were removed, 903 articles were eligible for title and abstract screening. An additional 752 articles were excluded, leaving 151 articles eligible for full-text review. The reasons for exclusion in each phase of the screening process are presented in Figure 2 .

A total of 87 SRs were included. Of the 87 SRs, 44 (44/87, 50.6%) reported adherence to PRISMA, 56 (56/87, 64.4%) found naltrexone as a favorable intervention, and 37 (37/87, 42.5%) did not report a funding source. Additionally, 18 SRs (18/87, 20.7%) reported harms as a primary outcome, 27 (27/87, 31.0%) reported harms as a secondary outcome, and 42 (42/87, 48.3%) did not report harms as a primary or secondary outcome. The general characteristics of included SRs can be found in Table 3 .

Of the 87 SRs in our analysis, one SR (1/87, 1.1%) classified grades/severity scales for harms in the methods, and four SRs (4/87, 4.6%) listed and separately defined harms in the methods. We found that 11 SRs (11/87, 12.6%) of the included studies discussed limitations to assessing harms. Five SRs (5/87, 5.7%) included harms language in their search strategies, 18 SRs (18/87, 20.7%) followed a protocol that addressed harms, and 36 (36/87, 41.4%) prespecified harms. A total of 17 SRs completed 50% or more of harms items (17/87, 19.5%). A comprehensive list of evaluated harms items can be found in Tables 1 and 2 .

Of our 87 included SRs, our CCA analysis included primary studies from 85 SRs. In total, 2,475 primary studies were cited. The total number of unique primary studies included across all SRs was 1,791. The fewest number of primary studies cited by an SR was 2, and the most was 151. Of our 85 included SRs for CCA analysis, there were 1,463 primary studies cited once. There were 284 primary studies cited in two to four SRs and 44 primary studies cited in five or more SRs. For the eligible 85 SRs, the overall CCA was 0.45%. Four dyads were considered “high” overlap, 35 dyads were considered “moderate” overlap, and the remaining dyads were considered “minimal” overlap. The results of CCA are found in Table 4 .

Of the 87 included SRs, two SRs (2/87, 2.3%) were graded as “high” quality, one SR (1/87, 1.1%) was graded as “moderate” quality, 12 SRs (12/87, 13.8%) were graded as “low” quality, and 72 (72/87, 82.8%) were graded as “critically low” quality ( Table 3 ).

A Kruskal–Wallis test showed a significant relationship between studies graded “critically low” via AMSTAR-2 and completeness of harms reporting (p = 0.0486). Also, a significant relationship was found between studies that specified harms as an outcome and completeness of harms reporting (p = 0.0001). No significant association was determined between completeness of harms reporting and whether the SR reported adherence to PRISMA.

Because our study found deficiencies in harms reporting on naltrexone, we first recommend an overall improvement in harms reporting. This could be attained by adherence to standardized methods of harms reporting, such as PRISMA and CONSORT harms extensions (17, 33). Second, we suggest improvements be made to SRs using grades or severity scales when reporting harms to reduce ambiguity. Petrova et al. and Koh et al. discussed potential methods and tools for reporting the severity of harms of medical interventions (34, 35).

Furthermore, to reduce ambiguity and improve comparability, systematic reviews could prespecify and apply standardized grading frameworks for adverse events [e.g., map events to Common Terminology Criteria for Adverse Events (CTCAE) (5-point grades) and use the Naranjo Algorithm for causality when attribution is unclear], classify suicidal ideation/behavior using C-SSRS, and use systems such as ABACUS for general drug reaction classification (40–42). Practically, protocols could name target scales a priori; define how non-standard labels (e.g., “serious”, “severe”, and “clinically significant”) will be mapped to scale grades, extract, and report both counts and grade distributions (e.g., Grade ≥3); and use consistent denominators and exposure windows for grade-stratified summaries. Applying these frameworks standardizes terminology, clarifies thresholds for seriousness, and supports meta-analysis where appropriate, thereby improving clarity, reproducibility, and clinical interpretability of harms reporting. Notably, 82.7% of SRs in our sample were rated “critically low” using AMSTAR-2, underscoring the need for better methods; until harms reporting improves, clinicians should exercise caution with naltrexone and monitor patients closely.

To operationalize these recommendations, future SRs could register a protocol that prespecifies adverse-event definitions/lists, ascertainment windows, severity grading (with protocol-listed scales and explicit mapping rules for non-standard terms), denominators/time-at-risk, and rules for zero-event data to reduce selective reporting and clarify rate calculations; expand information sources beyond trials to include long-term extensions, observational cohorts/registries, and post-marketing surveillance to better capture long-latency or infrequently collected harms; use dual, standardized extraction that records adverse event (AE) definition, assessment method, grade, timing window, denominator, exposure duration, and whether the AE was prespecified to increase accuracy and comparability; synthesize using both counts and rates and, for rare events, prespecify effect measures and sensitivity analyses or provide a structured narrative when meta-analysis is inappropriate to yield stable, transparent estimates; and report according to PRISMA harms with balanced presentation and public sharing of extraction sheets/code to strengthen transparency and reproducibility. Collectively, these steps improve capture of long-latency or infrequently collected harms, increase completeness and comparability, and enhance interpretability and reproducibility for clinicians.

Addressing study strengths, we executed a study design created specifically for transparency and reproducibility. Documenting our strategies prior to starting the project, we uploaded a detailed protocol to the OSF for reference (18). We routinely uploaded any changes, updates, or modifications. Additionally, we worked with an SR librarian to develop a search strategy including numerous bibliographic databases responsible for routinely cataloging reviews. Screening for harms and AMSTAR-2 in a masked, duplicate fashion allowed the authors to extract accurate data. While there were many strengths within our study, some limitations are noted.

Our analyses are limited to harms collected and reported in the included trials and SRs; because randomized trials often have restricted eligibility and short follow-up, long-term or infrequently captured adverse events may be underrepresented, and complementary sources (e.g., long-term extensions, observational cohorts, registries, and post-marketing surveillance) may be required to detect them. Unclear or unreported items were coded as not reported per prespecified rules (no imputation), which likely biases completeness estimates downward and may amplify between-review differences. Although extraction and AMSTAR-2 ratings were performed in masked duplicates with adjudication, some judgments remain partially subjective. Two SRs lacked full primary study lists and were excluded from the CCA, which could modestly affect overlap estimates and dyad composition. Generalizability is limited due to the cross-sectional nature of our study. Additionally, the quality assessment used AMSTAR-2, a checklist developed in 2012; therefore, studies published prior to this could not follow this set of guidelines.