With the success of immune checkpoint blockade (ICB), immunotherapy has revolutionized cancer therapy. More and more immunotherapies have been approved for treating various cancers (1, 2). The immune checkpoint blocking agents for immunotherapies include anticytotoxic T-lymphocyte antigen 4 (CTLA-4, ipilimumab, tremelimumab), antiprogrammed cell death protein 1 (PD-1, nivolumab, pembrolizumab), and antiprogrammed death-ligand 1 (PD-L1, atezolizumab, avelumab, durvalumab) agents (3, 4).

Clinical trials are considered essential to advancing and evaluating the use of ICB in cancer treatment (5). Biomedical publications of various journals are key methods for disseminating the design, conduct, results, and conclusions of these trials. The published reports should provide the reader with the ability to fully understand the trial and make informed judgments of trial results. Thus, it needs to be a unified standard to ensure the quality of the reports.

The Consolidated Standards of Reporting Trials (CONSORT) statement provides guidance to authors regarding essential items that should be included in trial reports and can be also applied to immune-oncology (IO) trials (6, 7). However, distinct mechanisms of IO therapies exhibit unique efficacy and toxicity compared with traditional cancer treatments such as chemotherapy, which may lead to additional considerations for reporting guidelines of IO clinical trials (8, 9). Based on this fact, the Trial Reporting in Immuno-Oncology (TRIO) statement is developed by American Society of Clinical Oncology (ASCO) and the Society for Immunotherapy of Cancer (SITC) to improve the interpretation and comparison across IO trials (10, 11). There have been literatures evaluating the quality of randomized clinical trials (RCT) reports based on the CONSORT statement (12–14), but no studies specifically evaluate the report quality of IO trials. Therefore, the purpose of this study is to evaluate the reporting quality of IO trials based on the TRIO statement. In addition, we also investigated the publications’ characteristics associated with higher quality in IO trial reporting.

Due to the lack of guidelines for reports of IO clinical trials until TRIO statement (10, 11) came out more than 2 years ago, the reporting quality for IO clinical trials was unsatisfactory. Concerns have been raised that more structured and transparent approach was important to the benefit-risk assessment in the evaluation of IO treatment as a new therapy. Therefore, the standardized reporting is essential. This is the first systematic evaluation of the reporting quality of IO clinical trials of cancer treatment in accordance to the TRIO statement.

Immune-oncology trials presented an unsatisfied reporting quality score based on the specific 11-item scoring system derived from the TRIO statement, which consist of six-item efficacy score and five-item toxicity score. The most common reported items in traditional clinical trials are also well described in IO trials, such as response evaluation and toxicity grade. This may be attributing the success to the well-established CONSORT guidance for clinical trials. However, treatment details beyond progression and toxicity onset time and duration were poorly reported, which are crucial to evaluate the efficacy and toxicity of immunotherapy for the cancer.

Pseudo-progression, as a unique phenomenon in IO treatment, is an event that denotes the appearance of new lesions (usually with shrinkage of baseline index tumor burden) or an initial increase in index lesions with subsequent index lesion response by clinical or radiographic assessment (15, 16). Thus, IO clinical trials often allow patients to continue therapy beyond objective progression and half of publications have reported the criteria of continue treatment. However, the details of treatment after progression was seriously underreported (12.8% reported). This failure may be related to the insufficient awareness of authors on the importance of continuous therapy and may limit the ability to make a comprehensive benefit assessment.

Although specific toxicity items for IO therapy, definition of “immune-related adverse events” or “adverse events of special interest” and management of IO toxicity, were relatively well described in over half of publications, the onset and duration of IO toxicity were rare reported. It is worth noting that, unlike the traditional cancer treatment, the toxicity of IO therapy can be latency occurrence and long lasting (17, 18). Therefore, reporting the onset and duration of toxicity is arguably as clinically important to assess the risk-benefit and useful to design the subsequent IO trails.

Different from that the reporting quality of clinical trials of traditional chemotherapy has been fully evaluated since the CONSORT guideline proposed, the quality of IO clinical trials reporting has not been assessed according to the TRIO statement which was specifically designed for IO trials. This divergence may be related to the insufficient awareness of differences between IO and traditional chemotherapy clinical trials and slower uptake of the TRIO statement. Continued use of CONSORT cannot fully reflect the unique characteristics of IO clinical trials (19, 20). More advanced than CONSORT, TRIO adopts toxicity reporting standards at the initially proposed. Notably, only four trails have no description of toxicity. It is also interesting that more than half of trials published in journals of IF >20 (n = 173, 58%).

It is worth noting that two trial reports received the highest score of 11 points (21, 22). Both reports are multicenter randomized controlled phase III clinical studies and are published in the New England Journal of Medicine in 2015. They are both immunotherapy alone studies of nivolumab and are funded by Bristol-Myers Squibb. The tumor types they studied are melanoma (CheckMate 066) and nonsmall-cell lung cancer (CheckMate 057), respectively. Although these two trial reports meet the requirement of each item according to the TRIO statement, some subcomponents are still insufficient. Neither of them mentioned the rationale for the chosen criteria used to evaluate response to therapy and for the selection of clinical diagnoses used in data collection belong to categories such as “immune-related adverse events” or “adverse events of special interest”. Although they both reported the number (proportion) of patients who are treated beyond progression and efficacy after initial progression, they did not mention treatment beyond progression duration and emergence of new toxicity. In addition, the report of CheckMate 066 did not mention time of onset of IO toxicity.

Factors associated with higher reporting scores were also investigated. The publications in journals of IF >20 had higher quality score, either for efficacy assessment or toxicity assessment, which might be related to the original requirements and review system of the journal (13). Specific cancer, such as lung cancer and urinary system cancer had higher quality score compared with the pan cancer. Most of the trials designed for a specific cancer category aimed to confirm the clinical efficacy of the IO treatment for this disease, not just similar to the exploratory purpose in the pan cancer categories. Therefore, there were more detailed reports on the efficacy and the whole trial. Simultaneously, this would possibly reduce authors’ interest for toxicity concerns, which lead to no difference in toxicity quality score between specific cancer and pan cancer. Compared with anti-CTLA-4 agent, trials involving other agents got a better quality score, especially for efficacy score. This is largely due to the fact that other agents came out later than anti-CTLA-4 agent, when IO clinical trials were relatively mature.

Although our study comprehensively assessed the reporting quality of IO clinical trials, the limitations should also be addressed. First, this study does not compare TRIO statement with the traditional CONSORT statement, which is mainly because that the purpose of this study is to evaluate all trials of IO rather than randomized control trails. Second, the quality score in our study was given equal weight to each item on the TRIO, which may weaken some important items or overemphasize some less-important items. At last, for those recommendations with several subcomponents, we only assign values to items, not to subcomponents. This may make the evaluation criteria broad, but it is friendly and practical for most trials.

In summary, our findings show that IO trials had an unsatisfied reporting quality score assessed by TRIO statement, especially in the reporting of treatment details beyond progression and toxicity onset time and duration. High IF journals have better reporting quality. Studies focused on specific cancer and studies containing anti-PD-1 or anti-PD-L1 agents have higher efficacy quality score. As the first step toward providing an overall landscape of IO trials reporting quality, we are expecting that it may shed light into future improvement of IO trial reporting for the better benefit-risk assessment of immunotherapy.

The original contributions presented in the study are included in the article/ Supplementary Material . Further inquiries can be directed to the corresponding authors.

Conceptualization: CC, SH, and LZ. Methodology: CC, YZ, and XZ. Software: YW and L-nH. Formal analysis: CC, YZ, and XZ: Data curation: CC, YZ, XZ, and ZL. Writing (original draft preparation: CC, YZ, and XZ. Writing (review and editing): all authors. All authors contributed to the article and approved the submitted version.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.