The gold standard for tumor identification is histopathological tissue sections, which also provide a wealth of information that can be utilized to determine disease progression, select individualized treatment plans, and predict patient survivorship. However, due to the enormous labor required of experts to extract information from complex images, traditional histopathology procedures are unable to satisfy the demands of precision medicine (9). Currently, AI-based digital pathology has been successfully used in tumor diagnosis and treatment. This technology has a wide range of future applications, including improving the accuracy of pathological diagnosis, formulating treatment plans, predicting patient prognosis, and reducing manual workload, and more (10). For instance, AI can segment and identify tumor cells in pathology slides and accurately quantify immunohistochemical staining results (11). Thus, machine learning techniques based on histopathological analysis provide novel strategies to predict response to tumor immunotherapy (12). Among these, immunohistochemical (IHC) analysis, tumor-infiltrating lymphocyte (TIL), tumor-stroma ratio (TSR), and microsatellite instability are extensively researched.

Tumor cells expressing PD-L1 can suppress the immune response by binding to PD-1 on T cells, and ICIs is an anti-tumor strategy to block this interaction. Studies have shown that PD-L1 expression levels correlate with immunotherapy response and clinical results. Previous research has demonstrated that an AI-driven PD-L1 tumor ratio score-based analyzer can identify non-small cell lung cancer (NSCLC) more effectively than a pathologist can when predicting the immunotherapy response, and its results are objective and repeatable without human error (13). In addition, the absence of DNA defective mis-match repair (MMR) mechanism caused by mutations in the MMR gene results in the accumulation of somatic genomic mutations, which are closely associated with the ICB response (14). For MMR-deficient and MMR-proficient colorectal tumors, Le et al. (15) discovered immune-related objective response rates of 40% and 0%, respectively, suggesting that MMR status can be used to predict clinical response in patients treated with immune checkpoint inhibitors. In further, many studies have confirmed that higher levels of T-cell infiltration and T-cell counts are related to improved immune checkpoint blockage (16).

Microsatellite refers to several short, repetitive DNA sequences in the genome. Microsatellite instability is associated with DNA mismatch repair and involved in the development of many malignancies. Tumors with high microsatellite instability were found to respond well to immunotherapy (17). Kather et al. (18) demonstrated that deep learning has the potential to rapidly and accurately screen patients suitable for immunotherapy by precisely identifying the microsatellite stability status of patients with gastrointestinal cancer. Furthermore, AI can recognize lymphocytes, tumor cells, and mesenchymal stroma in the section and use three-dimensional reconstruction to highlight the spatial distribution of different cell types, which is another potential factor to evaluate the efficacy of immunotherapy (19).

TIL is closely associated with immunotherapy, and deep convolutional neural network models can be used to determine its distribution from eosin-hematoxylin (HE) stained images. By assessing the immune cell types in the tumor tissue matrix and classifying patients into types A and B, Zheng et al. (20) constructed a prognostic-relevant immune phenotype classifier. Higher levels of ICB were observed in phenotype A, which indicated a superior immunotherapy effect and prognosis. Additionally, there is an end-to-end strategy to train DL directly on response or outcome data, which used convolutional neural networks (CNNs) or graph neural networks (GNNs) to predict immunotherapy responses. According to these trials, the AUCs for predicting responders in lung cancer and melanoma were 0.778 and 0.69, respectively (21).

Tumor mutational burden (TMB) is defined as the number of non-synonymous single nucleotide variants (NsSNV) on tumor cells that can be transcribed into new antigenic peptides and presented on the cell surface, thereby activating T cells (22). High TMB is shown to be a predictive biomarker for lung cancer (23). Recent research suggests that deep learning systems can be used to predict immune responses. The study used HE-stained images to predict the status of TMB and reached an AUC of 0.78-0.98 in the external validation group, much higher than utilizing clinical data alone (24).

AI has a promising development in predicting tumor immunotherapy response and can examine the state of tumor development through pathological data. Its algorithm and analysis results can be highly standardized and shared, which may assist advance medicine and actualizing precision medicine ( Figure 2 ).

With the constant advancement of medical imaging equipment and technology, medical images are no longer simply pictures or restricted to the traditional “computer assisted diagnostics (CAD)”, but also contain high-throughput mineable data that are not recognizable to the naked eye (25). Imaging histology refers to AI-based imaging characterization that can provide more detailed information than single graphics can, displaying macroscopic, molecular, and cellular features (26, 27). Additionally, AI-based predictive models can provide reliable non-invasive biomarkers for evaluating immunotherapeutic response. Among various biomarkers, PD-L1 expression has been well validated in immunotherapy, and a combined model based on CT radiomics and clinical features can assess PD-L1 expression levels non-invasively (28). Sun et al. (29) selected 135 patients with tumors at different sites derived from phase I PD-1/PD-L1 monotherapy clinical trials. They established a predictive imaging model based on CT imaging by combining enhanced CT images with RNA-seq genomic data from tumor biopsy tissues. This model can analyze tumor-infiltrating CD8 signals and distinguish between immune-infiltrating and immunodetect types, making it an efficient method for predicting clinical outcomes of patients with advanced solid tumors after immunotherapy. Mu et al. (30) analyzed baseline PET/CT data of 194 patients with stage IIIB-IV NSCLC treated with PD-1/PD-L1 inhibitors, and they developed multiparametric imaging histological signature models which successfully predict whether patients would receive sustained clinical benefit from immunotherapy. However, further research is still required to determine whether models utilizing several data sources can perform better than models that simply use radiomics.

TMB is also well known for being one of the important indicators of ICI efficacy. He et al. (31) used deep learning techniques to analyze CT images from patients with advanced NSCLC and establish TMB radiomic biomarkers, which have high predictive value for ICI treatment response, overall survival (OS), and progression-free survival (PFS). Furthermore, Trebeschi et al. (32) used AI techniques to study the pre-treatment enhanced CT image analysis of patients with progressive malignant melanoma and NSCLC treated with PD-1, and they discovered that lesions with more heterogeneous morphology, which means compact borders and inhomogeneous density, were more likely to respond to immunotherapy. Additionally, there is a substantial association between imaging histological indices and mitosis-related pathways, suggesting that higher proliferative potential may signal better immunotherapy efficacy.

However, despite the fact that hyperprogression—rapid tumor progression following immunotherapy—is associated with a poor prognosis, there are no validated biomarkers to identify patients at risk for it (33, 34). Vaidya et al. (35) retrospectively summarized clinical and imaging data from 109 patients with advanced NSCLC treated with PD-1/PD-L1 immunosuppressant monotherapy, 19 of whom showed hyper progression. And researchers extracted textural features from the patients’ baseline CT images reflecting the texture within and around the target lesion as well as histological features quantifying the degree of peri-lesion vascular tortuosity, which can somewhat predict whether patients will develop hyper progression.

In general, the majority of studies’ Radiomics Quality Scores (RQS) ranged from 11 to 20 out of a possible maximum score of 36 points (36). It indicates that AI-based imaging-omics analysis can delve into the spatiotemporal heterogeneity of tumors and plays an important role in predicting immunotherapy response, biomarker expression, and patient prognosis, particularly in the absence of histopathological specimens. Deeper studies of imaging histology can aid in the diagnosis of immunotherapy-eligible patients, disease risk assessment, and precision medicine.

Thanks to advances in sequencing technology, a large amount of cancer genomic data has now been accumulated, providing more precise recommendations for directing tumor immunotherapy. The development of next-generation sequencing (NGS) technologies has enabled comprehensive genomic and transcriptomic screening. This allows to the generation of datasets for analyzing tumor drivers, as well as sequencing cancer cells, stromal cells, and immune cells within the tumor microenvironment to reveal the characteristics of therapeutic effects (37). The human genome contains more than 3 billion base pairs, making it a vast space of high-dimensional data with very complicated information. Whole genome sequencing (WGS) offers the most comprehensive information about the genome, but sorting out the genes, phenotypes, and their inter-regulatory relationships requires the assistance of AI, particularly deep learning techniques. Xie et al. (38) developed a predictive model that integrated genomic data from multiple perspectives, such as TMB, microsatellite instability, and somatic cell copy number variation in many different tumor types to effectively differentiate between “cold” and “hot” immune patients. The model was further externally validated using data from clinical trials, showing that patients in the hyperimmune group were more responsive to immunotherapy and had a better prognosis. By sequencing WES and RNA sequences from 110 patients with metastatic melanoma, Van Allen et al. (39) found that individual response rates to anti-CTLA4 correlated with TMB and cytolytic markers. It has also been shown that the expression and variant levels of platelet-related genes are closely related to the prognosis and immunotherapy response in patients with triple-negative breast cancer (40). These studies provided a direction to explore the variable response to immune checkpoint inhibitors and the identification of prognostic biomarkers.

Moreover, some factors in the transcriptome are still required to be added to the genomic analysis in order to fully predict immune response and explain drug resistance (41). By combining exome and transcriptome sequencing with mass spectrometry, Yadav et al. (42) discovered immunogenic mutant peptides with MHC specificity that could be useful for individualized vaccine development. Furthermore, Mo et al. (43) developed a high-throughput screening platform using 384-well plates to observe tumor immune interactions by co-culturing peripheral blood mononuclear cells (PBMC) and cancer cells in each well to assess cellular value-added and viability and to detect cell growth phenotypes. They also tested the effects of multiple bioactive compounds and identified three potential antagonists for enhancing immune activity. Some scholars have also proved it by epigenetic profiling that the DNA methylation landscape of patient CART19 cells influences the efficacy of the cellular immunotherapy treatment in patients with B-cell malignancy (44).

Numerous studies have utilized AI for a variety of tumor immunotherapy applications ( Table 1 ). The current liquid biopsy technology offers a more accessible and flexible approach to tumor diagnosis and treatment, which is represented by the detection of circulating tumor cell DNA. In immunotherapy, liquid genetic biomarkers are increasingly being developed to predict the therapeutic efficacy of ICIs (59), and molecular biological information in liquid specimens can be automatically identified and detected by AI technology (45). Additionally, some biomarkers are used to exclude hyperprogressive or pseudoprogressive disease after immunotherapy, including plasma cytokines interleukins and circulating tumor cell DNA (60).

In addition to genomics and imaging, proteomics technologies have been extensively explored to identify biomarkers for the effectiveness of tumor immunotherapy (46). An AI-based serum proteomics test model has been developed to predict response to ICIs in patients with metastatic melanoma (48). The application of multi-omics-based AI models to predict tumor immunotherapy responses is also promising. Multi-omics integrates genomics, transcriptomics, epigenomics, proteomics, and radiomics, allowing for a more thoroughly characterization of the disease with data from different sources (49). Based on the multi-omics concept, Yi Yang et al. (50) constructed a deep learning-based 90-day prediction model using age, gender, medical history, baseline data, routine laboratory tests, and follow-up CT scans of NSCLC patients to better distinguish immunotherapy responders from non-responders. Similarly, Arsela Prelaj et al. (61) used machine learning techniques to predict OS and PFS in responders and non-responders in a real-world study. Additionally, Peng Song et al. (51) suggested combining DNA and RNA sequencing, immunohistochemical staining results, demographic baseline data, medical history, and laboratory test results to analyze the information related to efficacy and break the limitation of relying only on PD-1 expression level and TMB level to predict immunotherapy response, and develop a mature model for immunotherapy response prediction for Chinese lung cancer patients. Along with PD-L1 expression, TMB, and TIL, it has been demonstrated that miRNA abnormalities (62), EGFR mutations (54), TP53 mutations (63), ALK rearrangements (55), gut microbiota (56), Fc gamma receptor (FcγR) polymorphisms (57) and serum complement levels (58) all influence immunotherapy response to some extent. It would facilitate the robustness of prediction if this information could be merged with pathological sections and imaging histology to form multi-omics integrated data.

Moreover, tumor-like organs serve as a useful screening model for immunotherapy because they can accurately reproduce the tumor microenvironment while still generating in vitro immune-tumor interactions (47). Incorporating AI into organoids is anticipated to overcome the safety and individualization challenges of conventional prediction by establishing a productive platform for tissue collection, tumor in vitro culture, growth analysis, and medication screening (52). AI could also forecast ICB responses by focusing on antigen presentation pathways (64) and cell necroptosis index (CNI) (53).