The original bryophyte plant materials, including Sphagnum squarrosum (S. squ), Physcomitrium patens (P. Pat), and Ricciocarpos natans (R. nat), were gifted by Prof. Ruiliang Zhu and Prof. Yue Sun from the School of Life Sciences at East China Normal University. The original Wolffia australiana (W. Aus) was gifted by Dr. Li Feng from the High School Affiliated to Renmin University of China.

For bryophyte culture, within a clean workbench, the plant’s surface soil was first rinsed off with clear water, followed by immersion in a 0.05% Triton buffer solution for 5 minutes. Subsequently, the specimen was treated with a 5% NaClO solution for another 5 minutes and then rinsed with sterilized distilled water three times, each rinse lasting 2 minutes. Excess moisture on the surface of the thallus was absorbed using sterile filter paper. The specimen was then placed on a pre-prepared ½ GB5 medium with 1% sucrose, sealed with a sealing film, and then incubated in a room maintained at a constant temperature of 22°C with a light cycle of 16 hours (at 800 lux) to 8 hours.

For cultivation, thallus sections that were vibrant green and in good growth condition were placed on the same medium for propagation. During this process, the growth status of the scales was observed and documented. The culture medium was prepared by dissolving Gamborg B5 Medium powder and sucrose in distilled water, adjusting the pH to 5.7-5.8, adding agar, and then autoclaving. After sterilization, the medium was poured into Petri dishes and allowed to air-dry at room temperature.

For Arabidopsis Columbia (Col-0, WT) culture, the seedlings were transferred to soil 14 days after being cultured on 1/2 Murashige and Skoog (MS) agar medium in a controlled growth chamber (16-h-light/8-h-dark photoperiod, 22°C and 70% relative humidity).

For W. Aus culture, the plants were cultured in the liquid 1/2 Murashige and Skoog (MS) medium in a controlled growth chamber.

In this study, light microscope images of the samples were captured using an Olympus-BX43 biological microscope. Initially, we ensured the microscope slide was clean before adding 3–5 drops of distilled water onto it. Carefully, the liverwort scales were dissected using a dissecting needle and then laid flat in the water droplets on the slide. Once covered with a cover slip, we began our examination under the microscope at a low magnification to initially locate our target. Following this, we switched to a 10-fold eyepiece and a 60-fold objective magnifier for more detailed observation. After confirming regions where the chloroplasts were relatively dispersed without excessive clustering, photographs were taken, thus yielding a high-quality dataset of chloroplast images, setting the foundation for subsequent analysis. By carefully adjusting the longitudinal direction of the sample stage to move the microscope stage, a series of microscopic images (about 5–7 images) focusing on different planes were obtained. The image should include complete cells, ensuring that all chloroplasts in the cell could be clearly visible and counted.

The fluorescence microscope images were captured with a LEICA TCS SP8 (Germany) confocal microscope using a 40× objective or 63×objective oil immersion objective (excitation wavelengths: 632 nm).

For serial block face scanning electron microscopy (SBF-SEM) imaging, referring to a published protocol (Liu et al., 2024), the capitulum of S. Squ was fixed in a solution of 4% (w/v) paraformaldehyde in PBS at 4°C overnight. The fixed samples were stained with 2% (w/v) osmium tetroxide solution, treated with 1% (w/v) thiocarbohydrazide (TCH) solution, and restained with 1% aqueous uranyl acetate/Walton’s lead aspartate solution in turn. Samples were then washed and dehydrated through an ice-cold acetone series (25%, 50%, 75%, 100%, 100%, and 100% acetone) before being embedded in EPIN812 resin at room temperature. Blocks were modified into thin strips, mounted onto a stereomicroscope holder and then imaged and recorded using SEM (3VIEW-SEM, Zeiss). To obtain a three-dimensional image of the sample, a series of images (40 nm per slice) were obtained by removing each slice and imaging the next surface.

In this study, we captured approximately 300 bright-field light microscope images (191 images of R. nat, 51 images of P. pat, and 89 images of S. Squ), 119 fluorescence microscope images (Wolffia Arrhiza) and 512 SBF-SEM images (720×1200 pixels, S. Squ) for manual labeling.

After selecting images that excelled in terms of resolution, contrast, and chloroplast distribution, we employed an annotation tool, LabelImg software (https://github.com/tzutalin/labelImg; Tzutalin, 2015), to manually label the chloroplasts present in these images. Utilizing the bounding box feature of this tool, we meticulously assigned a label to each visible chloroplast. Over 20,000 chloroplasts for light microscopic images, 3600 chloroplasts for fluorescence microscope images and 2500 chloroplasts for the SBF-SEM images were labeled. A single annotator labeled the chloroplasts to eliminate inter-annotator variability to minimize potential impacts on model performance. 90% of the labeled chloroplasts were used as training data and the remaining 10% served as validation data. The annotated data were saved in.txt format, providing precise training and validation datasets for subsequent deep learning training.

The framework of DeepD&Cchl can be seen in Figure 1 . DeepD&Cchl, a set of models based on YOLO ( Supplementary Figure S1 ), were developed to detect and count chloroplasts. To maintain clarity and avoid confusion among DeepD&Cchl models, we established a systematic naming protocol: DeepD&Cchl_L for light microscope images, DeepD&Cchl_E for SBF-SEM images, and DeepD&Cchl_F for fluorescence microscope images. Each name reflects the unique features and applications of the corresponding model. The manually labeled datasets were used to train and tune the corresponding model ( Figure 1A ). For instance, light microscope chloroplast images of three bryophyte (S. squ, P. pat, and R. nat) were obtained ( Supplementary Figure S2 ) for training of DeepD&Cchl-L model, which specifically detect light microscope chloroplasts. The well-manual-labelled light microscopy chloroplast dataset was fed to the YOLOv7 framework loaded with YOLOv7.pt model (https://github.com/WongKinYiu/yolov7; Wang et al., 2023).

To run YOLOv7 locally, we established Python 3.9 environment using “conda create -n yolov7_env python=3.9”, and then activate it with “conda activate yolov7_env”. Within this environment, we installed the necessary libraries and dependencies using “pip install -r requirements.txt” (https://github.com/WongKinYiu/yolov7), including PyTorch, torchvision, numpy, scipy, pandas, matplotlib, seaborn, and OpenCV. The YOLOv7 source code was also cloned from the same GitHub repository, and pretrained weights were downloaded for initialization. Details can be also referred to the attached Manual.

We trained each model for 500 epochs using the Adam optimizer with a learning rate of 0.001 and a batch size of 16 for each type of images. The training process lasted 2.662, 2.123 and 3.216 hours for DeepD&Cchl-L, DeepD&Cchl-F, and DeepD&Cchl-E, respectively. All of our training and testing data were stored in an input directory which contained separate folders for training and testing, along with the corresponding label txt files. Our experiments were conducted on a desktop computer equipped with an Intel Core i7–10700 CPU @ 3.80 GHz and an NVIDIA GeForce RTX 3060 with 12GB of VRAM. The code was executed using the PyTorch 2.0.0 framework and was supported by CUDA version 12.2.

To extend chloroplast detection to 3D cellular structures, we captured a sequential set of 2D images at different focal depths from the same sample. These consecutive 2D image layers were aligned according to their focal depths, sequentially stacked to form a 3D cell model, and processed by DeepD&Cchl to detect the chloroplasts in each layer. An IOU module was implemented to prevent repeated counting. The IOU was calculated as the ratio of the intersection to the union of chloroplast bounding boxes identified in two adjacent focal planes. Its values range from 0 (no overlap) to 1(complete overlap). Considering potential lateral displacement during focal adjustments, a threshold was chosen. The two chloroplasts with an IOU above the threshold were considered a single one. Otherwise, they were treated as distinct ones, incrementing the total count by one. This approach enabled accurate 3D volume quantification of chloroplasts ( Figure 1B ).

Additionally, for accurate counting of chloroplasts in individual cells, we employed the published Cellpose tool (https://www.github.com/mouseland/cellpose; Pachitariu and Stringer, 2022) to segment the images on a single-cell basis. Specifically, the pre-trained Cellpose model (cyto2) was fine tuned with plant cell images to create the suitable plant cell segmentation model Cyto2Pro, enabling a more efficient segmentation of single plant cells. Chloroplast counts were then calculated within each segmented cell ( Figure 1C ).

We utilized manually labeled data as the ground truth to evaluate the performance of DeepD&Cchl. The mislabeled or omitted annotations were checked and counted manually. Several parameters were used to evaluate the performance, including precision (P), recall (R), average precision (AP), mean average precision (mAP) as well as F1 score (Rainio et al, 2024). Their definitions are shown in Equations 1–5.

In this context, True Positive (TP) represents the correctly detected number of target chloroplasts in the image, False Positive (FP) indicates the count of misidentifications, and False Negative (FN) signifies the number of particles in the image missed by the network. P is a metric measuring the ratio of correctly detected particles out of all detected chloroplasts, reflecting the model’s accuracy in object detection. On the other hand, R measures the ratio of correctly detected chloroplasts out of all chloroplasts in the sample, showcasing the model’s capability in detecting all target instances. AP is the mean precision value calculated across different recall levels, determined by the area under the precision-recall curve. AP offers insights into how well the model detects targets across various recall levels. Meanwhile, mAP is the average of AP values across all categories, providing an evaluation of the model’s overall performance. F1 score is the harmonic mean of precision and recall, used to comprehensively evaluate the balance between precision and recall of the model.

To thoroughly assess the performance of DeepD&Cchl-L in chloroplast detection, we employed the comprehensive evaluation metric, mAP, which combines precision and recall ( Figure 2A ; Supplementary Figures S3 , S4 ). In the validation dataset, when the confidence level was set to 0.5, the model achieved an average precision of 0.877. Furthermore, the F1 curve peaked at 0.84, further demonstrating the model’s excellent balance between precision and recall ( Supplementary Figure S5 ).

To evaluate the efficiency of DeepD&Cchl in detecting chloroplast ( Figures 1B , 2B ; Supplementary Figure S6 ), we manually labeled 9 unseen images (3 images from each different plant) as a test dataset. The DeepD&Cchl tool successfully detected 88 chloroplasts in S. squ (missing 3), 160 chloroplasts in P. pat (missing 10), 194 chloroplasts in R.nat (missing 20), and 475 chloroplasts in total images (missing 33), while falsely detected 0 chloroplasts ( Figure 2C ). The final counting results were calculated and the precision rates are 97.10%, 96.77%, and 92.72% respectively ( Figure 2D ). In all, the DeepD&Cchl-L tool showed an expert performance on automatic chloroplast detection and counting with light microscope images.

To accurately calculate the chloroplast in 3D, we used DeepD&Cchl on a multilayer of light microscope images covering the entire cell ( Figures 1B and Figure 3 ). To avoid duplicate counting, IOU calculations were performed between each detected target in the second image and the targets in the benchmark. The predetermined IOU threshold was set at 0.3. If the IOU exceeded the preset threshold, it was classified as an existing chloroplast in the benchmark to prevent duplicate counting; otherwise, it was identified as a new chloroplast and added to the benchmark as mentioned in Section 2.4 ( Figure 3 ). Sequenced images of S. squ, P. pat, and R.nat leaf cells were obtained at various focal planes. The DeepD&Cchl was applied for each individual layer, and IOU was used to monitor the overlaps of every target between layers. The accurate counts of chloroplasts in different plants were obtained ( Figure 3 ; Supplementary Figure S7 ). The precision was significantly improved in multilayer statistics than that in the single-layer image. For example, as shown in Figure 3A , about 254 chloroplasts were detected in the first layer, while the number was 326 in the total multilayer images. In Figure 3B , these numbers were 74 and 184, respectively, and in Figure 3C , they were 214 and 365. Overlaps between focused and out-of-focused chloroplasts might cause errors in detection and counting in a single layer. However, with the help of the IOU module, combined with training on focused chloroplast images, these errors were effectively minimized, yielding relatively accurate 3D quantification. These accurate 3D counts provide insights into chloroplast distribution, critical for investigating photosynthetic capacity.

To expand the application of the DeepD&Cchl tool, we have incorporated various types of chloroplast microscope images, including SBF-SEM and fluorescence microscopy images ( Figure 4 ). The same training strategies were used on various types of images, similar to those used for light microscope images. The evaluated metrics were also applied to DeepD&Cchl_E and DeepD&Cchl_F. In the corresponding validation dataset, at a confidence threshold of 0.5, DeepD&Cchl_E achieved an average precision of 0.927 and an F1 peak of 0.86 ( Supplementary Figures S8 - S11 ), while DeepD&Cchl_F attained a higher average precision of 0.973 and an F1 peak of 0.91 ( Supplementary Figures S12 - S15 ), both demonstrating excellent precision-recall balance.

To assess the performance of the three DeepD&Cchl models, we conducted tests on three distinct sets of untreated images ( Figure 4 ; Supplementary Figure S16 ). The results revealed that 66, 5, and 73 chloroplasts were individually identified from fluorescence, SBF-SEM, and light microscope images, respectively. Notably, for the light microscope images, we purposely utilized a set from multilayer cell leaves of W. Aus. They exhibited inferior clarity compared to single-layered cell images ( Figure 4 ). These results not only affirm the high adaptability and precision of the DeepD&Cchl tool in chloroplast detection and counting across various types and complexities of images, but also underscore the capability of deep learning methods in precise organelle quantification within diverse biological samples, highlighting their potential and universality in biological research.

To achieve single-cell counting of chloroplasts, we employed a Cellpose segmentation tool. We re-trained the original model cyto2 with our dataset and obtained a new model named ‘cyto2pro’ as mentioned in Section 2.4, and then utilized it to segment optical microscope images. We specifically chose intact individual cells as inputs for object detection, subsequently conducting chloroplast detection and counting within those individual cells using DeepD&Cchl at both 2D and 3D levels ( Figure 5 ; Supplementary Figures S17 , S18 ). Plotting cell area against chloroplast count in single cells of R. nat (Type A-B) and Arabidopsis thaliana (A. tha, Type C-D) reveals cell type-specific clustering ( Figure 6 ). This highlights the relationship between chloroplast count and cell size in determining cell type, providing valuable morphological information for single-cell studies.

There are currently four main chloroplasts estimation methods widely used ( Figure 6 ; Table 1 ): manual counting, semi-automated counting, deep-learning-based DeepLearnMOR (Li et al., 2021), and the DeepD&Cchl. Manual counting is simple but time-consuming, labor-intensive, and prone to errors. For instance, ImageJ offers a robust suite of counting tools like Cell Counter (Plugins-Analyze-Cell Counter) for manual labeling and auto-numbering (Arena et al., 2017). Semi-automated counting often needs user intervention and parameter setup for image segmentation and object counting. For example, ImageJ’s Analyze Particles feature allows one-click counting based on image pre-treatment with threshold-based segmentation (Rueden et al., 2017; Schneider et al., 2012). However, it’s worth noting that ImageJ has limitations in counting accuracy and lacks batch processing capabilities. Processing images one at a tim\

can be time-consuming and inconvenient for large-scale data analysis. DeepLearnMOR, based on YOLOv5 (YOLO version 5), counts chloroplasts in fluorescence images. It offers high precision and speed in two-dimensional object detection and fully automated counting. However, the need for fluoresces images limits its practicality.

Our proposed tool, DeepD&Cchl, accurately counts chloroplasts in 3D single cells and is versatile across an unlimited number of plant species. As demonstrated in Figures 6B, C , either 2D or 3D images are subjected to cell segmentation using Cellpose and chloroplast auto-detection and counting using DeepD&Cchl. This method enables automated chloroplast detection and holds significant potential for widespread application in single-cell resolution correlation studies, linking single-cell transcriptomics with individual chloroplast status.

DeepD&Cchl introduces an innovative approach to plant cell research, particularly in understanding photosynthesis efficiency and plant adaptability to changing conditions. Usually, scientists perform chloroplast detection using single-layer images, because it’s challenging and time-consuming to avoid repeated chloroplast counting in multilayer images (Grishagin, 2015). In contrast, DeepD&Cchl offers an effective method to detect and count the chloroplasts in multilayer images. It addresses counting errors in 2D counting due to severe out-of-focus and defocusing issues, enhancing efficiency, particularly for studying organelles within whole cells. DeepD&Cchl enables precise quantification of chloroplasts, providing insights into a plant’s photosynthetic capacity, growth, and developmental stages (Xiong et al., 2017; Fujiwara et al., 2019). This tool can also monitor changes in chloroplast distribution in response to environmental stresses, aiding in the study of plant adaptability (Zahra et al., 2022; Liu et al., 2018; Feng et al., 2019).

The current version of DeepD&Cchl includes three distinct chloroplast detection models trained for light microscope images, fluorescence microscope images, and SBF-SEM images, respectively. While effective within their specific scopes, these models require specialized knowledge for selection and operation, potentially limiting widespread adoption. To address this, further development of a multimodal model that can process diverse microscope images automatically is needed. Future research should focus on integrating various detection and classification techniques to create an efficient, accurate, and user-friendly chloroplast image analysis system. This will boost the model’s adaptability and flexibility, ensuring high-quality results across various observation scenarios.

The DeepD&Cchl method, utilizing deep learning, provides an innovative solution for accurately counting chloroplasts in plant cells. Its effectiveness would be amplified when integrated with object detection frameworks like Faster R-CNN (Faster Region-CNN) or SSD (Single Shot MultiBox Detector), improving precision and segmentation accuracy (Liu et al., 2016; Girshick et al., 2015). When combined with time series analysis tools like LSTM (Long Short-Term Memory), it can allow real-time monitoring of chloroplast dynamics under various environmental conditions (Hochreiter and Schmidhuber, 1997). The addition of multimodal data and technologies like autoencoders and VAEs (Variational Autoencoders) may enhance deep extraction of cellular features, offering a more efficient and comprehensive approach to plant cell research.

Despite these advances, limitations remain. For instance, the models may fail in out-of-focus cases or overlap errors in single-layer images (Section 3.2). Perhaps, some slightly out-of-focus images and overlapped cases in the training dataset could address these issues. However, beyond these limitations, DeepD&Cchl still has potential for broader applications within plant cells. Its robust and accurate image analysis makes it suitable for detecting and quantifying other cellular structures and organelles with further training. This could enrich our understanding of cellular mechanisms and processes, advancing plant cell biology.