Table 1 provides typical examples from the fourteen scanned and 3D-segmented root systems of cassava plants. The bottom row lists the count of detected root voxels within each volume. The 3D-segmentations of the depicted root structures have been generated by the RootForce algorithm (Gerth et al., 2021) with a vesselness filter in the range of 0.2–0.5 mm resulting in a volume of small roots, and furthermore used a 3D-Gauss filter (σ = 0.5mm) to support the extraction of large roots (see Section 2). Both obtained volumes are binarized and merged yielding the final binary root segmentation.

Figure 1A shows one vertical 2D image slice S_i extracted from volume V ₅. This side view depicts the large (bright) stem of the cassava plant (denoted by the yellow box in the top center) surrounded by the (dark) soil mixture, the planting pot below (red region) and the air above (dark blue region). The soil mixture is rather inhomogeneous with many components of differing gray values, including small pebbles. To enhance visibility of the very thin root structures depicted in the soil, the contrast was increased. One challenge in this exemplary data is highlighted in the green boxes in Figure 1A and enlarged Figure 1B , where two branches of a very thin root (orange and magenta) are denoted. Figure 1C depicts the two branches of the same root in the adjacent image slice S_{i+ 2}, (shifted two voxels perpendicular to the slicing plane) passing a small stone (cyan) with very a similar roundish structure and gray level values.

For the training, validating, and testing of the proposed SPP-network (see Section 4) the fourteen CT scans were divided. Four volumes (V ₁ – V ₄) were used as training data and one volume (V ₅) for the validation of the deep neural network. The remaining nine (V ₆ – V ₁₄) scans were used for testing. The training volumes were chosen carefully with the distinct aim of providing a diverse set of root examples despite the otherwise identical physical parameters.

When training a DCNN to delineate the complex root structures into the fore- and background labels ‘root’ and ‘non-root’, a sufficient amount of adequately labeled ground truth data is required. However, there is only limited availability of training data with a satisfactory image and label quality, thus hindering the training process. Since manual annotation of such CT data is time consuming and error prone, segmentation results of the training data D = {V ₁, V ₂, V ₃, V ₄ } (see Section 3.2) obtained from the RootForce algorithm were used as label volumes L ₀ (see Table 1 ) in an initial training data set T ₀ = (D, L ₀). The deep neural network (see Section 4) was trained by a weakly supervised learning approach, as proposed by Khoreva et al. (2017), to recursively refine the data set. During the weakly supervised learning iterations, the segmentation results of the network N ₀ from the first training iteration are used as label volumes for subsequent training iterations, see Section 5 for details.

Due to memory restrictions, an entire volumetric 3D scan of the roots (with a total of 1024 × 1024 × 900 voxels) cannot be processed at once. Therefore, we consider small sub-volumes or small 3D-patches from the complete volumes as input samples. Even though only the center voxel of each patch is classified into either label ‘root’ or ‘non-root’, the surrounding volume patch provides important contextual information. Consequently, the reference label of one sample is not a classification of the whole volume patch, but a binary label for the center voxel, encoding either ‘root’ or ‘non-root’.

In order to detect thin and thick roots (see Table 1 ; Figure 1 ) with equal accuracy, similar structural root features need to be represented across scales. To restrict the introduction of additional noise and expensive preprocessing during inference, scaling techniques utilizing filtering were avoided. Instead, as all samples originate from image data with the same voxel resolution, we vary the edge length of the patches for each sample; hence, the training sample size of an input patch varies only in the voxel count and thus the dimensions of the cuboid, but not in the physical voxel size. In the remains of this work, we will refer to the varying sample size as ‘field of view’ (FoV) which is created by extracting the neighboring region around each voxel. This region can extend between r = 2 to r = 7 voxels in each direction, hence resulting in cubic samples with edge lengths in the range from l = 5 to l = 15 voxels.

Since every volumetric data sample corresponds to a distinct voxel, which is either of class ‘root’ or class ‘non-root’, a naively constructed training data set would inherit a skewed class distribution, as there are much less root voxels than non-root voxels present in each CT scan. Randomly drawing samples from the pool of possible non-root voxels to match the count of root voxels, could yield many trivial negative samples, such as volume patches depicting only soil or air around the plant pot, see Figure 1A . To provide an adequate training data set with an ideally uniform class distribution, the applied data preprocessing includes a categorization of the training and validation samples into sub-labels, which distinguish the center voxel of the volume patch from the surrounding voxels.

Because during the later inference, the proposed network (see Section 4) shall be provided a volume patch, and its output should predict one of the classes ‘root’ or ‘non-root’ for only its center voxel, it is important to train the network with a sufficient set of ‘difficult negative’ examples. Specifically, in such ‘difficult negative’ samples, some root voxels should be present in the volume patch, while the center voxel is not of the class ‘root’. In this way, the network will be prevented from classifying the volume patch as a whole and instead is taught to differentiate between the contents in the center voxel and its surrounding.

Furthermore, another challenge with respect to the application of deep convolutional neural networks to plant root segmentation is the misclassification of voxels depicting the plant pot as class ‘root’. As the non-trivial geometry of the pot base (see Figure 1A ) could not be easily extracted or excluded by a mask, the sub-labels also differentiate within the ‘non-root’ class, namely between ‘sediment’ and ‘else’ contents, where ‘else’ is used as collective term for all voxels depicting the plant pot or air. This approach allows further specification for the included samples of the negative ‘non-root’ class in the training and validation data. For example, ‘non-root’ samples, which contain roots in the surrounding but not in the center voxel and are located next to the edges of the plant pot.

As depicted in Figure 2 , for the categorization of the training samples into adequate sub-labels, two label volumes V _Roots and V _Sediment are employed ( Figure 2A ): The first volume, V _Roots, is a binary volume and depicts an approximate labeling V _Roots = L_i of the ‘root’/’non-root’ classification, which in the first approximation is obtained from a RootForce segmentation (see Section 3.3). The second label volume V _Sediment approximates a sediment segmentation, which is generated from the inverse V _Roots label volume. Additional image processing steps are applied in order to remove air pockets in the soil and the plant pot from the sediment labeling. However, a conservative box-mask, which also excludes several sediment voxels, was needed for the lower volume due to the complex geometry of the pot bottom. Hence, it is assured that no voxels from ‘pot’ or ‘air’ are contained in V_Sediment, and thereby the sediment label volume helps differentiating between the content classes ‘sediment’ and ‘else’. The original gray level CT-volume, depicted in Figure 2A , only serves for a better description of the process, as its information is not needed to generate the sub-labels.

The following steps are repeated for all possible radius sizes r ∈ {2, 3, 4, 5, 6, 7} of the FoV: For every ‘interior voxel’ of both the approximated ground truth volume V _Roots as well as the approximated sediment label volume V _Sediment, sample patches, which are centered on the current voxel and sized according to the current radius r, are extracted and then used to categorize each sample. The denotation of ‘interior voxels’ refers to those voxels, which are at least d = ⌊r/2⌋ voxels away from the volume’s edges, such that the FoV can be placed around the voxel without needing to pad the volume.

Figure 2B depicts the interior voxels of the volume in white, while for the FoV the center voxel is marked in red, and yellow indicates the voxels in the surrounding regions. Exemplary resulting patches are shown in Figure 2C .

As motivated earlier, the categorization of all volume patches into sub-labels shall differentiate the occurring content combinations of the center voxel and the surrounding voxels, where the content classes are ‘root’, ‘sediment’, and ‘else’ – surrogating ‘pot’ and ‘air’. Figure 2D shows the considered sub-labels in a table-like structure, where the contents of the center voxel are organized along the rows and the contents of the surrounding voxels are organized along the columns. This is also reflected by the respective color-coding of Figure 2B . The secondary table headers are color-coded to match the three content types (‘root’, ‘sediment’ and ‘else’). In total nine different combinations of sub-labels are considered. The first row contains the sub-labels, which correspond to the original ‘root’ class, while all other sub-labels are of the ‘non root’ class. Each table entry gives an exemplary depiction of the three volume patches (edge colors indicate the affiliations to the source volumes). With the two binary volume patches, V _Roots and V _Sediment, and some binary logical operators, every sample can now be categorized into a unique sub-label. The output of this process is a table for each FoV radius size r ∈ {2, 3, 4, 5, 6, 7}, containing the total amount of available samples per sub-label and storing the respective sample coordinates.

By specifying the sample counts per sub-label, a class-balanced training data set can now be constructed. To this end, we define a distribution, dependent on the difficulty and availability of each sub-label.

This above-described procedure of sub-label generation is repeated for all five CT scans used as training or validation volume. Where possible, similar distributions for every volume and FoV combination were used. However, this goal is not always attainable since the availability of the sub-label depends on the spatial extend of the addressed root system and the FoV radius r. Specifically for the later, the proportion of samples containing sub-labels, which encode for a mix of content types, scales proportionally with the FoV size.

The iterative process starts at the top of the flow graph in Figure 5 by (manually) choosing a set of initial seed voxels, of which the user is certain that at least one of them belongs to the class ‘root’. These seed voxels are then pushed into a queue, together with the information that at processing time the largest available FoV size should be used for them. Even though the diagram in Figure 5 depicts the workflow for a single voxel at a time, multiple voxels can be pulled from the queue at once, to make use of the more efficient batch evaluation. When a voxel is pulled off the queue, its FoV (with radius r) from the neighboring voxels is extracted from the volume and prepared as a sample for the network’s input layer. If the network classifies the actual center voxel as ‘root’, the downwards arrow from diamond (a) (in Figure 5 ) is followed.

In the example depicted in Figure 6A , a single seed voxel was chosen. Its queued state is represented by a blue colored disk in the respective voxel. The largest used FoV size in the example has a radius of r = 3 voxels and an edge length of l = 7 voxels, the related voxels are denoted in blue. Figure 6E summarizes the color codings of all depictions. Next, in Figure 6B , the network has classified the seed voxel as ‘root’ with the given FoV (drawn as a red box with a red circle in the center voxel) and the voxel is thus represented (labeled) with a filled-in colored blue voxel. The process will then queue all yet unseen neighboring voxels (of the current voxel) for further evaluation as it is shown in Figure 6C . This step is denoted as ‘growing phase’, since it fills the queue and allows the detected root structure to grow. While the queue is not empty, and the newly queued voxels are classified to belong to the ‘root’ class, these steps are repeated. This is depicted in Figure 5 by looping back to the top in diamond (c), hence iteratively evaluating the collected and yet unseen neighbor voxels in the queue.

Figure 6D illustrates a slightly progressed state of the root volume with a growing amount of already classified as well as queued voxels. In contrast to a naïve brute-force approach, where each voxel in the complete CT-volume is evaluated and classified from top to bottom, this process operates similar to a classical ‘breadth-first’ flood-filling algorithm, as it starts at a seed voxel and locally expands in every direction equally, until a stopping criterion is met. Furthermore, to avoid evaluating every voxel in the volume, the goal is to stop this growing or ‘flooding’ process at the boundary of the already detected root structures. However, as these boundaries are not known beforehand, they must be detected as precisely as possible during the expansion process.

To robustly detect the boundaries between the root structures and the surrounding soil, advantages are made of the possibility to process FoVs of different spatial sizes. It can be assumed that large FoVs will lead to predictions with a high degree of certainty, where the network is confident in its classification, as it has much contextual information on which to base its estimate. Furthermore, it is assumed that small FoVs will lead to predictions which are locally more accurate, since smaller FoVs might detect the exact voxel position of a root-boundary with higher spatial accuracy than a large FoV, as there exist fewer off-center voxels competing for attention of the network. In contrast, a large FoV can contain many root voxels, none of which are in the center position. This effect could erroneously skew the prediction towards the ‘root’ class, especially when those root voxels are located directly next to a center voxel.

Under these considerations, the best achievable boundary detection of the root structures will likely be computed with small FoVs. For this reason, the proposed evaluation process starts with the largest FoV around the seed voxels, since it allows going from the most confident classification (large FoVs) to the spatially most accurate one (small FoVs). Hence, if a voxel is pulled from the queue and is classified as ‘non-root’, the size of the FoV is reduced to a smaller radius and the same voxel is pushed in the queue again for further evaluation. This extends the formerly mentioned loop in Figure 5 by an alternative route, following the ‘no’-arrows from diamond (a) and again from diamond (b), if the currently used FoV is not the smallest one possible.

As depicted in Figure 6F , this effect can happen close to the boundary of the root. Here, the currently processed voxel was queued with the largest FoV with radius r = 3, and edge size 7. However, in this example the network classifies the voxel as ‘non root’, indicated by a colored cross in the figure. With this classification result, the voxel is inserted in the queue again, now with a smaller FoV. Hence, in Figure 6G the next smaller FoV with radius r = 2 and edge size 5 is depicted with a green color-coding. The reduced FoV size now helps the network to focus more on the actual center voxel of the FoV. When the voxel is then classified as ‘root’, the following neighbor voxels will continue to use the smaller FoV, as can be seen in Figures 6H, I respectively. If the voxel is still classified as ‘non-root’, the FoV size is further reduced as shown in Figure 6J . Here a neighbor of the previously discussed voxel was classified as ‘non-root’ with a FoV of edge size 5. In this way, the voxels of the boundary layers of the root structures are grown by smaller FoVs. This especially comes into effect for regions in which the gradient of the gray-level values promotes a premature edge detection for larger FoVs.

An equivalent process takes place in regions in which the root structure tapers off too much for large FoVs to correctly classify them. This situation is shown in Figures 6K-O . Starting with a FoV of edge size 5, a voxel classified as ‘non-root’ is queued again with a smaller FoV of edge size 3, illustrated with an orange color-coding. Using the smaller FoV, the voxel is then correctly classified as ‘root’ and the growing phase now continues along the thinner root structure. The further progressed state in Figure 6O shows that this effect can happen with multiple voxels at the same time. Note that the processing order of the queued voxels in these sketches do not follow the standard FIFO order of a queue, but rather the order was chosen to illustrate the idea. During the inference process, the queued voxels are evaluated simultaneously as part of the same batch.

As described above, the ‘growing phase’ leaves the FoV size unchanged between the current voxel and the new neighbors which are pushed in the process queue. The FoVs edge size will only be decreased in areas, where a voxel has been classified as ‘non root’. This approach will carry the high confidence levels of the large FoVs through the already grown root structures and sacrifice it for a higher spatial accuracy only where needed. In this way, the radius of the FoV will decrease more and more until a given lower boundary value of r = 1 and edge size of 3 is reached (depicted with an orange color-coding). A classification as ‘non root’ with this FoV, as indicated in Figure 6O , will lead to the next phase of the process.Following the downward ‘yes’ arrow from diamond (b) in Figure 5 , the algorithm now knows with the highest confidence and spatial accuracy, that the current voxel is not from the ‘root’ class and marks it as such. However, this level of spatial accuracy cannot yet be assured for the previously classified root voxels. To tackle this problem, a so-called ‘pruning phase’ is initiated in the affected local region. At this point, it is possible that the already grown structure extends over the true root’s boundaries. In contrast to the premature edge detection in the growing phase, the gradients of the gray level values in the CT data can also promote a delayed edge detection for larger FoVs.

Figure 6P shows an example with a FoV of edge length l = 5 voxels. In Figure 6Q the process queues the same voxel again with a smaller FoV with radius r = 1 and an edge length l = 3, which is the smallest FoV possible. As the voxel is already located considerably across the root boundary, the smaller FoV classifies the voxel as ‘non root’ in Figure 6R . Since the smallest allowed FoV was already used, this initiates the pruning phase in that region of the volume, which allows to double check the currently proposed segmentation by the larger FoVs.

Under these circumstances it may happen that neighboring voxels, which have previously been classified as ‘root’ within a larger FoV, will be re-evaluated using the smallest FoV in later iterations. If the re-evaluation reveals misclassifications by the previously used larger FoV, the pruning process is successful, and the root boundary is corrected and shifted back by updating the classification result at that voxel from ‘root’ to ‘non-root’.

Figure 6S shows this effect of requeuing a voxel sample with a FoV of edge length l = 3, where previously a FoV of edge length l = 5 has predicted the class ‘root’ in an earlier iteration. When the re-evaluation detects the opposing class, the process is repeated in such a way that it again follows the downward arrow in Figure 5 from diamond (b), resulting in another alternative loop.

Even though it only affects voxels, which were classified as ‘root’ by a FoV larger than the smallest one, this path also results in a flood filling operation. Thereby, the boundary of the root is shifted slightly backwards to where the most spatially accurate FoV detects it to be. This is shown in Figure 6T , where in a slightly progressed state of the process, more voxels have been pruned and queued for re-evaluation. If the re-evaluation step confirms the previous result, the pruning phase terminates, and no new neighbors will be pushed to the queue. One example of this possibility is the lower neighbor of the originally discussed voxel in Figure 6P . If all voxels in that region reach this state, the segmentation of the local root structure can be considered complete.

Through the usage of the queue data structure, a parallel implementation of the inference evaluation processes as well as processes concerned with the enqueing of new neighbors is possible. Thus, multiple growing and pruning phases can be performed at the same time in different regions in the CT volume. This is also indicated in the lower right corner of Figure 6T , where a new pruning phase has started on the other side of the root. When the queue is empty, the inference algorithm will terminate for the whole volume.

Figure 7 depicts the loss curves for the training (solid) and validation (dotted) data set over all three training iterations. A vast improvement from the first to the second iteration can be observed, which also corresponds to an improvement of the label data available for successive iterations. The improvement from the second to the third iteration is significantly reduced. Hence it can be concluded that the chosen net architecture (Section 4) has exhausted its ability to generalize based on the available small training data set (Section 3.2), as the training has reached a minimum. Hence, the training was stopped after the third iteration and the network was stored as ‘final network’ N₂ .

Each of the three evaluated inference methods (RootForce, naïve inference, dynamic inference) apply an effective volume mask to reduce the region that needs processing; These are described here for reference:

Two exceptions were made for the dynamic inference results, which are marked by an asterisk in Figure 8 and Table 2 , namely volumes V ₁₃ and V ₁₄. Here the volumes were initially inferred with a mask that left the upper ridge of the pot bottom accessible to the flood-filling operation. This led to a segmented ring in V ₁₄, where the upper ridge around the pot was falsely classified as ‘root’. This in turn allows the network to find additional small roots that otherwise would not have been connected to the main root-structure, as the mask of the pot acts as a natural barrier to the detection of root branches that first grow to the bottom of the pot and then reach back up. In these exceptions, the additionally detected small roots were preserved, while the ring artifacts belonging to the pot bottom were deleted in a post-processing step with another mask. By this approach, the ability of the proposed network to classify roots that were not detected by the reference method RootForce is better demonstrated. Moreover, these upward growing roots are a prominent example of the network correctly classifying the pot mantle as ‘non-root’, since long segments of those branches practically stick directly to the plant pot. While this results in a disconnected segmentation structure, it enables a more sensible comparison of the voxel counts.

In contrast to the reference segmentation method RootForce, the proposed DCNN approach is able to detect and segment plant structures above ground. Nevertheless, the comparison has been restricted to root regions below ground.

For the initialization of the proposed dynamic inference algorithm, single seed voxels were used, which were a-priori known to be part of the ‘root’ class. These positions are usually not known exactly, however, they can be found via thresholding within the main plant stem. Alternatively, a small set of voxels enclosing a region in the volume that is likely to contain a root can be used as seed voxels.

In the following, the test data segmentations are presented with a visual comparison, based on a fused rendering of the 3D structure of the proposed dynamic inference method using N₂ as DCNN approach and the reference method RootForce (see Figure 8 ). Additionally, the segmentation results of the dynamic inference are visually compared to the segmentation of the same network using a naïve inference algorithm, as depicted in Figure 9 .

The affiliation between classified voxels and the underlying segmentation approach is denoted via set notation. Firstly, the set of all voxels segmented by the RootForce approach is denoted as S _RF. Secondly, the set of all voxels segmented by the proposed network N ₂ (from the third training iteration) and using a threshold θ = 50% on the class probabilities (the network has a confidence above 50% that a voxel belongs to the ‘root’ class) is referred to as S _N50. Finally, the network segmentation using a threshold of θ = 20% is denoted by S _N20.

The voxel coloring scheme in Figures 8 , 9 decodes the segmentation method and is denoted in set notation in Table 2 , where the ‘\’ sign denotes the difference of two sets (e.g. ‘A\B’ translates to ‘set A without set B’): ‘yellow’, S _N50 ∩ S _RF indicates the intersection between all voxels segmented by the proposed DCNN N₂ and the voxels proposed by RootForce; ‘green’, S _N50 \ S _RF denotes additionally detected root branches by the DCNN; ‘blue’, S _N20 \ (S _N20 ∪ S _RF) relates to further additionally detected thin root branches with the lower 20% threshold; ‘red’, S _RF \ S _N50 indicates root segments, which are only discovered by RootForce. To obtain a quantitative comparison of the investigated methods, in Table 2 the voxel counts for each color domain are listed separately.

For visualization purposes, the opacity of a one voxel thick layer around the yellow surface was set to zero in both Figures 8 , 9 . However, no root branches were deleted, and hence the qualitative extent and structure of the root system is not altered. This eliminates hollow red or green structures which wrap around the yellow domain and thereby reveals the true extent of the segmentations intersection. Nevertheless, this effect yields some apparent discrepancies in the voxel counts in Table 2 when comparing it to the figures, as the voxel counts still include these hidden voxels. This effect is later discussed in Section 8.1. A similar procedure could have been applied to the surface of the blue regions. However, since the overlapping voxels are less significant in their count, and thus do not disturb the visualized information, this was neglected.

Figure 8 shows the segmented root systems of all test volumes (V ₆ – V ₁₄). Generally, in comparison the segmentation results of the proposed DCNN N₂ approach and the analytical reference method (RootForce) are quite similar, and most of the root structures are detected by both algorithms (yellow voxels in Figure 8 ). However, in all test volume the proposed deep neural network detected numerous additional thin root branches with the 50% threshold (green voxels in Figure 8 ), and even more with the 20% threshold (blue voxels in Figure 8 ), which have not been discovered with the reference approach RootForce (of which exclusively detected voxels are red in Figure 8 ).

As one major finding, the quality of the root surface is increased using the proposed deep neural network approach. This effect is noticeable when focusing on the storage roots and comparing the count of different colored ‘blobs’, which are mostly misclassifications, attached to the root surface. For most of the test volumes, the red colored blobs outnumber the green and blue blobs. Thus, even with the low confidence threshold (θ = 20%), the proposed DCNN approach yields competitive results.

The achieved voxel counts of detected root structures are listed in Table 2 and can be used for a quantitative assessment of the different colored domains. The higher segmentation quality of the proposed approach is hinted at by an increased count of detected root voxels. However, there is a noticeable lack of visible, red-colored voxels in relation to visible green- and blue-colored voxels in Figure 8 and the comparatively small difference in voxel counts in Table 2 . This results from excluding voxels for an unbiased visual comparison, as explained in Section 7.3.

The excluded voxels of the red domain (voxels exclusively detected by RootForce) are concentrated along the surface of thin root branches, while excluded voxels of the green and blue domains (voxels exclusively detected by the DCNN) are scattered on the surface of large storage roots in similar quantity as the individual voxels which are still visible as dark dots on the yellow surface. Most voxels exclusively found by RootForce (red) widen the commonly detected (yellow) root branches, while voxels exclusively found by the proposed network approach (green, blue) extend the structure by new thin root branches. In terms of a qualitative comparison, there is a difference between detecting more surface voxels (like RootForce) versus detecting more root branches (like the DCNN). Contrary to the detection of wider branches, previously unknown root branches yield additional and desired information about the overall structure of the investigated plant root systems, especially when considering the ambiguity of ‘correctly’ discretizing a continuous surface with voxels.

The most noticeable flaws of the proposed DCNN approach are the large, misclassified regions in the test volumes V ₇, V ₁₀, and V ₁₄. These regions prohibit a useful comparison via voxel counts, since the misclassified voxels outnumber the ones that make up the small root branches, and hence cannot be easily separated from the true root system. Investigation of the cause for such errors showed an altered gray level profile in the affected areas, which was caused by higher water content in the soil and lead to an increase of the gray level values in the CT reconstruction. Unfortunately, these increased gray levels are very similar to the gray levels of many root structures. This observation indicates that the trained network has been fine-tuned to a specific soil mixture with a certain water saturation level which results in a characteristic gray level distribution adjacent to the plant. Hence, the obvious cause of this observed effect is the identical scanning parameters for the CT data acquisition of training data.

Focusing on the segmentation results of the DCNN with the lower threshold (θ = 20%, ‘blue’), further significant improvements in detecting thin root branches can be observed. Using the lower threshold, the network detects the same root voxels as when using the higher threshold (θ = 50%), and further extends the segmentation. Many of the test volumes (see Figure 8 , Table 2 ) have a similar amount of green- and blue-colored root branches, meaning using the lower threshold (θ = 20%) resulted in nearly twice the amount of detected root voxels. A few small root branches are also detected by the reference method as well as the proposed network with θ = 20%, but not with the 50% threshold. Two examples of this are in volume V ₆ (mainly red and partially blue branch, growing from the center to the lower right) and in volume V ₁₂ (furthest branch to the right, growing downwards).

Generally, the lower threshold (θ = 20%) also comes with the cost of an increased count of misclassified voxels. However, the effect is more subtle than expected, since most misclassifications happen in regions, where the network already struggles with a higher threshold (θ = 50%), e.g., at the humid sediment in volumes V ₇, V ₁₀, and V ₁₄. Though, misclassifications of soil lumps sticking to the root surface are more frequent with lower threshold compared to RootForce.

Some roots that follow the outer edge of the plant pot, as depicted for volumes V ₈, V ₁₃ and V ₁₄, highlight the success of the proposed and employed sub-labeling technique. In such situations, other networks models with the same network architecture and trained without the sub-labels, have frequently initiated the inference algorithm to follow the plant pot (by ‘flood-filling’) as well. This argument can also be made for the bottom of the plant pot as a counter example, since it was not focused on in training by the sub-labels (see Section 3.5). Note that the pot bottom was excluded from the segmentations in Figure 8 by a post-processing step as explained in the previous section.

It can be argued that the problem of the misclassification of the base of the pot shows the advantage of properly applying the proposed sub-labeling technique, as it allowed for exact control over samples from the pot mantle but not the pot bottom (see Section 3.5). More precisely, specifically choosing samples from the approximated segmentations of the pot mantle to augment the training data, resulted in successfully teaching the network to not recognize the ‘pot’ voxels as ‘root’ voxels. However, due to the complicated geometry of the pot bottom, no reference segmentations were available from which specific samples could augment the training data. Therefore, the alternative of randomly sampling from the bottom volume to statistically include challenging samples in the training data was used. Unfortunately, this did not focus learning on such samples sufficiently and hence was not successful in teaching the network to identify voxels from the pot bottom as ‘non-root’. This problem could be alleviated by possibly increasing the training samples from the pot bottom. However, this also could alter the proportion of multiple sub-labels to an unknown degree and therefore opposes the principle of fine-tuning the composition of the training data.

As described in Section 6, the naïve algorithm iterates through every voxel in the volume independently. This approach has the disadvantage that mostly ‘non-root’ voxels are queried, since they constitute most voxels in a volume and thus, waste expensive computation time. Moreover, this approach is more vulnerable to misclassifications as the deep neural network never connects the queried results within one volume, leading to a cluttered 3D structure (see Figure 9A ). Here, a helpful post-processing step is the application of a ‘connected component analysis’ (CCA), which extracts the largest detected structures and removes most of the clutter. In Figure 9 the segmentations from the proposed DCNN N₂ from the third training iteration are depicted, comparing the naïve and dynamic inference methods. To aid the visualization, the reference segmentation (RootForce) is included in the illustrations for an identical color-coding to Figure 8 . In Figure 9A , the raw output of the naïve inference algorithm is depicted for Volume V ₆. It is obvious that there exists a lot of clutter, namely misclassifications, generated by the network (depicted as green- and blue-colored voxels), which obscure the view onto the root structure in the center.

Figure 9B shows the result after applying the connected component analysis (CCA) which deletes all structures consisting of less than 1,000 connected voxels. This results in only some larger clumps of soil in the segmentation. Some large, misclassified structures are part of the pot (top left) and some topsoil sticking to the pot (top right, background). Also, small root structures that have been detected by the network, become visible in this view. Those are found around the root in the lower half of the illustration. The segmentation result of the proposed dynamic inference is depicted in Figure 9C . Since the proposed dynamic inference approach automatically generates only one single connected component, unwanted structures in the volume are automatically discarded. However, this comes at the cost of possibly undetected root sections, which could have been correctly classified by the network using the naïve inference (cf. Figure 9B ) but remain hidden for the flood-filling approach. For example, in Figure 9B there are three thin green root branches visible in the foreground (horizontally centered). In Figure 9C one of them is missing in the segmentation using a threshold of \theta = 50% but detected by the lower 20% threshold (colored in blue). Another branch (colored in red with some blue depicted in Figure 9C ), indicates that only the reference method (RootForce) and the deep neural network with threshold θ = 20% was able to detect it, while the same branch is colored yellow in Figure 9B , indicating that the network with threshold θ = 50% can detect it as well when it is not blocked off by the flood-filling process.

In conclusion, an obvious tuning parameter is θ, thresholding the class probability of the network output. Lower thresholding (e.g., θ = 20%) is able to segment thinner and finer roots, however, is also susceptible to more misclassifications. For the naïve algorithm, this means also more unwanted large, connected components which are harder to separate from the desired root branches, as can be seen in Figure 9B by the additional blue soil clumps. The segmentation of the dynamic algorithm is affected more positively by a low threshold, since it increases the connectivity of the root components as described above, while other parts of the segmentation are barely altered. This, on one hand, leads to more detected thin roots, which are connected to the main-root structure, but on the other hand, leads to a slightly rougher root surface on a few branches. This can be seen in Figure 9C by the thin blue root branches and the blue dots along the yellow surface respectively.

We also note that the naïve inference only uses one FoV for all queries, while the proposed dynamic inference adaptively changes the radius r of the FoV. A FoV with edge length l = 5 was applied for the naïve inference. Using larger FoVs with a constant radius would lead to less, but still obstructing, clutter in the raw segmentation and would furthermore fail to detect thin root branches. When investigating the effect of varied FoV sizes of the dynamic algorithm, it becomes apparent that most voxels are evaluated with the largest, initial FoV size. Adaptation to smaller sizes happens only at the direct root boundaries or very thin root structures with small local voxel counts. Hence, the effect is not statistically significant for the voxel count. However, using only larger FoVs with the flood-filling approach misses most of the thin roots and yields thicker root structures overall, due to the missing pruning step.

Furthermore, for the naïve inference the count of root segments kept after the CCA is dependent on a hyper-parameter. This could be set in such a way that only the largest connected component, or components which fulfill known geometric requirements such as elongation restrictions, are kept in the volume. This could yield segmentations, which closer resemble the segmentation of the proposed dynamic algorithm. However, choosing adequate criteria to discard disconnected components becomes challenging since small and thin roots are easily caught erroneously when trying to declutter the segmented volume.

Some of the soil lumps in Figure 9B can also show up within the dynamic inference, if its initialization consists of guessing where possible root voxels are located in the volume, e.g., by selecting a small surface of seed voxels in the volumes center. As long as the root structure crosses the surface, all parts of the roots will be detected as shown in Figure 9C . If some single soil lumps – depicted in Figure 9B – happen to cross this surface, they would be included in the segmentation of the dynamic inference as well.

In the following, the evaluation times of the different inference methods are compared. Improvements are mainly achieved with respect to the naïve method, as the RootForce algorithm is based on classic image processing methods whose run-time is highly optimized.

The RootForce algorithm always needs the same time duration to evaluate a volume of a fixed size. For our test volumes (1024 × 1024 × 900 voxels, see Section 3) this amounts to approximately 10 minutes on an Intel Xeon CPU (E5-2620v4@2.10GHz) with 256 GB system RAM.

Evaluation with the naïve method also takes the same time for a fixed volume size. However, the run-time significantly depends on the used FoV size. The limiting factor is the VRAM used on the graphics card for each batch. For sequential processing on one GeForce GTX 1080 Ti GPU, conducted measurements with FoV edge lengths of l = 5 and l = 15, allowed for batches of size 2¹⁴ = 16384 samples and 2⁹ = 512 samples respectively, and resulted in evaluation times of roughly 120 and 3,000 minutes respectively.

The run-time of the dynamic inference algorithm is dependent on the amount of connected root voxels, which are present in the volume, whereby the connectivity is decisively dependent on the used threshold on the class probability. A more subtle influence comes from the structure of the root itself. Since more voxels are queued simultaneously during the flood-filling of thicker roots, the evaluation can leverage a better parallelization via the batch size. Contrary, in thin roots, voxels are queued more sequentially which results in smaller batches per query. For a maximal speedup, the parallel processing capabilities mentioned in Section 6.3 are utilized. With four GeForce GTX 1080 Ti GPUs (in one computation node), the smallest root system (V₉ ) can be segmented in 5 minutes, the largest root system (V₈ ) needs up to 25 minutes. Misclassifications as in V₇ also cost extra time, which in this case amounted to 36 minutes.

For comparison we assume the best conditions for the naïve algorithm, i.e., inference with FoV of edge length l = 5 and perfect speedup for parallelization (i.e., the measured processing time of 120 minutes is divided by the count of used GPUs). When using four GPUs, the naïve algorithm can be quicker for exceptionally large root systems, however for our testing volumes this was not the case. When using more than four GPUs (in multiple computation nodes) the naïve algorithm will surpass the dynamic algorithm, since the communication overhead for parallel processing of the naïve algorithm is trivial and hence, has an advantageous scaling property.

However, when scaling with respect to volume size, the dynamic algorithm will have an advantageous scaling property, since the fraction of root voxels to volume size grows slower than the total voxel count in the volume. Therefore, less queries are required in the dynamic algorithm to process a large volume. Even more so, if no mask is available to generously reduce the processed region for the naïve algorithm (see Section 7.2). Scaling with respect to volume size applies to larger objects or objects of the same size being scanned with a higher voxel resolution.

One general complication of evaluating the quality of the proposed set of methods is the unobtainable, pristine ground truth data of the root structure. Even manual annotation would most likely not yield the desired quality, due to the error-prone fine structures and low contrast gray levels (see Section 3.1). Hence, the exact structure and total voxel count of the true root system is never known, and thus, it cannot be stated how many root branches are left undetected or are falsely segmented. Furthermore, it is challenging – if not even impossible – to decide whether undetected small roots are the result of a lack in generalization capabilities of the proposed deep network model or are due to too many false negative samples in the training data set or changes in the scanning system. Essentially, this effect blurs the line when deciding at which point the deep neural network starts overfitting to the true data distribution, as the network model possibly learns unknown characteristics of the flawed training data set, which can then not be called out by the validation and test data sets inheriting the same flaws. In this case, ‘successfully’ learning generalizations of adequate features on the flawed data might be equivalent to a data set specific adaptation, i.e., overfitting, when considering the unknown ground truth data. Nevertheless, typical cues for overfitting do not show up on the available training curves for the data sets (see Section 7.1).

An obvious shortcoming of the investigated approach is the problem of the bottom of the plant pot. Hence, for future extensions and applications, it is advisable to use plant pots with either a more trivial geometry or obtaining an alternative segmentation of the plant pot for the sub-labels (see Section 3.5) and thus focus more on the negative samples during the training. Alternatively, if the range of future application is limited, using a segmentation of the plant pot as mask to prohibit network queries outside the sediment region is also a viable option, since using such a mask during the inference run has no disadvantages for the computation time (see Section 8.3).

The comparison between the proposed dynamic inference and the naïve inference algorithm shows that inference by flood-filling can be a drawback if the root structure is not one single connected component as seen from a certain threshold. Basically, root branches which would be detected by the deep network model will not show up in the dynamic inference segmentation, when the branch is by accident disconnected from the main root structure. Increasing the connectivity of the detected root structures is equivalent to an increase in the network’s fidelity, which requires a training data set of high quality. If increased computation times can be handled or tolerated, an alternative approach could be to use the naïve inference with a conservative CCA as post-processing (see Section 8.2). The remaining components in the volume could be separated into soil lumps and root branches by a shallow learning model and basic features computed from the geometry of the connected region.

The most significant disadvantage of the proposed DCNN-based segmentation approach is the lack of diverse CT scans of roots. All described experiments were carried out on scans of the same (Cassava) plant type, grown in the same soil mixture and captured with the same scanning parameters and CT scanner (see Section 3). Alternating any of these influencing factors will considerably change the problem statement in the form of the gray level distributions. This would limit the current network model in scope and usefulness, as the discussion on the humidity levels in the sediment already indicated (see Section 8.1). Generalizing the proposed network at this level will require a careful composition of diverse CT scans for the training procedure, and most likely a larger network architecture. Other possible approaches could be to train a generative model, to map arbitrary CT scans of root systems to a common generalized latent space, in which a simple model like ours could be used for final segmentations.

A drawback of the dynamic inference algorithm is that it was specifically developed to work with single voxel predictions from the DCNN. This requires significantly more forward passes through the neural network to evaluate every voxel, than modern image-to-image DCNNs, like the U-Net (Smith et al., 2020; Soltaninejad et al., 2020), could achieve. Nevertheless, a trade-off between fast computation time (e.g., by using U-Net architectures) and carrying semantic context between individual predictions (e.g., by using flood-filling procedures) has to be made. A promising direction for future research was proposed by Januszewski et al. (2018) (and recently evaluated by Gruber et al. (2021)) who integrated the flood-filling approach directly into an image-to-image DCNNs framework called “Flood-Filling-Networks”.