This study was conducted at Maolan National Nature Reserve (107°52’E to 108°05’E and 25°09’N to 25°20’N) ( Figure 1 ), with a total area of 21,285 hectares. It is the only remaining, original, stable karst forest system at the same latitude worldwide, with a mean elevation of 800 m. The rocks in the region consist predominantly of pure carbonate limestone and dolomite, while the soil is primarily weakly alkaline black lime soil. The region has an average annual temperature of 15.3°C, averaging 5.2°C in January and 23.5°C in July. The growing period spans 237 days, while the frost-free period totals 10 days. The annual precipitation is 1752 mm, typically from May to October. Additionally, the annual average relative humidity is 83% (Zhang et al., 2012). The most widely distributed vegetation types in the nature reserve include SL, MTSF, CF, CBMF, and BF. The SL communities are mainly dominated by Cornus parviflora and Celtis sinensis, while Liquidambar formosana and Castanopsis fargesii were the dominant species in CF. Pinus kwangtungensis was the constructive species in the CF, and Pinus massoniana was the dominant species in CBMF. In addition, Acer wangchii and Boniodendron minus were the dominant species in BF.

According to the standard raised by Condit (1998) for the standard handbook of forest dynamic plots (FDPs) establishment, we established 9 to 10 FDPs for each of five dominant vegetation types: shrubland (SL), mixed tree and shrub forest (MTSF), coniferous forest (CF), coniferous broadleaf mixed forest (CBMF), and broadleaf forest (BF) ( Table 1 ). The distance between every two plots is greater than 50 m to avoid the confusing impacts of spatial autocorrelation and ecotone and all the woody individuals with DBH > 1 cm in the FDPs were tagged, measured, and identified to species (Condit, 1998).

The NMDS show the stress is less than 2, the result is meaningful, furthermore significant differences between CF and other vegetation types on one axis, and its most concentrated on two-axis. Therefore, significant species composition differences between CF and other vegetation types, and the species composition of CBMF was similar to that of MTSF and significantly different from that of BF and CF. ( Figure 2 ). The dominant species varies among vegetation types. SL is dominated by C. parviflora, with an IV of 6.92, while MTSF is dominated by L. formosana, with an IV of 10.27. As for CF, P. kwangtungensis is the dominant species, with an IV of 43.03, and P. massoniana dominates CBMF, with an IV of 23.73. A. wangchii, B. minus (6.03), and Platycarya strobilacea (4.69) are the dominant species in BF ( Supplementary Table S1 ).

BF had the highest Margalef’s index of species richness among the five vegetation types, indicating the most species in plots. No significant variation was found among the other four vegetation types in Margalef’s index ( Figure 3A ). The Pielou’s index of species evenness of CF was significantly lower than those of the other four vegetation types, indicating a significant species dominance trend compared with other vegetation types ( Figure 3B ). The rarefied species richness results showed significant observed variations among all five vegetation types after controlling the confusing impacts of abundance. Besides, the BF still had the highest richness and significantly different from other vegetation types, and the CF is the lowest and significantly different from SL and MTSF ( Figure 3C ).

CF has the highest decline rate and similar trends were observed on the species rank-abundance curves of MTSF and CBMF. BF and SL has slow decline rate on the species rank-abundance curves, which had lower species evenness than CF, MTSF and CBMF ( Figure 4A ). On the species accumulation curve, MTSF showed a similar trend to that of CBMF, and the species accumulation rate of CF was lower than other vegetation types and BF was largest ( Figure 4B ).

The NCM results showed that the R² of CF (0.679) and BF (0.533) were significantly higher than the other three vegetation types, suggesting that the neutral process dominated the community assembly in BF and CF, while the niche process dominated the community assembly of the other three vegetation types ( Figure 5 ). In addition, the highest Nm value in BF suggested a greater degree of species diffusion than other communities.

As the comprehensive component of biodiversity, species composition can reflect the ecosystem structure and function (Rahman et al., 2022). Since the species diversity pattern involves the number of species and the species evenness, any changes in both components can lead to species composition variations (Chao et al., 2014). Our results showed significant variations among vegetation types in the karst landscape ( Figure 2 ). Among all the five vegetation types, BF had the highest species richness, and CF had the lowest Pielou’s index of species evenness ( Figure 3 ).

As a single dominant species in CF communities, P. kwangtungensis significantly controls the vegetation type structure and the community environment formation, with no obvious subdominant species and numerous rare species with low species evenness ( Figure 4A ; Supplementary Table S1 ). CF is normally distributed in habitats with higher altitudes or deeper slopes, where fewer species could colonize (Wang and Cui, 2023). Additionally, the aggregation strength of P. kwangtungensis increases with increasing elevation, thus occupying the niche space of other species. This could explain the lower species richness and evenness in CF than other vegetation types. On the other hand, the higher migration rate of rare species in CF inhibits the increase of species richness, leading to a pattern with a lower species richness and slower species accumulation rate (Stanley Harpole and Tilman, 2006; Xu et al., 2019). Thorn et al. (2020) found that rare species were the main factor of species composition differences. A higher rare species proportion in CF might be the main reason for its species composition variation compared with others ( Figure 2 ). Consistent with the results of Zhang et al. (2012), our results showed that BF had the highest species richness and fastest species accumulation rate, with no obvious dominant species. BF in this study was established in subtropical evergreen deciduous broadleaf mixed forest, considered the climax in the karst landscape. Such a community normally features a complex community structure and a higher species richness (Qi et al., 2021). BF is used to considered in a relatively favorable environment. The relatively higher soil quality results in more individuals or abundance, possibly leading to a pattern with higher species richness (Zhang et al., 2022b). In addition, the stronger habitat heterogeneity at local scales supports various micro-habitats, which helps feed more species (Bátori et al., 2023). Thus, the relatively higher quality of habitat conditions and the stronger habitat heterogeneity might be the main reason for high species richness and evenness.

However, the species richness differed greatly among the five vegetation types after controlling the confusing impacts of abundance. A significant trend of BF > SL > MTSF > CBMF > CF was found in terms of rarefied species richness, which emphasized the importance of considering the impacts of sample size. In other words, individual variations among vegetation types could lead to a misunderstanding of species richness in the karst landscape. The species richness of SL was significantly higher than MTSF and CBMF after removing the influence of abundance on species richness ( Figure 3C ). In addition, the species richness of MTSF was significantly higher than that of CBMF, SL and MTSF are common secondary forests in karst areas, dominated by species such as C. parviflora, L. formosana, and C. sinensis ( Supplementary Table S1 ). The lack of human disturbance allows shrubs to persist, with small-diameter tree species being the mainstay, leading to a scattered community advantage. Therefore, their species accumulation rates are second only to that of BF ( Figure 4B ), with high species evenness and no significant differences in species composition from other vegetation types except CF. Xiao et al. (2023) also suggested that high habitat heterogeneity of rocks increased shrub richness. In the MTSF community, tall deciduous trees gradually closed the canopy, limiting the understory light conditions (Li et al., 2016; Lu et al., 2016). Those heliophilous shrub species can not adapt to such poor light conditions and are excluded, thus exhibiting decreased species richness. However, the occurrence of tall deciduous trees facilitates the decomposition of fallen leaves and litter and promotes soil function recovery (Zeng et al., 2020). As a result, the immigration of shade-tolerant tree species, such as Machilus nanmu, Quercus glauca, and Cornus kousa, is promoted, leading to a higher species richness relative to CBMF. Like CF, CBMF was strongly dominated by extremely few species, such as P. massoniana and L. formosana. Such a monodominant community often leads to higher dominance and lower evenness. Consistent with the results of Zhang et al. (2022a) that P. massoniana – L. formosana mixed forest has higher species richness than CF, this study also showed a similar trend after controlling the sample size. The main reason is that adding broadleaf trees can increase the number of plant species and provide different types of litter for forest soil, thereby improving the physical and chemical properties of the soil (Zhang et al., 2022a). Moreover, adding broadleaf tree species also changes the resource acquisition strategies of the plants. Qin and Shangguan (2019) reported that compared with P. massoniana plantations, P. massoniana – L. formosana mixed forests show the most significant increase in leaf nutrient content and the best resource acquisition strategy to adapt to different environmental conditions.

The stochastic spread of species mainly depends on the difference between local species richness and diversity and the degree of diffusion restrictions. As a neutral-based process model, NCM is a valid approach for inferring stochastic processes acting on community assembly (Chen et al., 2019). This study used neutral models to simulate different vegetation types and found that both CF and BF communities were primarily governed by neutral processes, with CF being more strongly influenced by neutrality (R² = 0.68, explained by NCM), indicating that stochastic processes dominate the community assembly of CF ( Figure 5C ). The main reason is that in the CF community, P. kwangtungensis occupies an absolutely dominant position, with no obvious co-occurring species and relatively high rarity of other species ( Figure 4A ; Supplementary Table S1 ). In contrast, the species are more susceptible to stochastic processes due to their small population size, high migration rate, and limited geographical range (Xu et al., 2019; Zhang et al., 2022c). Additionally, the P. kwangtungensis population is large as it is adapted to high altitudes with low survival pressure. Coniferous trees often have allelopathic effects that inhibit the occurrence of other species and compress the ecological niche of other species within the community (Ehlers et al., 2014; Wang and Cui, 2023), resulting in strong randomness in species occurrence and stochastic processes dominating the community.

As the typical climax community in this landscape, BF communities have higher species richness and diversity than CF ( Figure 3 ). Rare species are fewer within the communities, and the species distribution is relatively uniform. Therefore, the R² value of the NCM is slightly lower than that of CF ( Figures 5C, E ). However, according to the calculated Nm values, species dispersal between plots in BF is higher than that in CF, indicating species dispersal as one of the important mechanisms of species coexistence in BF communities. Contradicting the findings of this research, however, Su et al. (2023) found that humidity and soil significantly impact species composition in karst BFs, indicating the effects of environmental filtering on ecological processes in the community. The reason may be that in the current study, dominant species in the karst BF community have adapted to the moisture stress and regulated their water use strategies by root plasticity and leaf shedding, adapting to the severe karstic habitat (Ding et al., 2023).

Previous studies suggested that secondary forests driven by deterministic processes could recover their community structure to the state before disturbance (Finegan, 1996; Letcher and Chazdon, 2009; Norden et al., 2009). In this study, the dominant ecological process in SL, MTSF, and CBMF tends to be the niche process, indicating that deterministic processes dominate the community assembly of these forests, which was consistent with the studies of tropical secondary forests by Norden et al. (2009), Cequinel et al. (2018) and others, where deterministic processes drove the composition of forest species. Previous studies have also reported that the colonization and extinction processes in forest communities are primarily determined by environmental heterogeneity and species competition (He et al., 2022). SL, MTSF, and CBMF are karstic secondary forest communities. The biological and non-biological filtering frameworks may be important in its ecological process (Münkemüller et al., 2020).

Despite the abundant rainfall in the karst landscape with a subtropical monsoon climate, the aboveground and underground dual-structure hydrological system often causes severe precipitation leakage, resulting in severe surface drought and frequent temporary droughts (Ding et al., 2023). Moreover, the karst surface is steep and irregular, with low and discontinuous soil coverage (Qi et al., 2021). Chase (2007) found that the community species composition similarities increased after drought treatment, possibly due to the strong drought-induced environmental filtering that filters out species with weak competitiveness and increases species similarity between communities. Therefore, environmental filtering may affect the construction of secondary forest communities. Among the three secondary forest communities, SL has the lowest R² ( Figure 5A ), mainly because environmental filtering often has a smaller impact on larger species (Kristiansen et al., 2012). The SL communities are mostly dominated by small shrubs with a higher resource utilization efficiency than large trees and, therefore, a tendency to receive the most effects of ecological niche processes. As the typical dominant community in the karst landscapes, BF has high species diversities. Although CF communities’ species diversity is lower than BF communities, they are both dominant communities with stable community structures in karst areas. Thus, it can be indirectly inferred that the different developmental stages of karst communities may be governed by different community assembly mechanisms.