Hyperspectral data have revolutionized remote sensing by offering detailed spectral information across hundreds of narrow, contiguous bands, enabling fine-grained analysis of surface materials (Hamedianfar et al., 2023). This capability is particularly valuable in Earth sciences, where it supports the study of critical features such as urban growth, water resources, built-up areas and vegetation health. Hyperspectral data provide an unparalleled ability to distinguish subtle variations in surface composition, enhancing the understanding of processes like land-use change, water body dynamics, and ecosystem responses to environmental stresses.

Indices derived from hyperspectral data, such as the Normalized Difference Water Index (NDWI), Normalized Difference Red Edge (NDRE), Normalized Difference Built-up Index (NDBI) and Green Normalized Difference Vegetation Index (GNDVI), play a crucial role in monitoring Earth surface features. NDWI is widely applied in water resource management, aquaculture mapping, and coastal monitoring (Taloor et al., 2021; Guha et al., 2020; Xu et al., 2021; Laonamsai et al., 2023; Ihsan et al., 2023; Zhu et al., 2021; Fu-min et al., 2007). NDRE and GNDVI are essential for assessing vegetation health, monitoring land cover changes, and analyzing vegetation responses to climatic conditions (Frampton et al., 2013; Mangewa et al., 2022; Maccioni et al., 2001; Jorge et al., 2019; Bonfil, 2017; Jiang W. et al., 2021; Lu et al., 2020). NDBI serves as a vital tool for detecting built-up areas, enabling urban planning, infrastructure development, and the analysis of land-use changes (Zheng et al., 2021; Bhatti and Tripathi, 2014; Yasin et al., 2022; Hadeel et al., 2009; Liu and Zhang, 2011; Delogu et al., 2023; Enrique Valdelamar Martínez et al., 2024).

Despite their utility, the high dimensionality of hyperspectral data poses challenges in processing and analysis (Alcaras et al., 2021; Mekuriaw et al., 2017; Omran et al., 2023). Noise removal, dimensionality reduction, and band selection are essential preprocessing steps for extracting meaningful information from these datasets (Rasti et al., 2018; Rasti et al., 2020; Zheng et al., 2020). Current research primarily employs single-band combinations to compute indices, which does not fully leverage the rich spectral range offered by hyperspectral sensors (Oppelt, 2002; Zarco-Tejada et al., 1999). Furthermore, existing studies lack comprehensive methodology to select bands of hyperspectral data to compute indices.

The increasing volume and complexity of hyperspectral data necessitate automation in analysis workflows. Parallel processing is a promising solution to mitigate computational bottlenecks, distributing tasks across multiple cores to enhance efficiency and scalability. While parallel processing has been employed in various geospatial analyses, its application in hyperspectral data workflows, particularly for automating band selection, remains under explored. This study addresses these gaps by automating the selection of spectral bands from AVIRIS-NG data for computing key Earth science indices. The approach involves calculating spectral indices from all possible AVIRIS band combinations, comparing them with Sentinel-2-derived indices using root mean squared error (RMSE), and selecting optimal bands based on statistical criteria for specific Land Use Land Cover (LULC) categories. The RMSE serves as a critical metric for quantifying the disparity between datasets and evaluating the accuracy of spectral indices derived from them. RMSE is chosen for its sensitivity to larger deviations, which is crucial in hyperspectral analysis, where even minor discrepancies across bands can impact index reliability. Unlike metrics like Mean Absolute Error (MAE) and Mean Bias Deviation (MBD), RMSE’s quadratic weighting emphasizes substantial deviations, helping detect differences in spectral response characteristics. Its interpretability in the same units as the data further facilitate clear assessments of sensor compatibility, supporting accurate, consistent index derivation across sensors. The Skewness correction technique box-cox transformation, is employed to address any asymmetry in the distribution of RMSE values, ensuring robust comparisons between datasets. Subsequently, bands are selected based on their agreement metrics for specific LULC categories.

Automation is achieved through Python-based workflows that leverage parallel processing, significantly reducing computational time and enabling large-scale geo-spatial analysis (Wu et al., 2021). In this study, the efficiency of parallel processing for automating hyperspectral data analysis is evaluated using a high-performance computational machine running the Linux Mint operating system within a virtual box environment. The computational platform boasts 32 cores and 128GB RAM, but to ensure optimal performance and avoid interference with existing processes on the machine, only 25 cores are allocated for computation. By leveraging the multiprocessing capabilities of this setup, the study aims to expedite data processing tasks, such as band selection and spectral index computation, thereby enhancing overall workflow efficiency and reducing computational turnaround times.

By focusing on indices like NDWI, NDRE, NDBI, and GNDVI, this study highlights their relevance in Earth science applications, including water resource management, vegetation health assessment, and urban monitoring. Furthermore, it evaluates the comparability of hyperspectral and multispectral sensors, addressing a critical need for consistent and accurate geo-spatial analyses. This research advances hyperspectral data methodologies by integrating automation and computational efficiency, contributing to a deeper understanding of Earth’s dynamic systems.

The study area is located in Vuyyuru, located in Andhra Pradesh, India, sharing the border of Guntur and Krishna districts (Figure 1). The hyperspectral data from the Airborne Visible and Infra-Red Imaging Spectrometer–Next-Generation (AVIRIS-NG) campaign under Phase 2A, was collected on 26 Feb 2018 Figure 1. The data covers an area between 16°20′N and 80°42′E (site #114). This hyperspectral dataset comprises 430 narrow spectral bands ranging from 380 to 2510 nm at 5 nm intervals and has a spatial resolution of 4 m (Bhattacharya et al., 2024). After undergoing radiometric, atmospheric corrections, and bad band removal processes, the original 430 bands are reduced to 372 bands for analysis.

In addition to the AVIRIS-NG data, the study also incorporates data from the Copernicus Sentinel-2 mission. Sentinel-2 comprises two polar-orbiting, multispectral satellites, Sentinel-2A and Sentinel-2B, which are phased at 180° and share the same sun-synchronous orbit. These satellites offer a wide swath of 290 km and revisit the same area every 5 days. The Sentinel-2 data used in the study was acquired on 25 Feb 2018. This dataset includes 13 bands, encompassing visible, near-infrared (NIR) bands with a spatial resolution of 10 m, and Shortwave Infrared (SWIR) bands with a spatial resolution of 20 m. The Sentinel-2 data was corrected, and the reflectance values were used directly for the study.

This study employs an automated Python-based framework to streamline hyperspectral band selection and index comparison between AVIRIS and Sentinel-2 imagery, focusing on the efficient processing of spectral indices. The methodology starts with user-defined input parameters, including paths to AVIRIS and Sentinel-2 imagery, a shapefile containing LULC classes (such as Barren Land, River Sand, Urban, Vegetation, and Water), an output directory, and a target spectral index. Based on the selected index, Sentinel-2 bands are identified by calculating their central wavelengths and bandwidths to establish upper and lower bounds (Table 1). These bounds are then applied to the AVIRIS dataset to retrieve corresponding bands within the defined wavelength range, ensuring spectral alignment across sensors. For each LULC class, all possible band combinations within the range are calculated to compute the spectral indices, allowing for detailed inter-sensor analysis. The accuracy of these spectral indices is quantified using the RMSE, which provides a measure of the difference between Sentinel-2 and AVIRIS indices across each LULC class. To further normalize and refine RMSE values, a Box-Cox transformation is applied, which corrects for any skewness and enables more robust statistical comparisons. The final band selection process applies an RMSE threshold (Mean–Standard Deviation) to identify the bands with minimal spectral error for each LULC class. Additionally, the mean bands of index are chosen based on their frequency of occurrence across all LULC classes, prioritizing bands that consistently appear in the low-error subset. This automated pipeline integrates essential spatial and statistical operations, significantly enhancing the reproducibility and efficiency of hyperspectral band selection and inter-sensor spectral comparison, thus supporting robust and large-scale hyperspectral data analysis across diverse land cover types. The whole methodology is illustrated in Figure 2.

The band selection process for indices such as NDWI, NDRE, GNDVI, and NDBI is meticulously executed according to the established methodology. This involves identifying optimal band combinations to ensure the most accurate computation of each index. Figure 3 provides a visual representation of the datasets used in both the testing and validation phases of the study. Specifically, the test dataset is employed to determine the most suitable AVIRIS bands for each index, utilizing a range of spectral and statistical analyses. Once the optimal bands are selected, the validation dataset is used to compare and verify the results, ensuring the selected bands provide consistent and accurate index calculations. This dual-phase approach helps to confirm the reliability and applicability of the chosen bands across different datasets and conditions.

For most land cover classes for NDWI, the RMSE values in the adopted methodology are generally lower compared to the mean of all bands and single central bands. The adopted methodology consistently exhibits lower RMSE values across different land cover classes, indicating higher agreement and stability in NDWI values (Figure 4). Notably, the “Water” class demonstrates a substantial reduction in RMSE values in the adopted Methodology, suggesting consistency and accuracy in water index values derived using the proposed methodology. Barren Land, while exhibiting slightly higher RMSE values, still demonstrates relatively good agreement and stability when compared to single central bands.

Therefore, the selected methodology exhibits robustness in deriving NDWI values across various land cover types. The methodology consistently outperforms the mean of all AVIRIS bands and single central bands approaches, demonstrating its effectiveness in providing accurate and consistent water index values. While challenges may exist in accurately characterizing barren land, the adopted methodology presents a favorable outcome, indicating its reliability in water index assessment across diverse land cover types.

Figure 5 illustrates the root mean squared error (RMSE) values of Sentinel-2 and AVIRIS NDRE corresponding to the mean of all bands, single central bands, and the proposed methodology across various LULC categories. The adopted methodology consistently demonstrates lower RMSE values compared to the mean of all bands and single central bands across all land cover classes. This consistency suggests that the chosen methodology effectively enhances the agreement and stability of NDRE values across diverse land cover types. Vegetation exhibits the lowest RMSE value when the mean of all bands is considered. However, it's noteworthy that this RMSE value is lower than that derived from the single central bands. Urban areas consistently show higher RMSE values, even though the adopted methodology has lower RMSE values. Water bodies demonstrate moderate RMSE values, indicating variability in NDRE values. The bands selected using adopted methodology consistently outperform the mean of all bands and single central bands approaches, indicating its effectiveness in providing accurate and consistent NDRE values.

Across all land cover classes, the proposed methodology consistently yields lower RMSE values compared to those obtained by considering the mean of all bands for GNDVI (Figure 6). The methodology also generally demonstrates RMSE values lower than those derived from a single central band, indicating its effectiveness in calculating GNDVI value using the proposed methodology. In the “Barren Land” and “Vegetation” classes, the proposed methodology’s RMSE values fall between those of the mean of all bands and single central bands.

The bands selected using the proposed methodology consistently outperform both the mean of all bands and single central bands in providing accurate and consistent GNDVI assessments across different land cover classes. In the “Urban” class, the methodology yielded an RMSE value of 0.0944, compared to 0.1025 and 0.1096 for the mean of all bands and single central bands, respectively. This underscores our proposed methodology’s effectiveness in reducing noise and enhancing the reliability of GNDVI values, essential for vegetation assessment and monitoring applications.

While there are slight variations in RMSE values across land cover classes, the proposed methodology consistently yields RMSE values that are marginally lower than those obtained from considering the mean of all bands and single central bands fro NDBI. In the case of “Water,” the proposed methodology’s RMSE value is notably lower compared to the other two approaches. Overall, while there are differences in RMSE values across different land cover classes, there is no significant difference between the proposed methodology and the other approaches (Figure 7).

Comparing the RMSE values for the Normalized Difference Built-Up Index (NDBI) across different land cover classes, it's evident that our adopted methodology doesn't exhibit any detrimental effects. While there are slight variations in RMSE values across different approaches, including considering the mean of all bands and single central bands, bands selected using our methodology consistently maintain comparable or slightly lower RMSE values. This indicates that our adopted methodology effectively reduces noise and enhances the reliability of NDBI assessments without introducing any negative impacts on the accuracy of the results. Thus, it can be concluded that there is no adverse effect associated with the utilization of our methodology for NDBI computation across various land cover types.

The band selection approach enhances the results across diverse LULC types by prioritizing spectral bands that consistently exhibit high relevance and reliability. By focusing on bands that capture key biophysical and biochemical properties across various LULC classes, such as vegetation, urban areas, water bodies, and barren land, this method minimizes variability in spectral indices and ensures robust results. This is particularly beneficial for indices, which rely on precise discrimination of spectral features to accurately represent vegetation health, water presence, or urban areas. Additionally, the approach yields lower RMSE values compared to methods using all band reflectances or single central bands, demonstrating superior alignment with LULC spectral characteristics. By avoiding overfitting to specific datasets, the method enhances the transferability of results across regions and datasets, ensuring broader applicability. This current approach effectively mitigates these challenges through robust metrics like RMSE, ensuring reliable, transferable, and scalable hyperspectral analysis.

Python code was developed using open-source libraries to select bands from hyperspectral imagery using different indices such as NDWI, GNDVI, NDRE, and NDBI. Peddinti et al. (2021) included bands within the range of Sentinel-2 bandwidth in their study.

The results demonstrate lower RMSE values across various land cover classes for NDWI values, with a notable reduction in the Water class. While the Barren Land class may exhibit slightly higher RMSE values, it still maintains relatively good agreement and stability compared to single central bands. Similarly, the adopted methodology consistently shows lower RMSE values compared to the mean of all bands and single central bands across all land cover classes for NDRE values. While vegetation displays the lowest RMSE value with the mean of all bands, it's important to note that this value is still lower than that derived from single central bands. Conversely, urban areas consistently exhibit higher RMSE values despite lower RMSE values. For GNDVI, the proposed methodology consistently yields lower RMSE values across various land cover classes compared to considering the mean of all bands. It also generally demonstrates lower RMSE values compared to those derived from single central bands, indicating its effectiveness in calculating GNDVI values. Similarly, for NDBI, the proposed methodology consistently yields slightly lower RMSE values across various land cover classes compared to considering the mean of all bands and single central bands. Notably, in the Water class, the proposed methodology’s RMSE value is notably lower than the other approaches. Overall, while differences in RMSE values exist across different land cover classes, there is no significant difference between the proposed methodology and the other approaches. In addition to the proposed methodology, single central bands were considered to assess performance.

The proposed methodology demonstrates superior accuracy and reliability in deriving spectral indices (NDWI, NDRE, GNDVI, NDBI) across diverse land cover types by yielding consistently lower RMSE values compared to the mean of all bands and single central bands. It excels in earth science applications including water resource management, vegetation health assessment, and urban monitoring, with notable improvements in noise reduction and index stability. Parallel processing enhances efficiency, reducing computation time by up to 62%, making the approach scalable for large datasets. The methodology’s robustness, transferability, and broad applicability make it a valuable tool for advanced hyperspectral data analysis and environmental monitoring. Also the proposed methodology outperformed both the mean of all AVIRIS bands and single central bands, providing superior results. Parallel processing significantly reduced execution time, especially for AVIRIS data extraction and index computation. Optimized coding prevented redundant data extraction, further enhancing efficiency. Utilizing 25 cores, optimal performance was achieved with 30%–50% of the cores, while higher usage impacted execution time. Longer computation times for NDBI were due to the greater number of indices involved. Overall, the approach ensured accurate results with improved computational efficiency.