Crop type mapping is crucial for agricultural land cover monitoring and decision-making. State-of-the-art methods developed using recent Sentinel satellite data have already demonstrated their ability to accurately map crop types. However, crop type mapping for the pre-Sentinel era remains challenging due to the limited availability of higher spatial- and temporal-resolution data. This study addresses this knowledge gap by leveraging variable-length Landsat satellite time-series (L-SITS) data in combination with a self-supervised learning model, SITS-BERT, for crop type mapping. This case study, conducted in two German districts, demonstrates the potential of mapping two different crop type levels (CTL1 - 5 and CTL2 - 9 classes) in the pre-Sentinel era. The SITS-BERT model, pre-trained on unlabelled L-SITS data, was fine-tuned on single-year and 3-year datasets and evaluated using past and future years’ data, compared with the model’s training data. The SITS-BERT model achieved overall accuracies of 0.78–0.83 and 0.64–0.76 for CTL1 and CTL2, respectively, with fine-tuning on single-year data. The model fine-tuned with 3 years achieved higher accuracies (0.81–0.85 and 0.72–0.78). The results showed that the SITS-BERT model finetuned with single-year data outperforms the baseline random forest model trained on single-year fixed-length L-SITS data. The study highlighted that, with this approach, limited number of available SITS observations can still be useful. The findings of this study demonstrated the potential of the SITS-BERT model with L-SITS data for crop-type mapping in the pre-Sentinel era, contributing to a more comprehensive understanding of agricultural land cover dynamics and to the evaluation of agricultural policy impacts.
crop mappingdeep learningsatellite image time-series (SITS)SITS-BERTtransformersBundesministerium für Forschung, Technologie und Raumfahrt10.13039/501100002347031B1129AThe author(s) declared that financial support was received for this work and/or its publication. This research was funded by the German Federal Ministry of Research, Technology and Space (BMFTR) within the framework of the SYMOBIO project (grant number 031B1129A).section-at-acceptanceRemote Sensing Time Series AnalysisIntroduction
Information on agricultural land cover (e.g., crop types, their extent and location) and its changes over time is crucial for well-informed policy making. Particularly, decision-makers designing, implementing and evaluating food security, agriculture, biodiversity, and climate change policies benefit from high quality spatial and temporal agricultural land cover data (Karthikeyan et al., 2020). Monitoring of policy impacts is crucial to avoid long-term negative effects (e.g., unsustainable practices) and, if necessary, to take appropriate corrective actions.
Earth observation satellite image time-series (SITS) data have been successfully applied in research and practice to detect distinct crop growth patterns and stages, and their spatial distribution. This approach helps to distinguish crop extent at different spatial scales, and crop types at various stages of their phenological cycle (Zeng et al., 2020). Furthermore, the increasing availability of high spatial and temporal resolution remote sensing (RS) SITS (e.g., from Sentinel or harmonised Sentinel and Landsat satellites), newly developed machine/deep learning models and hardware/software development to compute large geospatial data improved the potential of monitoring agricultural land cover in the present and past (Knorn et al., 2009; Ofori-Ampofo et al., 2021; Asam et al., 2022; Nyborg et al., 2022). The performance of currently used models strongly depends on the availability of high spatial and temporal resolution SITS data. Most state-of-the-art models employ SITS data from Sentinel 1 and 2, which have been available since 2014. However, to assess long-term policy impacts before 2014, the only available option is to use SITS based on Landsat satellite data. Landsat provides data from 1970 to the present, enabling researchers to access more than 50 years of historical satellite data. However, compared to Sentinel 2, Landsat data has lower spatial resolution (30 m vs. 10 m) and a lower temporal resolution (16 days instead of 5–10 days for Sentinel).
Despite these limitations, researchers have demonstrated that models developed using Landsat SITS (L-SITS) data can map agricultural land cover (Kyere et al., 2019; Wijesingha et al., 2024). A random forest (RF) machine learning model trained on L-SITS data achieved 71% overall accuracy in mapping four crop types (grasslands, maize, summer crops, and winter crops) in a region of Germany (Kyere et al., 2019). Similarly, the same four crop types were mapped using L-SITS data and the convolutional neural network (CNN) model, achieving 90% overall accuracy (Wijesingha et al., 2024). Both studies used fixed time point observations as model inputs, which might have contributed to reduced accuracy.
Specific fixed time points (e.g., weekly, monthly) are mainly used because SITS data are limited throughout the entire crop growth period, e.g., due to the lack of cloud-free images at the required time intervals. However, most machine models (e.g., RF, CNN) can only handle fixed-length SITS. To use varying-length SITS in the abovementioned models, padding or truncation should be employed, which could alter the original data or remove necessary information (Lee and Shin, 2025). Alternatively, recent advanced deep learning models with transformers can be applied. These models can use variable-length SITS, avoiding the need for complex fixed-length SITS processing.
Transformers are a type of deep learning model, whichprimarily uses self-attention or scaled dot-product attention mechanisms, and is therefore able to handle sequences of variable lengths (Vaswani et al., 2017). Transformers were initially introduced for neural machine translation in natural language processing. However, nowadays, due to their multiple advantages, transformers are also employed to solve other modelling problems in computer vision or time-series data processing (Wen et al., 2022). For example, transformers have been used to solve time-series forecast and SITS classification problems in previously published studies (Voelsen et al., 2024). Further (Rußwurm and Körner, 2020), introduced a transformer-based approach that uses Sentinel SITS data and advanced temporal encoding to classify fourteen crop types, including fallow land. Additionally, harmonised Landsat and Sentinel 30 m SITS were also employed with transformer-based models to detect land surface phenology (Tran et al., 2025) and to map the in-season 37 crop type (Zhang et al., 2025). However, to the best of the authors’ knowledge, no applications of transformer-based models that exclusively use L-SITS for agricultural land cover mapping have been reported.
Labelled data availability is another common challenge in such models. Since deep learning models with transformers are commonly supervised models, to obtain good results, they usually require large amounts of data (Persello et al., 2022). Self-supervised learning (SSL) is another recently emerged method that can learn from unlabelled data and therefore successfully tackle the issue of scarce training data (Chaves et al., 2020; Miller et al., 2024). The standard SSL process involves two main stages: pretraining and fine-tuning. During pretraining, models acquire initial representations by solving a predefined pretext task. In the fine-tuning stage, these representations are refined for a specific downstream task through supervised training on task-related data. This approach allows deep learning models to transfer general knowledge from large-scale, unlabelled data to tasks with limited labelled data, thereby improving their generalisation ability and reducing the risk of overfitting. The successful application of SSL methods with transformers to Sentinel SITS data has already been reported by (Yuan and Lin, 2021; Xu et al., 2024).
Despite promising results, the SITS-based agricultural land cover mapping still has limitations in terms of model transferability across different temporal (e.g., years) and spatial (e.g., other regions) domains, as well as across spatial-temporal domains. The recent study by the authors assessed changes in model performance during transfer across different temporal and spatio-temporal domains (Wijesingha et al., 2024). L-SITS data for mapping of four crop types (grassland, maize, summer crops and winter crops) were used. Results showed that model performance decreased when the trained model was transferred to another temporal domain, and the performance was further reduced during spatial and spatio-temporal transfer. To overcome this problem, multi-year or multi-location data can be used for model training. It enables the model to understand multiple data distributions. In addition, a recent study showed that a transformer-based deep learning model with an advanced temporal encoding method could accurately transfer models trained with single-year data to another year (i.e., temporal transferability) (Pham et al., 2024). In that study, harmonised Landsat and Sentinel 2 SITS were used. The study employed high density SITS using currently available satellite data. This approach, however, does not solve the identified issues of L-SITS data models.
In light of these challenges, this study addresses the following research question: Can L-SITS data and recently developed transformer-based models using SSL methods effectively map crop types, thereby contributing to agricultural land cover mapping efforts in pre-Sentinel era periods (before 2014)? To answer this question, this work aims to;
Develop a crop type mapping SSL model with L-SITS data
Compare model performance using variable-length L-SITS data versus fixed-length SITS data
Investigate how the temporal transferability of these models changes by using different training data (single year versus multi-year training data)
Materials and methodsStudy area
This study was conducted in two districts of the Weser-Ems region in the federal state of Lower Saxony in Germany (see Figure 1). The two selected districts are: a) Ammerland (DE496) and b) Vechta (DE49F). The Ammerland district is located in the northwestern part of the Weser-Ems region and covers an area of approximately 73,064 km2. In 2019, the total agricultural area in Ammerland was 41,593 km2, which accounts for 57% of the district’s total area. More than 55% of the agricultural area in this district is covered by grassland, primarily used as forage for dairy cattle. The Vechta district is located in the western part of the Weser-Ems region and covers an area of 81,422 km2. Similar to Ammerland, about 59% of Vechta’s area is dedicated to agriculture. However, in Vechta, the share of grassland area was lower (11%) than in Ammerland. The most dominant crop types in Vechta were maize and winter cereals.
(a) Location of study area in Germany with German federal state boundaries and Corine land cover classes for the year 2018 (CLC Service, 2023) for (b) Ammerland and (c) Vechta districts.
Three-panel figure showing land cover in Germany. Panel a maps Germany’s major cities and highlights two study areas in red. Panels b and c display detailed land cover categories for these study areas, using colors to indicate features such as sealed areas, forest cover, vegetation, and water. A color legend clarifies each land cover type.DataCrop field data
The polygons of the crop field boundary and the crop type for each field were obtained from integrated administration and control system (IACS) data from the Ministry of Food, Agriculture and Consumer Protection of Lower Saxony between 2010 and 2018 for the whole Weser-Ems region. Due to heterogeneous crop types data (more than 200 crop types were recorded), this study grouped the types of crops into two different levels: a) crop type level 1 (CTL1) - basic types, and b) crop type level 2 (CTL2) - advanced level. In CTL1, all crop types were categorised into four major categories: permanent grassland, summer crops, winter crops, and others. Nine crop types were considered in the crop type level 2 (CTL2). The summary of the crop type levels categorisation is tabulated in Table 1.
Crop type classification for both crop type levels (CTL1 and CTL2).
Crop type level 1 (CTL1)
Crop type level 2 (CTL2)
C1 – Permanent grassland
C11 – Permanent grassland (pastures, meadows)
C2 – Summer crop
C21 – Maize (grain, silage)
C22 – Summer cereal (barley, wheat)
C3 – Winter crop
C31 – Winter wheat
C32 – Winter cereal (barley, rye)
C4 – Other
C41 – Oil seed (rapeseed, sunflower)
C42 – Permanent crop (orchard)
C43 – Root crop/vegetables (carrot, potato)
C44 – Mixed (field fodder, quiescence)
Landsat satellite image time-series (L-SITS)
L-SITS data (U.S. Geological Survey, 2025) from 2010, 2011 and 2013 to 2018 for the two districts were acquired and pre-processed using the Google Earth Engine’s Python API (Gorelick et al., 2017; Wu, 2020; Montero, 2021). Data for 2012 could not be obtained due to extensive cloud cover in the satellite images, and 2012 was excluded from further assessment. Each annual L-SITS contained all available images (from Landsat 5, 7, and 8 satellites) between March (01.03) and October (31.10). Even though there are differences between spectral responses for each corresponding band from three Landsat satellites, the Collection-2 Level-2 surface reflectance data were downloaded which systematically corrected to minimise these variations (Landsat images courtesy of the U.S. Geological Survey). Following the methodologies by (Wijesingha et al., 2024), from each image data (after cloud masking and scaling) of six spectral bands (blue, green, red, near-infrared1, shortwave-infrared1, and B7:shortwave-infrared2) and four vegetation indices (VIs) (normalised vegetation index–NDVI, enhanced vegetation index–EVI, optimised soil-adjusted vegetation index–OSAVI, and normalised difference moisture index–NDMI) were considered and extracted (equations for Vis are mentioned in the Supplementary Material). The size of the crop fields was calculated based on the crop field boundaries. Crop fields with a size less than 0.9 ha (less than 10 Landsat pixels) were excluded from the study. The mean band reflectance or VI value for each crop field polygon was extracted. For each crop field polygon, the extracted L-SITS data included a time-series of all available time points, Landsat reflectance bands, VI data, and the image acquisition date.
ModelsSelf-supervised learning (SSL) model
This study employed the SSL deep learning model previously developed by (Yuan and Lin, 2021), which was already utilised for land use, land cover, and crop type mapping using Sentinel SITS. In this model, variable-length SITS data with its time point are inserted as a one-dimensional array. First, the model was pre-trained with available SITS data without using any labels. Then, the pre-trained model was finetuned with available labels. During pre-training, the model learned specific spectral-temporal features of the SITS data, which enabled it to perform downstream tasks accurately with few labels. The model was an extension of bidirectional encoder representations from transformers (BERT), a remarkable SSL model in natural language processing (NLP). The modified BERT model to classify the SITS data was referred to as the SITS-BERT model (Yuan and Lin, 2021). The whole network consisted of two parts: a) an observation embedding layer and b) a standard transformer encoder. The observation embedding layer contained observations with the corresponding time encoded using positional encoding. The time information was provided as day of the year (DoY) to facilitate model transferability across multiple years. The transformer encoder comprises various stacked transformer blocks. A single transformer block consisted of a multi-head attention layer and a position-wise fully connected feed-forward network. More details about SITS-BERT can be found in the original article by (Yuan and Lin, 2021).
Pre-training of the SITS-BERT model was designed to do a predesigned pretext task: “some of the input observations are randomly chosen and added with noise, and then the model is forced to predict” (Yuan and Lin, 2021). The model can learn temporal connections between observations by solving this pretext task. According to the authors of the original SITS-BERT publication, the model was pre-trained to reconstruct the original, uncorrupted values of perturbed data using SITS, leveraging contextual information from the remaining sequence. During pre-training, the model optimisation objective was the mean-squared error between the predicted value of the corrupted data and its corresponding actual value. The SITS-BERT model in this study was pre-trained using only variable-length L-SITS data (6 bands and 4 VIs) and with respective DoY from 5 years (2011, 2013, 2014, 2016, and 2017). The model pre-training was done according to the given configuration (max length = 64, number of features = 10, validation rate = 3%, batch size = 256, number of attention heads = 8, learning rate = 0.0001) (Yuan and Lin, 2021). After pretraining, the model was fine-tuned for the time-series classification task—a process commonly referred to as fine-tuning. To do so, an additional output layer was added to the model. The original model was imported from the https://github.com/linlei1214/SITS-BERT.
Random forest (RF) model
The RF model served as the baseline model in this study to compare the performance of the SITS-BERT model. The RF models have been widely employed for crop type classification using SITS data and evaluated for their spatial and temporal transferability (Orynbaikyzy et al., 2022; Wijesingha et al., 2024). However, as a shallow/traditional machine learning model, the RF model requires fixed-length time-series data as input. Following the methods explained by (Wijesingha et al., 2024), the variable-length L-SITS data were converted to the fixed-length time-series by calculating median values for each bi-month (four observations per time-series–March-April, May-June, July-August, and September-October). Each L-SITS consisted of 40 observations (10 VIs or spectral bands × 4 time points). The mean NDVI bi-monthly time-series for CTL2 crops are shown in Supplementary Appendix Figure A1 of the Supplementary Material as an example.
Model experiments
This study conducted several experiments to understand the influence of the amount of training data and the temporal transferability of the trained model over various years. Considering the training data, two experiment cases were carried out: a) Case I–models were trained using only single-year L-SITS data (2015 data), and b) Case II–models were trained with multi-year L-SITS data (2014, 2015, and 2016). Regarding the trained models’ temporal transferability, two transferability scenarios were evaluated: a) Scenario I–trained models were tested with the L-SITS data from 2010 to 2011, and b) Scenario II–trained models were tested with the L-SITS data from 2017 to 2018. In Scenario I, the model transferability over the past years was tested, and in Scenario II, the model transferability for future years was checked.
The SITS-BERT and baseline RF models were evaluated for all defined experiment cases and model transferability scenarios. However, due to the rebuilding of fixed-length L-SITS data from variable-length L-SITS data, the number of data points and time-series observations used in the model and test data differed between the RF and SITS-BERT models (Table 2), as data for some fixed time points were unavailable.
The size of training and testing data sets (number of data points) in two different models.
Model
Training data – Case I
Training data – Case II
Test data – Scenario I
Test data – Scenario II
SITS-BERT
28,176
84,139
57,458
54,923
RF
24,076
79,799
56,610
52,217
Each experiment case (Case I and II) with model transferability scenarios (Scenario I and II) was evaluated by computing a confusion matrix using predicted and actual values for both class levels (CTL1 and CTL2). Based on the confusion matrix, four model performance evaluation metrics (Table 3) were derived: OA- overall accuracy (Equation 1), Pr - precision (Equation 2), Rc - recall (Equation 3, and F1 - F1-score (Equation 4). The Pr, Rc, and F1 were calculated at the class level. Due to uneven class distribution, each class-level metric (Pr, Rc, and F1) was summarised by calculating a weighted average, with the number of samples per class as the weight. Because the test data had different numbers of observations across the two models, the comparison was based on a similar number of observations from the RF model. That means a model comparison was based on 56,610 observations in Scenario I and 52,217 observations in Scenario II. However, to get an indication of the SITS-BERT model transferability, model performance metrics for the remaining observations were also computed. To do this, SITS-BERT model performances were calculated based on 848 samples and 2,706 samples for Scenarios I and II, respectively. Finally, to understand the SITS-BERT model performance behaviour with varying numbers of time-series observations, the OA values were summarised for each scenario based on the number of time-series observations per sample.
Equations and descriptions of the evaluation metrics. (Where TPi is the number of true positives, FPi is the number of false positives, FNi is the number of false negatives for the class i respectively, and n is the number of classes).
Metric
Equation
Description
Overall accuracy (OA)
OA=∑I=1nTPiTotalSamples (1)
The proportion of correctly classified samples among all tested samples
Precision (Pr)
Pri=TPiTPi+FPi (2)
In each class, the proportion of predicted positives that were positive
Recall (Rc)
Rci=TPiTPi+FNi (3)
In each class, the proportion of actual positives that were correctly predicted
F1 score (F1)
F1i=2×Pr×RcPr+Rc (4)
The harmonic mean of precision and recall
To understand the variations in SITS-BERT model predictions with respect to the number of L-SITS observations, the percentage of accurately predicted data points was calculated for each observation group per CTL type and scenario. The SITS-BERT model experimental results notebooks are at https://github.com/jmatics/L-SITS-BERT/tree/master/LSITS.
ResultsCrop type distribution
The median number of crop fields per year from the two districts was 27,482. The lowest number of crop fields was in 2015 (24,076), and the highest was in 2010 (28,939). However, the distribution of crop classes varied between the two regions, whereas the frequency of crop types within the same region over the years did not show substantial variation (Figure 2). About 50% of the crop fields in Ammerland were permanent grasslands (C11). Compared to Ammerland, the share of permanent grassland fields in Vectha was only 13%. Maize (C21) was the most dominant crop type in Vechta, at about 40%, and in Ammerland at 28%. About 13% of crop fields in Vechta were winter wheat (C31), and another 20% of fields were other winter cereals (C32). However, the proportion of winter wheat (C31) and winter cereals (C32) in Ammerland was 1% and 5%, respectively. The share of oilseed crop fields (C41) was 0.4% and 1.3% in Ammerland and Vechta, respectively. About 4% of crop fields in Ammerland were permanent crops (C42), and root crops and vegetables (C43) - in Vectha. In both districts, the crop type group “mixed” (C44) showed varying proportions of crop fields across years.
Crop type distribution in two districts between 2010 and 2018 (except 2012).
Stacked bar chart comparing yearly ratios of different crop parcel types from 2010 to 2018 in Ammerland and Vechta regions, with maize and permanent grassland dominant in Ammerland, and maize, winter cereals, and wheat more prominent in Vechta.SITS-BERT model pretraining
The SITS-BERT model was pre-trained on all L-SITS and DoY data from 5 years without crop type label data. The model was pre-trained for 50 epochs. The best model was obtained at the 49th epoch and was used for the downstream tasks. The mean square error loss for the best model training was 0.01639; for the validation, it was 0.01356.
Variable-length Landsat SITSTraining data (Case I and Case II)
The pretrained SITS-BERT models were finetuned under two Case experiments. Under Case I, the models were trained using single-year data, whereas Case II employed data from three consecutive years (2014, 2015, and 2016). The training data from Case I contained 28,176 observations, and they had variable-length data with five to fifteen observations (Figure 3). Most of the data points (about 31%) contained 9 observations each. The data with 10 and 8 observations were 24% and 19%, respectively. The minimum number of data points was either 5 or 15 (0.1%). Compared to Case I, 3 years of data from Case II contained from five to 25 observations (Figure 3). The total number of training data in Case II was 84,139, with 28,278 from 2014, 28,176 from 2015, to 27,685 from 2016. Of all the training data, 14% each contained 10 and 11 observations. Notably, the data from 2015 to 2016 only had observations between 5 and 15. Conversely, the 2014 data consisted of observations ranging from 10 to 25.
The distribution of the number of Landsat observations in the variable-length satellite time-series for the model fine-tuning cases: (a) Case I - using a single year (2015) data only, and (b) Case II - using 3 years (2014, 2015, and 2016) data.
Two side-by-side bar charts compare training data distributions. Chart A for Case I shows a unimodal distribution peaking at around ten observations, while chart B for Case II displays a bimodal distribution with peaks near twelve and twenty observations. Both charts label the x-axis as number of observations and the y-axis as frequency.Test data (Scenario I and Scenario II)
Similar to the training data distribution under Cases I and II, the test data distribution in Scenarios I and II also differed (Figure 4). In Scenario I, the SITS-BERT model was evaluated using data from 2010 to 2011, comprising 57,458 data points. These data ranged from 6 to 16 observations. Data points with 13 observations were the most frequent, accounting for approximately 25% (8,253 in 2010 and 5,895 in 2011). The 2017 and 2018 data were used in Scenario II for model evaluation, comprising 54,923 data points. Those data points contained observations ranging from three to 24. However, the dataset had 11 and 18 observations, which were the most common, comprising 22% of the data points (11% each from the 11 and 18 observations).
The distribution of the number of Landsat observations in the variable-length satellite time-series for testing (a) Scenario I (2010 and 2011) and (b) Scenario II (2017 and 2018).
Bar chart comparing two scenarios labeled Scenario I and Scenario II under the heading Test Data. Scenario I shows a single-peaked distribution centered near twelve, while Scenario II displays a bimodal distribution with peaks near eleven and nineteen. Both plots use frequency as the y-axis and number of observations as the x-axis.Models trained with single-year data (Case I)
During the Case I experiments, both pretrained SITS-BERT and the RF models were trained on single-year data (2015), and the models were transferred to predict values (both CTL1 and CTL2) in two scenarios (Scenario I and II).
In both Scenarios, the SITS-BERT model under Case I with CTL1 outperformed the RF model in all evaluation metrics (Table 4). The SITS-BERT model’s OA and weighted F1 score in Scenario I (0.833 and 0.826) were better than in Scenario II (0.784 and 0.785). A similar pattern of accuracy metrics was observed in the RF model, with slightly lower values: weighted F1 was 0.764 (OA = 0.752) and 0.740 (OA = 0.724) for Scenarios I and II, respectively. The SITS-BERT model performance was also calculated for the remaining observations, as the two models were compared with a similar number of observations. Based on the remaining observations, the SITS-BERT model achieved OA of 0.742 and weighted F1 of 0.718 for CTL1 in Scenario I. Under Scenario II, the remaining observations with the SITS-BERT model achieved an OA of 0.730 and a weighted F1 of 0.724.
Model performance summary for Case I (fine-tuned with single year data). Bold text highlights the best performance for each Scenario and crop type level.
Class levels
Scenario levels
Model
OA
Weighted precision
Weighted recall
Weighted F1
CTL1
Scenario I
SITS-BERT
0.833
0.825
0.833
0.826
RF
0.752
0.804
0.752
0.764
SITS-BERT (remaining)
0.742
0.724
0.742
0.718
Scenario II
SITS-BERT
0.784
0.810
0.784
0.785
RF
0.724
0.802
0.724
0.740
SITS-BERT (remaining)
0.730
0.750
0.730
0.724
CTL2
Scenario I
SITS-BERT
0.759
0.753
0.759
0.739
RF
0.702
0.757
0.702
0.704
SITS-BERT (remaining)
0.693
0.700
0.693
0.658
Scenario II
SITS-BERT
0.644
0.715
0.644
0.640
RF
0.652
0.738
0.652
0.673
SITS-BERT (remaining)
0.670
0.670
0.670
0.652
When the models were trained to predict more crop types (CTL2), both models showed slightly lower performance. Under Scenario I, the SITS-BERT model clearly outperformed the RF model, with a weighted F1 score of 0.739 (OA = 0.759) compared to 0.704 (OA = 0.702) for the RF model. Compared to the RF model, the SITS-BERT model yielded better evaluation metric values under Scenario II. However, as with the CTL1 prediction models, the performance of both models decreased slightly in Scenario II compared to Scenario I. The OA values for the remaining observations with the SITS-BERT model for CTL2 were 0.693 (F1 = 0.658) and 0.670 (F1 = 0.652), respectively, for Scenarios I and II.
Models trained with multi-year data (Case II)
During the Case II experiments, both SITS-BERT and the RF models were trained using multi-year (2014, 2015, and 2016) data. The models were then transferred to predict values (both CTL1 and CTL2) in two scenarios (Scenario I and II).
Compared to the models’ performances in Case I, both models showed improved results in Case II (Table 5). However, in CTL1 prediction under Scenario I, the SITS-BERT model indicated higher OA (0.855 vs. 0.844) and weighted F1 (0.848 vs. 0.841), whereas the RF models achieved better OA (0.835 vs. 0.807) and weighted F1 (0.838 vs. 0.812) in Scenario II. The OA values from the SITS-BERT model for the remaining observations were 0.756 (F1 = 0.737) and 0.821 (F1 = 0.820) for Scenarios I and II, respectively.
Model performance summary for Case II (fine-tuned with 3 years data). Bold text highlights the best performance for each Scenario and crop type level.
Class levels
Scenario levels
Model
OA
Weighted precision
Weighted recall
Weighted F1
CTL1
Scenario I
SITS-BERT
0.855
0.846
0.855
0.848
RF
0.844
0.841
0.844
0.841
SITS-BERT (remaining)
0.756
0.739
0.756
0.737
Scenario II
SITS-BERT
0.807
0.844
0.807
0.812
RF
0.835
0.845
0.835
0.838
SITS-BERT (remaining)
0.821
0.828
0.821
0.820
CTL2
Scenario I
SITS-BERT
0.771
0.805
0.771
0.762
RF
0.789
0.789
0.789
0.769
SITS-BERT (remaining)
0.708
0.731
0.708
0.689
Scenario II
SITS-BERT
0.725
0.816
0.725
0.731
RF
0.771
0.773
0.771
0.766
SITS-BERT (remaining)
0.749
0.769
0.749
0.743
The model prediction for the CTL2 in Case II indicated that the RF models (OA = 0.789 and 0.771 for the two Scenarios) were better than the SITS-BERT model (OA = 0.771 and 0.725 for the two Scenarios). Based on the remaining observations, the SITS-BERT model obtained an OA of 0.708 and a weighted F1 of 0.689 in Scenario I. Under Scenario II, the remaining observation predictions with the SITS-BERT model achieved an OA of 0.749 and a weighted F1 score of 0.743. The confusion matrices for each model are summarised in the Supplementary Material (Supplementary Appendix Figures A1–A8).
SITS-BERT model accuracy per number of observations
The percentage of prediction accuracy for CTL1 and CTL2 under Scenarios I and II in Case I is provided in Figure 5. In CTL1 Scenario I, the data with six observations had the lowest percentage (67%), while all other data with more than six observations obtained more than 80% accuracy. For Scenario II, the data with three, four, and five observations obtained 60%, 45%, and 70% accuracy, respectively. However, the data with more than 7 observations achieved more than 70% accuracy. Similar to the overall model performance, CTL2 in Scenarios I and II showed lower accuracy values for varying observations. For Scenario II, the data with six observations again yielded the lowest accuracy (67%) for CTL2, while all other data, ranging from seven to sixteen observations, achieved accuracies greater than 75%, except for the data with ten observations (74%). Considering Scenario II, the data with 15 observations provided the lowest accuracy (49%). Further, the data with four, five, sixteen, seventeen, eighteen, nineteen, and twenty observations obtained accuracies below 60%. At both crop type levels (CTL1 and CTL2) and across Scenarios, the highest accuracy was achieved with the data set with the largest number of observations.
Impact of the number of Landsat observations in the variable-length time-series on the accuracy in both Scenarios and fine-tuning Cases.
Bar chart with four panels comparing the percentage of accurately predicted data versus number of Landsat observations for CTL1 and CTL2, under Scenario I and Scenario II, with values generally higher in Scenario I.Discussion
In this study, the application of recently developed transformer-based deep learning models with SSL for crop type mapping using L-SITS data was explored. The results of this study showed that the L-SITS data, combined with the SITS-BERT model, could accurately predict both simple (CLT1) and advanced (CLT2) crop types. Furthermore, the findings demonstrated that fine-tuning of the pre-trained SITS-BERT model with 3 years data provided better results compared to using single-year data.
L-SITS data
This study explored the application of solely L-SITS data from 2010 to 2018 for crop type classification. During that period, Landsat 5 data were available until 2011, and Landsat 7 data were available from 1999 (till 2024) (U.S. Geological Survey, 2025). The Landsat 8 data have been available since 2013 (U.S. Geological Survey, 2025). As a result, in 2012, Landsat 7 data were the only available source, leading to limited cloud-free data availability and preventing the use of 2012 data in the study area. In all other years of the assessment, at least two Landsat satellite datasets were available to generate L-SITS data for each year. This ensured having the maximum temporal observations for the L-SITS data. Compared to the current variable-length SITS applications using Sentinel 1, 2, or a combination of Landsat and Sentinel 2 (Chaves et al., 2020). The L-SITS data explored in this study had a lower temporal resolution. Although L-SITS data have lower temporal resolution, variable-length L-SITS data showed potential for crop type mapping in Germany, which would be beneficial for areas or periods without Sentinel data. When it comes to crop type mapping before 2010, with the suggested model, SITS data from at least two Landsat satellites (either Landsat 4 and 5 or Landsat 5 and 7) can be used. Therefore, it enables crop type mapping going back to the 1980s.
Moreover, this study demonstrated that over 245 days (from March to October), a minimum of five and a maximum of 25 temporal observations could be obtained for a variable-length L-SITS in different regions within the study area. For example, a single L-SITS had five observations at 81, 105, 217, 232, and 272 DoY (March 22, April 15, August 5, August 20, and September 29). However, based on this example, we can see that even with fewer observations, the L-SITS data have well-distributed time-series within the main crop-growth season. This outcome is helpful for tasks that require differentiation of crop types based on their phenological patterns. Moreover, with very few observations, the prediction accuracy did not show a substantial reduction (Figure 5), further supporting the application of the SITS-BERT model for crop type mapping using L-SITS data. In contrast, the results for Scenario II indicated that L-SITS with 15 observations produced (Figure 5) the lowest correctly classified percentage (for CTL2). Further examination of the L-SITS dataset with 15 observations revealed that one incorrectly predicted example included five observations from July, with no observations at the beginning or end of the time-series and no observations from June. Such behaviour highlights a potential limitation of the model, indicating the need for further fine-tuning and the incorporation of novel adaptations, such as temporal encoding (Pham et al., 2024), to better capture temporal dependencies within the time-series.
Application of SITS-BERT with L-SITS data
This study is one of the first attempts to explore the SSL method with variable-length L-SITS data for crop type mapping. Considering this, the SITS-BERT model was pre-trained using 5 years of variable-length L-SITS data without labels. As mentioned before, the SITS-BERT model was introduced with the application of the Sentinel-2 SITS data by (Yuan and Lin, 2021). This study attempted to reimplement a similar model using L-SITS data. Compared to the original version with Sentinel data, this study also incorporated a similar pretext task during model pre-training to leverage the advantage of extensive unlabelled data.
The pre-trained SITS-BERT model was then fine-tuned using two different training datasets: a single year (Case I) and a multi–year (Case II) dataset. The single-year training dataset consistently showed a minimum accuracy of 0.644 for CTL-2 in future years (Scenario II). Compared to that, the multi-year training dataset achieved a minimum accuracy of 0.725 for the same scenario, representing an approximately 16% increase. Compared with the original work using Sentinel 2 data, these accuracy values were substantially lower (Yuan and Lin, 2021). However, when it comes to CTL2 predicting for the past years (Scenario I), both single-year and multi-year training cases showed similar performance (0.759 and 0.771 of OA). These results were better than those obtained by (Pham et al., 2024) using Landsat 5 and 7 data with temporal encoding to the irregular time-series. This indicated that the application of L-SITS data with the SITS-BERT model could be a potential tool for predicting crop types before the Sentinel era, overcoming challenges encountered by (Pham et al., 2024).
The baseline model used in this study was the RF model trained on fixed-length L-SITS data. The fine-tuned SITS-BERT model with variable-length L-SITS data outperformed the RF model with fixed-length L-SITS when trained on single-year data. This was a significant advantage when there was not enough labelled data to train the model. On the other hand, due to the fixed-length L-SITS, the number of data points used for training and testing the RF model was mainly lower than that of the SITS-BERT model. The increased number of data points improved the performance of the SITS-BERT model. Under Case I, the RF model was trained with 17% fewer observations than the SITS-BERT model. However, in Case II, only 5% fewer observations were used in the RF model, and the differences in model performance were not substantial.
Simple vs. advanced crop type classification
This study examined two different crop types (CTL1 and CTL2) at two distinct levels and evaluated model performance across both levels. The CTL1 contained simple crop type groups, which comprised broader crop categories. The SITS-BERT model with both cases (Case I and Case II) consistently showed higher OA values than the baseline RF model (Tables 4, 5). The winter crops (C3) showed the highest class-level F1 (Case I = 0.909 and Case II = 0.941). These results were better than this study’s baseline RF model results (F1 was 0.877 and 0.926) and findings from (Wijesingha et al. (2024) for winter crops using fixed-length L-SITS data with the RF and the CNN models transferred to another temporal domain (F1 was between 0.83 and 0.90). This also confirmed that broader crop type classification could benefit from variable-length L-SITS and SITS-BERT models.
The complex crop type labels were included in CTL2; thus, both SITS-BERT and baseline RF models underperformed compared to CTL1. Similar to all the results, however, the SITS-BERT model predictions for Scenario I (for years 2010 and 2011) showed higher OA compared to Scenario II (for years 2017 and 2018) (Table 4). However, results reported by (Pham et al., 2024) indicated higher model performances for later years (OA was 86.4 and 90.9 for 2017 and 2018) than for past years (OA was 61.9 and 62.8 for 2010 and 2011) for 14 crop types using temporally encoded SITS data. This again confirms that the application of the SITS-BERT model with variable-length L-SITS data is effective for past crop type mapping tasks, including those with complex crop classes.
The STIS-BERT model’s capability to separate maize (C21) from other summer cereals (C22) was substantially higher, where the maize F1 scores for Scenario I were 0.893 and 0.933 for Cases I and II, respectively. These F1 scores for maize were higher than the results reported by (Pham et al., 2024; Pham et al., 2024), where temporally encoded SITS data were used to predict crop type in the years 2010 (F1 = 0.8) and 2011 (F1 = 0.9) (similar to Scenario I). However, when separating winter wheat (C31) from other winter cereals (C32), models performed poorly in both scenarios. A possible reason for this performance could be the class distribution of the data. In the study regions, the number of fields with maize was significantly higher than that of summer cereals, enabling the model to identify distinct patterns in maize’s SITS data compared to those of summer cereals. On the other hand, the proportion of winter wheat compared to winter cereals was lower (especially in the Ammerland district). A similar pattern of results was reported by (Pham et al., 2024), though the models in this study were trained with similar class observations, yet the winter cereals (wheat, barley, and rye) reported 0.6 as an average F1 score.
The SITS-BERT model predictions for the complex crop categories, such as oil seeds (C41), permanent crops (C42), and root crops and vegetables (C43), clearly indicated that when the model was trained with multiple years, the predictability of these classes increased compared to a model trained with a single year of data. Among these crop classes, C42 showed the best performance (F1 score≥0.5) in both Scenarios (using the Case I and II models), though most of the C42 data was available in the district of Ammerland. The C41 class had an F1 score of more than 0.55 for the models in Case II. However, these class performances were lower than the F1-scores for separate classes of rapeseed and sunflower (>0.6) from the models with temporally encoded SITS data (Pham et al., 2024). The class C43 was a complex class that combined root crops and vegetables. The decision to merge root crops and vegetables into the C43 class was made to increase the number of data points per class to reduce the impact of the class imbalance. However, this grouping led to a decrease in prediction accuracy due to the complexity of phenological and morphological characteristics of both crop types. This outcome suggests that a standard approach is necessary when defining crop classes for SITS data, as simply increasing sample size does not overcome fundamental differences between crop types and the class imbalance problem.
Conclusion
This study evaluated the application of recently developed transformer-based deep learning models with SSL for crop type mapping using variable-length L-SITS for the pre-Sentinel era. The results clearly demonstrated that variable-length L-SITS extracted from available cloud-free data points can still be useful for crop-type or other mapping tasks when used with the SITS-BERT model. Similarly, the pre-trained SITS-BERT model could be finetuned with single-year data (fewer data points) and achieved reasonable accuracies for both simple (CTL1) and advanced (CTL2) crop type mapping. These results were better than those of the baseline RF model, which was trained on fixed-length L-SITS data. This is a significant advantage in cases where there is insufficient labelled data to train the model. Further, fine-tuning with multi-year data also improved model performance. The study also investigated the impact of the number of L-SITS observations on the accuracy of the SITS-BERT model for variable-length L-SITS data. The results showed that even with a small number of observations, prediction accuracy did not decline substantially. The model evaluation also assessed the SITS-BERT model’s predictions for past and future years relative to its training years to evaluate its generalisability across different temporal domains, and this study’s results showed somewhat better performance for past years than for future years. Overall, the use of variable-length L-SITS data with the SITS-BERT model enables the application of L-SITS data for crop-type mapping dating back to the 1980s.
Data availability statement
The data analyzed in this study is subject to the following licenses/restrictions: The Landsat data is available to download through respective sources (e.g., USGS Earth Explorer or Google Earth Engine), but the crop type data (polygons of fields) are restricted to share. Requests to access these datasets should be directed to jayan.wijesingha@uni-kassel.de.
Author contributions
JW: Data curation, Visualization, Software, Formal Analysis, Methodology, Investigation, Validation, Writing – original draft, Conceptualization. IB: Writing – review and editing, Project administration, Data curation, Conceptualization.
Acknowledgements
The authors sincerely thank Prof. Dr. Michael Wachendorf for providing the facilities and support necessary to carry out this research. We would like to thank the Ministry of Agriculture of Lower Saxony, Germany, for providing historical agricultural land use (IACS) data for the training and validation of the models.
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frsen.2026.1782148/full#supplementary-material
ReferencesAsamS.GessnerU.GonzálezR. A.WenzlM.KrieseJ.KuenzerC. (2022). Mapping crop types of Germany by combining temporal statistical metrics of Sentinel-1 and Sentinel-2 time series with LPIS data. Remote Sens. (Basel).14. 10.3390/rs14132981ChavesM. E. D.PicoliM. C. A.SanchesI. D. (2020). Recent applications of landsat 8/OLI and Sentinel-2/MSI for land use and land cover mapping: a systematic review. Remote Sens. (Basel).12. 10.3390/rs12183062CLC Service (2023). CLCplus backbone 2018 (raster 10 m), Europe. 10.2909/cd534ebf-f553-42f0-9ac1-62c1dc36d32cGorelickN.HancherM.DixonM.IlyushchenkoS.ThauD.MooreR. (2017). Google Earth Engine: planetary-scale geospatial analysis for everyone. Remote Sens. Environ.202, 18–27. 10.1016/j.rse.2017.06.031KarthikeyanL.ChawlaI.MishraA. K. (2020). A review of remote sensing applications in agriculture for food security: crop growth and yield, irrigation, and crop losses. J. Hydrol. (Amst).586, 124905. 10.1016/j.jhydrol.2020.124905KnornJ.RabeA.RadeloffV. C.KuemmerleT.KozakJ.HostertP. (2009). Land cover mapping of large areas using chain classification of neighboring landsat satellite images. Remote Sens. Environ.113, 957–964. 10.1016/j.rse.2009.01.010KyereI.AstorT.GraßR.WachendorfM. (2019). Multi-temporal agricultural land-cover mapping using single-year and multi-year models based on landsat imagery and IACS data. Agronomy9, 13–15. 10.3390/agronomy9060309LeeH.ShinD. (2025). Beyond information distortion: imaging variable-length time series data for classification. Sensors25, 621. 10.3390/s2503062139943260MillerL.PelletierC.WebbG. I. (2024). Deep learning for satellite image time-series analysis: a review. IEEE Geosci. Remote Sens. Mag.12, 81–124. 10.1109/MGRS.2024.3393010MonteroD. (2021). Eemont: a python package that extends google Earth engine. J. Open Source Softw.6, 3168. 10.21105/joss.03168NyborgJ.PelletierC.LefèvreS.AssentI. (2022). TimeMatch: unsupervised cross-region adaptation by temporal shift estimation. ISPRS J. Photogrammetry Remote Sens.188, 301–313. 10.1016/j.isprsjprs.2022.04.018Ofori-AmpofoS.PelletierC.LangS. (2021). Crop type mapping from optical and radar time series using attention-based deep learning. Remote Sens. (Basel).13, 1–17. 10.3390/rs13224668OrynbaikyzyA.GessnerU.ConradC. (2022). Spatial transferability of random forest models for crop type classification using Sentinel-1 and Sentinel-2. Remote Sens. (Basel).14, 1493. 10.3390/rs14061493PerselloC.WegnerJ. D.HanschR.TuiaD.GhamisiP.KoevaM. (2022). Deep learning and Earth observation to support the sustainable development goals: current approaches, open challenges, and future opportunities. IEEE Geosci. Remote Sens. Mag.10, 172–200. 10.1109/MGRS.2021.3136100PhamV.-D.TettehG.ThielF.ErasmiS.SchwiederM.FrantzD. (2024). Temporally transferable crop mapping with temporal encoding and deep learning augmentations. Int. J. Appl. Earth Observation Geoinformation129, 103867. 10.1016/j.jag.2024.103867RußwurmM.KörnerM. (2020). Self-attention for raw optical satellite time series classification. ISPRS J. Photogrammetry Remote Sens.169, 421–435. 10.1016/j.isprsjprs.2020.06.006TranK. H.ZhangX.ZhangH. K.ShenY.YeY.LiuY. (2025). A transformer-based model for detecting land surface phenology from the irregular harmonized landsat and Sentinel-2 time series across the United States. Remote Sens. Environ.320, 114656. 10.1016/j.rse.2025.114656U.S. Geological Survey (2025). Landsat missions. Available online at: https://www.usgs.gov/landsat-missions (Accessed October 24, 2025).VaswaniA.ShazeerN.ParmarN.UszkoreitJ.JonesL.GomezA. N. (2017). Attention is all you need. Available online at: http://arxiv.org/abs/1706.03762 (Accessed May 22, 2025).VoelsenM.RottensteinerF.HeipkeC. (2024). Transformer models for land cover classification with satellite image time series. PFG - J. Photogrammetry, Remote Sens. Geoinformation Sci.92, 547–568. 10.1007/s41064-024-00299-7WenQ.ZhouT.ZhangC.ChenW.MaZ.YanJ. (2022). Transformers in time series: a survey. Available online at: http://arxiv.org/abs/2202.07125 (Accessed May 22, 2025).WijesinghaJ.DzeneI.WachendorfM. (2024). Evaluating the spatial–temporal transferability of models for agricultural land cover mapping using landsat archive. ISPRS J. Photogrammetry Remote Sens.213, 72–86. 10.1016/j.isprsjprs.2024.05.020WuQ. (2020). Geemap: a python package for interactive mapping with google Earth engine. J. Open Source Softw.5, 2305. 10.21105/joss.02305XuY.MaY.ZhangZ. (2024). Self-supervised pre-training for large-scale crop mapping using Sentinel-2 time series. ISPRS J. Photogrammetry Remote Sens.207, 312–325. 10.1016/j.isprsjprs.2023.12.005YuanY.LinL. (2021). Self-supervised pretraining of transformers for satellite image time series classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.14, 474–487. 10.1109/JSTARS.2020.3036602ZengL.WardlowB. D.XiangD.HuS.LiD. (2020). A review of vegetation phenological metrics extraction using time-series, multispectral satellite data. Remote Sens. Environ.237, 111511. 10.1016/j.rse.2019.111511ZhangH. K.ShenY.ZhangX.LiJ.YangZ.XuY. (2025). Robust and timely within-season conterminous United States crop type mapping using landsat Sentinel-2 time series and the transformer architecture. Remote Sens. Environ.329, 114950. 10.1016/j.rse.2025.114950
Edited by:Peter Potapov, World Resources Institute, United States
Reviewed by:Ahmad Khan, University of Maryland, College Park, United States
Haijun Li, University of Maryland, College Park, United States