We used 16 years (1999–2014) of corn yield response to N fertilizer rate data from a field experiment conducted in central Iowa, USA (details in Puntel et al., 2016). The experiment was designed to study the effect of five N fertilizer rates (0, 67, 134, 201, and 268 kg N ha⁻¹) on corn yield in continuous corn (CC) and corn following soybean (SC) cropping systems. Application of N was either pre-plant or side-dress; see details in Puntel et al. (2016). Corn grain yield was reported at 15.5% moisture content. The corresponding EONR values per year and rotation were calculated using measured yields (see section Estimation of the Annual Economic Optimum Nitrogen Rate). The weather at the experimental site is humid continental with warm rainy summers with an average annual precipitation of 900 mm and annual temperature of 9°C. Over the 16-year experimental period, crops experienced warm and wet conditions (3 years), cool and wet conditions (3 years), warm and dry conditions (5 years), and cool and dry conditions (5 years; Figure S1a). The soil at the site is a deep fertile loamy (Clarion soil series) with topsoil organic matter of 3.4% and profile plant available water of 250 mm.

The APSIM model is an open-source advanced simulator of agricultural systems that combines several process-based models in a modular design (Holzworth et al., 2014). In this study, we used the recently calibrated version of the model for this site with no additional changes (see Puntel et al., 2016 for detailed calibration information). Model performance is also provided in this study (see yield predictions at maturity, results section). The simulation process was continuous, starting in 1999 and ending in 2014 without annual re-initialization of inputs to capture carryover effects on soil N, water, residue, and soil organic matter dynamics from 1 year to the next. The following APSIM models were used: corn and soybean crop models (Keating et al., 2003), Soil N (soil N and C cycling model; Probert et al., 1998), SWIM (soil water model using the Richard equation and fluctuated water tables; Huth et al., 2012); SURFACEOM (residue model; Probert et al., 1998; Thorburn et al., 2001; Thorp et al., 2005), soil temperature (Campbell, 1985), and the following management rules: planting, harvesting, fertilizer, tillage, and rotations (Keating et al., 2003).

To forecast yields and EONR in each study year (1999–2014) we combined actual weather data up to a specific crop stage and historical weather data thereafter (Figure S2). The following forecast times were considered in this study: planting, 6th leaf (V6), 12th leaf (V12), and silking (R1; Abendroth et al., 2011). We selected these times because farmers in rainfed production regions can make delayed N applications during vegetative and early reproductive stages (R1) without a significant negative effect on yield (Scharf et al., 2002). In this environment, there is not much evidence of yield response to N fertilizer application after corn silking, thus we did not explore yield and N forecasts during grain filling period.

Except for planting time, which only includes observed weather data until the planting date, the rest of the forecast times used observed in-season weather data until the forecast time (Figure S2). Historical daily weather data from 1980 to 2015 (35 years, maximum available for this site) was obtained from the Iowa Environmental Mesonet (2014) to fill the unknown weather. The combination of 35 years of simulations, four forecast times, 16 study years, five N rates, and two crop rotations resulted in more than 22,000 simulations. We calculated the yearly mean and standard deviation of yield and EONR from the 35 simulated values per year and forecasting time similar to Togliatti et al. (2017). The simulated site-mean EONR was calculated by averaging individual annual estimates of EONR and yield at EONR (YEONR) for each rotation, and then calculating the associated standard deviation (SD; across years mean).

Once the simulations were completed, we grouped the data to represent the following five weather scenarios and quantify the impact on yield and EONR forecasts:

Use simulated yields from calendar years with similar annual precipitation and temperature patterns as the year of interest (e.g., 2010 and 1984). These groups of years can be found in Figure S1a.

Scenarios I and II were investigated because previous findings showed that the used of analogous weather years instead of the full weather record could increase accuracy of yield predictions and thus, derived EONR predictions (Hammer et al., 2001; Hansen et al., 2004). Scenarios III to V were investigated to quantify the impact of using limited amount of weather data because long term (35-years) weather data are not available in every location (Hansen et al., 2004; Grassini et al., 2015).

Yield prediction at different forecast times did not significantly change during the growing season compared with yield prediction at maturity (Figure 2). On average, the RRMSE decreased by 1.2% from planting to maturity. Across five N rates, two crop rotations, and 16 years APSIM explained 77% of the observed variability of corn yield (Figure 2) when predictions were made early in the season (planting and V6 stage), about 79% of the variability during mid-season (V12 and R1 stages), and 81% of the variability at the end of the season (R6 stage). The simulated mean yield at the four forecasting times (with unknown weather after the corresponding growth stage) was similar to the final yield prediction at R6 simulated with known weather (Figure 2).

At early stages (planting to V12), APSIM was able to explain ~40% of the variability in observed YEONR and 57% of the variability at maturity (Figure 2). The APSIM model was better at explaining the observed variability in the YEONR during warm weather years than cold weather years across the different forecasting times (R² ~ 0.8 vs. 0.5, respectively; Table S3). Absolute differences between observed and predicted YEONR were 1.55 and 0.63 Mg ha⁻¹ for cold and warm weather years, respectively. Overall model performance improved (R² = 0.6; data not shown) when looking at this relationship while excluding the 2008, 2010, 2012, and 2014 extreme weather condition years in the 16-years of experiments (Figure S2).

Between the two cropping systems, yield predictions were slightly better for SC than CC rotations across all four forecasting times (RRMSE of 12.9 vs. 14.2%, respectively; Table S1). The standard deviation of the mean yield, a measure of the uncertainty around yield predictions, did not decrease from planting to R1 (Figures S3, S4). At maturity, there is no uncertainty (standard deviation = 0) because the actual weather was known (Figures S3, S4). In agreement with yield predictions, the uncertainty around the predicted mean YEONR did not decrease from planting to R1 in a significant number of the years (Figures 3, 4). However, the mean deviation (precision) of yield predictions across all years was 27% lower for R1 than at planting (Figures 3, 4).

In contrast to yield predictions, the EONR predictions were less accurate (Figure 2), and at different forecasting times showed different patterns across rotations and years (Figures 5, 6). In some cases, the EONR prediction was more accurate at planting or V6 than at maturity (e.g., 2004, Figure 5). In other cases, the EONR prediction was more accurate at maturity than early in the season, but the differences in terms of actual value were small (e.g., Figure 5, 2007, the difference between simulated EONR at planting and maturity was 18 kg N ha⁻¹ for CC). Overall, the mean deviation (precision) of EONR predictions across all years was 25% lower for R1 than at planting (Figures 5, 6). The forecasted site-mean EONR and the standard deviation was similar to the observed values, but low for SC and high for CC (Table 1).

At early forecasting times (planting, V6, and V12), the absolute average differences between observed and predicted EONR for both rotations were lower for warm than for cold weather years (37 vs. 50 kg N ha⁻¹, respectively; Table S3). The model better explained observed variability in the EONR in cold/dry than warm/wet seasons across forecasting times and crop rotations (R² = 0.32).

In terms of uncertainty, the standard deviation of the mean EONR prediction did not show a consistent pattern of decrease during the growing season (Figures 5, 6). Overall, the EONR predictions were more accurate in CC (RRMSE = 25%) than SC (RRMSE = 45%) across forecast times (Figure 2).

The APSIM model predicted EONR with an average error of ±38 kg N ha⁻¹ in 62% of the study cases (n = 31). Prediction error was below the threshold value of ±30 kg N ha⁻¹ in about 50% of the cases.

Using 35 years of weather data as input to the model resulted in marginally lower RRMSE (on average 1.4%) values and therefore better yield predictions across forecast times compared to the use of 5, 10, or 20 years of weather data or weather data that included only years with similar weather patterns (Figure 7A). Similar results were found for the EONR prediction; that is, use of 35-year weather data performed better than all other scenarios (Figure 7B). Across all forecasting times, RRMSE was 3.5% lower when using 35-years of weather data. Among the scenarios compared against the use of 35-years, the use of 20 years of weather data had the lowest RRMSE for yield and EONR at all forecasting times (Figure 7A). The exception was the yield predictions at V12 and silking stage where the RRMSE was the lowest for scenarios with 5, 10, or 20 years of weather data (Figure 7A).

Independent of the weather scenarios, RRMSE values for EONR predictions were four times higher compared to yield predictions. The selection of a specific weather scenario tended to have a greater impact on predictions early in the season than toward the end-of-season (Figure 7). For example, at flowering the RRMSE was on average across weather scenarios 0.8 and 2.3% lower than RRMSE at planting for yield and EORN, respectively.

We use the site-mean EONR and YEONR (see green horizontal line in Figures 3–6) to benchmark the direction and magnitude of error in annual APSIM predictions. Across 16-years in CC and 15-years in SC (n = 31 cases), the observed EONR values were in 11, 14, and 6 cases above, below and at site- mean EONR values respectively. APSIM predictions of EONR at planting time were in 11, 16, and 4 cases, above, below and at the site-mean EONR values respectively (Figures 5, 6). In 19 of 31 (62%) cases, APSIM correctly predicted the direction of annual EONR being above, below, or at average. For those cases, the average error was 38 kg N ha⁻¹ which represents 20 and 25% error base on the average N rate for CC (observed mean = 184 kg N ha⁻¹) and SC (149 kg N ha⁻¹), respectively.

The observed YEONR values were in 14, 11, and 6 cases above, below and at site-mean YEONR values respectively, across rotations and years. APSIM predictions of YEONR at planting time were in 8, 14, and 9 cases, above, below and at the site-mean YEONR values respectively (Figures 3, 4). In 21 of 31 (67%) cases, APSIM correctly predicted at planting time the direction of annual YEONR being above, below, or at average.

In low or high N need years the simulated EONR deviated more from observations than in average N need years (Figures 5, 6; Table 2). In 2 out of 16 years (extreme wet years; 2008 and 2010) the APSIM model greatly underestimated the EONR when > 30 kg N ha⁻¹. In low fertilization need years, APSIM over predicted the N rate on average by 36 and 69 kg N ha⁻¹ in CC and SC rotation, respectively (Table 2). Overall the distribution of the differences between APSIM predicted EONR and observed EONR was skewed to the right (~ + 30 kg N ha⁻¹) for CC and skewed to the left for SC, underestimating the observed EONR (Figure 8).

This study showed that end-of season corn yields and YEONR can be predicted within an acceptable error (RRMSE < 17%) as early as planting time in most of the study years (Figures 2–4 and Figures S3, S4). The 16-years of data used to test our yield forecasts accounted for different weather years (wet, dry, warm, cold, Figure S1a) and management practices, which further increased our confidence that our process-based forecasting approach is robust (He et al., 2017). This means that our science-based yield forecasting approach has the potential to inform corn producers in the USA Midwest that are using the yield-goal approach to predict N-rates by “guessing” end-of-season yields (Arbuckle and Lasley, 2013; Raun et al., 2017; Morris et al., 2018). Further testing of our approach across multiple environments is needed before application. This would minimize expected errors that are often encountered when using a conventional yield approach based on yield averages across years (Raun et al., 2017).

In most cases, the simulated yield at maturity was within the standard deviation of yield prediction at planting (Figures S3, S4). The uncertainty in yield predictions did not substantially decrease from planting to silking even though that the weather uncertainty decreased (Figure S3). We believe that if we had run the forecast at 150°C-days (or 15 calendar days) after silking we would have seen a decrease in the uncertainty as two of the major determinants of corn yield (kernel number and potential kernel weight) is set (Andrade et al., 1999; Borrás and Vitantonio-Mazzini, 2018). Previous studies have shown a decrease in the uncertainty around corn yield predictions at or about 150°C-days after silking (Thornton et al., 1997; Hansen et al., 2004; Togliatti et al., 2017). By R1 stage, about 50% of the weather is still unknown and the weather during grain fill period has substantial effects on yield. This weather uncertainty introduces yield variability in model predictions as different weather variables affects various plant and soil processes (and thus the final product that is yield) in different ways that are also phenologically time dependent (Semenov et al., 1993). For a thorough sensitivity analysis of weather effects on APSIM simulated grain yield for central Iowa, USA we refer to Togliatti et al. (2017).

In two of the years, yield predictions at maturity deviated substantially from the simulated yields at planting (2004 and 2008 in both rotations; Figures 3, 4). In these years, yield predictions changed during the season as a response to extremely low temperature conditions during grain fill compared to the 35-year average temperature (Figures S2A, S5). The model responded to this weather event by increasing the length of grain fill resulting in higher yield predictions at maturity than at early stages. However, in our study the response of the model was not enough to match the observed yields; perhaps use of a newer maize version (Soufizadeh et al., 2018) of the APSIM model may capture these dynamics better. Nevertheless, this example shows that use of crop model offers both predictability and the reasons behind yield predictions (Banger et al., 2017).

The APSIM model predictions at planting time were directionally correct in 62% of the study cases. This means that our N forecasting approach is promising and has future potential to directly and/or indirectly aid N rate decisions by providing a year-to-year opportunity to adjust N rate recommendations early in the season (Figures 5, 6). In terms of prediction accuracy, our modeling approach forecasted annual EONR values at all forecast times with an error range of ±38 kg N ha⁻¹ in about 62–69% of the simulated cases (Figures 5, 6, 8). These results emphasize the difficulty in predicting annual EONR in corn production. Given a potential threshold error range of ±30 kg N ha⁻¹, this means that further improvements are needed in modeling algorithms as well as development of more precise soil and crop management inputs to the model.

To our knowledge, there is no other tool that can provide both yield and EONR forecasts as early as at planting time. The soil nitrate test and remote sensing approaches are in-season tools. Pre-season tools such as the yield-goal approach rely on yield guesses (Van Es et al., 2006; Sela et al., 2017); while the MRTN tool incorporates N response variation and accounts for price fluctuations, but does not directly adjust for year-to-year variability (Sawyer et al., 2006).

As illustrated in Figure 1, there are many dynamic and interactive factors involved in EONR prediction (Scharf, 2001; Mamo et al., 2003; Scharf et al., 2006; Dhital and Raun, 2016). Every year, management, environment, and genotype interactions determine the shape of the yield response to N fertilizer relationship that is used to estimate EONR. Our modeling approach accounted for the majority of the factors by simulating residue decomposition, soil organic matter dynamics, changes in soil temperature, moisture, and nitrate levels during fallow periods and accounted for management such as planting date, row spacing and date of N application. All these factors affect soil N supply and crop N demand. Low EONR predictions from the model were usually associated with high estimates of soil residual nitrate levels at planting and delays in planting. But, there are many interactions that simultaneously occur among soil-plant processes within the system that makes it difficult to simplify and generalize the reasons of yield and EONR variability (see Figure S5 and Table S2). For example, in a previous work using APSIM model, we found that spring precipitation was related to N losses and EONR (the higher the spring precipitation the higher the EONR; Puntel et al., 2016). In contrast, a later study showed that the amount of precipitation had little influence on simulated grain yield in central Iowa because of the effect of groundwater tables (Togliatti et al., 2017).

The inter-annual prediction accuracy of the EONR forecast was lower than that of the yield forecast (Figures 3–fig66, Figures S3, S4). One reason is that EONR values are not a direct output of the model but a result from a regression analysis of model outputs (yields) which incorporates further uncertainty (Puntel et al., 2016). Across years, the simulated site-mean EONR value by the model was similar to the observed site-mean (Table 1). We believe that by running the model sequentially, annual over-and under-predictions are canceled out and the model is able to predict the site-mean values more accurately.

This study identified three areas for future research: (1) simulation set up, continuously vs. annual reset; (2) YEONR and EONR predictions in extreme years, and (3) weather data to drive simulations.

Our simulation protocol is characterized by a high degree of difficulty as we ran the model sequentially from 1999 to 2014 to avoid annual re-initialization of soil input parameters and to capture the carry-over effects on N dynamics (Constantin et al., 2011; Basso and Ritchie, 2015). That approach was followed because of the lack in data to update the model every year and because there was evidence from other studies that models can simulate yields and organic matter trends at different N rates in the long term (Ma et al., 2007; Puntel et al., 2016). The drawback of this approach is that simulation errors can accumulate over time and affect the next year's simulation, especially if the model fails to predict crop yield and N dynamics in one of the years (Salo et al., 2016).

We believe that yield and EONR forecasts at planting time can be further improved if additional information on soil nitrate, water, and surface residue was available to eliminate model uncertainties in simulating carry-over effects during fallow periods (Hansen et al., 2006; Carberry et al., 2009; Ines et al., 2013; Yin et al., 2017). Previous modeling work in Iowa has shown that the available soil water and N status at planting time largely affected predicted mean yields and the range of yield level probabilities (Archontoulis et al., 2016). Therefore, when the model is primarily used for accurate year-by-year yield and EONR forecasting purposes, an annual reset approach may be more appropriate than sequential (Ines et al., 2013; Iqbal et al., 2017). This information can be derived from emerging technologies such as remote sensing, and other common tools such as soil N testing (Basso et al., 2001; Jin et al., 2017; Reimer et al., 2017). Testing and possible improvements in the APSIM surface organic matter model for corn and soybean residues in future studies may improve the simulation of the carry over effects and predictions of EONR, especially in the SC rotation.

There is a need for further research to improve the annual EONR predictability in extreme weather years with precipitation above or below historical average precipitations. Differences between predicted and observed EONR tended to be higher for cold weather years (temperature below the historical average), and their agreement was largely affected by the inaccurate prediction on extremely wet (i.e., 2008) and dry (i.e., 2000) weather years (Figure S2). In this study, we noticed that EONR forecast was more accurate in years that had EONR values near the long-term site average and the prediction accuracy decreased in years with extreme high or low EONR measured values (±20% from long-term site average, Table 2).

For the farmer, both extreme high or low EONR predictions result in economic loss, with the greatest being yield loss when under-fertilizing in high N rate requirement years. Furthermore, when high N is predicted in low N need years the risk of N loss to the environment is greatest (Raun et al., 2017). In particular, the model failed to forecast accurate EONR values in extreme high precipitation years such as 2008 and 2010. We believe the main reason is the incorrect simulation of the carry-over effects from previous year (i.e., soil inorganic nitrogen, soil moisture, root and carbon and nitrogen inputs from previous crops) or possible underestimation of N losses from the system (He et al., 2017; Yin et al., 2017). Prediction of EONR is very complex and less accurate than yield (Figure 2). Perhaps an alternative way of using the model as a forecast tool may be to predict the key-components of the EONR such as yields and soil supply and plant N uptake dynamics, in which the model performs well (Archontoulis and Licht, 2017). In that way, we avoid accumulation of errors that leads to a low prediction accuracy of the EONR.

For weather uncertainty affecting yield and derived EONR forecasts (Hansen et al., 2006), our results indicated that use of the entire historical record (35 years) in the simulation process is better than selecting years based on similar weather events or using only a smaller set number of previous years (Figure 7). This was more evident in EONR predictions and less in yield predictions. Although the reasons are not sufficiently clear, based on other studies, we believe that simulated crop growth depends more on the distribution of weather within the season than the season average which was used to categorize the years used for the weather scenarios (Hansen and Indeje, 2004; Figure S1b).

We also found that the use of at least 10 and 20 years of weather data is associated with small error (2.5% less accuracy than the 35-year; Figure 7) which is an important finding given that not all sites have accurate weather records for 35 years (Hansen et al., 2004; Grassini et al., 2015). Our findings agree with Van Wart et al. (2013), who showed that 6–10 years of weather data is needed for sites with similar rain patterns as the one used in this study. However, we also recognize there are other ways of selecting years to be used in forecasting studies and this is a topic for further investigation (Hansen et al., 2004).

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.