We present a four-parameter Exploitation–Exploration with Forgetting (EEF) model that integrates an individualized forgetting rate into the reinforcement-learning update, thereby capturing person specific information loss while remaining highly parsimonious. Deck-specific initial weights are derived directly from the empirical first-choice frequencies, anchoring the model in observed behavior rather than arbitrary priors. To probe information decay dynamics we introduce two complementary indices—Sequential Exploration Decay (SED) and Forgetting Interval (FI)—which quantify, respectively, the rate at which exploratory value wanes and the typical lag between informative choices. In addition, the EEF model was fitted to data from healthy young adults as well as two heterogeneous groups—individuals with gambling disorder and older adults—and achieved the best (or statistically comparable) BIC and AIC scores across these cohorts while remaining competitive on the free energy criterion. Parameter-recovery simulations yielded high correlations between generating and recovered values, confirming that the model's parameters are both identifiable and psychologically interpretable.

This study provides a more precise and in-depth methodology for modeling and understanding complex decision-making behaviors. Particularly in terms of cognitive validity, our research shows that the forgetting parameter can effectively capture individuals' information processing characteristics during decision-making and predict future choice behavior. Furthermore, we have enhanced the previously established open-source toolbox, making this model widely applicable to various research contexts and enabling researchers to more efficiently analyze and understand the phenomenon of forgetting in the IGT (Ligneul, 2019).

The Exploitation and Exploration with Forgetting (EEF) model consists of two main components: an exploitation module and an exploration module. The exploitation module is conceptually similar to the core ideas in the PVL and VSE models, both of which primarily base their computations on gain and loss utilities. However, our model does not include the “loss aversion” parameter from the PVL model or the decay parameter from the VSE model. The EEF model specifically introduces two key control variables:

In each trial, the exploitation weight of each deck d, d∈{A, B, C, D}, is updated according to the following equation:

Exploitation_d(t) represents the exploitation weight of deck d at time t. Equation 1 describes how the exploitation weight of deck d is updated when it is not selected. Here, the weight gradually decreases at a rate controlled by the forgetting parameter λ, simulating the effect of forgetting—that is, the gradual decay of unused information.

Equation 2 calculates the value V(t) at time t, where θ modulates the sensitivity to gains and losses. Equation 3 illustrates how, when deck d is selected, the exploitation weight is updated based on the recent monetary feedback. In the exploitation module, an increase in the forgetting parameter λ indicates faster forgetting, meaning that previous information is lost more rapidly. In other words, as an individual's level of forgetting increases, the emphasis in exploitation weight updates shifts from the prior exploitation weight to the current value incentive.

Exploration weight represents the gradually increasing attractiveness of a specific deck when it has not been selected for a period of time. The exploration weight is an independent function separate from the exploitation weight, meaning it is not influenced by the monetary feedback experienced during the IGT. Previous models have used the delta rule combined with a learning rate and exploration reward parameter to simulate sequential exploration behaviors by adjusting the weights of unselected decks (Ligneul, 2019). In our study, we design the increase in exploration weight to be closely related to the forgetting process, with a particular focus on the impact of forgetting on exploratory behaviors.

The exploration weight is governed by the following equations:

Exploration_d(t) represents the exploration weight of deck d at time t. Equation 4 is the exploration weight update equation for a selected deck, indicating that the exploration weight of a selected deck is reset to zero. In contrast, Equation 5 includes the individual forgetting parameter λ and the exploration incentive parameter ϕ (ranging from −5 to 5). This equation describes that, starting from an initialized exploration weight, consistent with the cognitive reinforcement mechanisms observed in Yechiam et al. (2010), if a deck remains unselected over time, its exploration weight will gradually approach ϕ under the influence of exploration incentives, encouraging (if ϕ is positive) or discouraging (if ϕ is negative) the agent to re-explore options that have been ignored for a long period.

The equation also suggests that in groups with a high level of forgetting, the effect of exploration incentives may be weaker. In high-forgetting groups, individuals are inclined to ignore or quickly forget past information, making them more likely to forget the information related to “how long an option has been unchosen” when making decisions, causing the update of exploration weight to depend more on previous weights rather than on exploration incentives. To consider an extreme case, when the forgetting rate is 1, it implies that an individual with only memory of the current trial completely disregards past information and makes choices based solely on the rewards received in the current trial. In this scenario, the exploration weight remains unchanged from the initial value of zero, and each choice relies exclusively on the utility calculation of the exploitation weight. This distinction illustrates the different ways in which λ impacts exploration and exploitation weights.

After calculating the exploitation weight and exploration weight for each deck using the exploitation and exploration modules, the EEF model determines the probability of selecting each deck using the following equation:

The EEF model follows a similar approach to the PVL model in generating choice probabilities (Steingroever et al., 2013). This equation treats the decision-making process as a stochastic process regulated by a consistency parameter (subsequently referred to as “C”), which adjusts the randomness of choice behavior. In statistical physics, C is analogous to the “inverse temperature” parameter β, which controls the probability distribution of state selection. In decision-making models, an increase in C indicates that the choice process is more influenced by the combined effect of the computed exploration and exploitation weights, thereby reducing randomness and increasing reliance on these weights.

Notably, the value of C is derived from a transformation of the inverse temperature parameter β, with the specific transformation formula defined as C = 3^β − 1, where β ranges from 0 to 5. The model ultimately converts these weights into choice probabilities using the softmax function, ensuring that the probability distribution is normalized (i.e., the sum of the probabilities for all options equals 1). This method effectively balances the impact of randomness and the influence of weights, ensuring that the model's choice behavior not only reflects the computed weights but also adheres to the statistical characteristics of a stochastic process.

The structure of the IGT allows researchers to observe and analyze participants' decision-making patterns across multiple choices, which gradually become apparent as the task progresses. At the beginning of this sequence of decisions, the first choice plays a unique role. The first choice is made by participants without a full understanding of the long-term consequences of each deck, revealing their initial risk preferences and intuitive judgments. After constructing the core framework of the EEF model, we noted that, despite extensive research on the IGT, few studies have thoroughly explored participants' first-choice behaviors within the task, and it is even rarer to see these behaviors used as the foundation for model construction. Thus, this study aims to fill this gap by conducting an in-depth analysis of first choices in the IGT, with the goal of improving the model's parameter recovery when simulating human decision-making behavior.

Specifically, we seek to investigate first-choice patterns through a comprehensive dataset to provide prior knowledge that enhances model performance. We utilized a dataset containing 504 healthy participants (Steingroever et al., 2015), drawn from ten independent studies using standard or variant IGT payoff schemes, encompassing the classic 100-trial version of the task.

Figure 1 presents our visualization analysis of the choices made by 504 participants across each trial in the dataset (Steingroever et al., 2015). Figure 1A shows the choice frequency for each option across 100 trials by 504 participants, while Figure 1B reveals distinct preference patterns for card decks A, B, C, and D in the first round of the IGT. Statistical analysis indicates that Deck A is the most popular option, with a selection probability of 38.29%, chosen 193 times. This is followed by Deck B, which was chosen 126 times, accounting for 25.00% of the total choices. In contrast, Decks C and D have lower selection probabilities, at 20.83% (105 choices) and 15.87% (80 choices), respectively. The chi-square test further reinforces the significance of this pattern. The resulting chi-square statistic reached 68.46, with a corresponding p-value of 9.13 × 10⁻¹⁵. This p-value is far below the 0.05 threshold, clearly indicating that the observed choice frequencies differ significantly from the theoretical random probability of each option (25%). This suggests that under conditions of incomplete information, participants display a pronounced initial preference for certain options, particularly Deck A. Such a choice bias may be related to participants' intuitive judgments and their varying perceptions of risk, leading to the formation of preferences for certain decks in the early stages of decision-making.

The model's prior knowledge reflects the initial beliefs or experiences that humans rely on when making decisions. Incorporating this knowledge into the model allows it to better simulate human intuition and preferences, thereby enhancing its predictive accuracy and alignment with human decision-making patterns. To improve the model's fit to human decision-making behaviors, we incorporated the observed frequencies of participants' initial choices as prior knowledge in the model.

As previously mentioned, the EEF model combines the derived exploitation and exploration weights with a consistency parameter C and transforms them into choice probabilities using the softmax function. Thus, we can reverse-transform the observed choice frequencies from earlier through the softmax function (setting β = 3) to compute the initial weight for each deck, which serves as the model's prior knowledge.

The initial weights of the four card decks are calculated as follows:

This initial weight serves as the model's starting condition and is combined with the weights subsequently derived from the exploitation and exploration strategies to simulate participants' decision-making behaviors in the Iowa Gambling Task. By doing so, the model is provided with a more reasonable starting condition, allowing it to capture participants' intuitive tendencies during the early stages of decision-making.

More than a mere post-hoc adjustment, embedding the empirically observed first-choice bias as a prior state is not a generic post-hoc tuning device but an organic element of the EEF's forgetting architecture. The vector w_ini constitutes an initial memory trace that decays trial by trial at the individual specific forgetting rate λ, entering the same update equations as all subsequent experience. Benchmark models that lack an explicit memory-decay or exploration channel (e.g., EV, PVL-Δ) cannot accommodate such transient priors without redefining their core equations, which would blur the theoretical distinctions under comparison. For this reason the first-choice prior is implemented only in the EEF model, where it naturally complements the joint principles of forgetting and dynamic memory renewal.

To systematically analyze IGT decision-making patterns and validate the empirical fit of the new computational model, we selected several key datasets, all of which used the IGT as a research tool. The following section details these datasets and their applications in this study.

As experimental data and theoretical understanding continue to advance, the validation and comparison of decision-making simulation methods have become increasingly important. This section introduces and compares several decision-making simulation methods used for the IGT. In particular, we introduce two new evaluation metrics—Sequential Exploration Decay (SED) and Forgetting Interval (FI)—which help us further investigate the effectiveness of various methods in capturing forgetting and exploratory behaviors.

In this subsection, we conduct a detailed comparison of the effects of the two newly proposed evaluation metrics—Sequential Exploration Decay (SED) and Forgetting Interval (FI)—in the EEF model and five other previously established models, using the IGT dataset from 504 participants as the basis.

The Sequential Exploration Decay (SED) metric introduced in this study is designed to measure the extent of information value decay and changes in exploratory behavior between the first 50 rounds and the last 50 rounds in the IGT.

Figure 2A presents the actual SED ratio based on data from 504 participants and the results simulated by different decision-making models. A ratio >1 signifies that exploration events are more frequent in the first half of the task than in the second half, thereby operationalising the gradual decay of information value. Conversely, a ratio close to 1 indicates that participants (or models) maintain a comparable level of exploration throughout. Hence, the proximity of the EEF bar to the empirical dashed line visually conveys its ability to reproduce the tempo of exploratory decline observed in humans.

The experimental data show that the actual SED ratio is 1.549, reflecting a gradual decrease in exploratory behavior throughout the task. As illustrated in the chart, there are significant differences in the SED ratios among different decision-making models: the SED ratios for the EV and PVLdelta models are much higher than the actual data, at 79.857 and 74.375, respectively. The SED ratios for the ORL, VPP, and VSE models are 3.013, 3.399, and 2.351, respectively—much lower than those of the EV and PVLdelta models but still significantly higher than the actual data. Statistical analysis using the chi-square test confirms that, except for the EEF model, the SED ratios of all other models differ significantly from the actual SED ratio(p < 0.001).

Such a stark contrast between early and late-stage exploration behaviors may indicate an excessive dependence and exploitation of early-acquired information in these models. This near-cessation of exploration in the later stages of the task contrasts with the gradually diminishing yet sustained exploration behavior observed in the actual data, highlighting a deficiency in the models' dynamic adaptability to simulate human decision-making. In contrast, the EEF model shows a more balanced pattern of exploration between the early and late stages, with an SED ratio of 1.557, which is very close to the actual data value of 1.549. This suggests that the EEF model, by incorporating forgetting, can better simulate the actual exploration dynamics and more accurately reflect human behavioral patterns and the phenomenon of forgetting in an ever-changing decision environment. It also indicates that the continued exploration observed in the later stages of the IGT may be due to some degree of forgetting of previously acquired information in human decision-making.

As for the Forgetting Interval (FI) showed in Figure 2B, a new evaluation metric designed to quantify participants' degree of forgetting after receiving negative feedback, an analysis of the data from 504 participants revealed that the actual FI is 6.24 trials, indicating that participants typically delay a period of time before reselecting the same deck after experiencing negative feedback.

In terms of model comparison, the EEF model's FI value is 4.13 trials. Although this differs from the actual data (6.24 trials), it is the closest to the real data among all models. Other models, such as the EV model (3.69 trials), the ORL model (4.04 trials), the PVL-Delta model (3.61 trials), the VPP model (3.93 trials), and the VSE model (4.11 trials), show FI values lower than the EEF model, indicating that their performance in simulating participants' forgetting behaviors deviates further from the actual situation. This result highlights the relative advantage of the EEF model in capturing decision behaviors related to forgetting. One-tailed Welch's t-tests further confirmed that the FI value of the EEF model was significantly higher than that of the EV, PVL-Delta, and VPP models (p < 0.01), indicating better alignment with the actual data. Although the differences between EEF and the ORL(p = 0.14) and VSE(p = 0.42) models were not statistically significant, the EEF model still achieved the closest approximation to the real data among all models. This observation suggests that further refining and adjusting the parameters of the forgetting mechanism in future model developments may improve model accuracy and practical applicability.

By comparing the performance of each model in terms of SED and FI, we found that the EEF model, which incorporates forgetting factors, achieves the closest alignment with the actual data across both metrics. This finding underscores the necessity of incorporating forgetting factors into models and highlights the EEF model's significant advantage in simulating the phenomenon of forgetting and its impact on human behavior. These results suggest that future research and model development should further explore and refine the mechanisms of forgetting to enhance the model's realism and predictive accuracy.

In this section, we conduct fixed-effects and random-effects analyses on the Bayesian Information Criterion (BIC), Akaike Information Criterion (AIC), and Free Energy (F) metrics for the EEF model and five other previously established models. The experiments are still based on the IGT dataset from 504 participants. It should be noted that using Free Energy (F) directly as a metric for model comparison may not allow for a direct comparison with other statistical metrics such as BIC and AIC, due to differences in their calculation methods and scales. To address this issue, this study apply a transformation to the Free Energy (F) metric by multiplying its value by –2, i.e., calculating −2 × F (Daunizeau et al., 2014). The purpose of this transformation is to align the dimensionality and value range of Free Energy (F) with those of BIC and AIC, thereby enabling a clearer and more rational comparison between models.

Figure 3 uses the EEF model as the reference point and adopts a fixed-effects model comparison framework. In this context, positive bar values represent the amount of information loss incurred when selecting a model other than EEF. Because each bar reflects the difference between a given model's score and the EEF baseline, higher (positive) values indicate worse performance, while values at or below zero suggest comparable or superior performance.

As shown in the figure, the EEF model outperforms all other models across nearly all evaluation metrics. The only exception is the Free Energy criterion, where it performs slightly worse than the VSE model. Nonetheless, this marginal drawback is outweighed by its superior performance on the remaining metrics.

Under the Bayesian Model Selection framework, the EEF model achieved the highest estimated model frequency under both the BIC and AIC criteria, exceeding 70% and 40% respectively—well above the chance level (1/6) and other models. Notably, the confidence intervals of these estimates did not overlap with the chance level or other models, indicating a statistically robust group-level preference for the EEF model. These consistent results across multiple evaluation criteria provide strong Bayesian evidence for the robustness and generalizability of the EEF model in explaining human decision-making behavior in the IGT.

In this section, we present the results of the simulation experiments on model and parameter recovery. We focus on evaluating the EEF model's performance compared to the other five existing models in terms of prediction accuracy for fitting actual choice data and simulated choice data, as well as the quality of parameter recovery.

Figure 4 shows the performance of three metrics that measure model recovery. In the analysis of fitting actual data (Fitted Choices), the EEF model achieved 58.13% accuracy, significantly outperforming the EV model (44.01%; t = 14.02, p < 0.0001) and the PVLΔ model (45.51%; t = 12.15, p < 0.0001). Its accuracy did not differ significantly from the ORL model (57.47%; t = 0.64, p = 0.521), nor from the VPP (58.21%; t = −0.08, p = 0.940), or VSE models (58.64%; t = −0.50, p = 0.618). Notably, the simplified EEF model achieves any equivalent fitting performance with fewer parameters than ORL, VPP, and VSE, underscoring its efficiency.

In the analysis comparing whether the choices derived from the agent (Simulated Choices) can replicate the original choice data, the EEF model achieved an accuracy of 41.977%—the highest among all six models—demonstrating strong performance. Two-tailed Welch's t-tests showed that its accuracy was significantly higher than that of EV [27.05%; t(73861) = 18.16, p < 0.0001], PVLDelta [27.43%; t(78709) = 17.28, p < 0.0001], and VPP [38.77%; t(100756) = 3.11, p = 0.0018], but did not differ significantly from ORL [40.14%; t(100632) = 1.80, p = 0.072] or VSE (41.55%; t(100773) = 0.50, p = 0.619). Despite using fewer free parameters, the EEF model still outperformed most alternatives, underscoring its robustness in replicating experimental behavior and its efficiency in simulating participants' decision-making processes. These findings confirm the effectiveness and stability of the EEF model in capturing complex cognitive dynamics.

Figure 5 presents the results of the parameter recovery analysis, providing a visual assessment of each model's ability to recover simulated parameters. All axes display z-scored parameter values, standardized by subtracting the sample mean and dividing by the standard deviation. This rescaling ensures that heterogeneous parameters are comparable and can be displayed on common axes.

As a result of this standardization, data points may assume negative values even when the original parameters (e.g., λ, θ∈[0, 1]) remain within their theoretical bounds. Negative z-scores simply indicate that a participant's parameter estimate falls below the sample mean.

The dashed line in each subplot marks perfect equivalence between parameters estimated from the real dataset (x-axis) and those recovered from simulated data (y-axis). Among all models, the EEF model shows the tightest clustering of points around this ideal line, demonstrating superior consistency and accuracy in parameter recovery.

Overall, the composite correlation coefficient for parameter recovery in the EEF model is 0.849 (range: 0.69–0.96), which is significantly higher than those of the other models. Specifically, the EV model yielded a composite coefficient of 0.787 (range: 0.75–0.86), the ORL model 0.791 (range: 0.67–0.87), and the VSE model 0.792 (range: 0.63–0.94). The PVLdelta and VPP models showed relatively inconsistent parameter recovery, with lower overall coefficients and broader variability across parameters (PVLdelta: 0.725, range: 0.52–0.86; VPP: 0.698, range: 0.38–0.94). Pairwise Fisher z-tests confirmed that the EEF model's recovery performance was significantly better than all other models (p values < 0.05 in all comparisons). Notably, the correlation coefficients for the ForgettingRate and ExploreBonus parameters in the EEF model reached 0.896 and 0.956, respectively, highlighting the model's strong explanatory and predictive power in capturing individual differences in decision-making behavior.

We further extended the EEF model by adding a loss aversion parameter (LA), ranging from 0 to 5, to the exploration module. This parameter reflects the tendency for individuals to form a more negative impression of an option after encountering negative feedback, thereby reducing its probability of being selected. We named this extended model the Exploitation and Exploration with Forgetting and Loss Aversion (EEFLA). Specifically, we made the following modification to the utility calculation in the exploitation module:

In addition, all other parts of the exploitation and exploration modules in the EEFLA model remain consistent with the original EEF model. We conducted a comprehensive comparative analysis between the EEF model and the EEFLA model.

Figure 6 shows the performance of three metrics that measure model recovery across EEF and EEFLA. First, it is essential to highlight the EEFLA model's ability to fit actual data (Fitted Choices) and replicate original choices (Simulated Choices). As mentioned earlier, the EEF model outperformed previous models in terms of parameter recovery (Fitted Choices: 58.128%; Simulated Choices: 41.977%). After incorporating the loss aversion parameter, the EEFLA model showed even better performance on these two metrics, achieving a fit to actual data of 58.956% and replicating 42.613% of the original choices. This performance surpasses that of all models, including the EEF and the five comparison models. This result indicates that introducing the loss aversion parameter allows the model to better fit the original data.

Figure 7 shows the comparison of SED and FI, and we can find that although the EEFLA model performed slightly worse than the EEF model (the SED of EEFLA is 1.598, with a difference of 0.049 from the original data, while the SED of EEF has a difference of only 0.008 from the original data; the FI of EEFLA is 4.02, with a difference of 2.22 from the original data, while the FI of EEF has a difference of 2.11), it still outperformed most of the other models. This indicates that the EEFLA model is still capable of effectively simulating human decision-making behaviors.

However, it can be seen from Figure 8 that the performance of the EEFLA model is not outstanding in terms of parameter recovery and fixed-effects statistical model analysis. The composite correlation coefficient for EEFLA's parameter recovery is 0.785 (range: 0.58–0.93), which is lower than the EEF model's 0.849 (range: 0.69–0.96). Additionally, the EEFLA model did not perform well in the previously mentioned fixed-effects statistical model analysis.

In summary, although the performance of the EEFLA model with the loss aversion parameter is still inferior to that of the EEF model in many aspects, the EEFLA model may show superior performance when there is a higher demand for accurately reproducing human decision-making behaviors.

This study proposes and validates a new computational model—the Exploitation and Exploration with Forgetting (EEF) model—which, for the first time, introduced a forgetting parameter in the context of the Iowa Gambling Task (IGT) and incorporated the initial conditions based on the first-choice data of 504 participants collected by Steingroever et al. (2015). By comparing the EEF model with previous models, we verified its effectiveness and addressed the gap in IGT research concerning the phenomenon of forgetting, revealing how individuals adjust their strategies in response to constantly changing environmental information.

Through the introduction and analysis of two new metrics, Sequential Exploration Decay (SED) and Forgetting Interval (FI), we found that the EEF model performs better than the other five models in capturing the phenomenon of forgetting in the IGT. The analysis of SED showed that the EEF model maintains a more balanced pattern of exploration behavior between the early and late stages of decision-making, and its SED value is closer to the real data compared to models like VSE and PVLdelta. This indicates that the EEF model more accurately simulates the process of information value decay over time, highlighting the central role of forgetting in exploratory behaviors. Furthermore, the EEF model's superior performance on the FI metric further demonstrates its effectiveness in simulating human behavior under the influence of negative feedback.

However, this study still has some limitations. Although the introduction of the forgetting parameter enhanced the model's explanatory power, its consistency with actual individual decision-making behavior needs further improvement in certain contexts. This may be due to the model's current inability to fully account for other complex influencing factors, such as emotional states and risk preferences. Modeling and integrating these factors present significant challenges, involving multi-dimensional data collection and complex algorithm design. In addition, although this study examined the behaviors of different age groups, a deeper exploration of specific populations (e.g., patients with mental disorders) regarding the differences in forgetting and decision-making behaviors requires more detailed research design and ethical approval. Regarding the neural mechanisms of forgetting, although existing research suggests that forgetting is closely related to the activities of the prefrontal cortex and hippocampus, integrating these neural mechanisms into our model remains a challenge. The acquisition and analysis of neurobiological data are complex and require interdisciplinary collaboration and advanced technical support.

Therefore, future research will focus on the following directions: First, refining the forgetting mechanism and exploring methods for dynamically adjusting the forgetting parameter to better simulate cognitive processes in various contexts. This may involve developing new algorithms or using machine learning techniques to automatically optimize model parameters. Second, applying the EEF model to more complex decision-making scenarios, such as multi-armed bandit tasks or real-life decision-making contexts, to verify its broader applicability and improve its generalizability. Finally, we plan to incorporate more psychological and neurobiological data, such as electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), to further investigate the neural mechanisms of forgetting and decision behavior. By integrating these neural signals into the model, we hope to enhance the model's accuracy in simulating individual decision-making processes. Although this direction is highly complex, it has the potential to significantly improve the model's biological realism and predictive capability.