The present study employs data from a previous international survey of approximately 645 residents per city (total n = 3,224) that gathered data on participants’ exposure in urban settings to various extreme weather events, their experience with flood management interventions, their demographic characteristics, and their preferences among different policies geared toward reducing flood disaster risk. The primary analysis for the full survey instrument is currently under peer review as a manuscript with the Journal of Environmental Psychology (Tier et al., 2025).

Out of this larger survey instrument, we utilize a subset of items related to residents’ exposure to extreme weather events and their demographic characteristics (see Appendix A for this subset of the survey codebook). We discard 829 responses, out of the total surveyed in the sample, from participants who had lived in their city for less than 3 years in order to remove responses from those who may not have had enough time to experience hazards in their city. We then discard an additional 7 responses from participants responding with a non-binary gender identity; because this category is so small, it does not allow for robust statistical conclusions. In total, we use n = 2, 388 responses in this study. More specifically, we utilized the following survey items (a summary of responses to these survey items can be found in the Appendix):

The international survey that we drew on from Tier et al. (2025) was designed through an iterative process that included exploratory work in each of the case study locations. The cities were selected based on similarities in population size, national governance type, economic standing, and experience with extreme weather events. We also selected for global geographic range, and made sure that the case locations included multiple cities in the Global South. The survey instrument was designed as follows: (1) it was first drafted in English; (2) collaborators from each city checked for cultural fit of the questions and we edited the English version as needed; (3) collaborators provided translations in conjunction with additional rounds of translation and back-translation via DeepL and Google Translate; different colleagues from each city edited the translations as needed and we checked for consistency across all locations and versions. The survey platform was built using Qualtrics XM software, approved by Princeton University’s Institutional Review Board (#16462), and beta-tested internationally using Prolific. The actual data collection was conducted via the Centiment survey panel company.

This study aims to understand which characteristics of the survey population are more strongly associated with an individual participant’s past exposure to a particular type of extreme weather event. We assess nine participant characteristics, split into three categories: four commonly referenced climate vulnerability demographics (income, age, gender, and education level), three rarely-referenced climate vulnerability demographics (Queer identity, disability identity, and non-dominant primary language), and two self-reported perceptions of vulnerability (self-perceived harm from flood disasters and self-perceived discrimination).

Feature importance indicates the degree to which these participant characteristics are present in urban populations experiencing each of the extreme weather events included in the survey. In other words, feature importance assigns a weight to different participant characteristics (self-perceived discrimination, income, etc.) based on how relevant they are for predicting a given target variable (past exposure to coastal flooding, heatwaves, etc.).

We employ a supervised machine learning approach whereby importance analysis is performed through gradient-boosted decision trees to measure the importance of each of the participant characteristics relative to each other, for a given climate hazard type. We chose to employ this method over OLS or other regression-based approaches for three reasons. First, in this case it is not appropriate to assume a linear relationship between hazard exposure and participant characteristics. Second, decision trees are a better choice for interpreting feature importance because here they rank participant characteristics based on their contribution to exposure to hazard events. In particular, our decision trees create ranks based on how much each participant characteristic contributes to the decision-making at each split in the tree. Hence, those participant characteristics that are selected for splitting higher up in the tree are interpreted as more relevant. Third, the participant characteristic variables are either ordinal (e.g., Likert-type scales, income range, etc.) or binary; decision trees, being a non-linear model, have the capability to include variables of this nature. It is also worth noting that we use gradient-boosting trees as opposed to random forests. Gradient-boosted trees allow for higher accuracy because they are trained sequentially, rather than independently, to correct on each others’ errors.

We perform this analysis through the XGBoost library (Chen and Guestrin, 2016), which allows for implementation of the stochastic gradient-boosting algorithm (Friedman, 2001) with decision trees being used as the weak learner.

Like other supervised learning models, tree boosting training is done by defining an objective function and optimizing it. An additive model (Equation 1),

is fit by stages, also called additive training. In each stage, a weak learner is introduced, which minimizes the loss with each added tree. Trees continue to be added as long as their addition improves the model output. Training stops once the loss function reaches an acceptable level, or when it is no longer possible to improve on an external validation dataset. In this case, we employ a validation set for early stopping to find the optimal number of boosting rounds.

where (Equation 2) for any step m, ensemble step m equals ensemble step m minus 1 plus the learning rate (n) times the weak learner at step m.

The model hence continues training until the validation score stops improving; here our objective is to minimize the log loss. In other words, we employ negative log likelihood as our loss function. This measures how well the model predictions align with the data, or its capability to reproduce true data. This loss function calculates the logarithm of the likelihood of observing the data given the parameters of the model, penalizing the model when it assigns low probability to what in the data would be a correct feature. Following existing convention, the data were split for training (80 percent) and testing (20 percent).

XGBoost provides feature importance in the form of gain, cover, and frequency. Gain measures the improvement in accuracy brought by a feature to the tree branches it is on. Cover looks at the number of samples affected by the feature for each split. Frequency is the percentage of times a covariate is used in splitting. In this case, we employ gain in order to understand the contribution of our covariate of interest to the accuracy of the prediction. Once models are fitted, the weighted root mean square error (RMSE) of the model is used together with visual inspection to determine the appropriate threshold for results. Only models with a robust fit are included in the results. Out of the models represented, results that are not considered are those hazard-city combinations with a high RMSE and poor visual fit.

Figure 1 shows how the validation of the result was conducted. The series of plots show the performance of the top trained models, where the dots represent the median prediction and the bar represents the uncertainty of such prediction for each target class. Performance in this case is measured as the model’s ability to predict the true value of hazard exposure (represented in the x-axis) through the predictions represented along the y-axis. The predictions represent values from 1 to 5, which is the scale at which past exposure to each hazard was measured. For instance, exposure to heavy wind was expressed on a scale of 1 to 5, and the plot shows how accurate the model is at predicting such exposure given the inputted participant characteristics (income, gender, discrimination, etc.). The fit to the line represents the fit of the models: the closer to the dotted line, the better the model can predict exposure. The number in the top right corner of each figure is the weighted RMSE for the model. The transparency of some figures indicates that the fit of the model was not sufficiently strong.

Figure 2 shows the primary results of our study. The histogram bars represent the feature importance calculated by the best fitted model. Each bar represents a given characteristic that our model looks at. The height of histogram bars represents the relative importance of a participant characteristic with respect to the other characteristics for a given city and hazard. The error bar at the top of each histogram bar represents the range in importance when calculated with the top five best fitting models after training. For reference, a total of 1,000 models were trained. This is meant to show not only the model chosen as the best fit, but also the range of results if we had picked alternative models that constitute a good fit. This showcases that even though different training rounds can deliver different results, these are highly consistent across resulting models.

Across the 18 models deemed robust (nontransparent panels in Figure 2), some similarities stand out. First, more traditional social vulnerability indicators (i.e., education level, income, age, and gender) are less frequent among the most important characteristics for any of the surveyed city populations in our feature importance analysis. Second, some additional demographic variables from our survey were in fact more important than traditional vulnerability indicators (i.e., Queer identity, disability identity, and non-dominant primary language). Third, measures of perceived discrimination and perceived harm from flood disasters in particular were frequently the most important predictors across different types of extreme weather events and across cities.

In the Buenos Aires sample, the participant characteristic most important to exposure to flooding from extreme rainfall was self-perceived harm from flood disasters and the least important were income and non-dominant primary language (with relatively high participant characteristic importance across the board). Disability identity was the most important and gender was the least important to exposure to heavy winds (again with relatively high importance across the other characteristics). Disability identity was the most important to exposure to extreme heat, with non-dominant primary language somewhat important and all other characteristics ranked very low.

In the Johannesburg sample, self-perceived harm from flood disasters was by far the most important to exposure to flooding from extreme rainfall, while income and gender were the least important (and moderate importance across the other characteristics). Self-perceived harm from flood disasters was again the most important to exposure to flooding from river overflows, with education level and disability identity also highly important, self-perceived discrimination and non-dominant primary language moderately important, and all others ranked lower.

In the Seoul sample, self-perceived harm from flood disasters was again the most important by a significant amount to exposure to flooding from extreme rainfall, self-perceived discrimination and disability identity were moderately important, and Queer identity and non-dominant primary language ranked very low. Disability identity was the most important by a significant amount to exposure to flooding from coastal storms, non-dominant primary language was moderately important, and all other characteristics were ranked very low. Self-perceived discrimination was by far the most important to exposure to droughts, with moderate importance for all other characteristics (Queer identity was not included due to absence in the sample).

In the London sample, self-perceived discrimination was most important to exposure to flooding from extreme rainfall, with self-perceived harm from flood disasters and Queer identity also very important and all other characteristics moderately important. Self-perceived discrimination was also most important to exposure to flooding from river overflows, with all other characteristics moderately important. Queer identity was most important to exposure to extreme heat, with self-perceived harm from flood disasters, self-perceived discrimination, disability identity, and non-dominant primary language all moderately important and remaining characteristics ranked low. Queer identity was most important to exposure to droughts, with self-perceived harm from flood disasters, self-perceived discrimination, disability identity, and non-dominant primary language all moderately important and remaining characteristics ranked low. Disability identity was by far the most important to exposure to wildfires, with all other characteristics ranked quite low.

In the New York City sample, self-perceived harm from flood disasters was most important to exposure to flooding from extreme rainfall, with self-perceived discrimination as next-most important and all characteristics having moderate to high importance. Self-perceived discrimination was most important to exposure to flooding from coastal storms, with self-perceived harm from flood disasters a close second and all other characteristics moderately important. Self-perceived harm from flood disasters was also most important to exposure to flooding from river overflows, with Queer identity and non-dominant primary language as next-most important in turn and all other characteristics having moderate to high importance. Self-perceived harm from flood disasters was also most important to exposure to extreme heat, with disability identity and gender as next-most important in turn and all other characteristics having moderate to high importance. Finally, Queer identity was most important to exposure to wildfires, with disability identity as next-most important, self-perceived discrimination and non-dominant language as moderately important, and all other characteristics ranked quite low.

If we take frequency counts for which participant characteristics were ranked as most important across all 18 robust models, self-perceived harm from flood disasters is identified the most often (seven times), with self-perceived discrimination and disability identity both identified four times and Queer identity three times. Education level, income, age, gender, and non-dominant primary language are never the most important characteristics for any city-hazard pair. Conversely, gender is identified as the least important characteristic eight times – with non-dominant primary language identified as the least important three times; education level, age, and Queer identity identified two times each, and income identified once. Self-perceived harm from flood disasters, self-perceived discrimination, and disability identity are never the least important characteristic. Notably, non-dominant primary language often falls in the middle of the pack in terms of feature importance – more often than do education level, income, age, and gender.

These findings are important to highlight because composite indicators are typically built to measure proxies for exposure and vulnerability to hazard events. However, proxies are not always well fit to measure the complex relationships that lie behind them. Importantly, they are also not well suited to capture the lived experience of urban residents, which can be crucial in defining their resilience to hazard events. Our results show the importance of including measurements, either quantitative or qualitative, that can help decision-makers to have a sense of context-specific risk characteristics. Measurements of self-perception in particular are as crucial for composite indices as are other proxies of vulnerability. It is important to stress that our survey distinguishes between actual exposure to hazard events (albeit self-reported exposure) and participants’ evaluations of their own vulnerabilities. The hazard exposure survey items queried participants about the number of times that they had experienced each hazard type – rather than their exposure in relation to their neighbors or peers. On the other hand, the two vulnerability questions were posed to have participants consider their relational vulnerability within some perceived societal average. Thus, it is a consequential finding that participants’ perceptions of their own vulnerability does in fact match how their hazard exposure compares to that of others.

As previously mentioned, it is important to emphasize that the results presented here are based on the 18 models that proved to be robust based on their RMSE and visual fit. This translates into limitations on the types of conclusions that can be reached through the available results; although we can assert that commonalities and differences exist across cities and across hazards, we cannot make specific predictions for every hazard-city combination. Additionally, the scarce data related to earthquake vulnerability do not allow for applicability of the findings to this type of hazard.

Despite the commonalities described above, we also find that there is significant heterogeneity regarding feature importance across cities and across hazard types. For example, wildfires – as well as droughts, extreme heat, and coastal flooding for certain cities – were one hazard type in which self-perceptions of vulnerability were much more subdued while Queer and/or disability identity were far more prominent. For the three extreme weather events related to flooding (heavy rainfall, coastal storms, and river overflows), the two measures of self-perceptions of vulnerability (harm from flood disasters and/or discrimination) were more consistently important.

Local context also plays an important role, especially considering pre-existing variation regarding which identities experience marginalization. For example, disability identity repeatedly ranked highly in the Buenos Aires sample – and no other city – as a predictor for the three hazard types represented in this city’s robust models (exposure to flooding from heavy rainfall, heavy winds, and extreme heat), despite more variation in the other participant characteristics across hazard types for Buenos Aires. Another example is that Queer identity in the Seoul sample shows an unusual pattern compared to other cities: it ranks very low in importance for exposure to flooding from heavy rainfall and coastal storms, and has no data for droughts (i.e., no Seoul participants who identified as Queer also had experienced droughts). This could suggest that this participant characteristic is not tied to vulnerability in Seoul (unlike in other cities) – or, perhaps more likely, that participants were much less likely to self-report this identity in this location.

These results are in line with existing literature that highlights the importance of indicator weights (i.e., giving numerical importance to some indicators over others) when employing such vulnerability indicators for disaster decision-making (Papathoma-Köhle et al., 2019), as well as the importance of considering “dynamic vulnerability” to understand disaster risk (de Ruiter and van Loon, 2022). Dynamic vulnerability refers to the fact that vulnerability to a given hazard might change over time and can be influenced by changes in context and even by compound hazards. Our results showcase that climate risks need to be understood within the contextual idiosyncrasies of a given city, and vulnerability indicators cannot fully be generalized across cities or even across hazards within the same city. This is particularly true when selecting the types of questions or measurements that will be assessed about a population. Furthermore, certain aspects key to vulnerability may not be self-reported due to cultural or contextual reasons, so other approaches may need to be considered to truly gauge relevant features and to support those that are most vulnerable.

This article was written in the context of this journal’s special issue titled, Managed Retreat in Response to Climate Hazards. In this last section we include reflections on planning for managed retreat in order to demonstrate a concrete policy area that could benefit from our findings.

With the “solution space” for managing the global climate crisis shrinking as dangerous effects of greenhouse gas emissions become increasingly “locked-in” (Haasnoot et al., 2021), ambitious and transformative adaptation planning and implementation strategies are needed now. Adaptation actions are commonly categorized as either resistance (such as building a seawall), advancement (such as creating new land through reclamation), accommodation (such as elevating a structure at risk of flooding), avoidance (such as restricting development in high-risk floodplains), or retreat (such as planned relocation) (Mach and Siders, 2021). Retreat, the focal point of this special issue, typically comes into play when other adaptation measures are deemed insufficient given the level of exposure or vulnerability to the hazards that a given community or household is experiencing. Managed retreat refers to instances when relocation is purposeful and coordinated; although managed retreat can be initiated over a variety of different temporal and governance scales using various instruments by actors such as governments, the private sector, and civil society.

Even though effective in reducing hazard exposure, managed retreat faces many other barriers to success – and even the precise terminology for this type of policy initiative is highly debated.⁵ In the U. S., the limited number of climate-induced managed retreat programs that do exist have largely been tackled through buyout/acquisition strategies (Siders and Gerber-Chavez, 2021). Though there is a growing literature on these buyouts (see Greer et al. (2022) and Mach et al. (2019) for review papers), Greer and Brokopp Binder (2017) also conclude that minimal policy learning has occurred due to lack of both actionable data and formal government guidance. Evidence consistently shows that buyout programs suffer from a wide range of challenges, including: long wait times (Siders and Gerber-Chavez, 2021; Weber and Moore, 2019); inadequate federal funding (CRS, 2022; Peterson et al., 2020); complex and uncoordinated multi-level governance processes (Siders and Gerber-Chavez, 2021; Weber and Moore, 2019); lack of transparency in program procedures/structures (Greer et al., 2022); excessive focus on individual homeowners, as opposed to renters, communities, or other forms of housing tenure (Wilson et al., 2021); failure to incorporate social vulnerability indicators (Wilson et al., 2021) or to capture potential structural causes of immobility (Seeteram et al., 2023); insufficient monitoring and evaluation (McGhee et al., 2020); and misguided targets, such as measuring number of household recipients instead of favorable long-term outcomes (Manda et al., 2023).

Of particular importance to relocation programs is ensuring more distributive, procedural, and other forms of climate justice – especially given that current data suggest poor existing outcomes on these fronts. Mach et al. (2019) find that wealthier and denser counties are more likely to implement buyout programs – but within those counties, households in lower-income and more racially diverse areas are more likely to actually receive a buyout. Loughran et al. (2019) find that “the rate at which minority populations were growing” influences White households’ interest in accepting buyouts. Curran-Groome et al. (2021) also note immense inefficiencies in application processes, which place a burden on lower-income and less staffed local governments. Finally, Marino (2018) contends that buyouts as currently formulated do not work well in many Indigenous communities where property ownership is not always documented and individually-focused relocations conflict with community-oriented decision-making.

The most prevalent typology of managed retreat globally is still what Ajibade et al. (2022) describe as “techno-managerial,” or focused on simple hazard exposure reduction tactics. Present-day interventions lag behind not only in overcoming barriers initiate and implement a retreat process, but also in focusing on the outcomes from a justice and well-being perspective. We argue that the results of our analysis reinforce the pressing need for managed retreat efforts to include a focus on compensatory, transformative approaches that center justice. This is particularly true given the types of participant characteristics that we found to contribute to hazard exposure the most: people’s sense of their own flood hazard vulnerability, their experience with discrimination, and both disability and Queer identity. These characteristics are associated with larger systemic, institutional, and social barriers that have the potential to make residents undergoing relocation even more vulnerable to hazard exposure or subject to even further discrimination in the buyout selection process or in the new locations that they relocate to. Our findings do not necessarily suggest that relocation should not take place, but instead that such initiatives should have a strong focus on justice-centered outcomes.