Many state-of-the-art infrastructure management systems are making use of the Markov model for the prediction of deterioration and the determination of the optimal intervention strategies. When using the Markov model, elements of infrastructures are considered to be in discrete states (deterioration condition states) defined using physical characteristics, and the deterioration of elements over time is described as probable transitions between these states over time.

In estimating transition probabilities, there are two basic situations, (1) there are sufficient time-series data, i.e., when sufficient data are available for a minimum of two consecutive time intervals and (2) there are no sufficient time-series data, i.e., when there are no sufficient data available for a minimum of two consecutive time intervals. Needless to say, infrastructure managers should make decisions of interventions for all elements by reliable transition probabilities when the management system uses Markov models. In the first situation, infrastructure managers use statistical methods to estimate the transition probabilities of the Markov models. Many of the models were developed with the assumption that elements could jump no more than one state per time period, such as those used in Klein (1962), Carnahan et al. (1987), Jiang et al. (1988), Madanat and Ben-Akiva (1994), and Thompson et al. (1998). However, others were developed without this assumption, such as those developed by Tsuda et al. (2006) and Kobayashi et al. (2012a,b). The former have the advantage that they are relatively easy to compute. In the former, however, as the deterioration being predicted is relatively rapid with respect to the time periods selected, error becomes non-negligible. For example, if it is assumed that the transition probability from state i to i + 2 in 5 years is 0, the model cannot express rapid deterioration processes. The latter allow to avoid this assumption and have the advantage that resulting transition probabilities lead to more accurate prediction of deterioration. In addition, they can be used when data have been collected at non-uniform intervals.

Despite progressive development in the field of monitoring over the last decades so that data can be collected more frequently and more accurately, the second situation likely occurs in municipalities with relatively small scale and it can be readily imagined that not a few administrative organizations are facing this situation. They are especially pertinent in developing nations, in which there is too little attention paid to managing infrastructures, including a vast number of concrete bridges. Situations such as (1) time-series data cannot be used to estimate the transition probabilities because historical intervention records between inspections are unavailable, (2) time series data are not recorded as electronic data, and (3) time-series data in current criteria of states are not available because criteria of states had been changed recently, come under this situation as well. In the situation where there are no sufficient time-series data, infrastructure managers either rely solely on expert opinion or use expert opinion in conjunction with predictions made using mechanistic-empirical models.

In mechanistic-empirical models, deterioration processes are modeled as physical processes that can be described using the characteristics and properties of the materials of the elements and their chemical and physical response/reaction against factors of the environment. Using mechanistic-empirical models, once the values of endogenous and exogenous factors governing the deterioration process are determined, the condition evolution of the bridge element over time can be estimated without the need for inspection data for a minimum of two consecutive time intervals. Unlike Markov models, condition evolution over time is modeled as being continuous (DuraCrete, 1998; Kirkpatricka et al., 2002). When mechanistic-empirical models are used together with expert opinion, it is usually assumed that an element cannot transition more than one state in one time period.

An improvement to the estimation of transition probabilities to be used in infrastructure management systems, and, therefore, an improvement to infrastructure management, would be to have a methodology to be used to estimate transition probabilities using mechanistic-empirical models for use in the second situation. In addition, it is desired that elements are modeled so that they can transition more than one state in one time period. Such a methodology is proposed in this article. The proposed methodology makes use of two approaches: (1) the analytical approach and (2) the Bayesian approach. The first is used in situations where it is possible to find an analytical solution so that the transition probability can be derived directly from the mechanistic-empirical models. The second is used in situations where it is not possible to find the analytical solution. It makes use of Bayesian statistics, which requires the formulation of a likelihood function of the transition probabilities and the use of Markov Chain Monte Carlo (MCMC) simulation.

The remainder of the article is structured as follows: Sections “Finite State Markov Models and Transition Probabilities” and “Estimating Transition Probabilities Using Mechanistic-Empirical Models” include a background to position the contribution of the proposed method. Specifically Section “Finite State Markov Models and Transition Probabilities” contains an overview of finite state Markov models and transition probabilities, and Section “Estimating Transition Probabilities Using Mechanistic-Empirical Models” contains a literature review regarding methods to estimate transition probabilities using mechanistic-empirical models. Section “Mechanistic-Empirical Models” includes the mechanistic-empirical models to be used in the work presented in this article. In the Section “Relationship between Mechanistic-Empirical Models and Transition Probabilities,” it is explained how transition probabilities are to be estimated analytically using mechanistic-empirical models. In Section “Methodology,” the methodology is described, in which, the steps to formulate the mechanistic-empirical model of deterioration process is given along with a way to convert the condition evolution predicted using this model to the discrete states required by the Markov model. An example of how the method is to be used is given in Section “Example.” In Section “Comparison with the State-of-the-Art,” the Markov models developed using the proposed methodology and a state-of-the-art methodology are compared. In the Section “The Number of Deterioration Paths from the Mechanistic-Empirical Models,” the effect of the number of deterioration paths required to obtain a satisfactory result is shown and discussed. Section “Conclusion” contains the conclusions of the work and recommendations for future research.

In finite state Markov models, transition of condition states between time point t₁ and t₂ = t₁ + z is expressed with a transition probability matrix whose i* j (i = 1, …, I;j = 1, …, I) element is a transition probability defined as Prob[h(t₂) = j|h(t₁) = i] = π_ij. h(t) is a function which denotes a condition state at t. π_ij is a conditional probability which indicates the occurrence probability of condition state j at t₂ with given condition state i observed at t₁. In finite state Markov models, it is assumed that the transition probability between time points t₁ and t₂ is only dependent on the condition state at t₁ so as to satisfy the Markov property. Finite state Markov models have been used in the management of deteriorating systems since the 1960s, when there was a rapid development of mechanical and electrical systems (Howard, 1960; Gertsbakh, 2000; Kolowrocki, 2014). They have been used to ensure that to determine optimal intervention strategies, i.e., the strategy to follow to ensure that the costs of executing interventions are balanced with the costs of not executing interventions. Finite state Markov models allowed for the modeling of deterioration processes that could not be perfectly modeled deterministically, to be modeled as stochastic processes. They were, and are, considered to be good models for systems where the transitions from one state to another can be considered to be memoryless. They are less good where this is not the case but are still often used due to both their ease of use and ease of understanding in situations where the assumptions of memoryless do not lead to large deviations in condition evolution prediction from reality. To ensure that a Markov model is developed to give accurate predictions of deterioration using statistical methods, condition state data are required. In general, the longer the time series of inspections, the better.

Finite state Markov models are used in infrastructure management systems to model the deterioration of elements and to determine optimal intervention strategies (AASHTO, 2004; Swei et al., 2015). They are used instead of continuous Markov models due to their convenience, in terms of using visual inspections (it is easier to say that an element is in state 2 than to say if it is in 2.1, 2.2, or 2.3) and in terms of assessing the state of the element that is to trigger an intervention (Howard, 1960; White, 1992; Puterman, 1994). The estimation of the transition probabilities for the Markov models is ideally done using available condition state data that have been collected at uniform time intervals over a long period of time (Lee, 1970). Although somewhat more complicated, when data have been collected at non-uniform intervals of time over a long period of time, the transition probabilities can be estimated using statistical methods, such as survival analysis, maximum likelihood estimation, and Bayesian estimation approaches (Hastings, 1970; Lancaster, 1990; Kobayashi et al., 2012a; Mizutani et al., 2013; Lethanh et al., 2015). When little to no condition state data are available, transition probabilities have been estimated using expert opinion or estimated in various ways to obtain a best fit with the condition states predicted using mechanistic-empirical models (Golroo and Tighe, 2012; Indiana Department of Transportation, 2013).

To simplify the estimation of transition probabilities some researchers and developers of management systems have assumed that it is not possible for an element to move more than one condition state in one time interval, e.g., Jiang et al. (1988), Mishalani and Madanat (2002), and Robelin and Madanat (2007). However, others have explicitly developed models where that is not the case (Tsuda et al., 2006). Being able to estimate transition probabilities with data that have been collected at both uniform and non-uniform time intervals and allowing elements to move more than one condition state per time interval, it has been shown to increase the accuracy of the estimation of the transition probabilities in Markov models (Tsuda et al., 2006; Kobayashi et al., 2012b; Mizutani et al., 2013; Lethanh et al., 2015).

In situations where there are little to no time-series condition state data estimating transition probabilities so that the results of a Markov model fit those of mechanistic-empirical models, using mechanistic-empirical models is likely to yield more accurate deterioration predictions.

However, fitting can be done in different ways. No research has been conducted using an approach to find an analytical solution so that the transition probability can be derived directly from the mechanistic-empirical models. When this approach is unavailable due to relatively complicated functional forms of the mechanistic-empirical models, an approach is available to estimate the transition probabilities using the predictions of condition of the elements with the Markov model and with the mechanistic-empirical models. Roelfstra et al. (2004), which is perhaps the first work in this area, used a restricted least squares approach to minimize the difference between the predictions of the average condition of the elements with the Markov model and the predictions of condition of the elements with the mechanistic-empirical models. They assumed that there was a maximum transition of one state in one time interval. Overcoming this assumption, Lethanh et al. (2017) used a restricted least squares approach to minimize the difference between the prediction of the probabilities of the elements being in each condition state in each time interval within the investigated time period estimated using the Markov model and those predicted using the mechanistic-empirical models. In other words, they minimized the sum of the differences between the elements of the state vectors, which are estimated using the Markov model and the mechanistic-empirical models, respectively. In this process, however, information of the transitions of the states in each element, i.e., information of the transitions from state i to j of an element, is lost by aggregating this information into the values of the state vectors. To prevent this loss of information, in the proposed methodology, a likelihood function of the transitions of the states in each element is formulated, and the transition probabilities are estimated based on the likelihood function. The methodology used by Lethanh et al. (2017) is used as a reference methodology in this article.

Many mechanistic-empirical models can be used to predict the future condition state of reinforced concrete elements. In this work, a mechanistic-empirical model was selected to predict condition states of the element during the initiation phase of chloride-induced corrosion, and another was selected to predict condition states of the element during the propagation phase. The models used were based on those given in DuraCrete (2000). An illustration of the two phases, along with the ranges of chloride concentrations (kg/m³) and crack widths (mm) used to define the condition states, is given in Figure 1.

Chloride penetration was modeled using Fick’s second law of diffusion (Fick, 1855). (1)δCclδt=Dclδ2Cclδx2, where C_cl is the chloride ion concentration at the depth of the reinforcement (here small notation “cl” denotes the abbreviation for chloride); x is the depth; and D_cl is the chloride diffusion coefficient.

The solution for partial differential equation (Eq. 1) gives the following explicit form to calculate the chloride concentration as a function of the distance of the reinforcement from the concrete surface x_cl and time t (DuraCrete, 2000). (2)Ccl(xcl,t)=Cs1−erfxcl2Dclt, where C_s is the surface chloride content and erf[⋅] denotes the error function.

The time required for corrosion to start, that is time t in Eq. 2 was estimated by setting the value of C_cl to be equal to the chloride concentration. Then the value of C_cl is used to determine the entry point into a condition state. In other words, by setting the upper bounds on the values of C_cl to be used to define each discrete condition state i, the time to arrive at that condition state was obtained by solving Eq. 2 with respect to time t and a certain depth of concrete cover from the reinforcement. The value of variables C_cl, C_s, and D_cl in the above equations were considered to be random, with each one being represented with a probabilistic distribution.

After the value of chloride concentration reached a certain lower limit, corrosion of the reinforcement was assumed to start. After it reached a certain higher limit, cracking was assumed to start. The following equations from DuraCrete (2000) were used: (3)w(t)=w0+βP(t)−P0, where w(t) is the crack width (mm) over time; β is the parameter that controls the propagation; w₀ is the crack width when it is visible (≈0.05 mm); P₀ is the amount of loss of re-bar diameter (mm) when the crack width is visible; and P(t) is the amount of loss of re-bar diameter (mm) at time t, which is given by the following equation: (4)P(t)=∫0tVcorr⋅α⋅wet⋅τdτ, where V _corr is corrosion rate coefficient (mm/year); wet is the wet period in a year (equal to the ratio between total numbers of rainy day and 365 days); and α is pitting factor that takes non-uniform corrosion of the re-bars into consideration.

The relationship between the mechanistic-empirical models and the transition probabilities is explained in this section. The two phases of deterioration as illustrated in Figure 1 are expressed as a function of a set of random variable X, where X represents a vector of parameters such as D_cl, V _corr, α, and wet. When the mechanistic-empirical model, which is defined as the general notation of deterioration function y = g(t, x), includes a single random variable x with its probability density distribution f (x) and y = g(t, x) is a monotonic increasing function, the relationship between the mechanistic-empirical model and the transition probabilities can be derived. Here, t is elapsed time. y is an indicator of deterioration, and the value of y becomes larger as the reinforced concrete element deteriorates. The inverse function of y is denoted as x = m(y, t).

As x is a random variable, the value of i is also a random variable. The occurrence probability of observing condition state i is given by the following equation: (5)ξ(i,t)=∫m(yi−1,t)m(yi,t)f(x)e(t)dx, where e(t) is the probability density function of elapsed time t and (y_i−1, y_i] denotes the pre-defined range of y for condition state i.

The probability of observing condition state j at any subsequent time t + Δt is defined as follows: (6)η(i,j,t,Δt)=∫m(y¯,t)m(y¯,t)f(x)e(t)dx, where y¯ and y¯ are the lower bound and the upper bound of y at t, respectively, that function y = g(t, x) passes both ranges (y_i−1, y_i] at t and (y_j−1, y_j] at t + Δt, which are given by the following equations: (7)y¯=max(yi−1,g(t,m(yi−1,t+Δt))), (8)y¯=min(yi,g(t,m(yi,t+Δt))).

Since Markov transition probabilities are constant in the finite state Markov model, it is assumed that t is selected randomly. Consequently, the continuous uniform distribution U(0, t_z) is available as the probability density function of elapsed time e(t), and t_z is enough large number. The Markov transition probability π_ij(Δt) is given by the following equation: (9)πij(Δt)=π(i,j,Δt)π(i), where π(i) is the marginal distribution of i, which is given by the following equation: (10)π(i)=∫otZξ(i,t)dt=∫0tZ∫m(yi−1,t)m(yi,t)f(x)e(t)dxdt, and π(i, j, Δt) is the marginal distribution of (i, j), which is given by the following equation: (11)π(i,j,Δt)=∫otZη(i,j,t,Δt)dt=∫0tZ∫m(y¯,t)m(y¯,t)f(x)e(t)dxdt.

The methodology proposed to estimate the transition probabilities is shown in Figure 2. The first two tasks are required to select the mechanistic-empirical models. Once the models are selected, the values of their parameters are to be defined, as are the condition states to be used to map the continuous values determined using the mechanistic-empirical models to the discrete states used in the Markov model. When transition probabilities can be calculated directly from the mechanistic-empirical models, transition probabilities are determined analytically. When this is not possible, the transition probabilities are to be estimated using the Bayesian approach shown in Figure 2 as the sub-process. The use of the Bayesian approach is explained in more detail in the rest of this section.

The comparison between the proposed methodology and the state-of-the-art methodology (Lethanh et al., 2017) was done by estimating the transition probabilities using both methodologies from the mechanistic-empirical models shown in Eqs 32 and 33 and Table 1, and measuring the differences (or residuals) between the average condition values predicted by using the Markov models and the condition values predicted using the mechanistic-empirical models. Figure 3B shows the state vectors calculated using transition probabilities estimated by the state-of-the-art methodology shown in Table 4. The condition state distributions for each time interval calculated by the mechanistic-empirical models are shown in Figure 3C, i.e., the condition state distributions in Figure 3C are calculated as ratios of condition states at each time point in the generated sample deterioration paths in 7.3. The values of the log likelihood with the transition probabilities estimated using the proposed methodology and the state-of-the-art methodology were −1.0924E+05 and −1.2697E+05, respectively. The proposed methodology can maximize the likelihood function and fit transition probabilities to generated sample paths better than the state-of-the-art methodology. To discuss the results in detail, the differences of sums of squared residuals between the Markov models and the condition values predicted using the mechanistic-empirical models are given in Table 5. This table shows, for example, that the sum of squared residuals in the period between 11 and 20 years was 4.20E−01. It can be seen that the differences between 11 and 60 years using the models developed with the proposed methodology are smaller than the differences using the models developed with the state-of-the-art methodology. This tendency can be seen in the expected deterioration paths shown in Figure 5 as the path of the proposed methodology is closer to that of data than the state-of-the-art methodology after 10 years elapsed.

From Table 5, it can be also seen that the sums of differences over 100 years were 1.88 condition states and 3.46 condition states using the Markov models developed using the proposed and the state-of-the-art methodologies, respectively. From these results, it can be inferred that the Markov model developed using the state-of-the-art methodology overestimates life expectancy and underestimates the speed of deterioration because the result of the state-of-the-art methodology was influenced excessively by the data between 61 and 100 years to estimate transition probabilities π_3j, π_4j, and π_5j.

In the proposed methodology, the accuracy of the Markov models depends on the number of deterioration paths N used in the estimation of the transition probabilities. This relationship is shown in Figure 6 and Table 6 for all transition probabilities for values of N ranging between 1,000 and 10,000 at 1,000 step intervals. In all cases, as above, the following values were used a = 1 (year), c¯=1,000, and Z = 100. The posterior distributions calculated using sampled parameter π₁₁(1) are shown in Figure 7 for N = 1,000 and N = 10,000 as an example. From Figures 6 and 7 and Table 6, it can be seen that the parameter dispersion decreases from 1.16E−02 to 3.39E−03 as N increases from 1,000 to 10,000. As shown in Table 6, all of the other parameters π_ij have the same magnitude correlation to the case of π₁₁ as N increases from 1,000 to 10,000. The credible interval of π₁₅ with N = 4,000 is not shown in Table 6 because all sampled π₁₅ were 0.

It can be seen in the previous table and in Figure 8, which shows the 90% Bayesian credible intervals of life expectancies with N from 1,000 to 10,000 at 1,000 step intervals that the Bayesian credible intervals vary as a function of the number of deterioration paths used. With a threshold of R = 1 (year), where r=T¯5−T¯5, it seen that N is to be at least 3,000 to obtain a satisfactory result. Figure 9 indicates that the required N increases when R decreases. From Figure 9, it can be seen that N has to be greater than 5,000 or more when R = 0.8 (years), and 8,000 or more when R = 0.6 (years), respectively.

In this article, a methodology is presented to estimate the transition probabilities to be used in Markov models from mechanistic-empirical models. The methodology can be used when there is little to no available time-series inspection information, but mechanistic-empirical models are available. The proposed methodology includes the use of analytical solutions where possible and the use of Bayesian statistics where analytical solutions are not possible. When the Bayesian approach is used, the accuracy of the transition probabilities is to be estimated as a function of the number of deterioration paths calculated.

The methodology is demonstrated by using it to estimate the transition probabilities to be used in a Markov model for reinforced concrete bridge elements deteriorating due to chloride-induced corrosion of the reinforcement. It was shown in this example that the proposed methodology was an improvement on the state-of-the-art methodology, as the sum of residuals over 100 years with the proposed methodology was 100*1.88/3.46 = 54% of that of the state-of-the-art methodology, i.e., the proposed methodology gave a 46% point decrease. In addition, the later would have over-estimated the speed of deterioration.

In the work presented in this article, it was assumed that stationary transition probabilities are to be used. When Markov models with stationary transition probabilities are used, certain limitations are imposed which restrict the modeling of the deterioration processes. This is considered to be okay if the sum of residuals is close to 0. If, however, it is found that they do not, the use of non-stationary transition probabilities might provide better results. The use of non-stationary transition probabilities could improve the modeling of deterioration in these circumstances. They could, for example, be used to capture sudden decreases in the states elements. Future work should be focused on the use of non-stationary transition probabilities in bridge management systems. Some preliminary work is that done by Wellalage (2015).

In addition, the proposed methodology has potential of use for many other materials in addition to reinforced concrete, in situations where condition data are not available but mechanistic-empirical models are available. Exactly how this should be done is, for each type of material, a topic for future work.

DM contributed to theoretical formulation, program coding, computation, and preparation of the manuscript. BA, NL, and KK contributed to preparation of the manuscript.

NL is employed by POMPLUS Consulting Ltd. All other authors declare no competing interests.