The HIS platform connects individuals, objects, and their informational resources in both fixed and mobile environments through high-speed communication networks (24). This connectivity effectively coordinates the allocation of anti-epidemic resources. The online matching problem primarily involves pairing suitable resources from suppliers with recipients affected by the epidemic. The HIS allocates anti-epidemic resources among recipients and suppliers according to online matching rules, aiming to fulfill the needs for these resources as quickly and broadly as possible (Figure 1). In the pursuit of maximizing resource matching, the deployment time of anti-epidemic resources emerges as a critical influencing factor that plays a vital role in effectively managing the epidemic. Therefore, we concentrate on maximizing the matching of anti-epidemic resources while minimizing the total waiting time.

The recipient i ∈ D and the resource supplier j ∈ S dynamically arrive at HIS in a time series order. Recipient i can be represented by a quintuple i = (id_i, loc_i, dr_i, requesttime_i, q_i), where id_i indicates a specific identifier, loc_i indicates the location, dr_i ∈ {high, low} indicates the epidemic prevention level, requesttime_i indicates the request time, and q_i indicates the required quantity. Supplier j is represented by j = (id_j, loc_j, sr_j, requesttime_j, q_j, l_j), where id_j indicates a specific identifier, loc_j indicates the location, sr_j ∈ {1, 2, 3} indicates the supply level, requesttime_j indicates the supply time, q_j indicates the quantity supplied, and l_j indicates the largest geographical range that j can be supplied.

Anti-epidemic resources are categorized into both reusable and non-reusable types. The resource matching set M encompasses all matching responses requested by both the supplier and the recipient. Each response can be represented by a quintuple (i, j, q_ij, τ_ij, rate_ij), where i indicates the request from the recipient, j indicates the supplier capable of providing resources to i, q_ij indicates the quantity of resources that j can provide to i, τ_ij indicates the waiting time required for i to j, and rate_ij indicates the resource matching rate between j and i.

The foundational model is articulated as follows.

The objective (1) of the model is to maximize the total quantity of resource matching. Constraint (2) ensures that each recipient i can obtain anti-epidemic resources that do not exceed their demand during the decision period t. Constraint (3) ensures that the supplier j can provide anti-epidemic resources without exceeding their maximum quantity. Constraint (4) indicates that the supplier j must provide resources to the recipient i within their service radius. Constraint (5) prioritizes the recipient iwith a high level of epidemic prevention. Constraint (6) prioritizes suppliers j with a low supply level. Constraint (7) indicates that both the recipient i and the supplier j must arrive at the platform within the matching time window t. Constraint (8) ensures that the quantity of anti-epidemic resources allocated by suppliers to recipients is a non-negative integer. Constraint (9) indicates whether the supplier j allocates resources to the recipient i.

The MSODBM is a temporal and spatial dynamic decision-making problem characterized by unpredictability, major outbreaks, and limited information. The arrival of recipients and suppliers in the HIS is marked by uncertainty regarding both timing and quantity. The decision-making process for matching anti-epidemic resources is not only influenced by the current state of events but also has implications for the progression of future events. Given that the number of recipients and suppliers participating in the matching process, as well as the quantity of requested resources, varies across different matching time windows, the average wait time and the average matching rate serve as critical indicators for evaluating the outcomes of matching.

The total waiting time τ_i consists of waiting for the matching time τibatch and the travel time τitravel. τibatch refers to the difference between the match time and the request time for anti-epidemic resources. i.e., τibatch=matchtimei-requesttimei. τibatch is influenced by the capacity of resource supply, the selection of time windows for platform matching, and the matching strategy employed. τitravel represents the routing time following the allocation of anti-epidemic resources. Assuming a constant speed, τitravel is determined by the distance between the supplier and the recipient.

The average wait time avg_τ is the ratio of the total waiting time for matching to the number of requests made by all recipients within the matching time window. It can be calculated by the Equation 10.

The average match rate avg_rate is the ratio of the total completion rates of all recipients to the number of recipients within the specified matching time window. The average match rate can be calculated using Equation 11.

Where |M| indicates the total number of recipient i which participate in t; r_i(t) indicates the matching completion rate of the recipient i which participate in t, as shown in Equation 12.

Choosing the appropriate matching time window significantly influences the waiting time experienced by resource recipients. There are two primary types of matching time windows: instant matching and batch matching (25). While instant matching can minimize individual waiting times, it may not always be the most optimal solution. Conversely, batch matching enhances the overall matching efficiency of the HIS platform (11). The size of the batch matching time window is also crucial. This study examines the fixed time window batch matching strategy (FTWBM) and the adaptive time window batch matching strategy (ATWBM).

In the FTWBM strategy, both the recipient and the supplier arrive randomly at the HIS platform over a time series. During a specified time period T, the HIS platform divides this period into n matching time windows of equal duration (t₁, t₂, ⋯, t_n). The size of each matching time window is bs. The HIS platform collects requests from recipients and suppliers within each matching time window and organizes resource matching at the conclusion of the window. For instance, if the time period is set to 24 h (T = 24h) and the matching time window is 10 min (bs = 10), a total of 144 fixed matching time windows can be generated. This implies that 144 resource matching tasks must be executed within a single day. Simultaneously, the HIS platform seeks to maximize the match within each time window.

Unlike the FTWBM, the ATWBM minimizes waiting time by automatically selecting a series of matching window sizes based on a specific strategy. The HIS platform also divides the time period T into n matching time windows (t₁, t₂, ⋯, t_n), although the sizes of these windows may vary. Before each match begins, the HIS platform continues to gather requests from both recipients and suppliers to create a matching task set. Once the matching process is complete, the successfully matched recipients and suppliers are removed from their respective sets. In the subsequent matching time window, new requests from recipients and suppliers may arrive, or some recipients and suppliers may depart. Consequently, the recipient set D and the supplier set S are updated. The ATWBM dynamically selects an appropriate matching time window based on the feedback reward.

The HIS platform utilizes batch matching, which allows resource suppliers and recipients to wait for a specified period. This process can be modeled as a Markov decision process. In this framework, the agent continuously interacts with the environment, receiving the current state from it at time t. Based on this state, the agent selects an action A, which then influences the environment, resulting in a reward R_t and a new state being returned to the agent. The Markov sequence stipulates that the generation of the next state depends solely on the current state. Specifically, when new suppliers or recipients arrive, two states—holding and matching—can be adopted. The action involves selecting a matching time window. Once the matching time window is determined, a matching timestamp t is generated, allowing the matching action to proceed. If requesttime_i<t and the matching state is not a complete match (rate_i < 1), the recipient i is included in the recipient set D_t. If requesttime_j<t and the matching state is not a complete match (rate_j < 1), the supplier j is included in the supplier set S_t. Those that do not meet the matching conditions will remain holding in the buffer (see Supplementary Figure S1). Meanwhile, the elements in the recipient and supplier sets are sorted according to time, respectively.

This study addresses the problem of Markov decision processes with batch matching by employing reinforcement learning to select adaptive time window sizes. Reinforcement learning involves mapping the relationship between learning states and behaviors, focusing on how to make decisions under uncertain conditions to maximize benefits (26). The multi-armed bandit (MAB) problem is a classic reinforcement learning challenge in which a system contains k arms, each yielding a different reward when selected. Decision-makers initially lack knowledge of the rewards associated with each action but gradually accumulate information through interactions with the system over a time period T. The objective of the decision-maker is to maximize total revenue within T. Exploration and exploitation are two fundamental concepts in reinforcement learning. Exploration refers to the strategy of trying various actions to discover the maximum reward, while exploitation involves selecting an action that is known to yield a substantial reward. The MAB framework gathers information by exploring different actions and uses it to maximize rewards through exploration-exploitation trade-offs. Additionally, the MAB is a key component of stochastic scheduling and is widely applied in fields such as online advertising, recommendation systems, and crowdsourcing tasks (27).

Adaptive time window batch matching addresses the mechanism of dividing matching time windows within a model to achieve the optimization goal of maximizing matches while minimizing waiting time. Since the rewards associated with specific matching time windows are not known in advance, the HIS platform interacts with the environment through exploration and exploitation strategies. Candidate matching sets are formed by selecting various batch matching time windows, after which the performance of the current state is observed, allowing for dynamic adjustments in subsequent selections. Through this process, the matching time window that yields the best performance is identified and selected to optimize the average waiting time. The MAB algorithm can be employed to solve these types of sequential decision-making problems. The division of matching time windows can be modeled as an MDP, where the current state and actions directly influence the next state. The primary challenge lies in selecting the appropriate matching window at the optimal time to maximize benefits. Equation 13 utilizes the average waiting time R_t(a) of the matching time window t to calculate a score and update it accordingly.

Where a_ic indicates the average waiting time of the matching time window t for the c-th selection; c indicates the total number of times the matching time window t has been selected.

This study selects the matching time window based on the classic ε-greedy algorithm (26). The sizes of the matching time window set A = {bs₁, bs₂, ⋯, bs_k} are utilized as the k-arms of MAB. As illustrated in Algorithm 1, a matching time window t_i is randomly selected with the probability of ε(0 < ε < 1) to explore, while the time window with the highest current reward is chosen to exploit with a probability of 1−ε. Whenever the matching of t_i ends, the revenue of t_i is updated. To facilitate the rapid learning of the online version, this study introduces a continuous policy improvement interval K_θ and the decay coefficient λ ∈ (0, 1) of ε_k. When K_θ = 0, each time the online algorithm performs a match, the matching time window is updated to initiate a new round of matching, which is entirely optimistic (28). When K_θ>1, the algorithm executes several matches using the current strategy before refining it, which is considered partially optimistic. Furthermore, as the number of matches increases, ε_k typically decreases. The algorithm will employ the current strategy with a higher probability (29). Consequently, Equation 14 establishes the initial exploration probability ε₀, and each match decays exponentially according to the decay rate λ.

The supply-demand bipartite matching network is based on the service radius of suppliers. Equation 4 demonstrates that each supplier j is constrained by their service scope, with the maximum service scope being defined as l_j. In the bipartite network G<U, V, E(U, V)> consisting of suppliers and recipients, where U = D_t, and V = S_t, calculate Manhattan distance l_ij between the recipient i and the supplier j. According to the Equation 4, if l_ij ≤ l_j, there is an edge between i and j, and the distance is used as the weight of the edge, that is, E(i, j) = l_ij. if l_ij>l_j, there is no match between i and j, and set E(i, j) = −1.

The nearest neighbor first (NNF) strategy considers the distance between suppliers and recipients by incorporating distance-based priority matching constraints. Equation 15 prioritizes the allocation of anti-epidemic resources from suppliers to recipients that are closer in proximity, thereby maximizing matching while minimizing τitravel. Let L_i represent a set of l_ij, which includes all suppliers that can be matched with recipients. Select min(l_ij) as the prioritized matching for recipients (see Supplementary Figure S2).

Multi-stage bipartite graph matching is predicated on the priorities of suppliers and recipients. Based on the recipient's anti-epidemic priority attribute dr and Equation 5, the bipartite network G<U, V, E(U, V)> is partitioned into G_PH<U, V, E(U, V)> with high-priority recipients, and G_PL<U, V, E(U, V)>with low-priority recipients (see Supplementary Figure S3). Subsequently, G_PH<U, V, E(U, V)> and G_PL<U, V, E(U, V)> are divided into three stages according to the supplier's level sr = { 1, 2, 3 } , respectively (see Supplementary Figure S4). As shown in Algorithm 2, the set U matches the set V with sr = 1 in the first stage. If the matching is not fully completed, the second stage of matching is performed, wherein set U matches set V with sr = 2. Similarly, the third stage of matching is conducted until the matching is successfully completed. If, at any stage, set U completes matching, the process is terminated.

Prioritizing the allocation of resources to areas that have been significantly impacted by the epidemic can aid in containing outbreaks within these areas and in preventing further transmission. In the process of matching anti-epidemic resources, recipients with higher priority levels are given precedence in the matching procedure. The Greedy-based many-to-many bipartite matching algorithm with capacity (see Algorithm S1) operates under the assumption that the current high-priority recipient will consistently be matched with the most advantageous supplier, irrespective of the matching decisions made by other recipients.

For the set of recipients with low priority, the maximum matching is determined using the classic Hungarian algorithm, a combinatorial optimization algorithm for solving task allocation problems in polynomial time. This algorithm identifies the maximum matching by continuously locating augmenting paths to increase the number of matching edges and vertices. It effectively addresses task allocation challenges to optimize resource utilization and is widely applied across various fields involving resource matching problems. The classic Hungarian algorithm primarily addresses assignments involving a single task or resource, without accounting for capacity constraints. Conversely, the allocation of anti-epidemic resources must consider the quantity of resources allocated, presenting a matching problem with capacity limitations. Therefore, this study incorporates capacity attributes into the vertices of bipartite graph networks (see Algorithm S2) and defines a recursive function getPath(i, q_i) to identify augmenting paths. Additionally, the classic Hungarian algorithm is designed to solve one-to-one matching problems. However, there exists a mismatch of anti-epidemic resources between recipients and suppliers, resulting in a many-to-many matching problem. Consequently, it is essential to extend the classic Hungarian algorithm to accommodate many-to-many matching, which can be achieved by enhancing the augmenting path search mechanism.

This study delineates the findings of an experiment conducted utilizing the proposed algorithm on synthetic datasets. The decision period is established at |T| = 120. Within this timeframe, recipient requests and supplier resources are introduced randomly in chronological order, generating corresponding data based on the characteristics of both resource recipients and suppliers. The anti-epidemic level (dr = {high, low}) of the recipients is randomly assigned in accordance with the demand response, while the suppliers' levels (sr = {1, 2, 3}) are generated randomly based on the supply response. In consideration of the availability of anti-epidemic resources, the ratio of the quantity of resources requested by the recipient to those supplied by the supplier is approximately set at 0.8. A total of 3,000 sets of recipient requests and 1,800 sets of supplier resources are generated randomly.

The FTWBM experiment selects any matching time window from the BS for testing, while the ATWBM experiment randomly selects a matching time window t with a probability of ε_k for exploration. The remaining time window is then selected for exploitation with a probability of 1−ε_k. To facilitate rapid learning in the online version, Table 1 establishes the decay coefficient λ for ε_k and the continuous strategy improvement intervals K_θ.