Figure 1 is the proposed system designed as three multilevel hierarchies to include the top-level, middle-level, and lower-level. The top level consists of a high-speed server; the middle level consists of a moderate-speed server, and the lower level consists of a low-speed server. The primary reason for the hierarchy is that the cost per load (task) is generally proportional to the speed of the server. Figure 2 depicts the strategy for allocating the loads to only available servers and balancing the loads among the servers.

When a request (load) is sent to the hierarchy, the top-level hierarchy is checked to see if there is an available server that can process the request (load). If there is, then the request is completed. If not, the level is checked, with the process repeated until the request is processed. If the top level in the hierarchy cannot handle a request (load), a copy of the request is sent to other levels in the system. The demand-based method used in this study allows all the servers in each hierarchy to be randomly checked to see the available servers that can process the request (load). The level-switching operation was employed to distribute loads evenly across all levels in the system.

The matrix operation shown below is used to trace computations for easier processing. The trace is supported by two theorems that enable quick access to any available server at any level in the hierarchy.

Theorem 1

If A is an n×n matrix, the trace of A, written trace (A) is the sum of the diagonal elements,

Theorem 2

If A is an n×n matrix and B is an n×n matrix, then trace (AB) = trace (BA),

Different levels in the hierarchy have servers of varying sizes. The random check for available servers often allows multiple requests (loads) of different sizes to access all the servers at each level of the hierarchy to determine which servers can process them. This method speeds up request processing time because it eliminates the need to redirect requests (loads) to underloaded or idle servers when an overloaded server is detected.

In this study, the assignment of loads across all server categories (top-level, middle-level, and low-level) is based on server availability, processing and storage capacity, and the size of the new load or request. All these factors are considered to ensure proper and even distribution of cloud users' requests. The study employs an assignment problem model to enhance effectiveness in terms of resource utilization and system response time for each resource or facility handling each load or request. Therefore, the system assigns each resource to a single load or request to optimize performance. Table 2 presents the data matrix for assigning loads to servers or resources.

Let s_ijdenote the assignment of the server to load j such that.

Therefore, the mathematical model for the assignment problem is formulated as follows:

subject to the constraints

and s_ij = 0 or, for all i and j

The assignment model used allowed the system to check each server in all the hierarchies (levels) before the allocation of loads. The assignment operation starts from the top-level hierarchy servers to the low-level servers and returns to the starting point at the top-level hierarchy for the next allocation operation. This assignment method also minimizes the distance covered in terms of the cost of moving from one server to another server, the cost of moving from one level to another level, and the time taken to move around the levels. The assignment of cloud users' request operations continues as shown in the pseudocode.

The SIRO model determines the availability of each server and the service rate of each server since the allocation of cloud users' requests is based on the servers' capacity. In this study, the SIRO model uses Poisson and Erlang distributions.

This study used the Poisson distribution to model the arrival of cloud user requests (loads). This queuing system effectively captures the random nature of user requests over time. The Poisson distribution is defined by the formula in Equation 6:

where P(X−k) represents the probability of exactly k arrivals occurring within a specified period and λ denotes the average arrival rate of users per unit time.

In cloud computing, user requests (like accessing data, running applications, or storing files) arrive randomly over time. The Poisson distribution is widely used in queuing theory and network systems to model random arrivals of loads (in this case, user requests or loads) within a given time frame. This makes it a suitable choice for modeling request arrival rates in a data center.

Therefore, using Poison distribution to calculate the number of loads in each server determines the number of loads that the server is processing presently and the ones that are yet to be processed. This is one of the factors that determines whether the incoming loads will be assigned to the server or not.

The Poisson model simulates how often new user requests arrive in the cloud data center during a time window. If λ = 10, on average, 10 requests arrive every second. The model explains the probability of receiving 5, 10, or 20 requests in the next second. To dynamically monitor the loads, Servers monitor not only how many requests they are currently processing but also how many are expected to arrive soon, based on a Poisson model. This helps determine whether a server is suitable for taking on new requests. If a server is already handling many requests and the Poisson model predicts high incoming traffic, the load balancer may redirect new requests elsewhere, and if the probability of receiving too many requests is high, the system can preemptively avoid overloading that server. The Poisson distribution, when used with queue theory (like M/M/1 or M/M/c models), helps estimate the number of requests currently being processed and the number of requests waiting in the queue. This helps the demand-responsive system adjust assignment probabilities in the data matrix mentioned earlier. The use of the Poisson model contributes to real-time, data-driven decision-making and prevents bottlenecks, ensures fair distribution of loads, minimizes latency and response time, and maximizes server utilization without overload. Therefore, the Poisson distribution in this study is used to model the arrival pattern of user requests. By predicting the likely number of incoming requests per server, the load-balancing scheme can intelligently distribute requests in real time, optimizing performance, avoiding overloads, and ensuring demand-responsiveness across the cloud data center.

The Erlang distribution was employed to improve service time efficiency across all servers within the system. This model is characterized by a single service channel comprising k identical sequential phases, which represent the degree of variability in the service time distribution. In this framework, each server processes tasks through these k phases in series, ensuring consistent service behavior. A new service process begins only after the completion of all k phases for the previous task. Consequently, each new arrival increases the number of active service phases in the system by k. The modeling approach led to the formulation of a system of differential–difference equations to describe the dynamics of task servicing under the Erlangian model.

This study utilized the Erlang distribution to enhance service efficiency within the cloud service system. In the employed model, each task is processed sequentially through k service phases, with a new service initiation delayed until all k phases of the preceding task are completed. As a result, each incoming request increases the total number of active phases in the system by k. This modeling approach led to the derivation of a system of differential–difference equations, which characterizes the service dynamics. The Erlang distribution formula used in this context is presented in Equation 7:

The probability density function for Erlang is given by

Here, x denotes the service time, k is the shape parameter indicating the number of phases, and is the rate parameter. The μ is used to model the number of cloud services that customers can be served per unit of time, then kμ is the number of virtual machines that the system can serve per unit of time, and 1/kμ represents the average time taken by a server at each virtual machine.

Therefore, both Poison and Erlang distributions were used to determine the status of each server, whether the new incoming load will be assigned to a server or not, that is, a server demands and accepts load or rejects the load.

The sequencing technique was used to model the order in which incoming loads l₁, l₂, …., l_n are assigned to a set of servers s₁, s₂, …., s_n and the time t₁, t₂, …., t_n at which the loads are processed. Johnson's sequencing algorithm was used to determine the optimal sequence (Algorithm 1).

The load balancing proposed in this system satisfies the arrangement of a balanced distribution of loads among the servers. In the proposed system, the load balancer is capable of playing both the role of sending requests and receiving requests. This process enables the load balancer to balance the loads among all the servers. This method reduces unnecessary assignment of loads to a very busy server. Mathematically, if this system consists of m servers, and L is the total amount of load in the system, then the system achieves τ-convergence. The complexity and the system convergence are calculated in Equation 8:

This complexity and convergence analysis reduces the system running overhead time, which causes a delay in some studies examined. However, in assignment problem using algorithms like the Hungarian method, introduces a time complexity of O(n3)O(n3), making it computationally heavy but effective in ensuring optimal one-to-one matching between loads and servers. The queuing model, incorporating SIRO with Poisson distribution for request arrivals and the Enlarge distribution for processing rates, provides a realistic and dynamic representation of the system's behavior, with moderate computational overhead per request.

Game theory adds another layer of complexity by enforcing fair and balanced load allocation using mixed strategies. This prevents starvation and load redirection by considering each server's demand and capacity, typically resulting in a time complexity between O(n2)O(n2) and O(n3)O(n3). Additionally, the system's use of real-time server status checks, instead of static index tables, introduces continuous but relatively lightweight overhead for maintaining current server information.

The demand-based dynamic load-balancing scheme used in this study is computationally complex due to the integration of optimization algorithms, real-time data collection, and intelligent decision-making frameworks, as shown in Table 3. While techniques like round-robin and static ML models are far less demanding computationally, this scheme significantly enhances performance, scalability, and fairness in resource allocation. The trade-off lies in higher FLOPs and memory use, but this is justified in high-throughput, cloud-scale environments where load efficiency and fairness are critical.

The study used the game theory technique to incorporate the demand optimization method for each server to decide whether to accept incoming workload or reject it before passing it to another server. This game theory strategy depends on the server's current status, whether idle or busy, and the server's capabilities, such as CPU processing power and memory size. The game theory modeled the interactions among the three levels of servers (top-level, middle-level, and low-level) for predicting load distribution patterns. Thus, the interaction between Server A and Server B can be modeled as a two-player, zero-sum game, where the payoff matrix defines outcomes such as response time and CPU utilization, while the strategies include accept, reject, or demand load. The expected demand justification for every server in cloud computing, with an arbitrary demand justification matrix [a_ij] of order m × n is defined in Equation 9:

Servers B

Server A (a11a12a21a22)

The algebra solution is therefore used to determine the probability that a server demands and accepts loads or rejects the loads. Such a decision is based on its status (idle or busy) and the server's capability.

In game theory, with a payoff matrix[_aij]mxn,

Since Server A demands and accepts loads and expects at least U. Server A is given by Equations 10–13, respectively. That is, Server A would not accept more load unless it expects a minimum acceptable payoff, such as avoiding overload.

Similarly, Server B both requests and accepts loads, expecting at least a utility value of U. Its behavior is described by Equations 14–17. In other words, Server B will only accept a load if it determines that doing so will maintain or improve its performance; otherwise, it will reject the load. This is further illustrated in Table 4.

Therefore, U represents the value of the game corresponding to the minimum expected performance gain that a server aims to achieve to secure its strategy. This strategy focuses on balancing the load and minimizing overload.

The game theory using two players is represented by the algorithm below:

Step 1: players and strategies are defined

Players: player 1 is Server A, while Player 2 is Server B

Server A strategy: either demand and accept or reject the load

Server B strategy: either accept or reject the load

Step 2: payoffs definition for Server A and Server B based on resource usage and load-balancing success

If server A demands and server B accepts→server A benefits (+1), server B loses (−1)

If server A demands and server B rejects→server A loses (−1), server B gains (+1)

If server A rejects (not demanding load), the outcome depends on server B's response:

If server B accepts (idle), maybe a wasted opportunity (A: 0, B: 0)

If server B rejects, too, no effect (A: 0, B: 0)

Step 3: Build Server A's Payoffs.

This study deployed a demand-based dynamic load-balancing scheme with optimization techniques, which include assignment problems for the proper distribution of loads to available servers at all levels of servers. Each resource was distributed to one load or request based on the capacity of the server, and the resources requested were effective with no redirection of loads. The systems were optimized using queue theory and game theory. The SIRO model with Poisson distribution was used to calculate the rate at which loads are being sent to each server, while the enlarge distribution was used to calculate the rate at which each server can process cloud users' requests and complete a given request. The queue model allows the load balancer to quickly gather information about the status (available or busy) and processing speed of each server, instead of some of the existing studies that used an index table as a source of information. The game theory provides robust dynamic load balancing, which ensures the distribution is based on resource demand using mixed strategies. The processing speed of the servers had to be determined since every server cannot have the same processing speed, and loads cannot be assigned to servers that have not requested them. This prevents the redirection of loads for underloaded or idle servers. The game theory disallowed priority to any server to prevent starvation of low-priority servers and to limit the number of load failures in the system.

The system was simulated using load-balancing data from the GoCJ dataset for Distributed and Cloud Computing Infrastructures. Python codes were written to import the extracted datasets from Google datasets (Kaggle), and functions were written for the dataset. In this research, six relevant features were extracted as follows: load size, CPU, memory, network latency, disk I/O, connections, and response time. Of these, five features, excluding response time, were utilized as independent variables to serve as performance predictors. The system's response time, measured in milliseconds (ms), was treated as the dependent variable and used to evaluate the performance of the demand-responsive load-balancing system. CPU usage was measured in percentage terms, while all other variables were evaluated using appropriate units aligned with cloud system performance metrics. The system generates the results based on the techniques used in this study. The load-balancing results were captured, converted to frames, and presented in Table 5. The techniques were simulated with the number of loads, remote servers, and load balancers.

In Table 5, the simulation results show that a huge number of loads (requests) were coming concurrently from the cloud users, and the processing time was minimal based on the robust techniques that were used in the study. A load balancer was used to randomly assign and balance the incoming requests (loads) among the remote servers based on their usage and demand for incoming requests. Due to cloud users' requests that were coming concurrently, there was little traffic, which was managed by queue theory and game theory. The results of the simulation depicted the system response time, which showed that the technique used performs better than some of the existing systems that redirect loads from overloaded servers to underloaded or idle servers and studies that used priority-based techniques. The assignment technique also achieved a high degree of even distribution when it was tested by assigning different numbers of loads to a certain number of virtual servers. The processing time (response time) that was achieved per server is minimal, and the response time is not up to 20 ms, which was achieved by some studies. The system response time is shown in the output results, which are depicted in Figure 3, while other load-balancing results captured from the implementation were depicted in Figures 4–6.

In this study, a simulation-based approach was adopted to evaluate the effectiveness of a demand-responsive load-balancing scheme for optimizing cloud data center performance. The simulation was built upon real-world data extracted from the GoCJ dataset, which contains trace-level job and resource allocation data from distributed and cloud computing infrastructures. This dataset provided a realistic basis for modeling how user requests (or loads) are processed within a cloud environment, making it ideal for performance evaluation and optimization testing.

Python was chosen as the primary tool for implementing simulation due to its extensive support for data handling, statistical modeling, and simulation development. The extracted data from the GoCJ dataset were first imported into the Python environment, where they were preprocessed to isolate relevant load-balancing attributes. These attributes included load size in megabytes (MB), CPU usage as a percentage, memory usage in megabytes, network latency in milliseconds, disk I/O in input/output operations per second (IOPS), and the number of active connections. These parameters were then used to model the behavior of cloud servers under varying loads.

Within the simulation, specific Python functions were developed to model the arrival of user requests based on a Poisson distribution. The Poisson model was particularly important because it effectively captures the random and unpredictable nature of user behavior in cloud environments. By modeling request arrivals as a Poisson process, the simulation could account for fluctuations in demand and dynamically adjust load-balancing decisions accordingly. This stochastic modeling allowed the system to estimate the arrival rate of new loads per time interval, thereby informing the load balancer about potential surges or drops in request traffic.

Once the simulated loads were generated, the system distributed them across a network of virtual servers using the demand-responsive load-balancing logic. This decision-making process relied heavily on real-time resource monitoring, particularly on factors like current CPU load, memory pressure, and latency metrics. The goal was to assign each new load to the server most capable of handling it efficiently, thereby minimizing response times and avoiding bottlenecks.

To evaluate the effectiveness of the load-balancing strategy, the response time for each load was calculated using a composite function that considered all six key performance metrics (CPU usage, memory demand, load size, network latency, disk I/O, and number of connections). Each server's performance was evaluated based on its ability to minimize the response time given its current load and resource availability. These response times were then captured, converted into visual frames, and tabulated using Microsoft Excel for comparative analysis. The final response time results for a sample of 10 servers were presented in Table 5.

In Table 5, the data provide a detailed view of how different servers handle loads of varying intensities and resource demands. For instance, Server_3, which had a relatively low CPU usage (33%) and minimal network latency (16 ms), demonstrated a fast response time of 11 ms, despite processing a large load size of 9,378 MB. In contrast, Server_4, with a much higher CPU usage (88%) and memory demand (30,428 MB), yielded a higher response time of 19 ms. This suggests that CPU saturation and memory overload have a more significant impact on performance than load size alone.

Interestingly, Server_7 recorded the lowest response time of 10 ms, even though it experienced high network latency (483 ms). This anomaly can be attributed to its extremely low CPU usage (10%) and a moderate number of connections, suggesting that network latency, while important, can sometimes be mitigated by strong server-side processing capabilities. On the other hand, servers like Server_5 and Server_6 demonstrated average response times, despite having varied metrics, indicating that balanced resource usage across CPU, memory, and I/O yields better stability in performance.

These results support the central hypothesis of the study that demand-responsive load balancing leads to improved performance by dynamically assigning loads based on the real-time state of each server. By incorporating the Poisson-based arrival model, the system could anticipate and react to varying workload patterns, thereby ensuring that no server became a performance bottleneck. This proactive and adaptive approach is what differentiates demand-responsive systems from static or round-robin load balancers, which do not account for real-time resource constraints or request behavior.

In summary, the simulation demonstrated that incorporating real-world cloud job data, statistical load modeling using the Poisson distribution, and resource-aware load-balancing strategies can significantly improve cloud data center performance. The response time outcomes, as shown in Table 5, validate the effectiveness of the demand-responsive load-balancing scheme, highlighting its ability to manage resource contention, avoid overloads, and minimize latency across distributed systems. Such findings stress the practical applicability of intelligent load balancing in real-world cloud infrastructure, making it a vital component for ensuring service efficiency and user satisfaction in modern computing environments.