The multicast communication system consists of satellite part and terrestrial part as Figure 1. The satellite network is composed of hundreds of LEO satellites. Each satellite establishes communication links with the nearest satellites in adjacent orbits and adjoining satellites in the same orbit, thus a mesh satellite communication network is built in which each satellite can communicate directly with four satellites in its neighborhood. The terrestrial network consists of ground stations and user terminals. In the multicast communication system proposed in this paper, the communication paths can be divided into three parts: satellite routing paths, terrestrial routing paths and satellite-terrestrial communication links. The satellites connected to the terrestrial network are selected according to the longest visual time criterion. When the terrestrial users request the multicast data streams, the satellites send multicast data and routing topology status information to the terrestrial macro base stations, then the terrestrial macro base stations send multicast data to mobile users of the multicast group through terrestrial routing paths.

The structure of terrestrial networks is similar to the small cell networks (SCNs) (Andrews et al., 2014). SCNs deploy short-range, low-power, and low-cost small cell base stations (SBSs) operating in conjunction with macro base stations. The SBSs encompass microcell base stations, picocell base stations, and femtocell base stations. They are designed to reduce the distance between users and base stations, enhance network coverage and improve spectral and energy efficiency. The terrestrial network proposed in the paper consists of macro base stations, microcell base stations, and mobile users. The transmitting power of macro base stations is generally more than 10 W and their coverage range is more than 200 m. While for microcell base stations, the transmitting power is 500 mW-10 W and the coverage range is between 50 and 200 m. Connected with satellites, macro base stations receive the multicast data streams and forward them to microcell base stations. After receiving the multicast data streams, microcell base stations forward them to the users. Mobile users are low-speed pedestrians requesting different multicast contents. In this paper, the stochastic model of terrestrial nodes’ location distribution is based on the Poisson point process (PPP) (Saha and Dhillon, 2020). The integrated satellite-terrestrial network has the characteristics of large scale, complex, high dynamic and heterogeneousness. Adopting traditional routing methods in the integrated satellite-terrestrial network will cause problems of poor scalability and large routing overhead.

A Software-Defined Network decouples the control plane of switches and routers from their data plane, enabling the control and orchestration of satellites and terrestrial devices from a central entity. The controller is the core of the SDN, which manages a wide range of data plane resources. Based on the SDSN model proposed by the scholars (Zhu Tang et al., 2014), our SDN deployment model of the satellite-terrestrial hybrid network is proposed as Figure 2. The SDN controller is set as the controller layer located at the ground station, and a Network Operations and Control Center (NOCC) is set as the application layer. The infrastructure layer consists of satellites and terrestrial devices, which perform the basic forwarding functions of multicast communications. The SDN controller can abstract the topology and resources of the network and provide them through the Northbound interface to applications, thus the NOCC can use the network resources more efficiently.

The network is modeled as a graph G = { V , E } , where V and E denote the set of nodes and links, respectively. There are three kinds of nodes: satellite V s , ground station V g , mobile user terminals V t . Let some random satellites be the multicast sources. Each user can join or leave the multicast group i according to the multicast content at the beginning of each time slot t. Let w ( e ) denote the distance cost of the link e ϵ E . The capacities of node v and link e is c ( v ) ,  v ∈ V and c ( e ) , e ∈ E , which denote the number of forwarding entries that can be stored in node v and the maximum data rate allowed for link e .

To address the trade-off between the bandwidth consumption and rerouting overheads, the formulation is based on the Scalable Multicast Traffic Engineering (SMTE) problem (Huang et al., 2016). In time slot t , let T i t denote the multicast tree for multicast group i, and let p i , v t be the set of outgoing downstream paths from node v on the tree T i t . Let Υ i , v t denote whether node v is the node of the multicast group i , and ω i , v t denote whether node v has the same outgoing downstream paths on the tree T i t and T i t − 1 in two adjacent time slots as follows. γ i , v t = { 1 , ( v ∈ T i t ) 0 , ( v ∉ T i t )            ∀  v ∈ V (1) ω i , v t = { 0 ,   ( v ∈ T i t   and   p i , v t ≠ p i , v t − 1 ) 1 , ( v ∈ T i t  and   p i , v t = p i , v t − 1 )          ∀  v ∈ V (2)

In multicast group i, let r ( i ) be the data rate of the multicast source S i , and ε i , e t denote the cost of packet duplication on link e ∈ E in time slot t, which is the number of copies of the packet. The constraints of the nodes and links in the whole network can be described as follows. { ∑ i ( γ i , v t + γ i , v t − 1 − γ i , v t ⋅ γ i , v t − 1 ⋅ ω i , v t ) ≤ c ( v ) ∑ i ( r ( i ) ⋅ ε i , e t ) ≤ c ( e )       ∀  v ∈ V  and  ∀  e ∈ E  (3)

To examine the advantages of the proposed algorithm, let the multicast cost, the average delay of users and the congestion ratio be the simulation objects.

For each multicast tree T i t in time slot t, the multicast cost includes the tree link cost w ( T i t ) , the update cost b ( T i t ) and the transition cost h ( T i t ) , which are defined below. w ( T i t ) = ∑ e w ( e ) ⋅ ε i , e t             ∀ e ∈ T i t (4) b ( T i t ) = ∑ v ( γ i , v t − γ i , v t ⋅ γ i , v t − 1 ⋅ ω i , v t )           ∀ v ∈ T i t (5) h ( T i t ) = ∑ v γ i , v t ⋅ γ i , v t − 1 ( 1 − ω i , v t )         ∀ v ∈ T i t   (6)

The tree link cost w ( T i t )   denotes the whole size of multicast trees that combines their link lengths and packet duplications. The update cost b ( T i t ) represents the cost of rerouting when some existing nodes and new nodes need to add new rules. The transition cost h ( T i t ) represents the cost of storing the old and new rules in some existing nodes of multicast trees when the outgoing downstream paths of these nodes change. Then the total multicast cost is defined below. W = ∑ t ∑ i ( w ( T i t ) + σ 1 ⋅ r ¯ ⋅ b ( T i t ) + σ 2 ⋅ r ¯ ⋅ h ( T i t ) ) (7) where σ 1 and σ 2 are the parameters to differentiate the importance of tree link cost, update cost and transition cost. The values of σ 1 and σ 2 can be any number greater than or equal to 0 and they depend on the weight of the three costs according to the actual network. r ¯ is the average of link lengths in the whole multicast systems.

Example 1. Figure 3 represents an example to explain the multicast cost in this paper with one source A and four destinations H1, H2, H3, and H4, where σ 1 = σ 2 = 1 and link lengths are specified beside each link. Figure 3A shows that in time slot t, only H2 is there in the multicast group i, thus w ( T i t ) = 1 + 2 + 5 = 8 . Figure 3B shows that in time slot t+1, H1, H3, and H4 join the multicast group. The node D is a new switch in the multicast tree. Since the outgoing downstream paths of node B, C have changed, they need to update the restoring rules. Therefore, w ( T i t + 1 ) = 1 + 2 + 3 + 4 + 5 + 6 + 7 = 28 ,   b ( T i t + 1 ) = 3 (containing B, C, D), h ( T i t + 1 ) = 2 (containing B, C). The average of link distances in the multicast tree is 4, then the total multicast cost in time slot t+1 is W = 28 + 3 × 4 + 2 × 4 = 48 .

The events of the members joining and leaving the multicast group are Poisson processes. Let α ( t e ) and β ( t e ) denote the number of users who send reports to join or leave the multicast group in the interval [ 0 , t e ] , respectively. Since the delay in content acquisition has an impact on the routing path selection and congestion ratio calculation, the queuing theory is used to evaluate the average waiting time of users. The input and output node processes traffic in one direction and according to the Markov model, their previous statuses are not dependent. The service process of base stations is abstracted as an M/M/1 queuing system (Leila et al., 2011). According to the queuing system model, the average arrival rate in the interval [ 0 , t e ] is calculated as follows. λ = λ t e = α ( t e ) + β ( t e ) t e (8)

The service time of the RP switch follows an exponential distribution and let the average service time be 1 μ t e . Since the arrival rate λ and the service rate μ are constants at any moment in the queuing system, they are defined as follows. λ = λ t e = α ( t e ) + β ( t e ) t e (9) μ = μ t e (10)

Then the average queue waiting time for users is T s = λ μ ( μ − λ ) (11)

Finally, the average delay of users in the whole multicast system is T ¯ = T s + 1 μ + T r ¯ (12) where T r denotes the average propagation delay in the whole network.

The bandwidth occupancy ratio ξ v of node v is defined as the ratio of current bandwidth usage to the bandwidth capacity of the node v . ξ v = b v c ( v ) = ∑ i = 1 n r ( i ) c ( v ) (13) where n denotes the number of multicast trees containing node v . Then the congestion ratio η is defined as the percentage of nodes with a bandwidth occupancy ratio above R among all nodes. R is the parameter to determine the congestion state of node v according to ξ v . The definition is as below, where num ( V ) is the number of all nodes in the whole system. η = ∑ v τ v num ( V ) (14) τ v = { 0 ,      ( ξ v ≥ R ) 1 ,      ( ξ v < R ) (15)

Given a network topology G = { V , E } , the node constraint c ( v ) and the link constraint c ( e ) , the streaming rate r ( i ) of multicast source S i , the parameters σ 1 and σ 2 , the time interval [ 0 , t e ] , the number of users joining multicast groups α ( t e ) and the number of users leaving multicast groups β ( t e ) , the average service time of switches 1 μ t e , we aim to 1) build a dynamic multicast system with low latency for satellite-terrestrial hybrid networks, 2) minimize the multicast cost by balancing the rerouting frequency of satellite and terrestrial networks, and 3) decrease the congestion ratio by using bandwidth sensitive adaptive routing calculations.

Since PIM-SM protocol can effectively reduce the routing states and has stronger scalability, the multicast trees are initially constructed based on PIM-SM protocol. Constructing a multicast tree first requires to determine the location of the Rendezvous Point (RP), and the RP selection problem is an NPC problem that requires a heuristic algorithm. The previously proposed methods are broadly classified into two directions: one is to make a selection in a given finite set of candidates, such as LAMA, and the other is to search the optimal solution using the greedy algorithm. In order to optimize the multicast delay, our algorithm uses the idea of the Tabu search (Prajapati et al., 2020) for selection of RPs, and the simulation is compared with the LAMA algorithm’s. The steps to calculate the RP are as follows.

1) Set a forbidden table H to store the local optimal RP-selected solutions. Randomly select an initial node as xnow , and let f ( xnow ) denote the evaluation value of the node, which is the sum of the distance from multicast source S i to xnow and the distance from xnow to each other user node v in multicast group i . f ( xnow ) is defined as f ( xnow ) = D ( S i , xnow ) + ∑ v D ( xnow , v )    ∀ v ∈ H i . Set the maximum number of iterations N .

In order to optimize the rerouting strategies suitable for both satellite and terrestrial networks, a two-layered shared tree structure is used to determine RPs and construct shared trees in both satellite and terrestrial networks. Firstly, at the beginning of each time slot, we set the connection status of the whole network links and choose the satellite connected to the terrestrial networks through the longest visual time criterion based on terrestrial macro base stations. In the remainder of this paper, we use the terminology Feed to denote the satellite connected to the terrestrial network. Then we set that feed as the multicast destination node of the satellite network to calculate the satellite RP. And Dijkstra’s algorithm is used to establish the shortest paths from the satellite multicast sources to the RP node, as well as the shortest path from the RP to that feed, thus the multicast tree of the satellite network is constructed. In the terrestrial network, the feed is used as the source node to determine the RP node of the terrestrial network. Similar as the satellite network, the Dijkstra algorithm is used to construct the multicast tree from the source node to all multicast group users.

With multicast users joining or leaving multicast groups dynamically, in order to improve the multicast quality and relieve the centralized pressure of RP nodes, when the RP of a multicast group reaches the node capacity limit, the new users will become a new multicast group. In other words, when a new user requests the multicast content, the multicast group is chosen based on its request content in the first. Then at the time slot of joining, determine whether the terrestrial RP of the multicast group has reached the capacity limit, if not, the user communicates with the terrestrial RP and establishes shortest paths to it, and if it has reached the node capacity limit, the multicast group recalculates the multicast tree to ensure the smallest total delay of the multicast group, and the multicast group will not accept the new user. The user will become a new multicast group, then a new terrestrial RP node is calculated based on the new multicast group to build a new multicast tree.

In the previous research, the routing path of the new user joining or leaving a multicast group has two solutions: one is to recalculate routes based on the new topology, which is defined as full-rerouting, and the other is to directly add branches, pruning or local reconfiguration of the multicast tree, which is defined as partial-rerouting. Compared to the complexity and high dynamics of terrestrial networks, the downlink of satellite networks is more fixed. However, the satellites have higher link latency and fewer onboard resources, which can lead to a great waste of resources if the satellite network and terrestrial network are always allowed to reroute at the same time when a large number of users join or leave multicast dynamically.

Therefore, we propose a cost-aware rerouting algorithm. After constructing the multicast tree according to the previous algorithm, when the handover of the Feed occurs, the total multicast cost W 1 of full-rerouting system and the total multicast cost W 2 of partial-rerouting system are calculated according to the formulations in 3.3. Full-rerouting refers to reconstructing multicast trees of the satellite network with the new Feed as the destination node, while partial-rerouting refers to the new Feed directly establishing a link with the previous Feed without changing the basic structure of the original multicast trees. If W 1 > W 2 , partial-rerouting is selected, and vice versa, full-rerouting is selected.

In the satellite-terrestrial hybrid system, due to the wide coverage of satellites, the uplinks and downlinks connecting satellite networks and terrestrial networks are more fixed for a period of time. Meanwhile, some nodes connected to these fixed links are usually at the central part of multicast transmission paths, thus many multicast paths contain these nodes, which we call hot nodes. Due to the long delay and high packet loss ratio of satellite links, when congestion occurs in these hot nodes, it will reduce the transmission quality of the multicast system much more seriously than other nodes. Therefore, in this paper, a load factor is introduced to weight the real-time bandwidth occupancy of the network topology links to achieve joint optimization of delay and congestion ratio during routing calculation. Bandwidth occupancy ratio is the ratio of the node’s occupied bandwidth to the bandwidth capacity. The load factor is β . For each network node v , the weight of its neighbor node i as the next-hop in route paths is defined as H i = β A i A i 2 + B i 2 + ( 1 − β ) B i A i 2 + B i 2             0 ≤ β ≤ 1 (16) A i = ξ i ∑ j ϵ g ( v ) ξ j             B i = L i ∑ j ϵ g ( v ) L j (17) where g ( v ) denotes the set of neighboring nodes of node v , ξ i denotes the bandwidth occupancy ratio of the neighboring node i , and L i is the distance from node v to its neighbor node i . The factor β is 0 means only considering the node bandwidth occupancy effect, and β is 1 means only considering the link distance effect.

Then in order to find the optimal solution for the load factor β , the objective function in time slot t is set as W t = ∑ v ϵ V η v + T t ¯ (18) where η v is the congestion ratio of node v and T t ¯ is the average delay of users in time slot t . The optimization problem turns to find the value of β that makes the objective function minimum. Since it is a one-dimensional problem, the one-dimensional search algorithm based on golden partition method is proposed in this paper. The steps are as follows.

1) Given the initial interval [ a ( 1 ) , b ( 1 ) ] , the accuracy requirement is tol  ( tol > 0 ) , the golden mean coefficient T , and the number of iterations k. ( a ( k ) is the value of a at the k th iteration)

The stochastic model of terrestrial nodes’ location distribution is based on the Poisson point process (PPP). In this paper, the terrestrial network contains 2 macro base stations, and the coverage radius of macro base stations is 1000 m. Each macro base station has Poisson distribution (the mean value is 150) of microcell base stations within its coverage area, and the coverage area of a microcell base station is 100 m. The distance between the two macro base stations is 1500 m. The events of the members joining and leaving the multicast group in time interval [ 0 , t e ] are Poisson processes with the mean value of 40 and 5, respectively. Set the node constraints c ( v ) = 5 , c ( e ) = ∞ , the streaming rate r ( i ) = 5 M b i t / s for all multicast source, the parameters σ 1 = 1 , σ 2 = 1 , the time interval [ 0 , t e ] , t e = 40 m i n . the number of users joining multicast groups α ( t e ) = 44 and the number of users leaving multicast groups β ( t e ) = 8 , the average service time of switches 1 μ t e = 5 m s . For the newly joined multicast group members, their geographic locations are randomly generated. The terrestrial environment diagrams are as Figure 4.

Consider three connectable states for terrestrial base stations and users: 1) if the mobile user is not in the coverage area of any microcell base station, the mobile user can be directly connected to the nearest macro base station, 2) the macro base station can be connected to micro base stations within 100 m from it. 3) Each microcell base station is connected to its nearest 8 microcell base stations to form a mesh network.

The users are grouped according to the multicast content they request, and the probability of requesting the i th popular content among n contents is defined as f i = i − γ ∑ j = 1 n j − γ according to the Zipf distribution, where γ is the bias parameter. In this paper, set the bias parameter to 0.8 and the number of multicast content types to 4, which are listed as A, B, C, D in order of popularity from high to low. Then their probabilities of being requested are 0.431, 0.248, 0.179, and 0.142, respectively.

In the bandwidth occupancy-based congestion control algorithm, considering the value of β is between 0 and 1, set the initial simulation value a ( 1 ) = 0 ,   b ( 1 ) = 1 and t o l = 0.01 ,   T = 0.618 .

In our experiments, we compare our approach based on LAMA (LAMA-TSTM), our approach based on Tabu search (Tabu-TSTM), LAMA, and the original PIM-SM based on Tabu search. The multicast cost (containing the tree link cost, the update cost and the transition cost), the average delay, the total length of multicast trees, and the congestion ratio are adopted as performance metrics.