The scheduling of task sets with dependencies has been a well-studied topic within the real-time community. Task schedules with inter-task dependencies are computed in (Chetto et al., 1990) by modifying the offsets and deadlines of the individual tasks and then using EDF to schedule the new task set (Buttazzo, 2011). In (Choi and Agrawala, 2000) the notion of absolute and relative timing constraints (i.e., events are temporally dependent on each other) for source and sink task requirements are introduced. Furthermore, the authors present a scheduling approach for uniprocessor systems with complex timing constraints such as jitter requirements. In (Fohler, 1994) the authors compute static schedules for tasks that communicate through bounded delay protocols like TDMA or TTP using dependency graphs. The work in (Tindell and Clark, 1994) presents an analysis of the schedulability of tasks that communicate using the TDMA protocol. In (Abdelzaher and Shin, 1999) an optimal task schedule for communicating tasks is generated using a branch-and-bound approach. A similar approach is introduced in (Peng et al., 1997) with modified optimization criteria. (Craciunas et al., 2014) presents a similar heuristic scheduling approach to ours which uses EDF simulation to create static schedules for tasks with communication and precedence dependencies but in contrast to our work, the results only apply for dependencies between tasks with equal periods. The temporal dependencies between tasks presented in the prior work described above are not as complex as the ones arising from the ADAS task chains where not only task periods can be different, but the correctness of the chain dependency is related to individual task jobs. Multi-rate tasks and complex precedence constraints have been analyzed in (Forget et al., 2011, 2017; Mubeen and Nolte, 2015). Additionally, in (Isović and Fohler, 2000), a two-step approach for distributed systems is introduced, which is based on an offline computation and an online EDF mechanism for scheduling tasks with complex constraints like jitter and job-level precedence requirements.

In the context of ADAS, the complex task chain requirements have been addressed in terms of computing the worst-case end-to-end latency of multi-rate chains, c.f. (Becker et al., 2017), depending on the available system information, e.g., scheduling algorithm or task offsets. Our approach is different in that it generates schedules that already adhere to the task chain requirements, which does not necessitate a further analysis since the schedule construction guarantees the real-time requirements. In (Rajeev et al., 2010), the authors present a model-checking-based method to compute worst-case response times and end-to-end latencies of tasks that have chain dependency and communication constraints. In (Becker et al., 2016a), the authors introduce a task chain latency analysis that does not require information about the concrete scheduling algorithm. (Verucchi et al., 2020) use an existing list-scheduling algorithm but apply it on a directed acyclic graph (DAG), which is constructed from multi-rate task sets such that complex precedence and timing constraints are captured and satisfied.

In (Lukasiewycz et al., 2012) a modular framework for ILP-based scheduling of time-triggered distributed automotive systems is presented, where both bus access and operating system schedules are created. The end-to-end latency of chains only applies single-rate dependency chains, and the method suffers from an exponential increase in runtime with increasing the number of tasks and messages. An extension that adds an incremental step in order to reduce the runtime complexity of the schedule generation has been proposed in (Sagstetter et al., 2014), where the focus is on integrating locally optimized schedules into a globally non-optimal solution. In terms of the communication backbone, the scheduling problem for TSN networks has been addressed in, e.g., (Craciunas et al., 2016; Serna Oliver et al., 2018) for fully deterministic communication needs, including latency and jitter requirements without taking into account the schedule of the communicating tasks. Furthermore, the combined task and message scheduling problem has also been thoroughly studied for other types of networks, e.g., TTEthernet (Craciunas and Serna Oliver, 2016), shared registers (Becker et al., 2016a), or the Universal Communication Model (Pop et al., 2003).

In our earlier work (McLean et al., 2020) we considered that the communication backbone is done via Peripheral Component Interconnect Express (PCIe), and we have used a periodic real-time task model in which the worst-case execution time (WCET) of a task changes based on the core speed and the communication is modeled as overhead at the end of task instance execution. This paper extends our work to consider the IEEE 802.1 Time-Sensitive Networking (TSN) deterministic Ethernet standard for the communication. TSN is becoming a de-facto standard in several areas, e.g., industrial, automotive, avionics, space, with a broad industry adoption and several vendors developing TSN switches. This paper presents a heuristic-based scheduling algorithm for ADAS platforms that considers the different dimensions of timing and dependency requirements and is designed with scalability in mind. The optimization approaches are based on metaheuristics (Simulated Annealing and Genetic Algorithm), which take into account not only the timing constraints but also design goals, such as function allocation on computing units. We consider both PCIe and TSN for the communication. Future work may also include the LET model (Biondi and Di Natale, 2018) which is becoming increasingly popular in the automotive domain since it can provide deterministic communication behavior.

To the best of our knowledge, this is the first work to propose a heuristic-based solution to the combined scheduling problem in ADAS platforms that requires a solution for the task-to-core assignment, static task schedule generation, and the scheduling of TSN messages sent by the tasks, which respects both task and complex task chain timing constraints.

We start by introducing the platform and application models in Section 2 followed by a description of the scheduling problem in Section 3. We introduce the algorithm in Section 4 followed by an experimental evaluation in Section 5 and conclude the paper in Section 6.

The modern integrated ADAS hardware platform features a multi-core multi-SoC embedded ECU with a variety of CPUs and Graphics Processing Units (GPUs) running at different speeds, which are interconnected through either a deterministic Ethernet backbone, such as TSN (IEEE, 2016b) or TTEthernet (Steiner et al., 2011), or through PCIe. RazorMotion (TTTech Computertechnik AG, 2018), for example, features a Renesas RH850P/1H-C ASIL D MCU with lockstep cores running at 240 MHz and two Renesas R-Car H3 ASIL B SoCs with four Cortex A57, four Cortex A53, one Cortex R7, one IMP-X5, and one IMG PowerVR GX6650 GPU.

Figure 1 presents a high-level view of the ADAS platform, which is similar to the platform described in (Marija Sokcevic, 2020). Each host can run a different operating system depending on the safety and performance requirements. Each such OS can have a different scheduling policy, ranging from fixed-priority (AUTOSAR (Bunzel, 2011)) to table-driven or dynamic priority scheduling (typically in safety RTOSes). However, there is a growing tendency to use a table-driven static schedule execution due to the compositionality and isolation properties (Lukasiewycz et al., 2012; Sagstetter et al., 2014; Mehmed et al., 2017; Ernst et al., 2018), i.e., tasks that are already scheduled are not influenced by new tasks being added to the system. In order to provide a common execution environment and hardware abstraction, a middleware layer, e.g., the MotionWise (TTTech Computertechnik AG, 2018) layer, is running on top of each operating system. The middleware layer also ensures portability of software functions to be located according to their execution and safety requirements (Niedrist, 2018). Moreover, the middleware layer provides the capability to execute tasks according to a table-driven pre-computed schedule independent of the underlying OS dispatching mechanisms, which ensures temporal isolation (Mehmed et al., 2017). Hence, in this paper, we focus on creating static schedules for the table-driven dispatching mechanism of such ADAS systems.

Tasks performing software functions of different criticality levels communicate with each other both on- and off-chip through different means. On-chip communication is usually done through buffers, message passing, or shared memory, while off-chip communication is achieved either through PCIe or TSN. The safety-critical communication also has to adhere to stringent timing requirements and has to be aligned to the execution schedule of the real-time tasks. For example, when PCIe is used, a message sending cost has to be taken into account when scheduling the respective communicating tasks. When using time-aware switched Ethernet technologies like TSN, the schedule of the messages has to be aligned to the execution of the tasks, and the end-to-end latency requirements comprising both task execution and message transmission have to be met.

We model an ADAS platform as a graph A ( V , E ) . A vertex v i ∈ V is either an end system (ES) that performs computations, the set of all ESes denoted with ES , or a TSN switch (SW), their set being denoted with SW . The edges E are the communication links. Each ES v i ∈ ES has a set of computation cores C i .

Time-Sensitive Networking (IEEE, 2016b) addresses the need to have more determinism and real-time capabilities over standardized Ethernet networks. To achieve this, TSN defines a series of amendments to the IEEE 802.1Q standard, as well as stand-alone mechanisms and protocols (e.g., 802.1ASrev). TSN has already seen adoption in the industrial domain and is becoming increasingly relevant in the automotive domain. The main mechanisms out of the TSN ecosystem that we consider in this paper are the clock synchronization protocol IEEE 802.1ASrev (IEEE, 2016a), which provides a synchronized clock reference, and the timed-gate functionality of IEEE 802.1Qbv (IEEE, 2015) bringing scheduled communication capabilities on the egress ports of devices. The timed-gate mechanism is essentially a shaping gate that forwards selected message streams from each egress queue according to the transmission schedule encoded in so-called Gate-Control Lists (GCL). A TSN stream is defined by a payload size, a talker (sender), one or more listeners (receivers), and optional timing requirements in terms of jitter and latency. The global schedule synthesis has been studied in (Craciunas et al., 2016; Dürr and Nayak, 2016; Pop et al., 2016; Serna Oliver et al., 2018) focusing on enforcing deterministic transmission, temporal isolation, and compositional system design for critical streams with end-to-end latency requirements.

A communication link is modeled as a directed edge is represented by two vertices [ v a , v b ] ∈ E . All physical links in the system are bidirectional and so for each [ v a , v b ] ∈ E , there exists a [ v b , v a ] ∈ E with the same properties except that source and destination are swapped. In TSN-based systems, a directional virtual edge is created for each core, i.e., the edge [v _a , v _a ] is added to E . Each link [ v a , v b ] ∈ E has a set of attributes. We denote with [v _a , v _b ].s the transmission bandwidth of the link, [v _a , v _b ].c denotes the number of queues in associated egress port.

On top of this platform, many different software functions implemented by different vendors must be integrated and deployed. It is crucial that software functions (which may be tested independently) can be integrated with other software functions compositionally. The system is composed of applications (called tasks or runnables) that are either pre-assigned to cores or must be assigned by the scheduling algorithm. Tasks have real-time requirements, both in terms of execution (offset, deadline, jitter) as well as temporal dependencies arising from task chains (defined below). We model the applications as a set of n periodic tasks, Γ = {τ _i ∣1 ≤ i ≤ n}, similar to the model in (Liu and Layland, 1973). A task τ _i is defined by the tuple (σ _i , r _i , ϕ _i , C _i , T _i , D _i ) with σ _i representing the core, C _i denoting the worst-case execution time (WCET), T _i the period, r _i the earliest release time, ϕ _i the initial offset/displacement of task arrival times and D _i the relative deadline of the task under the assumption that D _i ≤ T _i . Each real-time task τ _i yields an infinite set of instances (jobs) τ _i,k , k = 1, 2, … (Buttazzo, 2011, p. 80). Tasks can be preempted at any time instant on a timeline with macrotick granularity given by the underlying OS capabilities.

If a task τ _i is pre-assigned to a core, then its core σ _i will be given. Otherwise, we decide their assignment to a specific core, in that the σ _i of a task τ _i can take any value from a finite set of core values C i . The assignment of tasks to cores is captured by the mapping function M : Γ → C i .

The scheduling allows preemption, i.e., a schedule table can be constructed such that a task is interrupted by another task and then resumes its execution. Currently, tasks cannot migrate at run time after they have been assigned to a core, but in the future, we envision that task migration, when done correctly with respect to the deterministic timing behavior, will allow even better resource utilization.

Tasks may exchange messages. A message is modeled as a flow (stream) Φ i ∈ L Φ , which has a period Φ _i .T which is the same as its sender and receiver tasks, a deadline Φ _i .D, which can be arbitrary but smaller than or equal to the period Φ _i .T, a message size Φ _i .P, and a fixed route Φ _i .r. Messages are transmitted as frames. If a message exceeds the maximum transmission unit (MTU) of 1,500 bytes defined for standard Ethernet, then the message is split into k = Φ i . P 1500 fragments. The message is split such that each resultant frame is its maximum size until the last, which has the last bytes. A frame θ i , m [ v a , v b ] is an instance of the frame on the link [v _a , v _b ]. It is associated with the ith flow and has sequence number m. A frame has a transmission duration of θ [ v a , v b ] . L microseconds on the link [v _a , v _b ] where it is transmitted. A frame can have a maximum payload of MTU 1,500 bytes plus the 42 bytes Ethernet header.

Each task may have implicit timing constraints arising from the task definition and explicit design parameters related to arrival offsets and/or deadline requirements. Hence, a task must execute periodically with the given period T _i , and in each period, it must finish its worst-case execution C _i within the defined deadline D _i , starting after the earliest release time r _i . In addition, tasks may also have jitter requirements, i.e., constraints on the variance of execution of consecutive period instances (Buttazzo, 2011, p. 81-82), due to control loop considerations (Di Natale and Stankovic, 2000). We denote the jitter requirements of a task τ _i with J _i and the observed jitter j _i , i.e., the maximal deviation of both starting and finishing times for any two consecutive task instances are bounded by J _i .

Other timing requirements are related to message passing between tasks, where the communication latency has to be considered. The most complex set of timing requirements come from the so-called task (or event) chains (c.f. (Becker et al., 2016a)). A task chain specifies that at least one instance of every task in the given task chain list has to be executed in the specified order within a given maximum end-to-end reaction latency. These chains also have a priority, p _i , which can be used for optimization criteria. Since the tasks in the chain can be on different hosts/cores, the communication needs have to be included in the end-to-end latency considerations. In a PCIe backbone, the latencies between communicating tasks are modeled and enforced as an additional delay after executing the sending tasks. In the case of TSN, the frame schedule must be aligned to the task execution to ensure that the messages are sent after the sending task has been executed and before the receiving task starts. Please note that tasks in the chains may have different activation patterns and periodicity, i.e., we are considering multi-rate chains (c.f. (Becker et al., 2017)).

We give a simple example of a task chain in Figure 2 composed of 4 tasks, a source (τ ₁), two processing tasks (τ ₂, τ ₃) and a sink (τ ₄) with periods of 20 ms, 20 ms, 10 ms and 20 ms, respectively. The critical communication between τ ₁ and τ ₂ is done off-chip through the TSN network over two switches (SW ₁ and SW ₂) since τ ₁ and τ ₂ are running on different SoCs. For the purposes of this illustration, tasks τ ₂, τ ₃, and τ ₄ are located on the same core, and the communication is assumed to be in 0-time. From each instance of the source, there needs to be a succession of instances of the other tasks in the right order such that the latency is not exceeded. It is allowed that an instance of the processing or sink tasks merges multiple signals. In the example, the sink merges the signal from two execution instances of the processing task τ ₃. The communication frames from τ ₁ to τ ₂ have to be scheduled in such a way that the message is forwarded through the switches and arrives at τ ₂ before the respective instance of τ ₂ executes. The latency of the communication needs to be also included in the total end-to-end latency.

Let L ℵ denote the set of task chains, where a task chain is given by the tuple ℵ _i = ({τ ₁ ≺…≺ τ _k }, L _i , p _i ) with L _i being the allowed end-to-end latency and p _i ∈ [0, 1] is the priority. For a chain ℵ i = ( { τ 1 ≺ … ≺ τ k } , L i , p i ) ∈ L ℵ , we formalize the correctness condition for the in-order execution and end-to-end latency requirement as follows: ∀ τ 1 , x , x ∈ 0 , … , h p i T 1 : ∃ { y 2 , … , y k } , y j ∈ 0 , … , h p i T j , ∀ j ∈ { 2 , k }  such that s t a r t ( τ 2 , y 2 ) ≥ e n d ( τ 1 , x ) ∧ e n d ( τ k , y k ) − s t a r t ( τ 1 , x ) ≤ L i ∧ ∀ j ∈ { 2 , k − 1 } : s t a r t ( τ j + 1 , y j + 1 ) ≥ e n d ( τ j , y j ) , (1) where hp _i is the hyperperiod of the chain ℵ _i calculated as the least common multiple of the periods of the tasks in the respective chain and interfering tasks. ¹ and start(τ _i,j ) and end(τ _i,j ) denote the start and end of the execution of the job τ _i,j , respectively.

If there is communication over the TSN network between any two tasks in the chain, the TSN network schedule needs to reflect several correctness conditions. Firstly, the correctness conditions from (Craciunas et al., 2016) for generating GCL schedules need to be fulfilled in order to have correct and deterministic frame transmission over IEEE 802.1Qbv TSN devices. In addition to the technological constraints of standard full-duplex Ethernet networks, a deterministic timing of frames is enforced in (Craciunas et al., 2016) through so-called frame/flow isolation constraints. In the timed-gate mechanism of IEEE 802.1Qbv, the transmission schedule applies to the entire traffic class (as opposed to individual frames like in, e.g., TTEthernet). Therefore, the queue state has to be known and deterministic in order to ensure that the right frames are sent at the right time. Hence, the isolation conditions in (Craciunas et al., 2016) enforce that a correct GCL schedule isolates frames of different flows either in the space domain by placing them in different egress queues or in the time domain, preventing frames of different flows from being in the same queue at the same time. Secondly, implementation or network-specific correctness conditions need to be fulfilled. Here we mention the synchronization error and the microtick of the timeline. The synchronization protocol defined in IEEE 802.1As-rev ensures a common clock reference; however, individual clocks may still have a bounded time differential towards the clock reference. The maximum of all the individual bounded clock errors is called the network precision (δ). Furthermore, the (hardware) realization of the required state machines defined in, e.g., IEEE 802.1Qbv implementing the TAS mechanism also has a certain overhead, resulting in a minimum mandatory spacing of scheduled events (called microtick or link granularity). The microtick or link granularity defines the fastest rate at which schedule events can be processed by the TSN hardware and hence, the granularity of the TSN scheduling timeline. Thirdly, the GCL schedule and, more specifically, the schedule offsets of the frame transmission need to be aligned to the task schedule. For example, a message transmitted between two tasks within a chain has to be scheduled for sending after the execution of the sending task and has to arrive before the receiving task executes. Additionally, the communication latency, which adds to the chain’s overall end-to-end latency, has to be within the given latency requirements.

For a given mapping M, we denote the schedule table with S . In this table, each task τ _i has a list of offsets O _i on its core σ _i . The first offset in O _i , denoted with ϕ _i , captures the initial offset of τ _i ’s arrival time within the schedule, and the rest of the offsets in O _i are the times when task τ _i resumes its execution if preempted. The schedule table S also contains the GCLs, which are captured via an offset θ [ v a , v b ] i , m . ϕ , where i and m are the flow and the frame instance, respectively, and [v _a , v _b ] is the link on which the frame is transmitted. Figure 2 shows a Gantt chart, which is a visual representation of a schedule table.

An overview of our optimization strategy is illustrated in Figure 3. The metaheuristics (SA or GA) decide the mapping M via an iterative search which generates neighboring solutions from the current solution, see “SA or GA search procedure” box in the figure, see the details in Section 4.2. The schedule S of flows and messages is decided by a combination of scheduling heuristics, i.e., List Scheduling (LS) and Earliest Deadline First (EDF), see the box with LS and EDF in the figure, and the details in Section 4.3 and Section 4.4, respectively. Finally, the quality of a solution is evaluated using a cost function, see “Cost Function” box in the figure and the discussion later in this section. We present our proposed solution for TSN-based systems. The same solution is used for PCIe-based systems, with the observation that the flows are not considered, i.e., they are modeled as an overhead added to the sending task’s WCET.

We use an LS-based heuristic to jointly schedule the flows and the tasks involved in communication, presented in Section 4.3. Once the communicating tasks and flows are scheduled, we use an Earliest Deadline First (EDF)-based scheduling heuristic to add the tasks not involved in communication across cores and optimize the schedule, also by introducing design-time “preemption”, i.e., task splitting, see Section 4.4. We then check if the schedule adheres to the timing requirements imposed by the jitter and task chain constraints. The EDF scheduling heuristic introduces design-time task preemption by simulating at design time an EDF scheduling policy parameterized by task offsets and local deadlines. The heuristics receive as an input the mapping of tasks to cores. The LS heuristic is controlled by the tasks and flow offsets (ϕ _i ≥ 0), which are the earliest times a task can be started, or a flow can be sent. The EDF heuristic is controlled by both the offsets and local deadlines D , see the arrow in Figure 3 from the “SA or GA search procedure” box and the LS and EDF box below.

The mapping and the controlling parameters (offsets, deadlines) for the scheduling heuristics are determined by the metaheuristics, as part of their search procedure. We have developed two metaheuristics, one based on Simulated Annealing (SA), see Section 4.2.1 and one based on a Genetic Algorithm (GA), see Section 4.2.2. Both metaheuristics modify the mapping of tasks M, the task and flow offsets (ϕ _i ≥ 0), and deadlines D , to find an optimal solution with respect to the cost function. The novelty in our approach is that our metaheuristics make use of the different dimensions that influence task execution, i.e., task mapping, task offsets, and task deadlines, to converge to a near-optimal solution faster than traditional approaches.

The cost function (Cost), defined in Eq. 2, captures both a minimization objective with respect to the end-to-end latency of task chains and penalties representing constraint violations given by the application. The function has two cases, 1) a value if the solution configuration meets all the timing constraints and 2) a combination of static and dynamic penalties if one or more timing constraints are violated. C o s t ( s ) = ∑ ℵ i ∈ L ℵ l i L i ⋅ p i | L ℵ | ⋅ w 1 i f χ ( s ) = t r u e w 1 + ρ ℵ + ρ D + ρ J i f χ ( s ) = f a l s e (2)

During the search, the metaheuristics do not reject the invalid solutions. Instead, we “penalize” an invalid solution by increasing its cost function value to be larger than the values for valid solutions in the hope of driving the search towards valid solutions. A solution is invalid if one of the three constraints is violated: 1) There is a task or flow which does not meet its deadline, i.e., the worst-case response time f _i of a task τ _i is larger than its deadline D _i (or the worst-case delay of a flow is larger than its deadline). 2) There is a chain ℵ _i which has an end-to-end latency l _i that is greater than its allowed latency L _i . 3) There is a task τ _i which has a jitter j _i greater than maximum allowed jitter J _i , see the notations in Section 2.4. We capture each of these constraint violations using a separate penalty term, which is zero if the constraint is not violated: 1) ρ _D for deadline violations, 2) ρ _ℵ for chain latency violations and 3) ρ _J for jitter violations. Hence, if the sum of these penalty terms is zero then the solution is valid, i.e., all constraints are satisfied and thus all penalty terms are zero. We capture this situation with a test defined by χ(s) = ρ _ℵ + ρ _D + ρ _J > 0. χ(s) is true if the solution is valid, i.e., case (1) in Eq. 2, and false if the solution is invalid, corresponding to case (2).

Let us first discuss case (1) when the solution is valid. In this case, the value of the cost function which has to be minimized is defined as the average weighted distance of the measured end-to-end latency l _i over the imposed constraint L _i , of all task chains. Basically, the smaller the chain latencies, the smaller the term. When we sum up l i L i we also multiply with the chain’s priority p _i to capture the relative importance of chains. The resulted summation is divided with the number of chains | L ℵ | . Note that when a solution is valid, l _i ≤ L _i . The divisions of l _i by L _i and of the summation by | L ℵ | are intended to normalize the cost function term. In addition, we multiply the thus resulted term with a static penalty weight w ₁. We will discuss the use of weights shortly when we cover case (2).

Let us consider the example in Figure 4 where we have three tasks and one chain on a single core. The details are given next to the figure, with the note that the allowed end-to-end latency of the chain L ₁ is 20, and its priority p _i is 1.0. Figure 4A shows a valid solution, whereas Figure 4B an invalid one. For Figure 4A we have l ₁ = 20, hence the cost function is 20 20 ⋅ 1.0 ⋅ w 1 = 1 ⋅ w 1 . Let us now discuss the cost function for the case 2) when a solution is invalid. In that case, ρ _ℵ + ρ _D + ρ _J will be greater than zero. To this term, we add w ₁ to ensure that any invalid solution will be rated worse relative to that of any valid solution. That is, it adjusts the score such that the minimal penalty value is higher than that of any feasible solution, thereby preventing the search process from accepting any invalid candidate over a valid one. As discussed, we use ρ _D to penalize deadline violations, ρ _ℵ to penalize chain violations and ρ _J to penalize jitter violations.

The values of these penalty functions are dynamic, i.e., a larger value is used bigger constraint violations. We first define the ρ _ℵ from Eq. 3 in detail and then discuss the other two penalties, which are similarly constructed. The relative importance of these penalties are determined by the weights w ₂, w ₃ and w ₃. ρ _ℵ in Eq. 3, measures the weighted average of end-to-end violation. The violation of a chain ℵ _i is defined as the difference between its highest observed chain latency l _i and its end-to-end constraint L _i . ρ ℵ = ∑ ℵ i ∈ L ℵ m i n ( L i , m a x ( 0 , l i − L i ) ) L i | L ℵ | ⋅ w 2 (3)

If l _i is smaller or equal to L _i then the chain constraint is satisfied and the term is zero. We discuss here the case when the constraint is not satisfied, i.e., l _i > L _i . When l _i > L _i the max operator will return l _i − L _i . To normalize the penalty value, we clamp any observed violation l _i − L _i to the interval [0, L _i ] using the min operator and divide by L _i , hence the term in the summation will be in the interval [0, 1]. We divide the sum with the number of chains | L ℵ | and then multiply with the static weight w ₂. Let us illustrate this with the example in Figure 4B, where the latency L _i = 20 for chain ℵ ₁ is violated, i.e., l _i = 23. Hence, ρ ℵ = m i n ( 20 , m a x ( 0,23 − 20 ) ) 20 1 ⋅ w 2 = 3 20 ⋅ w 2 .

Likewise, the additional deadline and jitter costs (ρ _D and ρ _J ) is listed by Eqs. 4, 5, respectively. Here ρ _D measures the weighted average of deadline violations with a violation range clamped in the interval [0, D _i ]. The deadline violation of a task or flow i is denoted as the difference between the maximal relative finishing time of all of i’s instances f _i and the relative deadline D _i . ρ D = ∑ i ∈ Γ m i n ( D i , m a x ( 0 , f i − D i ) ) D i | Γ | ⋅ w 3 . (4)

Finally, ρ _J measures the weighted average of jitter violations. We define the jitter violation of a task τ _i as the difference between the maximal observed jitter j _i and the threshold J _i . The violation range is then clamped in the interval [0, J _i ]. ρ J = ∑ τ i ∈ Γ m i n ( J i , m a x ( 0 , j i − J i ) ) J i | Γ | ⋅ w 4 , (5)

In Eqs. 2–5, we list w ₁, w ₂, w ₃ and w ₄ as static weights designed to capture the importance of the respective violation with the following constraints: w ₂ ≥ w ₁, w ₃ ≥ w ₁, w ₄ ≥ w ₁. The constants were determined based on manual experimentation and observations, with w ₁ through w ₄ set to 10,000, 40,000, 10,000, and 60,000, respectively. Please note that there are no optimal values for the weights, since they have to be adapted to the application domain, criticality definitions and design goals of the respective use-case.

The SA and GA metaheuristics aim to iteratively optimize solutions by randomly changing existing solutions s to create new solutions s′ and evaluate them by using the cost function. They take as input the platform model A and the applications Γ and return the best solution s* found according to the cost function. Both start from an initial solution s ₀. Metaheuristics can start from any initial solution, even a random one. However, in our case, we have developed a Greedy algorithm to generate the initial solution. With SA and GA the mapping is optimized during the search. With Greedy, the mapping is decided constructively as follows. For each task, considering the processor affinity constraints (that restrict the mapping to specific processors), we iterate through all cores and identify the core with the lowest utilization. The utilization of a task is its WCET divided by its period. The utilization of core is the sum of all task utilizations on that core. We then map the task to the respective core. This ensures a balanced utilization of cores in the initial solution. Once the mapping is decided, we use the same LS and EDF scheduling heuristics to schedule the tasks and flows, see Section 4.3 and Section 4.4, respectively. However, with Greedy, the input parameters to the scheduling heuristics, such as offsets for both LS and EDF and local deadlines for EDF, are not optimized. Thus, we consider that the offsets are all set to zero and the local deadlines are set to the absolute deadlines.

One approach to the task and flow scheduling problem is to solve the problems separately and then fit them together. This is a reasonable approach when the two sub-problems do not form a circular dependency. In our case, however, the two scheduling problems are closely linked together. Let us consider the example in Figure 5, where we have a topology of three end systems (ES ₁ to ES ₃) and one switch (SW ₁). We have four tasks τ _A , τ _B , τ _C , and τ _D with precedence constraints [τ _A ≺ τ _B ≺ τ _C ≺ τ _D ] that form a chain. τ _A needs to be executed on ES ₁, τ _B on ES ₂ and τ _C and τ _D on ES ₃. Task τ _A sends a message to τ _B , and τ _B sends a message to τ _C . For illustration purposes, the task WCETs and the transmission times of message frames on links are a single time unit, and network precision and macrotick are ignored. The period of all tasks is 8 time units, and the chain latency is also 8 time units.

If the messages are scheduled first, then the solution of the flow would look as shown in Figure 5A. This schedule minimizes the flow latencies, but since task τ _B must receive message 1 and send message 2, the schedule contains a lot of idle time; hence, the chain latency becomes 13 time units. Note that messages 1 and 2 scheduled at the beginning of the schedule are sent by tasks τ _A and τ _B from the previous period. The same issue exists if the tasks are scheduled without any knowledge of the network. However, if both task scheduling and message scheduling are optimized concurrently, then an optimized solution, shown in Figure 5B, can be produced. This reduces the chain latency substantially to 8 time units from 13 in Figure 5A, meeting thus the task chain latency constraint.

In this section, we propose a joint flow and task scheduling heuristic based on List Scheduling (LS) (Sinnen, 2007). LS is a widely used task scheduling heuristic that is known to obtain good quality solutions when determining static schedules for tasks on multiprocessors. We have re-purposed LS for jointly scheduling flows and tasks. Our LS is inspired by the individual flow scheduling heuristic from (Raagaard and Pop, 2017), which uses variants of ASAP (As Soon As Possible) and ALAP (As Late As Possible) scheduling. Both of these are a special case of the List Scheduling heuristic (Sinnen, 2007). Our LS is more general, scheduling both flows and tasks. This LS can use offset and ordering parameters to control the placement of frames, which is not considered in (Raagaard and Pop, 2017).

LS receives as input the architecture A , applications Γ and the solution s generated by the outer metaheuristic, containing the mapping M, offsets ϕ and deadlines D . LS returns a partial schedule table S ′ covering the hyperperiod of Γ and including all flows and those communicating tasks involved in sending and receiving the flows. The joint scheduling is achieved by adding those tasks involved in communication as “virtual flows” and the cores where they are mapped for execution as “virtual links”. Then, precedence constraints are added to ensure that a frame cannot be sent before its sending task has finished executing, and a receiving task cannot start before its input frames have arrived. Hence, in the following, flows and links also denote tasks and cores, respectively. Note that not all tasks are involved in the communication over TSN. Tasks that are not sending/receiving flows are added to the schedule in a subsequent step using the EDF-based heuristic from Section 4.4.

Algorithm 1

ScheduleFlow (θ, ϕ). Schedules a flow θ as soon as possible (ASAP), considering its offset ϕ given by the metaheuristic. All frames in the flow have initialized lower and upper bounds to − ∞, ∞, respectively.

Similar to (Raagaard and Pop, 2017), LS starts with an empty timeline and iteratively schedules one flow at a time. The metaheuristic specifies the order in which flows are chosen. Similar to (Raagaard and Pop, 2017), the flows are chosen according to their deadlines since flows with tight deadlines are the hardest to schedule and therefore should be picked first. The tie-breaker for the ordering is given by the flow period. Each flow is scheduled using the ScheduleFlow procedure in Algorithm 1 such that the end-to-end latency is minimized. The termination condition for the LS is that either a schedule has been found for all flows or the current iteration does not produce a feasible schedule with respect to the flow deadline.

We now examine the steps of Algorithm 1 in more detail. In step 1 frames are retrieved in the order given by Eq. 7, see Section 4.3.3 for an explanation of how the next frame is determined. In step 3, the frame offset ϕ is set to the earliest time in a feasible region, greater than the lower bound, determined in step 2. Section 4.3.2 and Section 4.3.1 present how we define and determine the feasible regions and the lower bound, respectively. If the algorithm reaches a state where a frame cannot be scheduled, e.g., there is not enough space, then it needs to find another solution. This search is done by backtracking: In step 4 backtracking is done by increasing the lower bound to the latest available time which is less than the frame offset found in step 3, then rescheduling the previous frame, see Section 4.3.4.

The List Scheduling heuristic from Section 4.3 has scheduled all the flows and the corresponding communicating tasks, resulting in a partial schedule S ′ . We propose the EDF-based heuristic in Algorithm 3 for scheduling the rest of the non-communicating tasks and optimizing the schedule. EDFScheduleSynthesis receives as input the architecture A , applications Γ the partial solution s containing the mapping M, offsets ϕ, deadlines D and partial schedule S ′ . The schedule synthesis heuristic from Algorithm 3 is based on simulating Earliest Deadline First (EDF) scheduling, similar to (Craciunas et al., 2014; Barzegaran et al., 2020). EDF is a scheduling algorithm (Buttazzo, 2011) that prioritizes tasks at each time instant depending on their deadlines, i.e., the one with the earliest deadline will get control of the CPU. Given the task WCETs, offsets, and deadlines, the partial schedule S ′ , the complete schedule table S is generated by simulating how EDF would execute tasks until the hyperperiod. For a given mapping, offsets, and deadlines, EDF will always produce the same schedule. We optimize the schedule produced by the EDF simulation by allowing the metaheuristics in an outer loop to change the mapping M, offsets ϕ _i of each task τ _i and deadlines D , and by allowing preemption for the tasks. The EDF simulation is implemented using the discrete-event simulation (DES) paradigm, where the operation of a system is seen as a discrete sequence of events in time, and an event captures the change of state in the system at a particular moment in time.

Algorithm 3

EDFScheduleSynthesis ( A , Γ , s ) . Schedules the tasks that are not involved in the communication on top of the partial schedule S ′ produced by Algorithm 1, creating the final schedule S

We start by assigning all tasks to their respective cores (step 1 in Algorithm 3). All tasks without a task mapping will be mapped according to a best-fit strategy with respect to utilization, i.e., balancing the processor and core utilization. We run the simulation for a length l (set in step 2), after which the schedule will repeat itself. l is defined by 2 ⋅hyperperiod + MaxOffset, where hyperperiod is determined as the Least Common Multiple of all tasks in Γ and the MaxOffset is the maximum over all offsets ϕ _i (Leung and Merrill, 1980). The iteration over the simulation length l is done in the steps 6–12. The current time is captured by cycle, and we advance the time to the next event that needs to be simulated.

The EDF simulation is performed per core σ (step 10), and we use a queue Q _σ containing all cores from all processors, ordered by the earliest event that needs to be simulated. To start the simulation, steps 3–5 add cores to the queue Q _σ , by visiting all the cores in the architecture A .

The next event to be simulated is determined by taking the head of the queue Q _σ (Dequeue) and calling NextCycle. NextCycle is called for a core σ and returns the time cycle when the next event of interest for scheduling occurs on that core. Our NextCycle implementation skips unnecessary cycles, i.e. when no events of interest to the scheduling occur. It does so by progressing towards the nearest event defined by either releasing a task from the waiting queue, choosing the task with the earliest deadline first from the ready queue, completing a task, or allowing preemption to occur on a certain break point defined by a parameter called macrotick for each core. The macrotick defines the preemption granularity. The macrotick is set such that it allows preemption, under the constraint that the overhead due to context switches on each processor should be low, see (Aichouch et al., 2013; Craciunas et al., 2014; Zuepke et al., 2015) for a discussion.

We add cores to be simulated in Q _σ only if we are still within the simulation length l. The algorithm stops when there are no cores to be simulated (Q _σ is empty). The EDF simulation logic is taking place in the EDFSimulation function, called at line 10, which simulates up to the next event, which is returned. The product of EDFScheduleSythesis is then a recording of all occurred events, from which we can derive the schedule table S of the current solution s. As mentioned, LS schedules jointly the flows and those tasks involved in the communication. EDF schedules the tasks not scheduled so far, i.e., those which are not involved in inter-core communication. The EDF algorithm allows task preemption, which in our case means that tasks in the schedule can be split at design-time. This optimizes the schedule table, as the scheduling search space becomes larger and latencies and jitters can be further optimized compared to the case such task splitting is not allowed. In addition, EDF will be allowed to split the communicating tasks already added to the partial schedule S ′ by the LS heuristic if such task splitting will not result in constraint violations.

We illustrate the EDF approach via the example in Figure 4, consisting of an architecture with a single processor with two cores, σ ₁ and σ ₂, both of which having a macro tick of 1 ms. The depicted application is modelled by the task set Γ = {τ ₁, τ ₂, τ ₃}. In this example all tasks have a jitter constraint of 0, meaning that they have to start in each period instance at the same time in relation to the start of the period. The tasks are defined as τ ₁ = (σ ₀, 0, ϕ ₁, 4, 10, 10), τ ₂ = (σ ₀, 0, ϕ ₂, 1, 4, 4) and τ ₃ = (σ ₁, 0, ϕ ₃, 4, 20, 20). Furthermore, the set of task chains is defined by L ℵ = { ℵ 1 } , with ℵ ₁ = ({τ ₁ ≺ τ ₂ ≺ τ ₃}, 20, 1.0). Please note, that given Eq. 1, only two chain instances are necessary to validate as the hyperperiod of ℵ ₁ is 20 ms, and the period of τ ₁ is 10 ms.

Figure 4A depicts a solution, where the jitter and the task chain end-to-end constraints are violated, whereas Figure 4B shows a valid solution. As seen from Figure 4A, τ ₁ violates its jitter constraints, as the start (and end) of execution within its periods varies. This is detected when the events for the respective tasks instances are raised. For example, the event triggering the start of execution with respect to τ _1,1 and τ _1,2 differs by 1 ms. While the initial offset ϕ _i for all tasks is 0, resulting in τ _2,1 and τ _3,1 starting their execution first, neither are source tasks with respect to ℵ ₁. Moving forward, τ _1,1 is started at cycle 1, causing the event to trigger the registration of a task chain instance ℵ _1,1. At cycle 4, τ _1,1 is preempted by τ _2,2 while τ _3,1 completes is execution. Although τ _3,1 is a sink task, and a chain instance has been registered, the instance has yet to receive the completion of τ _1,1 and τ _2,k before it is accepted. That is, the presence of an event from τ _2,k that happens after τ _1,1 must be registered. Subsequently, τ _1,1 completes at cycle 5, allowing the ℵ _1,1 to advance its state, waiting for τ _2,3. Lastly, τ _3,2 finalizes its execution at cycle 23, thus completing ℵ _1,1 with a resulting latency of 23, which incidentally violates the given constraints. The chain instance ℵ _1,2 is registered at cycle 10, and finalizes at cycle 23, yielding a latency of 14. Given that both latency and jitter constraints have been violated, the product of the EDFScheduleSynthesis is not feasible.

However, in an optimized solution, solving the associated violations can be achieved by manipulating the initial offsets ϕ _i for the tasks, as depicted in Figure 4B. Here, the schedule has been altered such that all executions of τ ₁ and τ ₃ have been deferred by ϕ ₁ and ϕ ₃. For τ ₁, the displacement ϕ ₁ solves the jitter constraint violation, because all jobs τ _1,i now start (and end) at the same cycle relative to its period. Finally, τ ₃ has been displaced by 9 cycles, such that its initial execution allows τ _3,1 to catch the events from τ _2,3 (and by extension τ _1,1), thus reducing the latency of ℵ _1,1. Likewise, by introducing ϕ ₁ for τ ₁ the latency of ℵ _1,1 was reduced even further. The combined effect of ϕ ₁ and ϕ ₃ is full compliance of all constraints with the resulting latencies of 10 and 20 ms for ℵ _1,1 and ℵ _1,2, respectively.

As a first experiment, we were interested in determining our proposed SA’s ability to find near-optimal solutions. We have implemented an exhaustive search that finds the optimal solution; however, we could only do that for small task sets of less than 10 tasks considering an architecture with two cores. Our SA was able to find the same optimal solution in less than 10 s. In the following sets of experiments, determining the efficacy of SA was achieved through a combination of synthetic and realistic test scenarios, benchmarked against two other heuristics: Greedy, presented as the initial solution for the metaheuristics in Section 4.2 and Genetic Algorithm (GA).

In this section, we considered that the communication is done via a TSN backbone. We have used synthetic test cases, for which we generated various TSN networks. We have used three different types of graphs with varying degrees of connectivity. The topologies (mesh, ring, and tree) are shown in Figure 9.

The number of end systems, switches, and chains are given in Table 3, similar to the setup used in (Craciunas and Serna Oliver, 2016), except for the number of chains. For each end system, 16 tasks are created, 8 which communicate and 8 which do not. Each communicating task sends a message to another communicating task. Thus, there will be |ES|⋅ 4 flows in the network. The utilization is set to be 50% for each end system, 25% of which corresponds to communicating tasks and 75% to the rest. Task WCETs are chosen such that they are divisible with the macrotick and fit within the assigned utilization. Message lengths are chosen at random between 84 and 1,542 Bytes. The macrotick of the end systems is set to 250 μs, and the granularity of the links are set to 1 μs. The speed of links from end systems to switches is set to 100 Mbps and to 1 Gbps between switches.

We have used three sets of randomly chosen periods, all in milliseconds, P1 = {10, 20, 25, 50, 100}, P2 = {10, 30, 100}, and P3 = {50, 75}. We use the shortest paths for routes. The chains are generated with a maximum task length of 15, and a minimum of 2, and lengths are chosen from a uniform random distribution. Two consecutive tasks in a chain must either be on the same end system or be communicating via the TSN network. In order to compare the performance between the two metaheuristics, 10 runs were performed on each of the test cases for a total of 360 runs for each of the metaheuristics. The time given for each size is as follows. 300 s for small test cases, 1,200 s for medium, 4,800 s for large, and 19,200 s for the largest test cases, called “huge”.

Figure 10 shows on the y-axis the percentage of test cases solved for each size and topology. The results are grouped per topology, mesh, ring, and tree, and for each topology, we use different sets of periods, P1 to P3. A test case is “solved” if all the requirements are satisfied. On the y-axis, 1 means that 100% of the requirements were satisfied, whereas 0 means that no requirements could be satisfied. On the x-axis, we show the type of test case, small, medium, large, and huge. As we can see, our GA and SA solutions can successfully solve all the test cases, except for some of the “huge” test cases, especially in the tree topologies, where a few requirements could not be satisfied. In those situations, GA performs better than SA. When considering the cost of the solutions (the value of Eq. 2), we noticed that SA is better than GA in terms of the cost function for small, medium, and large test cases. However, in the huge test cases, GA not only is able to find feasible solutions more consistently but is also able to find solutions of lower cost.

Finally, we were also interested in our approaches’ ability to find a feasible solution as fast as possible. That is, we wanted to determine what is the earliest time when all the requirements are satisfied. Once such a solution is found, the metaheuristics continue the optimization until the time limit is reached. Hence, we modified SA and GA to return once a feasible solution was found. The runtime results are shown in Figure 11. The figure shows that when SA can find a feasible solution for a test case, it generally finds it faster than GA. However, there are situations where GA outperforms SA.