A set of drill targets in a bench is given; at each target, a blast hole is to be drilled and filled with explosive material, which will then be detonated to produce rubble that will be processed into ore. The drill planning problem consists of computing a plan that involves machines reaching each drill target in a bench and performing the necessary operations to drill the blast hole. Drilling produces piles of excess material around the hole. These piles constitute obstacles for the machine itself and other machines, hence no machine can drive over them. A solution to the drill planning problem should take into account the emerging obstacles as well as all other common TAMP requirements (e.g., avoiding machine-machine collisions) to be executable by the drill rigs.

The drill planning problem can be seen as a combination of several sub-problems: task planning (consisting of deciding the sequencing of the targets to be drilled), motion planning, and coordination. These problems cannot be treated separately, as the solutions of each problem depend on each other. For instance, task planning must lead to a sequence of drill targets that accounts for the piles generated after drilling (which become obstacles that must be taken into account in motion planning). In other words, the order in which the targets are drilled will affect the ability of the machine itself and other machines to traverse on the bench. Hence, it is necessary to subject the possible choices made to solve one problem to the choices made in resolving the other problems, e.g., verifying through motion planning that a chosen sequence of targets to drill will be kinematically feasible and will avoid the piles of material produced by drilling. There are two approaches in the literature addressing the drill planning problem. One approach is based on meta constraint reasoning (Mansouri et al., 2016) in which the task planning, motion planning and coordination problems are modeled as CSPs at different levels of abstraction, and the meta-constraints capture inter-dependencies between the tasks and motions as a shared abstraction. The second approach implements a multi-abstraction search where an abstract solution is refined incrementally with different types of search at different levels of abstraction (Mansouri et al., 2017).

This application allows us to make several statements pertaining to questions Q1– Q3.

Q1: One common way to deal with multiple levels of abstraction is to abstract away the continuous (geometric) representation – the motions and the piles in this problem – in order to obtain a fully discrete (graph) representation. The graph representation of the drill planning problem forms a variant of the Traveling Salesperson Problem (TSP) where nodes are the drill targets, and edges represent abstracted motion between the nodes. Then, the problem is to find a shortest closed path (tour) in the graph such that every node is visited only once. In a TSP, regions to be visited are associated to nodes in a graph, and each node should be traversed exactly once. Roughly speaking, each region along a tour acts as an “obstacle” that appears dynamically once the node is visited, and which must be avoided while visiting other nodes. However, the TSP employs the abstract notion of a graph to represent locations and their connectivity, thus ignoring the geometrical extent of the locations. Ignoring the geometric reality of the nodes in the TSP, and the fact that paths between them are affected by this spatial extent, leads to solutions that may not be feasible in practice, as they ignore the further constraints to the motion space that derive from the drilling tasks. This points to a rather general observation: abstracting away certain aspects of the problem representation preserves correctness only if we do not lose information regarding the dependencies between different aspects of the problem (in this application, the geometrical extent of the drill targets). An alternative way is to keep each representation at its own level of abstraction, and to leverage a common language to combine relevant knowledge among different levels of abstraction. To enable the use of a common language, we should first identify sub-problems of the overall problem. Furthermore, we need to identify dedicated solvers, each of which focuses on a subset of aspects of the overall problem, e.g., a motion planner verifies kinematic feasibility and absence of collisions, while a scheduler verifies that coordination choices are temporally and spatially feasible. Validated solutions for each sub-problem can be see as constraints that account for particular aspects of the overall problem. As remarked below, constraints can play the role of a common language to facilitate joint reasoning.

Q2: Where we discard the continuous (geometric) representation of motion and piles under a fully discrete (graph) representation, we effectively disable joint reasoning for the drill planning problem. Nevertheless, one might solve a TSP over the graph representation as a proxy to solve the drill planning problem, and use a post-processing step to filter out TSP solutions that are infeasible with respect to motion and pile constraints. To enable joint reasoning in the second alternative of dealing with multiple levels of abstractions, we can use the common language, in this case constraints, in a common constraint network (Dechter, 2003) to model the search space of all problems jointly. In this way, each dedicated solver only operates in the relevant level of abstraction, and validating solutions to sub-problems is reduced to posting constraints in the common constraint network and verifying its consistency. This approach is realized in several different applications, including drill planning (Mansouri et al., 2016) and integrated task and motion planning for warehouse management (Mansouri et al., 2015).

Q3: Machine learning can be useful not only for generating planning models, but also to generate heuristics for efficiently exploring search spaces. In the drill planning problem, we can learn patterns from examples provided by human experts for sequencing decisions in regions where machines have limited space to manoeuvre (Mansouri et al., 2016). In general, learning from humans is important for uptake by the industry, as end users want machines to adhere to the best practices of humans while expending little effort in specification/knowledge engineering. We can also use clustering methods for analyzing the topology of a bench, which will allow to cluster targets into groups for which there are only few reasonable sequencing possibilities and that are easy to navigate in sequence. This again will alleviate the computational burden of finding sequences in a joint search space, which is strongly affected by constraints on motion (Mansouri et al., 2016; Mansouri et al., 2017).

A team of autonomous electric haulers transport material between stations in a quarry. At the unloading station, a hauler can unload gravel obtained from two crushers. The primary crusher (PC) constantly produces gravel, which is continuously output via a conveyor belt. The production of gravel at the PC cannot be stopped under normal circumstances, hence, there should always be a robot under the PC so that gravel does not accumulate on the ground, obstructing access to the PC and halting the entire process. The secondary crusher (SC) does not have this constraint, as the gravel produced there is loaded onto haulers manually. Also, robots are required to leave the PC when full. The aim is to maximize the throughput of the overall system, i.e., the amount of gravel dumped at the unloading point. The important constraint of this application is to guarantee that there is always a hauler under the PC. In this instance of TAMP, we require to solve task planning for the team-wide decisions of which robot visits which station in what order, and motion planning and coordination for the decisions of how, when and where the robots move. Henceforth, we refer to this instance as a multi-hauler planning problem.

In order to respect the constraint of “there should always be a robot under the PC”, we have to able to compute exactly how long it takes for a hauler to go from one station to another station to make sure that we dispatch the robot in an appropriate time. Being too conservative and sending as many robots as available to the PC, makes the SC useless, and negatively affects the throughput by wasting robots being in a queue to reach the PC. However, the durations of navigation actions of the haulers in the quarry is very uncertain. This uncertainty stems from many sources, e.g., the dynamics of individual robots are typically only partially known; robots may navigate differently in different parts of the environment (e.g., skidding over a sandy patch of terrain, proceeding more slowly in the vicinity of pedestrians); task-dependent factors may affect how robots navigate (e.g., slow movement due to a heavy load); and interactions between robots jointly navigating in a shared space introduce further unmodelled dynamics (e.g., robots yielding to, or avoiding, each other). The multi-hauler planning problem was addressed in the literature by a hierarchical approach based on Generalised Stochastic Petri nets (GSPN) for modeling team behavior, where accurate probabilistic models of path durations are obtained via integration with a lower-level team controller (Mansouri et al., 2019). The GSPN is then interpreted as a MDP for which policies can be generated so that team performance is optimized whilst avoiding the exponential blow-up associated with the construction of a full joint model.

This application is most relevant to two of our original questions.

Q3: Today’s commercial solutions for multi-robot path planning remove many source of uncertainty by engineering the environment. Such assumptions are not applicable in many real-world applications including multi-hauler planning. For this reason, current industrial practice relies on fixed, hand-crafted policies for selecting tasks for robots and dispatching them to their destinations. We should instead replace the current practice with an automated planning system that does not make assumptions on the map, the robot geometries, the paths followed by robots, or their kinematics and dynamics. The system should provide a means to easily specify high-level requirements on team behavior, including safety constraints, and it should scale to realistically-sized teams. In order to robustly maintain the safety specification for the PC, models of navigation task duration can be learned. In the absence of real data due to the difficulties of deploying real experiments, we can learn from simulations of the team navigating in the target environment in this case a quarry (Mansouri et al., 2019). To explore the range of multi-robot navigation experiences relevant for the target environment, the robot team must operate in a way that is as similar to the desired behavior as possible. To achieve this, the team should be controlled in simulation using a controller which integrates coordination, motion planning and robot control (e.g., Pecora et al., 2018), and supports the injection of external navigation choices for robots. Given these choices, the controller generates multi-robot paths that take into account the kino-dynamic constraints of individual robots. These paths are jointly executed and supervised by the controller. When generating data for learning, a randomised policy can be used to provide navigation choices.

Q5: The main source of uncertainty in this problem is the duration of navigation actions. We require a method for multi-hauler planning that accounts for this uncertainty. A popular approach to planning with uncertainly is to use MDPs, where uncertainly is modeled in the outcomes of actions. However, it is not accurate to directly model the learned probabilistic model of duration as an action outcome in an MDP. Instead, we can use a stochastic extension to Petri nets to model team behaviors with probabilistic models of path durations. This can then yields an MDP which can be solved to generate policies that optimise team behavior against the team requirements and performance objective.

A modern lightweight dual-arm robot, e.g., the ABB Yumi, is deployed to assemble pieces of wiper motors. The workstation is depicted in Figure 2C, where the rotors are already inserted into workpiece holders (A) on a conveyor system, arriving in groups of five. The stators with the brushes and the electric interfaces are supplied in transport containers (B). Mounting a stator on a rotor requires to place a cone-shaped tool on the motor shaft temporarily. (C) marks the position the robot picks up of a tool. Such flexible production requires fast methods to specify new tasks for these robots, and classical teach-in by means of fixed poses and paths is not appropriate. Flexible assembly planning involves three aspects: task planning of the necessary steps and actions to achieve the overall goal/task; scheduling of these steps and actions; and motion planning for each step and action. Dual-arm manipulation further requires to decide about the allocation of task steps and actions to the individual arms. Moreover, the complexity of scheduling and motion planning is increased heavily, due to the necessity to closely coordinate the manipulators to prevent self-collisions of the robot. All four aspects – task planning, scheduling, allocation and motion planning – are closely interrelated and must be combined to achieve optimal plans with regard to some objective e.g., makespan. Henceforth, we refer to this instance of combined planning as an assembly planning problem.

Obtaining an optimal solution to the assembly planning problem depends not only on the motion of the manipulators but also on the orders in which a workpiece is assembled, the components are taken from boxes or conveyor belts, processed by other machines, etc. These dependencies are all the more complex if connected systems or machines impose further temporal constraints. In addition, different assignments of sub-tasks to arms, while taking the individual working ranges into account as well as task steps in which the arms have to cooperate, lead to a further combinatorial complexity. The assembly planning problem was addressed in the literature via different methods, including prioritized TAMP (Kurosu et al., 2017), fixed-path planning (O’Donnell and Lozano-Pérez, 1989), and fixed-roadmap planning. For this paper, we analyze the latter approach, which uses a flexible model and solver for simultaneous task allocation and motion scheduling that is based on constraint programming (CP) and constraint optimization (Behrens et al., 2019a). The core modeling concepts was Ordered Visiting Constraints, which describe routine sequences of actions in production and time-scalable motion series. These are linked by so-called Connection Variables that act as the shared abstraction between the task and the motion models.

Four of our original questions are relevant in this application.

Q1: Similarly to the drill planning problem, we can keep each representation at its own level of abstraction, and employ a common language to pass relevant knowledge among those levels. Assembly planning for a large scale of items can possibly lead to a massive search space of mutually feasible solutions. However, industrial workplaces often have several characteristic properties that we can leverage to simplify the problem. For example in task modeling, it is safe to assume that many production routines can be described concisely by sequences of actions (e.g., drilling, picking, welding or joining) to perform with one of the robot arms at given locations, with temporal constraints and dependencies between them. This can be easily specified using Constraint Processing (CP) languages (Behrens et al., 2019a). Also, many industrial workplaces provide a controlled and unobstructed environment in which motions can be pre-computed in the form of time-scalable roadmaps. The obtained representation of motion is then discrete and ready to be connected to the high-level CP-based task model by some auxiliary variables so that it can be directly used by a constraint optimisation solver.

Q2: When we flatten out all levels of abstractions into one uniform level, or use an interface representation to manage interactions among abstraction levels, it then becomes straightforward to employ a dedicated solver that can read the uniform or the interface representation. In assembly planning, we can follow this logic, and employ a dedicated constraint optimisation solver for CP languages, e.g., Google Operation Research tools, to obtain an executable optimal assembly plan. The resulting plan is effectively a mutually feasible solution for all sub-problems: task planning, scheduling, allocation and motion planning.

Q3: Instead of directly specifying a sequence of production routines into a planning domain languages (e.g., a constraint problem), a multi-model learning method can be used for robot programming. In particular, a combination of learning from demonstration and requirements specification through natural language has been shown to be effective in preparing robot assembly planning domains for flexible manufacturing (Behrens et al., 2019b).

Q5: Industrial workplaces provide by design a controlled and unobstructed environment. Therefore, it can be assumed that all object locations and possible placements are known in advance, which allows for offline pre-calculation of motion roadmaps and a profile of potential collisions of the arms in motions. Furthermore, depending on the industrial setting, we may able to assume the absence of external interference, e.g., from humans.

AUVs are required to autonomously accomplish missions such as coverage or inspection of a sequence of regions in the ocean or sea. To perform such missions, an AUV must employ a mission planner that can reason about both high-level sequencing of the regions to be visited and low-level motions for navigating through them. While an AUV executes a series of tasks that can span over a period of several hours, the environment could change drastically due to the presence of tide and currents. An AUV mission planner should generate a combined task and motion plan that take into account not only the nonlinear dynamics of AUVs, natural obstacles in water, kinematic constraints, but also drift caused by the time-varying ocean currents. If these requirements, especially those imposed by dynamically changing environment, are not met, AUVs would attempt to carry out highly costly missions that are no longer feasible.

In order to generate a feasible motion plan for an AUV, the intertwined dependencies between the tasks and dynamics of the AUVs and the environment should be considered in the initial planning phases. If the interactions with the environment are overlooked, it may be difficult or impossible to reach the regions of interest that the high-level task planner prescribes. Also, drift is usually modeled via a function whose inputs are position, depth, and time. Therefore, a particular ordering of visitation and position, for instance, could push the UAV further away from its goal because of drift. The AUV mission planning problem has been addressed in the literature by several different approaches. The one we analyze here builds a high-level navigation roadmap by sampling waypoints over the operational area and connecting neighboring waypoints to construct a network of navigation routes. This network avoids known obstacles, areas that are deemed too dangerous for the AUV, or other forbidden regions (McMahon and Plaku, 2016). The navigation roadmap is then combined with a Deterministic Finite Automaton (DFA) representing a regular language to compute sequences of waypoints that are compatible with the mission specification. This combined representation is then used to effectively guide a sampling-based motion planner that takes into account a model of the time-varying ocean currents in each its edge expansion.

For this application, we analyze two of our original questions.

Q1: As explained earlier, one way to deal with multiple levels of abstraction is to abstract away the continuous representation. In AUV mission planning, this particular choice would lead to discretising the relevant portion of the ocean in order to be able to impose high-level specifications for task and motion planning. It is problematic, however, to account for the nonlinear dynamics of the AUV and of the ocean currents in this discretisation. A complementary approach is to use a roadmap type of discretisation (i.e., a graph) which does not represent knowledge regarding the dynamics of the vehicle/environment, but contains knowledge about feasible states that satisfy the high-level task specification. An example of such a representation is roadmap coupled with a DFA (McMahon and Plaku, 2016). This is used as a guide to sample the continuous domain for obtaining a motion tree that is aware of constraints normally imposed on a continuous motion representation e.g., kinodynamic constraints or current drift.

Q3: The obvious candidate to leverage learning methods in this problem is to learn a drift model. In absence of reliable data to build predictive models of ocean currents, a simulator can take advantage of synthetic data derived from what is known of the physics of oceans.