As mentioned in the introduction, our approach relies on an architectural setup similar to ORCA. Therefore, we first reinforce the link between our approach and ORCA and related works. Next, we build on these analogs to preceding works to detail our approach – from the perspective of a generic design concept as well as of its concrete implementation.

Tensile structures play an important role in postmodern architecture [see, e.g., Lewis (2003)], and they promise to become increasingly important still, considering their unique versatility and flexibility in combination with advances in technologies in built material and construction methods (Schock, 1997). They have also been subject to Aerial Robotic Construction (ARC) research (Willmann et al., 2012; Augugliaro et al., 2013) due to their light mass, load-carrying ability, and their ability of connecting large distances. Quad-rotors have been identified as vehicles apt for aerial manipulation mainly due to their robust flight behavior and their hovering capability (Mahony et al., 2012). In Augugliaro et al. (2013), prototypic building primitives such as single and multi-round turn hitches, knob and elbow knots as well the trajectories resulting from their concatenation have been discussed. Different from pre-calculating trajectories, we have been working on a self-organizing approach to building tensile structures. We detail our approach below, followed by elaborations on its OCbotic-specific features.

Typically, a spider weaves its web by itself (von Frisch, 1974; Hansell, 2005). Complex web constructions, however, may require collaborative entanglement and tightening of ropes. This can, for instance, be achieved by synchronized flight through pre-calculated control points to cross the ARC quad-rotors’ trajectories. Alternatively, the swarm individuals may coordinate themselves relying on local stimuli, like social insects do (Bonabeau et al., 1999; Camazine et al., 2003). In this section, we present an accordingly motivated ARC experiment.¹ It features an autonomous quad-rotor that tightens a rope around a tent pole’s four suspension lines, see Figures 2 and 3.

For our lab-experiments, we employ the AR.Drone Parrot 2.0 quad-rotor system. It is connected to a standard PC via WLAN. In order to emulate performant autonomic control of the drone, a PC retrieves the sensory data of the quad-rotor and issues the according navigational instructions. We make use of the quad-rotor’s VGA camera that has a 90° field of vision, built-in image-processing capabilities such as recognition of QR markers, and the estimates of its ultrasonic distance sensor. As this sensor and a downward directed camera are used by the quad-rotor to stabilize its flight, we attached a coil at the top of the vehicle and unwind the cord through an eye at its back. We interface with the quad-rotor relying on Nodecoper.js and the node-ar-drone module (Childers, 2014).

The quad-rotor behaves only based on locally available sensory information. In particular, it implements the reactive behavior schematically summarized in Figure 4A: after taking off, it searches for an orange–green–orange marker, which is one of the designs that the vehicle is programed to recognize automatically. It keeps spinning right until it eventually finds one. If the distance to the marker is less than a certain threshold (1 m worked quite well), it drifts left. As a consequence, the detected marker moves outside of its field of view. At this point, the quad-rotor has surpassed the previous marker and looks for the next one, which is attached to the next suspension line (also consider Figure 2). The distance to the next marker along the circumference of the pole is greater than the given threshold. The quad-rotor can go straight ahead, if the tag is within the right-hand side of its view (this condition is labeled “tag in area” in Figure 4A). Otherwise, it needs to shift a bit to the left.

Programmatically, the quad-rotor’s behavior (Figure 4A) is represented as a set of simple if-then rules. As such, they can be easily subjected to standard LCS implementations and its extensions such as XCS (Section 1.3). Hereby, those rules with the best prediction accuracy in terms of marker detection may be reinforced to gain the greatest fitness over time, yielding the best possible behavior. In this way, the quad-rotor of the ARC example would learn to query the proper sensors at the right times to react in the best possible way, if the behavioral rule set was enriched with according alternatives. At the interface of level 0 (the SuOC) and level 1 (the reinforcement learner), the measure of success can typically be calculated based on locally available information such as the distance flown or the number of recognized tags. For good learning results, the parametrization of the behavior should be realized at a rather high level, focusing on the selection of queries and operations and only cover small ranges of variability. Potential benefits of level 1 learning would not only be optimization of one particular learning pattern but also behavioral rules that adapt to hardware particularities such as deviating sensory intake or imbalanced motor control.

At level 2 of the multilayer O/C design concept, behaviors can be created by means of generative model building approaches such as evolutionary algorithms and be optimized for deployment by means of simulations. As a first OCbotics prototype of offline level 2 generation and optimization, we have evolved quad-rotor behavior for collaborative surface maintenance. In this section, we introduce the challenge of optimizing collaborative surface maintenance. We detail the technical setup we relied on for both simulation and optimization, and we describe the behavioral options of each swarm individual. Afterward, we draw a very rough picture of the evolutionary experiments that we have run, and we discuss the interactions between layers 2 and 3 for propagating successfully bred behaviors that require synchronization between the individuals in an OCbotics swarm.

Consider the facades of large office buildings as examples of vertical surfaces: they are subject to cleaning (Bohme et al., 1998; Elkmann et al., 2002), trimming greenery (Pérez et al., 2011), and other maintenance tasks. As in the previous example, these tasks might benefit more from collaborative efforts than only in terms of efficiency. For instance, fast growing greenery might require one machine to bend, the other one to cut a branch. Equally, during cleaning, several hovering robots might have to join to build up sufficient pressure to remove persistent dirt. Of course, the respective operations might also be split into several procedures performed by individually optimized machines. In this example, however, we only consider the most modest objective, namely collaborative efficiency.

The technical setup of our level 2 experiment comprises (a) a simulation environment to calculate aviation and robotic mechanics and (b) a machine learning environment with a generative model component and an optimization component. Figure 4B depicts the software modules that we have used in order to simulate collaborative quad-rotor swarms. The Robot Operating System (ROS) acts as a hub for these modules. It provides a high-level software interface for programing and communicating with different kinds of robots (Quigley et al., 2009). Gazebo is a simulation engine that natively integrates with ROS, offering 3D rendering, robot-specific functionality, and physics calculations (Koenig and Howard, 2004). Thanks to a ROS driver for the AR.Drone Parrot quad-rotor (Hamer et al., 2014), and thanks to a Gazebo plug-in that simulates the quad-rotor’s behavior based on the very same ROS-based instruction set (Huang and Sturm, 2014), any of the generated behaviors immediately work in silico and in vivo. For the generation of novel behaviors as well as for their evolution, we decided to use the Evolving Objects framework (EO) by Keijzer et al. (2002). EO is an open framework for evolutionary computation featuring an extensible, object-oriented design concept, and turnkey implementations of genetic algorithms, particle swarm optimization, and genetic programing.

Our approach to collaborative facade maintenance is inspired by nest construction of social insects (Bonabeau et al., 1999). Each individual works on a small part of the construction proportional in size to the insects’ physique. Accordingly, each simulated quad-rotor divides the target surface in a grid, each cell measuring 2 m × 2 m, its field of view covering six cells, two rows of three (Figure 5). This partitioning scheme is a result of the size of the quad-rotor itself and its perceived area from a vantage point close to the surface. Without loss of generality, a dirtiness value is assigned to each cell that indicates whether it needs to be worked on or not. The quad-rotor’s internal state, i.e., its remaining battery life, as well as the configuration of dirty and clean cells that reveals itself in front of it trigger specific actions. The quad-rotor may return to the base station to recharge. It may fly to one of the cells in its field of view and clean it. Alternatively, it may move to one of the four neighboring vantage points to inspect the respective neighboring batches of cells.

In our last example, we demonstrate an early prototype of the user interfacing component of layer 3 of the multilayered O/C design concept that drives the OCbotics approach. As hinted at in Figure 1, layer 3 mediates the user’s goals vertically to all system layers below and horizontally to all OCbotics individuals of the system.

The preceding examples of web-weaving quad-rotors in Section 2 and collaborative swarms in Section 3 implement spatial operations. To some extent, they all culminate in the tandem of local cues and resultant trajectories. As a consequence, defining spatial targets for arbitrary subsets of a swarm deems to be an adequate task generalization for a first prototype of a level 3 user interface. A “human-in-the-loop” system design forces one to clearly define the level of influence a user may exercise versus the level of autonomy the system may keep (Narayanan and Rothrock, 2011). Therefore, we elaborate on the different levels of access implemented by our prototype right after we outline its technical foundation.

Focusing on interactivity, we decided to utilize the turnkey infrastructure of one of the comprehensive game and simulation engines. In particular, we decided to use Unity as it provides a very shallow learning curve (compared to its competitors) while still providing a powerful coding infrastructure that allows to write custom plug-ins in C# and which offers a wide range of third-party plug-ins in a dedicated asset store, see Unity Technologies (2014). Aiming at the implementation of a high-level interface, we tapped into these resources as much as possible and bought, for instance, commercial code bases for simulating flocking behaviors (Different Methods, 2014) and automated path finding in three-dimensional environments (Allebi, 2014). We further built on Unity demos and plug-ins that support current hardware solutions such as the Oculus Rift head-mounted display (Oculus VR Inc, 2014) and the Razer Hydra motion controller (Razer Inc, 2014). In combination, these hardware solutions allow us to emulate an augmented reality scenario for controlling an OCbotics swarm. Figure 11 shows the model of an OCbotics swarm being setup in Unity3D. The light green circles depict waypoints computed by the path finding algorithm, the dual-view perspective at the bottom-left corner of the screen indicates the current view of the attached head-mounted display. The bottom-right window displays the library of components used for modeling the scene, the list at the right-hand side of the screen shows the components that already constitute the scene.

Our user interface prototype immerses the user into a virtual reality shared with the OCbotics swarm. In the long run, the simulated swarm is meant to make way for a real one, and the virtual reality for an augmented reality. Already, the user can observe the whole swarm or a subset tracking it with a virtual camera that follows in a distance and which aims at the center of the set of selected individuals. The user can exercise control on any subset of the swarm, hence he may direct flocks of individuals or single individuals at a time. The interface provides all kinds of state information about the selected individuals, such as (averaged and variance of) remaining battery life, current target, current trajectory, and currently perceived neighbors. The user may switch between individuals and greater subsets of the swarm by simply selecting them. Next, he may change the target of flight or even individual control points along the way. Of course, he may also change the parameters of the selected individuals such as their urge for alignment. In our prototype, the user is immersed into the scene of the simulated swarm (see Figure 12A) so he can easily trace its activity, understand its relationship to the current target and to obstacles, and to rectify it, whenever necessary.

The presented simulated prototype for immersive swarm control shows how high-level goals such as setting a new target of the swarm can be communicated in an intuitive way. Differentiated selection of swarm individuals as well as setting local attributes, such as local targets or local waypoints, are simple yet clear examples of moving from abstract, high-level goal descriptions (target/swarm) to specific low-level commands (trajectory waypoints/individual). For a swarm and an individual to reach the specified targets or waypoints, complex calculations have to be performed. In the given example, the need to avoid obstacles and to find optimal paths as well as the coordination among swarm individuals on their way are outsourced to third-party plug-ins (Allebi, 2014; Different Methods, 2014). In the general case, also considering other tasks communicated on layer 3, the necessary behaviors could evolve in sandboxed simulations (layer 2) and be optimized based on local performance feedback (layer 1), see. We are currently working on furthering the accessibility of human–swarm interfaces by designing and evaluating game-like test scenarios in virtual reality (Figure 12B). In addition, we derive mutual influences among distributed robot societies at runtime to further improve the cooperation of individuals (Rudolph et al., 2015a,b, 2016).

In this article, we have introduced OCbotics as a comprehensive approach to designing self-organizing aerial robotic ensembles. OCbotics is driven by a multilayered observer/controller design concept that allows to optimize and adapt an adaptable system. Adaptation is required in order to maintain or increase the performance exhibited by the system under observation and control – either by optimizing or extending existing behaviors, or by innovating, i.e., generating, simulating, and optimizing novel behaviors. The performance, in turn, is measured in terms of user-defined goals that may also change over time. In comparison to concepts from the state-of-the-art [see, e.g., Augugliaro et al. (2013) where trajectories are pre-calculated], the OCbotics approach provides a large step toward applicability in non-lab and unstructured conditions, since it establishes a self-adaptation loop that considers safety constraint and continuously self-optimizes the robot’s behavior.

We have presented three different projects that operate at different levels of the discussed design concept: web-weaving quad-rotors with an emphasis on optimized local reactive behavior, evolution of collaborative behavior to efficiently work on surfaces, and an immersive user interface for setting and changing user-defined goals. While the three examples slightly vary regarding their applications, they are connected through the common themes of self-organization, rule-based behavior, and adaptation, and of course, the O/C design concept to host them all. With the pieces of the puzzle at hand, the next obvious step is to put them into place, to forge the software components into one (if heterogeneous) code base, to connect the layers of the design concept, to develop a repertoire of recombinable goal definitions, and to transfer the partially still virtual implementations of all levels onto an actual OCbotics infrastructure.

SM: main author, main project supervisor. ST and JH: coauthors, secondary project supervisors.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.