Toward mission-dependent long robotic arm enhancement: design method of flying watch attachment allocation based on thrust drivability

Abstract

Long robotic arms are useful for many applications such as nuclear plant decommissioning, inspection, and firefighting. A major problem for designing and operating long robotic arms is that even small end effector reaction forces and arm gravity can result in large loads on proximal arm joints because of long moment arms. To solve that problem, previous researches focus on specifically designed long arms with certain compensation mechanisms. However, those specialized arm designs are difficult to be applied to existing long robotic arms and to be customized for different missions. To overcome those two drawbacks, we recently proposed a watch-like thrust-generating modular device, called flying watch, with the following two major advantages. Firstly, flying watch can be attached to different kinds of existing long robotic arms and generate thrusts to enhance arm strength. And we have proposed a thrust planning method for flying watch in our previous work. Secondly, since different flying watch attachment allocations can enhance the same robotic arm in different ways, flying watch attachment allocations can be customized to meet the needs of a specific mission. However, up to now, customizing flying watch attachment allocations to different missions is still based on human experience and there is no clear performance metric and automated design method for flying watch attachment allocation. To facilitate mission-dependent long arm enhancement, in this paper, we first propose a novel performance metric, called thrust drivability, which measures the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces. Then based on thrust drivability, we propose an automated design method, called Allocation Optimization based on Weighted Situations (AOWS), for generating mission-dependent flying watch attachment allocations counteracting both unexpected and known external forces. Simulations show that AOWS based allocation designs can counteract both known and unexpected external forces much better than human-experience-based allocation designs.

Introduction

Long-reach robotic arms are very useful for many applications such as nuclear power plant decommissioning [1, 2], inspection [3,4,5], and firefighting [6]. One significant problem for designing and operating a long-reach arm is that even small end effector reaction forces and arm gravity can result in large loads on proximal joints that can exceed their actuation abilities. To solve this problem, previous studies focus on specialized arm designs with passive force compensation mechanisms based on buoyancy [3], spring [4, 7, 8], and weight [9] and active force compensation mechanisms based on tendon actuation [1, 2, 10], water jet propulsion [5, 6], and thrust propulsion [11,12,13]. However, specialized arm designs have two major problems. (1) They are difficult to be applied to existing long arms. (2) They are difficult to be customized to different missions.

To solve those two problems, we recently proposed a watch-like modular device called flying watch [14], as shown in Fig. 1. Flying watch has two major advantages. (1) Flying watch can be easily attached to existing long robotic arms and generate thrusts to enhance arm strength and we have already proposed thrust planning method for planning appropriate flying watch thrusts for arm enhancement in [14]. Flying watch is clearly different from thrust driven arms [11,12,13] in that flying watch is used for cooperating with existing actuators on long arms and the thrusters in [11,12,13] are used to drive passive joints. (2) Since different flying watch attachment allocations can enhance the same robotic arm in different ways, flying watch attachment allocations can be customized for a specific mission. For example, in Fig. 2a, when the mission of a long arm is to pick up a heavy object, we can attach four flying watches with thrust generating directions in vertical planes containing the corresponding attached links to counteract the gravity of the arm and the heavy object. When the mission of the arm is to push or pull a heavy object on the ground in different directions, we can attach two flying watches with thrust generating directions in the horizontal planes and another two flying watches with thrust generating directions in the vertical planes containing the corresponding attached links. Such flying watch attachment allocation helps counteracting vertical arm gravity and horizontal frictions of the heavy object.

Fig. 1

a Mechanical design of a flying watch (one propeller rotates in clockwise (CW) direction and the other propeller rotates in counterclockwise (CCW) direction). b Flying watch attached to a robotic arm link. A wrist watch is also included in the photo to illustrate the structure similarity between a flying watch and a wrist watch. c Flying watch module before attaching to robot links. This figure is adapted from [14]

Fig. 2

a Flying watches help a long arm pick up an object. b Flying watches help a long arm push debris on the ground in different directions. Rotation axes of red flying watches are on horizontal planes. Rotation axes of green flying watches are on vertical planes containing the corresponding attached links

However, toward mission-dependent long robotic arm enhancement, two significant problems remain. Firstly, for a certain mission, there is no clear metric for comparing different flying watch attachment allocations. Although there are researches about robotic kinematic and dynamic performance metrics such as manipulability [15,16,17,18], mobility [19, 20], and dexterity [21, 22]. Those metrics cannot evaluate thrust enhancement effect to reduce joint loads. Secondly, although we can roughly design the flying watch attachment allocation based on human experience, the experience-based designs are far from optimal designs and may take lots of human efforts. This problem can be more severe as the degree of freedom of the arm and mission complexity increase.

In addition, it is highly demanded that the evaluation metric and automated design method for flying watch attachment allocation can consider unexpected end effector reaction forces, whose exact directions and magnitudes cannot be predicted before a mission. Such unexpected end effector reaction forces happen very often due to winds, interaction with unknown environment, and collision. A flying watch attachment allocation design without considering unexpected end effector reaction forces may fail to enhance a long arm when the directions and magnitudes of the unexpected end effector reaction forces fall within the design weakness. When such enhancement failure happens, even small unexpected end effector reaction forces can overload proximal joints and cause very serious consequences. For example, although the four flying watches in Fig. 2a can enhance the arm to counteract gravity, if an unexpected side wind happens, the horizontal wind force may overload the 1st and 3rd joints and result in arm failure.

To facilitate mission-dependent long arm enhancement, in this paper, we will first propose a novel performance metric, called Thrust Drivability, which measures the ability of flying watch attachment allocation to counteract unexpected end effector reaction forces in “Thrust drivability” section. The derivation of Thrust Drivability also involves a more illustrative graphical representation of enhancement performance of flying watch attachment allocation, called Thrust Drivability Surface (TDS). Then based on Thrust Drivability, in Section III, we will propose an automated design method, called Allocation Optimization based on Weighted Situations (AOWS), for designing mission-dependent flying watch attachment allocations that can counteract both unexpected and known external forces (including arm gravity). Then in Section IV, based on simulations, we will demonstrate designing flying watch attachment allocations using AOWS for a static object holding mission and an object manipulation mission and show AOWS based allocation designs can counteract both known and unexpected external forces much better than human-experience-based allocation designs. Finally, we will conclude this paper in Section V.

The researches about sensor/actuator placement optimization for robotic or mechatronic systems are closely related to this paper, since AOWS is essentially an automated optimization algorithm for actuation unit placement. In [23,24,25], sensor/actuator placement optimization methods are proposed for active vibration control systems. In those systems, sensors and actuators are placed on flexible structures to control unwanted vibrations and optimal placements of sensor/actuators can make unwanted vibration decay faster. In [26], the actuator placement of a direct-drive stage is optimized to increase control bandwidth. In [27], the actuator placement of an exoskeleton is optimized to a given mission. The optimized actuator placement can maximize the stiffness of the exoskeleton in the mission workspace. In [28], the placement of curvature sensors along a continuum robotic arm is optimized in order to reduce shape reconstruction error of the arm. Although those researches also considered optimizing the placement of sensors or actuation units to improve performance of a robotic/mechatronic systems, they have very different specific goals compared with AOWS. The goal of AOWS is to find a flying watch attachment allocation that can minimize the arm joint loads under expected or unexpected disturbances. The goals of the mentioned sensor/actuator placement optimization methods include vibration suppression, control bandwidth maximization, and stiffness maximization, etc.

The contributions of this paper are as follows. (1) Proposing a novel performance metric, called Thrust Drivability, which can measure the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces. (2) Proposing an automated design method based on thrust drivability, called Allocation Optimization based on Weighted Situations (AOWS), for designing mission-dependent flying watch attachment allocations counteracting both unexpected and known external forces (including arm gravity). (3) Verifying through simulations that AOWS can generate flying watch attachment allocations with much better ability to counteract both known and unexpected external forces than human-experience-based allocation designs.

Thrust drivability

Overview

In this section, we will first briefly review the flying watch Equation of Motion (EoM) and reduction rate proposed in [14] to make this paper self-contained. Based on the EoM and reduction rate, we will introduce Thrust Drivability Surface (TDS), which is an illustrative graphical method for analyzing the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces from different directions. Finally, based on TDS, Thrust Drivability will be introduced as a quantitative metric representing the performance of a flying watch attachment allocation to counteract unexpected end effector reaction forces.

Equation of motion and reduction rate

The equation of motion of a robotic arm actuated by K actuators and attached with N flying watches is introduced in [14] as follows.

$${\varvec{M}}\left({\varvec{q}}\right)\ddot{{\varvec{q}}}+{\varvec{h}}\left({\varvec{q}},\dot{{\varvec{q}}}\right)-{{\varvec{J}}}^{T}\left({\varvec{q}}\right){{\varvec{F}}}_{e}-{\varvec{R}}\left({\varvec{q}}\right){\varvec{s}}+{\varvec{g}}\left({\varvec{q}}\right)={\varvec{\tau}}$$

(1)

${\varvec{M}}$ is the manipulator inertia tensor. ${\varvec{q}}$ is the generalized coordinates defining arm configuration.${\varvec{h}}$ is the velocity term. ${\varvec{J}}$ is the Jacobian matrix. ${{\varvec{F}}}_{e}$ is the end effector reaction force.${\varvec{s}}={\left[{s}_{1},\dots , {s}_{N}\right]}^{T}$ is the flying watch thrust magnitude vector collecting the thrust magnitudes of all N flying watches. ${\varvec{g}}$ is the gravity term. ${\varvec{\tau}}$ is the joint load vector. ${\varvec{R}}$ is the attachment style matrix defined as follows.

$${\varvec{R}}\left({\varvec{q}}\right)=[{{\varvec{J}}}_{1}^{fT}\left({\varvec{q}}\right){{\varvec{a}}}_{1}\left({\varvec{q}}\right),\dots , {{\varvec{J}}}_{k}^{fT}\left({\varvec{q}}\right){{\varvec{a}}}_{k}\left({\varvec{q}}\right),\dots , {{\varvec{J}}}_{N}^{fT}\left({\varvec{q}}\right){{\varvec{a}}}_{N}\left({\varvec{q}}\right)]$$

(2)

In (2), ${{\varvec{J}}}_{k}^{f}$ and ${{\varvec{a}}}_{k}$ are respectively the Jacobian matrix and the unit direction vector of the k th flying watch. Attachment style matrix represents the mapping between thrust induced joint loads and thrust magnitudes.

Many long robotic arm tasks, such as those related to infrastructure inspection, prefer slow arm motions. For example, if we use a long arm with a camera on the end effector to inspect a bridge. The camera requires static status or quasistatic movements to obtain clear images or videos. In addition, long robotic arms are highly susceptible to collisions because of their size and large workspaces. Slow arm motions are preferred to predict and avoid collisions. Because of those two reasons, in this paper, we assumed static or quasistatic situation as a first step to study flying watch allocation customization. Therefore, (1) can be simplified as follows.

$$-{{\varvec{J}}}^{T}\left({\varvec{q}}\right){{\varvec{F}}}_{e}-{\varvec{R}}\left({\varvec{q}}\right){\varvec{s}}+{\varvec{g}}\left({\varvec{q}}\right)={\varvec{\tau}}$$

(3)

Since different arm actuators may have different specifications, it is necessary to normalize joint loads before comparing them. Normalized joint loads is defined in [14] as $\stackrel{\sim }{{\varvec{\tau}}}={\varvec{H}}{\varvec{\tau}}$, where ${\varvec{H}}=diag(\frac{1}{{\tau }_{p1}},\dots ,\frac{1}{{\tau }_{pK}})$ is the normalization matrix and ${\tau }_{p1},\dots ,{\tau }_{pK}$ are the maximum permitted loads of the K joints. When all arm actuators have the same specification, for simplicity, we can make ${\varvec{H}}$ an identity matrix. By multiplying both sides of (3) with ${\varvec{H}}$, and defining $\stackrel{\sim }{{\varvec{J}}}={\varvec{J}}{\varvec{H}}$, $\stackrel{\sim }{{\varvec{R}}}={\varvec{H}}{\varvec{R}}$, and $\stackrel{\sim }{{\varvec{g}}}={\varvec{H}}{\varvec{g}}$, we can obtain the follows.

$$-{\stackrel{\sim }{{\varvec{J}}}}^{T}\left({\varvec{q}}\right){{\varvec{F}}}_{e}-\stackrel{\sim }{{\varvec{R}}}\left({\varvec{q}}\right){\varvec{s}}+\stackrel{\sim }{{\varvec{g}}}\left({\varvec{q}}\right)=\stackrel{\sim }{{\varvec{\tau}}}$$

(4)

Reduction rate $\zeta$ was originally proposed in [14] as $\zeta ={(\| \stackrel{\sim }{{\varvec{\tau}}}}_{nw}{ \Vert }_{\infty }-{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{*}\Vert }_{\infty })/{\| \stackrel{\sim }{{\varvec{\tau}}}}_{nw}{ \Vert }_{\infty }$, where ${\stackrel{\sim }{{\varvec{\tau}}}}_{nw}$ is the joint load vector when no flying watch is operating and ${\stackrel{\sim }{{\varvec{\tau}}}}^{*}$ is the joint load vector when flying watches are operating with optimal thrusts. Both ${\stackrel{\sim }{{\varvec{\tau}}}}_{nw}$ and ${\stackrel{\sim }{{\varvec{\tau}}}}^{*}$ are calculated under the same attachment allocation and the purpose of that reduction rate is to evaluate the performance of a flying watch thrust planner. As a further step, in this paper, we extend reduction rate to compare different flying watch attachment allocations. Since both flying watch thrusts and flying watch attachment allocations influence the maximum normalized joint loads, we need to optimize flying watch thrusts before using reduction rate to compare flying watch attachment allocation. The extended reduction rate is defined as follows.

$$\zeta =\frac{{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}\left({{\varvec{s}}}_{bm}^{*}\right)\Vert }_{\infty }-{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{coi}\left({{\varvec{s}}}_{coi}^{*}\right)\Vert }_{\infty }}{{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}\left({{\varvec{s}}}_{bm}^{*}\right)\Vert }_{\infty }}$$

(5)

${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}\left({{\varvec{s}}}_{bm}^{*}\right)\Vert }_{\infty }$ is the infinity norm of the normalized joint load vector for a benchmark case, given optimized flying watch thrusts ${{\varvec{s}}}_{bm}^{*}$. ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{coi}\left({{\varvec{s}}}_{coi}^{*}\right)\Vert }_{\infty }$ is the infinity norm of the normalized joint load vector for a case of interesting, given optimized flying watch thrusts ${{\varvec{s}}}_{coi}^{*}$. Under this more general definition of reduction rate, the reduction rate defined in [14] is a special case when the flying watch thrusts (and flying watch mass) of the benchmark case are forced to be zero and the flying watch attachment allocations of the benchmark case and the case of interest are forced to be the same. One way to obtain ${{\varvec{s}}}_{coi}^{*}$ and ${{\varvec{s}}}_{bm}^{*}$ is to solve Problem 1, which will be used throughout this paper for computing reduction rate if there is no special indication.

Problem 1: thrust optimization (infinity norm and hard constraint)

$$\underset{{\varvec{s}}}{\mathrm{min}}{\Vert \stackrel{\sim }{{\varvec{\tau}}}\left({\varvec{s}}\right)\Vert }_{\infty } s.t. {{\varvec{s}}}_{l}\le {\varvec{s}}\le {{\varvec{s}}}_{u}$$

In Problem 1, $\stackrel{\sim }{{\varvec{\tau}}}$ is the normalized joint load vector calculated using (4). ${{\varvec{s}}}_{l}$ and ${{\varvec{s}}}_{u}$ are the lower and upper bounds of the flying watch thrust magnitude vector. Problem 1 optimizes flying watch thrusts to minimize the infinity norm of the normalized joint load vector, subject to thrust upper and lower bounds. This optimization problem can be solved by interior point method [29].

Reduction rate can be used as a metric to measure the performance of flying watch attachment allocations to counteract known arm gravity and known end effector reaction forces. However, the drawback of reduction rate is that it cannot measure the performance of flying watch attachment allocations to counteract unexpected end effector reaction forces since we must know end effector reaction force to optimize flying watch thrusts using (4).

Thrust drivability surface

Since unexpected end effector reaction forces (resulting from object manipulation, collision, wind, etc.) happen very often in practice and reduction rate cannot measure the performance of flying watch attachment allocations to counteract unexpected end effector reaction forces, we propose TDS based on reduction rate to solve this problem.

Firstly, since the performance of flying watch attachment allocations to counteract known arm gravity and known end effector reaction force can be evaluated using reduction rate, we focus on unexpected end effector reaction force by separating flying watch EoM (4) as follows.

$$-{\stackrel{\sim }{{\varvec{J}}}}^{T}({\varvec{q}}){{\varvec{F}}}_{k}-\stackrel{\sim }{{\varvec{R}}}\left({\varvec{q}}\right){{\varvec{s}}}_{k}+\stackrel{\sim }{{\varvec{g}}}\left({\varvec{q}}\right)={\stackrel{\sim }{{\varvec{\tau}}}}_{k}$$

(6)

$$-{\stackrel{\sim }{{\varvec{J}}}}^{T}\left({\varvec{q}}\right){{\varvec{F}}}_{u}-\stackrel{\sim }{{\varvec{R}}}\left({\varvec{q}}\right){{\varvec{s}}}_{u}={\stackrel{\sim }{{\varvec{\tau}}}}_{u}$$

(7)

In (6) and (7), ${{\varvec{F}}}_{e}={{\varvec{F}}}_{k}+{{\varvec{F}}}_{u}$, where ${{\varvec{F}}}_{k}$ is the known end effector reaction forces and ${{\varvec{F}}}_{u}$ is the unexpected end effector reaction forces. ${\varvec{s}}={{\varvec{s}}}_{k}+{{\varvec{s}}}_{u}$, where ${{\varvec{s}}}_{k}$ is the flying watch thrusts for counteracting known arm gravity and known end effector reaction forces and ${{\varvec{s}}}_{u}$ is the flying watch thrust magnitude vectors for counteracting unexpected end effector reaction forces. $\stackrel{\sim }{{\varvec{\tau}}}={\stackrel{\sim }{{\varvec{\tau}}}}_{k}+{\stackrel{\sim }{{\varvec{\tau}}}}_{u}$, where ${\stackrel{\sim }{{\varvec{\tau}}}}_{k}$ are the joint loads resulted from known arm gravity and known end effector reaction forces and ${\stackrel{\sim }{{\varvec{\tau}}}}_{u}$ is joint loads resulted from unexpected end effector reaction forces.

Then let us optimize the thrust magnitude vector in (7) by solving the following Problem 2.

Problem 2: thrust optimization (2-norm and soft constraint)

$$\underset{{\varvec{s}}}{\mathrm{min}}\| {-{\stackrel{\sim }{{\varvec{J}}}}^{T}{{\varvec{F}}}_{u}-\stackrel{\sim }{{\varvec{R}}}{{\varvec{s}}}_{u}\Vert }_{2}^{2}+{{\lambda }^{2}\Vert {{\varvec{s}}}_{u}\Vert }_{2}^{2}$$

Problem 2 intends to minimize the 2-norm squares of both normalized torque vector and flying watch thrusts vector. Problem 2 can be solved using damped least squares method [30, 31] as follows.

$${{\varvec{s}}}_{u}^{+}={-\stackrel{\sim }{{\varvec{R}}}}^{+}{\stackrel{\sim }{{\varvec{J}}}}^{T}{{\varvec{F}}}_{u}=-{\stackrel{\sim }{{\varvec{R}}}}^{T}{\varvec{b}}$$

(8)

where

$${\varvec{b}}={\left(\stackrel{\sim }{{\varvec{R}}}{\stackrel{\sim }{{\varvec{R}}}}^{T}+{\lambda }^{2}{\varvec{I}}\right)}^{-1}({\stackrel{\sim }{{\varvec{J}}}}^{T}{{\varvec{F}}}_{u})$$

(9)

In (8), ${\stackrel{\sim }{{\varvec{R}}}}^{+}={\stackrel{\sim }{{\varvec{R}}}}^{T}{\left(\stackrel{\sim }{{\varvec{R}}}{\stackrel{\sim }{{\varvec{R}}}}^{T}+{\lambda }^{2}{\varvec{I}}\right)}^{-1}$ is the damped pseudoinverse matrix of the normalized attachment style matrix $\stackrel{\sim }{{\varvec{R}}}$. Since ${\stackrel{\sim }{{\varvec{R}}}}^{+}$ includes the inverse of $\stackrel{\sim }{{\varvec{R}}}{\stackrel{\sim }{{\varvec{R}}}}^{T}+{\lambda }^{2}{\varvec{I}}$, which could be a large matrix for arms with many degrees of freedom, directly computing ${\stackrel{\sim }{{\varvec{R}}}}^{+}$ is computationally intensive. Therefore, we introduce an intermediate load vector ${\varvec{b}}$, which is the solution of $\left(\stackrel{\sim }{{\varvec{R}}}{\stackrel{\sim }{{\varvec{R}}}}^{{\varvec{T}}}+{\lambda }^{2}{\varvec{I}}\right){\varvec{b}}={\stackrel{\sim }{{\varvec{J}}}}^{T}{{\varvec{F}}}_{u}$. Such linear equation can be efficiently solved using LU decomposition. It is theoretically possible that $\stackrel{\sim }{{\varvec{R}}}{\stackrel{\sim }{{\varvec{R}}}}^{T}+{\lambda }^{2}{\varvec{I}}$ in (9) is not invertible. If that happens, please slightly change $\lambda$ and $\stackrel{\sim }{{\varvec{R}}}{\stackrel{\sim }{{\varvec{R}}}}^{T}+{\lambda }^{2}{\varvec{I}}$ will be invertible again.

We optimize flying watch thrusts for TDS and Thrust Drivability by solving Problem 2 instead of Problem 1 for the following two reasons. (1) TDS and Thrust Drivability are intrinsic properties of a flying watch attachment allocation characterizing its ability to counteract uncertain end effector reaction forces. Therefore, TDS and Thrust Drivability should not depend on flying watch specifications (such as the maximum thrust of a specific flying watch). Therefore Problem 2 with soft constraints on flying watch magnitudes is better than Problem 1 with hard constraints enforcing specific flying watch thrust capacity. (2) Problem 2 with a concise analytical solution (8), (9) can be solved much more efficiently than Problem 1 solved with iterative interior point method [29]. We will later see that this advantage is the basis of computing TDS with high resolution and incorporating Thrust Drivability into the cost function of AOWS, both of which optimize flying watch thrusts intensively.

Now we can evaluate how much ${{\varvec{s}}}_{u}^{+}$ can reduce ${\| \stackrel{\sim }{{\varvec{\tau}}}}_{u}{\Vert }_{\infty }$ compared with no thrust case given an unexpected unit end effector reaction force ${{\varvec{F}}}_{u}={\varvec{e}}(\theta ,\phi )$. $\theta$ and $\phi$ are respectively polar and azimuthal angles of the unexpected unit end effector reaction force. From (7), we can obtain the follows.

$$\rho \left(\theta ,\phi \right)=\frac{{\Vert -{\stackrel{\sim }{{\varvec{J}}}}^{T}{\varvec{e}}\left(\theta ,\phi \right)\Vert }_{\infty }-{\Vert -{\stackrel{\sim }{{\varvec{J}}}}^{T}{\varvec{e}}\left(\theta ,\phi \right)-\stackrel{\sim }{{\varvec{R}}}{{\varvec{s}}}_{u}^{+}\Vert }_{\infty }}{{\Vert -{\stackrel{\sim }{{\varvec{J}}}}^{T}{\varvec{e}}\left(\theta ,\phi \right)\Vert }_{\infty }}$$

(10)

In (10), $\rho \left(\theta ,\phi \right)$ gives the relative ratio of ${\| \stackrel{\sim }{{\varvec{\tau}}}}_{u}{ \Vert }_{\infty }$ that ${{\varvec{s}}}_{u}^{+}$ can reduce compared with no thrust case. We can see from (8) and (9) that the mapping from ${{\varvec{F}}}_{u}$ to ${{\varvec{s}}}_{u}^{+}$ is a linear transformation. For a given arm configuration and a given flying watch attachment allocation, the Jacobian matrix (${\varvec{J}}$) and attachment style matrix (${\varvec{R}}$) are fixed. As a result, ${{\varvec{s}}}_{u}^{+}$ equals a constant matrix times ${{\varvec{F}}}_{u}$ which is a linear transformation. Without loss of generality, we calculated $\rho$ assuming unit unexpected end effector reaction force as described by (10). Suppose the unexpected end effector reaction force is magnified by a factor. The optimal flying watch thrusts will be magnified by the same factor and the $\rho$ calculated from (10) will not change. Therefore, the unit unexpected end effector reaction force assumption of (10) has no problem.

Now let us look at another interpretation of $\rho \left(\theta ,\phi \right)$ from the perspective of the total joint load $\stackrel{\sim }{{\varvec{\tau}}}={\stackrel{\sim }{{\varvec{\tau}}}}_{k}+{\stackrel{\sim }{{\varvec{\tau}}}}_{u}.$ Let the reduction rate $\zeta ={(\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }-{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{k}^{*}\Vert }_{\infty })/{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }$, where ${\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}$ is the joint loads of a benchmark allocation due to known effects corresponding to ${{\varvec{s}}}_{bm}^{*}$ solved from Problem 1 without constraints and ${\stackrel{\sim }{{\varvec{\tau}}}}_{k}^{*}$ is the joint loads of the current allocation due to known effect corresponding to ${{\varvec{s}}}_{k}^{*}$ solved from problem 1 without constraints. Also, (10) can be concisely written as $\rho =({\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }-{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{+}\Vert }_{\infty })/{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }$, where ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }$ and ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{+}\Vert }_{\infty }$ are respectively the maximum absolute value of normalized joint loads resulting from unexpected end effector reaction forces with zero flying watch thrusts and optimal flying watch thrusts ${{\varvec{s}}}_{u}^{+}$ obtained from Problem 2. The optimum infinity norm of total normalized joint load ${\stackrel{\sim }{{\varvec{\tau}}}}^{\boldsymbol{*}}$ solved from Problem 1 without constraints satisfies the follows.

$${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{*}\Vert }_{\infty }\le {\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{k}^{*}+{\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{+}\Vert }_{\infty }\le {\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{k}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{+}\Vert }_{\infty }={\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }\left(1-\zeta \right)+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }(1-\rho )$$

(11)

In (11), the first inequality is true because ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{*}\Vert }_{\infty }$ is the minimum value of Problem 1 without constraints. The second inequality is triangle inequality. The equality is because the definition of $\zeta$ and $\rho$. Dividing (11) by ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }$, we can obtain

$$\frac{{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{*}\Vert }_{\infty }}{{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }}\le \left(1-k\right)\left(1-\zeta \right)+k\left(1-\rho \right)\le \mathrm{max}(1-\zeta ,1-\rho )$$

(12)

where $k={\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }/({\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty })$ is the weight factor of the unexpected end effector reaction force. Therefore $\mathrm{max}\left(1-\zeta ,1-\rho \right)$ is the upper bound of the relative infinity norm of optimal normalized joint load vector ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{\boldsymbol{*}}\Vert }_{\infty }/({\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty })$ given both known arm gravity, known end effector reaction forces, and unexpected end effector reaction forces. A flying watch attachment allocation with larger $\rho \left(\theta ,\phi \right)$ helps reducing the relative infinity norm of optimal normalized total joint load vector, ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{\boldsymbol{*}}\Vert }_{\infty }/({\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }),$ given that the flying watches have sufficient thrust capability. Also, for a certain arm, if the unexpected end effector reaction force and arm configuration are fixed, ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{bm}^{*}\Vert }_{\infty }+{\Vert {\stackrel{\sim }{{\varvec{\tau}}}}_{u}^{0}\Vert }_{\infty }$ is also fixed. Therefore, increasing $\rho \left(\theta ,\phi \right)$ helps reducing the infinity norm of optimal normalized total joint load ${\Vert {\stackrel{\sim }{{\varvec{\tau}}}}^{\boldsymbol{*}}\Vert }_{\infty }$ given the same arm configuration, the same unexpected end effector reaction force, and sufficient maximum flying watch thrust.

If we regard $\rho$ as radial distance then (10) is actually the parametric equation of a closed surface. We called this surface Thrust Drivability Surface (TDS). We can uniformly sample $\theta \in \left[0,\pi \right], \phi \in [\mathrm{0,2}\pi ]$ with certain resolution and calculate $\rho \left(\theta ,\phi \right)$ to visualize TDS. TDS can visualize the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces with different magnitudes and directions. An example TDS is shown in Fig. 6 for the long arm and flying watch attachment allocation shown in Fig. 4. The arm is requested to hold an object with 15 N gravity while counteracting 10 N horizontal wind. The arm configuration for holding the object is shown in Fig. 6(a) and the front, back, top, and bottom views of the TDS is shown in Fig. 6b–e (the front view shares the same view direction of Fig. 4). In Fig. 4, the flying watches are represented by cylinders for simplicity and the thrust generating direction of a flying watch is represented by the arrow on the cylinder. We will further explain Figs. 4, 6 in Sect. 4.

Thrust drivability

We now hope to derive a quantitative metric from TDS to measure the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces with different magnitudes and directions.

Let us first consider what an ideal TDS looks like. The ideal situation is that a flying watch attachment allocation can counteract 100% unexpected end effector reaction force in any direction with any magnitude. In other words, ${\Vert -{\stackrel{\sim }{{\varvec{J}}}}^{T}{\varvec{e}}\left(\theta ,\phi \right)-\stackrel{\sim }{{\varvec{R}}}{{\varvec{s}}}_{u}^{+}\Vert }_{\infty }$ in (10) is always zero regardless of $\theta$ and $\phi$. In this ideal situation, the TDS is a unit sphere.

Based on this observation, we can obtain a quantitative metric by comparing the volume of a TDS with a unit sphere. To measure the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces within polar angle $\theta \in [{\theta }_{l},{\theta }_{u}]$ and azimuthal angle $\phi \in [{\phi }_{l}, {\phi }_{u}]$, we can compare the corresponding volumes of the TDS and unit sphere.

$$D=\frac{{V}_{TDS}\left({\varvec{\omega}}\right)}{{V}_{unit}\left({\varvec{\omega}}\right)}$$

(13)

In (13), ${\varvec{\omega}}=[{\theta }_{l},{\theta }_{u},{\phi }_{l},{\phi }_{u}]$ is the direction region of the unexpected end effector reaction force. ${V}_{TDS}\left({\varvec{\omega}}\right)$ and ${V}_{unit}\left({\varvec{\omega}}\right)$ are respectively the volume of TDS and unit sphere inside the direction region defined by ${\varvec{\omega}}$. The ratio of those two volumes is defined as Thrust Drivability, denoted as D throughout this paper. Thrust Drivability is a quantitative metric of the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces with directions in ${\varvec{\omega}}$ and unknown magnitudes.

In spherical coordinate, since the infinitesimal volume can be written as $dV={\rho }^{2}sin\theta d\theta d\phi d\rho ,$ we can write (13) as follows.

$$D=\frac{{\int }_{{\theta }_{l}}^{{\theta }_{u}}{\int }_{{\phi }_{l}}^{{\phi }_{u}}{\rho }^{3}\left(\theta ,\phi \right)sin\theta d\theta d\phi }{(cos{\theta }_{l}-cos{\theta }_{u})({\phi }_{u}-{\phi }_{l})}$$

(14)

We can discretize (14) as follows to make it easier for numerical calculation. When deciding the discretization resolution for implementing Thrust Drivability, the purpose of Thrust Drivability and the tradeoff between computation ability and evaluation precision should be considered. If Thrust Drivability is used as a part of an optimization cost function that is heavily evaluated, a rough resolution that ensures reasonable computation time but still preserves the rough shape of the corresponding TDS is recommended. When Thrust Drivability is used only for attachment allocation evaluation, more precise resolution is recommended.

$$D=\frac{\sum_{{\theta }_{l}}^{{\theta }_{u}}\sum_{{\phi }_{l}}^{{\phi }_{u}}{\rho }^{3}\left(\theta ,\phi \right)sin\theta \Delta \theta \Delta \phi }{(cos{\theta }_{l}-cos{\theta }_{u})({\phi }_{u}-{\phi }_{l})}$$

(15)

Up to now, we derived TDS and Thrust Drivability only considering unexpected end effector reaction forces. Therefore, the TDS and Thrust Drivability we derived are more specifically Force TDS and Force Thrust Drivability. In addition, we can also derive Torque TDS and Torque Thrust Drivability by applying similar derivation process to unexpected end effector reaction torques. Without special indications, TDS and Thrust Drivability in the following Sections refer to the Force TDS and Force Thrust Drivability.

Allocation optimization based on weighted situations

Overview

In this Section, we propose a novel design method for designing mission-dependent flying watch attachment allocation, called Allocation Optimization based on Weighted Situations (AOWS). We will first introduce a mission model, called Weighted Situation Model (WSM) and a convenient description of flying watch attachment allocation based on Denavit–Hartenberg convention (DH Convention) [32], called DH-Allocation Description (DHAD). Then based on WSM and DHAD, we will propose AOWS. Finally, we will summarize the overall attachment allocation design procedure.

Weighted situation model

In an arm manipulation mission, an arm may experience a combination of static and moving processes and different kinds of external forces may be exerted on the end effector. A mission can be described by a 2-tuple ${\varvec{\Pi}}=({\varvec{E}},{\varvec{P}})$, where ${\varvec{E}}=\{{{\varvec{e}}}_{1},\dots ,{{\varvec{e}}}_{W}\}$ is a set of $W$ representative situations during a mission and ${\varvec{P}}=\{{p}_{1},\dots ,{p}_{W}\}$ is a set of importance factors corresponding to situations in ${\varvec{E}}$. The $i$ th situation is defined as a 2-tuple ${{\varvec{e}}}_{i}=({{\varvec{q}}}_{i}, {{\varvec{F}}}_{ei}$), where ${{\varvec{q}}}_{i}$ is the generalized coordinates defining arm configuration in the situation and ${{\varvec{F}}}_{ei}$ is the known end effector reaction force at this situation. A situation depicts an arm status at a representative moment during a mission. The importance factor of a situation describes the significance of a situation, which may be determined by the probability of a situation or designers’ judgement. For example, the designer may assign a high importance factor to a rare situation that may cause serious consequences.

DH allocation description

To optimize flying watch attachment allocation, an intuitive way is to directly optimize the absolute positions and orientations of flying watches regarding the fixed base frame. However, since flying watch must be fixed on arm links as the arm moves, optimizing such absolute allocation description will result in complicated constraints. To simplify the allocation optimization problem, we propose to use a relative allocation description based on DH Convention.

In DH Convention, the $i$ th DH frame is attached to the $i$ th arm link and four parameters are defined to specify the relative movement between the i th DH frame and the i-1 th DH frame. Since a flying watch is rigidly attached to a link and there is no relative movement between the watch and the link, it is not necessary to assign an additional DH frame to a flying watch. We can instead just define the position and orientation of the flying watch regarding the DH frame corresponding to the attached link to specify flying watch attachment allocation. Such allocation description is independent from the arm configuration and can help simplify the flying watch allocation optimization. Based on that motivation, the allocation of a flying watch on the $i$ th link is defined by a vector of 6 elements, ${{\varvec{\mu}}}_{f}=\left[ {{\varvec{\mu}}}_{d},{{\varvec{\mu}}}_{\eta }\right]=[{\mu }_{d1},{\mu }_{d2},{\mu }_{d3},{\mu }_{\eta 1},{\mu }_{\eta 2},{\mu }_{\eta 3}]$. ${{\varvec{\mu}}}_{d}=$[${\mu }_{d1},{\mu }_{d2},{\mu }_{d3}$] is the position of the flying watch regarding the $i$ th DH frame. ${{\varvec{\mu}}}_{\eta }=[{\mu }_{\eta 1},{\mu }_{\eta 2},{\mu }_{\eta 3}]$ is the unit thrust direction vector regarding the $i$ th DH frame. We call such kind of description as Full DH Allocation Description (DHAD). And ${{\varvec{\mu}}}_{f}$ is called Full DHAD vector. We can easily convert ${{\varvec{\mu}}}_{d}$ and ${{\varvec{\mu}}}_{\eta }$ to position vector ${\varvec{d}}={{\varvec{R}}}_{i}{{\varvec{\mu}}}_{d}+{{\varvec{r}}}_{i}$ and unit thrust direction vector ${\varvec{a}}={{\varvec{R}}}_{i}{{\varvec{\mu}}}_{\eta }$ regarding the base frame, where ${{\varvec{R}}}_{i}$ and ${{\varvec{r}}}_{i}$ are respectively the rotation matrix and position vector of the $i$th DH frame. The absolute positions and directions of flying watches are necessary for computing the attachment style matrix or visualizing flying watches. Please note that the DHAD is different from DH convention. The four-parameter description in DH convention describes the relative motion between two neighbor DH frames while the DHAD vector is essentially a combination of position and direction coordinates of a flying watch regarding the DH frame corresponding to the attached link.

Under the following assumptions, which are often applicable in practice, it is possible to simplify the full DHAD description. (1) All links are straight and perpendicular to their joint axes. (2) All joints are rotary. DHAD vector of a flying watch attached to the $i$ th link can be further simplified as a vector with two elements, ${{\varvec{\mu}}}_{s}=({\mu }_{sd}, {\mu }_{s\eta })$. ${\mu }_{sd}$ is the coordinate of the flying watch on X axis of the $i$ th DH frame, which reflects the position of the flying watch. ${\mu }_{s\eta }$ is the angle between the flying watch thrust vector and Y axis of the $i$ th DH frame which reflects the orientation of the flying watch. We call such simplified description as Simplified DHAD. And ${{\varvec{\mu}}}_{s}$ is called simplified DHAD vector. We can use ${{\varvec{\mu}}}_{f}=\left[{\mu }_{sd},0, 0, 0,cos{\mu }_{s\eta }, sin{\mu }_{s\eta }\right]$ to simply convert ${{\varvec{\mu}}}_{s}$ to ${{\varvec{\mu}}}_{f}$.

The advantage of DHAD is that it separates all allocation information (flying watch attachment position and orientation) from arm configuration and thusly significantly simplifies constraints in allocation optimization process.

Allocation optimization based on weighted situations

We propose a design method called Allocation Optimization based on Weighted Situations (AOWS) for designing mission-dependent flying watch attachment allocation considering both known arm gravity, known end effector reaction forces, and unexpected end effector reaction forces. AOWS solves the following problem.

Problem 3: allocation optimization based on weighted situations (AOWS)

$$\mathop {\min }\limits_{{{\varvec{\mu}},{\varvec{s}}_{1} , \ldots {\varvec{s}}_{W} }} {(1} \ldots \beta {)}\sum\limits_{i = 1}^{W} {p_{i} \frac{{\left\| {\tilde{\user2{\tau }}_{ki} ({\varvec{\mu}},{\varvec{s}}_{i} ,{\varvec{q}}_{i} ,{\varvec{F}}_{ki} )} \right\|_{\infty } }}{{\left\| {\tilde{\user2{\tau }}_{ki}^{0} ({\varvec{q}}_{i} ,{\varvec{F}}_{ki} )} \right\|_{\infty } }}} + \beta \sum\limits_{i = 1}^{W} {p_{i} \sum\limits_{j = 1}^{V} {\alpha_{j} \left( {1 - D_{ij} \left( {{\varvec{\mu}},{\varvec{q}}_{i} ,{\varvec{w}}_{i} } \right)} \right)\;s.t.\;{\varvec{\mu}},{\varvec{s}}_{1} , \ldots {\varvec{s}}_{W} \in {\varvec{C}}} }$$

In Problem 3, ${\varvec{\mu}}$ is the DHAD vector describing the flying watch allocation. Both full DHAD vector and simplified DHAD vector can be used in Problem 3. However, if the assumptions for the simplified DHAD are applicable, the simplified DHAD is recommended since the coordinates of simplified DHAD are completely independent while the last 3 coordinates of a full DHAD vector must form a unit vector, which is an additional constraint. We used simplified DHAD in this paper for simplicity. ${{\varvec{s}}}_{1},\dots ,{{\varvec{s}}}_{W}$ are the corresponding flying watch thrust magnitude vectors of $W$ situations. ${{\varvec{q}}}_{1}$,…,${{\varvec{q}}}_{W}$ are the corresponding generalized coordinates vectors describing arm configurations in the $W$ situations. ${{\varvec{F}}}_{k1}$,…,${{\varvec{F}}}_{kW}$ are the corresponding known end effector reaction forces of the $W$ situations. ${\stackrel{\sim }{{\varvec{\tau}}}}_{k1}$,…,${\stackrel{\sim }{{\varvec{\tau}}}}_{kW}$, which can be reduced by optimizing ${\varvec{\mu}}$ and ${{\varvec{s}}}_{1},\dots ,{{\varvec{s}}}_{W}$, are the corresponding normalized joint load vectors of the $W$ situations resulting from arm gravity (including flying watch gravity) and known end effector reaction forces. ${\stackrel{\sim }{{\varvec{\tau}}}}_{k1}^{0}$,…,${\stackrel{\sim }{{\varvec{\tau}}}}_{kW}^{0}$ are the corresponding joint load vectors of the $W$ situations assuming zero flying watch thrusts and initial flying watch attachment allocation. We used the infinity norm of ${\stackrel{\sim }{{\varvec{\tau}}}}_{k1}^{0}$,…,${\stackrel{\sim }{{\varvec{\tau}}}}_{kW}^{0}$ to normalize the infinity norm of ${\stackrel{\sim }{{\varvec{\tau}}}}_{k1}$,…,${\stackrel{\sim }{{\varvec{\tau}}}}_{kW}$ to range between 0 and 1.${p}_{1},\dots ,{p}_{W}$ are the corresponding importance factors of the $W$ situations. $\beta$ is the unexpected effect weight factor. ${D}_{i1},\dots ,{D}_{iV}$ are the thrust drivabilities for the $i$ th situation in the corresponding $V$ direction regions ${{\varvec{\omega}}}_{1},\dots ,{{\varvec{\omega}}}_{V}$ that we want to maximize. ${\alpha }_{1},\dots ,{\alpha }_{V}$ are the drivability weight factors. ${\varvec{C}}$ is the constraints of ${\varvec{\mu}},{{\varvec{s}}}_{1,\dots ,}{{\varvec{s}}}_{W}$ considering robotic arm and flying watch specifications.

AOWS is minimizing the weighted summation of two terms. The first term (known effect term) is trying to minimize the infinity norm of the joint load vector of each representative situations. The second term (the unknown effect term) is a weighted summation of $1-{D}_{ij}$. Therefore, the purpose of the second term is to maximize Thrust Drivability. We can imagine that there is a trade-off between these two terms since maximizing Thrust Drivability tends to increase the orientation diversity of flying watches while counteracting certain known effect tend to reduce the orientation diversity of flying watches so that all flying watches can focus on counteracting the same known forces. The unexpected effect weight factor $\beta$ allows the user to handle this trade-off. As we have mentioned in subsection 2.2, since the joint load vector ${\stackrel{\sim }{{\varvec{\tau}}}}_{ki}$ depends on both flying watch thrust magnitudes and flying watch attachment allocations, the flying watch thrusts must be optimized together with flying watch attachment allocations. Therefore, both DHAD vector ${\varvec{\mu}}$ and flying watch thrust magnitude vectors ${{\varvec{s}}}_{1},\dots ,{{\varvec{s}}}_{W}$ are the optimization variables. Based on our analysis, we can understand that Problem 3 is solving the flying watch allocation ${\varvec{\mu}}$, which has optimal balanced performance for counteracting both known arm gravity (including flying watch gravity) and known end effector reaction forces as well as unexpected end effector reaction forces. Problem 3 can be solved using interior point method [29].

Overall allocation design procedure

Up to now, we have introduced reduction rate as a metric measuring the ability of flying watch attachment allocations to counteract known arm gravity (including flying watch gravity) and known end effector reaction force. Also, we introduced Thrust Drivability as a metric of flying watch attachment allocations measuring their ability to counteract unknown end effector reaction forces. Based on Thrust Drivability, we proposed AOWS as an automated design tool for designing mission-dependent flying watch attachment allocation. In this Section, we discuss how to put all those components together as a systematic design process.

The overall allocation design process is shown in Fig. 3. The first step is to do mission planning, where the trajectory of the robotic arm during a mission is generated. Then we need to select representative arm situations in the mission and decide their corresponding important factors to form a WSM ${\varvec{\Pi}}=({\varvec{E}},{\varvec{P}})$. Also we need to determine the direction regions ${{\varvec{\omega}}}_{1},\dots ,{{\varvec{\omega}}}_{V}$ of unexpected end effector reaction forces and their corresponding drivability weight factors ${\alpha }_{1},\dots ,{\alpha }_{V}$ as well as the unexpected effect weight factor $\beta$. The mentioned parameters (WSM, direction regions, drivability weight factors, and unexpected effect weight factor) are design parameters. Then those initial parameters are fed to the AOWS to generate an optimal flying watch attachment allocation. Next we need to evaluate the performance of the obtained optimal allocation. The following three methods with different features can be used. (1) Reduction rate is a metric to measure the performance of a flying watch attachment allocation to counteract known arm gravity (including flying watch gravity) and known end effector reaction forces. (2) TDS is a graphical method visualizing the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces with different magnitudes and directions. (3) Thrust Drivability is a numerical metric measuring the ability of a flying watch attachment allocation to counteract unexpected end effector reaction forces with uncertain magnitudes in a direction region.

Fig. 3

The overall design process of flying watch allocation

After evaluation using those three methods, if some performance is not satisfactory, we can adjust the design parameters and run AOWS again. For example, if the obtained allocation cannot counteract unexpected wind forces on the end effector well enough, we can increase unexpected effect weight factor $\beta$. If the obtained allocation cannot counteract known end effector reaction forces in horizontal directions well enough, we can increase importance factors of corresponding situations that include the horizontal forces. This redesign process is repeated until a satisfactory allocation is obtained.

Please note that the acceptable mission scenario for AOWS is that the arm does not significantly deviate from the preplanned trajectory due to unexpected end effector reaction forces. It happens very often in practice that a robotic arm deviates from the preplanned trajectory due to unexpected end effector reaction forces. If the deviation is large, the AOWS-based flying watch attachment allocation based on the preplanned arm trajectory may not perform well. One way to increase the robustness of AOWS to such deviations is to include those situations where the arm deviates from the preplanned trajectory into the weighted situation model so that AOWS can also consider those situations.

Simulation

Overview

In this Section, we simulate a 9-DoF long-reach robotic arm doing two typical arm missions to test the effectiveness of AOWS. The first mission is a static object holding mission, where an arm is required to hold an object with 15 N gravity statically. The second mission is an object manipulation mission, where the arm is required to move the same object from one position to another position along a trajectory. Those two typical missions are components of many more complicated missions. During those two missions, the robotic arm needs to counteract its own gravity, the object gravity, and occasional 10 N side wind on the object in horizontal directions (assuming wind forces on other parts of the arm are neglectable). We will design mission-dependent flying watch allocations for those two missions using AOWS and compare the AOWS-based allocations with human-experience-based allocations. In the rest of this subsection, we will firstly introduce the arm and flying watch specification. Then we will introduce the human-experience-based allocation design. Finally, we will introduce some common implementation details for those two missions.

The arm specification is shown in Fig. 4. The arm is a 9-Dof redundant arm with a motor in each joint. The arm links are made of CFRP with inner diameter of 31 mm and thickness of 2 mm. The arm joint mass (including motors, gears, and housing) is 0.59 kg. The total mass of the arm is 8.81 kg. The base frame is represented by purple arrows and the end effector frame is represented by red arrows. Please note that all coordinate systems in this paper are right-handed coordinate systems. The arm design is derived from an existing long-reach arm in our lab [2] and our feasibility investigation on motors and gears in the market.

Fig. 4

Specification of the simulation arm and the human-experience-based design of flying watch allocation. For simplicity, flying watches are represented by cylinders and the thrust generating direction of a flying watch is represented by the arrow on the cylinder. J1,…,J9 are joint numbers and W1,…, W8 are flying watch numbers

The specification of flying watch is the same as the physical prototype in [14]. The propeller diameter of a flying watch is 381 mm and the maximum thrust of one flying watch is 18 N. The mass of one flying watch is 542 g. The dimension of a flying watch (not including propellers) is length (66 mm), width (56 mm), and height (48 mm).

The human-experience-based flying watch allocation is also shown in Fig. 4. The flying watches are represented by cylinders for simplicity. The thrust generating direction of a flying watch is represented by the arrow on the cylinder. Under the arm configuration of Fig. 4 when all links are in the same vertical plane, green cylinders represent flying watches with thrust generating directions in the vertical link plane and yellow cylinders represent flying watches with thrust generating directions in horizontal planes. Since the gravity of the arm is 88.1 N and the thrust of a flying watch is 12.58 N excluding its gravity, about 7 flying watches are necessary to counteract the arm gravity. We attached 7 flying watches to distal links with thrust generating directions in the vertical link plane for counteracting arm gravity and other vertical end effector reaction forces. We also attached one flying watch on the eighth link with the thrust generating direction in the horizontal plane for counteracting end effector reaction forces in horizontal planes. The distance between a flying watch and its closest joint is 0.4 m. We index the flying watches from 1 to 8 from the most proximal one to the most distal one.

The common implementation details for the static object holding mission and object manipulation mission are as follows. We used Matlab R2019a to simulate the 9-DoF arm and implement AOWS based on the optimization toolbox of Matlab. We assumed that the size and weight of the end effector are negligible. Therefore, for simplicity, the end effector weight is set to zero. In practice, if the end effector is not small and lightweight, the end effector weight can be incorporated by changing the parameters of the equation of motion (4) and the size of the end effector can be incorporated by modifying the constraint parameters of inverse kinematics and motion planning in the simulations. Therefore, the flying watch attachment allocation optimization process demonstrated in the simulations is still applicable even if the end effector is not small and lightweight.

For simplicity, we assumed the motor on each joint are identical. As a result, we used identity matrix as the normalization matrix. For AOWS, we constrain the minimum distance between the flying watch attachment point and its closest joint to be 0.3 m. We can derive from this minimum distance and flying watch dimensions that to avoid collision, the maximum rotation of the fifth and ninth joints should not exceed $135.58^\circ$ and the maximum rotation of the sixth, seventh, eighth joints should not exceed $96.16^\circ$. $5^\circ$ safety margins are included when calculating those joint constraints. For the second to fourth joints, 10 degrees margin is left to ensure adjacent links will not collide. The base joint is free to rotate. Those joint constraints are guaranteed when computing inverse kinematics and generating arm path. We constrain the two flying watches on the same link to be attached in two regions respectively. Those two regions are on different sides of the middle point of the link and the distance between the middle point and the closest boundary point of a region is 0.23 m. That means the minimum distance between the two regions is 0.46 m, which is larger than the propeller diameter of the flying watch (0.381 m). About 8 cm clearance is left for safety reasons. For example, the user may not precisely attach the flying watches. Therefore, no collision will happen between flying watches on the same link.

We used the human-experience-based flying watch allocation with no flying watch thrusts as the initial value of AOWS. When calculating the intermediate load vector using (9), we used $\lambda =0.01$. When calculating Thrust Drivability (D) as a part of AOWS cost function, the resolution for $\theta$ (the polar angle) is $\pi /4$ and the resolution for $\phi$ (the azimuthal angle) is $\pi /2$. We used a computer with Intel i7 CPU (1.8 GHz frequency and 16 GB RAM) for running simulations. Since the Thrust Drivability (D) is a part of the AOWS cost function, which is evaluated tens of thousands of times by the interior point method, the rough resolution is chosen to make sure that AOWS can return a result in reasonable time (i.e. several hours) while the shape of the TDS can roughly be preserved. When calculating Thrust Drivability for evaluating flying watch attachment allocation (e.g. the Thrust Drivability in Tables 2 and 5), more precise resolution is used since those Thrust Drivability are only evaluated a few times and should more precisely reflect the performance of flying watch attachment allocations. The resolution for $\theta$ is $\pi /9$ and the resolution for $\phi$ is $2\pi /9$. In choosing the discretization resolution for Thrust Drivability, we recommend try and error methods. If Thrust Drivability is used in AOWS, firstly, try several rough Thrust Drivability resolutions for AOWS and analyze the result with acceptable high precision Thrust Drivability. Then pick the rough Thrust Drivability resolution for AOWS that lead to acceptable results in reasonable time. If Thrust Drivability is used purely for evaluation, please try several high resolutions and pick a resolution considering computation time. Please note if there is no special indication, all coordinates in this Section are referring to the base frame of the simulation arm.

Please note that we assumed ideal controllers for the arm and flying watches, since the development of arm or flying watch controller is not in the scope of this paper. In other words, the simulated arm configuration is the same as the desired arm configuration.

Static object holding mission

In this mission, the robotic arm will hold an object with gravity of 15 N at [11,2,0.8] m with the end effector frame coincides with the base frame. A 10 N wind load with direction of positive y axis direction of the base frame (probability = 30%) and negative y axis direction of the base frame (probability = 30%) will act on the end effector. We choose [11,2,0.8] as the end effector position since it is far from base frame and the large torques on proximal joints are challenging for the long arm.

To use AOWS, we need to construct a WSM. We firstly solve an inverse kinematics problem with joint constraints to obtain the robot configuration ${{\varvec{q}}}_{m1}=[0.00,-\mathrm{0.07,0.39,0.02},-0.02,-0.01,-\mathrm{0.49,0.05,0.12}]$. Then based on the known end effector reaction forces, we can find the following three representative situations. (1) ${{\varvec{e}}}_{1}=({{\varvec{q}}}_{1}, {{\varvec{F}}}_{1})$, where ${{\varvec{q}}}_{1}={{\varvec{q}}}_{m1}$ and ${{\varvec{F}}}_{1}=\left[\mathrm{0,0},-15\right]N$. (2) ${{\varvec{e}}}_{2}=({{\varvec{q}}}_{2}, {{\varvec{F}}}_{2})$, where ${{\varvec{q}}}_{2}={{\varvec{q}}}_{m1}$ and ${{\varvec{F}}}_{2}=[\mathrm{0,10},-15]N$. (3) ${{\varvec{e}}}_{3}=({{\varvec{q}}}_{3}, {{\varvec{F}}}_{3})$, where ${{\varvec{q}}}_{3}={{\varvec{q}}}_{m1}$ and ${{\varvec{F}}}_{3}=[0,-10,-15]N$. Based on those three representative situations, we can construct a WSM ${{\varvec{\Pi}}}_{\mathrm{m}1}=({{\varvec{E}}}_{m1},{{\varvec{P}}}_{m1})$, where ${{\varvec{E}}}_{m1}=\{{{\varvec{e}}}_{1},{{\varvec{e}}}_{2}, {{\varvec{e}}}_{3}\}$ and ${{\varvec{P}}}_{m1}=\{0.4, 0.3, 0.3\}$. Here we used the probabilities of events as the corresponding importance factors. Although we know the arm needs to counteract its own gravity, the object gravity, and wind forces, we also hope the arm to be able to counteract unexpected environmental forces from varieties of directions due to unexpected wind and collisions. Therefore, we set the direction region of unexpected end effector reaction forces as ${{\varvec{\omega}}}_{m1}=[0,\pi ,0, 2\pi ]$ with drivability weight factor ${\alpha }_{m1}=1$. We set the unexpected effect weight factor ${\beta }_{m1}=0.5$.

We input those design parameters to AOWS. The AOWS-based allocation and the human-experience-based allocation are shown in Table 1. The flying watch positions and orientations in Table 1 correspond to ${\mu }_{sd}$ and ${\mu }_{s\eta }$ respectively, which are Simplified DHAD introduced in subsection 3.3. ${\mu }_{sd}$ is the coordinate of a flying watch on X axis of the corresponding DH frame. ${\mu }_{s\eta }$ is the angle between a flying watch thrust vector and the Y axis of the corresponding DH frame. The human-experience-based allocation and AOWS-based allocation in the second representative situation (${{\varvec{e}}}_{2}=({{\varvec{q}}}_{2}, {{\varvec{F}}}_{2})$, where ${{\varvec{q}}}_{2}={{\varvec{q}}}_{m1}$ and ${{\varvec{F}}}_{2}=[\mathrm{0,10},-15]N)$ are shown in Figs. 5, 6, 7 and 8. We also calculate TDSs for both allocations and show them in Figs. 5, 6, 7 and 8. In this paper, we define front view (the same view as that of Fig. 4) as the view when seeing along positive Y axis direction of the base frame, back view as the view when seeing along negative Y direction of the base frame, top view as the view when seeing along negative Z direction of the base frame, and down view as the view when seeing along the positive Z direction of the base frame. When drawing TDS alone, the center of TDS is moved to the origin of base frame so that the reader can more easily understand the dimensions of TDS. The positive directions of X, Y, Z axes of the base frame are respectively shown using magenta, green, and cyan lines. The blue lines indicate the links of the arm and the black dots indicate the position of the joints. The cyan lines on the joints indicate the joint rotation axes. The end effector reaction force is shown using orange lines. The corresponding optimized flying watch thrusts are also shown using red lines.

Table 1 Aows-based and human-experience-based allocations of static mission

Fig. 5

The front, back, top, and bottom views of the human-experience-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the static object holding mission. a Front view. b Back view. c Top view. d Bottom view

Fig. 6

a Overall view of the human-experience-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the static object holding mission. b Front view of TDS. c Back view of TDS. d Top view of TDS. e Bottom view of TDS

Fig. 7

The front, back, top, and bottom views of the AOWS-based flying watch allocation, end effector reaction force, and the corresponding average thrust drivability surface in the static object holding mission. a Front view. b Back view. c Top view. d bottom view

Fig. 8

a Overall view of the AOWS-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the static object holding mission. b Front view of TDS. c Back view of TDS. d Top view of TDS. e Bottom view of TDS

From Table 1 and Figs. 5, 6, 7 and 8, we can firstly see that compared with the human-experience-based design, AOWS moves several flying watches (1st, 3rd, 7th) significantly closer ($\ge 5$ cm) to the end effector to increase moment arms. The reason why not all flying watch are moved closer to the end effector to increase moment arms is that we used the interior point method to solve the AOWS problem (Problem 3) and the interior point method only returns a local minimal instead of a global minimum. More specifically, the AOWS problem (Problem 3) is a high-dimension nonconvex and nonlinear optimization problem, whose optimization variable has 40 dimensions for the static holding mission (16 dimensions for flying watch attachment allocation and 24 dimensions for flying watch thrusts) and 280 dimensions for the object manipulation mission (16 dimensions for flying watch attachment allocation and 264 dimensions for flying watch thrusts). Therefore, the search space for AOWS has complicated high-dimensional geometry and the interior point method will only find a local optimal solution near the initial point. Also, in the human-experience-based design, only the 7th flying watch is attached with thrust generating direction in the horizontal plane and all other flying watches are attached with thrust generating directions in the vertical planes containing the corresponding attached links (when the arm is observed in Fig. 4 configuration). By contrast, in the AOWS based design, the thrust generating direction of the 4th flying watch is partially tilted (50.71$^\circ$) to the horizontal plane and all other flying watch have thrust generating directions close to vertical plane containing the corresponding attached links (when the arm is observed in Fig. 4 configuration). This means AOWS choose to use the 4th flying watch instead of the 7th one to counteract horizontal wind, which is better for two reasons. Firstly, since vertical external forces (arm gravity = 88.1 N and vertical end effector reaction force = 15 N) influence the arm much more significantly than the horizontal external force (wind load = 10 N), it is better to use distal flying watches with longer moment arms to counteract vertical external forces. Secondly, the partially tilted 4th flying watch can also counteract vertical external forces when there is no wind. By contrast, in human-experience-based design, since the thrust generating direction of the 7th flying watch is on the horizontal plane, when there is no horizontal wind, the 7th flying watch itself will be a load to the arm.

Also, we can observe from Table 1 and Figs. 5, 6, 7 and 8 that in the AOWS based allocation, the thrust generating directions of the two flying watches on the same link forms skew lines (lines that are not parallel and not intersecting). As a result, the generated thrusts along the skew lines are capable to form equivalent forces and torques (regarding the DH frame of the closest proximal joint) with diverse directions and magnitudes, which can counteract unexpected effect induced forces and torques and prevent them from further propagating to more proximal links. On the other hand, in the human-experience-based design, the thrust generating directions of flying watches on the same link (except the 8^th link) are parallel. For a given arm configuration, the resulting parallel thrusts can only form equivalent forces and torques (regarding the DH frame of the closest proximal joint) with fixed directions, which may fail to counteract unexpected effect induced forces and torques with unknown directions and magnitudes. The advantage of the AOWS design for counteracting unexpected effect can also be seen from TDSs in Figs. 5, 6, 7 and 8. The TDS of AOWS design is clearly closer to a unit sphere than that of human-experience-based design. We can learn from the AOWS design that the diversity of thrust generating directions of flying watches is very important for counteracting unexpected end effector reaction forces.

To quantitively evaluate the ability of the AOWS based allocation to counteract known effects (the arm gravity, the object gravity, and the known wind loads), we used the human-experience-based allocation as the benchmark case and compute the average reduction rate of all representative situations based on (5). We found the average reduction rate of the AOWS based allocation over the human-experience-based allocation is 37.43%. Therefore, the AOWS based allocations can counteract known effects significantly better than the human-experience-based allocation.

To quantitively evaluate the ability of the AOWS based allocation to counteract unknown end effector reaction forces, we calculated the Thrust Drivability (D), using (15), of AOWS based allocation and human-experience-based allocation in all directions (${{\varvec{\omega}}}_{all}=[0,\pi ,\mathrm{0,2}\pi ]$), in the left region (${{\varvec{\omega}}}_{left}=[\frac{\pi }{4},\frac{3\pi }{4},\frac{3\pi }{4},\frac{5\pi }{4}]$), in the right region (${{\varvec{\omega}}}_{right}=\left[\frac{\pi }{4},\frac{3\pi }{4},0,\frac{\pi }{4}\right]\cap [\frac{\pi }{4},\frac{3\pi }{4},\frac{7\pi }{4},2\pi ]$), in the front region (${{\varvec{\omega}}}_{front}=[\frac{\pi }{4},\frac{3\pi }{4},\frac{5\pi }{4},\frac{7\pi }{4}]$), in the back region (${{\varvec{\omega}}}_{back}=[\frac{\pi }{4},\frac{3\pi }{4},\frac{\pi }{4},\frac{3\pi }{4}]$), in the top region (${{\varvec{\omega}}}_{top}=[0,\frac{\pi }{4},\mathrm{0,2}\pi ]$), and in the bottom region (${{\varvec{\omega}}}_{bottom}=[\frac{3\pi }{4},\pi ,\mathrm{0,2}\pi ]$). The Thrust Drivability (D) results are shown in Table 2. From Table 2, we can see AOWS-based design performs better than human-experience-based design in counteracting unknown effect in different directions. The Thrust Drivability (D) of all directions of the AOWS based design (0.946) is 31.6% higher than that of the human-experience-based design (0.719). Such direct comparison between Thrust Drivability (D) of different allocations is meaningful because according to (10), both Thrust Drivability used the same benchmark, which is the arm in configuration ${{\varvec{q}}}_{m1}$ with no flying watch assistance. We can also understand the strong and weak regions of one allocation by comparing Thrust Drivability (D) of different regions of the allocation itself. We can observe from Table 2 that the strong regions of the both human-experience-based allocation and AOWS-based allocation are top and bottom regions. The human-experience-based allocation is relatively weak in left and right regions while the AOWS-based allocation is relatively weak in front and back regions. Such information is helpful for arm operation since the operator should pay attention to avoid obstacles or other disturbances that may result in end effector reaction forces in weak regions of the deployed flying watch attachment allocation.

Table 2 Thrust drivability of static object holding mission

We can further analyze why there are weak and strong regions. A major reason is moment arm disproportion of flying watch thrust. Let us first observe the human-experience-based allocation. From Table 2, we can see the weak regions of the human-experience-based attachment allocation are left and right regions and the strong regions are top and bottom regions. The Thrust Drivability difference between the strongest and weakest regions is 0.32, which is very large. Supposing there is an unexpected end effector force ${F}_{u1}^{weak}$ added along the weakest direction (roughly the direction along the arm as seen from Fig. 5c), the moment arms of ${F}_{u1}^{weak}$ regarding the 1st and the 7th joint are similar and the first and second largest respectively. Therefore, the largest joint load is on the first joint and the second largest joint load, which is very close to the largest joint load, is on the seventh joint. The only flying watch that can counteract the horizontal end effector reaction force in the human-experience-based allocation is the seventh flying watch. In contrast with the moment arm proportion of ${F}_{u1}^{weak}$, the moment arm of the 7th flying watch thrust regarding the 1st joint is significantly larger than that regarding the 7th joint. Such moment arm disproportion of the flying watch thrust means even if the flying watch successfully counteracted the load on the 1st joint, significant load will remain on the 7th joint, which is still likely to cause arm failure. On the other hand, let us now observe the AOWS-based allocation. Different regions of the AOWS-based attachment allocation have similar and very high Thrust Drivability (difference between the maximum and minimum is only 0.052). If we observe Fig. 7c, when the same unexpected end effector reaction force ${F}_{u1}^{weak}$ is exerted, we can see 6 out of 8 flying watch can counteract such force. Since joint loads due to the unexpected end effector reaction force is distributed on 6 watches, the problem of moment arm disproportion of flying watch thrust does not exist.

Finally, we simulate static holding missions under four more combinations of end effector positions (for orientation, the end effector frame is always aligned with the base frame) and known end effector reaction forces (more specifically object gravity). There are two purposes for doing that. Firstly, it helps verifying that AOWS can stably generate reasonable flying watch attachment allocations for static object holding missions under different conditions. Secondly, it helps understanding AOWS performance of object manipulation missions under different conditions as well. We will evaluate AOWS performance in an object manipulation mission in the next subsection and from Problem 3, we can see that AOWS essentially treats an object manipulation mission as a combination of multiple static object holding mission components. As a result, we can imagine the AOWS performance of an object manipulation mission depends on AOWS performance of the corresponding static object holding mission components. The reduction rate $(\zeta )$ and Thrust Drivability in all directions (${D}_{all}$) of all five conditions up to now for static object holding missions are summarized in Table 3. Please note that since we always use the same initial parameters for the inverse kinematics, the arm configurations only depend on the end effector positions.

Table 3 Aows performance under different conditions

From Table 3, we can firstly see that AOWS-based designs perform better than the human-experience-based design in counteracting both known reaction forces and unexpected end effector reaction forces under different conditions. Secondly, it is interesting to notice that for the same end effector position of [11,2,0.8], when the object gravity changes from 20 to 15 N, there is no significant change in reduction rate and when the object gravity changes the same amount from 15 to 10 N, the reduction rate drops significantly. We analyzed the joint loads under different object gravity and found the reason is that the human-experience-based design can counteract object gravity under 10 N very well and there is very small room left for AOWS to improve. However, when the object gravity is larger than 10 N, the joint loads of the human-experience-based allocation increase very rapidly and there is larger room for AOWS to improve. Therefore, when the object gravity is small enough, the AOWS-based design and the human-experience-based design will have similar performance for counteracting known reaction forces.

Object manipulation mission

In this mission, the arm will manipulate an object with 15 N gravity from position [7,2,0] to position [7,− 2,0] following an ellipse trajectory with parametric equation of $\left(x,y,z\right)=(7,2\mathrm{cos}\left(\pi \xi \right),\mathrm{sin}\left(\pi \xi \right))$, where $\xi \in [\mathrm{0,1}]$. The end effector frame will always coincide with the base frame. During the mission, a 10 N wind load with a direction of positive Y axis direction of the base frame (probability = 30%) and negative Y axis direction of the base frame (probability = 30%) will act on the end effector.

To use AOWS, we need to construct a WSM for the mission. We firstly do trajectory planning. Since there is no obstacle, trajectory planning can be done by generating a sequence of arm configurations and interpolating those configurations with certain time function. We evenly discretize $\xi$ with a step size of 0.1 to generate 11 trajectory sample points and then solve inverse kinematics for those points while forcing the configurations of adjacent points to be as close as possible to generate a sequence of arm configurations. Since a WSM does not concern when a representative situation happens, time interpolation of the sequence of arm configuration is not necessary. The set of arm configurations ${{\varvec{Q}}}_{m2}=\{{{\varvec{q}}}_{1},\dots ,{{\varvec{q}}}_{11}\}$ is shown in Fig. 9. The set of possible end effector reaction forces is ${{\varvec{F}}}_{m2}=\left\{{{\varvec{F}}}_{1}, {{\varvec{F}}}_{2}, {{\varvec{F}}}_{3}\right\}$, where ${{\varvec{F}}}_{1}=[\mathrm{0,0},-15]N$, ${{\varvec{F}}}_{2}=[\mathrm{0,10},-15]N$, ${{\varvec{F}}}_{3}=\left[0,-10,-15\right]N$. The set of representative situations is ${{\varvec{E}}}_{m2}={{\varvec{Q}}}_{m2}\times {{\varvec{F}}}_{m2}$, where “$\times$” is Cartesian product. We assume the configurations in ${{\varvec{Q}}}_{m2}$ are equally important and the importance of each possible end effector reaction forces depends on their probabilities. Therefore, for the 33 representative situations in ${{\varvec{E}}}_{m2}$, those situations include ${{\varvec{F}}}_{1}$ have importance factors of $\frac{1}{11}\times 0.4=\frac{2}{55}$ and those situations include ${{\varvec{F}}}_{2}$ or ${{\varvec{F}}}_{3}$ have importance factors of $\frac{1}{11}\times 0.3=\frac{3}{110}$. After defining the importance factor set ${{\varvec{P}}}_{m2}$, we can construct a WSM ${{\varvec{\Pi}}}_{m2}=({{\varvec{E}}}_{m2}, {{\varvec{P}}}_{m2})$. In addition to those known effects, such as the arm gravity, the object gravity, and the known wind load, we also hope that the flying watch allocation can counteract unexpected effects, such as unexpected wind loads and unexpected collisions. Therefore, we set the direction region of unexpected end effector reaction forces as ${{\varvec{\omega}}}_{m2}=[0,\pi ,0, 2\pi ]$ with drivability weight factor ${\alpha }_{m2}=1$. We set the unexpected effect weight factor ${\beta }_{m2}=0.5$.

Fig. 9

The sequence of arm configurations in the object manipulation mission

We input those design parameters to AOWS and the AOWS-based allocation and the human-experience-based allocation are shown in Table 4 for comparison. The flying watch positions and orientations in Table 4 still correspond to ${\mu }_{sd}$ and ${\mu }_{s\eta }$ respectively, which are Simplified DHAD introduced in subsection 3.3. We also draw human-experience-based allocation and AOWS-based allocation in the 17th representative situation (${{\varvec{e}}}_{17}=({{\varvec{q}}}_{6}, {{\varvec{F}}}_{2})$, where ${{\varvec{q}}}_{6}=$[-1.06,-0.41,-0.04,0.27,1.56,0.26,0.54,-0.28,-0.98] rad and ${{\varvec{F}}}_{2}=$[0,10,-15] N$)$in Figs. 10, 11, 12 and 13. The meaning of colors is the same as that of Figs. 5, 6, 7 and 8.

Table 4 Aows-based and human-experience-based allocations of manipulation mission

Fig. 10

The front, back, top, and bottom views of the human-experience-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the object manipulation mission. a Front view. b Back view. c Top view. d Bottom view

Fig. 11

a Overall view of the human-experience-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the object manipulation mission. b Front view of TDS. c Back view of TDS. d Top view. e Bottom view of TDS

Fig. 12

The front, back, top, and bottom views of the AOWS-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the object manipulation mission. a Front view. b Back view. c Top view. d Bottom view

Fig. 13

a Overall view of the AOWS-based flying watch allocation, end effector reaction force, and the corresponding average TDS in the object manipulation mission. b Front view of TDS. c Back view of TDS. d Top view of TDS. e Bottom view of TDS

From Table 4 and Figs. 10, 11, 12 and 13 and compared with the human-experience-based allocation, we can observe several patterns of AOWS similar to that of the static object holding mission. Firstly, AOWS moves the 1^st, 3^rd, and 7^th flying watches significantly closer ($\ge 4$ cm) to the end effector to obtain longer moment arms. Secondly, AOWS makes the thrust generating directions of all flying watches close to the vertical planes containing the corresponding attached links and tilts the thrust generating directions of the 2nd, 4th, and 7th flying watches to the horizontal planes ($\ge 9^\circ$) (when the arm is observed in Fig. 4 configuration). In such allocation, all flying watches focus more on counteracting the much larger vertical effect (vertical end effector reaction forces and arm gravity) while the tilted flying watches can also counteract horizontal wind force. Thirdly, the thrust generating directions of flying watches on the same link also form skew lines. As we have discussed in the static object holding mission, such diversified thrust generating directions of flying watches can help counteracting unknown end effector reaction forces. The advantage of diversifying flying watch orientation can also be seen from TDSs in Figs. 10, 11, 12 and 13. The TDS of the AOWS design is clearly closer to a unit sphere than that of the human-experience-based design.

To quantitatively evaluate the performance of AOWS based design to counteract known end effector reaction forces, we calculated the average reduction rates in the same way as the static object holding mission. The average reduction rate of the AOWS based design (using the human-experience-based design as benchmark) is 22.01%. Therefore, AOWS based design performs better than human-experience-based design in counteracting known end effector reaction forces.

To quantitatively evaluate the performance of AOWS based design to counteract unknown end effector reaction forces, we also calculated Thrust Drivability (D), using (15), in different direction regions as shown in Table 5. The definition of those direction regions is the same as that of the static object holding mission. We can see from Table 5 that the all-direction Thrust Drivability (D) of AOWS based design (0.922) is 24.6% higher than that of the human-experience-based design (0.740). We can also understand the strong and weak regions of each design by comparing Thrust Drivability (D) of the design itself. For the human-experience-based allocation, the weak regions are the left and right regions and the strong regions are the front and back regions. For the AOWS based allocation, the weak regions are the front and back regions and the top, bottom, left, and right regions have similar Thrust Drivability.

Table 5 Thrust drivability of object manipulation mission

A major reason that there are strong and weak regions is the same as that of the static object holding mission. Let us observe the human-experience-based allocation first. From Table 5, we can observe that the left and right regions are the weakest regions while the front and back regions are the strongest regions. The Thrust Drivability difference between the strongest and weakest regions is 0.27, which is very large. Supposing there is an unexpected end effector force ${F}_{u2}^{weak}$ added along the weakest direction (roughly along x direction of the base frame as seen from Fig. 10), the ratio of moment arms of ${F}_{u2}^{weak}$ regarding the 7th and the 5th joint is about 71%. And the joint loads on the 5^th and 7th joints are the first and second largest respectively. The major flying watch that can counteract the horizontal ${F}_{u2}^{weak}$ is the 7th flying watch. In contrast with the moment arm proportion of ${F}_{u2}^{weak}$, the ratio of moment arm of the 7th flying watch thrust regarding the 7th joint and the 5th joint is about 41%. Such moment arm disproportion of the flying watch thrust means even if the flying watch successfully counteracted the load on the 5th joint, significant load will remain on the 7th joint, which is still likely to cause arm failure. On the other hand, let us now observe the AOWS-based allocation. Different regions of the AOWS-based attachment allocation have similar and very high Thrust Drivability (difference between the maximum and minimum is only 0.063). If we observe Fig. 12c, when the same unexpected end effector reaction force ${F}_{u2}^{weak}$ is exerted, we can see all 8 flying watches can counteract such force. Since joint loads due to the unexpected end effector reaction force is distributed on 8 watches, the problem of moment arm disproportion of flying watch thrust does not exist either.

Conclusion

In this paper, to facilitate mission-dependent long-reach robotic arm enhancement, we propose an automated design method for designing attachment allocations of flying watches (thrust-generating attachable modules for arm enhancement), called Allocation Optimization based on Weighted Situations (AOWS). Given a mission description, AOWS can generate flying watch attachment allocations counteracting both known and unexpected end effector reaction forces as well as arm gravity. The ability of AOWS to consider unexpected end effector reaction forces attributes to a novel performance metric proposed in this paper, called Thrust Drivability, which measures the ability of a flying watch attachment allocation to counteract unknown end effector reaction forces. AOWS enables fast customization of flying watch attachment allocations for different missions. To the best of our knowledge, no such automated design method like AOWS and performance metric like Thrust Drivability have been explored in the literature. Based on simulations of two typical arm missions (static object holding mission and object manipulation mission), We show that AOWS based flying watch attachment allocations have much better performance than human-experience-based allocations.

In the future, we will verify the effectiveness of AOWS using a physical robotic arm and flying watches. In addition, we believe AOWS could also be used for designing future thruster-motor hybrid long arms (arms actuated by both thrusters and motors) that combine the strength of thrusters and the agility of motors. We will explore designing such arms based on AOWS in the future. Also, although we assumed static or quasistatic condition throughout this paper, flying watch attachment allocation that considers dynamic effects is also desirable. For example, a small vibration on a proximal link of a long arm can result in large vibrations on distal links. Flying watch attachment allocation that can suppress such vibration propagation would significantly improve arm stability. We will work on such flying watch attachment allocation design method considering dynamic effects. Finally, the AOWS problem (Problem 3) is solved using interior point method that only returns a local minimal. We will explore solving the global solutions of the AOWS problem.