Skip to main content
  • Research Article
  • Open access
  • Published:

Hardware-in-the-loop simulation of massive-payload manipulation on orbit

Abstract

This paper describes a hardware-in-the-loop simulation of massive-payload manipulation on orbit using a master–slave teleoperation system. The main problems in teleoperating a space robot arm from the earth to manipulate a massive payload are communication delay, unexpected excessive force generated between the slave arm and the payload, and geometric/dynamic modeling error. In order to overcome those problems, a teleoperation system using mixed force and motion commands is discussed. The teleoperation system is verified by performing hardware-in-the-loop simulations.

Introduction

Space robotic manipulators such as SRMS (Shuttle Remote Manipulation System) and SSRMS (Space Station Remote Manipulator System) have played a vital role in satellite orbit injection and construction of the ISS (International Space Station). These space robots have been operated by astronauts on-site.

However, if such manipulation can be performed by teleoperating the space robot from the earth, there is no need to send astronauts to space, which is attractive from both safety and financial consideration.

In general, satellites or modules of space structures have huge mass compared to the robot arm. Astronauts must be very careful when operating the robot arm not to generate excessive force between the end effector of the robot arm and the massive payload being manipulated, especially when starting and stopping the movement. The main problems in the manipulation of a massive payload by teleoperating the robot arm from the earth will be communication delay and the unexpected excessive force generated between the space robot and the payload.

It is well known that a communication delay between a master arm and a slave arm greatly reduces the performance of the teleoperation [1]. In general teleoperation (e.g. [2]) in which the master and the slave arms are directly coupled, an operator must take “move and wait” strategy, in which the operator must wait the delayed information of the slave-arm to confirm at every slight operation if the communication delay is large. Semi-autonomy is a solution for efficient teleoperation under communication delay [3]. Model-based teleoperation via a virtual environment has also been enthusiastically studied [4,5,6]. However, in model-based teleoperation via a virtual environment, an unexpected excessive force could be generated by a modeling error in the geometric or dynamic properties of the slave arm and the payload.

In order to overcome the problem of modeling error in the teleoperation, the use of mixed force and motion commands has been proposed [6, 7]. However, model-based teleoperation assumes that the master and the slave arm have similar sizes, and therefore this teleoperation system cannot be directly applied to massive-payload manipulation. In [8], a small size master-arm was used to operate the ETS-VII manipulator, which was an actual space robot. However, only the contact force was fed-back to the operator, and hence force scaling was not considered. The power scaling was considered in [2], however the method proposed in [2] has the problem of inefficiency in teleoperation if the communication delay is large, as discussed above. In this work, model-based teleoperation is modified considering the difference in size between the master and the slave arms for application to massive-payload manipulation.

Furthermore, the modified model-based teleoperation is verified using a hardware-in-the-loop simulation (HILS). Various HIL simulators for space robots were surveyed in [9]. HIL simulators for space robots have been developed mainly to simulate the capture of space targets such as satellites [10,11,12]. However most HIL simulators for space robots have used conventional serial robots to reproduce the movement of the space target. Serial robots do not generally move very quickly, and hence the response delay is relatively large. The large response delay of the motion table (the robot arm used to reproduce the movement of the space target) may cause instability in HILS [13]. Hence, this work uses a parallel-robot-based HIL simulator [14], applying delay time compensation [13] in order to validate the model-based teleoperation by simulating massive-payload manipulation.

The main contributions of this paper are the proposal for a modified version of model-based teleoperation using mixed force and motion commands and the parallel-robot-based HILS of the massive payload manipulation. The proposed method deals with force scaling. In force scaling, mitigation of excessive force at the remote site will be the key issue, because the operator’s commands are scaled-up at the remote site, and there is a serious communication delay. The mitigation of generated force at the remote site using the proposed method was verified by performing HIL simulations.

Hardware-in-the-loop (HIL) space robot simulator

Fig. 1
figure 1

Concept of a space robot teleoperation system

The concept of a space robot teleoperation system is illustrated in Fig. 1. In the space center on earth, an operator manipulates the master arms by looking at the virtual environment and predicting the movement of the space robot on orbit. The control inputs from the master arms and the position data of the virtual arm are transferred to the slave arm on orbit with a communication delay. The slave arms on orbit follow the motion commands generated by the master arms on earth. When the slave arms contact with the payload or the environment, the contact force is locally controlled using force sensor data. However, the slave arms’ position data arrive at the space center on earth with a communication delay. Hence, the positions of the slave arms at the moment of teleoperation are unknown on earth. In order to overcome the communication delay problem, simulated motions of the slave arms (virtual arms) are displayed for the human operator. The simulated motion is transferred to the slave arm controller as well as the motion command generated by the master arms (see Fig. 1). The motion of the virtual arms modifies the slave arm’s position in order to eliminate the error between the slave arm and virtual arm. The joint angle data of the slave arm are transferred to the virtual environment in order to display the movement of the slave arm, which is superimposed on the movement of the virtual arm. The system can be divided into the hybrid motion simulator, the slave system, and the master system.

HIL simulator

The HIL simulator [14] is composed of a 6 axis force sensor for acquiring the external force, an X-Y motion table for reproducing the translational large movement of the payload, and a HEXA-type [15] motion table for reproducing 6-DOF guide movement. The force sensor is mounted at the boundary of the hardware simulation and software simulation. In the HIL simulation, a force and torque acting on the payload are measured by the force/torque (F/T) sensor. The relative position and orientation with respect to the payload of interest are calculated by solving a dynamic equation with the measured force/torque data. Finally, the calculated relative position and orientation are realized on the hardware side by a servomechanism in real time (see Fig. 2). The response delay time compensation method [13] is applied to the hybrid simulator.

Fig. 2
figure 2

Concept of the hybrid simulation

Slave system

Fig. 3
figure 3

Overview of the slave system

The slave system is composed of two 7-DOF manipulators with a serial link mechanism and a camera head system for monitoring the working environment (see Fig.  3). The camera head can pan and tilt to see the workspace. A wrist camera, a 6-axis force sensor and a BarrettHand™ (Barrett Technology, LLC) for grasping the various payloads are mounted on each end effector. The 6-axis force sensor acquires the force generated at the wrist of the slave arm. The BarrettHand is able to open and close fingers to grip a payload. The end effector camera is used to obtain detailed information with a different field of view angle of the camera head.

Master system

The master system (see Fig. 4) is composed of two 6-DOF haptic interfaces with a parallel link mechanism [16]. The force sensor used in the 6-DOF haptic interface is NANO 5/4-A (Bl Autotec). The 6-DOF haptic interface is used in the joystick mode (the haptic device does not move and does not display forces in the joystick mode) in this work. The virtual environment is displayed on a monitor.

Fig. 4
figure 4

Overview of the master system

Development of virtual environment and control law

Operation scenario

The assumed task is to convey a massive payload by teleoperating a space robot. A grip mounted on the HEXA-type motion table (Fig. 3) is assumed to be a part of the massive payload. The force applied to the grip is measured by an F/T sensor mounted between the grip and the HEXA-type motion table. The measured force and torque are considered to be the external force applied to the virtual massive payload. The motion of the massive payload is calculated by solving a dynamic equation, and the motion is reproduced by the HEXA-type motion table mounted on the X-Y table.

In the operation scenario, it is assumed that a human operator on earth teleoperates a space robot arm (slave arm) to manipulate a massive payload.

Control law

In this work, the haptic device shown in Fig. 4 is used as a master arm in the joystick mode. The force applied by the human operator to the stylus of the master arm is measured by an F/T sensor and is transformed into a velocity command of the slave arm. A block diagram of the teleoperation system is illustrated in Fig. 5.

Fig. 5
figure 5

Block diagram of the system

Let \({\varvec{x}}_i = [ {\varvec{r}}_i^T \quad \varvec{\phi }_i^T]^T\) and \({\varvec{v}}_i = [ {\varvec{\dot{r}}}_i^T \quad {\varvec{\omega }}_i^T]^T\) (i = m, vir, s, or t) be the six dimensional position and velocity vectors, respectively. Subscripts mvirs, and t, respectively, represent the master arm, virtual arm, slave arm, and massive payload. \(\varvec{\phi }_i\) and \({\varvec{\omega }}_i\) denote a set of three variables for orientation (e.g., Euler angles) and the angular velocity vector, respectively. The derivative of \({\varvec{x}}_i\) is transformed into \({\varvec{v}}_i\) by the matrix \({\varvec{B}}(\varvec{\phi }_i)\) as \({\varvec{v}}_i = {\varvec{B}}(\varvec{\phi }_i){\varvec{\dot{x}}}_i\).

In the teleoperation with a communication delay, both geometric and dynamic modeling errors are serious problems. If there are very large discrepancies between the real environments at the remote site and their models, it may not be possible to catch/contact a payload with the slave arm via teleoperation, because the operator on earth would not be able to monitor the remote site in real time. Large discrepancies between the dynamic properties of the slave arm/payload and those of the models may cause an unexpected excessive force to be generated between the slave arm and the payload. In order to overcome those problems, model-based teleoperation using mixed force and motion commands has been proposed [6, 7].

This work uses model-based teleoperation [6, 7], but it is modified to apply to massive payload manipulation. The main modification pertains to the scaling between the master arm and the slave arm. The original model-based teleoperation using mixed force and motion commands [6, 7] assumed that the force and motion in both arms (master and slave arms) were of the same scale. However, in massive-payload manipulation, the force/motion command given by a human operator should be scaled up so that the large-sized slave arm can manipulate the massive payload. In the following sections, modified model-based teleoperation using mixed force and motion commands is presented.

Control law of slave arm

The velocity command to the slave arm, \({\varvec{v}}_{s,c}(t)\), is given as follows:

$$\begin{aligned} {\varvec{v}}_{s,r}(t)={\varvec{K}}_s{\varvec{G}}{\varvec{f}}_{m}(t), \end{aligned}$$
(1)
$$\begin{aligned} {\varvec{v}}_{s,c}(t) = &\,{\varvec{v}}_{s,r}(t-t_{delay}) - {\varvec{K}}_s{\varvec{f}}_{s}(t) + {\varvec{K}}_c{\varvec{B}}({\varvec{x}}_{vir}(t-t_{delay}) - {\varvec{x}}_{s}(t)) \nonumber \\&-\,{\varvec{D}}({\varvec{v}}_{s}(t-\Delta t) - {\varvec{v}}_{s}(t-2\Delta t)), \end{aligned}$$
(2)

where \({\varvec{f}}_m\) is the reference force commanded by the human operator using the master arm, \({\varvec{f}}_s\) is the actual force generated at the slave arm, \({\varvec{G}}\) is a diagonal scaling matrix between the force commanded by the master arm and the force the slave arm is expected to generate. In order for the slave arm to manipulate the massive payload, the force command \({\varvec{f}}_m\) given by the human operator must be scaled up for the slave arm. The matrix \({\varvec{G}}\) scales up the reference force. Substituting (1) into (2), the following equation is obtained:

$$\begin{aligned} {\varvec{v}}_{s,c}(t)=&\,{\varvec{K}}_s({\varvec{G}}{\varvec{f}}_m(t-t_{delay})-{\varvec{f}}_{s}(t)) +{\varvec{K}}_c{\varvec{B}}({\varvec{x}}_{vir}(t-t_{delay}) - {\varvec{x}}_{s}(t)) \nonumber \\ & - \, {\varvec{D}}({\varvec{v}}_{s}(t-\Delta t) - {\varvec{v}}_{s}(t-2\Delta t)), \end{aligned}$$
(3)

\({\varvec{v}}_{s,r}\) is the reference velocity given by (1). \({\varvec{x}}_{vir}\) is the position vector of the slave arm. They are transferred from the earth and arrive at the space robot \(t_{delay}\) later. Hence, \({\varvec{f}}_{m}\) and \({\varvec{x}}_{vir}\) used in (3) are \(t_{delay}\) ahead of the local (space) time. \({\varvec{x}}_{s}\) is the position vector of the slave arm, \({\varvec{K}}_c\) is a gain matrix for the restraint force, \({\varvec{f}}_{s}\) is a force vector measured by the F/T sensor of the slave arm, and \({\varvec{K}}_s\) is a diagonal force feedback gain matrix. \({\varvec{D}}\) is a diagonal damping gain matrix. \(\Delta t\) is the sampling period of the controller. The original model-based teleoperation using mixed force and motion commands [6, 7] was not always stable in HILS, and hence the damping term (the last term of the right hand side of 2) is added in this work.

Virtual arm

Model-based virtual robots between the real robot and the master arm have been widely used to overcome communication delay (e.g., [6]), or unexpected events (e.g., [17, 18]).

In this work, the virtual arm is used to compensate for the modeling error in dynamic properties as well as to overcome the communication delay. The difference in the positions between the virtual slave arm and the real slave arm is feedback to the real slave arm control, as shown in the second term of the right-hand side of (2). The term helps reduce the positioning error caused by the modeling error in the dynamic properties of the slave arm. The reference velocity of the slave arm is also given to the virtual arm as follows:

$$\begin{aligned} {\varvec{v}}_{vir,r}(t)={\varvec{v}}_{s,r}(t)={\varvec{K}}_s{\varvec{G}}{\varvec{f}}_m. \end{aligned}$$
(4)

Similar to the velocity command to the slave arm (2), the velocity command to the virtual arm is given as follows

$$\begin{aligned} {\varvec{v}}_{vir,c}(t) & ={\varvec{v}}_{vir,r}(t) - {\varvec{K}}_s{\varvec{f}}_{vir}(t) \\ & - {\varvec{D}}({\varvec{v}}_{vir}(t-\Delta t) - {\varvec{v}}_{vir}(t-2\Delta t)), \end{aligned}$$
(5)
$$\begin{aligned} {\varvec{x}}_{vir}(t)={\varvec{x}}_{vir}(t-\Delta t) + {\varvec{B}}^{-1}{\varvec{v}}_{vir}(t)\Delta t, \end{aligned}$$
(6)

where \({\varvec{v}}_{vir}\) is the velocity of the virtual arm. The matrix \({\varvec{B}}^{-1}\) transforms \({\varvec{v}}_{vir}\) into \({\varvec{\dot{x}}}_{vir}\). The virtual arm control law is almost the same as the control law of the slave arm to predict the movement of the slave arm and present it in the virtual environment. When the slave arm employs high-ratio reduction gears at the joints, \({\varvec{v}}_s\) can be approximated by \({\varvec{v}}_{s,c}\) (see Appendix A). Therefore, the velocity of the virtual arm, \({\varvec{v}}_{vir}\), is approximated by \({\varvec{v}}_{vir,c}\).

Fig. 6
figure 6

Concept of virtual force

In the virtual environment, the contact force between the massive payload and the end effector of the slave arm is simulated by a spring-damper system, as illustrated in Fig. 6. \({\varvec{f}}_{vir}\) is simulated as follows:

$$\begin{aligned} {\varvec{f}}_{vir}(t) & = - {\varvec{K}}_t{\varvec{B}}({\varvec{x}}_{vir}(t-\Delta t) - {\varvec{x}}_{t}(t-\Delta t)) \\ &- {\varvec{D}}_t({\varvec{v}}_{vir}(t-\Delta t) - {\varvec{v}}_{t}(t-\Delta t)), \end{aligned}$$
(7)

where \({\varvec{K}}_t\) is a spring constant matrix, \({\varvec{D}}_t\) is a viscosity coefficient matrix, \({\varvec{x}}_{t}\) is a position vector of the virtual massive payload, and \({\varvec{v}}_{t}\) is a velocity vector of the payload. The acceleration of the massive payload is expressed by

$$\begin{aligned} {\varvec{\dot{v}}}_{t}(t) = {\varvec{M}}^{-1}{\varvec{f}}_{vir}(t), \quad {\varvec{M}} = \left[ \begin{array}{cc} m\varvec{E} & {}\mathbf{0}\\ \mathbf{0}&{}{\varvec{I}} \end{array} \right] \end{aligned}$$
(8)

where m is the mass, \(\varvec{E\in }\mathfrak {R}^{3\times 3}\) is an identity matrix, and \({\varvec{I}}\in \mathfrak {R}^{3\times 3}\) is the inertia tensor of the massive payload. The massive payload will be a satellite or a module of a space structure, and hence m and \({\varvec{I}}\) are known (they can be obtained from the design parameter). However, the inertia matrix \({\varvec{M}}\) may include a modeling error. The position of the payload is estimated by (810), hence the modeling error in \({\varvec{M}}\) in (8) causes an estimation error. The position error caused by the modeling error is compensated for by the coupling term in the slave arm controller (the third term of the right hand side of 2).

The velocity and position of the massive payload are calculated by

$$\begin{aligned} {\varvec{v}}_{t}(t) = {\varvec{v}}_{t}(t-\Delta t) + {\varvec{\dot{v}}}_{t}(t)\Delta t,\end{aligned}$$
(9)
$$\begin{aligned} {\varvec{x}}_{t}(t) = {\varvec{x}}_{t}(t-\Delta t) + {\varvec{B}}^{-1}{\varvec{v}}_{t}(t)\Delta t. \end{aligned}$$
(10)

Virtual environment

As discussed in Section "Virtual arm", the velocity of the virtual arm, \({\varvec{v}}_{vir}\), was approximated by \({\varvec{v}}_{vir,c}\) assuming high reduction gear ratios at the joints. Only the dynamics of the massive payload was considered in the virtual environment (710). The dynamics of the massive payload, (710), is used in the computation of the force feedback term of the velocity command (5). The calculation time of the virtual environment is 1 ms/loop, which is small enough compared to the video frame rate (33 ms). The virtual arm and the massive payload from the four different viewpoints are displayed on a screen (see Fig. 7). Zooming the camera is also possible on the computer graphics. The motions of the virtual arms are displayed with a solid model. The motions of the slave arms constructed from the joint data transferred from space are superimposed with a wire frame model on the motion of the virtual arms. This makes it easier to check whether the slave arm follows the virtual arm after a time delay.

Fig. 7
figure 7

Overview of virtual environment

Verification experiments

Validity of HIL simulation

HIL simulations were performed to confirm the validity of the system. In the HIL simulation, it is assumed that the slave arm grips a massive payload of 3000 kg and pulls it along the x-axis at a speed of 0.01 m/s for 20 s. The mass and inertia of the virtual payload assumed in the experiment are presented in Table 1. The gains used in the experiment are shown in Table 2.

Table 1 Mass and inertia parameters of the payload
Table 2 Gain parameters of the experiment

In the HIL simulation, the step input of the reference velocity of \({\varvec{v}}_{s,r}\) = 0.01 m/s was given instead of human teleoperation. The velocity command \({\varvec{v}}_{s,c}\) was calculated by (2) with \({\varvec{v}}_{s,r}\) = 0.01 m/s. \({\varvec{v}}_{s,r}\), \({\varvec{v}}_{s,c}\), and \({\varvec{v}}_{s}\) during the HIL simulation are plotted in Fig. 8. As it satisfies this condition, the end effector velocity \({\varvec{v}}_{s}\) is reduced by force feedback and asymptotically converges to \({\varvec{v}}_{s,r}\) (0.01 m/s).

Fig. 8
figure 8

Comparison between \({\varvec{v}}_{s,c}\) and \({\varvec{v}}_{s,r}\) (x-axis)

Figure 9 shows the force \({\varvec{f}}_{s}\) measured by the force sensor at the wrist of the slave arm. In order to accelerate the massive payload of 3000 kg to a velocity of 0.01 m/s, a momentum of 30 Ns is necessary. As shown in Fig. 8, a step reference velocity \({\varvec{v}}_{s,r}\) = 0.01 m/s was given at t = 176.5 s and \({\varvec{v}}_{s}\) almost converged to \({\varvec{v}}_{s,r}\) at t = 186 s. \({\varvec{f}}_{s}\) plotted in Fig. 9 was integrated from 176.5 to 186 s, and the integral value was 28.43 Ns. It is assumed that the velocity of the massive payload is almost the same as the velocity of the slave arm. Therefore the movement of the massive payload in the HIL simulation almost agrees with theory. The error in the momentum required to accelerate the massive payload from 0 to 0.01 m/s was about 5%. The force feedback term of the control law of the slave arm adequately mitigated the applied force and the velocity of the slave arm asymptotically converged to the given reference velocity.

Fig. 9
figure 9

Force of slave arm (x-axis)

Comparison of the results with and without force feedback

Another HIL simulation was performed to verify the effectiveness of the force feedback in (2) in reducing the resultant force during the massive-payload manipulation mission. When a virtual payload of 3000 kg is manipulated in a HIL simulation without force feedback, a huge force will be generated between the payload and the end effector of the slave arm, and hence there is a risk that the slave arm used in this research will be broken. Therefore, in this HIL simulation, a massive payload of 500 kg is used instead of 3000 kg.

The massive payload of 500 kg was grasped by the slave arm and pulled automatically for 1 min at a speed of 0.01 m/s along the x-axis with and without force feedback. Figure 10 shows the force generated between the slave arm and the payload in an automatic manipulation experiment for the payload of 500 kg without force feedback. In the experiment without force feedback, a maximum force of 23 N was generated when the payload was grasped, the arm started to pull the payload, and then the arm stopped pulling, as shown in Fig. 10.

Fig. 10
figure 10

Force and position of slave arm without force feedback

Fig. 11
figure 11

Force and position of slave arm with force feedback

Figure 11 shows the force generated between the slave arm and the payload in an automatic manipulation experiment of 500 kg with force feedback. In the experiment with force feedback, a maximum force of 18 N was generated at the time when the payload was grasped, the arm started to pull the payload, and then the arm stopped pulling, as shown in Fig. 11. During the manipulation, the generated force was kept small. The mean square error of \({\varvec{f}}_{s}\) is calculated as follows to compare the results of the experiments with and without force feedback:

$$\begin{aligned} MSE({\varvec{f}}_{s}) = \frac{1}{N}\sum _{t=0}^{\infty } \Vert {\varvec{f}}_{s}\Vert ^2, \end{aligned}$$
(11)

where N is the number of number of data items. \(MSE({\varvec{f}}_{s})\) was 5.41 N\(^2\) and 15.35 N\(^2\) when the force feedback was applied and was not applied, respectively.

Thus, it can be concluded that the force feedback in the slave arm controller helps reduce the force generated at the slave arm when the arm conveys a massive payload.

HIL simulation of massive-payload manipulation

Scenario of experiments

Two types of HIL simulations (HILS 1 and HILS 2) were performed.

First, a virtual massive payload of 4500 kg was grasped by the slave arm and was transferred along the x-axis at a constant speed of 0.01 m/s for 20 s (HILS 1). Next, a virtual massive payload of 4500 kg was grasped by the slave arm and was transferred along the x-axis by human teleoperation with a communication delay of 5 s (HILS 2) (see Additional file 1). In the teleoperation experiment, the input values for the y and z axes components are assumed to be zero.

The effectiveness of the model-based teleoperation using the mixed force and motion commands for the problem of communication delay was verified in [6] by performing a teleoperation experiment between German Aerospace Center (DLR, Germany) and Tohoku University (Japan). In this work, the effectiveness of the modified version of the model-based teleoperation for massive-payload manipulation is verified.

Although the same HILSs were carried out with virtual payloads of 3000 kg and of 6000 kg, only the result with the virtual payload of 4500 kg is presented in this paper. The mass and inertia used in the HILS are shown in Table 3. The gain parameters of the experiment were the same as the parameters used in the experiments presented in “Verification experiments” section.

Table 3 Mass and inertia parameters of the payload (4500 kg)

Comparison between the end effector force of the slave arm and the end effector force of the virtual arm (HILS 1)

In order to evaluate the accuracy of the simple virtual force model illustrated in Fig. 6, the end effector force of the slave arm generated along the x-axis is compared with that of the virtual arm as shown in Fig. 12.

Fig. 12
figure 12

Force of slave arm and virtual arm (x-axis) (in HILS 1)

Although the oscillation of the force was not reproduced by the simple spring-damper model, the model reproduced the mean value of the force with sufficient accuracy. It can be concluded that the dynamics of the massive payload of the slave side was modeled well by the simple spring-damper virtual force model.

Comparison between the end effector position of the slave arm and the end effector position of the virtual arm (HILS 1)

The positions of the end effectors of the slave and virtual arm along the x-axis are plotted in Fig. 13. Because the positions of both arms along the y and z axes were almost constant and identical, only the positions along the x-axis are presented.

Fig. 13
figure 13

Position of slave arm and virtual arm (x-axis) (in HILS 1)

Because the position of the virtual arm was sent to the slave arm, and it was used in the slave arm control (the third term of the right hand side of 2), the motion of the virtual arm almost corresponds to the motion of the slave arm (note that the time delay was zero in HILS 1).

Comparison between the end effector force of the slave arm and the end effector force of the virtual arm (HILS 2)

Figure 14 shows the end effector force along the x, y, and z axes of the slave arm and the end effector position along the x-axis of the slave arm, respectively, when a virtual massive payload of 4500 kg was transferred by human teleoperation with a time delay (HILS 2). Figure 15 shows the reference velocity of the slave arm given by human teleoperation (see 4).

Fig. 14
figure 14

Force of slave arm and position of slave arm (x-axis) (in HILS 2)

Fig. 15
figure 15

Reference velocity of slave arm (x-axis) (in HILS 2)

In HILS 2, the maximum end effector force was approximately 28 N. Compared to the result in HILS 1, the maximum end effector force generated in HILS 2 was larger, because the end effector reference velocity \({\varvec{v}}_{s,r}\) generated by human teleoperation tends to be larger than the constant reference velocity \({\varvec{v}}_{s,r}\) = 0.01 m/s given in HILS 1 (see Fig. 15).

Figure 16 shows the forces generated at the slave and virtual arms. Note that the time axis on earth (for the virtual arm) is shifted 5 s later for the time axis on orbit (for the slave arm) in Figs. 16 and  17 in order to compare the results of the slave and virtual arms.

As shown in Fig. 16, the force generated at the virtual arm simulates the force of the slave arm well, although a small oscillation of the force was not reproduced.

Fig. 16
figure 16

Force of slave arm and virtual arm (x-axis) (in HILS 2)

Fig. 17
figure 17

Position of slave arm and virtual arm (x-axis) (in HILS 2)

Comparison between the end effector position of the slave arm and the end effector position of the virtual arm (HILS 2)

Figure 17 shows the positions of the end effectors of the slave and virtual arms. The maximum error between the positions of the arms was 0.002 m, which was slightly larger than the errors in HILS 1.

Thus, it is proved that the motion of the virtual arm predicts the motion of the slave arm well, and hence the proposed method is effective for the massive-payload manipulation task by teleoperation with time delay.

Conclusion

In this paper, a teleoperation system for a massive payload transportation task was proposed. The proposed control strategy was verified by performing hardware-in-the-loop simulations (HILSs).

It is confirmed that the control law to reduce the force generated between the slave arm and the massive payload works well. Furthermore, a virtual arm control law to reproduce the dynamics of massive payloads in a virtual environment is proposed. Several HILSs were carried out to verify the proposed system, and the differences between the results with and without the force feedback term were compared. It was verified that the massive-payload manipulation task can be performed while reducing the force generated between the end effector and the massive payload. HILSs of massive payload transport were also performed under a time delay environment, and it was verified that the movements and forces on the slave side were well reproduced in the virtual environment. The modified version of the model-based teleoperation will be applied to more complicated tasks in future work.

References

  1. Cl J, Craig RC, David LA (2001) Time delay and communication bandwidth limitation on telerobotic control. Int Syst Smart Manuf Int Soc Optics Photon 4195:405–419

    Google Scholar 

  2. Secchi C, Stramigioli S, Fantuzzi C (2008) Variable delay in scaled port-Hamiltonian telemanipulation. Mechatronics 18(7):357–363

    Article  MATH  Google Scholar 

  3. Sayers CP, Richard PP, Louis LW, Dana RY (1998) Teleprogramming for subsea teleoperation using acoustic communication. IEEE Jo Ocean Eng 23(1):60–71

    Article  Google Scholar 

  4. Niemeyer G, Slotine JJE (2004) Telemanipulation with time delays. Int J Robot Res 23(9):873–890

    Article  Google Scholar 

  5. Ryder CW, Sean MS, Elliot WH, David LC, Hao J, Mark RC, Allison MO (2014) Time-delayed teleoperation for interaction with moving objects in space. In: 2014 IEEE international conference on robotics and automation (ICRA)

  6. Tsumaki Y, Goshozono T, Abe K, Uchiyama M, Koeppe R, Hirzinger G (2000) Verification of an advanced space teleoperation system using internet. In: Proceedings of the 2000 IEEE/RSJ international conference on intelligent robots and systems (IROS 2000), vol. 2. pp 1167–1172

  7. Tsumaki Y (1998) A model-based space teleoperation system using mixed force and motion commands. Trans JSME 64(626):3901–3906

    Article  Google Scholar 

  8. Yoon WK, Goshozono T, Kawabe H, Kinami M, Tsumaki Y, Uchiyama M, Oda M, Doi T (2004) Model-based space robot teleoperation of ETS-VII manipulator. IEEE Trans Robot Autom 20(3):602–612

    Article  Google Scholar 

  9. Yoshida K (1996) Experimental platforms for research and development of space robots. J Robot Soc Japan 14–1:18–21

    Article  Google Scholar 

  10. Wenfu X, Bin L, Yangsheng X, Cheng L, Wenyi Q (2007) A ground experiment system of free-floating robot for capturing space target. J Intell Robot Syst 48(2):187–208

    Article  Google Scholar 

  11. Rouleau G, Rekleitis I, L’Archeveque R, Martin E, Parsa K, Dupuis E (2006) Autonomous capture of a tumbling satellite. In: Proceedings 2006 IEEE international conference on robotics and automation (ICRA 2006)

  12. Boge T, Wimmer T, Ma O, Zebenay M (2010) Epos-a robotics-based hardware-in-the-loop simulator for simulating satellite rvd operations. In: 10th International symposium on artificial intelligence robotics and automation in space

  13. Osaki K, Konno A, Uchiyama M (2010) Delay time compensation for a hybrid simulator. Adv Robot 24(8–9):1081–1098

    Article  Google Scholar 

  14. Takahashi R, Ise H, Konno A, Uchiyama M, Sato D (2008) Hybrid simulation of a dual-arm space robot colliding with a floating object. In: 2008 IEEE international conference on robotics and automation (ICRA). pp 1201–1206

  15. Uchiyama M, Iimura Ki, Pierrot F, Unno K, Toyama O (1992) Design and control of a very fast 6-dof parallel robot. In: Robotics mechatronics and manufacturing systems. pp 473–478

  16. Tsumaki Y, Naruse H, Nenchev DN, Uchiyama M (1998) Design of a compact 6-dof haptic interface. In: Proceedings of the 1998 IEEE international conference on robotics and automation, vol. 3, pp 2580–2585

  17. Kikuuwe R, Yasukouchi S, Fujimoto H, Yamamoto M (2010) Proxy-based sliding mode control: a safer extension of pid position control. IEEE Trans Robot 26(4):670–683

    Article  Google Scholar 

  18. Kikuuwe R, Yamamoto T, Fujimoto H (2008) A guideline for low-force robotic guidance for enhancing human performance of positioning and trajectory tracking: it should be stiff and appropriately slow. IEEE Trans Syst Man Cybern 38(4):945–957

    Article  Google Scholar 

Download references

Authors’ contributions

AK led and directed the project. AK showed the need for dealing with massive payloads in teleoperation. JS proposed the control law to reduce the force generated between the end effector and the massive payload using the force sensor value of the slave arm. JS also proposed the control law of the virtual arm to reproduce the dynamics of massive payloads in the virtual environment. AK participated in the discussion on the optimization of the control law. All authors read and approved the final manuscript.

Acknowledgements

The authors acknowledge Tohoku University for the experimental equipment and we are grateful Dr. Sugai for the robot programming support.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Funding

The authors received no specific funding for this research.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jaesung Yang.

Additional file

Additional file 1.

This is a movie file of the massive-payload manipulation simulation experiment.

Appendix A: Dynamics of the virtual arm

Appendix A: Dynamics of the virtual arm

If high-ratio reduction gears are employed at the joint, the joint angles and joint torques between before and after the gear reduction are related by

$$\begin{aligned} {\varvec{G}}_{r}{\varvec{q}}={\varvec{q}}_a, \end{aligned}$$
(12)
$$\begin{aligned} {\varvec{\tau }}={\varvec{G}}_{r}^{T}{\varvec{\tau }}_{a} \; (\because \Delta {\varvec{q}}^{T}{\varvec{\tau }}=\Delta {\varvec{q}}_{a}^{T}{\varvec{\tau }}_{a}=\Delta {\varvec{q}}^{T}{\varvec{G}}_{r}^{T}{\varvec{\tau }}_{a}), \end{aligned}$$
(13)

where \({\varvec{G}}_{r}\) is a diagonal reduction matrix, the diagonal elements of which correspond to the reduction ratio; \({\varvec{q}}\) is the joint angle vector of an arm; \({\varvec{q}}_a\) is the angle vector of the actuators; \({\varvec{\tau }}\) is the joint torque vector; and \({\varvec{\tau }}_a\) is the torque vector of the actuators.

The dynamics of the arm is expressed by

$$\begin{aligned} {\varvec{\tau }}={\varvec{M}}{\varvec{\ddot{q}}} + {\varvec{h}} + {\varvec{J}}^{T}{\varvec{F}}, \end{aligned}$$
(14)

where \({\varvec{M}}\) is the inertia matrix of the arm, \({\varvec{h}}\) represents the Coriolis force and centrifugal force and gravitational force, \({\varvec{J}}\) is the Jacobian matrix, and \({\varvec{F}}\) is the external force. Substituting (12) and (13) into (14), the following equation is obtained:

$$\begin{aligned} {\varvec{\tau }}_{a}={\varvec{G}}_{r}^{-T}{\varvec{M}}{\varvec{G}}_{r}^{-1}{\varvec{\ddot{q}}}_{a} + {\varvec{G}}_{r}^{-T}({\varvec{h}} + {\varvec{J}}^{T}{\varvec{F}}). \end{aligned}$$
(15)

Assuming that the actuator is the velocity command type, the actuator torque is calculated as

$$\begin{aligned} {\varvec{\tau }}_{a}={\varvec{K}}_{va}({\varvec{\dot{q}}}_{ca}-{\varvec{\dot{q}}}_{a})={\varvec{K}}_{va}{\varvec{G}}_{r}({\varvec{\dot{q}}}_{c}-{\varvec{\dot{q}}}) (\because {\varvec{\dot{q}}}_{ca}-{\varvec{\dot{q}}}_{a}={\varvec{G}}_{r}({\varvec{\dot{q}}}_{c}-{\varvec{\dot{q}}})), \end{aligned}$$
(16)

where \({\varvec{\dot{q}}}_{ca}\) is the joint angle velocity command. \({\varvec{K}}_{va}\) corresponds to the gain of the hardware velocity servo.

Substituting (16) into (15), the following equation is obtained:

$$\begin{aligned} {\varvec{\dot{q}}}_{c}-{\varvec{\dot{q}}} = {\varvec{G}}_{r}^{-1}{\varvec{K}}_{va}^{-1}{\varvec{G}}_{r}^{-T}{\varvec{M}}{\varvec{G}}_{r}^{-1}{\varvec{\ddot{q}}}_{a} +{\varvec{G}}_{r}^{-1}{\varvec{K}}_{va}^{-1}{\varvec{G}}_{r}^{-T}({\varvec{h}} + {\varvec{J}}^{T}{\varvec{F}}). \end{aligned}$$
(17)

By multiplying both sides of (17) by \({\varvec{J}}\), the following equation is obtained:

$$\begin{aligned} & {\varvec{J}}({\varvec{\dot{q}}}_{c}-{\varvec{\dot{q}}}) \\& ={\varvec{J}}{\varvec{G}}_{r}^{-1}{\varvec{K}}_{va}^{-1}{\varvec{G}}_{r}^{-T}{\varvec{M}}{\varvec{G}}_{r}^{-1}{\varvec{\ddot{q}}}_{a} +{\varvec{J}}{\varvec{G}}_{r}^{-1}{\varvec{K}}_{va}^{-1}{\varvec{G}}_{r}^{-T}({\varvec{h}} + {\varvec{J}}^{T}{\varvec{F}}). \end{aligned}$$
(18)

Therefore, by using \({\varvec{v}}_{c}={\varvec{J}}{\varvec{\dot{q}}}_{c}\) and \({\varvec{v}}={\varvec{J}}{\varvec{\dot{q}}}\), (18) is rewritten as

$$\begin{aligned} {\varvec{v}}_{c}-{\varvec{v}}= {\varvec{J}}{\varvec{G}}_{r}^{-1}{\varvec{K}}_{va}^{-1}{\varvec{G}}_{r}^{-T}{\varvec{M}}{\varvec{G}}_{r}^{-1}{\varvec{\ddot{q}}}_{a} + {\varvec{J}}{\varvec{G}}_{r}^{-1}{\varvec{K}}_{va}^{-1}{\varvec{G}}_{r}^{-T}({\varvec{h}} + {\varvec{J}}^{T}{\varvec{F}}). \end{aligned}$$
(19)

If the reduction ratio is large enough, the two terms on the right hand side of (19) can be ignored, because the first term is proportional to \({\varvec{G}}^{-3}\) and the second term is proportional to \({\varvec{G}}^{-2}\). In this case, \({\varvec{v}}\) can be approximated by \({\varvec{v}}_{c}\). In massive-payload manipulation simulation, \({\varvec{F}}\) in the second term may become very large and hence it may not be possible to ignore the second term. However, if \({\varvec{F}}\) is adequately controlled by force feedback (e.g. \({\varvec{K}}_{s}{\varvec{F}}_{s}\) in 2), the second term of the right hand side of (19) will be sufficiently small and the assumption \({\varvec{v}}\approx {\varvec{v}}_{c}\) will hold.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, J., Konno, A., Abiko, S. et al. Hardware-in-the-loop simulation of massive-payload manipulation on orbit. Robomech J 5, 19 (2018). https://doi.org/10.1186/s40648-018-0116-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40648-018-0116-8

Keywords