B-PaDY: robot co-worker in a bumper assembly line

Abstract

In an automobile assembly line, many processes require control and human intervention. A human’s dexterity and ability to react to unpredictable changes in production volume and product specifications are necessary for these processes. Thus, it remains difficult to rely on robots or other automated systems. Conversely, some tasks, such as moving to a tool rack and delivering parts, do not require human skill. In the bumper assembly process of automobile assembly, a worker must move to pick up the bumper twice in one procedure. This task may strain the worker and increase work time for each bumper. To address this situation, we developed a new co-worker robot, named B-PaDY, to work in cooperation with human workers on the automobile bumper assembly line. This co-worker robot is not directly involved in the assembly processes, but performs tasks to support human workers. The robot delivers the bumper to the worker on the production line at the right time. We propose hardware concepts and designs to ensure worker’s safety and allow for handling of all bumpers types. This robot uses a suspended rail system and handles the bumper, attaching to its reverse side. Additionally, we propose a method to determine the proper delivery timing based on a logistic regression model and confirm its effectiveness by performing experiments. We statistically model the worker’s state upon completion of the assembly task and stochastically determine the proper timing. By recognizing the worker’s behavior and supporting the worker with appropriate timing, our system effectively supplies the bumper to the worker and improves work efficiency.

Background

Industrial robots have been used extensively in manufacturing industries to enhance productivity and ensure the quality of products. While industrial robots have been extended to various fields such as electronics, food manufacture, and automobile assembly, many processes are still performed manually by humans. The final assembly line in automobile production is a process that still faces challenges in applying existing industrial robots. These processes may require human dexterity and adaptability because they may change in an unpredictable manner according to the amount of production and the specification of each product. Although autonomous dual-arm robots, including YuMi (ABB) [1] and Baxter (rethink robotics) [2], have been developed to realize flexible automation processes, it is difficult to completely replace humans with robots in these processes.

To address this situation, we propose the use of a co-worker robot called PaDY (in-time Parts/tools Delivery to You robot [3]). The co-worker robot is not directly involved in the assembly processes but rather performs nonessential tasks to support human workers. This robot supports the human worker in attaching parts to an automobile in the assembly line by delivering the required parts and tools to the worker. Because this reduces the time spent on walking to the workbench and selecting parts and tools, this system improves the work efficiency and reduces unnecessary labor. Hoffman et al. [4, 5] also proposed a cooperative robot system that selects the next task by itself based on the validity and risk of each action. Hayakawa et al. [6] proposed an assembly system whereby the robot works with the human by determining the work state based on a self-organizing map. Tan et al. [7] proposed a collaborative working system in a cell production system.

Although PaDY targeted processes can deal with small parts, there are many processes that require handling of large parts and tools. In these processes, the worker must handle heavy parts and move through a large workspace. Thus, there is room for substantial improvement in the workload. Peshkin et al. proposed the worker assistance system “Cobots”, whose mechanism conforms with the nonholonomic constraint. It assists with the detaching task in the door assembly process [8]. Yamada et al. [9] proposed a power-assisted precise positioning support system for the window mounting process. Both of these studies proposed efficient systems for reducing the physical burden on human workers.

Fig. 1

Worker trajectory in bumper assembly process. a The current procedure: the worker must move to the AGV two times. b The improved procedure with the B-PaDY system: the worker does not need to move to the AGV to pick up the bumper by hand

In this study, we apply the concept of PaDY to the bumper assembly process-one of the automobile assembly processes dealing with large-sized parts and tools. In the current state of this process, the worker picks up the bumper delivered by an automatic guided vehicle (AGV) and attaches the bumper to the vehicle body manually. Although the bumpers are automatically supplied from outside of the factory using the AGV, human still install the bumpers in the assembly process. Figure 1 shows the worker’s movement path while working in the assembly process. In the current practice of the bumper assembly, the worker must move to the AGV twice to pick up the bumper on the AGV in one procedure. As a result, work time is increased and the worker must focus on carefully picking up the bumper to avoid damaging it. Therefore, a system that picks up the bumper and delivers it to the worker automatically would be highly beneficial. The installation of an automatic bumper delivery system using B-PaDY (Bumper PaDY) may be expected to decrease the physical burden on the worker and improve work efficiency.

To decrease the work time, it is necessary to reduce the worker’s wait-time as much as possible. A robot that is always present beside the worker can disturb not only the worker who is performing regular assembly tasks, but also humans who need to pass through this workspace. Here, we examine a system that decides the proper delivery timing based on the worker’s current state. By delivering the bumper with proper timing, the robot avoids disturbing assembly tasks while minimizing work time. In our previous study, we proposed a method that predicts the worker’s arrival time at the next position based on a Markov model of the worker’s trajectory [10]. However, this considers only the worker’s position information and is difficult to apply to a process with limited motion, such as bumper assembly. Tamura et al. [11] proposed a method to communicate the worker’s intention to the robot by hand signals in a cell production system. Although this is an effective way to reduce wasted time, it results in an increased workload.

In this study, we discuss a method to determine the proper delivery timing without explicit signs from the worker. Instead, we consider the worker’s current state besides the position information. This decision system is expected to improve work efficiency by stably supplying the bumper without adding to the worker’s task. We use a logistic regression model to decide the proper delivery timing. We model the worker’s state at the time when the bumper supply is needed such that our system stochastically determines whether the current time is appropriate for bumper delivery.

The remainder of this paper is organized as follows. In “Concept and hardware design” section, we describe the proposed system concept and the hardware design for collaborating with human workers. In “Determining proper delivery timing” section, we present the details of the system components. In “Experiment” section, we describe the experiments conducted to evaluate the validity of the system. Finally, “Conclusion” section concludes this paper.

Concept and hardware design

In this study, we apply the concept of PaDY-whose validity in processes dealing with small-sized parts has already been confirmed-to the bumper assembly process. By having the bumper delivered by the robot instead of the worker, our proposed system eliminates the task of moving to the AGV and reduces the worker’s physical burden.

Fig. 2

Installation image of B-PaDY. a The installation position of B-PaDY in the bumper assembly process. b The bumper assembly process from the top view. The arrows and numbers indicate the sequence of work performed by B-PaDY

Figure 2 shows how B-PaDY is installed in the actual bumper assembly process. B-PaDY hangs on the rail at the upper part of the production line and provides coverage for a substantial work space in the bumper assembly process. The worker is on the production line at all working times and B-PaDY supports the worker flexibly depending on the work progress. The numbers in the right subfigure of Fig. 2 describe the sequence of the bumper assembly process after the B-PaDY system is installed. First, B-PaDY picks the bumper up from the AGV and waits at the edge of the work space while synchronizing the flow of the production line to avoid disturbing the workers. Then, observing the worker’s state using its external sensor and checking the work progress, B-PaDY delivers the bumper to the worker on the production line. Proper delivery timing is decided based on the statistical model. After bumper delivery, B-PaDY moves to the AGV and picks up the next bumper.

Hardware configuration

Fig. 3

Hardware design of B-PaDY. The robot has six degrees of freedom and consists of three parts (rail, arm, and gripper)

A prototype of B-PaDY is shown in Fig. 3. This robot has six degrees of freedom and consists of three major parts: the rails (including the running and transverse rails), the arm (including components for rotation and vertical movement), and the gripper. In the following sections, we summarize each component and its functions and focus on the details of the arm and gripper.

Rail design

The rail, includes running and transverse rails that allow movement within a plane. Because the worker and the vehicle body move long distances in the production line, B-PaDY must operate over a wide workspace. To address this situation, we adopt a suspended system that moves along rails. Although a robot moving on the floor can have a greater working range, it can be affected by a rough floor surfaces or obstacles, and has difficultly in managing stairs in the work space. The bumper assembly process has a wide workspace and its corresponding floor space synchronizes with the production line. Considering these features, we selected a suspended system.

The rail consists of the two rails (running and transverse) and a robot base. In the actual assembly process, the length of rail is approximately 4 m in the running direction and approximately 1 m in the transverse direction. Movement along the running rail is parallel to the production line direction. It serves to deliver the bumper and synchronize with the movement of the production line. Movement along the transverse rail runs perpendicular to the production line. This facilitates motion to the AGV and approach to the worker on the production line. We use a rack and pinion mechanism for driving on the rails. The robot base moves along each rail by meshing the rack with the driven part, which connects to a servo motor, a reducer, and a pinion.

Arm design

Fig. 4

Hardware design of arm part. a The multi-step up–down mechanism. b The contracted state of the arm part. c The state when grasping a bumper

The arm can rotate and move a gripper, attached to the robot base, along the vertical axis. The rotation of the gripper occurs around the vertical axis. We adopt a multi-step vertical movement mechanism. Fig. 4a shows an overview of the multi-step up–down mechanism. By connecting the five-step slide rails with the chains and the sprockets, all of the slide rails are driven relative to the movement of the top rail. The rotary motion of the servo motor is transformed to linear motion by the ball screw allowing the top slide rail to move up and down in the vertical direction. This mechanism is based on the extension mechanism of a ladder truck. Fig. 4b, c show the installation of the multi-step up–down mechanism in the actual process. When the vertical component is contracted, all parts are held above the worker, which prevents collisions with the worker and ensures the worker’s safety.

Gripper design

The gripper serves to hold the bumper and control its orientation. The gripper has two degrees of freedom: the rotation around the tilt axis and the rotation around the roll axis. Because the automobile assembly line handles many types of vehicle bodies, B-PaDY must flexibly and firmly hold several types of bumpers with different shapes and mass characteristics. To achieve this, we adopt a vacuum grasping system. By vacuuming the back side of the bumper using the suction pads attached to the gripper, we achieve this flexibility regardless of the type of bumper and simultaneously improve the grasping success rate. The diameter D and the number n of the suction pads are determined by using the following equation.

$$\begin{aligned} D = 2\sqrt{\frac{mgS}{\pi nP}} \end{aligned}$$

(1)

where, m is the mass of the bumper and P is the vacuum pressure. S is the safety factor and is generally equal to eight in the case of vertical hanging. Considering the mass of the actual grasped bumper, four suction pads with diameters of 60 mm are attached to the gripper. The arrangement of the suction pads are determined considering the shape of all the bumpers treated in the actual assembly process.

Fig. 5

Grasping several types of bumpers. In this experiment, we test the front (left column) and rear (right column) bumpers of three car types for a total of six types of bumpers

Figure 5 shows an experiment on bumper grasping by B-PaDY. This experiment targeted six types of bumpers consisting of the front and rear bumpers of three car types. We confirm that our mechanism succeeds in grasping all bumpers without changing the diameter and the arrangement of the suction pads.

Determining proper delivery timing

In this section, we explain the method for deciding the proper delivery timing. By delivering the bumper with the proper timing, our system reduces worker wait-time and improves work efficiency without disturbing assembly tasks. Pattern recognition with a statistical model based on prior information has been used for modeling such decision making. In particular, this problem is regarded as binary classification. Solving such a classification problem typically entails machine learning techniques such as neural network [12, 13] or support vector machine [14, 15]. In this study, we instead emphasize the generalization of the worker model. We analyze the worker information based on the logistic regression model and determine the significant model parameters to avoid the over-fitting problem.

Definition of model variables

Many studies have addressed the handover between human and robot. Strabala et al. [16] grouped the handover task into the three processes: delivery process, signal process, and handover process. They analyzed the handover motion between humans. Then they applied the desired handover timing, robot configuration, and the approach based on the analysis results to the handover task between the worker and the robot. Basili et al. [17] focused on the transfer part in the handover between the humans. They performed a numerical analysis of the handover including the distance and velocity. Cakmak et al. [18] focused on the changes in the robot arm configuration at the start of the handover. They discussed the effect of the ambiguity of the robot configuration during the handover task and summarized the design process for a feasible robot configuration.

These studies used the following information to decide the handover timing.

• a. Direction of the human’s gaze.

• b. Orientation of the human’s body.

• c. Relative position between the human and the robot (end-effector).

• d. Elapsed time.

• e. Configuration of the robot while holding the object.

• f. Actions and signals by the human.

In this system, we use (a), (b), and (c) to determine the handover timing. We do not use (d) because of the variance in the work completion time among the workers. We also do not use (e) because it requires a priori consideration of the feasible handover configurations. Though (f) provides valuable information to directly understand the worker’s intentions, we do not use it because it increases unnecessary work. Consequently, we use the following input variables for this system.

• Worker’s position $(M_x, M_y)$

• Worker’s velocity $(V_x, V_y)$

• Worker’s body orientation $(O_x, O_y, O_z)$

• Worker’s face orientation $(F_x, F_y, F_z)$

To measure these variables, we use the Kinect v2, which is one of the most popular RGB-D sensors because of its usability and versatility. It can be used to capture details of the human face and body with high precision. In this system, worker’s mass position $(M_x, M_y)$, orientation $(O_x, O_y, O_z)$ and face orientation $(F_x, F_y, F_z)$ are obtained from Kinect. The worker’s velocity $(V_x, V_y)$ is calculated from the worker’s mass position.

Logistic regression model

The logistic regression model is widely used in various fields because of its ability to represent two-state occurrence probability using continuous values. This makes it suitable for numerical decision modeling [19]. Miravitlles et al. [20] used logistic regression model to analyze risk factors of hospital admission and exacerbation for chronic obstructive pulmonary disease patients. Merlo et al. [21] analyzed social epidemiology with logistic regression model. In this system, we apply logistic regression analysis to statistical decision making of the robot to determine the proper delivery timing.

Here we describe the general case of the logistic regression model. Let $\varvec{x}=(x_1,\ldots , x_r)$ be the variables for describing an occurrence of a two-state phenomenon and y be the dependent variable for describing the result of an occurrence, where $y=1$ signifies occurrence and $y=0$ signifies non-occurrence. The vector of n-th sample data $(n=1,\ldots ,N)$ used in the logistic regression model is defined as

$$\begin{aligned} (y_n,x_{1n},\ldots ,x_{rn}) \end{aligned}$$

(2)

where N is the total number of samples. Using the logistic function, the conditional probability $p(y=1|\varvec{x})$ is expressed as follows.

$$\begin{aligned} p(y=1|\varvec{x})=\frac{\mathrm{exp}(Z)}{1+\mathrm{exp}(Z)} \end{aligned}$$

(3)

where Z is the linear multiple regression equation representing the influence of r variables and is given by

$$\begin{aligned} Z=\beta _0+\beta _1x_1+\cdots +\beta _rx_r \end{aligned}$$

(4)

where $\beta _i$ is the i-th partial regression coefficient. The logistic regression model is generated by estimating the partial regression coefficients ${\varvec{\beta }} = (\beta _0, \ldots , \beta _r)$ from the sample data. The maximum likelihood estimation method is generally used to calculate these coefficients. The likelihood function is as follows:

$$\begin{aligned} \mathrm{like}(\varvec{\beta })=\prod _{n=1}^N p(\varvec{x}_n)^{y_n}(1-p(\varvec{x}_n))^{1-y_n} \end{aligned}$$

(5)

Obtaining the maximum value of this likelihood function requires partial differentiation of the log likelihood $L(\varvec{\beta })=\log (\mathrm{like}(\varvec{\beta }))$ by $\varvec{\beta }$ and solving the following simultaneous equations.

$$\begin{aligned} {\left\{ \begin{array}{ll} \frac{dL}{d\beta _0} = \sum\limits _{n=1}^N (y_n-p(\varvec{x}_n)) = 0 &{} \\ \frac{dL}{d\beta _s} = \sum\limits_{n=1}^N x_{ns}(y_n-p(\varvec{x}_n)) = 0 &{} \end{array}\right. } \end{aligned}$$

(6)

The logistic regression model is generated by calculating the coefficients $\varvec{\beta }$ from the above equations.

Variable selection

When multiple regression models such as logistic regression model are used, the over-fitting problem should be considered. In multiple regression analysis, the prediction performance tends to improve as the number of the input parameters increases. However, with too many degrees of freedom, the model will lose its ability to deal with newly observed data. Avoiding this over-fitting problem requires decreasing the number of parameters included in the model and simplifying it without compromising its prediction accuracy. The Akaike information criterion (AIC [22]) is often used to select the input parameters effectively. AIC is defined as

$$\begin{aligned} \mathrm{AIC}=-2\log p(\varvec{Y}|\tilde{\theta },M)+2K \end{aligned}$$

(7)

where M is the target model, $\varvec{Y}$ is the observed data, $\tilde{\theta }$ are estimated as the most probable parameters for the target model M, and K is the number of input variables. AIC is used when comparing several models and it is convenient because of its simple calculation and clear interpretation. A model with a smaller AIC is regarded as the better model for explaining a phenomenon.

We use the stepwise procedure to select the significant variables. The stepwise procedure is a common approach for selecting explanatory variables based on criteria such as the AIC in multiple regression analysis. The next most significant variable is added to the target model until the AIC stops decreasing. Forming the model to explain the worker’s state just before the handover with only these significant variables improves the generalization performance of the model.

Experiment

Experimental setup

Fig. 6

Experimental environment

Once we implemented the system described above, we performed an experiment to verify its validity. Figure 6 shows the experimental environment from the top view. First, the worker waits for the starting signal of the experiment while holding a bumper in the work area (green area in Fig. 6). After the starting signal of the experiment, the worker mounts the bumper on the front vehicle body (upper part in Fig. 6) and closes the hatch. Then, the worker moves to the handover area (blue area in Fig. 6) and receives the bumper from B-PaDY. This bumper is mounted on the next vehicle body (lower part in Fig. 6). The coordinate frame used for this experiment is shown in the right side of Fig. 6. We use the following two methods to decide the delivery timing for this experiment.

• Exp.1 Delivery timing determined based on the worker’s current position.

• Exp.2 Delivery timing determined based on the logistic regression model.

For Exp.1, B-PaDY starts the delivery movement at the moment that the worker enters the handover area. For Exp.2, the delivery timing is decided based on the logistic regression model constructed for each worker.

In this experimental bumper assembly process, we define proper delivery timing as the moment when the worker starts to move to the handover area after closing the hatch. We use 100 data samples, including 50 samples of the worker’s state just before and after the moment of proper timing and 50 other samples, to generate the logistic regression model. We obtained the data for each of the four subjects and constructed four unique models.

Table 1 shows the selected variables and AIC calculated from the each model. Although the models of all of the subjects consist of similar variables, they do exhibit slight differences. For example, the same two variables $V_y$ and $F_z$ are selected for all of the models. However, the third variable differs between most of the models and thus the calculated AIC is also different for each subject.

When deciding the proper timing based on the logistic regression model in a real-life situation, the dependent variable must be decided from the observed explanatory variables. We set the threshold probability $p_T$ and calculate the dependent variable y as follows.

$$\begin{aligned} y = \left\{ \begin{array}{l} 1, \quad p(y=1|\varvec{x})\ge p_T\\ 0, \quad p(y=1|\varvec{x})< p_T \end{array} \right. \end{aligned}$$

(8)

In this experiment, we used $p_T=0.9$ for the threshold. To avoid erroneous decisions caused by unexpected outliers in the sensor data, the delivery is started when the calculated occurrence probability $p(y=1|\varvec{x})$ exceeds the threshold $p_T$ for three consecutive frames. B-PaDY moves along the predetermined trajectory in 2.0 s. When Kinect sensor failed to track the worker, the number of failures was recorded and the experiment was conducted again.

Table 1 Model variables selected for each subject by using the stepwise method based on AIC

Experimental result

Each subject performed ten trials of two experiments for a total of 80 experiments. We define the waiting time as the difference between the arrival time of the worker and the robot at the handover area. We used the waiting time to evaluate the proposed method. In this experiment, we define the arrival time of the robot as the time when the worker has just entered the handover area (a circle with a diameter of 0.6 m) and the worker’s velocity has become less than 0.2 m/s.

Fig. 7

Example of experiment procedure (subject B). After assembling the bumper to the front vehicle body, the worker closes the hatch (a). Then, the worker moves to the handover area and B-PaDY starts moving depending on the result of the timing decision (b, c). After moving, the handover is performed in the handover area (d)

Fig. 8

Experimental result. The blue bars show the mean waiting times for Exp.1 whereby the delivering timing is decided based on the worker’s current position. The green bars show the mean of waiting times for Exp.2 whereby the delivering timing is decided based on the logistic regression model. The black line indicates the variance of each result

Figure 7 shows the experiment performed by subject B using the proposed system. It is confirmed that the bumper delivery is performed at the proper timing after closing the hatch. Figure 8 shows the mean and variance of the experimental result of the waiting time for each subject. We confirmed that the waiting time of all subjects was reduced in Exp.2 using the method based on the logistic regression model. Therefore, work efficiency is enhanced by implementing the proposed system.

Discussion

Table 2 Results of statistical test for each subject

We use a statistical test to verify if there is a significant difference between the result of Exp.1 and Exp.2. Table 2 summarizes the results of the statistical test for each subject. First, we used the Kolmogorov–Smirnov test to check the normality of these results. In Table 2, the p value of the Kolmogorov–Smirnov test exceeds the significance level 0.05 for all of the experiments, signifying that these results are likely to follow the normal distribution. Thus, we conducted the statistical test under the assumption of normality. Then, we checked whether there is a significant difference between the mean of the results using the paired t test. In Table 2, the p values of the paired t test for all subjects are below the significance level 0.05, signifying that there is a significant difference between the result of the two experiments. Therefore, we consider that the proposed system effectively decreases the waiting time.

However, the variance of the waiting time in Exp.2 is high, especially for subjects B and D. The main reason for this is because the current system only determines the delivery timing and the trajectory of the robot is not considered. Consequently, the variance in the delivery start timing between each trial is affected by factors such as the worker’s velocity and work time. Because these variations can increase the mental load on workers, it is important to reduce the variance and accomplish stable bumper delivery timing.

From Table 1, we confirmed that selected model variables are different among the subjects. In this timing decision system, we adopt a worker-dependent decision model which is tuned for each subject based on the AIC. While it is time-consuming to construct the model for each subject, it has the advantage of considering the individual differences and can enhance the precision of timing decision. Since the worker and the work environment are frequently changed in the actual assembly process, the worker-dependent model could be suitable.

In this experiment, there were three tracking failures by the Kinect sensor. Our system cannot operate when the Kinect system fails to track the worker. Therefore, a redundancy mechanism is required for this system by adding a new sensor for measuring the worker’s state. In addition, this experimental process simplifies the actual assembly process. When using only a simple model such as logistic regression within a complex real-life process, the occurrence of erroneous decisions may increase. Thus, delivery timing should be decided by estimating the worker progress based on the worker’s state.

Conclusion

In this paper, we propose a robot co-worker system B-PaDY for use on an automobile assembly line to deliver bumpers for workers to install. We summarize the functions used to implement and explain the hardware design, and describe how these functions related to the current bumper assembly process. We detail the concepts and design related to the components of B-PaDY including its rail, arm, and gripper. We confirm that our proposed grasping system can hold several types bumpers.

Additionally, we propose a method to determine proper delivery timing using a statistical approach. This system decides a suitable delivery timing by modeling the decision making using the logistic regression model based on the current worker’s state. We performed a verification experiment to confirm the validity of the proposed method.

In future work, we aim to prepare the system for installation in an actual assembly line. In the actual process, the target work area on a production line moves at a constant speed with the worker and vehicle body. Thus, we must develop a system that considers the movement within the work area.

Furthermore, we must implement safety measures to ensure the feasibility of the system in a real work environment. When a robot works with a human in the same space, it is essential to ensure the safety of workers. We must implement functions that detect potential collisions with the human and abnormal actions of the worker to cooperate harmoniously with human workers.