A robotic end-effector with rolling up mechanism for pick-and-release of a cotton sheet

Abstract

In this paper we introduce a novel robotic end-effector for pick-and-release of a sheet of cotton cloth. One of the main operations in a factory for sewing cloth products is to pick up cloth parts, provide them to a sewing machine, and sew them together. Fabric products are assembled by performing this work in various processes. Each work starts with a lot of cloth parts stacked near a sewing machine. In the case of underwear, thin cotton cloth is the target of manipulation, but such cloth parts tend to stick to each other, and a certain skill or technical acuity is required for picking up. Therefore, this work is basically done manually in the present. The purpose of this study is to automate such manipulation, so we devise an end-effector that picks up only the top one of stacked cotton sheets. The proposed end-effector has a mechanism that is a cylindrical brush attached with a remover cloth on its surface. The cylindrical brush is grounded on a cotton sheet, and then is rotated to roll up only the cotton sheet. This mechanism makes it possible to release the sheet by rotating the brush in the reverse. We constructed hardware with this function and attached it to the tip of a robotic manipulator. In addition, we performed system integration so that the initial position of rolling up a cotton sheet could be automatically determined from images taken by a camera. We also conducted evaluation experiments.

Introduction

One main operation in a factory for sewing cloth products is to pick up cloth parts, provide them to a sewing machine, and sew them together. Fabric products are assembled by performing this work in various processes. This work begins with a large number of cloth parts stacked near a sewing machine.

In the case of manufacturing underwear, cloth parts made of thin cotton sheet are sewn together. In such cloth parts, stacked two sheets easily stick to each other, and it takes a certain knack to pick up only one of the stacked sheets. Therefore, this work is basically done manually. If this part could be automated, it is expected to contribute greatly to improving the automation rate of the entire sewing work by combining with already automated work using sewing machines. As a result, a great effect is expected as a measure to cope with labor shortage.

The purpose of this study is to automate the work of providing a cotton sheet, which is often used for underwear, to a sewing machine. In particular, we target the work of picking up one cotton sheet from stacked sheets and releasing them at the desired timing. The end-effector and manipulation procedure to realize this series of work are examined.

Fig. 1

Left: stacked cotton sheets, right: picking up the top sheet

Cotton cloth is a thin, light, and easily-to-stick cloth material. Therefore, when cotton sheets are stacked, picking up only one sheet as shown in Fig. 1 is a delicate work even when performed by a human. There have already been methods of automating the manipulation of cloth products, but their gripping method cannot be basically applied to pick up a cotton sheet. The reason is that conventional end-effectors are not suitable for handling cotton sheets.

The authors have studied a method of picking up a sheet of cotton cloth using a brush [1]. We succeeded in rolling up only the top cotton sheet by pressing the brush on the edge of the cotton sheet and then rotating the brush. However, there were problems such as rolling up multiple sheets at once when the bristles of the brush penetrate deeply into the stack of cotton sheets. In this study, we propose a novel hardware and manipulation procedure that can pick up a cotton sheet more stably with keeping the idea of rolling up. In addition, the proposed hardware was mounted on the tip of an articulated manipulator, and the picking up and release of cotton sheet was evaluated.

The contributions of this work are as follows:

• We propose an effective way, which introduces a cylindrical brush, for picking up thin sheets of fabric that easily adhere together.

• We designed the hardware to realize the proposed picking up method and devised an appropriate manipulation procedure.

• We evaluate the effectiveness of the proposed end-effector by performing the picking procedure using an actual robot system with a vision sensor.

The paper is structured as follows. In the next section we discusses related work. The third section introduces our problem settings and concept, and the forth section explains the proposed end-effector with rolling up mechanism. The fifth section presents fundamental evaluation of the proposed hardware. The sixth section introduces experimental results with an integrated robot system, and seventh section concludes the paper.

Related work

Mechanical hardware for garment picking

One approach for picking cloth items from a stack is the use of dedicated robotic hands. Research and development from a similar perspective has been ongoing [2, 3]. Schulz [4] introduces several types of gripper for garment picking. One is “Needle” gripper that aims to entangle the fabric by card clothing. Others are “Bonding” gripper for picking up a sheet of cloth by sticking out an adhesive tape, and “Freezing” gripper for adhere a target sheet by freezing and bonds after giving water. These methods may damage the fabric or temporarily stain it. Ono et al. [5] proposed a robot hand to pick up a piece of cloth. In that work, the number of stacked items was assessed using measurements from a sensor in the tip of the hand. The hand was then inserted into the stack so as to grasp a given number of items.

There are other approaches as well. Related to the rolling motion adopted in our study,

Kabaya et al. [6] proposed a mechanism to pinch a piece of fabric by rotating the rollers facing each other. Its performance was verified by experiments using relatively thick fabrics such as towels and felts. There is another work on flipping pages using roller-type effector [7].

In [7], a roller equipped with a piezoelectric sensor was used to flip pages of a booklet and assess whether a page actually flipped. However, the cotton sheets targeted in this study are more flexible than paper and tend to adhere together, hence one can easily foresee that it would be difficult to flip up individual sheets with this type of approach. For the same reason, it is difficult to use a conventional robot hand [8, 9] for the purpose of handling cloth. Therefore, a new method is required.

Cloth manipulation system

Within the field of Intelligent Robotics, progress is being made in robotic cloth manipulation [10, 11]. As picking cloth up is the first step in manipulation, this subtask is addressed in many lines of cloth manipulation research. Willimon et al. [12] address the task of selecting a single grasp point for lifting up a cloth item from arbitrary messy shape configurations. Doumanoglou et al. [13] proposed a method for spreading out unknown cloth items presented in arbitrary shape configurations. They used Hough forests for grasp point detection. Ramisa et al. [14] proposed a method for detecting characteristic parts like collars and cuffs in images of shirts etc. in arbitrary shape configurations. Yamazaki [15] proposed a method for simultaneous detection of multiple grasp points on cloth items in arbitrary shape configurations. As these examples illustrate, many studies have demonstrated success in picking up cloth items. However, these methods assume problem settings where the cloth can be properly grasped by just extending the hands to approximately detected grasp points, whereas we target dexterous picking of densely stacked items.

Focusing on high-precision picking, Le et al. [16] developed an end-effector equipped with proximity sensing abilities. Nagata et al. [17] proposed a towel-picking method using touch sensors. Both approaches contribute towards increased dexterity. However, these end-effectors cannot easily be applied to the problem setting of densely stacked thin fabric sheets assumed here.

Problem settings and our concept

Problem settings

We assume that cotton fabric sheets used to produce underwear etc. are cut to a specified shape and stacked. The number of sheets in the stack is assumed to vary from one to tens. The sheets’ edges may be well-aligned as when the stack has just been cut to shape, or may have slightly misalignment as when the sheets are re-stacked after another processing step. The fabric is thin, and when attempting to lift up the top sheet, the underlying sheets easily adhere to and move along with the top sheet.

Our goal is to separate and lift up the top sheet from the stack. Afterwards, the sheet would be fed into e.g. a sewing machine. This requires that we can release the grasped sheet.

Hardware concept

The basic policies for constructing an end-effector are as follows:

• Do not damage the cloth or attach liquids to it during picking up. It should be a method that does not apply excessive force to cloth.

• The end-effector must be small enough not to interfere with the supply of fabric to sewing machines. Also, it is desired not to move manipulators unnecessarily during picking up the cloth. For example, after the end-effector is brought into contact with a cotton sheet, the sheet can be held only by the movement inside the end-effector.

• Suitable for repetitive work. In other words, even the case that a large number of cotton sheets are stacked, it is possible to continue picking up only the top cotton sheet without significantly affecting the state of the remaining sheets.

Fig. 2

Top: appearance of end-effector and its major sizes, bottom: front view

The end-effector for rolling up a cloth sheet

Overview

Figure 2 shows an overview of the proposed end-effector. The maximum length of each side is 100 mm in length, 132 mm in width, 124 mm in height. The mass is 809 g. This end-effector is divided into an upper part and a lower part. A plate is attached to the uppermost part of the upper part, and this plate is attached to the tip of the robotic manipulator for use. The upper part and the lower part are joined by a linear bushing.

Small tactile sensors (Shokac Chip made by Touchence Co., Ltd.) is attacned to the upper part, whereas plates with 1 mm thickness is pasted to the lower part. There are almost no gap between the sensor and the plate. According to this combination, almost no up and down movement occurs in the lower part, but the force applied to the tactile sensor while picking can be measured.

The main parts are explained below:

• Cylindrical brush: It is a cylindrical part that is actively rotated by an actuator. Two cylinders are attached to one rotating shaft, and it is placed at the bottom of the end effector. ’Remover cloth’ is attached to the surface of the cylinders. The end-effector is approached from directly above to a cotton sheet placed on a horizontal table, and then the sheet is rolled up by utilizing the fact that the brush and the cotton sheet adhere. Details of this part are explained in the next sub-section.

• Actuator: The end-effector is operated by one unit of Dynamixel AX-18 (holding torque 1.8 N$\cdot$m, no-load rotation speed 97rpm) manufactured by Best Technology Inc. Since this actuator is small and lightweight, it can be embedded inside the end-effector.

• Driving force transmission section: The driving force from the actuator is transmitted to the cylindrical brush via a timing belt, gears, and pulleys. This timing belt is located in the center of the end-effector and rotates the two brushes. At the same time, the end-effector body is moved horizontally in the front-rear direction. This mechanism will be explained in detail in the final sub-section of this section.

Fig. 3

Cylindrical brush

Cylindrical brush

There is a previous study [1] as a background of using the cylindrical brush. In this study, a cotton sheet was rolled up using a cylindrical brush used for the purpose of polishing the inside of tubes. The brush selected during the course of the previous study had a diameter of 6 mm and a shaft diameter of 3 mm. In that study, the basic effectiveness was clarified, but two problems remained. One is that the strength of the shaft is low, which may cause unnecessary bending. Another is that it is difficult to make the lowest point of the brush lower than the bearing. Therefore, the main body of the end-effector including the bearing had to be placed outside a table where cotton sheets are placed. Another problem was that the length of the brush was so short that it could only pick up the corners of the cotton sheet. In such a case, the cotton sheet cannot be held in an unfolded state, and it is difficult to release it in the desired shape.

Therefore, in the end-effector proposed here, we bring in a new brush part. A cylinder with a radius of 12 mm is molded with ABS resin so that the position where the brush and a cotton sheet meets is lower than the protrusion of the bearing, and a remover cloth is attached to the circumference, as shown in Fig. 3. The remover cloth has flexible bristles of about 1 to 3 mm that grow at an inclination of about 1 to 5 degrees. It was confirmed experimentally that the picking and releasing of a cotton sheet were good when these bristles were entangled in cotton sheet. The remover cloth have weaker adhesion to cotton sheet than the previous brush. However, by using two wide cylindrical brushes, a large area can be brought into contact with the cotton sheet, so it is sufficient for picking up and releasing. In addition, since one hem of the cotton sheet is touched at once, it is easier to keep the cotton cloth in an expanded shape, as compared with the case of rolling only the corner portion as in the previous study.

The rationale for choosing the remover cloth was based on experiments, but as another candidate, we also considered using card clothing with reference to related work. For the results of this verification, refer to the “Fundamental experiments” section.

Fig. 4

Structure of pulley and gear for rolling-up manipulation

Table 1 Details of rotating components

Horizontal movement mechanism associated with brush rotation

When a cotton sheet is rolled up, if the cylindrical brush is simply rotated on the fixed position, not only the top cotton sheet but also sheets below it might be moved. Then, repetitive work becomes difficult. Therefore, it is desirable to move the axis of the cylindrical brush in the horizontal direction according to the rolling amount of the cylindrical brush.

The proposed end-effector achieves this movement with one servo motor, multiple timing belts and spur gears. The structure of these components is shown in Fig. 4. Pulleys 2 and 3 are attached to the main shaft of the motor, then Pulleys 3, 4 and 5 rotate the brushes. Meanwhile Pulleys 1 and 2 and Gears 1 and 2 achieves horizontal movement. Here, simply thinking, the amount of the brush rolling and the amount of horizontal movement should be set the same. However, we wondered if a good effect could be obtained for the picking work by making a difference between the two amounts. Therefore, Pulley 2 was designed so that the movement amount can be changed by preparing three types of teeth, 16, 24 and 32. The details of the components are shown in Table 1.

Using the rolling amount of the cotton sheet Lmm, brush radius, and pulley tooth ratio, the rotation angle $\theta _{motor}$rad of the motor is calculated by the following equation:

$$\begin{aligned} \theta _{motor}= \displaystyle {\frac{z_{pulley4}}{z_{pulley3}}} \times \displaystyle {\frac{z_{pulley5}}{z_{pulley4}}} \times \displaystyle {\frac{1}{r_{brush}}} \times L, \end{aligned}$$

(1)

where z indicates number of teeth. Using the angle, the amount of horizontal movement can be calculated by the following equation:

$$\begin{aligned} X = \displaystyle {\frac{PD_{gear1}}{2}} \times \displaystyle {\frac{z_{pulley2}}{z_{pulley1}}} \times \displaystyle {\frac{PD_{gear1}}{PD_{gear2}}} \times \theta _{motor}, \end{aligned}$$

(2)

where PD indicates pitch diameter.

As mentioned the above, the ratio of horizontal movement can be adjusted by changing Pulley 2. The following shows the relationship between the amount of horizontal movement X on each number of teeth and the amount of change L due to the brush rotation. L is equal to the amount of cotton sheet rolled.

$$\begin{aligned} X= \left\{ \begin{array}{lll} L &{} (Pulley 2=32) \\ 0.75L &{}(Pulley 2=24) \\ 0.5L &{} (Pulley 2=16) \end{array} \right. \end{aligned}$$

(3)

Based on this setting, we make it possible to verify which ratio is suitable for rolling up the cotton sheets stably.

Fundamental experiments

Examination of gear ratio

As explained in the previous section, the proposed end-effector can adjust the moving ratio by making the Pulley 2 replaceable. Therefore, the number of teeth of the pulley was changed in three patterns of 16, 24, and 32, and the performance was investigated.

The experimental settings were as follows. Cotton sheet to be manipulated had a size of $134 \times 170 mm$, a thickness of 0.5 mm, and a weight of 3.6g. An NR sponge rubber plate was placed on the desk for cushioning and non-slip. Five cotton sheets were placed on the plate, and they were picked up one by one in order and then released at a predetermined place. The location of the cotton sheet was known, and the start position of the rolling up was specified in advance. The proposed end-effector was attached to the tip of a 6-DoF manipulator (Manipulator H made by Robotis Inc. [18]) installed on the desk.

The motion of the robot in the pick-and-release experiment was as follows. First, the manipulator was moved so that the cylindrical brush is placed directly above the edge of the cotton sheets with the vertical distance about 40 mm. Next, it was moved vertically downward at a speed of 4 mm/s until a predetermined pressure value was detected by the tactile sensor. After that, the cylindrical brush was rotated by 202.5 degrees for rolling up a cotton sheet. Then, the cotton sheet was moved to another position and then was released by rotating the cylindrical brush in the opposite direction by the same amount.

The achievement level of the task was classified into the following 6 types:

• (a) The holding of a cotton sheet was in good condition, and also released successfully.

• (b) The holding of a cotton sheet was not in good condition, but successful until releasing.

• (c) Succeeded to pick up, but failed to release (cotton sheet does not peel)

• (d) Several sheets were picked up at the same time.

• (e) Any sheet could not be picked up

• (f) One cotton sheet was able to picked up, but it dropped while carrying.

In this experiment, both (a) and (b) were treated as success and others were failures. A concrete example of (a) and (b) is shown in Fig. 5. In (b), although a part of a cotton sheet is separated from the cylindrical brush, but it is held by other parts and therefore it can be carried to another place.

Fig. 5

Success case of picking up

Table 2 Results on pick-and-release with changing gear ratio

Table 2 shows the results of picking and releasing for different gear ratios. Here, for each gear ratio, the motion of pick-and-release from the 5 stacked cotton sheets one by one were performed 5 sets. Therefore, the number of trials should be 25. However, if the robot picked up more than one sheets at a time, the experiment was continued without returning the extra cotton cloth. Therefore, the number of trials was reduced by that number. From the table, when the number of teeth is 32 and 24, the situation of picking up multiple sheets at once occurred several times. On the other hand, when it was not possible to pick up in the first trial (case (e)), picking up the same cotton sheet was tried again. This appeared one times for each with 32 and 24 teeth, and both could be picked up in the second trial.

The following equations were used to calculate three values shown in the bottom of the table.

$$\begin{aligned} \begin{array}{lll} pick\,up\,success\,rate = \{ (a) + (b) + (c) + (f) \} / N \\ release\,success\,rate = \{ (a) + (b) \} / \{ (a) + (b) + (c) \} \\ overall\,success\,rate = \{ (a) + (b) \} / N \end{array} \end{aligned}$$

(4)

In the case of 32 teeth, all three cotton sheets were picked up at once when the remaining sheets were three. When the number of teeth was 24, it was improved, but in the case of the remaining two sheets, two cotton sheets were often picked up at once. On the other hand, when the number of teeth was 16, all were successful except for one failure in release. From these results, it was found that 16 is suitable for the number of teeth.

Let consider the factors on the resulted shown the above. If the number of teeth is 16, the horizontal movement amount is half the brush rotation amount as shown in Eq. (3). In this case, as shown in Fig. 6, the edge of the top cotton sheet is pulled in the direction of brush rotation. This resulted in the effect of peeling the adhesion between two top cotton sheets, making it easier to roll up only one sheet.

Fig. 6

Peeling of cloth by pulling horizontally

Surface material of the cylindrical brush

Some of the previous studies have achieved the pick-up of cloth sheets using needles. Based on this finding, the performance of a similar method was also investigated in this study. Card clothing as shown in Fig. 7 was used by wrapping it around a cylinder with the same as remover cloth.

The results are shown in Table 3. Although the experimental conditions were the same as with the remover cloth, the bristle tips of the card clothing pierced the cotton sheet deeply, and in many cases, two or three sheets were picked up at a time. For the same reason, once rolled cotton sheets could not be released. For example, in the case of 16 teeth, there was only a case where multiple sheets were picked up at once, and even if the cylindrical brush was subsequently rotated in reverse, the cotton sheet did not separate from the brush. From the above, it was found that the card clothing is not suitable for picking up the cloth materials targeted in this study.

Fig. 7

Cylindrical brush with nylon needle fabric

Table 3 Results on nylon needle fabric

Verification experiments

In addition to the cotton fabric used in the above-mentioned experiments, the following three cotton fabrics were also used:

• (1) The size is 89 × 290 mm, the thickness is 0.6 mm, and the weight is 4.4g. 100% cotton on both front and back.

• (2) The size is 135 × 170 mm, the thickness is 0.6 mm, and the weight is 5.8g. Jersey stitch. 100% cotton on front and 100% wool on back.

• (3) The size is 135 × 170 mm, the thickness is 0.3 mm, and the weight is 2.0g. Rib knitting.

All of these were soft and thin materials that are used for underware, and are materials that stick together easily.

The number of teeth on Pulley 2 was fixed at 16. Five sets of pick-and-release experiments were conducted against ten stacked cotton sheets. Then, the success or failure of the work was classified by the above-mentioned (a) to (f). Table 4 shows the results.

Table 4 Results on three different cotton sheets

For fabric (1) and (2), It was possible to pick up and release with a high success rate. On the other hand, the fabric (3) had a bad result. This is probably because the fabric (3) is lighter, thinner, and softer than other fabrics. As an additional verification, for the fabric (3), an experiment was conducted in which the upper 10 sheets were picked and released when the number of stacked sheets was 20. The result was as follows: (a) 23 times, (b) 0 times, (c) 21 times, (d) 3 times, (e) 0 times, (f) 0 times. As a result, the pick-up success rate was 44/47=93%, and the release success rate was 21/44=47%. There was an improvement in the pick-up success rate compared to the case of stacking only 10 sheets. From this result, two things can be inferred. First, if the surface on which the cotton sheets is placed is made of a softer material, there is a possibility that the pick-up success rate can be increased even if the number of stacked sheets is small. Secondly, in the case of fabric (3), it is difficult to release it using its own weight. Possible countermeasures are as follows: (i) sticking a material with a large frictional force to the place where the cotton sheets will be placed. (ii) adding a mechanism to actively push out from the cylindrical brush at the time of release.

Integration experiment

System integration and experimental settings

The purpose of this study is the automation of pick-and-release of cotton sheets. In the previous section, the experiments were performed on the assumption that the end-effector contacts a cotton sheet in an appropriate positional relationship. In this section, we report the results of an integrated experiment that automates the positioning part of the cylindrical brush using a camera.

Figure 8 shows the appearance of the experimental system. The proposed end-effector was attached to the tip of Dobot-M1, which is 4-axis SCARA robot manufactured by Dobot Inc. A downward facing color camera was attached to the front of the end-effector to detect the edge of the cotton sheet. To improve the success rate of image processing, a lighting to create shadows of the edges of the cotton sheets was also installed. The cotton fabric for this experiment was the same one introduced in the first sub-section of the previous section. A pick-and-release experiment was performed from the state where 15 cotton sheets were stacked.

Fig. 8

The proposed end-effector attached to an articulated manipulator

Detection of the edge of cotton sheets

Image processing for detecting a target clothing edge is as follows. First, an image is captured from the camera mounted on the wrist of the manipulator. Next edge-detection is performed by means of Sobel operator, obtaining the image’s derivatives in vertical and horizontal direction. Then the image is thresholded to leave just the pixels that represent strong edges. We identify pixels with a directional component between −20 and 20 degrees as horizontal edges and pixels with a directional component between 70 and 110 degrees as vertical edges. Next, edges with a surface area below a given threshold are discarded.

From among the remaining edges, we select the edge closest to the image’s origin (its top-left corner) in the horizontal direction as our vertical edge and the edge closest in the vertical direction as our horizontal edge, and fit a straight line to each. Then we find the intersection of these lines to identify the corner of the cloth. On the basis of this result, we select a brush placement position on the detected edge.

Fig. 9

An example of the pick-and-release experiments (i) edge detection for determining contact position, (ii) contact to the top cotton sheet, (iii) rolling up the edge of the sheet, (iv) picking up, (v) releasing, (vi) finished the releasing

Continuous pick-and-release from stacked cotton sheets

Experiments that pick-and-release the top 10 cotton sheets one-by-one from the stacked 15 sheets were conducted in three sets. Figure 9 shows an example. In this experiment, if the picking of a cotton sheet failed, the targeted sheet was manually removed. As a result, the success rate of picking was 83% and the success rate of releasing was 80%. As a failure example, there was a case in which the stacked cotton sheets were slightly moved during a picking up manipulation, which affected the subsequent work.

The time required for one pick and release was about 10 seconds. However, the motion sequence was not optimized yet, so there is plenty of room for speeding up.

To seek appropriate motion of the end-effector in more detail, pressure force to a cotton sheet needed for appropriate pick-and-release was investigated. The tactile sensor shown in the lower part of Fig. 2 was used to measure the force. The value obtained here is different from the value applied to the cotton sheet, but the mass of the lower part of the end-effector is invariant, so it was judged that a quantitative evaluation could be made by observing the change in this value. The horizontal axis in Fig. 10 is the height of the end-effector from the reference plane, and vertical axis shows the success rates of rolling up and releasing

when a fixed number of cotton sheets were placed on.

As a reference value, the pressure force was 0.05N at maximum when the height was 111.5 mm. Meanwhile, it was about 0.27N when the height was 110 mm and about 3N when it was 107.5 mm. As can be seen from Fig. 10, the success rate of both picking and releasing was high when the pressure force was about 0.27N. From this fact, the success rate of both picking and releasing can be maintained by setting the magnitude of the force applied to cotton sheets appropriately.

Fig. 10

Height of the end-effector vs. Success rate

Conclusions

In this paper, we designed and developed a novel end-effector for pick-and-release of stacked cotton sheets one by one. Based on the findings of previous study [1], we proposed the use of cylindrical brush and incorporated it in a compact end-effector. This makes it easy to mount the end-effector on an articulated manipulator. We also discovered that we can improve the success rate of picking-up and releasing by adopting a remover cloth as the material of the rolling part and by halving the horizontal movement amount with respect to the brush rotation amount.

Future work include the investigation to improve the success rate when the number of stacked cotton sheets is increased significantly. In addition, we will study an approach to appropriately supply a cotton sheet to the target position when releasing.

Further issue, it is very important to properly determine the contact position of the end-effector against various shapes of cloth parts. It will increase the applications of the proposed end-effector significantly.