Rubber artificial skin layer with flexible structure for shape estimation of micro-undulation surfaces

Abstract

In this paper, we propose a wearable tactile sensor for measuring the shape of micro-undulations on a hard surface. The proposed sensor has two layers. The inner one is a thin rubber layer into which a strain gauge is embedded that is formed around a user’s finger. The outer layer is a flexible structure that consists of numerous pins on a flexible sheet. The shape of micro-undulations on an object surface can be measured when a user wearing the sensor traces the surface. The results of an experiment, in which we compare the cases with and without the flexible structure of the sensor, show that our proposed sensor is sufficiently accurate to measure the shape of micro-undulations due to its flexible structure, which contributes significantly to improving its signal-to-noise ratio.

Introduction

In manufacturing workplaces, skilled and experienced craft workers touch and inspect the surfaces of products or parts to find irregularities. Such surface inspections are important for maintaining product quality. For example, if an irregular undulation exists on the surface of a car body, the appearance of the entire car is degraded and its quality as a product is reduced. In the case of a car, detecting surface undulations is simplified by the ability to observe the distortions of light rays irradiated onto the painted body. However, since such light ray inspections must be performed on the downstream of the product line after painting, significant expenses are incurred before a car arrives for the inspection. Therefore, it would be much better to detect irregular undulations and exclude poor quality parts on the upstream of a product line.

Such upstream product line surface inspections require the following:

• 1. They must be capable of being performed quickly on product lines where processed parts are flowing continuously one after another.

• 2. They must be capable of covering wide areas of up to several hundred millimeters.

• 3. They must be capable of determining surface shapes that are visually indistinct due to dirt and/or metallic reflections.

These requirements were clarified in worker interviews discussed in [1,2,3], in which it was also stated that workers need about 10 years of training to detect micro-undulations in the abovementioned conditions. The detection undulation is not defined, but the height is several tens of micrometers. Assuming that the undulation is a sinusoidal wave, undulations that smoothly change with a period of 10 mm or more will be detected. While micrometers can be used for detecting surface undulation of size up to several tens of micrometers, and are thus suitable for precisely inspecting a narrow range, they are unsuitable for inspections over wide areas. Furthermore, although laser sensors or methods based on image processing can be considered, it is difficult for those methods to sense surfaces when they are obscured by dirt or metallic reflections. Therefore, as part of efforts to create an undulation surface detection method that can be used even on dirty surfaces immediately after processing, we propose the new tactile sensor shown in Fig. 1 for surface strain detection use. Because it is wearable, the proposed device can be used as a tool in current work processes and can be easily incorporated into existing production lines without the necessity of introducing new equipment. Since this sensor can also be used to add a tactile sensing ability to a robot finger, we plan to incorporate it into an automatic surface undulation detection system in the future. As a preliminary step, this paper investigates whether irregularities on a surface can be detected by attaching the sensor to a user’s finger and then using it to perform active touch-based detection.

Fig. 1

Schematic diagram showing a the structure and b a photo of the proposed sensor. Deformation due to an undulation on the object’s surface is amplified by the pins of the flexible structure, after which information of the object’s surface is transmitted through the flexible structure to the sensor element

Related work

While a micrometer is generally used to detect micro-undulations, that device is normally only useful for inspecting a specific portion of a surface in which an undulation has been found or is suspected, and is unsuitable for extensive searches. In general surface inspections, a highly skilled worker traces the surface to detect undulations. One alternative method, which involves the use of a tactile device designed by Sano et al. called TouchLens, has been proposed [1, 4, 5]. TouchLens is a tactile device that enhances undulation surface perceptions using a lever mechanism. Although anyone can easily find micro-undulation using this device, the tactile information obtained cannot be handled as numerical information, and there is a problem that it is difficult to evaluate objectively.

Focusing on the point of sensing, Kikuuwe et al. developed a hand-bell type cantilever haptic sensor [6]. Since the inspector holds that device by hand when using it to trace surfaces, undulation locations can be immediately and intuitively determined. However, due to its large size, this device is difficult to use on large curved surfaces or surfaces with narrow measurement ranges.

On the other hand, humans can move their fingers and hands dexterously, press them flexibly on various shapes, and easily trace them over their surfaces. Furthermore, surface information can be obtained through tactile sensations. With these points in mind, it is reasonable to consider wearable finger-mounted tactile sensors. Tanaka et al. developed a tactile sensor of this type for undulation surfaces detection [2] and showed how they could conduct inspections by active touch and achieve real-time measurements using wavelet transform. However, quantitative evaluations such as those required for undulation size discrimination have not been conducted in detail. Another device, GelSight [7, 8], was developed as a measuring device for detecting surface unevenness with high accuracy. However, although GelSight is able to reconstruct the shapes of surfaces and soft materials onto which it has been pressed via image processing, this device is unsuitable for undulation detection over a wide area because it has to be pressed over all of the measurement ranges.

In order to measure wide areas, methods that work by tracing surfaces such as those discussed by Kikuuwe et al. [6] and Tanaka et al. [2] are more suitable. We developed an artificial skin layer in which two thin rubber coatings were formed around the wearer’s finger and a strain gauge was embedded between the rubber layers [3]. The tactile sensor outputs a signal corresponding to detected deformations when the wearer’s finger is traced along the surface to be measured. Although this device allows surface curvature information traced from the output signal to be reconstructed, it is difficult to use when reconstructing the 10 micro-level undulations that we are aiming to measure.

Research involving sensors embedded in flexible materials such as the abovementioned developed device has been conducted for a long time now. In the field of robotic manipulation studies, a number of tactile sensors have been developed for dexterous robots [9]. For example, Yamada et al. [10] developed a distributed tactile sensor with ridges that could be used to realize grip force control via tactile sensing, while Hosoda et al. [11] developed a robot finger in which polyvinylidene difluoride (PVDF) film was randomly embedded in a flexible material. This resulted in a robot finger with improved manipulation capabilities. However, distributed tactile sensors require numerous sensor elements and complicated wiring. A potential solution to this issue is the center of pressure (CoP) sensor developed by Shimojo and Ishikawa et al. [12, 13], which dramatically reduced the number of wires required.

When a flexible object is used, it can be designed to deform according to the shape of the contacted surface, which means that tactile information can be more easily extracted from the deformation. With that point in mind, inexpensive vision-based sensors with simple configurations have been developed to measure flexible object deformations. For example, Nagata et al. [14] developed a tactile sensor that can detect the shape of an object onto which it has been pressed by capturing the displacement of a pin aligned with a flexible object using a camera. In another example, Kamiyama et al. [15] attached a marker to the surface of a flexible object and used a camera to capture the marker deformation. The developed device, GelForce, is a tactile sensor that can measure the magnitude and direction of force. Similarly, Chorley et al. [16] and Lepora and Ward-Cherrier [17] developed TacTip as a more accurate tactile sensor, while Yamaguchi et al. [18] developed FingerVision as a small and inexpensive tactile sensor for robotic manipulation. The abovementioned GelSight is also a vision-based tactile sensor [7, 8]. Although vision-based tactile sensors can achieve high accuracy with simple configurations, problems occur when vision-based systems become larger than the flexible tactile sensor in which the sensor element is embedded.

Recent evolutionary developments in semiconductor and fabrication technology have made it possible to produce artificial skins as even thinner tactile sensors that are much like human skin. In the electronics field, very light, thin, and stretchable electronic circuits that can be applied to skin surfaces in the form of tattoos and artificial skin have been developed to create tactile sensors [19,20,21]. Unfortunately, the fabrication of these types of artificial skin requires advanced technology. In another development, artificial skin capable of rapid prototyping has been produced. For example, Tenzer et al. [22, 23] developed a microelectromechanical system (MEMS) embedded tactile sensor that makes it easy to perform rapid typing and showed that it could be applied to artificial skin, while Charalambides and Bergbreiter [24] proposed a method of embedding electronic circuits in soft materials via a simple manufacturing method and developed an artificial skin that can detect slip. While artificial skin developments have often been pursued for use robot tactile sensors, numerous wearable sensors have also been developed for extracting human hand motions and tactile sensory information [25]. For example, Park et al. and Hammond et al. [26, 27] developed an artificial skin using liquid metal and extracted hand movement and pressure distribution information, while Li et al. [28] developed an artificial skin that could be used for gesture recognition.

Although many of the tactile sensors and artificial skin studies described so far are aimed at force sensing, there have been few studies aimed at the detection of micro-undulations for applications such as surface strain testing. We have proposed a rubber artificial skin layer in which two thin rubber coatings are formed around the wearer’s finger and a strain gauge is embedded between the rubber layers [3]. The tactile sensor outputs a signal corresponding to detected deformations when the user’s finger was traced along the surface to be measured. We have shown its applicability to surface inspection with a simple configuration of one sensor element and thin layers. The sensor has the advantage of not interfering with other tasks since it is a thin wearable sensor. Although this device allowed surface curvature information traced from the output signal to be reconstructed, it is difficult to use when reconstructing the 10 micro-level undulations that we are aiming to measure.

Accordingly, we used the results of [3] as baseline and aimed at the realization of even smaller undulation detections. The main contributions of this study are considered to be as follows:

• We show that the shape of a micro-undulation with an amplitude of 10 $\upmu$m can be measured by our proposed wearable tactile sensor. This performance level is considered to be sufficient for surface inspections.

• We show that, by using a flexible structure, it is possible to improve the shape estimation capabilities of active touch surface sensing to much higher than has been seen in previous studies.

Device

Concept

The basic design of the proposed sensor is the same as the rubber artificial skin with embedded strain gauges [3]. The rubber artificial skin is a wearable tactile sensor in which a thin rubber form with an embedded strain gauge is placed around a user’s finger. Previous studies have shown experimentally that the output signal is linear for curvatures between − 60 and 100 1/m. This means that, by performing calibration in advance using some known curvatures, it is possible to estimate a curvature from the output signal. In order to sense smaller undulations, we improved this tactile sensor. As part of efforts to make them more sensitive, research has been conducted aimed at creating tactile sensors with contact surfaces that have structures inspired by human skin [29,30,31]. These studies have the effect of expanding the deformations of small stimuli, such as micro-undulations, from the viewpoint of the outside world, and we realized that it would be possible to add this effect to a flexible tactile sensor with a simple structure [32]. In the results of [32], we showed that it was possible to create an rubber artificial skin with embedded strain gauges that could amplify output signals by adding a structure that expands minute deformations in the shape of the tracing surface. Since TouchLens is used as the sensor component, the structure is the same. We use the results of [32] to measure the shape of micro-undulations. As described previously, a flexible structure is interposed between the rubber artificial skin equipped with embedded strain gauges and the surface to be measured. The flexible structure consists of numerous pins emplaced on a flexible sheet. When a user traces an undulating surface with a finger while wearing our proposed sensor the output signals caused by the surface undulations are amplified.

Prototype

In this section, we describe the prototype design of our proposed sensor. The rubber layer of the proposed sensor is formed around a user’s finger by direct dipping. More specifically, the user’s finger was dipped directly in liquid rubber without using a mold, thus creating an artificial skin layer fitted to the user’s finger. However, since the liquid rubber we use contains latex, we need to consider its potential as a latex allergy trigger. The dipping process was repeated three times with the strain gauges emplaced after the first dip and before the third dip. The liquid rubber used to make the skin layer of the prototype was general purpose natural rubber latex containing sulfur. The sensor element embedded in the second thin rubber layer was a strain gauge for large deformation (KFEM-2-120-C1L1M2R, Kyowa Electronic Instruments Co., Ltd., Japan). The dimensions of the flexible structure bottom plate are 20 mm $\times$ 20 mm, and the thickness is 0.2 mm. The pins were cylindrical with a diameter of 2 mm and extended 4 mm from the bottom plate. The distance between the pins was 3 mm, and each bottom plate contained 36 pins. This flexible structure was fabricated with three-dimensional (3D) printers, and the material is carbon fiber reinforced ABS filament ($\text {CarbonX}^{\text {TM}}$ ABS+CF, 3DXTECH, USA). To operate the device, the user places a sensor-equipped finger on the flexible structure so that the strain gauge is always positioned horizontally above the pins. Figure 2 is a schematic diagram showing the position of the finger on the flexible structure.

Fig. 2

Finger position on the flexible structure

Shape estimation of micro-undulation

We compared our proposed tactile sensor, which has a flexible structure interposed between rubber layers and the surface to be contacted, with the conventional sensor proposed in [3]. To accomplish this, we first prepared touch samples with sinusoidal surfaces. Users wearing both tactile sensors then traced those surfaces with their sensor-equipped fingers, after which the surface curvature changes of the samples were estimated from the output form of the tactile sensor signals.

Method

The proposed tactile sensor was calibrated to estimate the curvature from the output signal. Similar to the previous study, the relationship between the signal from the tactile sensor and the curvature of the touched surface was measured by pressing the tactile sensor against a cylindrical surface with a constant curvature. Although the output varies depending on the pressing force exerted, the sensor eventually becomes saturated as the pressing force is increased. The relationship between the output signal and the curvature of the surface being touched was measured using the value that registers when the output change due to the pressing force becomes saturated. Here, convex shapes are treated as positive curvatures and the concave shapes are treated as negative curvatures. In the previous study, it was shown that the relationship between the sensor signal and curvature is linear between − 20 and 100 1/m. Since this study aims at measuring curvatures from − 10 to 10 1/m, there is no particular problem using this method. Furthermore, it is shown that when the flexible structure is interposed between the sensor and the surface to be contacted, the output signal is multiplied about four [32]. In the touch samples prepared for this study, the depth from the base plane changes sinusoidally in the longitudinal direction. All touch samples were machined from square $30\,{\text {mm}} \times 30\,{\text {mm}} \times 200\,{\text {mm}}$ polyacetal resin poles. A 20 mm flat base position area is set at both ends of the undulation surface, which means the length of the undulation surface is 160 mm. The two-dimensional (2D) shape of the touch sample undulation surface can be expressed by the following equation:

$$\begin{aligned} z(x)= \,& {} A \cos \left( \frac{2\pi }{T}x\right) , \end{aligned}$$

(1)

where A and T are the amplitude and the spatial period of the sine surface, respectively. The coordinate system is set up as shown in Fig. 3. The curvature $\kappa (x)$ of the sinusoidal surface in the x direction is expressed as follows:

$$\begin{aligned} \kappa (x)= \,& {} \frac{-A \left( \frac{2\pi }{T}\right) ^2 \cos \left( \frac{2\pi }{T} x\right) }{\left\{ 1+A^2\left( \frac{2\pi }{T}\right) ^2 \sin ^2\left( \frac{2\pi }{T} x\right) \right\} ^{\frac{3}{2}}} \end{aligned}$$

(2)

where the positive and negative are reversed from the curvature of the mathematical notation because the convex type is positive and the concave type is negative. Since, in our case, $(A^2( \frac{2\pi }{T})^2) \ll 1$, we can approximate $\kappa$(x) as

$$\begin{aligned} \kappa (x)\approx \,& {} {\tilde{A}} \cos \left( \frac{2\pi }{{\tilde{T}}}x\right) \end{aligned}$$

(3)

where ${\tilde{A}}$ and ${\tilde{T}}$ are defined by

$$\begin{aligned} {\tilde{A}}:= \,& {} A \left( \frac{2\pi }{T}\right) ^2 \nonumber \\ {\tilde{T}}:=\, & {} T. \end{aligned}$$

(4)

In this paper, we treat ${\tilde{A}}$ as the curvature amplitude, and ${\tilde{T}}$ as the spatial period. We prepared touch samples which doubled ${\tilde{A}}$ or ${\tilde{T}}$ and quadrupled ${\tilde{A}}$ or ${\tilde{T}}$ based on a touch sample that has an amplitude of 10 $\upmu$m and a spatial period of 10 mm. In order to change ${\tilde{A}}$ and ${\tilde{T}}$, the amplitude was changed. However, in order to fix ${\tilde{A}}$ and change ${\tilde{T}}$, it was necessary to change not only the spatial period but also the amplitude. Thus, when only ${\tilde{T}}$ is doubled or quadrupled, not only must the spatial period be doubled or quadrupled, but also the amplitude needs to be multiplied by four or 16.

Fig. 3

Undulation surface coordinates. The flat surface and the undulation peak are at the same level

A user traced the surface of these touch samples with a finger wearing the tactile sensor and estimated curvature of the surface from the output sensor signal. Figures 4 and 5 show photographs and schematic diagrams of the experimental setup. Friction condition differences arise because the materials of the tactile samples that come into contact with the conventional and proposed sensors are different. To eliminate such friction condition differences, a piece of tissue paper is inserted between the sensor and the tactile sample. The side view motion of the finger was captured by a high-speed camera operated at 120 frames/s. Next, the position of the finger in the x direction was measured from the captured video using motion tracking software. In order to synchronize the sensor signal recording at 0 s with the position measurement time, the moment at which the light-emitting diode (LED) illuminated at the start of the sensor signal recording was taken as the position measurement start. Since each undulation sample begins and ends with a 20 mm flat section, it is possible to use the position measurement results to extract signals from when the sensor position is traveling from 20 to 180 mm. The user, which is first author, traces one section in about 4 s according to the metronome. For 100 s, the user traces the tactile sample with the finger wearing the sensor. The tracing was performed so that the pressing force was within the range of about 10 N to 15 N.

Fig. 4

Photograph of a finger wearing our proposed sensor during tracing. The marker attached to the tactile sensor is used for camera position measurements

Fig. 5

Sensor system. The sensor signal is uploaded into a personal computer (PC) via a strain gauge amplifier and an analog/digital (AD) converter

A low frequency component trend that was added to the sensor signal was removed. The low frequency component trends in the extracted signal were eliminated by subtracting a 4th-order polynomial curve fitted to the signal. The signal value was converted to a curvature value using the result of a calibration that had been done in advance. Since the position measurement sampling time (which is 1/120 s) is longer than the sensor signal sampling time (which is 1 ms) the number of measurement points in the position measurement is smaller than the number of measurement points in the sensor signal. Therefore, linear approximation was used to calculate the missing points so that the number of signal and position measurement points could be equalized.

Results

Figure 6 is a graph showing the actual curvature, which is represented by a solid line, of the sinusoidal curved surface with an amplitude of $10\, \upmu$m and a spatial period of 10 mm. The curvature, which is represented by the solid line, is estimated from the sensor signal. In this figure, the phases are shifted so that the two cross-correlations become maximum in order to compare the actual curvature change with respect to the position and curvature change estimated from the sensor signal. The horizontal axis represents the position in the x direction, while the vertical axis represents the curvature. Figure 6a shows the estimation result using the conventional sensor, while Fig. 6b shows the estimation result using the proposed sensor. In Fig. 6a, we can see the periodic signal, but the wave is disturbed. In contrast, we can see that the estimated curvature is similar to the actual curvature in Fig. 6b.

Fig. 6

Comparison between actual and estimated curvature. Each graph shows the output signal of (a) a conventional sensor (without a flexible structure) and b our proposed sensor (with a flexible structure)

Figure 7 is a graph showing the results of the Fourier analysis of the curvature estimation from the sensor signal. The horizontal axis represents the spatial frequency, while the vertical axis represents the amplitude in curvature variation. The dotted line shows the actual spatial frequency (0.1 1/mm). Figure 7a shows the estimation result using the conventional sensor, while Fig. 7b shows the estimation result using the proposed sensor. In these figures, we can see that both results show peaks in the vicinity of the actual spatial frequency, but the amplitude of the estimated curvature produced by the proposed sensor is closer to the actual value than the estimated curvature of the amplitude produce by the conventional sensor. In Fig. 7b, since a wave with a frequency lower than the actual spatial frequency is added to the conventional sensor signal, we can see that the wave is disturbed.

Fig. 7

Fourier analysis result of sensor signal. Each graph shows the result of a conventional sensor, and b our proposed sensor. The dotted line represents the curvature amplitude of the traced surface

Next, we calculate the normalized mean squared error (NMSE) to compare later curvature amplitudes ${\tilde{A}}$ with the modified measurement results. Assuming that the curvature of the surface to be measured is $\kappa$ and the curvature estimated from the sensor signal is ${\hat{\kappa }}$, the NMSE is as follows:

$$\begin{aligned} NMSE= \,& {} \frac{ \sum _{i}(\kappa _{i}-{\hat{\kappa }}_{i})^{2}}{ \sum _{i}(\kappa _{i})^{2}}. \end{aligned}$$

(5)

After calculating the average NMSE of the 10 measurements of each sensor, we found that the NMSE was 0.61 for the conventional type and 0.29 for the proposed type. Therefore, from the proposed sensor results, we can see that the estimated value was closer to the actual curvature value.

Next, we evaluate the noise produced using the signal to noise ratio (SNR), which is defined as follows:

$$\begin{aligned} SNR:= \,& {} 10\log _{10}\frac{\sum _{i}({\hat{\kappa }}_{i})^{2}}{ \sum _{i}(\kappa _{i}-{\hat{\kappa }}_{i})^{2}} \end{aligned}$$

(6)

The SNR was − 0.43 for the conventional type and 7.22 for the proposed type. Therefore, it can be said that the proposed sensor can reduce noise effects.

Next, we show the results obtained when we fixed the period ${\tilde{T}}$ of the surface to be measured and changed the curvature amplitude ${\tilde{A}}$. Table 1 shows the estimation results and their error rates. Although there is no change in the estimation ability of ${\tilde{T}}$, we can see that the proposed sensor improves the ability to estimate ${\tilde{A}}$. However, as ${\tilde{A}}$ increases, we can see that the error increases as well.

Table 1 Results of curvature amplitude ${\tilde{A}}$ and spatial period ${\tilde{T}}$ estimated by the haptic sensor when only the curvature amplitude is changed, and the respective error rates

Figure 8 is an NMSE graph calculated for each curvature amplitude ${\tilde{A}}$. The horizontal axis represents the curvature amplitude ${\tilde{A}}$ and the vertical axis represents the NMSE. Error bars show the standard deviation for 10 samples. In this figure, we can see that the estimated value using the proposed sensor is more accurate even if the curvature amplitude becomes larger. Furthermore, it shows that the NMSE decreases as the curvature amplitude increases.

Fig. 8

Graph showing the relationship between the curvature amplitude ${\tilde{A}}$ and the NMSE

Figure 9 is a graph of the SNR calculated for each curvature amplitude ${\tilde{A}}$. The horizontal axis represents the curvature amplitude ${\tilde{A}}$, and the vertical axis represents the SNR. Similarly, error bars indicate the standard deviation for 10 samples. In Fig. 9, it can be seen that the noise influence is reduced by the estimated value using the proposed sensor even if the curvature amplitude becomes large, and that the noise influence does not change as the curvature amplitude changes.

Fig. 9

Graph showing the relationship between the curvature amplitude ${\tilde{A}}$ and the SNR

Table 2 shows the estimation results and error rates when the measurements are made by fixing ${\tilde{T}}$ and changing only ${\tilde{A}}$. Even in this result, it can be seen that there is no change in the estimation ability of ${\tilde{T}}$, but we can see that the proposed sensor improves the ability to estimate ${\tilde{A}}$. We can also see that, even with small curvature change, errors can be measured accurately.

Table 2 Results of curvature amplitude ${\tilde{A}}$ and spatial period ${\tilde{T}}$ estimated using the haptic sensor when only the spatial period is changed, and the respective error rates

Figure 10 is a graph of the NMSE calculated for each spatial period ${\tilde{T}}$. The horizontal axis represents the spatial cycle ${\tilde{T}}$, and the vertical axis represents the NMSE. Error bars show the standard deviation for 10 samples. In Fig. 10, we can see that the proposed sensor gives better estimates even if the spatial period changes. However, this result also shows that the accuracy deteriorates when the spatial period is 20 mm. These results indicate that there is a spatial period where the NMSE peaks between 10 and 40 mm.

Fig. 10

Graph showing the relationship between spatial period ${\tilde{T}}$ and the NMSE

Figure 11 is a graph of the SNR calculated for each spatial period ${\tilde{T}}$. The horizontal axis represents the spatial period ${\tilde{T}}$ and the vertical axis represents the SNR. Similarly, error bars indicate the standard deviation for 10 samples. In this figure, we can see that the noise effect is smaller for the estimation results produced using the proposed sensor for any spatial period. Furthermore, this result suggests that the noise influence does not change even if the spatial period becomes excessive in the conventional sensor. However, the noise influence decreases as the spatial period becomes excessive in the proposed sensor.

Fig. 11

Graph showing relationship between the period ${\tilde{T}}$ and the SNR

Taken together, the above experimental results show that the curvature estimation with the proposed sensor is less affected by noise than conventional sensors and can be estimated more accurately.

Figures 12 and 13 show the position and pressing force for each time in each experiment. No extreme differences in speed and pressing force were noted. Other experiments were performed under the same conditions shown here.

Fig. 12

In the experiments conducted using our proposed sensor, the graphs show a the position and b pressing force for each time

Fig. 13

In the experiment involving the conventional sensor, the graphs show a the position and b pressing force for each time

Discussion

Advantages and limitations

One of the main effects of the proposed sensor is its ability to reduce the influence of noise and to estimate the shape of the surface traced more accurately than the conventional sensor. The fact that this result could be achieved via a tactile sensor with a simple flexible structure and a simple configuration of one strain gauge is a significant advancement.

Of course, there are some limitations as well. From Table 1, we can see that, for the proposed sensor, the error rate of the estimated curvature amplitude ${\tilde{A}}$ when the curvature amplitude ${\tilde{A}}$ of the standard undulation surface is quadrupled becomes very large compared to the conventional sensor. However, NMSE evaluations show that the shape of the sliding surface is estimated better than the other results. Figure 14 shows the estimated value from the proposed sensor signal and the curvature of the tracing surface when the curvature amplitude ${\tilde{A}}$ of the reference surface is quadrupled. In Fig. 14, it can be seen that the valley shape in the undulating surface is missing. This omission is the cause of the error in the estimated value. It can also be seen that the structure of the proposed sensor does not follow the shape. Therefore, it is not possible to estimate an undulating surface within a short spatial period. However, this limitation does not create a problem because the surface strain inspection that we are assuming is the detection of a wave with a spatial period of several tens to several hundreds of millimeters.

Fig. 14

This graph shows the output signal when a sinusoidal surface with an amplitude of 40 $\upmu$m and a spatial period of 10 mm is traced by the proposed sensor

Practical issues for surface undulation detection

In this study, we found that we could estimate the shape of the undulation surface with sufficient accuracy. However, some practical problems in actual surface strain inspections were identified. One of those practical problems is that extensive inspections are necessary. Since the proposed sensor in this research is a device attached to a user’s fingertip, it is unsuitable for use when searching for multiple undulations over a wide area because the process would take longer than the other methods presently used. In actual surface strain inspections, examinations are performed using the palm of the inspector’s hand, which makes it possible to access a wide area quickly with a single stroke of his or her arm. Accordingly, we consider it likely that extending our proposed sensor system to cover the palm of the user’s hand will make the system more applicable to a wider measurement range, and this extension is planned for the future. Additionally, since undulation surface inspections are performed on production lines, immediate judgments are required. Hence, real-time implementation of surface shape estimations is essential if practical use of our system is to be considered. Since Masaki et al. [33] developed an artificial rubber skin with embedded strain gauge system for detecting undulations in real-time that utilizes machine learning with neural networks, we consider it likely that incorporating machine learning into our proposed system could facilitate the creation of a more acceptable method for real-time implementation. From Fig. 6, the phase difference between the estimated and actual curvatures can be observed. This phenomenon suggests that the proposed sensor has dynamic characteristics. Although this phenomenon was also observed in [3], the cause of this phenomenon could not be clearly explained and will be a topic for our future research.

Conclusion

In this study, we proposed a wearable haptic sensor that can be used to detect surface undulations as small as $10 \,\upmu$m. Users wearing this finger-mounted sensor can measure surface shapes simply by tracing their finger across the surface of the object being inspected. The sensor consists of a rubber layer molded around the user’s finger that is equipped with an embedded strain gauge and a flexible structure that is interposed between the strain gauge and the surface to be inspected. When the proposed sensor is attached to a user’s finger and then pressed against the surface to be measured, the curvature change of the traced surface can be estimated from the sensor signal. At this time, in order to eliminate the influence of the pressing force, the user should continue pressing until the saturation level is reached. Although it is considered likely that other people will also be able to perform measurements using this method, as part of our future studies, it will be necessary to perform experiments to determine whether this is the case. With respect to surface curvature, we showed that it is possible to estimate the NMSE with an accuracy level of 0.5 or less, even if the amplitude or the spatial period is changed. Furthermore, the SNR is 5 or more, which indicates that it is not susceptible to noise.

Our future work will include the following:

• Verification by experiments with multiple people using the proposed sensor,

• Extending the sensor to cover the palm of the user’s hand in order to make it easier to inspect wide surfaces,

• Advanced modeling of this sensing system, including the proposed sensor and the flexibility of human skin, and

• Construction of a sensing system that is capable of operating in real-time.