Skip to main content

Wearable finger pad deformation sensor for tactile textures in frequency domain by using accelerometer on finger side

Abstract

In this study, we set out to develop a method for estimating the fine and fast shear deformation of a finger pad, that is, the palm side of a fingertip, as it scans the surface of a material. Using a miniature accelerometer, we measured the acceleration at the radial skin, the deformation of which is accompanied by the shear deformation of the finger pad. Using a transfer function, as specified in a separate experiment, between the pad and side of a finger, we estimated the shear deformation of the finger pad in the frequency domain. A comparison between an estimate based on the accelerometer and another based on a precise force sensor for the tangential component of the interaction force between the fingertip and material surfaces showed that the estimation accuracy was sufficient for frequencies in excess of approximately 20–50 Hz and for skin deformation above \(10^{-6}\) m. Our technique merely requires that an accelerometer be attached to the side of the fingertip, which allows active texture exploration. These estimates or measurements of the finger skin deformation caused by touching materials will help us to comprehend the relationships between material surfaces and the resulting texture sensations.

Introduction

In general, the physical properties of material surfaces characterize their tactile textures. From this standpoint, many research groups have investigated the relationships between different types of material properties and the textures that are perceived by touching them. Such an approach has significantly contributed to industries involved with clothing [1]. In contrast, texture perception arises as a result of the skin deformation caused by the interaction between the finger and material. Hence, the mechanical properties of the fingertips of individuals influence the textures perceived by those individuals. A typical example is the moisture content of a fingertip influencing the friction-induced pleasantness of the surface of a material [2]. Therefore, many researchers have attempted to link the information related to the deformation of a fingertip and the perceived texture when a finger touches a material. However, the measurement of a finger pad’s deformation is not easy because the finger pad is in contact with the material surface and is not exposed to any sensing devices. In this study, we developed a technique for estimating the fast and fine lateral deformations of a finger pad when a finger actively explores the surface of a material. To the best of our knowledge, such techniques have not previously been reported.

Many research groups have measured information related to the deformation of a fingertip by using a range of approaches. Nonetheless, observation through a transparent material such as glass is a very distinctive way of directly measuring deformation [3, 4]. In principle, this method is applicable only to transparent material. Another approach involves using the differential outputs of two accelerometers [5] to measure the macroscopic deformation of a finger pad. One accelerometer is attached to the dorsal side of the fingertip and another is fixed to the surface of the material contacted by the fingertip. Provided there is no slippage, the difference between the values measured with these two accelerometers will be the acceleration of the deformation of the finger pad. However, this technique cannot be applied to a finger sliding on the surface of a material.

To date, several indirect measurement methods have been employed to overcome the difficulties and limitations of direct measurement. For example, a number of researchers have exploited the propagation of skin deformations. Any high-frequency deformation at a point on the skin is transferred to the adjacent elastic skin tissues. Bensmaïa et al. attempted to measure these high frequency components as a finger was sliding over the surface of a material by using a Hall-effect transducer attached to the finger [6]. Similarly, the propagated vibratory information was acquired using a laser Doppler velocity meter [7, 8], microphones [9], accelerometers [10], and PVDF films [11] attached to the finger close to the tip, as well as a microphone [12] on the forearm. Although these vibrations in the adjacent skin originate from the vibratory deformations of the finger pad, these methods were not to measure the deformations of the individual finger pad.

Many researchers measured the interaction force between the fingertip and material surfaces as reliable information related to the finger pad’s deformation because the forces cause the skin to deform [13,14,15,16,17,18,19]. For example, Wiertlewski et al. [13] precisely measured the interaction forces when a material slid beneath a fingertip. However, they all measured interaction forces that were distinct from the skin deformations.

Given the abovementioned background and related studies, this study set out to develop a technique for estimating the lateral deformation of a finger pad and which could be applied to situations involving active touching. We used the propagation of skin deformation, whereby the lateral deformation of a finger pad leads to other deformation of the skin at the radial side of the fingertip. The skin deformation at the radial side can be measured by an accelerometer designed for the measurement of small and fast displacements. This study is based on our previous publications [20, 21] in which we introduced the principles of measurement and the linear causality between the deformations of the pad and the radial side of a finger. In this paper, we also assess the accuracy of our method by means of a comparison with a method based on force sensing after specifying the skin impedance of individuals, as well as describing the principles used for the estimation. Nakatani et al. [22] employed a similar approach in which they took advantage of the Poisson effect at a fingertip. They measured the deformation at the side of a fingertip by using strain gauges to estimate the normal load applied to the finger pad. In principle, however, their method was not designed to be able to handle rapid changes in the load.

In the present study, we were interested in the intensity or energy of the skin deformation for each frequency band. In terms of texture and vibrotactile perception, the phases of the skin vibrations were less influential, and the power or intensity of the spectral density in the frequency domain are considered important [6, 8, 23,24,25]. Some researchers have demonstrated that virtual textures or surfaces can be presented even without a representation of the phase information of the vibratory signals [26,27,28]. Furthermore, our method was used to estimate the macroscopic deformation of the finger pad. Actual skin deformations in the contact area between the finger pad and material surface are not spatially unique and include phase differences. However, our method does not rely on estimating the spatial distribution of the deformations in the finger pad. Therefore, the phase information acquired by our method may be weakly related to texture perception. Furthermore, the miniature accelerometers used for our method are not responsible for the low frequency signals, which means that our method cannot acquire those signals in the time and frequency domains. Hence, in this study, we did not set out to address the phases or time series data of skin deformations. Rather, we focused on the magnitudes of the skin deformations within a limited frequency range.

Principles used to estimate finger pad deformations

Here, we describe the principles on which our estimation is based. First, we introduce our proposal for estimating a finger pad’s deformation by using the propagation of the skin deformation. Second, for comparison, we describe a method for estimating a finger pad’s deformation based on the tangential component of the interaction force between the finger pad and material.

Estimation via acceleration measured at the side of the finger

Fig. 1
figure 1

Propagation of finger skin deformation. The lateral deformation of finger pad along X-axis propagates to the radial side

As shown in Fig. 1, the lateral deformation of the finger pad propagates to the radial side through the elastic body. Matsuura et al. [20] recently confirmed that the deformations at the pad and side of a finger exhibit significant causality that can be repeatedly observed in shaking tests. Hence, the deformation at a finger pad can be linked to that at the side of the finger through the following procedures.

As a calibration procedure, we simultaneously measure the acceleration at the finger pad \(\ddot{x}_p(t)\) and side \(\ddot{x}_s(t)\) through a shaking test. The transfer function \(G_1(s)\) between these two types of acceleration is then determined by

$$\begin{aligned} G_1(s) &= \dfrac{\mathcal {L}[\ddot{x}_s(t)] A(s)}{\mathcal {L}[\ddot{x}_p(t)] A(s)} \nonumber \\ &= \dfrac{X_s(s)}{X_p(s)} \end{aligned}$$
(1)

where \(\mathcal {L}\) and A(s) are the Laplace transformation and gain characteristics of the accelerometer, respectively. As long as we use the same accelerometers for the finger pad and radial side, their gain characteristics are easily canceled. In the frequency range where \(A(s) = 1\) (0 dB), we can make use of this transfer function to estimate \(X_p(s)\). The acceleration at the finger side \(\ddot{x}_{s1}(t)\) is measured, and then the lateral deformation of the finger pad in the frequency domain can be estimated by

$$\begin{aligned} \bar{X}_{p1}(s) = \dfrac{1}{s^2} \dfrac{\mathcal {L}[\ddot{x}_{s1}(t)]}{G_1(s)}. \end{aligned}$$
(2)

In our method, the acceleration of the hand movement is also measured by the accelerometer, which may interfere with the estimates of the finger pad’s deformation. However, the frequency of human hand motions is usually a few Hertz at most, which the miniature accelerometers used in the present study cannot detect, so this range is excluded from our method. As described later, the accelerometers that we used can respond to frequencies in excess of 20 Hz.

This method functions on the assumption that the transfer function of the fingertip does not vary during the measurement. It is known that the mechanical impedance of a finger pad varies depending on the normal load [29], whereby the stiffness and viscosity of the finger pad increase by approximately 30% as the load increases from 1 to 3 N. In our experiments, described below, each participant controlled the load applied with his finger to maintain a value of 1 N, by referring to an electronic scale. The actual load may have slightly differed from this target value for a brief instant, but we nevertheless assumed that, for the greater part of the measurement, the load was fairly controlled and the change in the mechanical impedance of the skin was minor, which allows us to statistically discuss the amplitudes of skin deformation in the frequency domain.

Furthermore, we assumed that the shear deformation of the finger pad is the unique factor that determines the skin deformation at the radial side. This indicates that only the tangential translation of the finger motion is considered, and the rotation around and translation along the longitudinal axis of the finger should not interfere with the skin deformation at the radial side. In the experiments, only tangential finger motions were allowed. Furthermore, the participants attempted to maintain the posture of their finger. Nonetheless, the frequency of the rotational motion of the finger is smaller than the active range of the miniature accelerometer that we used.

The abovementioned control of the finger load and motions does not allow the wearers of the sensor to conduct fully active and free touching; however, these control actions are acceptable for research or investigation purposes under laboratory conditions.

Estimation based on tangential force applied to finger pad

Fig. 2
figure 2

Spring-mass-damper model of the tangential deformation of a fingertip

Some earlier studies successfully expressed the deformation of the skin of a finger pad as caused by the tangential force on the finger pad by using a spring-mass-damper model [29,30,31], indicating a linear relationship between the deformation and force. This assumption holds provided that the skin deformation is not large and the normal load on the finger is constant. Based on this model, the finger pad’s deformation can be estimated from the tangential force.

When the finger pad is expressed by the one-degree-of-freedom model shown in Fig. 2 and the displacement of the representative mass point is defined by \(x_p(t)\), the equation of motion is given by

$$\begin{aligned} f(t) = m_p \ddot{x}_p(t) + c \dot{x}_p(t) + k x_p(t) \end{aligned}$$
(3)

where f(t), \(m_p\), c, and k are the tangential force on the finger pad, the mass, viscosity, and spring coefficient of the fingertip, respectively. Hence, the transfer function between f(t) and \(x_p(t)\) is

$$\begin{aligned} G_2(s) = \dfrac{X_p(s)}{F(s)} = \dfrac{1}{m_p s^2 + cs + k}. \end{aligned}$$
(4)

When the tangential force \(f_2(t)\) is measured, the finger pad’s deformation can be estimated by

$$\begin{aligned} \bar{X}_{p2}(s) = G_2(s) F_2(s) \end{aligned}$$
(5)

where \(F_2(s)\) corresponds to \(f_2(t)\) in the frequency domain. The mechanical parameters of the finger pad were experimentally identified for individual participants in the experiments.

To the best of our knowledge, the use of the tangential forces for estimating the finger pad’s deformation as a result of touching material surfaces has not been used previously; nevertheless, experienced researchers would likely have devised a similar setup.

Experiment 1: Transfer function for fingertip accelerations

Here, we describe how we experimentally estimated the transfer function between the accelerations of the pad and the side of the finger.

Apparatus

The main components of the experimental apparatus shown in Fig. 3 are two piezoelectric accelerometers [2302B, Showa Sokki Co. Ltd., Japan (one degree-of-freedom, 1.3 g), valid in 20–1000 Hz] and a vibration generator.

One of the accelerometers was fixed to the side of the finger by using adhesive tape. This attachment of the accelerometer does not alter either the stiffness or the viscosity of the finger pad, and does not lead to any subjective influences on the textures experienced by wearers. Another was mounted on the contactor on which the finger pad was placed, to measure the acceleration of the finger pad’s deformation. The displacement of the vibrating contactor was small, being a maximum of ±2.5 mm, for which the finger pad does not slip on the contactor. Hence, the displacement of the contactor is in good agreement with that of the finger pad. The sensitivity of the accelerometers was set along the X-axis. The outputs from both accelerometers were recorded at 8 kHz.

The vibration generator, which was based on a voice coil motor (X-1741, Neomax Engineering Co. Ltd., Japan), was used to deform the fingertip at various frequencies. The finger was fixed using a cradle at the proximal interphalangeal (PIP) joint. The accelerations of the applied vibration were set to approximately ±10 m/s2 at frequencies in excess of 10 Hz. The accelerations were smaller than this value at frequencies below 10 Hz.

Fig. 3
figure 3

Apparatus used for estimating the transfer function between the finger pad and radial side. Bottom left Side view. Bottom right Front view. The sensitivity of the accelerometers and the vibration applied to the finger pad was along X-axis

Procedure

Three male volunteers in their 20s participated in the experiments after providing informed consent. Their finger pads were wiped using cleaning paper before the measurement. Swept sinusoidal vibrations of 1–500 Hz were used. Twenty sweeps, including ten rising and falling series, were applied to the index finger of each participant. While the responses of the fingertip were recorded, each participant tried to keep his finger on the cradle while maintaining a load of 1 N. The participant could monitor the load by viewing the display of an electronic scale.

Analysis

We computed the Fourier transform of the outputs from the two accelerometers, \(\ddot{X}_s(s)\) and \(\ddot{X}_p(s)\), to acquire the accelerations of the skin deformations in the frequency domain. Following (1), we then computed the transfer function \(G_1(s)\) of the fingertip. Note that this transfer function includes not only the properties of the fingertip but also those of the accelerometer attached to the finger side. Furthermore, the coherency between the input \(\ddot{X}_s(s)\) and output \(\ddot{X}_p(s)\) of the transfer function was computed to check the linearity between the input and output. We applied these computation procedures for the individual participants.

Experiment 2: Estimation of mechanical parameters of fingertip

As previously mentioned, to estimate the finger pad’s lateral deformation based on the measured lateral force and (5), we experimentally determined the mechanical parameters in \(G_2(s)\). The parameters determined here were used for the analysis undertaken as part of Experiment 3.

Apparatus

Fig. 4
figure 4

Apparatus used for estimating the mechanical impedance parameters of fingertip. The index fingertip was placed on the contactor and its PIP joint was fixed by a cradle during the measurement

As shown in Fig. 4, we used a vibration generator with a high-precision piezoelectric force sensor [9217A, Kistler, Switzerland (minimum sensitivity 1 mN)] placed between the vibrator and finger contactor. The force sensor was operated with a charge amplifier (5015A, Kistler, Switzerland, with a response of up to 30 kHz) and measured the lateral component (X-direction) of the force applied to the finger pad. The participant’s fingertip was placed on the contactor while his PIP joint was fixed to a cradle in the same way as in Experiment 1. The deformation of the finger pad along the X-axis was measured by an accelerometer mounted on the contactor. The outputs from both sensors were recorded at 8 kHz.

Procedure

The same participants as those taking part in Experiment 1 also participated in this measurement. A series of 20 swept sinusoidal vibrations of up to 500 Hz were presented to each participant. The participants were also instructed to maintain a finger load of 1 N during the measurement.

Analysis

We analyzed the mechanical impedance of the fingertip of each participant. When the fingertip was modeled as a spring-mass-damper system, as shown in Fig. 2, the mechanical impedance of the fingertip plus the aluminum contactor is given by

$$\begin{aligned} Z(s)&= \dfrac{F(s)}{\dot{X}_p(s)} \nonumber \\ &= m_c s + c + \dfrac{k}{s}. \end{aligned}$$
(6)

Note that \(m_c\) is not the mass of the finger pad but the sum of the effective mass of the contactor and that of the mass point of the finger pad. Because the available data consisted of the accelerations of the finger pad, we converted this formula, as follows:

$$\begin{aligned} sZ(s) &= s^2\dfrac{F(s)}{\ddot{X}_p(s)} \nonumber \\ &= m_c s^2 + cs + k. \end{aligned}$$
(7)

By substituting \(j\omega \) for s, this formula can be expressed as

$$\begin{aligned} j\omega Z(j\omega )= (k-m_c\omega ^2) + j\omega c \end{aligned}$$
(8)

where \(\omega \) is the angular velocity. This equation indicates that its real part is characterized by the stiffness and mass of the system. Furthermore, the imaginary part is characterized by the viscosity. We computed the real and imaginary parts of this equation by using the measured \(\ddot{x}_p(t)\) and f(t) samples, and then fitted the samples into the equations for the real and imaginary parts by using the least-squares method to estimate the \(m_c\), k, and c values for individual participants.

Experiment 3: Comparison of acceleration- and force-based estimates of lateral deformation

To compare the estimates of the finger pad’s deformation as determined by the two methods, we simultaneously measured the acceleration at the finger side and the tangential force while the participant scanned the surface of a material.

Apparatus

Fig. 5
figure 5

Simultaneous measurement of the acceleration at the finger side and the tangential component of the interaction force

As shown in Fig. 5, a fingertip to which the accelerometer was attached at the radial side was used to scan the surface of a textured material. The material was placed on an aluminum plate to which the force sensor [9217A, Kistler, Switzerland (nominal force threshold: 1 mN, resonance frequency: greater than 20 kHz)] same as in Experiment 2 was installed to measure the tangential component of the interaction force. The plate was supported by a pair of leaf springs so that the normal load would not interfere with the force sensor.

Four types of materials were used for the measurement. They were a piece of cotton cloth, a flat and finely polished aluminum plate, and two types of grating scale boards made of ABS plastic. The grating scales have alternative ridges and grooves on their surfaces. The width of each ridge was 1 mm, with the ridges repeatedly arranged at intervals of 1 mm (fine) or 9 mm (coarse). The height of the ridges was also 1 mm in both cases.

Procedure

Each participant participated in this experiment after the transfer function \(G_1(s)\) for his fingertip had been specified in Experiment 1. Each participant scanned each type of material along the X-axis, having been instructed to rhythmically scan a length of 5 cm over 1 s, i.e., 5 cm/s, for 10 s. In the same way as in the previous experiments, the finger load was also maintained at approximately 1 N.

Analysis

For each participant, the measured tangential force \(f_2(t)\) was converted into the frequency domain by a Fourier transform and, according to (5), for each combination of participants and materials, we computed \(\bar{X}_{p2}(s)\), that is, the finger pad’s deformation in the frequency domain. For this computation, we used the k and c values estimated in Experiment 2 for the individual participants. Because it was difficult to estimate \(m_p\) with our experimental setup, for the analysis in Experiment 3, we used a representative value taken from the literature [31]: \(m_p = 0.1 \times 10^{-3}\) kg. This value is potentially determined by the weight of local epidermal tissue of finger pad [31], and it is unlikely to be affected by the mass of an accelerometer at the finger side.

Similarly, from the accelerations at the finger side, \(\ddot{x}_{s1}(t)\), following (2), we computed \(\bar{X}_{p1}(s)\). We used \(G_1(s)\) that was estimated for each participant in Experiment 1 to compute \(\bar{X}_{p1}(s)\) for each participant.

Furthermore, to discuss the accuracy of the estimated finger pad’s deformation, we computed the difference in the magnitudes of these two types of estimates: \(|\bar{X}_{p1}(s)|\) and \(|\bar{X}_{p2}(s)|\).

Results

Gain of transfer function for fingertip (Experiment 1)

Fig. 6
figure 6

Gain (top) and coherence (middle) of the smoothed transfer function between the accelerations of the finger pad and side for one participant. Bottom figure compares the gains of the transfer function for three participants

The top part of Fig. 6 shows an example of the gain of the transfer function between the acceleration of the pad and the side of the finger for one participant. As previously mentioned, because we are interested in the magnitudes of the skin deformations, the figure shows the magnitude of \(G_1(s)\). The first and second peaks were observed at around 70 and 300 Hz, respectively. Typically, in terms of the lateral skin deformation of the finger pad, a one degree-of-freedom system can be applied, as previously mentioned. In our setup, however, the mass of the accelerometer at the side of the finger constitutes the system with two degrees of freedom. As a result, the magnitude of the transfer function showed a few typical features of a system consisting of two resonances and an anti-resonance between them.

Figure 6 also shows the magnitude of the transfer function for each of the three participants. They were largely flat up to 80–120 Hz, and then moderately increased followed by a cave for which the bottom peak was between 120–150 Hz. We suggest that the major causes of the individualities originate from the anatomical individualities of the participants’ fingertips [32] as well as the individualities of the mechanical impedance of their fingertips. Especially, the nail profile and thickness of the fingertip fatty tissue may influence the transfer function between the skin deformation at the pad and radial side of the finger. Given the main goal of the present study, this was not pursued any further.

For all the participants, the coherence of the transfer function was sufficiently high, being greater than 0.8, within a range of 10–450 Hz, indicating that the estimation based on the transfer function is valid within this frequency range.

Mechanical parameters of fingertip (Experiment 2)

Fig. 7
figure 7

Example of the real and imaginary parts of (8), the impedance of the mechanical parameters of the finger tip. Dotted curves and solid lines are measured samples and linear approximations, respectively

Table 1 Mechanical parameters of finger pad estimated in Experiment 2

Figure 7 shows examples of the real and imaginary parts of (8). From the gradient and intercept of a line into which the measured samples were fitted, the \(m_c\) and k values were estimated. Furthermore, c was estimated from the gradient of the approximated line of the imaginary part. Table 1 lists the values estimated for individual participants. The mean estimated \(m_c\) value was 0.029 kg for all the participants. As previously described, \(m_c\) was mainly the effective mass of the contactor and not that of the finger pad. The k and c values estimated here are close to those reported in other studies in which the mechanical parameters involved in the lateral deformation of a finger pad were investigated. Nakazawa et al. [29] reported that \(k = 310\) N/m and \(c = 2.1\) Ns/m while the finger load was 1 N. Wiertlewski and Hayward [31] reported that these values were within a range of 600–1700 N/m and 0.8–2.4 Ns/m for a 0.5 N load. The values estimated in our study are largely consistent with these previously reported values.

Comparison between the acceleration- and force-based estimates of finger pad deformations (Experiment 3)

Examples of estimated finger pad deformation

Fig. 8
figure 8

Finger pad’s lateral deformation estimated by the accelerations at the finger side. Examples of one participant

Figure 8 shows the estimated finger pad deformation \(\bar{X}_{p1}(s)\) for each type of material when scanned by any one participant. The skin deformation includes the wide range of frequency components as observed in another study [13]. The deformation for the grating scales, which have a surface roughness that is larger than both the cloth and aluminum, is greater than those for the cloth and aluminum. Naturally, the greater surface roughness led to the greater skin deformation when the exploratory motion of the participant was controlled to be approximately the same across the materials. The profiles of the frequency responses differ depending on the materials, although we do not thoroughly discuss the differences because the active hand movements of the participants were not precisely controlled. Note that Fig. 8 shows the results for computational estimation, for which the accuracy is discussed in the latter part of this manuscript.

Because the grating scales have a periodic surface texture, the skin vibration might have also been periodic to some extent. The estimated lateral skin deformation for the coarse grating scale, for which the spatial surface period was 10 mm, exhibited a small peak at around 14–20 Hz. Since the participant was instructed to scan the material over a length of 5 cm in 1 s, the skin vibration was predicted to concentrate up to 8–10 Hz if the hand velocity was well controlled and followed a sinusoidal or bell-shaped curve. Unfortunately, these frequencies are out of the range of sensitivity of the accelerometer. However, the expected and observed peak frequencies did not substantially disagree with each other, taking account of the moderate control of hand exploration in our study. Similarly, the skin deformation for the fine grating scale, for which the surface period was 2 mm, was predicted to concentrate up to 40–50 Hz. The estimated skin deformation for the fine grating scale was mostly greater than those for the cloth and aluminum plates below this frequency range, which is consistent with the expectations.

Comparison between the acceleration- and force-based estimates

Fig. 9
figure 9

Example of the time series data of the accelerations at the finger side \(\ddot{x}_{s1}(t)\) and tangential component of the interaction force \(f_2(t)\) when the finger slid on material surfaces

Figure 9 shows examples of the time-series data of the accelerations at the finger side and the tangential forces. Apparently, the fine material, which is cloth in the figure, exhibited smaller variations in the accelerations and forces than the rough material, which is the grating scale in the figure. Naturally, the skin accelerations and interaction forces depend on the materials, is as shown in these time-series data.

Figure 10 shows examples of the magnitudes of the finger pad deformations estimated using (2) and (5) for each material. As previously stated, because the linearity of the transfer function \(G_1(s)\) was valid for 10–450 Hz, these estimates are shown for this range. Smaller interaction forces [14] and skin deformations [8] at higher frequency bands are typical responses of the finger when sliding on a material. For any types of material, the two types of estimates appear to be in reasonably good agreement with each other. We should note that the human perceptual threshold to vibrotactile stimulation [33, 34] is approximately 10−7 m at the most sensitive frequency. For shear skin displacements, the threshold may be even smaller by a few tens of a percentage point [35]. Furthermore, taking the smallest resolution of the force sensor into account, an estimated skin deformation of less than 10−7 m may be practically insignificant.

As none of the specific materials involved differences that were notably larger than those of the others, we computed the average differences between the two types of estimates across all the types of materials and participants, as shown in Fig. 11. The estimated differences for the representative frequencies are also listed in Table 2. Note that we do not know which type of estimate is more accurate, although the differences between them become a criterion whereby we can judge whether the estimates are reliable. If the difference is significant for some frequency bands, then we cannot rely on the two types of estimates. As a criterion for judging the accuracy, we adopted the differential thresholds of human perception for the vibrotactile stimuli. The differential threshold is the minimum difference between two physical stimuli that humans can distinguish. If the differences between the two types of estimates are smaller than the differential thresholds, then such differences are not significant to the perception and can be ignored. Because the differential thresholds towards the amplitudes of the vibrotactile stimuli are typically within a range of 10–20% of the stimuli [36, 37], we set 10–20% of the absolute thresholds, as presented by Gescheider et al. [34] as the criteria for judging a significant difference. As shown in Fig. 11 and Table 2, the differences between the two types of estimates fall below the differential thresholds of 20 and 10% at frequencies in excess of 23 and 50 Hz, respectively.

Fig. 10
figure 10

Two types of estimates. Solid curves are the finger pad’s deformation estimated based on the accelerations at the finger side. Dotted curves are those estimated based on the tangential components of the interaction forces

Fig. 11
figure 11

Perceptual differential thresholds towards the amplitudes of vibrotactile stimuli and average difference between the two types of estimates: \(|\bar{X}_\mathrm{p1}(\omega )|\) and \(|\bar{X}_\mathrm{p2}(\omega )|\). Differential thresholds are drawn based on literature [34]

Table 2 Differences in displacements estimated by the two types of methods at representative frequencies. DT indicates the differential threshold of human vibrotactile perception

Discussion

Using the propagation of skin deformation at a fingertip, we developed a technique for sensing the shear deformation of a finger pad. Here, we discuss the estimation accuracy and limitations of our method based on the experimental results.

Accuracy of the estimates

A comparison, in Experiment 3, of the two types of estimate of a finger pad’s deformation indicated that our method, based on the accelerations at the finger side, can be applied to a deformation in excess of 23–50 Hz. In contrast, skin deformation at less than a few tens of Hertz, at which our method may incur significant errors, is regarded as being less effective for the perception of fine textures [6, 24]; nonetheless, they still influence the perception of coarse surfaces. At the high-frequency range, in which the deformation is smaller than \(10^{-7}\) m, because of the limitations of the sensitivity of the force sensor and the noise level of the accelerometer, the accuracy of our method has not been validated; nonetheless, such a small deformation is unlikely to influence the perception of texture as previously mentioned.

Because of the reasons given below, it is assumed that the estimation based on the force sensor and that based on the accelerometer are, respectively, superior to the other method for a frequency range smaller than a few tens of Hertz and that greater than 100 Hz.

In the low-frequency range (less than a few tens of Hertz), the normal and tangential deformation of the finger pad are not decoupled and collectively affect the deformation at the finger side. The normal deformation of the finger pad also leads to skin deformation at the finger side. Hence, a large normal deformation in the low-frequency range may interfere with the estimates of the tangential deformation of the finger pad. At high frequencies, the adverse effects of the normal deformation on the estimates of tangential deformation are minute, because this effect of normal deformation on the radial side is attenuated [22]. From this aspect, at a frequency range of less than a few tens of Hertz, the estimation method based on the tangential force is regarded as being better than our accelerometer-based method, although further comparisons would be necessary to draw a firm conclusion.

In the high-frequency range above 100 Hz, the interaction forces between the finger pad and material are small and hardly detectable even when using one of the most sensitive commercially available rigid force sensors. As shown in Fig. 10, in our experiments, estimates based on the force sensor were found to change linearly with a gradient of −40 dB/dec, indicating that the outputs from the sensor were mostly noise of a constant level. There were no profound signals. On the other hand, because the acceleration is proportional to the square of the frequency, the accelerometer is good at detecting high-frequency signals, indicating that estimations based on the accelerometer are suitable at a high-frequency range. Indeed, the skin deformation estimated by the accelerometer reflected the different surface properties of materials up to 450 Hz, as shown in Figs. 8 and 10.

Effects of whole finger vibration

We did not restrain the finger joints during the measurement in experiment 3. Hence, the finger could rotate in ab/adduction at the metacarpophalangeal joint. Ideally, such rotation should be removed throughout the experiments. One may say that the effect of the finger ab/adduction is minor. According to a study [38] in which the mechanical impedance of the index fingers in abduction was measured, the resonance point of the ab/adduction motion is a few tens of Hertz, above which the rotational vibration is reduced. This also indicates that the whole finger vibration may be magnified at low frequencies and adversely affects the estimation based on the accelerometer at the finger side. Indeed, the finger pad deformations estimated by the accelerometer were greater than those resulting from the shear force at low frequency ranges, as shown in Fig. 10. Bracing the MP joint may lead to the two types of estimates being more congruent.

Limitations on usage

In this section, we discuss the limitations of the method we developed, and present cautions on its use.

First, our method functions at a constant finger load in the normal direction. Therefore, the method should not be used when the finger load varies dynamically while exploring a surface. This is because the profile of the transfer function of the fingertip depends on the load. Hence, when our method is used, the finger load has to be controlled in some way: e.g., instruction followed by practice. To compensate for this dependence on the finger load, a calibration method involving variations in the finger load is necessary, which will be the subject of a future study. In addition, it should be mentioned that most of the previously introduced indirect measurement methods did not consider the changes in the finger load.

Another caution regarding the usage is the effect of rotation or the gradient of the fingertip while the participants tried to maintain the posture of their finger during the exploration in our experiments. The acceleration at the finger side is also affected by gravity, which should not be confused with that of the skin deformation. Although we used only one accelerometer, the simultaneous use of a 3-axial accelerometer on the nail would be able to compensate for any adverse effects of the finger rotation.

Finally, calibration for an individual fingertip should be performed before measurement by using our method. The transfer functions vary considerably depending on the individual. We believe that such individuality originates from the profiles of fingertips and nails as well as the mechanical characteristics of the finger pad. Hence, the transfer function for one person cannot be applied to another. Moreover, as previously mentioned, because the function depends on the attachment of the accelerometer, calibration should be performed before each use.

Conclusion

We developed a technique for estimating the shear deformation of a finger pad, which is applicable even when actively touching a material. This technique requires only an accelerometer attached to the radial side of the fingertip, and the finger pad is not covered with any materials or devices that disturb the natural touch. The propagation of skin deformation on which our method is based has been used by some research groups in the past; however, they simply measured skin vibrations or deformations near the fingertip immediately after simple signal processing, and did not refer to the actual skin deformation of the individual. Therefore, our method is a unique general technique for measuring the shear deformation of a finger pad when actively touching the surface of a material.

As a result of comparing our accelerometer-based method with another method based on the tangential component of the interaction force between the fingertip and material, for a range in excess of higher than approximately 20–50 Hz and \(10^{-6}\) m, the estimates obtained with both methods are found to be in good agreement, with the differences being smaller than the human discrimination thresholds. This indicates that our method is effective for the analysis of the perception of fine roughness surfaces with which high-frequency skin deformation is mediated.

For practical purposes, it is preferable to solve a few related problems, such as those related to the variation in the finger load and further tests of the estimation accuracy for deformation of sub-micro meter. Nonetheless, this technique is expected to impact researchers in the field of haptics, because skin deformation is the direct source of tactile sensations and thus far, many research groups have attempted to link this information on skin deformation with human perceptions.

References

  1. Kawabata S, Niwa M (1989) Fabric performance in clothing and clothing manufacture. J Textile Inst 80(1):19–50

    Article  Google Scholar 

  2. Klöcker A, Arnould C, Penta M, Thonnard JL (2012) Rasch-built measure of pleasant touch through active fingertip explorations. Front Neurorobotics. doi:10.3389/fnbot.2012.00005

    Google Scholar 

  3. Levesque V, Hayward V (2003) Experimental evidence of lateral skin strain during tactile exploration. In: Proceedings of EuroHaptics, pp 261–275

  4. Tada M, Kanade T (2004) An imaging system of incipient slip for modelling how human perceives slip of a fingertip. Proc IEEE Int Conf Eng Med Biol Soc 1:2045–2048

    Google Scholar 

  5. Okamoto S, Wiertlewski M, Hayward V (2013) Anticipatory vibrotactile cueing facilitates grip force adjustment. In: Proceedings of IEEE World Haptics Conference, pp 525–530

  6. Bensmaïa SJ, Hollins M (2005) Pacinian representations of fine surface texture. Percept Psychophys 67(5):842–854

    Article  Google Scholar 

  7. Weber AI, Saal HP, Lieber JD, Cheng J-W, Manfredi LR, Dammann JF, Bensmaïa SJ (2013) Spatial and temporal codes mediate the tactile perception of natural textures. Proc Natl Acad Sci 110(42):17107–17112

    Article  Google Scholar 

  8. Manfredi LR, Saal HP, Brown KJ, Zielinski MC, Dammann JF, Polashock VS, Bensmaïa SJ (2014) Natural scenes in tactile texture. J Neurophysiol 111(9):1792–1802

    Article  Google Scholar 

  9. Tanaka Y, Horita Y, Sano A, Fujimoto H (2011) Tactile sensing utilizing human tactile perception. In: Proceedings of IEEE World Haptics Conference, pp 621– 626

  10. Tanaka Y, Horita Y, Sano A (2012) Finger-mounted skin vibration sensor for active touch. In: Isokoski P, Springare J (eds) Haptics: perception, devices, mobility, and communication, vol 7283. Lecture Notes in Computer Science. Springer, Berlin, pp 169–174

    Chapter  Google Scholar 

  11. Tanaka Y, Nguyen DP, Fukuda T, Sano A (2015) Wearable skin vibration sensor using a PVDF film. Proceedings of IEEE World Haptics Conference, pp 146–151

  12. Delhaye B, Hayward V, Lefèvre P, Thonnard JL (2012) Texture-induced vibrations in the forearm during tactile exploration. Front Behav Neurosci 6:37

    Article  Google Scholar 

  13. Wiertlewski M, Lozada J, Hayward V (2011) The spatial spectrum of tangential skin displacementcan encode tactual texture. IEEE Trans Robot 27(3):461–472

    Article  Google Scholar 

  14. Wiertlewski M, Hudin C, Hayward V (2011) On the 1/f noise and non-integer harmonic decay of the interaction of a finger sliding on flat and sinusoidal surfaces. In: Proceedings of the IEEE World Haptics Conference, pp 25–30

  15. Nonomura Y, Fujii T, Arashi Y, Miura T, Maeno T, Tashiro K, Kamikawa Y, Monchi R (2009) Tactile impression and friction of water on human skin. Colloids Surf B Biointerfaces 69:264–267

    Article  Google Scholar 

  16. Smith AM, Chapman CE, Deslandes M, Langlais JS, Thibodeau MP (2002) Role of friction and tangential force variation in the subjective scaling of tactile roughness. Exp Brain Res 144(2):211–223

    Article  Google Scholar 

  17. Platkiewicz J, Mansutti A, Bordegoni M, Hayward V (2014) Recording device for natural haptic textures felt with the bare fingertip. In: Auvray M, Duriez C (eds) Haptics: neuroscience, devices, modeling, and applications, vol 8618. Lecture Notes in Computer Science. Springer, Berlin, pp 521–528

    Google Scholar 

  18. Nagano H, Okamoto S, Yamada Y (2014) Haptic invitation of textures: perceptually prominent properties of materials determine human touch motions. IEEE Trans Haptics 7(3):345–355

    Article  Google Scholar 

  19. Tanaka Y, Bergmann Tiest WM, Kappers AML, Sano A (2014) Contact force and scanning velocity during active roughness perception. PLoS ONE 9:e93363. doi:10.1371/journal.pone.0093363

    Article  Google Scholar 

  20. Matsuura Y, Okamoto S, Yamada Y (2014) Estimation of finger pad deformation based on skin deformation transferred to the radial side. In: Auvray M, Duriez C (eds) Haptics: neuroscience, devices, modeling, and applications, vol 8619. Lecture Notes in Computer Science. Springer, Berlin, pp 313–319

    Google Scholar 

  21. Sato S, Okamoto S, Yamada Y (2015) Wearable finger pad sensor for tactile textures using propagated deformation at finger side: assessment of accuracy. In: Proceedings of IEEE international conference on systems, man, and cybernetics, pp 892–896

  22. Nakatani M, Shiojima K, Kinoshita S, Kawasoe T, Koketsu K, Wada J (2011) Wearable contact force sensor system based on fingerpad deformation. In: Proceedings of IEEE World Haptics Conference, pp 323–328

  23. Bensmaïa SJ, Hollins M (2000) Complex tactile waveform discrimination. J Acoust Soc Am 108:1236–1245

    Article  Google Scholar 

  24. Bensmaïa SJ, Hollins M, Yau J (2005) Vibrotactile intensity and frequency information in the pacinian system: a psychophysical model. Percept Psychophys 67(5):828–841

    Article  Google Scholar 

  25. Okamoto S, Yamada Y (2011) An objective index that substitutes for subjective quality of vibrotactile material-like textures. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, pp 3060–3067

  26. Okamoto S, Yamada Y (2013) Lossy data compression of vibrotactile material-like textures. IEEE Trans Haptics 6(1):69–80

    Article  Google Scholar 

  27. Cholewiak SA, Kim K, Tan HZ, Adelstein BD (2010) A frequency-domain analysis of haptic gratings. IEEE Trans Haptics 3(1):3–14

    Article  Google Scholar 

  28. Culbertson H, Unwin J, Kuchenbecker KJ (2014) Modeling and rendering realistic textures from unconstrained tool-surface interactions. IEEE Trans Haptics 7(3):381–393

    Article  Google Scholar 

  29. Nakazawa N, Ikeura R, Inooka H (2000) Characteristics of human fingertips in the shearing direction. Biol Cybern 82(3):207–214

    Article  Google Scholar 

  30. Pataky TC, Latash ML, Zatsiorsky VM (2005) Viscoelastic response of the finger pad to incremental tangential displacements. J Biomech 38:1441–1449

    Article  Google Scholar 

  31. Wiertlewski M, Hayward V (2012) Mechanical behavior of the fingertip in the range of frequencies and displacements relevant to touch. J Biomech 45(11):1869–1874

    Article  Google Scholar 

  32. Nohara K, Tada M, Umeda K, Mochimaru M (2007) Synthesizing possible variations of finger structure using principal component analysis for non-rigid volume registration results. In: Proceeding of 3rd international symposium on measurement, analysis and modeling of human function, pp 247–253

  33. Verrillo RT, Gescheider GA (1983) Vibrotactile masking: effects of one- and two-site stimulation. Percept Psychophys 33(4):379–387

    Article  Google Scholar 

  34. Gescheider GA, Bolanowski SJ, Hardick K (2001) The frequency selectivity of information processing channels in the tactile sensory system. Somatosens Motor Res 18(3):191–201

    Article  Google Scholar 

  35. Biggs J, Srinivasan MA (2002) Tangential versus normal displacements of skin: relative effectiveness for producing tactile sensations. In: Proceedings of IEEE symposium on haptic interfaces for virtual environment and teleoperator systems, pp 121–128

  36. Craig JC (1972) Difference threshold for intensity of tactile stimuli. Percept Psychophys 11(2):150–152

    Article  Google Scholar 

  37. Gescheider G, Bolanowski S, Zwislocki J, Hall K, Mascia C (1994) The effects of masking on the growth of vibrotactile sensation magnitude and on the amplitude difference limen: a test of the equal sensation magnitude-equal difference limen hypothesis. J Acoust Soc Am 96(3):1479–1488

    Article  Google Scholar 

  38. Hajian AZ, Howe RD (1997) Identification of the mechanical impedance at the human fingertip. J Biomehcanical Eng 119:109–114

    Article  Google Scholar 

Download references

Authors' contributions

SS and SO are the main contributors of this study, including formulating the concepts and principles, conducting experiments and analyses, and editing the manuscript. YM and YY are credited with the principles of the methods. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

Not applicable.

Ethics approval and consent to participate

Voluntary participants provided written informed consent before experiments.

Funding

This study was partly supported by MEXT KAKENHI Shitsukan 25135717 and 15H05923.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shunsuke Sato.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sato, S., Okamoto, S., Matsuura, Y. et al. Wearable finger pad deformation sensor for tactile textures in frequency domain by using accelerometer on finger side. Robomech J 4, 19 (2017). https://doi.org/10.1186/s40648-017-0087-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40648-017-0087-1

Keywords