One-touch calibration of hum-noise-based touch sensor for unknown users utilizing models trained by different users

Hum‑noise‑based touch sensors (HumTouch) are capable of recognizing human touch on semiconductive materi‑ als using the current leaking from the finger to the surface. Thus far, calibration for these hum‑noise‑based touch sensors has been performed for individual users because of the individual differences in hum‑driven electric currents in human bodies. However, for applications designed for unknown users, time‑consuming calibration for individual users is not preferred, and a new user should be able to use the sensor immediately. For this purpose, we propose a new calibration method for HumTouch. In this method, learning datasets collected from multiple people and a few extra samples from a new user are collectively used to establish a touch localization estimator. The estimator is com‑ puted using the kernel regression method with weighted samples from the new user. For a 20 × 18 cm 2 paper, the mean localization error is reduced from 1.24 cm to 0.90 cm with only one sample from the new user. Hence, a new user can establish a semipersonalized localization estimator by touching only one point on the surface. This method improves the localization performance of HumTouch sensors in an easy‑to‑access manner.


Introduction
Capacitive sensing has been adopted in most commonly-used and commercially successful touch-sensitive methods. This sensing method detects the changes in the electric charges within a uniform electrostatic field [1][2][3], and is extensively used in touch panels and buttons. Recently, touch localization technology that can be applied to nonspecialized surfaces or objects has attracted the attention of researchers. Most of these techniques are aimed at furniture and cloth, rather than touch panels for electronic appliances. An example is the electrical tomography method [4][5][6][7], which uses several pairs of electrodes to apply voltages to a resistive surface.
The touched location can be computed by recording the changes in the electrical impedance between the electrodes. Optical methods such as cameras, which detect and trace human fingertips, can be applied to any surface as long as the occlusion problem is not severe [8][9][10][11][12]. In addition, soft or thin pressure-sensitive sheets have been used for constructing touch-sensitive areas on free-form surfaces [13][14][15][16][17]. Other approaches include radio-frequency identification tags for sensing touch events and localization [18], inertial measurement unit sensors for tracking fingertips and recognizing gestures [19], and methods utilizing the propagation of acoustic waves in solid materials [20]. Multi-axial force sensors can also turn objects such as furniture into touch sensitive interfaces [21][22][23]. Furthermore, a method based on spray coating conductive substances can be easily applied to objects with complex shapes for forming a touch-sensitive surface [6,7,24]. Moreover, wearable sensors, such as [25,26], are a potential approach to detect a touch on objects without sensory instruments.
We focus on a sensing method that utilizes environmental AC-hum-noise [27][28][29][30][31][32][33]. For example, Cohn et al. demonstrated that the touch positions on a wall could be discriminated by monitoring the voltage at an electrode on human skin [27]. In their experiment, the hum-noise in the wall was detected through human skin when a person touched the wall. The hum-noise-based sensing method employed in this study is called HumTouch [34][35][36][37], which uses the environmental noise leaking from human fingers onto the surfaces of conductive or semiconductive objects. Touch or finger localization is achieved based on the voltages recorded at multiple electrodes attached to the surface [35][36][37]. In previous studies on hum-noise-based touch sensors, calibration for localization was performed for individual users, using machine learning or statistical approaches comprising support vector machines and regression analysis [27][28][29][30][35][36][37]. Because of the individual differences in human bodies, the voltages recorded on material surfaces differ among users, leading to inaccurate localization of touch. For certain applications, calibration for individual users before using the sensors is acceptable or suitable; however, other applications require easy access to hum-noisebased touch sensors by unknown users with less effort for sensor calibration.
Herein, we propose a calibration method in which only one or a few samples from a new unknown user and many samples collected from various people are combined to build a semipersonalized calibration model. The extra samples from the new user are weighted to tailor the calibration model to the user. We examine a method that uses 20 × 18 cm 2 paper coated with conductive ink. We investigate the suitable weighting values and the extra samples required for improving the localization accuracy. To the best of our knowledge, a method similar to our semipersonalized method for touch sensors has not yet been developed. This is mainly because, in principle, most of the previous touch sensors do not require calibration for individuals because their measurement accuracy does not depend on individual users. For example, electrical tomography methods utilize the changes in the electrical impedance caused by the deformation of the surface touched by the user. Hence, calibration models are required for individual surfaces, but not for individual users.
Touch-sensing methods utilizing hum-noise have distinctive features from other sensing methods. First, this method does not require any sensing structures on the surface to be measured. It does not require any resistive or capacitive materials and layers installed on the surface as long as the surface material is semiconductive. Hence, HumTouch may allow us to turn wooden products [36], such as furniture, and those covered with paper [35,37] or cloth into touch-sensitive interfaces. Furthermore, HumTouch can localize a human body in water [38]. The second prominent feature is the passivity. Methods based on hum-noise do not excite object surfaces by applying voltage or vibration, for example; hence, they may be suitable for sensing large-area surfaces such as a wall and floor. Camera-based touch-sensing methods [8][9][10][11][12] and acoustic methods [20] also share these two features whereas the former suffers from occlusions and the latter is not applied for soft materials. Our calibration method for HumTouch is expected to realize touch-detection functions for a variety of objects described above in public and domestic spaces.

General principle of the HumTouch sensor
Electrical grids are an essential infrastructure in modern cities. AC power lines constantly generate electromagnetic waves known as hum-noise during the transmission of electricity, which react with conductive or semiconductive objects nearby. The human body includes conductive substances such as minerals, which react with these hum-noise-producing currents in the body, as shown in Fig. 1. Therefore, when a human touches a conductive object, current leaks from the finger onto the object. These currents can be detected using voltage acquisition devices attached to the surface of the object. HumTouch sensing technology uses these currents for touch localization on papers [35,37], woods [36], and in water [38] and gesture recognition [34].

Material and setting
We used a 20 × 18 cm 2 wiping paper (Kimtowel, Nippon Paper Crecia Co. Ltd., Japan) as the experimental Fig. 1 Principle of HumTouch. HumTouch utilizes the current leaking from a human body when conductive substances in the body react with the electromagnetic waves generated by AC power lines in buildings material. The paper was painted with a semiconductive ink [34], which contained 15 g of polyvinyl alcohol, 300 mL of ultrapure water, 75 mL of polyethylene glycol 400, and 37.5 mL of glutaraldehyde. After painting, the paper was dried at room temperature for a week. After these processes, the paper remained dry and flexible, as shown in Fig. 2.
Further, 7 × 7 points were marked on the paper at intervals of 2.5 cm and 2.25 cm. The ratio of these intervals follows the dimensions of the paper. An electrode was attached to the center of each side of the paper (total of four electrodes), following [37], as depicted in Fig. 2a. These electrodes were then connected to an oscilloscope (HS6DIFF, TiePie Engineering, Netherlands; sampling frequency: 500 kHz) to record the voltages on the surface.

Experimental procedure
Experiments were conducted in an office room. Six participants (male university students above 20 years), who provided written informed consent, were involved in the experiments. Each participant was asked to touch each of the 49 marked points on the paper with their right index finger for approximately 1 s. This procedure was repeated five times to collect five sample sets from the individual participants. The contact force was not restricted because the voltage rarely changed with the contact force [37]. In addition, to confirm that the participants were well grounded, they sat still with their two feet firmly on the ground. At least, one foot needed to be on the ground to record the voltages driven by hum noise. Further, they removed their shoes and remained in socks because some shoes deterred the data collection.

Data analysis
Localization methods are described in Section 3 that includes three methods based on kernel regression analysis. The first method, i.e., personalized model, establishes estimation models by using only the samples of a certain person for localizing his/her touch. The second method, i.e., general model, establishes the model by using the samples of several people for localizing the touch of another individual. The third method, i.e., semipersonalized model, uses a few samples from a certain participant and many samples from other participants to prepare a model for that participant. Three methods are based on the kernel regression analysis method, and comparison with other potential methods is not considered.

Voltage-data preprocessing
The hum-noise in our experimental setting was a 50 Hz voltage signal. To reduce the influence of noise, we applied a moving average filter of length 0.2 ms with cut-off frequency of 2.2 kHz. After the noise-removal process, we recorded the maximum value detected in each electrode during 1 s as v i,j,e , where i ∈ {1, ..., 245} , j ∈ {1, ..., 6} , and e ∈ {1, ..., 4} indicate the trial, participant, and electrode, respectively. The normalized maximum voltage z i,j,e is then computed as follows: where v i,j and σ i,j are the mean and standard deviation for v i,j,1 -v i,j,4 .

Kernel regression model for fully-personalized localization (Personalized model)
We applied kernel regression analysis, known as nonlinear multiple regression analysis, to find the relationship between the touched locations and the recorded voltages because the relationship between the two variables is nonlinear. As a personalized localization model [35,37], a regression model was constructed using data recorded from a single participant as the learning data. Note that the learning and test data were separated in a leave-oneout cross-validation manner, where four of the five sample sets were used to build the model and the remaining set was used as the test set.
The location of each marked point was defined using a two-dimensional Cartesian coordinate system. The estimated location (x,ŷ) for the vector of normalized voltages z ∈ (z 1 , ..., z 4 ) T is computed as follows: where n is the number of samples (four trials × 49 points = 196), and z i = (z i,j,1 , ..., z i,j,4 ) T are the normalized voltages for the learning samples of a single participant's i-th trial, and . is the L2 norm. Coefficients α x,i and α y,i are the i-th elements of coefficient vectors α x ∈ R n×1 and α y ∈ R n×1 , respectively. The least squares solution of the coefficients are computed as follows: where x = (x 1 , ..., x n ) T and y = (y 1 , ..., y n ) T are the actual x and y locations for the learning samples, respectively, and matrix K ∈ R n×n is given by With reference to our previous study [37], the regularization value = 0.001. I is an n × n unit matrix.

Kernel regression model built using data from various participants (General model)
As a general model for a certain participant, a regression model was constructed using the data recorded from the other five participants. Hence, the general model is the opposite of the personalized model, which only uses the data of a single target participant. The general model was computed for each of the six participants. The estimated location is computed as follows: where m is the number of samples (five participants × five trials × 49 points = 1470) and z s = (z i,j,1 , ..., z i,j,4 ) T are the normalized voltages of learning sample s ∈ {1, ..., m} .
Coefficients α x,s and α y,s are the s-th elements of coefficient vectors α x and α y , respectively. These coefficients are computed using (3) and (4), with K ∈ R m×m , x ∈ R m×1 , and y ∈ R m×1 .

Weighted kernel regression model for semipersonalized localization (Semipersonalized model) Addition of extra samples from the target user
To build a semipersonalized model for a certain participant, we added one or more samples recorded by the participant to the learning data set collected from the other participants. This expanded learning dataset for kernel regression analysis includes 1470 + l samples, where l is the number of extra samples from the target participant. l ranges from one-five ( l ∈ {1, ..., 5}).

Weighting of the extra samples from the target user
We expected that including extra samples from the target participant reduces the localization error. Weighted kernel regression analysis was performed by adjusting coefficients α x and α y as follows: Matrix W ∈ R (m+l)×(m+l) is given by where I is an n × n unit matrix and U = diag(w, w, ...) ∈ R l×l is the weight matrix, where w is the weight value. Weight value w = 1, 10, 100, or 1000.

Personalized model: touch localization with the model constructed using samples from one participant
We established localization models for the individuals. For each participant, four sample sets were used as the training dataset and one was used as the test set, applying leave-one-out cross-validation. Figure 4 shows the mean and standard deviation of the localization for each participant. Further, we computed the arithmetic mean of the absolute errors for each marked point and participant. The mean estimation error among the six participants was 0.60 cm. Participant P4 exhibits the least mean localization error of 0.44 cm, whereas P6 exhibits the highest mean error of 0.83 cm. These estimation errors are generally considered small because they are lesser than the mean width of index fingertips of Japanese males: 13.9 cm [39]. Note that the error values of the fully personalized models can be used as the benchmark scores for evaluating the other methods in this study.
The estimation errors of the semipersonalized model are expected to be close to these errors. Figure 5 shows the results of the regression models constructed using the data of participants other than the tested participant. The mean estimation error among the six participants is 1.24 cm. For all the participants, the estimation errors are greater than those of the personalized model. Participant P4 exhibits a mean error below 1 cm; however, the mean errors for the other participants are equal to or greater than 1 cm. The errors of the semipersonalized model are expected to be lesser than those of the nonpersonalized general model. The aforementioned results suggest the extent of difference among individuals. P5 and P6 may be distinctive from the others in terms of the mean estimation errors. Their mean estimation errors were 1.69 cm and 1.58 cm, respectively, whereas those for the other participants were less than 1.30 cm. Further, as shown in Fig. 5, P6 exhibited large standard deviations. For this participant, reproducibility was relatively low whereas the standard deviations were minor for the other participants. The root cause of such variability within participants is unclear. The number of the participants is small and we cannot estimate the properties of population.  When single extra samples from the target participants are used for calibration, the total mean error ranges from 0.91-0.90 cm, irrespective of the location of the extra sample. When point 44 is used, the mean errors for P5 and P6 are 1.33 and 1.25 cm, respectively, which are lesser than the errors when the other points are used. Hence, point 44, which is near the center, exhibits the best localization accuracy among all the tested points, although the differences in the accuracy are negligible.

Semipersonalized model: localization with the model constructed using a few target-user samples and the sample sets of other participants
As shown in Table 1, when the extra sample (i.e., point 44) is weighted with w = 10 , 100, and 1000, the mean errors are 0.90 cm for all w values. Figure 6  participants are lesser than those of the general model, as depicted in Fig. 5. When two extra samples from the tested target participants are used for calibration, the total mean localization error is approximately 0.90 cm, irrespective of the combination of points. When three, four, or five extra samples are used, the total mean errors range 0.89-0.90 cm. Figure 7 depicts the localization errors among the general, several semipersonalized, and personalized models. The mean error of the semipersonalized model using point 44 with no weight value ( w = 1 ) is lesser than that of the general model (two-tailed t-test with one sample, t = 15.77 , p = 1.86 × 10 −5 with no p-value adjustment for multiple comparisons). The mean error of the semipersonalized model using point 44 weighted by 1000 is lesser than that of the general model (two-tailed t-test with one sample, t = 15.50 , p = 2.03 × 10 −5 with no p-value adjustment for multiple comparisons). There are significant differences between the other semipersonalized models using more than one extra point and the general model, as shown in Fig. 7.

Discussion
As listed in Table 1, the localization error of the personalized model is 0.60 cm on an average and substantially lesser than that of the general (nonpersonalized) model (1.24 cm). For the individual participants, the errors of the general model are nearly equal to or more than twice that of the personalized model. The average errors of the semipersonalized models range from 0.89-0.92 cm, indicating that the localization accuracy of the semipersonalized model developed in this study is between those of the personalized and general models. As the size of the contact-area of the fingertip is approximately a 1.0 cmdiameter circle, localization errors of 0.90 cm may be acceptable for most touch-sensor applications, although our study does not assume specific applications. When the HumTouch technique is applied to square paper, the points near the corner and center involve relatively large localization errors. This is because the localization accuracy is relatively poor at points far from the electrodes [37]. The points near the corner and center of the paper are far from all the electrodes placed at the center of each side of the paper. The semipersonalized model can reduce the localization errors near an area by incorporating an extra sample of the target participant near the area. Thus, the calibration ability with different points has been investigated in this study. As listed in Table 1, when single extra points are used for the semipersonalized model, the estimation error obtained by adding a point in the center (i.e., point 44) has the least total mean error of 0.90 cm. In addition, by weighting the sample for point 44, the mean estimation error is further reduced, although this reduction is practically null. However, adding the extra sample is effective only for the surrounding area. The localization errors for the points in the other areas remain large. For example, as illustrated in Fig. 6, for the points near point 44 (the center point in each subfigure), the localization errors are corrected compared to those with the general model; however, the points near point 77 (right-down in each sub-figure) include localization error. Nonetheless, this study establishes that by including only a single sample, the nonpersonalized general model can be converted into a semipersonalized model with small localization errors. Figure 7 shows that the localization errors of the nonpersonalized general model are reduced by 27.8% by adding only one sample of the target participant. However, the addition of more than one sample is not markedly effective in reducing the error. For example, adding five extra samples improves the localization accuracy by merely 0.007 cm, compared to the case where only one This study has certain limitations. Only six participants were involved in the experiment. It is unclear whether semipersonalized models should be based on a large number of participants. The number of participants and samples may influence the additional number of samples necessary for constructing semipersonalized models. Potentially, the more the participants and samples, the more are the extra samples necessary to establish semipersonalized models because the effect of extra samples of the target user are obscured by the large number of samples from others. In contrast, semipersonalized models based on a few people may not be highly adaptive to unknown users. Hence, the number of people incorporated in the semipersonalized model is speculated to be a crucial design parameter. This study did not aim at optimizing the localization method. The methods based on kernel regression analysis were not compared with others. Kernel regression analysis is a suitable method for the problems discussed here; however, more effective methods must be explored. Furthermore, in hum-noisebased sensors, the causes for individual differences are unclear and have not been thoroughly discussed. If the individual differences are determined by certain parameters, such as the body weight, then another approach completely different from the one adopted in this study can be employed to build semipersonalized models. In addition, it is to be noted that localization errors are Fig. 7 Comparison of the total mean errors. The general model obviously exhibits greater error than the semipersonalized one. The personalized model with smaller error than those of the general and semipersonalized models is also shown for reference. *** indicates p < 0.001 as per the two-tailed t-test with one sample with no p-value adjustment for multiple comparisons generally greater for larger surfaces [37]. If the method proposed in this study is applied to a larger surface, the localization errors may not be as small as the fingertip size.

Conclusion
We proposed an easy-to-use calibration method for unknown users of HumTouch, a hum-noise-based touch sensing method. Because of the individual differences in hum-noise-driven currents in human bodies, HumTouch sensors need to be calibrated for individual users, limiting their range of application. The proposed method incorporates numerous samples collected from other people and a few weighted new samples from a new target user to establish a semipersonalized model for the new user. When a square paper was used as a touch-sensitive surface, the mean localization error was reduced from 1.24 to 0.90 cm by adding a new point from the target user to the sample set obtained from other people. This localization error is smaller than the size of a human fingertip. Therefore, the semipersonalized localization model using a weighted or nonweighted sample from the target user is practical for hum-noise-based touch sensors.