Interaction force estimation on a built-in position sensor for an electrostatic visuo-haptic display
© Nakamura and Yamamoto. 2016
Received: 14 January 2016
Accepted: 13 April 2016
Published: 29 April 2016
This paper discusses a force sensing method using a built-in position sensing system for an electrostatic visuo-haptic display. The display provides passive haptic feedback on a flat panel visual monitor, such as LCD, using electrostatic friction modulation via multiple contact pads arranged on a surface-insulated transparent electrode. The display demonstrated in previous studies measured the positions of the pads in a similar manner to surface-capacitive touch-screens. This paper extends the sensor such that the system can monitor the electrostatic interaction force provided to the contact pads. The extension is realized by estimating the capacitance between the contact pads and the display surface. The paper investigates its basic characteristics to show that the force estimation is possible, regardless of the pad positions and the pushing force exerted by users. The force estimation capability is used for feedback control of interaction force, which improves the accuracy of the interaction force. The paper further extends the method such that the system can detect the moving direction of contact pads. By dividing the electrode of a contact pad and comparing their capacitances, the system can detect in which direction the user is trying to move the pad. Such capability is effective for solving the sticky-wall problem, which is known to be a common problem in passive haptic systems. A pilot experiment shows that the proposed system can considerably reduce the sticky-wall effect.
Haptic interaction is imperative for intuitive operation of computer systems. In modern computer devices, the touch-screen technology has realized intuitive operations that allow us to directly touch graphical icons on the screen. However, users are sometimes frustrated due to the lack of appropriate feedback; implementing haptic feedback is expected to enrich the user experience. On flat screens, providing force feedback in vertical direction is considerably difficult, and thus many studies have tried to implement lateral force feedback, in which the feedback force is provided within lateral two dimensions. Such lateral force feedback has been regarded as effective enough, since surface geometry recognition of humans is closely related with lateral force as reported in some studies [1, 2]. For example, lateral force can create an illusion of corrugated shapes on flat surfaces .
In recent studies, lateral haptic feedback on a flat visual displays, such as liquid crystal displays (LCD), has often been realized utilizing ultrasonic vibration [3, 4] or electrovibration [5, 6]. These technologies modulate surface friction for haptic rendering, by inducing a squeeze film effect or an electrostatic adhesion force between a fingertip and a flat surface. The change of friction, however, is not large enough for kinesthetic force rendering; they can only provide up to 0.1 N of friction change , which can only realize surface texture rendering, such as rendering of surface roughness; it is not possible to render, for example, virtual walls. Some studies have mentioned that electrostatic adhesion can realize large force almost up to 2 N between a fingertip and an electrode by using Johnsen-Rahbeck effect . However, the effect requires special electrode material, which is not transparent, and thus it has not been applied to haptic feedback on an LCD. In addition, the technology needs further investigation on the time response, as it seems the effect becomes dominant only when DC voltage is applied.
To provide haptic feedback in accordance with the visual information displayed on the LCD, it is imperative to detect the pad positions. The prototype systems detected the positions by superposing sensing signal on the haptic voltage; the positions were estimated in a similar manner as surface-capacitive-type touch-screens. Using the position sensing capability, the previous studies demonstrated a hockey game with haptic feedback, in which users can feel impact when hitting a puck or can feel walls of the hockey arena [10, 12].
A next challenge for the visuo-haptic system is to improve the quality of the rendering force. In the previous implementations, the system controlled the force through the applied voltage in an open-loop manner, assuming that the electrostatic adhesion force is proportional to the square of the applied voltage. Such simple rendering was effective enough for typical applications, such as games as demonstrated in . However, for some applications that require higher accuracy of haptic rendering, such open-loop rendering was not precise enough due to the following two reasons.
The first is fluctuation and limited time-response of the electrostatic adhesion force. When voltage is applied between the pad electrode and the screen electrode, the electrodes deform due to the electrostatic adhesion force. The resulting gap variation between the electrodes can fluctuate the electrostatic force. Especially, when a soft conductive rubber is utilized as the pad electrode, the electrostatic adhesion force suddenly increases as the voltage is gradually increased, probably due to a sudden change of the gap between the electrodes . The deformation of the electrodes also affects the time-response of the electrostatic force. As a result, on the actual device, the force does not simply follow the square-law.
The second is about the sticky wall problem, which is well-known problem in passive haptic systems [13, 14]. Due to passive nature of the friction force, virtual walls rendered by passive haptic systems tend to be sticky; a user feels resistive force during retrieving from the wall. To solve the problem, the system needs to detect the direction of the operation force given by the user.
These two problems require monitoring of interaction forces, such as the haptic feedback force and the operation force from users in tangential directions. To achieve such monitoring, this paper proposes an interaction force estimation method on the built-in position sensing system of the electrostatic visuo-haptic display. The basic concept, which was simply introduced in , is to measure the total amount of electric current flowing in the sensing system. The current amount would correspond to the capacitance between the pad and the screen electrode, which could be utilized for the force estimation. Based on the concept proposed in the previous work , this paper investigates detailed characteristics of the force estimation and applies it to closed-loop force control and to elimination of the sticky wall problem.
The structure of this paper is as follows. “Built-in force estimating system” introduces the concept of the proposed method and shows basic performance of the capacitance measurement on a prototype device. “Haptic force estimation” discusses monitoring of the haptic force, where the measured capacitance is used for estimation of the electrostatic adhesion force. Then, the estimation is utilized for closed-loop control of haptic force in “Closed-loop control of haptic force”. “Wall rendering with lateral-force-direction sensing” proposes another application of the sensing, which estimates direction of the operation force from the difference of capacitances of divided electrodes, to solve the sticky wall problem.
Built-in force estimating system
Haptic rendering is realized by applying high voltages to the contact pads. The sensing system superposes high-frequency (much higher than that of the haptic voltage) signal to the haptic voltage using transformers, which results in corresponding high-frequency signals flowing through the current sensors. The signals are used for simultaneous estimation of position and force, whose concept is depicted in Fig. 2b. Position and force estimation for multiple pads are realized by time division; high-frequency sensing signal is alternately applied to each pad, one by one, within a short switching period.
The force estimation method discussed in this paper, on the other hand, utilizes the sum of the current amplitudes measured at all the peripheral points. From the sum, the system can estimate capacitance between a pad and the ITO electrode. Since the capacitance should correspond to the normal force applied on the pad, including the electrostatic adhesion force, the system can estimate the force from the capacitance.
In the sensor system, the position and force sensing are independent from each other. Our previous work revealed the position sensing is not affected by the change of capacitance or interaction force . The following calculation reveals the opposite: the force (or capacitance) sensing is not affected by the position change of the pads.
Capacitance measurement on prototype
The capacitance measurement is fundamental for the force estimation. To investigate the capacitance measurement performance in the actual setup, currents were measured in the setup used in . The ITO screen electrode in the setup has approximately 860 mm × 500 mm in its size and 150 Ω/sq. of sheet resistance. The pad electrode is ϕ30-mm round-shaped conductive rubber sheet. The surfaces of the two electrodes are insulated with PET film whose thickness is 8 μm. As the sensing signal, a 100-kHz sine wave was used. In the setup, the combined resistance, which was measured at several points on the ITO electrode using a circuit tester, is approximately 500 Ω in maximum, while the capacitive impedance between the two electrodes, which was measured using an impedance analyzer, is approximately 5 kΩ. This large difference in impedance satisfies the condition for Eq. (6). Therefore, on this setup, the capacitance estimation, and in turn the force estimation, should be independent from the position of the pad.
Haptic force estimation
This section investigates the estimation of haptic force caused by the electrostatic adhesion. The previous work  simply assumed that the capacitance change is caused by the normal force applied to the pad, which is the sum of user’s pushing force and the electrostatic adhesion force. Therefore, the previous work tried to directly relate the measured capacitance (more exactly, the sum of the current sensor outputs) with the sum of the pushing force and electrostatic force. Such assumption, however, showed large errors probably due to the non-linearity of pad deformation. Therefore, this paper tries to estimate the haptic force alone using a theoretical model of electrostatic force.
As Eq. (8) indicates, the haptic voltage should be estimated from the product of the sensor output and the haptic voltage. Figure 6c plots the relation between the product and the increment of friction, with a quadratic fitting curve. The plot clearly shows that Eq. (8) provides a good estimation of the haptic force, regardless of the pushing force. The maximum error was found less than 0.1 N for these conditions.
When the haptic voltage was raised, the friction force gradually increased. The measured voltage on the upper plot shows gradual decrease, which is due to the change of the capacitance between the pad and the ITO. As the normal force, and in turn friction, increases, the capacitance and the corresponding current also increase. The increased current leads to larger voltage drops at the current limiting resistor and the transformer that were arranged between the voltage source and the pad. The sudden rises of the friction at around 1.0 s and 2.8 s would correspond to “pull-in”, in which the gap suddenly decreases when the balance between the linear elastic force (from the insulating material) and non-linear electrostatic force is broken. The estimation successfully described the change of the friction force, including the sudden change due to pull-in. In addition, in contrast to the previous work , there was no residual output in the estimated value after the haptic voltage turned off. The error for this dynamic measurement was within the maximum error of 0.1 N that was found for the static measurement.
Closed-loop control of haptic force
Our previous system [9, 12] has controlled the haptic force in an open-loop manner, based on the assumption that the haptic force is proportional to the square of the haptic voltage. As shown in the previous section, however, the assumption is not true due to the capacitance change; the haptic force fluctuates even when a constant haptic voltage is applied. Such behavior can be compensated by closed-loop control using the haptic force estimation, which is demonstrated in this section.
Wall rendering with lateral-force-direction sensing
Experiment for wall rendering
However, this simple rendering does not completely solve the problem. The sensor sometimes mis-detected the force direction when the pad is going to enter the virtual wall, which vanished the wall. In addition, during the retrieving phase, chattering vibration was sometimes observed. The reason of these behavior was low accuracy/sensitivity of the prototyped force-direction sensing, which might be solved by adding some signal processing filters. Combination use of motion direction might be useful to compensate the sensing accuracy. These solutions would be tackled in our future work.
This paper discussed interaction force sensing using a built-in sensing system for an electrostatic visuo-haptic display. The sensing system can estimate the capacitance between the contact pad and the ITO electrode, in addition to, and independent from, the position sensing. Based on the estimated capacitance, the sensing system is able to estimate the haptic feedback force. The estimation enables precise control of haptic feedback in a closed-loop manner. The control reduces the error from the target force and improves the response speed.
By modifying the structure of the pad, the sensing method can be extended to detect the direction of the operating force given by the user. The direction sensing successfully reduced the stickiness of the virtual wall. These interaction force sensing has just opened the door to high-fidelity haptic feedback on flat visual displays. The rendering method would be further investigated in our future work.
TN contributed to the conception, design of experiments, acquisition of data, and drafting of the manuscript. AY took part in the conception, interpretation of data, and revising of the manuscript. Both authors read and approved the final manuscript.
This work was supported in part by Grant-in-Aid for JSPS Fellows (No. 26 9272) and Grand-in-Aid for Scientific Research (B) (No. 26280069) from JSPS, Japan.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Christou C, Wing A (2001) Friction and curvature judgement. In: Proc. EurohapticsGoogle Scholar
- Robles-De-La-Torre G, Hayward V (2001) Force can overcome object geometry in the perception of shape through active touch. Nature 412(6845):445–448View ArticleGoogle Scholar
- Takasaki M, Kotani H, Mizuno T, Nara T (2005) Transparent surface acoustic wave tactile display. In: 2005 IEEE/RSJ international conference on intelligent robots and systems (IROS 2005), pp 3354–3359Google Scholar
- Lévesque V, Oram L, MacLean K, Cockburn A, Marchuk ND, Johnson D, Colgate JE, Peshkin MA (2011) Enhancing physicality in touch interaction with programmable friction. In: Proceedings of the SIGCHI conference on human factors in computing systems. CHI ’11, pp 2481–2490Google Scholar
- Bau O, Poupyrev I, Israr A, Harrison C (2010) Teslatouch: electrovibration for touch surfaces. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology. UIST ’10, pp 283–292Google Scholar
- Linjama J. Mäkinen V (2009) E-sense screen: novel haptic display with capacitive electrosensory interface. Presented at HAID 09, 4th Workshop for haptic and audio interaction design, Dresden, GermanyGoogle Scholar
- Meyer DJ, Peshkin MA, Colgate JE (2013) Fingertip friction modulation due to electrostatic attraction. In: 2013 world haptics conference (WHC), pp 43–48Google Scholar
- Shultz CD, Peshkin MA, Colgate JE (2015) Surface haptics via electroadhesion: expanding electrovibration with johnsen and rahbek. In: World haptics conference (WHC), IEEE, pp 57–62Google Scholar
- Nakamura T, Yamamoto A (2013) Multi-finger electrostatic passive haptic feedback on a visual display. In: World haptics conference (WHC), IEEE, pp 37–42Google Scholar
- Nakamura T, Yamamoto A (2014) Built-in capacitive position sensing for multi-user electrostatic visuo-haptic display. In: Asia haptics 2014Google Scholar
- Nakamura T, Yamamoto A (2015) Simultaneous measurement of position and interaction force on a multi-user electrostatic visuo-haptic display. In: 2015 IEEE world haptics conferenceGoogle Scholar
- Nakamura T, Yamamoto A (2015) A multi-user surface visuo-haptic display using electrostatic frcition modulation and capacitive-type position sensing. IEEE Transactions on Haptics, no. 1, pp. 1, PrePrints. doi:10.1109/TOH.2016.2556660
- Colgate JE, Peshkin MA, Wannasuphorasit W (1996) Nonholonomic haptic display. In: Proceedings of 1996 IEEE international conference on robotics and automation, vol. 1, pp 539–544Google Scholar
- Furusho J, Sakaguchi M, Takesue N, Koyanagi K (2002) Development of er brake and its application to passive force display. J Intell Mater Syst Struct 13(7–8):425–429View ArticleGoogle Scholar