Interaction force estimation on a built-in position sensor for an electrostatic visuo-haptic display

Abstract

This paper discusses a force sensing method using a built-in position sensing system for an electrostatic visuo-haptic display. The display provides passive haptic feedback on a flat panel visual monitor, such as LCD, using electrostatic friction modulation via multiple contact pads arranged on a surface-insulated transparent electrode. The display demonstrated in previous studies measured the positions of the pads in a similar manner to surface-capacitive touch-screens. This paper extends the sensor such that the system can monitor the electrostatic interaction force provided to the contact pads. The extension is realized by estimating the capacitance between the contact pads and the display surface. The paper investigates its basic characteristics to show that the force estimation is possible, regardless of the pad positions and the pushing force exerted by users. The force estimation capability is used for feedback control of interaction force, which improves the accuracy of the interaction force. The paper further extends the method such that the system can detect the moving direction of contact pads. By dividing the electrode of a contact pad and comparing their capacitances, the system can detect in which direction the user is trying to move the pad. Such capability is effective for solving the sticky-wall problem, which is known to be a common problem in passive haptic systems. A pilot experiment shows that the proposed system can considerably reduce the sticky-wall effect.

Introduction

Haptic interaction is imperative for intuitive operation of computer systems. In modern computer devices, the touch-screen technology has realized intuitive operations that allow us to directly touch graphical icons on the screen. However, users are sometimes frustrated due to the lack of appropriate feedback; implementing haptic feedback is expected to enrich the user experience. On flat screens, providing force feedback in vertical direction is considerably difficult, and thus many studies have tried to implement lateral force feedback, in which the feedback force is provided within lateral two dimensions. Such lateral force feedback has been regarded as effective enough, since surface geometry recognition of humans is closely related with lateral force as reported in some studies [1, 2]. For example, lateral force can create an illusion of corrugated shapes on flat surfaces [2].

In recent studies, lateral haptic feedback on a flat visual displays, such as liquid crystal displays (LCD), has often been realized utilizing ultrasonic vibration [3, 4] or electrovibration [5, 6]. These technologies modulate surface friction for haptic rendering, by inducing a squeeze film effect or an electrostatic adhesion force between a fingertip and a flat surface. The change of friction, however, is not large enough for kinesthetic force rendering; they can only provide up to 0.1 N of friction change [7], which can only realize surface texture rendering, such as rendering of surface roughness; it is not possible to render, for example, virtual walls. Some studies have mentioned that electrostatic adhesion can realize large force almost up to 2 N between a fingertip and an electrode by using Johnsen-Rahbeck effect [8]. However, the effect requires special electrode material, which is not transparent, and thus it has not been applied to haptic feedback on an LCD. In addition, the technology needs further investigation on the time response, as it seems the effect becomes dominant only when DC voltage is applied.

To extend the possibility of haptic feedback on an LCD, the authors have proposed an indirect electrostatic visuo-haptic rendering system that can provide large haptic feedback force to multiple users or fingers [9–12], without using Johnsen-Rahbeck effect. The proposed system utilizes “contact pads” that have electrodes on their bottom surfaces. They are placed on an LCD surface to exert large enough electrostatic adhesion force. The user obtains haptic feedback via the contact pads, and therefore, the rendering method is called “indirect”. One of the prototypes, which is used in [10–12], is shown in Fig. 1. The system is implemented on an off-the-shelf 40-inch LCD monitor without a touch-sensing function. The surface of the LCD monitor is covered with a transparent electrode, which is electrically grounded. Multiple contact pads placed on the transparent electrode are connected to different voltage sources, such that each pad can generate different electrostatic adhesion force. When the pad is moved by a user, the adhesion force is converted into friction force, which is perceived as haptic feedback; the resulting haptic feedback is passive due to the nature of friction force. The system is capable of providing more than 1 N of friction change to multiple fingers [9], which is large enough for kinesthetic haptic rendering including virtual wall rendering.

Fig. 1

Appearance of prototyped visuo-haptic display. (Reprinted from [11])

To provide haptic feedback in accordance with the visual information displayed on the LCD, it is imperative to detect the pad positions. The prototype systems detected the positions by superposing sensing signal on the haptic voltage; the positions were estimated in a similar manner as surface-capacitive-type touch-screens. Using the position sensing capability, the previous studies demonstrated a hockey game with haptic feedback, in which users can feel impact when hitting a puck or can feel walls of the hockey arena [10, 12].

A next challenge for the visuo-haptic system is to improve the quality of the rendering force. In the previous implementations, the system controlled the force through the applied voltage in an open-loop manner, assuming that the electrostatic adhesion force is proportional to the square of the applied voltage. Such simple rendering was effective enough for typical applications, such as games as demonstrated in [10]. However, for some applications that require higher accuracy of haptic rendering, such open-loop rendering was not precise enough due to the following two reasons.

The first is fluctuation and limited time-response of the electrostatic adhesion force. When voltage is applied between the pad electrode and the screen electrode, the electrodes deform due to the electrostatic adhesion force. The resulting gap variation between the electrodes can fluctuate the electrostatic force. Especially, when a soft conductive rubber is utilized as the pad electrode, the electrostatic adhesion force suddenly increases as the voltage is gradually increased, probably due to a sudden change of the gap between the electrodes [12]. The deformation of the electrodes also affects the time-response of the electrostatic force. As a result, on the actual device, the force does not simply follow the square-law.

The second is about the sticky wall problem, which is well-known problem in passive haptic systems [13, 14]. Due to passive nature of the friction force, virtual walls rendered by passive haptic systems tend to be sticky; a user feels resistive force during retrieving from the wall. To solve the problem, the system needs to detect the direction of the operation force given by the user.

These two problems require monitoring of interaction forces, such as the haptic feedback force and the operation force from users in tangential directions. To achieve such monitoring, this paper proposes an interaction force estimation method on the built-in position sensing system of the electrostatic visuo-haptic display. The basic concept, which was simply introduced in [11], is to measure the total amount of electric current flowing in the sensing system. The current amount would correspond to the capacitance between the pad and the screen electrode, which could be utilized for the force estimation. Based on the concept proposed in the previous work [11], this paper investigates detailed characteristics of the force estimation and applies it to closed-loop force control and to elimination of the sticky wall problem.

The structure of this paper is as follows. “Built-in force estimating system” introduces the concept of the proposed method and shows basic performance of the capacitance measurement on a prototype device. “Haptic force estimation” discusses monitoring of the haptic force, where the measured capacitance is used for estimation of the electrostatic adhesion force. Then, the estimation is utilized for closed-loop control of haptic force in “Closed-loop control of haptic force”. “Wall rendering with lateral-force-direction sensing” proposes another application of the sensing, which estimates direction of the operation force from the difference of capacitances of divided electrodes, to solve the sticky wall problem.

Built-in force estimating system

Concept

Figure 2a shows the built-in sensing system for the electrostatic visuo-haptic display, which was briefly introduced in [11]. The system shares most of its components with haptic feedback system. The shared components include a transparent surface-insulated indium-tin-oxide (ITO) electrode that covers the whole screen surface, and multiple contact pads with electrodes on their bottom surfaces. Current sensors are inserted between the ITO electrode and the ground at six peripheral points, as depicted in the figure.

Fig. 2

Schematic (a), concept (b), and circuit model (c) of the built-in sensing system for the electrostatic visuo-haptic display. (Modified from [11])

Haptic rendering is realized by applying high voltages to the contact pads. The sensing system superposes high-frequency (much higher than that of the haptic voltage) signal to the haptic voltage using transformers, which results in corresponding high-frequency signals flowing through the current sensors. The signals are used for simultaneous estimation of position and force, whose concept is depicted in Fig. 2b. Position and force estimation for multiple pads are realized by time division; high-frequency sensing signal is alternately applied to each pad, one by one, within a short switching period.

Before discussing the force estimation, the position estimation method, whose details are addressed in [12], is reviewed using the circuit model shown in Fig. 2c. When the sensing signal, $V_s \sin (\omega _s t)$, and the haptic voltage, $V_h \sin (\omega _h t)$ (note that $\omega _h \ll \omega _s$), are applied on a pad, corresponding current running through each terminal becomes

$$\begin{aligned} {\varvec{I}}_i = \frac{1}{R_i}\left( \frac{j\omega _s CR_p}{1+j\omega _s CR_p}{\varvec{V}}_s + \frac{j\omega _h CR_p}{1+j\omega _h CR_p}{\varvec{V}}_h\right) , \end{aligned}$$

(1)

where ${\varvec{V}}_s$ and ${\varvec{V}}_h$ are the sensing signal and the haptic voltage in complex representation respectively, $R_i$ is the resistance of the ITO electrode from the contact point to the i-th terminal, C is the capacitance between the pad and the ITO, and $R_p$ is the combined resistance of ITO resistances $R_i$. The resistances $R_i$ are assumed to correspond to the distance between the terminals and the pad. Due to the resistances, the currents running toward the terminals depend on the pad position. Therefore, from the ratios of the current amplitudes, which are calculated as,

$$\begin{aligned} r_x&= \frac{-I_1 - I_2 + I_3 + I_4}{I_1 + I_2 + I_3 + I_4} = \frac{-\frac{1}{R_1} - \frac{1}{R_2} + \frac{1}{R_3} + \frac{1}{R_4}}{\frac{1}{R_1} + \frac{1}{R_2} + \frac{1}{R_3} + \frac{1}{R_4}}, \end{aligned}$$

(2)

$$\begin{aligned} r_y&= \frac{I_1 - I_2 - I_3 + I_4}{I_1 + I_2 + I_3 + I_4} = \frac{\frac{1}{R_1} - \frac{1}{R_2} - \frac{1}{R_3} + \frac{1}{R_4}}{\frac{1}{R_1} + \frac{1}{R_2} + \frac{1}{R_3} + \frac{1}{R_4}}, \end{aligned}$$

(3)

where $I_i$ is amplitude of ${\varvec{I}}_i$, the system can estimate the position of the pad in x–y coordinate. In the actual setup, current amplitudes corresponding to the sensing signal are distilled from the total currents.

The force estimation method discussed in this paper, on the other hand, utilizes the sum of the current amplitudes measured at all the peripheral points. From the sum, the system can estimate capacitance between a pad and the ITO electrode. Since the capacitance should correspond to the normal force applied on the pad, including the electrostatic adhesion force, the system can estimate the force from the capacitance.

In the sensor system, the position and force sensing are independent from each other. Our previous work revealed the position sensing is not affected by the change of capacitance or interaction force [12]. The following calculation reveals the opposite: the force (or capacitance) sensing is not affected by the position change of the pads.

The total current running through a pad can be calculated as

$$\begin{aligned} {\varvec{I}} = \frac{j\omega _s C}{1+j\omega _s CR_p }{\varvec{V}}_s + \frac{j\omega _h C}{1+j\omega _h CR_p} {\varvec{V}}_h, \end{aligned}$$

(4)

in complex representation. For the force sensing, amplitude of the current corresponding to the sensing signal is focused, since the current amplitude, unlike the current phase, becomes independent from the pad position in a certain condition as discussed in the following. The amplitude corresponding to the sensing signal, which can be detected by using synchronous detection, becomes

$$\begin{aligned} A = \frac{\omega _s C}{\sqrt{1+(\omega _s CR_p )^2}}V_s. \end{aligned}$$

(5)

The amplitude includes not only the capacitance, C, but also the combined resistance, $R_p$, which depends on the position of the pad. However, if the combined resistance is small enough to satisfy $R_p^2 \ll \left( \tfrac{1}{\omega _s C}\right) ^2$, the detected amplitude becomes

$$\begin{aligned} A \approx \omega _s CV_s, \end{aligned}$$

(6)

which is independent from the pad position.

Capacitance measurement on prototype

The capacitance measurement is fundamental for the force estimation. To investigate the capacitance measurement performance in the actual setup, currents were measured in the setup used in [12]. The ITO screen electrode in the setup has approximately 860 mm × 500 mm in its size and 150  Ω/sq. of sheet resistance. The pad electrode is ϕ30-mm round-shaped conductive rubber sheet. The surfaces of the two electrodes are insulated with PET film whose thickness is 8 μm. As the sensing signal, a 100-kHz sine wave was used. In the setup, the combined resistance, which was measured at several points on the ITO electrode using a circuit tester, is approximately 500 Ω in maximum, while the capacitive impedance between the two electrodes, which was measured using an impedance analyzer, is approximately 5 kΩ. This large difference in impedance satisfies the condition for Eq. (6). Therefore, on this setup, the capacitance estimation, and in turn the force estimation, should be independent from the position of the pad.

Figure 3 shows the sum of the outputs of the current sensors, which corresponds to A in Eqs. (5) and (6), against the position of the pad in several load conditions. The load, representing user’s pushing force, was applied by putting a weight on the pad. As expected, these results are almost flat regardless of the pad position, which verifies the independence of the capacitance sensing from the pad position.

Fig. 3

The sum of outputs from the current sensors in several load and position conditions

Figure 4 shows the relation between the load and the sum of the signals in several haptic voltage conditions, measured at a fixed pad position. The error bar indicates the maximum and the minimum values within three-time measurements. It is clearly observed that two factors affect the total current: electrostatic adhesion force represented by the haptic voltage and the user’s pushing force. The next section will focus on the effect of electrostatic adhesion force on the rendering force estimation. The later section, “Wall rendering with lateral-force-direction sensing”, on the other hand, will utilize the effect of pushing force, to realize lateral-force-direction sensing.

Fig. 4

The sum of outputs from the current sensors in several load and voltage conditions

Haptic force estimation

This section investigates the estimation of haptic force caused by the electrostatic adhesion. The previous work [11] simply assumed that the capacitance change is caused by the normal force applied to the pad, which is the sum of user’s pushing force and the electrostatic adhesion force. Therefore, the previous work tried to directly relate the measured capacitance (more exactly, the sum of the current sensor outputs) with the sum of the pushing force and electrostatic force. Such assumption, however, showed large errors probably due to the non-linearity of pad deformation. Therefore, this paper tries to estimate the haptic force alone using a theoretical model of electrostatic force.

In the parallel-plate capacitor model, the electrostatic force can be expressed as

$$\begin{aligned} F_e = \frac{\left( CV_h\right) ^2}{2 \varepsilon S}, \end{aligned}$$

(7)

where C is the capacitance between the pad and the ITO electrode, $V_h$ is the haptic voltage applied between the electrodes, $\varepsilon$ is the dielectric constant, and S is the area of the interface. Thus, the haptic force (increment of friction force), which is proportional to the electrostatic adhesion force, can be estimated as

$$\begin{aligned} F_h = k(C'V_h)^2, \end{aligned}$$

(8)

where k is a coefficient of the estimation and $C'=\beta C$ is the output of the sensing system with $\beta$ being a proportionality coefficient.

To verify the haptic force estimation, an experiment on a 1-DOF setup was conducted. Figure 5 shows a schematic of the experimental setup, which consists of a simplified electrostatic haptic system on a motorized stage and a load-cell. The simplified system includes a current sensor, which consists of an I/V converter and a synchronous detection circuit to detect the current amplitude in the sensing signal frequency. By pushing the pad toward the load-cell using the motorized stage, the setup measures friction force between the pad and the ITO. In the experiment, a weight was put on the pad to represent pushing force from a user’s finger. The voltage on the pad (which is the voltage after the transformer for superposing sensing signal) was measured as the haptic voltage.

Fig. 5

Experimental setup for friction measurement

First, a calibration for the estimation was conducted, whose results are summarized in Fig. 6. Figure 6a shows the change of the measured friction in several haptic voltage conditions. Square and round markers represent two weight conditions, 100 g and 20 g respectively. In the plot, the two series of the measured friction force have offsets due to the weight. Since the objective is to estimate the haptic force, which is the increment of the friction from the no-voltage condition, the increment is plotted in Fig. 6b using blue markers. A quadratic curve is also plotted using a black line for reference, which shows that the haptic force does not follow the square-law. This behavior of the haptic force is probably due to the change of the gap between the pad electrode and the ITO screen electrode. Since a rise of the haptic voltage would decrease the gap through increase of electrostatic attraction force, the electrostatic force would be enhanced deviating from the square-law. This assumption is supported by the sensor outputs measured at the same time, which are plotted in red color with reference to the right vertical axis. The sensor output, which is assumed to represent the capacitance, suddenly increases above 150 V_rms, which means that the gap decreased above that voltage.

Fig. 6

Results of haptic force estimation in static conditions. a Measured friction in several haptic voltage conditions. b Increment of the friction and sensor output. c Relation between the increment of the friction and the product of the sensor output and the haptic voltage

As Eq. (8) indicates, the haptic voltage should be estimated from the product of the sensor output and the haptic voltage. Figure 6c plots the relation between the product and the increment of friction, with a quadratic fitting curve. The plot clearly shows that Eq. (8) provides a good estimation of the haptic force, regardless of the pushing force. The maximum error was found less than 0.1 N for these conditions.

By using the calibration result, the step response of the friction force was estimated and compared with the output of the load-cell. Figure 7 shows the result when a weight of 50 g was applied and haptic voltage of 350 V_rms was commanded. It should be noted that the commanded haptic voltage is different from the actual haptic voltage measured at the pad, due to the voltage drop at the transformer and the current-limiting resistor. The upper plot indicates the temporal change of the applied voltage, and the lower plot shows the response of the friction force measured by the load-cell (blue line) and the estimated force (green line).

Fig. 7

Results of haptic force estimation in dynamic condition

When the haptic voltage was raised, the friction force gradually increased. The measured voltage on the upper plot shows gradual decrease, which is due to the change of the capacitance between the pad and the ITO. As the normal force, and in turn friction, increases, the capacitance and the corresponding current also increase. The increased current leads to larger voltage drops at the current limiting resistor and the transformer that were arranged between the voltage source and the pad. The sudden rises of the friction at around 1.0 s and 2.8 s would correspond to “pull-in”, in which the gap suddenly decreases when the balance between the linear elastic force (from the insulating material) and non-linear electrostatic force is broken. The estimation successfully described the change of the friction force, including the sudden change due to pull-in. In addition, in contrast to the previous work [11], there was no residual output in the estimated value after the haptic voltage turned off. The error for this dynamic measurement was within the maximum error of 0.1 N that was found for the static measurement.

Closed-loop control of haptic force

Our previous system [9, 12] has controlled the haptic force in an open-loop manner, based on the assumption that the haptic force is proportional to the square of the haptic voltage. As shown in the previous section, however, the assumption is not true due to the capacitance change; the haptic force fluctuates even when a constant haptic voltage is applied. Such behavior can be compensated by closed-loop control using the haptic force estimation, which is demonstrated in this section.

Haptic forces with open-loop and closed-loop control were compared on the same setup as in Fig. 5. Figure 8 shows the two controllers used in the experiment. One is the open-loop feedforward controller used in the previous system. Supposing that the haptic force is proportional to the haptic voltage squared, the command voltage is controlled as

$$\begin{aligned} V_{ref} = K\sqrt{F_t}, \end{aligned}$$

(9)

where $F_t$ is the target haptic force and K is a coefficient and was empirically set to 600 $\text {V/N}^{-\tfrac{1}{2}}$ in this particular work. The other is a feedback controller combined with the previous feedforward controller. The feedback controller is a simple proportional one, whose proportional gain, $K_p$, was empirically chosen to 1000 V/N. The coefficient for the estimation [k in Eq. (8)] was manually adjusted.

Fig. 8

Block diagram of the two controllers: a open-loop control and b closed-loop control

Figure 9 shows the results of the two controllers for several conditions of target force: square wave and single-sided sine wave in different amplitudes. Left-side figures are the results of the previous controller, and the right-side figures are the results of the proposed controller. For each condition, the upper plot and the lower plot indicate the haptic force and the applied haptic voltage over time, respectively. Red, blue, and green lines represent target force, the measured force, and estimated force, respectively. Comparison of the two controllers in the square-wave case shows that the closed-loop controller improves the rising of the haptic force by increasing the applied voltage. In sine-wave case, whereas the previous controller lost the shape of the force from the target, the closed-loop controller achieved the force closer to the target. These results suggest that the closed-loop control with haptic force estimation can render the haptic feedback more precisely by improving the instability of the electrostatic adhesion.

Fig. 9

Results of open-loop (a) and closed-loop (b) control for several target-force conditions: (1) 0.3 N square, (2) 0.6 N square, (3) 0.3 N single-sided sine, and (4) 0.6 N single-sided sine

Wall rendering with lateral-force-direction sensing

The system provides haptic feedback passively by using friction. Such a passive haptic system suffers from the sticky wall problem, which can be solved by considering direction of the force/movement of user’s finger [13, 14]. Figure 10 explains the problem, which occurs when the reaction force from a virtual wall is rendered using passive haptic force, such as brake force and friction. When the pad enters the virtual wall, the brake or friction force acts as the reaction force from the wall (Fig. 10a). When the pad is retrieving from the wall, the brake force is still exerted until the pad completely getting off from the wall (Fig. 10b). This extra brake force is perceived as stickiness of the wall. To solve the problem, wall rendering should consider the direction of the force or the movement given from the user; the force rendering should be cut when the operating direction is the retrieving one. The direction of the movement, which can be obtained from the position sensor for the pad, however, has a limited effect in solving the sticky-wall problem, since the movement occurs after the operating force overcomes the maximum static friction under the haptic voltage application. If we can directly monitor the direction of the operating force, the haptic voltage can be cut before the operating force reaches the maximum static friction; the resulting sticky force would be much smaller. Toward the monitoring of the operating force direction, this section modifies the force sensing described in the previous sections.

Fig. 10

Schematics of sticky wall problem. In pushing phase (a), the friction properly acts as reaction force from the virtual wall. In retrieving phase (b), however, the friction acts as sticking force

Lateral-force-direction sensing

Figure 11 shows the concept of the lateral-force-direction sensing. The concept is to divide the pad electrode into a few parts and to detect the eccentric amount of the pressure as the lateral-force direction. Each part of the pad electrode, connected to an independent voltage source, detects its 2-D position, ${\varvec{x}}_i$ (i is the index for each part), and normal force, $F_i$. The center position of the whole pad can be calculated by the mean position of all the electrode parts as ${\varvec{x}}_c = \tfrac{\sum _{i=1}^{n}{\varvec{x}}_i}{n}$. Here, n is the number of the divided parts. Then, the center of the pressure (CoP) can be calculated as ${\varvec{x}}_p = \tfrac{\sum _{i=1}^{n}F_i{\varvec{x}}_i}{\sum _{i=1}^{n}F_i}$. Finally, the subtraction of the CoP from the center position, $\Delta {\varvec{x}}_p = {\varvec{x}}_p - {\varvec{x}}_c$, would correspond to the lateral force direction.

Fig. 11

Concept of the force-direction sensing

To validate the proposed method, an experiment on a 1-DOF prototype was conducted. The experimental setup, whose schematic is shown in Fig. 12, consists of a 1-DOF prototype pad, a sensing table, and an indenter. The 1-DOF pad has two sectioned electrodes connected to independent voltage sources, and sensing signal is alternately applied to each pad for multiple sensing. On the sensing table, an ITO electrode sheet covering an acrylic base plate are connected to current sensors at the two ends to detect the force and 1-D position of each sectioned electrode of the pad. In the experiment, the actual eccentric amount and the sensor output were compared in several haptic voltage conditions. This particular experiment directly uses the signal ratios between the two current sensors, $\tilde{x}_i = (-I_1+I_2)/(I_1+I_2)$ [1-DOF version of Eq. (2), $i = 1, 2$], as a normalized position of each pad electrode. The sum of the signal amplitudes, $a_i = I_1 + I_2$, was supposed to correspond to the force applied on each sectioned electrode.

Fig. 12

Experimental setup for force-direction sensing

Figure 13 shows the relation between the actual eccentric distance (displacement of the indenter from the center of the pad) and the estimated eccentric distance, which was calculated as

$$\begin{aligned} \Delta \tilde{x}_p = \tilde{x}_p - \tilde{x}_c = \frac{\sum _{i=1}^{2}a_i \tilde{x}_i}{\sum _{i=1}^{2}a_i} - \frac{\sum _{i=1}^{2} \tilde{x}_i}{2}, \end{aligned}$$

(10)

where $\tilde{x}_p$ and $\tilde{x}_c$ represent the normalized CoP and center position respectively. The color of the plots represents the haptic voltage as the legend shows. For all conditions, the estimated eccentric distance changes approximately linearly to the change of the actual distance. The estimated distance has some offset due to variance of sensitivity of each electrode. Fabrication error of the sectioned electrodes and amplitude difference of voltage sources would be the cause of the variance. In addition, as same as the pushing force measurement, the sensitivity of force direction measurement decreases as the haptic voltage increases. The dependence of the offset and the degrading sensitivity due to the haptic voltage complicates the sensing of the lateral force, and should be improved in the future work. However, for solving the sticky wall problem, the important thing is not detecting the magnitude of the lateral force but just direction of the force. The prototype has enough performance in detecting the force direction.

Fig. 13

Static characteristic of the force-direction sensing

Experiment for wall rendering

To demonstrate the capability of the force-direction sensing, a wall rendering experiment was conducted on the modified setup of the previous subsection. The appearance of the setup is shown in Fig. 14. The setup added a force sensor for evaluation, which was attached on a newly introduced linear guide for supporting the 1-DOF movement of the pad. In the experiment, the rendered forces with/without considering force direction were compared when the pad was moved manually through the force sensor to interact with a virtual wall. The basic virtual wall (without consideration of force direction) was rendered by controlling the haptic voltage as

$$\begin{aligned} V_h = K_w \tilde{x}_c & \quad (\tilde{x}_c < 0). \end{aligned}$$

(11)

The wall rendering with considering the force direction was as follows:

$$\begin{aligned} V_h = K_w \tilde{x}_c & \quad (\tilde{x}_c < 0 \cap \tilde{x}_c \cdot \Delta \tilde{x}_p \le 0). \end{aligned}$$

(12)

Fig. 14

Appearance of experimental setup for wall rendering

Figure 15 shows the results of the wall rendering. From above, these plots indicate time-variable plot of the position, the lateral force (eccentric distance) estimated by the built-in sensor, haptic voltage, and haptic force measured by the evaluation force sensor. Left-side plots show the result without considering force direction, and the right-side plots for considering force direction. The negative position means that the pad is within the virtual wall. In the left-side plots, the sticking force was exerted until the pad position returns to positive. On the other hand, sticking force was rapidly switched off in the right-side plots. This clearly shows that the force-direction sensing can reduce the stickiness of the virtual wall.

Fig. 15

Results of wall rendering without (a) / with (b) considering force direction

However, this simple rendering does not completely solve the problem. The sensor sometimes mis-detected the force direction when the pad is going to enter the virtual wall, which vanished the wall. In addition, during the retrieving phase, chattering vibration was sometimes observed. The reason of these behavior was low accuracy/sensitivity of the prototyped force-direction sensing, which might be solved by adding some signal processing filters. Combination use of motion direction might be useful to compensate the sensing accuracy. These solutions would be tackled in our future work.

Conclusion

This paper discussed interaction force sensing using a built-in sensing system for an electrostatic visuo-haptic display. The sensing system can estimate the capacitance between the contact pad and the ITO electrode, in addition to, and independent from, the position sensing. Based on the estimated capacitance, the sensing system is able to estimate the haptic feedback force. The estimation enables precise control of haptic feedback in a closed-loop manner. The control reduces the error from the target force and improves the response speed.

By modifying the structure of the pad, the sensing method can be extended to detect the direction of the operating force given by the user. The direction sensing successfully reduced the stickiness of the virtual wall. These interaction force sensing has just opened the door to high-fidelity haptic feedback on flat visual displays. The rendering method would be further investigated in our future work.