Journal of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

Haptic actuators for large display panels play an important role in bridging the gap between the digital and physical world by generating interactive feedback for users. For example, large touchscreen displays with haptic sensations can be utilized effectively in interactive education for children [1], tabletop medical training [2], and digital musical instruments [3]. Unlike small touchscreen displays in mobile devices, there are technical issues with generating meaningful haptic feedback for large display panels [4]. For example, if only a few actuators are employed, dead zones arise where no haptic feedback is sensed [5]. In contrast, localizing tactile feedback in the subregions of large display panels is challenging in the presence of multiple actuators as one must control the traveling wave generated by each actuator [6].

Attempts have been made to localize the subregions of the display by adjusting the parameters of each actuator such as the magnitude and phase. Studies on haptic localization evaluation have been empirical and experimental [7]. However, it is difficult to find the global maximization using an empirical approach because there are infinitely many choices of parameters for the actuators. Therefore, there is a need to precisely evaluate haptic feedback and optimize algorithms for localization.

In this paper, we propose a new algorithm for haptic actuator localization on a large display based on an optimizationtype algorithm to determine the maximizer of the localization factor (LF). The difficulty lies in the fact that it is almost impossible to find the partial derivatives of the LF with respect to the actuator parameters such as the magnitude or phase. Therefore, we employed particle swarm optimization (PSO) because it involves a stochastic process without a gradient [8-11]. For convenience, electrostatic actuators were considered in the analysis because it is easy to control the magnitude and phases of the excitations using actuators [12]. In addition, the transverse displacement under the effect of excitation was modeled using the Kirchhoff-Love (KL) plate theory [13-14].

To evaluate the LF, the KL solutions were reconstructed using a finite difference method (FDM) algorithm, where the forward Euler scheme was used for time discretization. To determine the maximizer of the LF, random samples were generated for the actuator parameters. Then, using the PSO concept, the location of each particle was updated to find better positions for a higher LF.

The remainder of this paper is organized as follows. In Section 2, we propose a PSO-based optimization algorithm to determine the maximizer of the LF. Section 3 presents the results using the proposed algorithm. Finally, the conclusions are presented.

In this section, we describe the PSO algorithm to localize electrostatic vibration actuators. The model equation is described in Subsection A. Next, the FDM-based solution reconstruction algorithm is presented in Subsection B. Finally, the PSO algorithm is proposed in Subsection C.

A. Modeling of the Display Panel Under the Effects of Electrostatic Vibration Actuators

To analyze the vibrating plates, we employ the KL plate theory [11-12]. We consider a rectangular-shaped plate Ω = [0, L] ⊂ ${ℝ}^2$ under the excitation f (x, t) by the electrostatic vibration actuators. Then, the transverse displacement denoted by u (x, t): Ω×$ℝ$→$ℝ$ is calculated using the governing equation

Here, D = EH³/12(1 − ν²) is the flexural rigidity, where E is the modulus of elasticity, ρ is the density, H is the thickness of the plate, c is the damping parameter of the plate and ν is the Young’s modulus. We consider four actuators attached to the plate at Ω_i, (i = 1, ..., 4) having different magnitudes and phases (see Fig. 1). Hence, the excitation generated by the actuator systems of frequency f is described as

Figure 1. Illustration of a large display with four actuators. The actuators are attached to the display on Ω_i (i = 1, ..., 4). The quadrants of the plate divided by mid-lines (dashed) are denoted by A₁, ..., A₄.

Here, the parameters w_i’s and φ_i’s are the magnitudes and phases of the actuators, respectively, which can be determined by the PSO optimization algorithm. For a well-posed system, the (homogeneous) boundary and initial conditions are imposed as:

Then, the governing equation (1) together with (3-4) yields a system with a unique solution.

The goal of this study is to localize the effects of actuators on the subregions of the display (see Fig. 1), which is denoted by A_i (i = 1, ..., 4). The haptic LF in region A_i (i = 1, ..., 4) is defined by the relative kinetic energies of the subregion. Here, the kinetic energy on subset D ⊂ $ℝ$ at time t is defined by

In summary, the LF on A_i (i = 1, ..., 4) is expressed mathematically as

From equation (5), it is clear that 0 ≤ LF_i ≤ 1. Here, time integration is performed on [T, T₀] to obtain the average kinetic energy, where the values T and T₀ are determined as presented in the Results section. Without loss of generality, we can fix i = 1 and drop the sub-index i in LF_i. Our optimization algorithm can be applied to the other subregions in the same manner.

Now, the optimization problem for localization can be stated as:

Problem 1. Find w_i, φ_i (i = 1, ..., 4) that maximizes the LF.

B. FDM based Prediction of Solution

As it is difficult to obtain analytical solutions of equation (1), we employed the FDM algorithm to solve it. The domain is triangulated by uniform rectangles of size h to yield N × N nodes, i.e.,

0 = x₁ < x₂ < ... < x_n = L,

0 = y₁ < y₂ < .. .< y_n = L.

The trial function in the discretized space is denoted by u_h. The forward Euler approach is employed for time discretization. The time step is denoted by Δt. The nodal values u_h (x_i y_j, nΔt) are denoted by $u_{i,j}^n$. Similarly, f (x_i y_j, nΔT) is denoted by $f_{i,j}^n$. When there is no worry of confusion, we drop superscript n in $u_{i,j}^n$.

Now, we need to approximate the operator ∇²∇²u_h = u_xxxx + 2u_{xx, yy} + u_yyyy in equation (1) on the internal points of the domain. The fourth order derivatives and ∇²∇²u_h at (x_i y_j) can be numerically approximated as in [15]:

Now, the forward FDM algorithm is expressed as

Equation (6) is repeated with increasing n indexes up to N to produce numerical solutions up to the target time T^target = NΔT. Once the solution is generated by the FDM, the LF can be numerically evaluated by equation (5), which is denoted as LF_wi,φi to emphasize the dependency with respect to the parameters.

C. Particle Swarm Optimization-Based Haptic Localization Algorithm

To solve optimization Problem 1, we employ particle swarm optimization (PSO) [8,9]. While the dimension of the parameters in Problem 1 is eight (four weights and four phases), we can reduce the dimension to six by fixing w_i = 1 and φ₁ = 0. This is valid because 1) the governing equation (1) is linear in u, and 2) the excitation function in (2) is periodic with respect to t.

Although an objective of this study is to determine the parameters w₂, w₃, w₄, ϕ₂, ϕ₃, ϕ₄ to maximize the LF, the PSO algorithm is a minimization-type algorithm. Therefore, we modify the optimization problem (with six parameters) slightly as:

Problem 2. Find w_i, φ_i (i = 2, ..., 4) that minimizes the loss function Λ(w_i, φ_i) = 1 − LF_wi,φi .

Regarding this new objective, we provide a brief remark regarding the equivalence of Problem 1 and Problem 2:

Remark.

As 0 ≤ LF_wi,φi ≤ 1, minimizing Λ(w_i, φ_i) in the interval [0,1] is equivalent to maximizing LF_wi,φi .

The PSO algorithm for Problem 2 is as follows. First, n randomly generated particles ($X_1^i$, i = 1, ..., n) and their velocities ($V_1^i$, i = 1, ..., n) are defined. Here, the particle $X_1^i$ contains parameters ($w_2^i,w_3^i,w_4^i,{\varphi }_2^i,{\varphi }_3^i,{\varphi }_4^i$) and $V_1^i$ that are sampled from the uniform distribution in [-2,2]. The main idea of PSO is to evolve the locations of the particles to determine the best maximizing position for the LF. Once random sampling is generated, the personal best vectors (Pbests) are initialized with these particles, i.e.,

Now, each particle is substituted in the loss function to find the particle with the lowest loss function (Gbest) amongⁿ particles, i.e.,

Then, the particles and velocities are updated recursively as (l = 1, 2, ...)

Equation (8) determines the updated locations of the particles with respect to the velocities. The velocities are updated using equation (7), where Pbests and Gbest are used to correct the current velocities with cognitive (c₁ > 0) and social weights (c₂ > 0). Here, U(0, 1) represents a uniform distribution in the interval [0,1]. Also, the so-called inertia parameter x > 0 and decaying parameter 0 < κ < 1 are used to prevent sudden change of velocities in iterations. At the end of each iteration, Pbests and Gbest are updated to save the best position of each particle and the global best position, respectively.

We state the optimization algorithm for haptic localization with Nit number of iterations:

In this section, the capability of the PSO algorithm is demonstrated for haptic localization. First, the parameters of the governing equations are described. The domain is Ω = [ 0, 0.2 m]² and the locations of the actuators are

Ω₁ = [0.03 m, 0.04 m] × [0.03 m, 0.042 m]

Ω₂ = [0.03 m, 0.042 m] × [0.158 m, 0.17 m]

Ω₃ = [0.158 m, 0.17 m] × [0.03 m, 0.042 m]

Ω₄ = [0.158 m, 0.17 m] × [0.158 m, 0.17 m]

The physical properties of the plates are E = 1 G Pas, ρ = 10³ kg/m³, H = 0.5 mm, ν = 0.33, and f = 290 Hz. For the FDM simulations, 27 × 27 nodal points were used with a time-step ΔT = 8·10⁻⁵ The time parameters in equation (5) were T = 0.1s with T₀ = 0.05s. Finally, we used twenty particles for the PSO algorithm with Nit = 80. The inertial, cognitive and social weights were set to w = 4, c₁ = c₂ = 2 with decaying parameter κ = 0.99.

For comparison, let us consider parameters with the same magnitudes and phases assigned to each actuator, i.e.,

w_i = 1, φ_i = 0, i = 1, ..., 4

The graphs of u_h obtained from the FDM simulations with the above choices are shown in Fig. 2(a). It can be observed that the displacements are symmetrical. As expected, the LF is 0.25, indicating that all subregions vibrate equally.

Figure 2. Graphs of u_h for the three cases. (a) w_i = 1, φ_i = 0, i = 1, ..., 4, (b) w₁ = 1, w₂ = w₃ = w₄ = 0. (c) Parameters obtained by PSO. For all the cases, the left and right figures correspond to the solution at t = 0.0464 s and t = 0.0696 s, respectively.

Now, we intend to localize region A₁ by controlling the actuator parameters. A naïve method is to activate an actuator only in region A₁, i.e.,

w₁ = 1, w₂ = w₃ = w₄= 0.

As shown in Fig. 2 (b), the bottom-left side has a high magnitude for u_h whereas the other regions have a small magnitude. In this case, the LF is 63.67%, indicating the need for an optimization process to enhance the localization.

Finally, we report the optimizing factors obtained by the PSO algorithm. The loss function with respect to the number of iterations is shown in Fig. 2. The cost function gradually decreases as the number of iterations increases. In the last iteration, the LF is 72%. Considering the area of A₁ (25% of the whole plate), the result of 72% of the kinetic energy being focused on A₁ is meaningful. The actuator parameters obtained by PSO are listed in Table 1.

Table 1 . Actuator parameters obtained by the PSO algorithm.

w1	1
w2	0.221
w3	0.221
w4	0.074
φ1	0 (radian)
φ2	0.5π (radian)
φ3	0.5π (radian)
φ4	0.596π (radian)

The graphs of u_h obtained using the PSO algorithm are plotted in Fig. 2(c). Compared with Fig. 2(b), we observe that the transverse displacement decreases on A₂, A₃, A₄, leading to localization on A₁. Therefore, the capability of the proposed algorithm to localize sensations using multiple actuators is verified.

For a large display panel, a small number of actuators leads to the appearance of a dead zone with no haptic sensation. In contrast, the traveling wave generated by the actuators must be controlled when there are multiple actuators to localize the sensations. In this study, we propose a PSO-based haptic localization method for large plates with four electrostatic vibration actuators of different magnitudes and phases. We modeled and simulated the behavior of the transverse displacement of the plates under the effects of actuators by employing a Kirchhoff-Love plate. The actuator parameters were controlled based on the parameters of the PSO algorithm. The transverse displacement generated by the actuators tuned by the PSO algorithm has a high amplitude only in the targeted quadrant, leading to the desired localization effect. Compared with previous studies (e.g., [7]), our algorithm robustly updates the locations of the actuators because the excitation function can be changed easily.

We believe that this PSO algorithm can be easily extended to various haptic localizations for other types of devices such as small panels in mobile devices or bar-shaped display panels. In addition, our methods can be extended to various localization-sensing applications such as interactive medical learning and interactive learning.

Figure 3. A plot of the cost function with respect to PSO iterations.

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIT) (No. 2020R1C1C1A01005396).