Identification of continuous-time Hammerstein systems by simultaneous perturbation stochastic approximation
Introduction
The modelling of real-world plants and processes, which are nonlinear in nature, remains a challenging problem. Both an expert and intelligent systems are therefore required to model accurately such plants and processes. One way to cope with this difficulty is to introduce identification of block oriented models. These models include a Hammerstein model (a static nonlinear subsystem followed by a linear subsystem), a Wiener model (a linear subsystem followed by a static nonlinear subsystem), or a Hammerstein–Wiener model (a linear subsystem sandwiched by two static nonlinear subsystems or vice-versa). In particular, an intelligent system, such as a system with a neural feed-forward controller, is modeled as a Hammerstein model. These models have been adopted by many researchers partly because they closely reflect actual nonlinear systems with relatively simple structures. As a result, these models have been successfully used to describe many practical plants, such as fuel cells (Li, Zhu, Cao, Sui, & Hu, 2008), valve actuators (Wang & Zhang, 2014), wind turbines (van der Veen, van Wingerdeen, & Verhaegen, 2013), spark ignition engine torques (Togun, Baysec, & Kara, 2012), and stirred tank reactors (Shi, Xu, & Dai, 2011).
Among various types of nonlinear system models, the Hammerstein model is quite popular. In fact, the identification of Hammerstein systems has been widely reported in the literature (Bai, 2004, Bai, Li, 2004, Ding, Chen, 2005, Ding, Liu, Liu, 2011, Ding, Shi, Chen, 2006, Ding, Shi, Chen, 2007b, Greblicki, 2000, Hasiewicz, Mzyk, 2004, Liu, Bai, 2007, Pawlak, 1991, Zhao, 2006). Over the past two decades, various methods for identification of Hammerstein systems have been studied extensively. These can be roughly classified into several categories, such as the iterative method (Liu, Bai, 2007, Narendra, Gallman, 1966, Rangan, Wolodkin, Polla, 1995, Stoica, 1981, Voros, 1997), the over-parameterization method (Chang, Luus, 1971, Ding, Chen, Iwai, 2007a, Hsia, 1976), the blind approach (Bai & Fu, 2002), the subspace method (Verhaegen & Westwick, 1996), the least squares method (Ding, Chen, 2005, Goethals, Pelckmans, Suykens, Moor, 2005), the parametric instrumental variables method (Laurain, Gilson, Garnier, 2009, Stoica, Soderstrom, 1981), the stochastic method (Bilings, Fakhouri, 1978, Greblicki, 1996, Pawlak, 1991) and the non-parametric identification method (Bai, 2003, Greblicki, Pawlak, 1987, Krzyak, 1993, Krzyak, 1996).
Recently, a decomposition-based Newton iterative identification approach for a Hammerstein nonlinear FIR system with ARMA noise was presented by Ding, Deng, and Liu (2014). Here, it was claimed that a fast convergence rates with more accurate parameter estimation can be achieved by using the Newton iterative method. In Wang, Ding, and Ximei (2015), a hierarchical least squares method has been proposed for the identification of Hammerstein state space model. A similar approach is applied to Hammerstein nonlinear controlled autoregressive models (Chen & Ding, 2015). Both results show that the hierarchical identification principle may improve the computational efficiency by decomposing one nonlinear system into several subsystems with smaller dimensions and fewer variables. In Ma and Liu (2015), a nonlinear recursive instrumental variables (RIV) identification method for Hammerstein ARMAX system is adopted. The effectiveness of the RIV method is shown in terms of identification accuracy and convergence speed, especially under colored noise. Meanwhile, a blind approach with new over-sampling strategy was adopted in Yu, Zhang, and Xie (2014) to produce a consistent parameter estimation in the presence of noise.
In spite of such abundant literature, several restrictions are inevitable in their studies from the theoretical view-point.
- (i)
Most of the identification methods are restricted to the models in discrete-time, while it is natural to express a real system in a continuous-time domain.
- (ii)
Many approaches assume that the static nonlinear system is given by a linear combination of several basis functions.
- (iii)
In the over-parameterization identification method, the identification model contains the products of nonlinear and linear parameters, causing redundant parameter identification and a large computational load.
Though it looks that that we can handle general class of nonlinear subsystems by adopting so many basis functions (such as higher order polynomials and piecewise affine functions), this is not tractable in reality in the existing identification frame work.
On the other hand, there are different types of identification methods which utilize evolutional computation, such as the cuckoo search algorithm (Gotmare, Patidar, & George, 2015), the stochastic gradient (Chen, Wang, 2015, Mao, Ding, 2015), and the PSO (particle swarm optimization) (Jingzhuo, Juanping, Jingtao, Meiyu, Juwei, Lei, 2014, Ko, 2011, Nanda, Panda, Majhi, 2010, Wang, Ren, Liu, Han, 2014). These methods are quite flexible in nature, and do not suffer from (i) and (iii) mentioned above. In particular, the PSO is known to be effective in various systems control supplication (Maruta, Kim, Song, Sugie, 2013, Maruta, Kim, Sugie, 2009). However, they have a serious draw back.
- (iv)
In swarm based optimization (including PSO), the computation times per iteration are proportional to the number of swarms. As a result, these methods require heavy computation time in the identification process, especially for static nonlinear systems with a large number of basis functions.
Hence, it is not tractable to handle static nonlinear subsystems consisting of large number of basis. Consequently, these evolutional computation based methods cannot avoid (ii) as well.
To the best of the authors’ knowledge, the simultaneous perturbation stochastic approximation (SPSA) method (Spall, 1992) could be only candidate to provide us with a promising tool for such system identification problems. This is because the SPSA method is well known to be effective for a variety of optimization problems, even for high-dimensional parameter tuning (Ahmad, Azuma, & Sugie, 2014b). In comparison to the existing results, the SPSA method does not suffer from the afore-mentioned theoretical restrictions (i)–(iv). It may be expected to identify both linear and nonlinear subsystems simultaneously, even for large number of basis functions with less computational load. Meanwhile, one major drawback of SPSA may be to guarantee the local convergence only from the theoretical points of view.
Based on the above observations, this paper thus presents an identification method of Hammerstein systems in continuous-time using simultaneous perturbation stochastic approximation. We assume that the structure (i.e., the system order) of the linear subsystem is known in advance, while the structure of the nonlinear subsystem remains unknown. Here, a piecewise affine function is then used to approximate the unknown nonlinear function in the Hammerstein model. Next, based on the input and output data, the SPSA-based method is used to identify the coefficients of the linear time-invariant (LTI) model and the piecewise affine function simultaneously. In order to clarify the benefit of the SPSA-based method, a large number of parameters in the piecewise affine function are considered here. So far, there have been few papers discussing the identification of such Hammerstein models. Therefore, it is worth evaluating the effectiveness of the SPSA method.
The remainder of this paper is organized as follows. Section 2 formulates the identification problem for Hammerstein models. In Section 3, the identification method using simultaneous perturbation stochastic approximation-based algorithm is presented. The nonlinear function identification based on piecewise affine function is also described in the same section. The method is validated through several numerical examples in Section 4. Finally, some concluding remarks are given in Section 5. This paper is based on our preliminary version (Ahmad, Azuma, & Sugie, 2014a), published in a conference proceedings, and contains the full explanations and experiments omitted there.
Notation: The symbols and represent the set of real numbers and the set of positive real numbers, respectively. For the vector θ, we use ∥θ∥2 to express the standard Euclidean norm. For sat denotes the saturation function whose ith element given as follows: where is the ith element of .
Section snippets
Problem formulation
Consider the continuous-time single-input-single-output (SISO) Hammerstein model in Fig. 1, composed of a nonlinear function f and a linear dynamical system G described by the differential operator p (): Here, u(t) is the input, is the unmeasurable output of the nonlinear function, namely is the measurement of y(t) but is corrupted by the noise v(t). The input–output relationship is described as follows:
Identification method using simultaneous perturbation stochastic approximation
This section proposes a method to solve Problem 2.1. First, the standard SPSA algorithm (Spall, 1992) is briefly explained. An identification technique of the nonlinear function, based on piecewise affine function, is then described. Finally, the identification method is presented based on the SPSA-based algorithm.
Numerical examples
In this section, the effectiveness of the proposed identification method is demonstrated with several numerical examples.
Conclusion
In this paper, an identification method of continuous-time Hammerstein models has been presented, which is based on Simultaneous Perturbation Stochastic Approximation (SPSA). The simulation results demonstrate that the proposed method has a good potential to identify continuous-time Hammerstein models, even for high-dimensional parameter identifications. In particular, the method is shown to be effective in approximating both linear and nonlinear subsystems in the Hammerstein models from the
Acknowledgment
This work was partly supported by Grant-in-Aid for Scientific Research (A) 25249058 from the Ministry of Education, Culture, Sports, Science and Technology of Japan.
References (52)
- et al.
Performance analysis of model-free PID tuning of MIMO systems based on simultaneous perturbation stochastic approximation
Expert Systems with Applications
(2014) Decoupling the linear and nonlinear parts in hammerstein model identification
Automatica
(2004)- et al.
Identification of Hammerstein systems with continuous nonlinearity
Information Processing Letters
(2015) - et al.
Identification of Hammerstein nonlinear ARMAX systems
Automatica
(2005) - et al.
Identification methods for Hammerstein nonlinear systems
Digital Signal Processing
(2011) - et al.
Auxiliary model-based least-squares identification methods for Hammerstein output-error systems
Systems and Control Letters
(2007) - et al.
Identification of MIMO Hammerstein models using least squares support vector machines
Automatica
(2005) - et al.
Nonlinear system identification using a cuckoo search optimized adaptive Hammerstein model
Expert Systems with Applications
(2015) Integration of support vector regression and annealing dynamical learning algorithm for MIMO system identification
Expert Systems with Applications
(2011)- et al.
Identification of the Hammerstein model of a PEMFC stack based on least squares support vector machines
Journal of Power Sources
(2008)
Iterative identification of Hammerstein systems
Automatica
Synthesis of fixed-structure robust controllers using a constrained particle swarm optimizer with cyclic neighborhood topology
Expert Systems with Applications
Fixed-structure H-infinity controller synthesis: a meta-heuristic approach using simple constrained particle swarm optimization
Automatica
Improved identification of Hammerstein plants using new CPSO and IPSO algorithms
Expert Systems with Applications
Global identification of wind turbines using a Hammerstein identification method
IEEE Transactions on Control System Technology
Least squares algorithm for an input nonlinear system with a dynamic subspace state space model
Nonlinear Dynamics
A new deterministic identification approach to Hammerstein systems
IEEE Transactions on Signal Processing
Identification of continuous-time Hammerstein models using simultaneous perturbation stochastic approximation
Proceedings of the 14th international conference on control, automation and systems
Controller design for optimizing fuel consumption of hybrid electric vehicles: a model-free approach based on simultaneous perturbation stochastic approximation
Transactions of the Society of Instrument and Control Engineers
Frequency domain identification of Hammerstein models
IEEE Transactions on Automatic Control
A blind approach to Hammerstein model identification
IEEE Transactions on Signal Processing
Convergence of the iterative Hammerstein system identification algorithm
IEEE Transactions on Automatic Control
Identification of a class of nonlinear systems using correlation analysis
Proceedings of the Institution of Electrical Engineers
A noniterative method for identification using Hammerstein model
IEEE Transactions on Automatic Control
Hierarchical least squares identification for Hammerstein nonlinear controlled autoregressive systems
Circuits, Systems, and Signal Processing
Adaptive digital control of Hammerstein nonlinear systems with limited output sampling
SIAM Journal Control Optimization
Cited by (17)
Fast and optimal tuning of fractional order PID controller for AVR system based on memorizable-smoothed functional algorithm
2022, Engineering Science and Technology, an International JournalA hybrid metaheuristic algorithm for identification of continuous-time Hammerstein systems
2021, Applied Mathematical ModellingCitation Excerpt :To observe the efficiency of our proposed algorithm, the statistical results are also being compared with the conventional PSO, GWO, MVO and SCA. In addition, we also compare the statistical results with the existing work [31] which is based on Simultaneous Perturbation Stochastic Approximation (SPSA) method. Furthermore, the Wilcoxon's rank test results of the fitness function with the significance level of 5% is shown in Table 4.
Generalized spline nonlinear adaptive filters
2017, Expert Systems with ApplicationsCitation Excerpt :Estimating the mathematical model of a system based on input-output observations is a critical task in many engineering solutions (Ahmad, Azuma & Sugie, 2016; Ewen & Weiner, 1980; Garrido, Curadelli & Ambrosini, 2017; Soderstrom & Stoica, 1988) including satellite communication system (Panicker, Mathews & Sicuranza, 1998), acoustic echo cancellation (Comminiello, Scarpiniti, Azpicueta-Ruiz, Arenas-Garcȡa & Uncini, 2014), active noise control (Das & Panda, 2004) and feedback cancellation in digital hearing aids.
Identification of the transmission characteristics of a current transformer based on compressed sensing
2023, Beijing Huagong Daxue Xuebao (Ziran Kexueban)/Journal of Beijing University of Chemical Technology (Natural Science Edition)Two enhancement levels for male fertility rate categorization using whale optimization and pegasos algorithms
2022, Diverse Perspectives and State-of-the-Art Approaches to the Utilization of Data-Driven Clinical Decision Support SystemsMetaheuristics algorithms to identify nonlinear Hammerstein model: a decade survey
2022, Bulletin of Electrical Engineering and Informatics