Real and/or Complex Roots Calculation of Nonlinear Equations Systems through Modified Particle Swarm Optimization1
Cálculo de raíces reales y/o complejas de sistemas de ecuaciones no lineales mediante el método de enjambre de partículas modificado2
Cálculo de raízes reais e/ou complexas em sistemas de equacoes nao lineares pelo método de enxame de partículas modificado3
Sergio Reyes-Sierra4
Julián Plata-Rueda5
Rodrigo Correa-Cely6
1This article is derived from a research modality graduation paper, as part of the Simulation Tools Development Project. Internal financing. Developed by the research group Control, Electronics, Modelling andSimulation, CEMOS of Universidad Industrial de Santander, Bucaramanga, Colombia.
2Fecha de recepción: 12 de marzo de 2011. Fecha de aceptación: 15 de febrero de 2012. Este artículo se deriva del proyecto de Trabajo de grado modalidad investigación como parte del proyecto de desarrollo de herramientas de simulación. Financiación interna. Desarrollado por el grupo de investigación Control, Electrónica, Modelado y Simulación (CEMOS), de la Universidad Industrial de Santander, Bucaramanga, Colombia.
3Data de recebimento: 12 de março de 2011. Data de aceite: 15 de fevereiro de 2012. Este artigo é derivado do projeto de Trabalho de graduação modalidade pesquisa como parte do projeto de desenvolvimento de ferramentas de simulação. Financiamento interno. Desenvolvido pelo grupo de pesquisa Controle, Eletrônica, Modelado e Simulação (CEMOS), da Universidade Industrial de Santander, Bucaramanga, Colombia. ]]>
Submitted on: March 11,2011. Accepted on: February 15,2012.
Resumen
En este artículo se describe una alternativa numérica para solucionar sistemas de ecuaciones no lineales con raíces reales y/o complejas. Para ello se convirtió el problema de solución directa de tales sistemas en un problema de optimización, y se resolvió utilizando el método de enjambre de partículas apropiadamente modificado para tal tarea. A título demostrativo se incluyen algunos resultados con sistemas de dos, cinco y diez ecuaciones, resueltos en un computador convencional y en un arreglo de cuatro nodos. Se concluyó que la estrategia es válida para solucionar este tipo de sistemas. Por otra parte, no se detectó una mejora en los tiempos de computación cuando se utilizó el cluster.
Palabras clave: Optimización mediante enjambre de partículas, sistemas de ecuaciones no lineales, metaheurística, cluster.
Abstract
In this article we describe a numerical alternative to sovlve systems of nonlinear equations with real and/or complex roots. The problem of solving directly such systems was transformed into an optimization one, which was solved using a specially modified particle swarm method. As an example, system of two, five and ten equations were solved using a conventional personal computer as well as a cluster of four nodes. It was concluded that this strategy is valid for solving this type of systems of equations. Moreover, using the cluster no computational time improvement was detected.
Key words: Particle swarm optimization, nonlinear system of equations, metaheuristic, cluster
Este artigo descreve uma alternativa numérica para resolução de sistemas de equações não lineares com raízes reais e/ ou complexas. Para isso, o problema de solução direta de tais sistemas virou-se para um problema de otimização e resolveu-se utilizando o método de enxame de partículas devidamente modificado para esta tarefa. A maneira de demonstração incluem-se alguns resultados com os sistemas de dois, cinco e dez equações, resolvidos em computador convencional e num arranjo de quatro nós. Concluiu-se que a estratégia é válida para resolver este tipo de sistema. Além disso, nenhuma melhoria foi detectada nos tempos de computação quando o cluster foi utilizado.
Palavras-chave: Utilização mediante enxame de partículas, sistemas de equações não lineares, meta-heurística, cluster.
Introduction
Given the difficulty for representing physical phenomena through one or more non-linear systems of equations (NSE), it is necessary to have several analytic and/or numerical methods available for solving them (Grosan and Abraham, 2008; Bianchini et al., 2001; Floudas, 1999; Ortega and Rheinboldt, 1970). Some of these systems do not have a single root (solution), so finding them becomes a mathematical and computational challenge, which is, as of today, an open research topic. In some cases, algebraic approaches can be used to find some of them. However, most NSE are too difficult to solve in this way, so numerical approaches are used to obtain approximate solutions, with a given error margin (Luo et al., 2008). One of the most used methods is multidimensional Newton Raphson (NR). In spite of its simple algorithm and high convergence speed, its main weakness is that it is dependent on a user-defined starting point, which needs to be close to the desired solution, meaning that it requires a previous knowledge of the solution. When a system with several variables and relatively complex equations is to be solved, the near-solution starting point becomes almost impossible to guess. Another traditional approach is the method of the gradient. This, however, requires that the system complies with a differentiability condition, which can prove to be difficult to achieve. Through past years, the so-called evolutionary algorithms have become a powerful choice for numerical solution (Geng et al., 2009; Hatanaka et al., 2004; Yang et al.,2008). When the NSE gets bigger, traditional approaches become excessively difficult and non-practical, increasing the computational cost, thus being necessary to use more recent methods (Brits et al., 2002; Hui and Zhao, 2008; Grosan and Abraham, 2008; Cui and Cai, 2010).
During the past decades, the metaheuristic optimization approaches (based on the imitation of natural, biologic, social or cultural processes) have been successfully applied to several optimization problems on engineering. One of these techniques is Particle Swarm Optimization (PSO), which has a high ability to efficiently explore multidimensional search spaces (Aijia et al, 2009; Rao, 2009; Clerc, 2006), and which has been recently applied to the current problem, e.g. (Tsoulos et al, 2010; Ouyang et al, 2009).
This article describes the development of an algorithm that allows solving NSE with real and/or complex roots, based on PSO. Striving to lower the computation time and the required memory, a theorem is presented that allows the transformation of a finding roots problem into an optimization one, and its demonstration is presented later on. Moreover, some results for a couple of demonstrative examples, using a traditional computer and a four node cluster, are shown.
1. Fundamentals
An NSE with m functions and m unknowns is assumed, as shown by equation (1), where F(z1 ,z2 ,...,zi), i = 1,2,...,m are the non-linear equations, while zi , i = 1,2,...,m, are the complex variables.
Theorem 1: Let C be the set of complex numbers, X a subset of C" and consider the following system of equations:
Where, for each i, fi. is a function whose domain contains X, and its range belongs to the complex numbers. Let f: X → R be defined as:
With z = (z1 ,z2 ,...,zn) and where ||c|| represents the magnitude of the complex number c, c is its complex conjugate and ||c||2 = cc. Note that f is properly defined, and, besides, the images of the function are non-negative real numbers. Thus, it follows that:
Proposition 1, Suppose that the system (2) has a solution in X, and let α = α1, α2, ..., αn) ∈ X. Therefore:
α satisfies (2) if, and only if, α minimizes f
Proof. If α satisfies (2) then fi(α) = 0 for every i = 1,2,...,m. Thereforef (a) = 0 and, since f(z) ≥ 0 for all z ∈ X, then α is a minimum for f
Now, if αminimizes f but does not satisfies (2) then f (α) must be positive, sincef (z) ≥ 0 for all z ∈ X. Since the system has a solution in X, there is a z* e X such that f(z*) = 0 y z*≠ α (X. Therefore, f (z*) <f (α) which violates (X being a minimum for f. Note the importance on the general consistency condition over the system (2) in X, since given a system of equations, it is always possible to build f and, if (X minimizes it, it does not generally imply that the system has a solution. Therefore, the problem of finding an NSE roots in a given set X, can be transformed into an optimization problem (minimization for this case), for a function f (built as previously shown) in the set X. An algorithm based on the previously stated is:
]]> Algorithm 1.It is important to remark that it is not mandatory to use the squared magnitudes of each expression on the NSE; the magnitude itself can be used without any problem. A brief fundamentals of metaheuristic optimization algorithms is presented below.
Particle Swarm Optimization (PSO): this stochastic, adaptive optimization technique was developed by Eberhart and Kennedy in 1995. It is directly related to the ability that have individual members of a group to gain and share knowledge of their surroundings. From the analysis of this behavior, a simple mathematical model was obtained, which reveal its potential to solve optimization problems (Rao, 2009). This algorithm generates a population with a given initial speed and position, which relates to a possible solution. For an N-dimensional problem, the position and speed are given by M x N matrices, as shown by equations (4) and (5).
Where X and V are the position and speed matrices, respectively. The population is given by a set of M particles. Each row in the X matrix represents the position of a particle in the search space (Rao, 2009; Clerc, 2006). Each iteration, the particles memorize and follow their best position (Pbest), and the whole population best position (Gbest), to update the speed matrix. Pbest is defined as shown by equation (6).
With these results, the position and speed of the particles can be updated, using equations (8) y (9).
Where i = 1,2,...,M and j = 1,2,...,N; iter is the current iteration; w is an inertia factor that regulates the effect of previous speeds into the new one; C1 is a self-trust factor, while C2 relates to the social trust. Rand( ) and rand ( ) are functions that return an uniformly distributed random number between [0,1], and they weigh in the regulations of the individual and social information for each particle.
In order to apply the algorithm, some steps need to be followed (Rao, 2009, Parsopoulos and Vrhatis, 2010):
Algorithm modification: In order to calculate complex roots, it was necessary to modify the original algorithm, specifically in the update of speeds (8). The new equation is given by (10).
Where real(...) is the real part and imag(...) the imaginary one. These particles allow for movement in the whole complex plane.
2. Experiments
Experiments were carried out in both, a regular computer and a four node cluster, with the following specifications:
3. Results and Analysis
Following are some results, related to the solution of a NSE of two, five and ten equation, striving to show the method validity. Some other tests were also performed, but due to space limitations are not shown. Given these systems complexity, no analytical tool foretells how many real and/or complex roots they have, in case they even exist. Table 1 shows the real roots for the NSE (11).
There, a set of solutions that satisfy the system is shown. The first couple is real and the rest is imaginary. For this case, 1000 particles were used, and the solution was found in a relative short time.
3.1. Five equations system
The NSE (12) was implemented in computer C1; some results are shown in tables 2 and 3. Only the complex roots are reported.
It was observed that as the number of particles goes up, the number of required iterations goes down. In a similar fashion, if the particles are too spread at the beginning, it will take more iterations than if they were closer. Unlike deterministic algorithms (Floudas, 1999), this kind of approaches generates a different set of solutions for each run (in case the system has a solution). Should a given solution be required to improve, the search space can be tightened.
3.2. Ten equations system
As in the previous case, NSE (13) was implemented in C1. Table 4 summarizes the results for 100 particles.
It is observed that by increasing the size of the NSE, as well as the number of particles, the required computer resources increase almost exponentially. For every case, the number of iterations was in the same range, even though they are inversely proportional to the number of particles. Therefore, the following experiments were performed on C2.
3.3. System of five equations in the cluster
NSE (12) was implemented in C2 and the results are summarized in tables 5 and 6.
4. Observations and Conclusions
It is evident that it is of great help to transform a problem of solution of a NSE into an optimization one, especially for finding its real and complex roots. Even though this strategy is well known in literature, a mathematical demonstration is provided here. The modified PSO used during this research, is an efficient optimization method that can be easily implemented to find the solution of these systems.
For the cases studied here, the fact of increasing the amount of particles, increased the run time, but a general correlation was not found. Moreover, and because it is a stochastic method, results are not expected to be fully repeatable. However, the solutions found by this algorithm could be feed to traditional deterministic ones.
A proper choice of the amount of particles, as well as their initial position, provided an increased convergence speed. Regarding the cluster, it is not enough to simply have the hardware, storage and networking resources, but the algorithm has to be parallelizable, which could not be done here. It was found too that the convergence speed depends on both, the size of the NSE and the complexity of its equations. A ten equation system was tried to solve in the cluster, but due to the high RAM requirements, the program did not run.
References
AIJIA, O.; YONGQUAN, Z. and QIFANG, L. Hybrid particle swarm optimization algorithm for solving systems of nonlinear equations. Nanning, China: Guangxi University for Nationalities, Granular Computing, 2009, GRC '09. [ Links ]
BIANCHINI, M.; FANELLI, S. and GORI, M. Optimal algorithms for well-conditioned nonlinear systems of equations. IEEE Trans On Computers. 2001, vol. 50, no. 7, pp. 689-698. [ Links ]
BRITS, R.; ENEGELBRECHT, A. and VAN DEN BERGH, F. Solving systems of unconstrained equations using particle swarm optimization. Pretoria, South Africa: Department of Computer Science, University of Pretoria, 2002. [ Links ]
CLERC, M. Particle swarm optimization. 1st Edition. ISTE, 2006. Chap. 3 [ Links ]
CUI, Z. and CAI, X. Using social cognitive optimization algorithm to solve nonlinear equations. Shanxi, China, 2010. Proc. 9th IEEE Int. Conf. On Cognitive Informatics, June 2005. pp. 199-203. [ Links ]
FLOUDAS, C. Deterministic global optimization. Kluwer Academic Publisher, 1999. Chap. 12. [ Links ]
GENG H. T.; SUN Y. J.; SONG Q. X. et al. Research of ranking method in evolution strategy for solving nonlinear system of equations. The 1st International Conference on Information Science and Engineering (ICISE2009), IEEE Computer Society, pp. 348-351. [ Links ]
]]>GÓMEZ, L. Propuesta de demostración del teorema sobre la relación entre sistemas de ecuaciones y el problema de optimización (Comunicación interna UIS. Nov 11, 2010). [ Links ]
GROSAN, C. and ABRAHAM, A. A new approach for solving nonlinear equations systems. IEEE Trans. On Systems and Cybernetics-Part A: Systems and Humans. 2008, vol. 38, no. 3, pp. 698-713. [ Links ]
GROSAN, C. and ABRAHAM, A. Multiple solutions for a system of nonlinear equations. International]. of Innovative Computing. Information and Control, ICIC. 2008, pp. 76-82. [ Links ]
HATANAKA, T; UOSAKI, K. and KOGA, M. Evolutionary computation approach to block oriented nonlinear model. Control Conference. 2004. 5th Asian, pp. 90-96. [ Links ]
HUI, W and ZHAO, Z. A neural network algorithm for solving systems of nonlinear equations. Changsha, Hunan, China: College of Electrical & Information Engineering, University of Science & Technology, 2008. [ Links ]
]]>LUO, Y. Z.; TANG, G. J. and ZHOU, L. N. Hybrid approach for solving systems of nonlinear equations using chaos optimization and quasi-Newton method. Applied Soft Computing. 2008, 8, pp. 1068-1073. [ Links ]
ORTEGA, J. and RHEINBOLDT, W Iterative solution of nonlinear equations in several variables. New York: Academic Press, 1970. [ Links ]
OUYANG, A.; ZHOU, Y and LUO, Q. Hybrid particle swarm optimization algorithm for solving systems of nonlinear equations. Granular Computing, 2009, GRC '09. pp.46-465. [ Links ]
PARSOPOULOS, K. and VRHATIS, M. Particle swarm optimization and intelligence: advances and applications. Ed. Information Science Reference, 2010, Chaps 1-4. RAO, S. Engineering optimization, theory and practice. 4th edition. Ed. John Wiley & Sons, 2009. Chap. 2-13. [ Links ]
TSOULOS, I. and STAVRAKOUDIS, A. On locating all roots of systems of nonlinear equations inside bounded domain using global optimization methods. Nonlinear Analysis: Real World Applications. 2010, vol. 11, no. 4, pp 2465-2471. [ Links ]
]]>YANG, B.; ZHANG, Z. and SUN Z. Computing nonlinear lst estimator based on a random differential evolution strategy. Tsinghua Science and Technology. 2008, vol. 13, no.1, pp.1007-0214. [ Links ]
]]>