Services on Demand
Journal
Article
English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mailIndicators
-
Cited by SciELO -
Access statistics
Related links
-
Cited by Google -
Similars in
SciELO -
Similars in Google
Share
Permalink
Revista Colombiana de Estadística
Print version ISSN 0120-1751
Rev.Colomb.Estad. vol.46 no.2 Bogotá July/Dec. 2023 Epub July 12, 2023
https://doi.org/10.15446/rce.v46n2.104019
Original articles of research
An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces
Un método adaptativo para optimizar la función de verosimilitud en modelos lineales mixtos bajo espacios de búsqueda restringidos
1 School of Statistics, Faculty of Sciences, Universidad Nacional de Colombia, Medellín, Colombia
Linear mixed effects models are highly flexible in handling correlated data by considering covariance matrices that explain variation patterns between and within clusters. For these covariance matrices, there exist a wide list of possible structures proposed by researchers in multiple scientific areas. Maximum likelihood is the most common estimation method in linear mixed models and it depends on the structured covariance matrices for random effects and errors. Classical methods used to optimize the likelihood function, such as Newton-Raphson or Fisher's scoring, require analytical procedures to obtain parametrical restrictions to guarantee positive definiteness for the structured matrices and it is not, in general, an easy task. To avoid dealing with complex restrictions, we propose an adaptive method that incorporates the so-called Hybrid Genetic Algorithms with a penalization technique based on minimum eigenvalues to guarantee positive definiteness in an evolutionary process which discards non-viable cases. The proposed method is evaluated through simulations and its performance is compared with that of Newton-Raphson algorithm implemented in SAS® PROC MIXED V9.4.
Key words: Hybrid genetic algorithm; Linear mixed model; Optimization; Positive definite matrices
Los modelos lineales mixtos son muy flexibles cuando se trabaja con datos correlacionados ya que estos consideran matrices de covarianza que explican los patrones de variación entre individuos y dentro de sus observaciones. Para estas matrices de covarianza existe una amplia lista de posibles estructuras propuestas por investigadores en múltiples áreas científicas. El método de máxima verosimilitud es el más común para la estimación de los parámetros en modelos lineales mixtos y depende de las matrices de covarianza estructuradas para efectos aleatorios y errores. Los métodos clásicos utilizados para optimizar la función de verosimilitud, como Newton-Raphson o Fisher's scoring, requieren desarrollos analíticos para obtener restricciones sobre los parámetros que garanticen matrices estructuradas y definidas positivas, y en general, esto no es una tarea fácil. Para evitar lidiar con restricciones complejas, proponemos un método adaptativo que incorpora los llamados Algoritmos Genéticos Híbridos con una técnica de penalización basada en valores propios mínimos con el fin de garantizar matrices positivas definidas en un proceso evolutivo que descarta casos no viables. El método propuesto se evalúa a través de simulaciones y se compara su desempeño con el algoritmo de Newton-Raphson implementado en SAS® PROC MIXED V9.4.
Palabras clave: Algoritmo genético híbrido; Modelo lineal mixto; Optimización; Matrices definidas positivas
Acknowledgments
We thank the school of statistics from the Universidad Nacional de Colombia at Medellín for its constinuous support to research initiatives. This work was partially supported by Colombian Institute for Science and Technology (Colciencias) Scholarship Program No. 647.
References
Box, G., Jenkins, G. & Reinsel, G. (1970), Time Series Analysis: Forecasting and Control, fourth edn, John Wiley and Sons, New Jersey. [ Links ]
Chehouri, A., R. Younes, J. P. & Ilinca, A. (2016), ‘A constraint-handling technique for genetic algorithms using a violation factor’, Journal of Computer Science 12(7), 350-362. [ Links ]
Coello, C. (2002), ‘Theoretical and Numerical Constraint-Handling Techniques used with Evolutionary Algorithms: A Survey of the State of the Art’, Computer Methods in Applied Mechanics and Engineering 191, 1245-1287. [ Links ]
Coley, D. A. (1998), Introduction to genetic algorithms for scientists and engineers, first edn, World scientific, River Edge, NJ, USA. [ Links ]
Demidenko, E. (2004), Mixed models: Theory and applications with R, second edn, Wiley, New Jersey. [ Links ]
Dempster, A., Laird, N. & Rubin, E. (1977), ‘Maximum likelihood from incomplete data via the em algorithm’, Journal of the Royal Statistical Society 39(1), 1-38. [ Links ]
El-Mihoub, T. A., Hopgood, A. A., Nolle, L. & Battersby, A. (2006), ‘Hybrid Genetic Algorithms: A review’, Engineering Letters 13, 124-137. [ Links ]
Henderson, C. R. (1984), Applications of linear models in animal breeding, Technical Report Press, University of Guelph, Guelph, Canada. [ Links ]
Holland, J. H. (1975), Adaptation in Natural and Artificial Systems, first edn, University of Michigan Press, Ann Arbor. [ Links ]
Kuri-Morales, A. F. & Gutiérrez-García, J. (2002), ‘Penalty Function Methodsfor Constrained Optimization with Genetic Algorithms: A Statistical Analysis’, MICAI 2002: Advances in Artificial Intelligence 2313, 108-117. [ Links ]
Laird, N. M. & Ware, J. H. (1982), ‘Random-effects models for longitudinal data’, Biometrics 38, 963-974. [ Links ]
Lin, C. (2013), ‘A rough penalty genetic algorithm for constrained optimization’, Information Sciences 241, 119-137. [ Links ]
Lindstrom, M. J. & Bates, D. M. (1988), ‘Newton-Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measures Data’, Journal of the American Statistical Association 83, 1014-1022. [ Links ]
Madar, V. (2015), ‘Direct formulation to Cholesky decomposition of a general nonsingular correlation matrix’, Statistics & Probability Letters 103, 142-147. [ Links ]
Mebane, W. R. J. & Sekhon, J. S. (2011), ‘Genetic optimization using derivatives: The rgenoud package for r’, Journal of Statistical Software 42(11), 1-26. https://www.jstatsoft.org/v42/i11/ [ Links ]
Michalewicz, Z. (1998), Genetic algorithms + data structures= evolution programs, second edn, Springer-Verlag, Berlin Heidelberg. [ Links ]
Munkres, J. R. (2000), Topology, second edn, Prentice Hall, Upper Saddle River. [ Links ]
Patterson, H. D. & Thompson, R. (1971), ‘Recovery of inter-block information when block sizes are unequal’, Biometrika 58, 545-554. [ Links ]
Pinheiro, J. C. (1994), Topics in Mixed Effects Models, PhD thesis, University of Wisconsin-Madison, USA. [ Links ]
Pinheiro, J. C. & Bates, D. M. (1996), ‘Unconstrained parametrizations for variance-covariance matrices’, Statistics and Computing 6, 289-296. [ Links ]
Pothoff, R. & Roy, S. (1964), ‘A generalized multivariate analysis of variance model useful especially for growth curve problems’, Biometrika 51, 313-326. [ Links ]
Rao, C. (1972), ‘Estimation of variance and covariance components in linear models’, Journal of the American Statistical Association 67(337), 112-115. [ Links ]
Reeves, C. & Rowe, J. E. (2002), Genetic algorithms: Principles and perspectives, first edn, Springer, New York. [ Links ]
SAS Institute Inc. (2008), SAS/STAT® 9.2 User’s Guide. The MIXED Procedure (Chapter). Cary, NC: SAS Institute Inc. [ Links ]
Schott, J. R. (1997), Matrix analysis for statistics, third edn, Wiley, New Jersey. [ Links ]
Scrucca, L. (2017), ‘On some extensions to ga package: hybrid optimization, parallelisation and islands evolution’, The R Journal. 9(1), 187-206. [ Links ]
Verbeke, G. & Molenberghs, G. (1997), Linear Mixed Models in Practice- A SAS-Oriented Approach, first edn, Springer, New York. [ Links ]
West, B., Welch, K. & Galecki, A. (2006), Linear Mixed Models-A practical guide using statistical software, second edn, Chapman and Hall/CRC, London. [ Links ]
Wolfinger, R. (1993), ‘Covariance structure selection in general mixed models’, Comunications in Statistics - Simulation and Computation 22, 1079-1106. [ Links ]
Wolfinger, R., Tobias, R. & Sall, J. (1994), ‘Computing gaussian likelihoods and their derivates for general linear mixed models’, SIAM Journal on Scientific Computing 15(6), 1294-1310. [ Links ]
Received: July 2022; Accepted: February 2023
This is an open-access article distributed under the terms of the Creative Commons Attribution License
