ANÁLISIS DE MODELOS Y ARQUITECTURAS METACOGNITIVAS EN SISTEMAS INTELIGENTES
MANUEL FERNANDO CARO PIÑERES,
Ph. D.(c), Grupo de investigación: TIC aplicadas a la Educación EdupMedia, Universidad de Córdoba, mfcarop@unal.edu.co
JOVANI ALBERTO JIMÉNEZ BUILES
Ph. D., Grupo de investigación: Inteligencia Artificial en Educación, Universidad Nacional de Colombia, jajimen1@unal.edu.co
Received for review July 17th, 2012, Accepted March 26th, 2013, final version June, 15th, 2013
KEYWORDS: Artificial Intelligence, Metacognition, Metamemory, MetaComprehension, SelfRegulation.
RESUMEN: Los Sistemas Inteligentes (SI) han aumentado en gran medida la autonomía en la toma de decisiones, lo que se han logrado gracias a la mejora de las habilidades metacognitivas. El término metacognición en Inteligencia Artificial (IA) se refiere a la capacidad que tienen los SI para el seguimiento y control de su propio proceso de aprendizaje. Este artículo describe diferentes modelos utilizados para abordar la aplicación de la metacognición en los SI. Luego presenta un análisis comparativo entre los diferentes modelos de metacognición. Así como, una discusión sobre las siguientes categorías de análisis: tipos de arquitecturas de metacognición que soportan los componentes de la metacognición, núcleos de las arquitecturas e implementaciones computacionales.
PALABRAS CLAVE: Inteligencia artificial, Metacognición, Metamemoria, Metacomprensión, Autoregulación.
1. INTRODUCTION
In recent years in Artificial Intelligence (AI) robust and Intelligent Systems (IS) have been developed. IS have highly increased the autonomy of their decisions, which has been achieved by improving metacognitive capabilities.
The term metacognition in AI refers to the capability of IS to monitor and control their own learning processes [1,2]. The metacognitive capabilities of IS are known in the specialized literature as "metacognitive skills" [3,4]. Metacognition is composed of three elements or components [5]: metamemory, selfregulation and metacomprehension, in which all the metacognitive skills are grouped.
Metamemory is about the capabilities and strategies that systems have to control and monitor their own memory processes. This is called selfawareness of memory [4]. SelfRegulation refers to the system's ability to make adjustments of its own learning processes. These adjustments are done in response to system perception about the feedback obtained from its current status of learning [5]. Metacomprehension is a specific application of metacognition. It refers to the degree of understanding that an IS has about the information that is supplied to it. [6,7,8].
Due to the importance that metacognition has for improving the performance of IS, this study seeks to answer the following question What issues of metacognition have been modeled in IS?
]]> This paper is organized as follows: first a description of the more referenced metacognitive models in the specialized literature is given. Next, a summary table of the features about metacognition in these models is provided. Finally, we discuss categories of analysis and then present the main conclusions.
2. MATERIALS AND METHODS
The stages followed in the methodology of this research are described below.
2.1. Categories of analysis
To answer the research question, categories of analysis have been defined:
2.1.1 Types of metacognitive architectures
This category makes reference to the description of models, frameworks and architectures of metacognition, which have been proposed by several authors.
2.1.2 Support of metacognition components
This category is related to the support of metamemory, metacomprehension and selfregulation in metacognitive models.
2.1.4 Computational implementations
This category alludes to the computational techniques used for the implementation of IS based on metacognitive models.
2.1.5 Terminology
This refers to metacognition terminology that is used in metacognitive models.
2.2. Identification of models, frameworks and architectures of metacognition in artificial systems
As a result of the review of this literature, a series of models are described that have been referenced in the development of IS with metacognitive abilities.
2.2.1 Theoretical framework for the operation of human memory [8]
A first important contribution in this area was the Theoretical Framework for the Operation of Human Memory [8], see Figure 1, which introduced the three principle keys for metacognition: Cognitive processes can be divided into several levels. The metalevel contains a dynamic model of the objectlevel. There are two dominant relationships called control and monitoring.
Today, the twotier architecture is the basis for the architectural design of metacognition in IS. See Figure 1.
2.2.2 MetaAQUA [9,10]
Is a model based on the theory of Introspective Multistrategy Learning (IML) [10] and a cognitive model of introspection. The main functionality of the MetaAQUA system is the "story understanding", which is considered to be the base level in a MultiAgent System (MAS) [11]. The metalevel is structured by the implementation of IML [10], which in turn uses Case Based Reasoning (CBR) [12]. The learning strategy in MetaAQUA is implemented using a model of goaldriven learning (GDL) [13] and produces structures called metaexplanations.
2.2.3 CLARION theoretical framework [14]
Clarion theoretical framework is an overall architecture of the mind [14]. The architecture is used to construct models of specific metacognitive processes such as selfmonitoring and selfregulation (of cognitive processes). Clarion is used to capture experimental data related to metacognition with humans.
2.2.4 The MetaCognitive Loop (MCL) [15,16]
This is an architecture focused on detection of anomalies in learning processes and how to respond to them. MCL presents a general architecture and has three sets of ontologies [17], which are: ontology for anomaly types, failure ontology for use in assessment, and response ontology for selecting repair types to guide.
2.2.5 Simple model for metareasoning [18]
Cox and Raja [18] proposed a simple model for metareasoning. This model presents a double cycle of reasoning. The first cycle, refers to the traditional conception of cognitive science and AI about the reasoning in IS. Where an intelligent agent [11] receives perceptions of the environment, with which it makes decisions (reasoning), and acts, making changes to the environment. On the other hand, the second cycle of the simple model refers to the perception that the metalevel has about the objectlevel. The metalevel receives information from the objectlevel and make decisions (metareasoning).
2.2.7 Distributed metacognition Framework [20,21]
This is a conceptual architecture for distributed metacognition with contextawareness and diversity. A distributed metacognitive architecture is one in which all metalevel reasoning components can be monitored and controlled by other components of metalevel [20,21].
2.2.8 A metacognitive integrated dualcycle architecture (MIDCA) [22]
MIDCA is a novel architecture that incorporates both a perceptionaction cognitive cycle and a monitorcontrol metacognitive cycle [22]. In the metalevel the agent recognizes the problem, explains what causes the problem, and generates a new goal to remove the cause [9]. The metalevel reasoner can change the goals, the processes, and the input. MIDCA is based on the previous work by Norman [23], who designed a cognitive architecture.
2.2.9 Metalevel control agent architecture (Framework) [11]
The architecture is centered on how to make effective metalevel control decisions and has been implemented in MAS. This architecture is one of the precursors to distributed metacognition. The metalevel control uses an abstract representation of the agent state. The decisionmaking process is supported by decision trees.
2.3 Comparison between models
A comparison table (Table 1) is shown below as a source of information to supplement the description of the metacognitive models.
In Table 1 five main issues are presented which are used as the basis for the analysis of metacognition models, these are: The main focus of the model, Learning strategy used, Theory or approach on which the model is based, Strategy on which the model is based, and Computational technology used for the implementation.
2.4. Organization of terminology
The terms related to metacognition were organized into an ontology, due to the large number of terms related to metacognition that appear in the metacognitive models, and the semantic ambiguities of these terms among different models. The ontology was developed using Protégé 3.4 [24].
The objectives that guide the construction of the ontology are: a) to organize the terms and concepts that allude to metacognitive skills and clarify their characteristics and semantic relationships, b) to develop a generic and reusable semanticmodel based on ontology that includes the necessary features to provide clarity about metacognition concepts.
2.4.1 Methodology for ontology construction. The methodology of Noy and McGuinness [17] was followed to construct the ontology
The range of the ontology domain is supported by the following questions: What are the terms and concepts related to metacognition? What is the class and subclass hierarchy that makes the organization of terms and concepts related to metacognition possible? What are the main relationships among metacognitive skills? What relationships are necessary in the ontology to generate recommendations of actions, which can be used to design strategies for metacognitive skills scaffolding?
No ontology was reused, because no ontologies related to metacognitive capabilities in IS were found. Beneath, the elements and terms taken from metacognitive models are described. See Table 2.
For each category of metacognitive skill, class hierarchies were identified. For SelfRegulation a class hierarchy composed by SelfReinforcement, SelfInstruction and SelfControlling was built (Figure 2).
]]>
Regarding metamemory, the taxonomy consists in the classes and subclasses associated with control and metacognitive judgments (Figure 3).
The reasoning core for metacognitive skills, is formed by the relationships between instances of classes and a series of statements, which structure the semantics of the process. The properties that relate instances of different classes in the ontology are exposed in Figure 4.
3. RESULTS AND DISCUSSION
The results and discussion were organized according to the categories of analysis.
3.1 Types of metacognitive architectures
In metacognition models, two predominant types of architectures were found, which are: (1) Centralized architectures such as MetaAQUA, CLARION and MCL, which are predominant and are formed by a single metalevel that controls the entire metacognitive activity of the system. (2) The decentralized or distributed architectures such as MLCAA and DMF are composed of many metalevels. These kinds of architectures are managed by agents, using complex policies for communication and negotiation.
In relation to metamemory, it can be seen in Table 3, that MetaAQUA has a complex multifaceted memory [9,18] and reasons about memory events. While, in MCL aspects referring to metamemory strategies that can be used to learn from detected failures are left out [22]. Moreover, MCL has only basic mechanisms of shortterm memory, which in the metalevel are matched with longterm memory. EMONE implements a metamemory strategy based on a CBR system. MIDCA has a memory mechanism that can be accessed from the objectlevel and the metalevel. The rest of the architectures do not present clear support to control and monitor the memory process.
Regarding metacomprehension, MetaAQUA, EMONE and MIDCA are the architectures that offer adequate support. MetaAQUA uses introspection [3,18] to represent reasoning traces with metaexplanation. EMONE has a strategy known as mental critics [19] that use commonsense narratives to suggest courses of action to deliberate about the circumstances and consequences of those actions.
In respect to Selfregulation, it can be clearly appreciated that all architectures provide full support for this component of metacognition. In MIDCA the metalevel can act as an executive function in a similar manner to CLARION. CLARION and MCL have better developed metacognitive processes than the rest of the architectures. Note that MetaAQUA, EMONE and MIDCA, are the most complete metacognitive architectures, because they provide support to three main components of metacognition: metamemory, metacomprehension and selfregulation.
3.3 Core of architectures
The metacognitive architectures are founded on learning strategies. These are: Introspective Learning (IL) [3,10,12,18], Reinforcement Learning (RL) [3,11,14,25], Learning by Experience (LE) [3,12] and Cooperative Learning (CL) [11,12]. See Figure 5.
In computation, IL consists of the selfexamination or rational selfobservation of system reasoning state [3,10,26]. RL refers to the problem faced by an agent that learns some behavior through trialanderror interactions with a dynamic environment [25]. LE is a learning technique used in IS, that is based on the solution to new problems by adapting solutions to known problems [12,18]. CL is a learning technique used in MAS, which is based on communications policies and collaborative work [11].
]]> MetaAQUA, MCL and EMONE are focused on responding to failures, but the first is based on Goaldriven learning (GDL), which is focused on IL. Second is founded on the NoteAssessGuide (NAG) [1] cycle, that is based on RL and the third uses LE because it uses errordriven adaptive systems with the purpose of finding solutions to presented problems.CLARION, MLCAA, MIDCA and Simple Model for Metareasoning are focused on metacognitive monitoring [11,14,15], but CLARION and MLCAA implement decision making strategies for metalevel control, however, Simple Model for metaReasoning differs in the last aspect. EMONE and MetaAQUA share commons features, both have very complete support for problem solving and implement LE based on CBR systems. Moreover, CL is the base of MLCAA and DFM, both are architectures based on social context awareness and problem diagnosis in MAS.
CLARION and MetaAQUA are the architectures that implement more aspects related to learning approaches refering to metacognition.
3.4 Computational implementations
Continuing with the discussion, the computational aspects used for implementing IS [33] based on metacognitive architectures are listed in this paragraph. RL is the preferred strategy used by authors to implement learning capabilities in IS [11,14,15]. Moreover, RL is used in CLARION with Qlearning [27] and is implemented using backpropagation networks [28], see Table 4. While, in MCL, RL is implemented using Bayesian Networks (BN) [29].
MLCAA implements SL following a decision making strategy denominated selfguided learning, which uses a Markov decision process (MDP) [11] over decision trees. All implementations of RL described above were developed using QLearning algorithms [11,1416].
Another computational technique implemented is CBR [3,12, 18]. CBR is a computational strategy for solving problems based on experience [30,12]. In metacognitive architectures, CBR is used to manage system memory and support the metamemory capabilities of IS [12,18]. In MetaAQUA, CBR supports both metamemory and metacomprehension, due to introspective metaexplanations [9]. While, in EMONE [19] CBRbased similarity measure, with an implementation of Common LISP Environment, is used.
MIDCA and Simple Model for Metareasoning do not present details of their implementation, they are in the design phase.
RL implementations based on QLearning is the most common support to selfregulation in metacognitive architectures. While metamemory and metacomprehension are implemented using strategies based on CBR.
]]> 3.5 Metacognitive ontologyIn this section a list with definitions of the characteristics (facets) of the semantic relationship between the terms referring to metacognition is shown (See Table 5).
The specification of semantic relationships can contribute to provide clarity in relation to metacognitive concepts.
4. CONCLUSIONS
In this paper we presented a research work based on four categories of study where it was found that there are two predominant types of architectures of metacognition in IS, centralized and decentralized.
The learning strategies present in metacognitive architectures are: Introspective Learning (IL), Reinforcement Learning (RL), Learning by Experience (LE) and Cooperative Learning (CL).
]]> The most used computational implementation to support selfregulation in metacognitive architectures are RL implementations based on QLearning. While metamemory and metacomprehension are implemented using strategies based on CBR. CLARION and MetaAQUA are the architectures that implement most aspects related to the different learning approaches of metacognitionMetaAQUA, EMONE and MIDCA, are the most complete metacognitive architectures as they provide support for the three main components of metacognition.
An ontologybased semantic model was proposed which is composed of the terms and concepts found in the studied architectures.
Finally, the specification of semantic relationships among terms and concepts can contribute to provide clarity about understanding metacognition.
REFERENCES
[1] Anderson, M. and Oates, T., Metacognition in Computation: Papers from 2005 AAAI Spring Symposium., in AAAI Technical Report SS0504, Menlo Park, CA, 2005. [ Links ]
[2] Minsky, M., Singh, P. and Sloman., A., Designing Architectures for HumanLevel Intelligence, in The St. Thomas Common Sense Symposium. AI Magazine, pp. 113124, 2004. [ Links ]
[3] Cox, M., Metacognition in computation: A selected research review, Artificial Intelligence, vol. 169, no. doi:10.1016/j.artint.2005.10.009, pp. 104141, 2005. [ Links ]
[4] Anderson, M., Fults, S., Josyula, D. and Oates, T., A SelfHelp Guide for Autonomous Systems. AI Magazine, 29(2), AI Magazine, 29(2), pp. 6776, 2008. [ Links ]
[5] E Vockell, Metacognitive Skills, Educational Psychology: A Practical Approach.: (Online Ed.) available at http://educationcalumetpurdueedu/vockell/edPsybook/Edpsy7/edpsy7_meta.htm. Viewed 10/11/2011, 2009. [ Links ]
[6] T Johnson, T Archibald, and G Tenenbaum, Individual and team annotation effects on students' reading comprehension, critical thinking, and metacognitive skills, Journal of Computers in Human Behavior COMPUT HUM BEHAV, vol. 26, no. 6, pp. 14961, 2010. [ Links ]
[7] R Maki and S Berry, Metacomprehension of text material, Journal of Experimental Psychology:Learning, Memory, and Cognition, vol. 10, pp. 663-679, 1984. [ Links ]
[8] T Nelson and L Narens, Metamemory: A Theoretical Framework and New Findings, In T. O. Nelson ed. Metacognition: Core Readings, 924. Boston: Allyn and Bacon. (Originally published in 1990), 1992. [ Links ]
[9] M Cox, Perpetual Selfaware Cognitive Agents, AI Magazine, vol. 28(1), pp. 3245, 2007. [ Links ]
[10] M Cox and A Ram, Introspective Multistrategy Learning: On the construction of learning strategies, Artificial Intelligence 112: , vol. 112, pp. 155, 1999. [ Links ]
[11] A Raja and V Lesser, A framework for metalevel control in multiagent systems, Autonomous Agents and MultiAgent Systems, vol. 15(2), pp. 147196. doi:10.1007/s104580069008z, 2007. [ Links ]
[12] A. Aamodt, KnowledgeIntensive CaseBased Reasoning and Intelligent Tutoring, AI Communications, vol. Vol.7(1), pp. 39-45., 1994. [ Links ]
[13] I., Demetriadis, S., Karakostas, A. Magnisalis, Adaptive and Intelligent Systems for Collaborative Learning Support: A Review of the Field, IEEE Transactions on Learning Technologies, vol. 4(1). doi:10.1109/TLT.2011.2, pp. 5-20, 2011. [ Links ]
[14] R Sun and R Mathews, Modeling MetaCognition in a Cognitive Architecture, Cognitive Systems Research, vol. 7(4), pp. 327338, 2006. [ Links ]
[15] M Anderson and D Perlis, Logic, Selfawareness and Selfimprovement: The Metacognitive Loop and the problem of brittleness, Journal of Logic and Computation 15(1), 2005. [ Links ]
[16] M Schmill, M Anderson, S Fults, and D Josyula, The Metacognitive Loop and Reasoning about Anomalies, in Metareasoning: Thinking about thinking Cambridge. MA: MIT Press., 2011, pp. 183198. [ Links ]
[17] N Noy and D McGuinness, Ontology Development 101: A Guide to Creating your First Ontology, Stanford Knowledge System Laboratory Technical Report and Stanford Medical Informatics Technical Reports, 2001. [ Links ]
[18] M Cox and A Raja, Metareasoning: An introduction, in Metareasoning: Thinking about thinking. Cambridge, MA: MIT Press., 2011, pp. 314. [ Links ]
[19] P Singh, EMONE: An Architecture for Reflective Commonsense Thinking. Dissertation of Doctor of Philosophy in Computer Science and Engineering. Cambridge, MA: Massachusetts Institute of Technology, 2005. [ Links ]
[20] C Kennedy and A Sloman, Autonomous Recovery from Hostile Code Insertion using Distributed Reflection, Journal of Cognitive Systems Research, vol. 4(2), pp. 89117., 2003. [ Links ]
[21] C Kennedy, Decentralised Metacognition in ContextAware Autonomic Systems: some Key Challenges, AI Magazine, 2010. [ Links ]
[22] M Cox, T Oates, and D Perlis, Toward an Integrated Metacognitive Architecture. Advances in Cognitive Systems, Papers from the 2011 AAAI Fall Symposium, vol. (FS1101), pp. 7481, 2011. [ Links ]
[23] D Norman and T Shallice, Attention to action: willed and automatic control of behavior, in Consciousness and Self Regulation: Advances in Research and Theory. Plenum, New York.: G. Schwartz and D. Shapiro, (eds.) , 1986, ch. vol.4, pp. 118. [ Links ]
[24] Stanford. (2012, Julio) Protégé Home Page. [Online]. http://protege.stanford.edu/ [ Links ]
[25] L Kaelbling, M Littman, and A Moore, Reinforcement Learning : A Survey, Journal of Artificial Intelligence Research, vol. 4, pp. 237285, 1996. [ Links ]
[26] D Schultz and S Schultz, A history of modern psychology , 10th ed. Belmont, CA: Wadsworth, Cengage Learning. pp. 67-77, 88-100. ISBN 13:9781133316244., 2012. [ Links ]
[27] C Watkins, Learning with delayed rewards, Cambridge University, Cambridge, UK., Ph.D. thesis 1989. [ Links ]
[28] D Rumelhart and J McClelland, Parallel distributed processing: Explorations in the microstructures of cognition, The PDP Research Group, Cambridge, MA, MIT Press. 1986. [ Links ]
[29] J., Liu, H., Deng, A. Ding, Application of Bayesian Network Knowledge Reasoning Based on CBR in ITS, Third International Joint Conference on Computational Science and Optimization, 2010. [ Links ]
[30] J. L., Cox, M. T., & Gonzalezcalero, P. A. Kolodner, Casebased reasoninginspired approaches to education, Engineering, vol., 00, pp. 14. DOI:10.1017/S000000000000000, 2005. [ Links ]
[31] A Alter, D Oppenheimer, N Epley, and E Eyre, Metacognitive difficulty activates analytic reasoning, Journal of Experimental Psychology: General, vol. Vol 136(4) DOI:10.1037/00963445.136.4.569, pp. 569576, 2007. [ Links ]
[32] M Minsky, The Emotion Machine. New York: Simon and Schuster, 2006 [ Links ]
[33] Alvarez, J.; Jiménez, J.; Ramirez, J. Design Cycle of a Robot for Learning and the Development of Creativity in Engineering. Dyna, vol. 78, n. 170, pp. 5158, 2011 [ Links ] ]]>
