The interconnection of learning ability and the organization of metacognitive processes and traits of personality

Karpov, A.V. P.G. Demidov Yaroslavl State University, Yaroslavl, Russia

Karpov, A.A. P.G. Demidov Yaroslavl State University, Yaroslavl, Russia

Karabushchenko, N.B. P.G. Demidov Yaroslavl State University, Yaroslavl, Russia

Ivashchenko, A.V. Peoples’ Friendship University of Russia (RUDN), Moscow, Russia

Introduction

As is well known, one of the fundamental problems in psychology is the prob­lem of general abilities. Because learning is one of these abilities, the study of it is an integral and important aspect of the problem. however, to date the concept of learning ability has not been explained fully and has not been well defined. In addi­tion, methodological support, which allows diagnosis of the level of learning abili­ties, is also underdeveloped. With its different definitions this concept is presented in a number of psychological disciplines, particularly developmental psychology, pedagogical psychology, and the psychology of general abilities. According to the classification of general abilities proposed by V. N. Druzhinin, learning ability is one of three general abilities alongside intelligence and creativity. There are also a number of other views on the definition of learning ability, among them those from the viewpoint of developmental psychology, pedagogical psychology, and the psy­chology of labor (Bodalyov, 1984; Druzhinin, 1999; Kalmykova, 1981; Kholodnaya, 2002; Shadrikov, 1985; Verbitskiy, 1987; Vygotsky, 1983; Yasyukova. 2005).

Thus, for example, learning ability is characterized as the ability to master new educational material (new knowledge, new actions, new forms of activity). In addi­tion, this concept is defined as a complex dynamic system of individual personal­ity traits that contribute to the productivity of educational activity and to the rate and quality of mastering social experience. Another approach — perhaps the most common and justified in theoretical terms — is the interpretation, proposed by Druzhinin (1999), of learning ability as a general ability. In this approach, learning ability is considered similar in status to intelligence and creativity. Druzhinin also proposes a rather clear criterion for the differentiation and composition of general abilities as a whole and of their specific features in relation to each other. On the basis of the “cognitive paradigm,” he suggests that this criterion is the correspon­dence of this or that ability to a certain basic component (aspect) of information processing. For example, intelligence is correlated primarily with the application of knowledge, creativity — with its transformation, learning ability — with its acquisi­tion. Thus, in this study we will stick with the approach by Druzhinin as the most grounded in theoretical terms. According to this approach, the need to introduce the concept of learning ability is conditioned by the existence of a general ability that has a status similar to that of general intelligence and creativity.

Another direction, which is more recent than the study of general abilities, is metacognitivism; it is currently no less relevant and popular in psychology. having appeared in the1970s, metacognitivism is a kind of platform for new developments and discoveries; it is still insufficiently developed but is promising. Many psycholo­gists, both in Russia and abroad (the United States, Canada, Australia, Japan), have been and are engaged at present in the study of metacognitive processes of person­ality. Despite the relatively short period of time during which metacognitivism has been studied, its theoretical base is expanding every year and new concepts, psy­chodiagnostic methods, directions, and scientific trends are being created within metacognitivism itself. Special psychological laboratories in universities of major cities in North America and Europe are devoting their work exclusively to research in the field of metacognitive personality traits. In addition, the range of tasks in this psychological field is always expanding (A. A. Karpov & A. V. Karpov, 2015; Metcalfe & Son, 2000).

However, as often happens in science, two large and important areas exist and are developing largely in parallel and independently of each other. This is the case with general-ability psychology and metacognitivism. Research carried out on the border of these two areas is presented to an insufficient degree, particularly in ex­perimental studies. Despite the ever-increasing number of studies both in the field of general abilities as a whole and of learning-ability development in particular and in the area of metacognitive personality processes, the possible correlation of a given class of processes with the level of learning ability has barely been considered and neither has, correspondingly, its interpretation. In other words, both scientific trends are developing quite rapidly, but their interaction with each other is not in­vestigated to any great extent.

In our view, the existence of such a correlation is assumed for quite important reasons and because of clear preconditions. Thus, previous studies conducted by us show that there is a relationship between intelligence level and creativity level, on the one hand, and the main metacognitive processes, on the other. This dependence can be found not only in analytical terms — that is, between the development of in­telligence and creativity and the development of certain metacognitive processes — but also in structural terms. In this respect, both intelligence and creativity are sig­nificant determinants of the structural organization of metacognitive processes as a whole. however, as has been noted, intelligence and creativity are two of the three general abilities according to the classification of Druzhinin. Therefore, because we have already proved the relationship of metacognitive processes and personality traits with intelligence and creativity, it is quite justified to assume their relation­ship with learning ability (A. A. Karpov, 2016; A. V. Karpov & Skityayeva, 2005).

In addition, we should take into account a number of studies conducted by the staff of the psychological laboratory of Columbia University (United States) aimed at determining the influence of metacognitive processes on learning in primary school–aged children. Among the most important research directions in this field are the application of mnemonic techniques to the learning process, time manage­ment and metacognitive control in the learning process, the so-called effect of a large time gap, as well as the role of metacognitive processes in collaborative learn­ing (also known as proximal learning). In our opinion, the interest of the leaders of the U.S. metacognitivism school in learning is not accidental. Common sense tells us that learning ability can be considered the result of the successful development of metaprocesses, mainly in the period of childhood and adolescence, which is the focus of American researchers (Metcalfe & Dunlosky, 2008; Schwartz, 2006).

Thus, the combination of the high theoretical importance of this issue with a clearly insufficient level of its elaboration at present proves the obvious relevance of it and therefore the need for its study. In this regard, the goal of this study is to determine the possible correlation between the level of learning ability (as a certain kind of general ability) and basic parameters of metacognitive processes and per­sonality traits.

This goal was concretized in the following tasks:

• the sampling design of the study, which was adequate to its purpose
• the diagnosis of the studied factors (variables)
• the statistical process in goftheresearch data and their subsequent interpre­tation

Accordingly, the objects of the study are learning ability and metacognitive processes and personality traits; and its subject is the correlation of learning-ability level and the extent of the development of metacognitive processes and personality traits. On the basis of the goals of the study, we can formulate the following hypoth­esis: there is a regular interconnection between the level of learning-ability devel­opment and the development of metacognitive traits of personality.

Method

The study used eight psychodiagnostic methods.

1. The technique for diagnosing reflexivity by A. V. Karpov and V. V. Ponomare­va. This technique explores, among other properties, reflexivity, which, ac­cording to the authors, can be presented in people at three levels of inten­sity (advanced, average, and underdeveloped).
2. The technique for determining the intensity level and the direction of reflection by M. Grant. The American psychologist M. grant developed a research questionnaire that allows estimation of the level of intensity and the direc­tion of reflection of a subject. The questionnaire consists of two scales — self-reflection and socioreflection — and the ratio of the two forms is ana­lyzed. The most adaptive level is an average level of self-reflection and a high (or above-average) level of socioreflection.
3. The scale of self-assessment of metacognitive behavior by D. LaCosta. This technique was developed in 1998 and is a short questionnaire, fairly easy to use and interpret. Subjects are required to assess the cognitive strategies they use.
4. The Metacognitive Awareness Inventory by G. Schraw and R. S. Dennison. This questionnaire refers to a rather small number of methods that exist currently to investigate metacognitive personality traits. Its main purpose is to trace the patterns of metaprocesses involved in the course of perform­ing activities.
5. The D. Everson questionnaire. This technique diagnoses four major meta­cognitive personality traits (metacognitive involvement activities, use of strategies, action planning, self-test).
6. The questionnaire for diagnosing metadecisions by A. V. Karpov. This survey consists of 26 questions and has a seven-response scale.
7. The technique Signs of a Quadrilateral by L. V. Zankov. This technique is used to diagnose the dynamic aspect of learning ability as one of the gen­eral personality abilities included in the cognitive subsystem. It consists of four successive stages.
8. The technique Puzzles. This research technique was created to measure the time-rate parameter of learning-ability development. The procedure has four levels of difficulty, six tasks at each level.

Students and graduates of Yaroslavl universities as well as employees of several commercial organizations took part in the research. The total sample consisted of 100 subjects.

The overall procedure of the research differentiated all the results of the prima­ry diagnostics of the subjects into two main sections. The first section contains the results of all the subjects on the techniques aimed at diagnosing the level of learn­ing. It includes data obtained by methods 7 and 8. The second section includes the data for all the subjects in each of the methods 1–6, which are aimed at diagnosing metacognitive qualities.

The research sample consisted of students in years 3 and 4 in the departments of natural sciences and humanities in higher educational institutions in Yaroslavl and also of postgraduate students, who were from 20 to 32 years old. In addition, the sample included employees in the educational and communications sections of several government organizations, as well as representatives from marketing and service organizations, who were from 25 to 48 years old. In the total sample of 100 people, 34% had a liberal arts education, and 66% had a technical and mathemati­cal education.

Results and discussion

All the results obtained were divided into two large blocks. The first block included the results of all subjects tested by the two techniques aimed at diagnosing the level of learning ability. The second block included the data of all subjects for each of the six methods for diagnosing metacognitive personality traits.

The results for both blocks were further subjected to statistical analysis. Before the data processing of the results of the test of learning-ability development, all the subjects were divided into four groups. group I included subjects with the lowest level of learning ability; group II — subjects with below-average learning ability; group III — subjects with above- average learning ability; group IV — those with the highest level of learning ability.

The differentiation of subjects in the specified subgroups was based on the gen­erally accepted method of contrast groups by Flanagan (1954). This method, in its standard version, assumes the differentiation of a sample into three subgroups of the subgroups with the lowest and highest value of the differentiating criterion. In this case, the individual development of learning ability in a quantitative ratio, respectively, is 17% in the first two subgroups and 66% in the third subgroup. This method, however, was modified to increase its diagnostic capabilities. Specifically, the whole sample was differentiated into four subgroups, not three. This was done through a “secondary” differentiation of the third subgroup (the largest subgroup) into two additional groups, which also significantly differed in the level of the dif­ferentiating criterion. For determining the differentiating indicators of threshold values for splitting the sample into the four subgroups, the following procedure was implemented. Because, for the diagnostics of individual measures of learning-ability development we used two methods, the diagnostics of its general level were determined by summing their results. The “raw” primary data were first transferred to the standard (sten) indicators, and then were summed. Thus was formed a continuum of all the potential individual indicators ranging from 2 to 20 points. Further division into subgroups was made on the basis of the main, standard requirements according to which the average values of the differentiation parameter (in this case, learning) should be statistically significantly different in them. In this research, the limiting level of significance was taken as a confidence interval localized within the boundaries of above p = .95. As a result the thresholds were determined for the formation of the relevant subgroups (for each of the subgroups, respectively, 8, 11, 14 and 17 stens). The specific values of mathematical expectation and standard deviation in the differentiated subgroups had the following values: 1st subgroup: M = 9.1; δ = 1.1; 2nd subgroup: М = 12.4; δ = 1.4; 3rd subgroup: M = 15.4; δ = 1.4; 4th subgroup: М = 17.2; δ = 1.2.

As a result of this procedure, we could calculate the total value of all the me­thods used for each of the four groups of subjects. The results are presented in Figure 1.

Figure 1. The dependence of the total intensity of metacognitive processes on learning-ability level. Designation: Σс is the average total indicator of the develop­ment of metacognitive personality traits in standard-ten points for the subjects of the respective groups; this indicator is the sum of the stens (standard-ten evalua­tion scale); see the text. The abscissa axis contains groups of subjects differing in the degree of learning-ability development, in ascending order.

From the data presented it can be seen that with the increase of learning-ability level as a general ability, there is a quite pronounced and statistically reliable (by the Kruskal-Wallis test) tendency toward an increase in the general development of metacognitive personality traits (р = .95). Consequently, it is possible to ascertain a direct relationship between these categories.

The next stage of the research was to identify the matrices of the intercorrela­tion of metacognitive personality traits corresponding to the scales of the eight psychodiagnostic techniques. In other words, on the basis of the results obtained after applying these techniques, the structure-level analysis of the correlation of the metacognitive personality traits with one another was carried out. Each of the matrices was calculated on the basis of four different groups of subjects united ac­cording to the learning-ability level of each of them. Thus, each of the four groups had its own matrix of intercorrelation.

Further, on the basis of these matrices we identified the indices of coherence, di­vergence, and general organization of the structures of the metaprocesses. The index of coherence (ICS) was measured by adding up the positive correlations among ele­ments of the system; the index of divergence (IDS) was measured by adding up the negative correlations; and the index of general organization (IgS) in this case was calculated as the difference of the values of ICS and IDS. When summing numerical values of the relations among the elements of each of these correlations, a score was assigned depending on the significance level. So, at p =.90 the score was equal to 1; at p = .95, to 2; at p = .99, to 3. In addition, for each group structure-grams were created, expressing the overall “picture” of the structural organization of the metacognitive traits and the existence of meaningful correlations among them.

As an illustration of the obtained structure-grams, we present the data for the groups with low and average learning-ability levels (Figures 2 and 3).

Figure 2. Structure-grams of metacognitive traits for group I subjects (the low­est level of learning ability). Symbols: The technique for diagnosing reflexivity (A. V. Karpov and Ponomareva): RR – retrospective reflexivity, RP – reflexivity of the present: RF – reflexivity of the future. The technique for determining the intensity and the direction of reflection (grant): Sel R – self-reflection, socR– socioreflection. The scale of self-assessment of metacognitive behavior (LaCosta): s-a MB – self-assessment of metacognitive behavior. The Metacognitive Awareness Inventory (Schraw and Dennison): MA – metacognitive awareness inventory. The Everson questionnaire: МА I (E) – involvement in metacognitive activities, US – use of strategies, PA – planning of actions, ST – self-test. The questionnaire for diagnosing metadecisions (A. V. Karpov): МD – metadecisions.

In the calculation of structural indices it was found that for a given structure ICS = 19, IDS = -11, IgS = 8.

Figure 3. Structure-gram for group III subjects (middle-level learning). The designations are the same as in Figure 2.

In the calculation of structural indices it was found that for a given structure ICS = 33, IDS = -7, IgS = 26.

Next, on the basis of the obtained values of ICS, IDS, and IgS we created a general graph of the dependence of the structural organization of metacognitive personality traits on learning-ability development (see Figure 4).

Figure 4. The dependence of the structural organization of metacognitive per­sonality aits on learning ability. Designations: ICS – index of the coherence of a structure; IDS – index of the divergence of a structure; IgS – index of the general organization of a structure. The abscissa axis contains groups of subjects differing in the degree of learning-ability development, in ascending order.

The results show the direct dependence of the structural organization of meta­cognitive processes and personality traits on the level of learning ability. With an increase in learning ability (as one of the general abilities) the structural organiza­tion of metacognitive traits increases significantly. In other words, subjects with the highest degree of learning ability had the highest indicators of metacognitive processes and personality traits, and vice versa. In connection with this mutual dependence, previous research has shown that, between the level of development of psychometric intelligence and the basic parameters of the cognitive sphere of personality, there is also a natural and fairly stable dependence of the “optimum” type. In other words, the highest level of intelligence is associated not with the minimum or maximum values of metacognitive processes and personality traits and their general organization but with the average (optimum) value of their inten­sity (A. V. Karpov & Skityayeva, 2005). And another study has shown that the same stable and logical relationship exists between another common ability (creativity) and parameters of the metacognitive sphere of personality and their organization. This relationship, however, is fundamentally different and has a kind of inverse proportion (A. A. Karpov, 2016). Thus two important results are obtained. First, such a connection does exists and is inherent in all three general abilities (intelli­gence, creativity, and learning ability). Second, in relation to each of them this con­nection is specific. The dependence of the parameters of the metacognitive sphere of personality and their organization with general abilities is thus fundamentally diversified.

In addition, we applied to the results obtained the method of Express-χ2 to de­termine the degree of homogeneity-heterogeneity of the intercorrelation matrices of metacognitive traits corresponding to the four different groups of subjects. With this method, the structural indicators of all elements of each of the four structures (in this case, the four groups of subjects) are ranked. After the assignment of ranks, the correlation analysis of each of the structures with one another is carried out. Thus, in the case of significant correlations among the groups of subjects, one can speak only of the quantitative differences among the structures. Conversely, non­significant differences would be indicative of qualitative changes within the struc­tures being considered.

As a result of using the Express-χ2 method the data in Table 1 were obtained. The results show that differences among the structures of metacognitive traits are, mostly, insignificant.

Table 1. Correlation matrix of the structures of each group as a result of the implementation of the Express-χ2 method

Thus, returning to Figure 4, which shows the dependence of the structural or­ganization of metacognitive traits on the level of intensity of learning ability as a whole, we can conclude that in the base of these intercorrelations are only quantita­tive modifications of structures. The discovered structures are generally homoge­neous as to the content of correlations within them.

In previous studies of the correlation between the two other general abilities (intelligence and creativity) and metacognitive personality traits, it has been found that changing them leads not only to quantitative changes in the degree of struc­tural organization of metaprocesses but also to its qualitative changes and transfor­mations. In addition, the above results show that a change in the learning-ability level leads only to quantitative changes in the degree of structural organization of metaprocesses but not to qualitative modification. Thus, of the three general abili­ties, learning ability is the least strong in relation to its influence on the structure of metacognitive traits.

In our opinion, the direct correlation between the level of learning ability and the degree of development of metacognitive processes found in this study is in most general terms due to the fact that this very degree is exactly one of the most important effective manifestations of learning ability as such. Learning ability is significantly associated with the development, assimilation, and generalization of metacognitive strategies. In other words, components and factors of learning, mastering different operating tools of metamemory, metathinking, and cognitive monitoring are, for example, included in the content of metacognitive processes.

however, the most important result of the research on this problem is the fol­lowing. In addition to the complex and multidimensional intercorrelation of learn­ing abilities and metacognitive processes and personal traits revealed in this study, the relationship of these two categories goes significantly beyond the limits of their dependence on the analytical and structural levels. Considering the very content of the concept of learning ability, its basic psychological characteristics, it is pos­sible to conclude legitimately that the given general ability is a part of the struc­ture of metacognitive processes and personality traits and is thereby differentiated into two main types: self-learning ability and learning ability realized in interaction with the environment. The first of these abilities is exactly of the greatest interest in the context of involvement in the metacognitive subsystem of the psyche and is subject to further study.

In addition, in view of our results, learning ability should be interpreted some­what differently and more widely than it currently is, but at the same time it should be interpreted in a differentiated way. The whole complex of metacognitive process­es and personality traits (as well as other metacognitive aspects) is in fact included in the very content and structure of learning ability as a general ability. Learning ability on the whole is formed largely by these means and therefore is largely iden­tical to them. In this regard, two basic structural levels can be differentiated in the learning-ability structure. The first level is direct learning ability, which is based on the “primary” mental processes and personality traits (that is, the “primary” cognitive processes). The second level is indirect learning ability, which is based on “secondary” processes and traits, primarily of the metacognitive kind. These processes and traits are formed mainly through conscious procedures and are then used by the subject as an additional means for expanding his or her potential and enhancing the resource capabilities of the psyche, including the acquisition of new knowledge and the formation of new activity tools — that is, for expanding and enhancing learning ability.

This result not only is in accordance with our general (resource) approach to the interpretation of metacognitive processes and personal qualities but also con­tributes substantially to the deepening and development of this approach (A. A. Karpov & A. V. Karpov, 2016). Our study reveals and explains exactly how a com­mon subject potential can be optimized as a result of the synergistic interactions between the individual development of learning ability and the parameters of the metacognitive sphere of personality, and this potential is as well a measure of the organization and integration of these interactions. In addition, these data contrib­ute to a deepening of the understanding of a more general plan developed for the concept of mental resources of the personality by Kholodnaya (2002). In our opin­ion, it should be emphasized that the validity of the resource approach to the inter­pretation of metacognitive processes and qualities, which is clearly explicated in re­gard to learning as a general ability, is associated with another significant fact: that is, that an important consequence of the integrative action of the metacognitive factors of learning is one of the classic phenomena that are currently revealed in metacognitivism, the phenomenon of the “region of proximal learning” (Metcalfe & Son, 2000). This concept is similar in meaning and etymology to the well-known phenomenon of the “zone of proximal development” by Vygotsky (1935/1983). In general, the meaning of this concept is that learners, based on their assessment of their cognitive potential, past experience, and other individual characteristics and preferences, independently select for priority study some part of the new material. Such a metacognitive strategy has been experimentally identified recently, but re­searchers have recognized it as one of the most effective and efficient strategies that contribute to the quality of the study of new material (Metcalfe & Son, 2000). In the region of proximal learning a specific pattern is highlighted.

First, during the learning of new material there is a pattern according to which for the study initially selected the most easy or familiar “chunks” of new material (in the specific region of proximal learning) act as a kind of basis for the productive, gradual transition to more complex elements.

Second, the effectiveness of the implementation of this strategy is directly related to the level of development of metacognitive processes and qualities. Consequently, our results, which reveal a similar, direct connection of learning ability and the level (and structural characteristics) of metacognitive processes and qualities, offer new ways to interpret and, respectively, to explain the phenomenon of the region of proximal learning. In our view, it needs to be understood not just as a phenomenon or even as a strategy but as one of the specific, but rather general in meaning and functional role, mechanisms underlying the same general ability (learning ability). In addition, in our opinion another “classic” phenomenon with a similar status (the status of the mechanism), which is set in metacognitive “judgments of learning,” should be interpreted. It would be better to label it more generically as “judgments about metamemory” because it includes several distinct but related substantive components: ease-of-learning judgments, paired-associate judgments of learning, ease-of-recognition judgments, and free-recall judgments of learning (Eakin, 2005; Schwartz, 2006). however, this conclusion is by necessity tentative.

Third, more interesting is the following pattern. Because of the characteristics, first and foremost, of a region of proximal learning, students do not choose the most simple elements of new material, but they also do not choose the most dif­ficult; they choose those of the approximately average level of difficulty. In other words, the efficiency of learning new material and probably all learning activity overall depends on the level of difficulty of the educational material, which can be characterized as being of the “optimum” type. This dependence has been shown in a large number of studies in various areas of psychological knowledge (A. A. Kar­pov & A. V. Karpov, 2015).

Finally, in our opinion, in the interpretation of our results, their most general meaning is that between learning ability and the parameters of the metacognitive sphere of personality there is a definite and fairly stable regular relationship. howe­ver, a similar result (the existence of a stable and natural relationship) was obtained earlier, as already noted, for the two other general abilities (intelligence and creativ­ity). Thus, we can conclude that the community of explicated dependencies (the community in regard to their existence but, of course, not to their specific content) is an important argument in answering one of the most general theoretical ques­tions in the psychology of abilities: the question of the psychological status of the concept of learning (whether it should be included in the category of general abili­ties). This question has a rather long history and is currently the subject of intense debate in the field of the theory of abilities and in modern metacognitivism. Our results are one of the arguments in favor of a positive answer to this question.

Conclusion

Thus, according to the results of our study we can draw the following main conclu­sions.

1. A correlation exists between learning-ability development as one of the general abilities and the individual measure of the development of meta­cognitive processes and personality traits.
2. On the analytical level of the research, this dependence consists in the fact that as learning-ability development increases, the degree of development of metacognitive traits increases too.
3. On the structural level of the research this dependence consists in the fact that as learning ability develops, the overall level of the structural organiza­tion of metacognitive traits also increases.
4. It is known that learning ability, as a general ability, is a determinant of suc­cessful learning. In turn, metacognitive personality traits serve as an auxil­iary tool in the process of mastering new material. Thus, the involvement of learning ability in the structure of metacognitive processes and traits enhances the efficiency of learning in professional activity.

Acknowledgements

The research was supported by the Russian Scientific Fund, project No. 16-18­10030.

References

Bodalyov, A. A. (1984). O napravleniyakh i zadachakh razrabotki problemy sposobnostey [On directions and tasks of development of the problem of abilities]. Voprosy psikhologii [Issues in Psychology], 1, 31–40.

Druzhinin, V. N. (1999). Psikhologiya obshchikh sposobnostey [Psychology of general abilities]. St. Petersburg: Piter Kom.

Eakin, D. K. (2005). Illusions of knowing: Metamemory and memory under conditions of retroactive interference. Journal of Memory and Language, 52(4), 526–534. doi:10.1016/j. jml.2005.01.009

Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. doi: 10.1037/h0061470

Kalmykova, Z. I. (1981). Produktivnoye myshleniye kak osnova obuchaemosti [Productive think­ing as a basis for learning ability]. Moscow: Pedagogika.

Karpov, A. A. (2016). Fenomenologiya i diagnostika metakognitivnoy sfery lichnosti [Phenom­enology and diagnostics of metacognitive sphere of personality]. Yaroslavl, Russia: YaRgU.

Karpov, A. A., & Karpov, A. V. (2015). Vvedeniye v metakognitivnuyu psikhologiyu [Introduction to metacognitive psychology]. Moscow: MPSU.

Karpov, A. A., & Karpov, A. V. (2016). Vzaimosvyaz psikhometricheskogo intellekta s orga­nizatsiyey metakognitivnykh protsessov i kachestv lichnosti [The interconnection of psy­chometric intelligence with the organization of meta-cognitive processes and personality traits]. Psikhologicheskiy zhurnal [Psychological Journal], 2(37), 69–79.

Karpov, A. V., & Karpov, A. A. (2016). Metakognitivnyye factory obuchaemosti kak mentalnyy resurs lichnosti [Metacognitive factors of learning as a mental resource of the personality]. In M.A. Kholodnaya, g.V. Ozhiganova (Eds.), Mentalnyye resursy lichnosti: Teoreticheskiye i prikladnyye issledovaniya [Mental resources of the personality: theoretical and applied re­search] (pp. 15–20). Moscow: Institute of Psychology RAN.

Karpov, A. V., & Skityayeva, I. M. (2005). Psikhologiya metakognitivnykh protsessov lichnosti [Psychology of metacognitive processes of the personality]. Moscow: Institute of Psychol­ogy RAN.

Kholodnaya, M. A. (2002). Kognitivnyye stili: O prirode individualnogo uma [Cognitive styles: On the nature of the individual mind]. Moscow: PER SE.

Metcalfe, J., & Dunlosky, J. (2008). Metamemory. In Learning and memory: A comprehensive reference (pp. 351–360). Oxford, U.K.: Elsevier.

Metcalfe, J., & Son, L. K. (2000). Metacognitive and control strategies in study-time allocation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(1), 204–221.

Schwartz, B. L. (2006). Tip-of-the-tongue states as metacognition. Metacognition Learning, 1, 149–158. doi:10.1007/s11409-006-9583-z

Shadrikov, V. D. (1985). Sposobnosti i deyatelnost [Abilities and activity]. Moscow.

Verbitskiy, A. A. (1987). Psikhologo-pedagogicheskiye osobennosti kontekstnogo obucheniya [Psy­chological and pedagogical features of contextual learning]. Moscow.

Vygotsky, L. S. (1983). Igra i eye rol v razvitii [Play and its Role in Development]. In Sobraniye sochineniy [Collected works]. Moscow: Pedagogika. (Original work published 1935)

Yasyukova, L. A. (2005). Zakonomernosti razvitiya ponyatiynogo myshleniya i ego rol v obuchenii [Regularities of development of conceptual thinking and its role in learning]. St. Petersburg: IMATON.