Intelligent Tutoring Systems Research Paper

The first ITS were built in the 1970s (for an overview of these early systems see Wenger 1987). They were a response to the rather inflexible conventional computer-aided instruction (CAI) systems which were (and are) mostly built on the principles of reinforcement learning. Whereas the reactions of CAI systems follow predetermined paths, ITS aim to provide instructional interventions that are custom tailored to the strengths, weaknesses, and the actual knowledge of a given learner. This flexibility of ITS which in theory also extends to unexpected situations and which is the justification for calling them ‘intelligent’ is achieved by using techniques from artificial intelligence (AI) research. The most salient property of ITS which takes them apart from other AI applications such as expert systems is the diagnosis of the current state of the student knowledge. This diagnosing facility and the other parts of ITS are described next. Then follows a description of how ITS work. After that, several problems are pointed out that have led to discrepancies between theory and practice of ITS. Finally, several new developments are mentioned.

1. System Architecture

In the classical view ITS consist of three modules that contain domain knowledge, a model of the learner’s current state, and teaching knowledge. These modules have been given slightly different names by different authors. Here, using Dede’s (1986) terms they will be called knowledge base, student model, and pedagogical module, respectively. The importance of the communication between learner and system has been increasingly recognized and this user interface is now widely recognized as the fourth module of ITS.

1.1 Knowledge Base

The knowledge base contains the domain knowledge, both declarative (knowing what) and procedural (knowing how). For instance, in ITS that deal with geometry proofs, the declarative part may contain the fact that the three angles of a triangle add up to 180 degrees; and the procedural part may contain the ‘side-angle-side’ rule, that is, the rule that if two triangles coincide in two corresponding sides and the angle in between one may conclude that the triangles are congruent. To represent the domain knowledge many different kinds of representational schemes have been used such as semantic networks, frames, constraint satisfaction networks, logic, propositional representations, and variations or combinations thereof. However, the most common technique used to represent domain knowledge seems to be production systems which consist of a memory buffer (working memory), a set of production (if–then) rules (procedural memory), a collection of facts (declarative memory) and an interpreter that decides what to do next. Production system architectures are often used in expert systems and indeed, expert systems have served as the knowledge base in early ITS. The best known example is GUIDON which used the expert system MYCIN as its knowledge base (Clancey 1987). In more recent ITS, production systems form the basis of an ideal student, who knows how to perform the task the student is expected to perform (e.g., Anderson et al. 1995).

1.2 Student Model

The student model should represent the student’s current state of knowledge. That includes a representation of the knowledge, concepts, and skills the student has acquired either fully or partially. It also includes the representation of a given student’s special skills and needs. Moreover, there should be a mechanism to represent misconceptions, bugs, or erroneous information which the student might have acquired. For instance, in ITS that deal with geometry proofs, the student model might represent that the student has understood the side-angle-side rule, that the student (erroneously) thinks there is also an angle-angle-angle rule which allows one to conclude that two triangles are congruent, and it may have inferred that this given student prefers delayed over immediate feedback. All this information should be used to provide optimal teaching interventions. To come up with such a student model one needs assumptions about how people learn and how they represent in memory what they have learned. Specific contents of the student model are inferred by asking the students questions and analyzing their reactions toward prompts of the system or the way they are going to tackle a problem posed by the ITS. For the student model, basically the same AI techniques are used as for the knowledge base.

1.3 Pedagogical Module

Teaching can be considered a knowledge-based skill, guided by strategies and techniques which are selected and combined dynamically in reaction to the student’s actions. The pedagogical module incorporates just this skill. It should determine the presentation method, the balance of tutor and student control, and it should give feedback when necessary. One important task of the pedagogical module is how to deal with student errors, for instance, when to interrupt and what to say. Pedagogical strategies may include presenting increasingly complex concepts or problems, Socratic tutoring, simulating phenomena, and modeling of expert problem solving via coaching. The teaching strategies and techniques used should have proved successful for the particular content matter. The pedagogical module should also be able to answer student questions.

1.4 User Interface

The user interface is essential for the success of an ITS: even if the three other parts were perfect, a weak user interface could make the ITS nearly useless. The best solution would be for the student to communicate with the ITS in natural language. This, unfortunately, is not possible to date, despite huge advances in the AI field of natural language comprehension and generation. Solutions to this problem include ‘canned language,’ multiple choice selection, and the almost exclusive use of graphical interfaces. The user interface must take into account attentional and memory constraints of the student but it should also use the strengths of human information processing. In many earlier ITS, the user interfaces have been constructed more or less ad hoc, but recently, theoretical considerations about transfer and domain specific issues have increasingly influenced the construction of interfaces (e.g., Anderson et al. 1995, Sedlmeier 1997).

2. How Do ITS Work?

The four components of ITS described above capture the different aspects included in ITS and give a good idea about how ITS might work in principle. Although parts of real ITS can be mostly assigned to one of these four parts there are not usually four independent modules at the level of implementation that interact with each other on equal terms. Usually one of the three classical components can be regarded as playing the central part in the teaching process.

2.1 Expert-Centered Instruction

In most ITS, the knowledge base plays the central part in the teaching process. Usually, the knowledge base is conceptualized as a domain expert or an ideal student and the course of action taken is dependent on the deviations of the student model from the expert or ideal student. In an Overlay model the knowledge of the learner is seen as a subset of the expert’s knowledge (e.g., Clancey 1987) and if a deviation between the current student model and the expert knowledge is diagnosed, the system tries to remove that discrepancy by suitable instructional measures. It has been found that the student’s knowledge is not just a subset of the expert’s knowledge but may contain nonexpert ways to arrive at the solution of a problem. Extensive research on how students solve problems correctly and what kinds of errors they commit led to the model of an ideal student and to ‘bug libraries,’ that is, to possible explanations for errors (Brown and Burton 1978). This approach has been elaborated into the technique of model tracing (e.g., Anderson et al. 1990). In this approach, a performance model specifies how, for a particular problem, a student’s current knowledge will map onto performance. Instructions address diagnosed confusions and keep the student on the right solution path. A learning model specifies how the student’s current knowledge will change as a result of problem-solving experiences and this learning model can be used to trace the student’s knowledge over time.

2.2 Teacher-Centered Instruction

One might argue that the most natural choice of an ITS component to control the instructional process should be the pedagogical module. Unfortunately, the pedagogical module is the most neglected component of ITS. This might in part be due to still existing deficiencies in instructional theory but it might also be the result of a practical problem: Teachers or education scholars are seldom involved in the construction of ITS. One approach that puts the teacher into the center of the instructional process is to use planning techniques that make it possible to develop and flexibly adjust global teaching strategies (e.g., Peachey and McCalla 1986, Sedlmeier and Robles de Acuna-Ponseti 1992).

2.3 Student-Centered Instruction

In expert and teacher-centered instruction, the learning process is fully dominated by the tutor. This contrasts with the current emphasis in instructional theory on learning by doing and situated learning. One way to put the student into the center of the instructional process would be to use techniques of machine learning (Self 1985). Research on this kind of instructional process is still in its infancy.

3. The Practice Of ITS

Whereas ITS seem to be ready to be used for specialized applications on the commercial market (Norton et al. 1998), to date, by far the most ITS did not pass the status of prototypical systems and there are very few ITS employed in the classroom. One important reason for this state of affairs is probably the diversity of researchers involved in the construction of ITS. A second might be that there is still no convincing evidence of ITS’s superiority over more conventional teaching procedures, and finally, there are severe problems with constructing good student models.

3.1 Diversity Of Interests

Constructing ITS is a multidisciplinary endeavor, including computer scientists, mathematicians, cognitive psychologists, and educational researchers, to name the most important groups. With some notable exceptions (e.g., Anderson et al. 1995), researchers are usually most interested in those aspects of ITS that are closest to their own field of research: Educators are most interested in effective tutors but often do not know how to write computer software, and researchers in cognitive science are more interested in developing and testing theories about how humans learn than in the sometimes tedious process of implementing computer software in schools. Another problem was that many ITS were developed on workstations and Lisp machines that schools could not afford and maintain.

3.2 How Effective Are ITS?

There is still a lack of thorough evaluation studies in which the effectiveness of ITS is compared to that of conventional teaching techniques. Keeping in mind that the development costs for ITS are quite high, ITS have to achieve noticeably better learning results than conventional CAI systems. One motivation to use ITS in the classroom has been the assumption that individualized instruction is much more effective than group instruction and that ITS can provide such an individualized instruction. However, a collection of meta-analyses shows that overall, the benefits of individualized instruction are almost indistinguishable from those of group instruction (Fraser et al. 1987). This is in line with the conclusion which Legree and Gillis (1991) reached in a survey of evaluation studies: For the majority of the ITS analyzed, there was no noticeable advantage over the learning success achieved with group instruction.

3.3 Problems With The Student Model

Research on student models since the early 1970s has shown that even for relatively small and well-defined domains such as simple arithmetic it is not possible to construct a complete cognitive model (e.g., ElsomCook 1993). One reason is that the representation of everyday knowledge which is needed in addition to domain knowledge is an almost insurmountable task. Another reason is that the basis for diagnosis, the student’s behavior, is often not clearly interpretable. A given student error encountered by the system may have many potential causes. However, the underlying misconceptions and not the observable behavior should be the basis for corrective procedures. Even if it were possible to construct an optimal student model, could it be used effectively? All available evidence suggests that it could not: Good teachers often have incomplete and even wrong ‘student models’ but are nonetheless very successful. They achieve their success not by a thorough diagnosis but by moving through a curriculum script, that is, a loosely ordered but well-organized and sequenced set of skills and concepts they expect students to learn, and by using adequate activities and strategies for teaching this material (Putnam 1987). Thus it seems that, at least for the purpose of effective instruction, the student model in its present state has severe deficiencies and might be discarded without much loss as far as teaching efficiency is concerned (Gugerty 1997).

4. Perspectives: Cognitive Science Research S. Educational Practice

Research in ITS is still flourishing (e.g., Goettl et al. 1998) but the emphasis is more on the second aim stated in the beginning of this research paper, that is, to develop cognitive models about the components of the instructional process. This could eventually also lead to the achievement of the first aim, that is, to provide effective instructional devices for all kinds of domains. It seems that for highly specialized industrial applications, this first aim has already been reached, at least in part. The deployment of ITS in the classroom is, however, only in its starting phase. For practical purposes it might be worthwhile in the short run to de- emphasize the reliance on the problematic student model and put more effort into the construction of theory-guided flexible interfaces. Such flexible interfaces could be the front-end to existing computer tools, simulations, games, or microworlds (Cumming and Self 1990) or could take part in guiding the teaching process (Sedlmeier 2000). Technological advances and the possibility of using the Internet will greatly facilitate the deployment of ITS in schools but ultimately whether or not schools will profit from the potentially huge benefits of ITS depends on successful collaboration among educators and system builders.

Bibliography:

1. Anderson J R, Boyle F, Corbett A T, Lewis M W 1990 Cognitive modeling and intelligent tutoring. Artificial Intelligence 42: 7–49
2. Anderson J R, Corbett A T, Koedinger K R, Pelletier R 1995 Cognitive tutors: Lessons learned. The Journal of the Learning Sciences 4: 167–207
3. Brown J, Burton R R 1978 Diagnostic models for procedural bugs in basic mathematical skills. Cognitive Science 2: 155–92
4. Clancey W J 1987 Knowledge-Based tutoring: The GUIDON Program. MIT Press, Cambridge, MA
5. Cumming F, Self J 1990 Intelligent educational systems: Identifying and decoupling the conversational levels. Instructional Science 19: 11–27
6. Dede C 1986 A review and synthesis of recent research in intelligent computer-assisted instruction. International Journal of Man–Machine Studies 24: 329–53
7. Elsom-Cook M 1993 Student modelling in intelligent tutoring systems. Artificial Intelligence Review 7: 227–40
8. Fraser B J, Walberg H J, Welch W W, Hattie J A 1987 Syntheses of educational productivity research. International Journal of Educational Research 11: 145–252
9. Goettl B P, Halff H M, Redfield C L, Shute V J (eds.) 1998 Intelligent Tutoring Systems. Springer, Berlin
10. Gugerty L 1997 Non-diagnostic intelligent tutoring systems: Teaching without student models. Instructional Science 25: 409–32
11. Legree P J, Gillis P D 1991 Product effectiveness evaluation criteria for intelligent tutoring system. Journal of Computer-Based Instruction 18: 57–62
12. Norton J E, Jones J A, Johnson W B, Wiederholt B J 1998 Are intelligent tutoring systems ready for the commercial market? In: Bloom C P, Loftin R B (eds.) Facilitating the Development and Use of Interactive Learning Environments. Erlbaum, Mahwah, NJ, 259–88
13. Peachey D R, McCalla G I 1986 Using planning techniques in intelligent tutoring systems. International Journal of Man– Machine Studies 24: 77–98
14. Putnam R T 1987 Structuring and adjusting content for students: A study of live and simulated tutoring of addition. American Educational Research Journal 24: 13–48
15. Sedlmeier P 1997 Basic Bayes: A tutor system for simple Bayesian inference. Behavior Research Methods, Instruments and Computers 29: 328–36
16. Sedlmeier P 2000 How to improve statistical thinking: Choose the task representation wisely and learn by doing. Instructional Science 28: 227–62
17. Sedlmeier P, Robles de Acuna-Ponseti J 1992 ‘Intelligente’ Hilfe beim Losen von alltagsnahen Wahrscheinlichkeitsproblemen: Modellierung dynamischer Wissensinhalte fur ein flexibles Tutorsystem [‘Intelligent’ help in solution of probability problems with everyday applications: Modeling of dynamic knowledge for a flexible tutor system]. Kognitionswissenschaft 3: 24–37
18. Self J 1985 A perspective on intelligent computer-assisted learning. Journal of Computer Assisted Learning 1: 159–66
19. Wenger E 1987 Artificial Intelligence and Tutoring Systems. Morgan Kaufmann, Los Altos, CA