Lecture Notes in Computer Science

DReSDeN: Towards a Trainable Tutorial Dialogue Manager to Support Negotiation Dialogues for Learning and Reflection

Carolyn P. Rosé and Cristen Torrey

Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh PA, 15213

{cprose,ctorrey}@cs.cmu.edu

Abstract. This paper introduces the DReSDeN¹ tutorial dialogue manager, which adopts a similar Issues Under Negotiation approach to that presented in Larsson [20]. Thus, the information state that is maintained in DReSDeN represents the items that are currently being discussed as well as their inter-relationships. This representation provides a structure for organizing the representation for the interwoven conversational threads [26] out of which the negotiation dialogue is composed. We are developing DReSDeN in the context of the CycleTalk tutorial dialogue system that supports the development of critical thinking and argumentation skills by engaging students in negotiation dialogues. We describe the role of DReSDeN in the CycleTalk tutorial dialogue system, currently under development. We then give a detailed description of DReSDeN’s underlying algorithms and data structures, illustrated with a working example. We conclude with some early work in using machine learning techniques to adapt DReSDeN’s behavior.

1 Introduction

Current tutorial dialogue systems focus on a wide range of application contexts including leading students through directed lines of reasoning to support conceptual understanding [27], clarifying procedures [33], or coaching the generation of explanations for justifying solutions [32], problem solving steps [1], predictions about complex systems [10], or descriptions of computer architectures [13]. Formative evaluation studies of these systems demonstrate that state-of-the-art computational linguistics technology is sufficient for building tutorial dialogue systems that are robust enough to be put in the hands of students and to provide useful learning experiences for them. In this paper we introduce DReSDeN, a new tutorial dialogue planner as an extension of the APE tutorial dialogue planner [12]. This work is motivated by lessons learned from the first generation of tutorial dialogue systems, with a focus on Knowledge Construction Dialogues [27,16] that were developed using the APE framework.

The DReSDeN tutorial dialogue planner was developed in the context of the CycleTalk thermodynamics tutoring project [29] that aims to cultivate self-monitoring skills by training students to ask themselves valuable questions about the choices they make in a design context as they work with the CyclePad articulate simulator [11]. The CycleTalk system is meant to do this by engaging students in negotiation dialogues in natural language as they design thermodynamic cycles, such as the Rankine Cycle displayed in Figure 1. A thermodynamic cycle processes energy by transforming a working fluid within a system of networked components (condensers, turbines, pumps, and such). Power plants, engines, and refrigerators are all examples of thermodynamic cycles. In its initial development, the CycleTalk curriculum will emphasize the improvement of the simple Rankine cycle. Rankine cycles of varying complexities are used in steam-based power plants, which generate the majority of the electricity in the US.

Figure 1: A Simple Rankine Cycle

Beyond understanding thermodynamics concepts and how and why individual factors can affect the efficiency of a cycle, design requires students to weigh and balance alternative choices in order to accomplish a particular purpose. Furthermore, design requires not only a theoretical understanding of the underlying science concepts but also a practical knowledge of how these concepts are manifest in the real world under non-ideal circumstances. Because of the intense demands that design places on students, we hypothesize that design problems will provide the ultimate environment in which students will be stimulated to construct knowledge actively for themselves. Figure 2 contains an example dialogue between a human tutor and a student discussing design trade-offs in connection with a rankine cycle. This is an actual dialogue extracted from a corpus of dialogues between Carnegie Mellon University Mechanical Engineering graduate students (as tutors) and Mechanical Engineering undergrads (as students) while working together on a Rankine cycle optimization task, although some details have been omitted for simplicity.

Notice that in the dialogue in Figure 2. the student and tutor are negotiating the pros, cons, hows, and whys of alternative design choices. Negotiation dialogues are composed of multiple, interwoven threads, each addressing a single proposal under negotiation [26]. The dialogue in Figure 2 begins with a single thread that addresses the general topic of factors that affect cycle efficiency. In turn (6), the student introduces a subordinate thread (i.e., thread 1) that addresses one specific way to increase cycle efficiency. Next, the tutor introduces a second subordinate thread (thread 2), parallel with thread 1, that introduces a second method for improving cycle efficiency. In turn (9), the tutor builds on this by further elaborating the proposal in thread 2. Thus, the resulting thread 3 is subordinate to thread 2. Two additional parallel threads are introduced in turns (11) and (13) respectively. In turn (15), the focus shifts back to the original more general topic, and then returns to subordinate thread 3. Notice that this dialogue proceeds in a mixed-initiative fashion, with both the tutor and the student taking the initiative to introduce proposals for consideration. This is significant since our ultimate goal is to provide only as much support as students need, while encouraging them to take more and more leadership in the exploration process as their ability increases [7]. This is not typical with state-of-the-art tutorial dialogue systems that normally behave in a highly directed fashion. Thus, one of our challenges has been to develop a dialogue manager that can support this type of interaction. Note that our focus is not to encourage the students to take initiative in the dialogue [8], but in the exploratory task itself. Allowing the student to take initiative at the dialogue level is simply one means to that end.

Figure 2: Example Negotiation Dialogue

In the remainder of the paper, we outline the theoretical motivation for the DReSDeN tutorial dialogue manager. We then describe how it is used in the CycleTalk tutorial dialogue system, currently under development. We then give a detailed description of DReSDeN’s underlying algorithms and data structures, illustrated with a working example. We conclude with some early work in using machine learning techniques to adapt DReSDeN’s behavior.

2 Motivation

The development of the DReSDeN tutorial dialogue manager was guided by concerns specifically related to supporting negotiation and reflection in a tutorial dialogue context. The role of DReSDeN in CycleTalk is to support student exploration of the design space, encourage students to consciously reflect on the design choices they are making, and to offer feedback on their ideas.

The idea of using negotiation dialogue for instruction is not new. For example, Pilkington et al. (1992) argue the need for computer based tutoring systems to move to more flexible types of dialogues that involve challenging and defending arguments to support students’ information gathering processes. When students participate in the argumentation process, they engage higher-order mental processes, including reasoning, critical thinking, evaluative assessment of argument and evidence, all of which are forms of core academic practice [24]. Negotiation provides a context in which students are encouraged to adopt an evaluative epistemology [18], where judgments are evaluated using criteria and evidence in order to weigh alternatives against one another. Baker (1994) argues that negotiation is an active and interactive approach to instruction that is an effective mechanism for achieving coordination of both problem solving and communicative actions between peer learners, or between a learner and a tutor. It keeps both conversational participants equally active and engaged throughout the process. Nevertheless, the potential for using negotiation as a pedagogical tool within a tutorial dialogue system has not been thoroughly explored. While much has been written about the potential for negotiation dialogue for instruction, very few controlled experiments have compared its effectiveness to that of alternative forms of instruction, and no current tutorial dialogue system that has been evaluated with students fully implements this capability.

On a basic level, the DReSDeN flavor of negotiation shares many common features with the types of negotiation modelled previously. For example, all types of negotiations involve agents making proposals that can either be accepted or rejected by the other agent or agents. Some models, such as [5,15,9], also provide the means for modeling justifications for choices as well as the ability to modify a proposal in the light of objections received from other agents. Nevertheless, at a deep level, the DReSDeN flavor of negotiation is distinctive. In particular, previous models of negotiation are primarily adversarial in that the primary goal of the dialogue participants is to agree on a proposal or even to convince the other party of some specific view. The justifications and elaborations that are part of the conversation are in service to the goal of convincing the other party to adopt a specific view, or at least a mutually acceptable view. In the DReSDeN flavor of negotiation, on the other hand, the main objective is to explore the space and to reflect upon the justifications. Thus, the underlying goals and motivation of the tutor agent are quite different from previously modeled negotiation style conversational agents and may lead to interesting differences in information presentation and discourse structure. In particular, while the negotiation dialogues DReSDeN is designed to engage students in shares many surface features with previously explored forms of negotiation, the underlying goal is not to convince the student to adopt a particular decision or even to come to an agreement, but instead to motivate the student to reason through the alternatives, to ask himself reflective questions, and to make a choice with understanding that thoughtfully takes other alternatives into consideration.

Much prior work on managing negotiation dialogues outside of the intelligent tutoring community is based on dialogue game theory [22] and the information state update approach to dialogue management [31,19]. Larsson (2002a, 2002b) presents an information state update approach to managing negotiations with plans to implement it in the GoDiS dialogue framework [4]. The information state in his model is a representation of Issues Under Negotiation, that explicitly indicates what has been decided so far and which alternative possible choices for as yet unmade decisions are currently on the table. Lewin (2001) presents a dialogue manager for a negotiative type of form filling dialogue where users negotiate the contents of a database query, including both which pieces of information are required as well as the values of those particular pieces. The DReSDeN tutorial dialogue manager adopts a similar Issues Under Negotiation approach to that presented in Larsson (2002b). Thus, the information state that is maintained in DReSDeN represents the items that are currently being discussed as well as their relationships to one another. This representation provides a structure for organizing the representation for the interwoven conversational threads [26] out of which the negotiation dialogue is composed.

We build on the foundation of our prior work building and evaluating Knowledge Construction Dialogues (KCDs) [27]. KCDs were motivated by the idea of Socratic tutoring. KCDs are interactive directed lines of reasoning that are each designed to lead students to learn as independently as possible one or a small number of concepts, thus implementing a preference for an “Ask, don’t tell” strategy. When a question is presented to a student, the student types a response in a text box in natural language. The student may also simply click on Continue, and thus neglect to answer the question. If the student enters a wrong or empty response, the system will engage the student in a remediation sub-dialogue designed to lead the student to the right answer to the corresponding question. The system selects a subdialogue based on the content of the student’s response, so that incorrect responses that provide evidence of an underlying misconception can be handled differently than responses that simply show ignorance of correct concepts. Once the remediation is complete, the KCD returns to the next question in the directed line of reasoning.

KCDs have a very simple underlying dialogue management mechanism, specifically a finite state push down automaton. Thus, they do not make full use of the reactive capabilities of the APE tutorial dialogue manager. They make use of very simple shallow semantic parsing grammars to analyze student input, classifying it into one of a small number of pre-defined answer classes. A set of accompanying authoring tools [16] makes it possible for domain experts to author the lines of reasoning underlying the KCDs. These authoring tools have been used successfully by domain experts with no technical or linguistic background whatsoever. KCDs invite students to enter freeform natural language responses to tutor questions. These tools make KCD development fast and easy. The most time consuming aspect of developing a knowledge construction dialogue is taking the time to thoughtfully design a line of reasoning that will be compelling enough to facilitate student understanding and student learning. Thus, the simplicity of the KCD technology allows developers to invest the majority of their time and energy on pedagogical concerns.

Thus, KCDs are a means for directly encoding the pedagogical content knowledge that is required to teach a concept effectively. Nevertheless, while KCDs have proved themselves robust enough to stand up to evaluations with real students, they fall short of the ideal of human tutorial dialogue. For example, KCDs are designed to lead students through a predetermined directed line of reasoning. While they have the ability to engage students in subdialogues when they answer questions incorrectly, they are designed to keep the student from straying too far away from the main line of reasoning. In order to do this, they respond to a wide range of responses that do not express the correct answer to questions in the same way. The DReSDeN tutorial dialogue manager provides a level of dialogue management above the level of individual KCDs. The goal is to build on what was valuable in the KCD approach while enabling a more flexible dialogue management approach that makes it practical to support mixed initiative and multi-threaded negotiation dialogues.

3 Dialogue Management in DReSDeN

In this section we discuss the main data structures and control mechanisms that are part of the implemented DReSDeN dialogue manager and present a working example that uses toy versions of the required knowledge sources. Further developing these knowledge sources is one of our current directions. DReSDeN has four main data structures that guide its performance. First, it has access to a library of handwritten KCDs. We also plan to generate some KCDs on the fly using a data structure called an ArgumentMap that encodes domain information to provide the foundation for the negotiation or discussion. The KCD library contains lines of reasoning used for exploring pros and cons of typical design scenarios and for remediating deficits in conceptual understanding that are related to issues under negotiation. The KCD library also contains generic KCDs for eliciting explanations and design decisions from students. Next, there is a threaded discourse history, generated in the course of a conversation, which is a graph with parent-child relationships between threads. Each thread of the discourse is managed separately with its own KCD like structure. The flexibility in DReSDeN comes from the potential for multiple threads to be managed in parallel. The final data structure, the discourse model describes the rules that determine how control is passed from one thread to the next.

Each dialogue begins with a single thread, initiated with a single KCD goal. With the initiation of this thread, a tutor text is produced in order for the dialogue system to introduce the topic of discussion. When the student responds, the system must decide whether the student’s text addresses the currently in focus thread, a different thread, or begins a new thread. This decision is made using the discourse model, which is a finite state machine. Each state is associated with rules for determining how to relate the student’s turn to the discourse history as well as rules for determining what the tutor’s next move should be. For example, part of this decision is whether the tutor should continue on the currently in focus thread, shift to a different existing thread, or create a new thread. Currently the conditions on the rules are implemented in terms of a small number of predicates implemented in Lisp. In the next section we discuss how we have begun experimenting with machine learning techniques to learn the conditions that determine how to relate student turns to the discourse history.

Figure 4 presents a sample working example. This example was produced using a discourse model that favors exploring alternative proposals in parallel. In its KCD library, it has access to a small list of lines of reasoning each exploring a different proposal as well as a thread for comparing proposals. It’s discourse model implements a state machine that first elicits proposals from the student until the student has articulated the list that it is looking for. Each proposal is maintained on its own thread, which is created when the student introduces the proposal. After all proposals are elicited, the discourse model causes the focus to shift from parallel thread to parallel thread on each turn in a round robin manner until each proposal has been explored. It then calls for the introduction of a final thread that compares proposals and elicits a final decision.

F
(1) Tutor(thread 0): How do you think you could make the simple Rankine cycle more efficient?

(2) Student(thread 1): you could add a reheat cycle

(3) Tutor(thread 2): What is something else you could try?

(4) Student(thread 2): you could try heating up the water more so that it's hotter when it exits the boiler

(5) Tutor(thread 1): Adding a reheat cycle is a good idea. Can you explain why?

(6) Student(thread 1): It rejects less heat to the environment. Also, it lowers the pressure of the steam before it enters the compressor.

(7) Tutor(thread 2): Sounds good. Superheating the water is a good idea. Can you explain how it would help the efficiency?

(8) Student(thread 2): it causes the phase change of the liquid to occur at a lower level

(9) Tutor(thread 1): Okay. How would you implement a reheat cycle?

(10) Student(thread 1): add an extra heater and an extra turbine

(11) Tutor(thread 2): Fine. Can you explain how you would implement your super heated water solution?

(12) Student(thread 2): Just crank up the heater

(13) Tutor(thread 3): Fine. Which of these options would you choose?

(14) Student(thread 3): i think reheat I would pick the reheat option.

(15) Tutor(thread 3): I would choose superheating the fluid because it is the simplest choice, and keeping it simple is good.
igure 3 Example DReSDeN Dialogue about Rankine Cycle Design Possibilities
See Figure 3 for a dialogue created using this dialogue model. First a thread is introduced into the discourse in turn (1) for the purpose of negotiating design choices about improving the efficiency of a rankine cycle. Next, two separate threads, each representing a separate design choice suggested by the student in response to a tutor request are introduced in turns (2) and (4) and processed in turn using a general elicitation KCD construct. Both of these threads are related to the initial thread via a design-possibility relation. Control passes back and forth between threads as different aspects of the proposal are explored. Note the alternating thread labels. After the final design choice elicitation thread is processed, an additional thread, which is subordinate to the two parallel threads just completed, is introduced in order to encourage the student to compare the two proposals and make a final choice, to which the student responds by suggesting the addition of a reheat cycle, a preference observed among the students in our data collection effort. The system responds by offering an alternative suggestion. As noted, with an alternative discourse model, this dialogue could have been processed using a different strategy in which each alternative proposal was completely explored in isolation, in such a way that we would not observe the thread switching phenomenon observed in Figure 3.

4 Machine Learning Techniques for Adapting DReSDeN’s Behavior

Our learning hypothesis is that negotiation dialogue will prove to be a highly effective form of tutorial dialogue. Within that framework, however, there exist a multiplicity of more specific research questions about how this expansive vision is most productively implemented in tutorial dialogue. Many local decisions must be made in the course of a negotiation that influence the direction that negotiation will take. Examples include which evidence to select as supporting evidence, which alternative design choice or prediction to argue in favor of, or when to challenge a student versus when to let the student move on. When the goal is to encourage exploration of a space of alternatives rather than to lead the student to a pre-determined conclusion, then there are many potential answers to all of these questions. Thus, we will explore the relative pedagogical effectiveness of alternative strategies for using negotiation in different contexts. Part of our immediate plans for future work is to explore this space using a machine learning based optimization approach such as reinforcement learning a [30] or Genetic Programming [17]. The learned knowledge will be encoded in the discourse model that guides the management of DReSDeN’s multi-threaded discourse history.

In the KCD approach to dialogue management [27], student answers that do not express a correct answer to a tutor query are treated as wrong answers. Thus, one challenge in expanding from a highly directed, tutor dominated approach to dialogue management to a mixed-initiative one is to distinguish the cases where the student is taking the initiative from the cases where the student’s answer is wrong. Thus, we began our explorations of machine learning approaches to adapting DReSDeN’s behavior by addressing the problem of distinguishing between student answers to tutor questions and student initiatives. We used as data the complete transcripts from 5 students’ corresponding with human tutors over a typed, chat interface, while working on a rankine cycle optimization problem. Altogether, the corpus contains 484 student contributions, 59 of which were marked as student initiatives by a human coder. We considered as student initiatives unsolicited observations, predictions, suggestions, and questions (apart from hedges [3]).

We used Ripper [6] as a classification algorithm to learn rules for distinguishing student initiatives from direct answers based on the bag of words present in the student contribution. The initial results were discouraging, yielding only a 10% reduction in error rate over the initial baseline error rate of 12.2% that would be obtained by consistently assigning the majority class. However, we noticed that the difficulty seemed to arise from trouble learning rules to distinguish hedges from true questions. Thus, in a second round of experimentation, we used Ripper again to distinguish student contributions that were either initiatives or hedges from other contributions. This time there was a 17.6% baseline error rate. In a 10 fold cross-validation evaluation, Ripper was able to learn rules to reduce this error rate to 5.8%, roughly one third of the baseline error rate. Furthermore, a simple heuristic of considering complete sentences to be true questions and fragments to be hedges yielded an accuracy of 82% over all student questions, and 87% over the full set of initiatives+hedges. Our encouraging preliminary results demonstrate that very simple techniques can make significant headway towards solving this important problem. We expect to be able to achieve better performance than this in practice since students tend not to use hedges with tutorial dialogue systems [3], and since the dialogue context normally provides strong expectations for student answers that can be used to unambiguously determine that correct answers constitute direct answers rather than initiatives.

5 Conclusion and Current Directions

In this paper we have introduced the DReSDeN tutorial dialogue manager as an extension to the APE tutorial dialogue planner used in our previous research. We currently have a working prototype implementation of the DReSDeN. We are continuing to collect Wizard-of-Oz data in the thermodynamics domain, which we plan to use as the foundation for building our domain specific knowledge sources and for continued machine learning experiments as described.

Acknowledgements

This project is supported by ONR Cognitive and Neural Sciences Division, Grant number N000140410107.

References

1. Aleven V., Koedinger, K. R., & Popescu, O.: A Tutorial Dialogue System to Support Self-Explanation: Evaluation and Open Questions. Proceedings of the 11th International Conference on Artificial Intelligence in Education, AI-ED (2003).

2. Baker, M.: A Model for Negotiation in Teaching-Learning Dialigues, International Journal of AI in Education, 5(2), pp 199-254, (1994).

3. Bhatt, K., Evens, M. & Argamon, S.: Hedged Responses and Expressions of Affect in Human/Human and Human/Computer Tutorial Interactions, Proceedings of the Cognitive Science Society (2004).

4. Bohlin, P., Cooper, R., Engdahl, E., Larsson, S.: Information states and dialogue move engines. In Alexandersson, J. (Ed.) IJCAI-99 Workshop on Knowledge and Reasoning in Practical Dialogue Systems, (1999) pp 25-32.

5. Chu-Carroll, J., Carberry, S.: Conflict resolution in collaborative planning dialogues. International Journal of Human-Computer Studies, 53(6):969-1015. (2000)

6. Cohen, W.: Fast Effective Rule Induction. Machine Learning: Proceedings of the Twelfth International Conference. (1995)

7. Collins, A., Brown, J. S., Newman, S. E.: Cognitive Apprenticeship: Teaching the Crafts of Reading, Writing, and Mathematics, in L. B. Resnick (Ed..) Knowing, Learning, And Instruction: Essays in Honor of Robert Glaser, (1989) Hillsdale: Lawrence Earlbaum Associates.

8. Core, M. G., Moore, J. D., & Zinn, C.: The Role of Initiative in Tutorial Dialogue, in Proceedings of the 10^th Conference of the European Chapter of the Association for Computational Linguistics. (2003)

9. Di Eugenio, B., Jordan, P., Thomason, R., Moore, J.: The Acceptance Cycle: An empirical investigation of human-human collaborative dialogues, International Journal of Human Computer Studies. 53(6), (2000) 1017-1076.

10. Evens, M. and Michael, J.: One-on-One Tutoring by Humans and Machines, (2003) Lawrence Earlbaum and Associates.

11. Forbus, K. D., Whalley, P. B., Evrett, J. O., Ureel, L., Brokowski, M., Baher, J., Kuehne, S. E.: CyclePad: An Articulate Virtual Laboratory for Engineering Thermodynamics. Artificial Intelligence 114(1-2): (1999) 297-347.

12. Freedman, R.: Using a Reactive Planner as the Basis for a Dialogue Agent, Proceedings of FLAIRS 2000, (2000) Orlando.

13. Graesser, A., Moreno, K. N., Marineau, J. C.: AutoTutor Improves Deep Learning of Computer Literacy: Is It the Dialog or the Talking Head? Proceedings of AI in Education (2003)

14. Graesser, A., VanLehn, K., the TRG, & the NLT: Why2 Report: Evaluation of Why/Atlas, Why/AutoTutor, and Accomplished Human Tutors on Learning Gains for Qualitative Physics Problems and Explanations, LRDC Tech Report, (2002) University of Pittsburgh.

15. Heeman, P. and Hirst, G.: Collaborating on Referring Expressions. Computational Linguistics, 21(3), (1995) 351-382.

16. Jordan, P., Rosé, C. P., & VanLehn, K.: Tools for Authoring Tutorial Dialogue Knowledge. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.), Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future, Proceedings of AI-ED 2001 (pp. 222 233). (2001) Amsterdam, IOS Press.

17. Koza, J.: Genetic Programming: On the programming of computers by means of natural selection, (1992) Bradford Books.

18. Kuhn, D.: A developmental model of critical thinking. Educational Researcher. 28(2), (1999) pp 16-26.

19. Larsson, S. & Traum, D.: Information state dialogue management in the trindi dialogue move engine toolkit. NLE Special Issue on Best Practice in Spoken Language Dialogue Systems Engineering, (2000) pp 323-340.

20. Larsson, S.: Issue-based Dialogue Management, PhD Dissertation, Department of Linguistics, Goeteberg University, Sweden (2002)

21. Larsson, S.: Issues Under Negotiation, Proceedings of SIGDIAL 2002.

22. Levin, J. A. ;  Moore, J. A.: ‘Dialogue-Games: Meta-communication Structures for Natural Language Interaction’. Cognitive Science, 1 (4),  (1980) 395-420. 

23. Lewin, I.: Limited Enquiry Negotiation Dialogues, Proceedings of Eurospeech (2001).

24. McAlister, S. R.: Argumentation and a Design for Learning, CALRG Report No. 197, (2001) The Open University

25. Pilkington, R. M., Hartley, J. R., Hintze, D., Moore, D.: Learning to Argue and Arguing to Learn: An interface for computer-based dialogue games. International Journal of Artificial Intelligence in Education, 3(3), (1992) pp 275-85.

26. Rosé, C. P., Di Eugenio, B., Levin, L. S., Van Ess-Dykema, C.: Discourse Processing of Dialogues with Multiple Threads, Proceedings of the Association for Computational Linguistics (1995).

27. Rosé, C. P., Jordan, P., Ringenberg, M., Siler, S., VanLehn, K., & Weinstein, A.: Interactive Conceptual Tutoring in Atlas-Andes, In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.), Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future, Proceedings of AI-ED 2001 (pp. 256-266). (2001) Amsterdam, IOS Press.

28. Rosé, C. P., Roque, A., Bhembe, D., VanLehn, K.: A Hybrid Text Classification Approach for Analysis of Student Essays, Proceedings of the HLT-NAACL 03 Workshop on Educational Applications of NLP (2003).

29. Rosé, C. P., Aleven, V. & Torrey, C.: CycleTalk: Supporting Reflection in Design Scenarios With Negotiation Dialogue, CHI Workshop on the Designing for the Reflective Practitioner (2004).

30. Sutton, R. S., & Barto, A. G.: Reinforcement Learning: An Introduction. (1989) The MIT Press: Cambridge, MA.

31. Traum, D., Bos, R., Cooper, R., Larsson, S., Lewin, I, Mattheson, C., & Poesio, M.: A model of dialogue moves and information state revision. (2000) Technical Report D2.1, Trindi.

32. VanLehn, K., Jordan, P., Rosé, C. P., and The Natural Language Tutoring Group: The Architecture of Why2-Atlas: a coach for qualitative physics essay writing, Proceedings of the Intelligent Tutoring Systems Conference, (2002) Biarritz, France.

33. Zinn, C., Moore, J. D., & Core, M. G.: A 3-Tier Planning Architecture for Managing Tutorial Dialogue. In S. A. Cerri, G. Gouardères, & F. Paraguaçu (Eds.), Proceedings of the Sixth International Conference on Intelligent Tutoring Systems, ITS 2002 (pp. 574-584). Berlin: Springer Verlag, (2002).

1 DReSDeN stands for Debate-Remediate-Self-explain-for-Directing-Negotiation-dialogues.