In Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference, C. Hoadley & J. Roschelle (Eds.) Dec. 12-15, Stanford University, Palo Alto, California. Mahwah, NJ: Lawrence Erlbaum Associates.
University of Edinburgh, Human Communication Research Centre
Glasgow Caledonian University, Centre for Learning and Teaching Innovation
Abstract: The Vicarious Learner project is investigating the role of dialogue in learning and, more specifically, how learners benefit from opportunities to 'overhear' other learners. We describe evidence of such learning by students using "vicarious learning resources" in a computer-supported learning environment. We argue that education dialogue has properties that are different from "everyday" dialogue. We see educational dialogues in particular as being about the complex alignment of concepts where the participants know that an initial misalignment is fairly certain. We focus on the distinction between exposition and derivation in discourse and discuss how this might describe what happens in learning dialogues.
Keywords: dialogue, vicarious learning, overhearers, conceptual alignment
Dialogue is clearly an important aspect of a rich learning experience, of the sort one hopes to provide in higher education for example. We have argued elsewhere (McKendree et al 1998) that dialogue is central to the learner's "enculturation" into the patterns of language and thought, discussion and criticism, that are characteristic of an academic discipline; we have noted also that, on a more detailed and local level, dialogue is often the most effective way for a learner to overcome, for example, a particular impasse during problem-solving, or to resolve a difficult conceptual issue. We have proposed, further, that learning can occur not only through participation in dialogue, but also through observing others participating in it. We call this vicarious learning, and we believe that in some manifestations it is very common, as when silent students in a tutorial group learn from discussions between the tutor and others or when "lurkers" on a discussion list benefit from reading contributions of others. We are working to understand this process more clearly, and to investigate the ways in which we can take advantage of it to create databases of re-usable dialogues as a helpful resource for distance learners and others.
In several experimental laboratory studies and university courses, we have developed and refined our understanding of how to design, capture, store, index, retrieve and re-use educational discussions. This new type of learning resource holds out promise both for isolated or distance learners and for students who might need greater exposure to language 'put to hard conceptual labour' than they are likely to get in traditional educational encounters in the classroom and lecture hall.
Our research on vicarious learning has found benefits that are both cognitive, resulting in an increase in knowledge and understanding in the particular curriculum area, and social. One social effect is that exposure to peer discussion creates positive feelings of being part of a learning community. Perhaps more importantly for our goals, we also find that students rapidly begin to model the language and structure of the discussions to which they were exposed. We think that this introduction into 'specialised arenas of language use' is precisely the type of result that we would want to foster, as well as promoting learning of new domain knowledge.
We are particularly interested in how to better incorporate dialogue into Computer-Supported Cooperative Learning systems. We think it is important to keep in mind that in some cases actual opportunities to engage in dialogue may be restricted. Resources can be scarce. For instance, in the UK, local calls are charged by the minute, so people may not be able to chat for hours each week, but may prefer to log into a chat area only for very specific, focused discussions that they feel are productive. Another problem we have encountered is that a course may have only a few students. Coordinating times for participants in several time zones can be a limiting factor on the amount of synchronous discussion possible.
Thus, we think that one possibility is to make recordings of various types of discussions available to students as 'reusable vicarious learning resources'. These could be included on a CD or could be downloaded and viewed off-line as well as being held in a central server as is the customary model. First, however, we need to establish that such resources are indeed of any use to learners and to present our theory about why such "overhearing" of dialogue might be beneficial.
Our initial question was, "Do vicarious learning resources promote learning?" If the resources do not seem to increase understanding over the usual textbook or multimedia expository materials, then it is hardly worth the effort of capturing and storing such resources. So, an initial study compared primary expositions - text and worksheets - to versions annotated with dialogue. A second study looked at a wider variety of media and focused more specifically on the language aspects of vicarious learning and how overhearers acquire the language of a new domain.
In our course on Human Communication, students are faced with the problem of learning formal techniques for analysing English sentences. They typically find this quite difficult. There were exercises in constructing syntax-trees depicting the grammar of sentences, and other kinds of formal diagrams, and the students experienced many problems in completing these. We developed a computer-based tool to assist the students in creating and editing these diagrams - a typical piece of instructional courseware. We were now able to experiment with different kinds of vicarious material that might further support the students' learning.
Using this system, we looked at the difference between expert monologue and student-tutor dialogue as learning materials (Cox et al., in press). In the one case, a tutor constructed a diagram while explaining the activity for the benefit of students; this was captured as a movie of the manipulation of the computer tool, along with a transcript of the tutor's commentary. In another case, a novice student constructed the diagram, with assistance from the tutor where needed - which was often. These materials were presented to the students also as animated diagrams accompanied by transcripts of the recorded speech. We observed that although there was no clear difference between students given the dialogues and those given the "direct instruction" tutorial monologues, both of these produced significantly better results than conditions where students were given only animations of the diagrams or only "primary text" materials. So it does seem that these dialogic materials can increase students' understanding when attached to more traditional instructional material.
Another experiment looked at learning effects, but in particular we focused the analysis on the impact of vicarious resources on the students' acquisition of language and discussion skills reflected in the vicarious learning resources. Dineen developed a set of Task-Directed Discussions (TDDs) that focus dialogue between a tutor and student or between peers on problematic or complex concepts in a domain (Lee et al., 1998). This results in more pointed, shorter, and deeper discussions for reuse than a typical small group discussion or free-form Internet session. We videotaped over 30 hours of discussions among students, and between students and an expert, using the TDDs. From primary instructional materials and integrated clips taken from these videos, we created an architecture that allows a multimedia database of video and audio clips, text transcriptions, and annotated graphics to be integrated with primary expository teaching material, all delivered via the Web. Using this system, an experiment was run to investigate the efficacy of 'vicarious learning' resources in a controlled laboratory setting.
The experimental system used a portion of an on-line Masters level course in Computers in Teaching and Learning that had been taught twice before. A self-contained section on Models of Learning with Technology was extracted. In addition, there were a set of "vicarious learning" resources integrated into the on-line readings.
Thirty-six students were divided into two groups based on their pretest scores on a knowledge test of the domain and attitude questionnaires. This resulted in two groups who did not differ significantly on knowledge, stated media preferences, or attitude toward discussion. One group saw the Web-based primary learning material only (Notes group); the other saw the same material with the additional vicarious learning resources (Vicarious group).
On the final day of the experiment, after three days of studying the material, the students were divided into groups of either two or three (the same across groups) and participated in a 40 minute, on-line, synchronous discussion of the material using Internet Relay Chat. They were simply told to discuss the course content to clear up anything they did not understand. They then took a final knowledge posttest.
In this paper, we are concentrating primarily on the discussion content. (Results are presented in more detail in Lee et al, 1999.) Our initial analyses indicate a number of interesting differences between the groups reflected in the way they engage in discussion.
We found significant differences in the amount of discussion generated, averaging 834 words for the Notes group and 1075 words for the Vicarious group, (Mann-Whitney U=9.0, p<.04). More importantly, when scored by a blind rater for the relevance of each statement, the Vicarious students stayed on topic significantly more than the Notes group (82% vs 68%).
Looking more closely at the patterns of interaction, we examined the instances where the discussion strayed from the content of the course to unrelated topics. This generally occurred either because no one could think of more questions or topics to discuss, or because no one knew the answer to the current question. We counted the number of typed utterances between the initial one that was judged to be off-topic and the one where the group was judged to be again on topic. We found that the Vicarious group had significantly shorter stretches of off-topic utterances than the Notes group as judged by the rank order of number of utterances (Mann-Whitney U = 139, p<.0001). The Vicarious group had a large number of instances in which only one or two utterances are off-topic, ie. a single person interjects an irrelevant comment which is ignored by the other participants or there is perhaps one acknowledging response before the group resumes discussion of the course topics. In the Notes group, while this happens occasionally, there were far more long stretches of irrelevant banter.
What is perhaps most interesting to us is that when the students engage in discussions themselves, we find that those who had seen the vicarious resources were modelling the tasks and language used in them. For instance, in Vicarious groups, when the discussants ran out of things to say on a topic, they sometimes suggested trying one of the discussion games they had seen in the resources, as in the following excerpt:
<Discussant1> OK, perhaps we could just pick a concept and try and describe it to another person.
<Discussant2> well you can go first then
<Discussant3> accretion?
<Discussant2> adding of new knowledge to existing schema, most common form of learning
<Discussant1> Accretion is the second stage in the learning process, after structuring, and it involves adding bits of knowledge into a schema
<Discussant3> ah, I see.
The students in the Notes group, when the discussion hit a lull, tended to talk about many unrelated topics: World Cup football, television, holidays, and Elvis.
Further exploratory analysis of the discussions showed significantly more occurrences of several educationally relevant discourse features for the group exposed to vicarious materials. This group:
This is further valuable evidence of the power of watching dialogues to influence positively the subsequent behaviour of students in discussions. These effects are all likely to have beneficial educational consequences in the longer run. If vicarious learning is, on our current evidence, effective in promoting this "social modelling" type of learning than traditional expository teaching methods, it may encourage valuable practices that promote learning skills across domains.
While these results are promising in themselves, we think it is important to indicate why we believe that important effects result from such vicarious encounters and to have some theory about what aspects of educational dialogue make it particularly amenable to capture and reuse for overhearers in contrast to typical 'everyday' conversations. Many of the best learning dialogues involve the making explicit of inferences which would normally remain unspoken. Assumptions are made by the participants which often turn out not to be shared, not to be part of the common ground needed for understanding (Clark, 1996). A process has to occur that results in these assumptions being aligned. Often, though not necessarily, this means that the student has to be brought more into line with the tutor; but we find that this occurs most effectively not through the student simply being told (accepting an exposition) as much as being led through the reasoning involved. By working through a more or less logical process of derivation, the authority of the tutor is largely abrogated in favour of the more abstract authority of reasoning norms that can be explicitly discussed and agreed, the procedure can be observed. The student thus learns, not only about the subject matter, but also about the process of argumentation. The development of the common ground being thus relatively out in the open, derivational dialogues are well suited to vicarious following of the process by overhearers.
It is likely that in many cases of vicarious learning, dialogues are also in other respects specifically developed in such a way as to facilitate the overhearer's construction of a common understanding. Our hypothesis in the Vicarious Learning project is that, by capturing such learning discourse, new learners listening in may be included in a more diverse conversation; the alternative is that much grounding may remain unexplored and available only to the expert. This facilitation of a common ground often happens in tutorial sessions, where the tutor may, for the benefit of the others, be more explicit than would be strictly necessary for the student being directly addressed. It certainly happens in other homely examples such as radio discussion and interview programmes, for example with politicians, where the presenter will (if sometimes in a relatively heavy-handed manner) interrupt to demand expansion of abbreviations, to elicit repetition and clarification, or even to interpolate information which the speaker is taking for granted but which might not be shared by all of the audience. In cases like these, the dialogue participants are well aware that there are or will be overhearers, and they take this into account. The very fact that this happens, of course, acts on the one hand as evidence that overhearers are known to have effective interpretative strategies, but on the other as evidence that these strategies require specific assistance from the dialogue participants.
In a brief example from a classroom, Clark (1996, p.298) states that the classroom is a closed situation in which the teacher has the greater 'goods' to transfer and there is generally no more negotiation needed. It is true that his examples are ones in which that might be the case, requesting a student to sit down or to answer a simple factual question. However, in the more complex situations that we would consider examples of more effective learning dialogues, there is a great deal of negotiation and the unequal equity that assigns power to the teacher is not nearly as obvious.
Many language models such as Clark's tend to focus on common ground essentially as the naming and identifying of discourse objects, often ones that could be picked out by ostension. It does not as clearly address building and organising a dialogue aimed at, in our case, the particular goal of bringing almost guaranteed misalignment of abstract conceptual knowledge into better alignment. The fundamental abstract concepts of most fields are exactly those where ostension is of little use (see Stenning 1999). It is true that Clark's models are not attempting to account necessarily for learning dialogues and many examples are from more mundane exchanges in shops and on street corners. However, Clark (1996) and Schober and Clark (1989) at least imply that the model should apply to such dialogues and that only direct participants in a discourse are able to establish such alignment to a desirable degree.
Clark's model seems to agree with Sperber and Wilson's (1986) claim from 'relevance theory' that the first interpretation is the one speakers are most likely to 'latch into', whereas in educational conversations this first reading is very likely to be wrong or at least incomplete and the students are well aware of that. (Note, though, that Clark rejects other aspects of Sperber and Wilson.) Clark's model does not deal with conversations in which participants know that a conceptual misalignment in 'common ground' is just about guaranteed. In these circumstances, we want the students to challenge a speaker in ways that might be unacceptable in another context. Such dialogues may provide a very good means for indirect participants, overhearers, to vicariously establish not only the common ground, but to challenge their own understanding through comparison with the example dialogues.
How can we characterise what is special about educational communication, especially the type of educational dialogue where conceptual learning is the goal? We start from a distinction between exposition and derivation (Stenning 1978). An expository relation between one sentence and the next in a discourse is one in which the second sentence provides new information to be added to that provided by the first. This information is normally to be accepted on the authority of the speaker (in educational dialogue this is often the teacher) and it is assumed that the hearer does not already know this information. In logical terms, it is the presentation of a new assumption. This kind of transition is typical of a narrative or descriptive discourse. When we are telling a story, we generally don't expect the listener to already know what happens next.
In contrast, when one sentence is derived from previous one(s), it adds no new information, and its assertion is founded not on the authority of the speaker, but because it is merely a new expression of what is already shared by both participants. For example, the exchange "X is a prime number greater than 2. X is odd." exhibits a derivational relation. By definition, it provides information the hearer already has, if they share an understanding of the concepts. In a purely expository discourse, such a transition is aberrant because it is unnecessary to spell out the common ground and could even be condescending to do so (as Grice (1975) observes in his Maxim of Quantity which states that a speaker should say only as much as is necessary to be informative).
In educational dialogue about conceptual issues, doing explicit derivation is one way of providing information about conceptual alignment. To take a rather contrived case of conceptual learning, the following dialogue might arise between an academic (A, perhaps a rather obtuse communicator?) and someone only partially acquainted with the rather arcane employment relations in a university department (L). Perhaps the partial foreignness of the UK jargon helps illustrate the uninitiated student's epistemic position:
A: John's a member of staff, not a student.
L: He has tenure!
A: No, he's not faculty.
L: He's a computer officer or a secretary?
A: No, he's academic-related.
L: They're librarians, aren't they?
A: No he's contract research staff, so he's not faculty but he's an untenured academic-related member of staff.
In each case, L makes what she assumes to be a derivation from an assertion of A's. Since in this situation A is the expert and L is novice, and A also has the particular knowledge of John, its clear to A that these are derivations by L, and its possible for A to correct the misunderstanding which the derivation exhibits. Whereas valid derivation provides no 'object-level' information (if it did, the derivation would be invalid), but does provide meta-level information about conceptual relations. The fact that the derivation is valid is part of what defines what we mean by the concepts involved. Derivations may be highly informative about concepts, but only at the metalinguistic level - they can tell us things about what our words mean.
Doing explicit derivation is often useful in educational dialogue, because it provides information about conceptual interpretation. Learning new concepts is usually a matter of modifying the interpretations of old words. The teacher modelling derivation 'out loud' can be useful for the student by revealing the intended conceptual relations; the student's attempt at derivation is highly informative to the teacher about the students' misconceptions. However, doing derivation 'out loud' is only informative if we realise that derivation is what it is. In the following dialogue there is confusion about what is exposition and what is derivation:
A: Your friend is a member of staff, not a student.
L: Does he have tenure?
A: Only if he's faculty.
L: He's a computer officer or a secretary?
A: He's academic-related.
L: They're librarians, aren't they?
A: Not necessarily. If he's contract research staff, he's not faculty but he's an untenured academic-related member of staff.
Here A knows the concepts but not who is being talked about, and L knows the identity of the individual but not fully the concepts. A may think that L knows the concepts and that L is merely trying to ascertain the facts about his friend, whereas L is really trying to learn what the conceptual relations are (perhaps as well as the facts, or maybe not)? At the second utterance, L may be asking whether A is asking whether the friend has tenure (ie. asking about whether staff implies tenure) - a derivation from information about known relationships. A, however, interprets it as a question about the friend - a request for expositional information. The confusion continues.
This rather trivial example of conceptual confusion seems to us illustrative of a common problem in educational communication about concepts. Derivation is a powerful way of providing information about concepts, but derivation is an aberrant discourse relation in common expository discourse, and it can be unclear whether a particular assertion is a derivation or an exposition. We can provide some clues but we can't expect to be universally understood. Connectives like 'so', for example can help to mark derivation, but in domains where the concepts are abstract and not easily interdefinable, or related to earlier conceptual interpretations of the same words in complex ways distinguishing derivation from exposition is a major task, both for teacher and for student. `So' also marks other relations in exposition. Students may not be used to the idea that uttering apparently vacuous truths can be a useful method for achieving conceptual alignment. Students may not even realise that conceptual alignment is what is at stake.
We believe it is also very likely that much of the dialogue in computer supported cooperative learning systems will also involve a need for this negotiation of acceptable mutual understanding, whether the dialogue is between the human and a computer, or between two humans mediated by technology. It is also important to keep in mind that complex understanding will evolve and may require revisiting of concepts at various points. Thus, we feel that traditional language models are too limited to guide what may be the most crucial aspects of design of effective cooperative learning systems, both in the characterisation of common ground as primarily a 'one-shot' problem of naming and identification, and also in its claim, at least in Schober and Clark (1989), that only direct participants can learn effectively from discourse.
Understanding the difference in kind between the informativeness of exposition and of derivation, opens up the possibility of deepening the concept of agency in communication. Derivation internalises agency. Once we are aware that derivation can re-represent what we know, and that it rests on conceptual authority rather than teacher's authority, we can truly become active in our learning. Frustratingly for our teachers, at least some students can do it while they lurk.
We want to re-emphasise that we do not see vicarious learning as a replacement for direct participation, but in these days of growing numbers and dwindling resources, especially in universities, as well as increasing delivery of Internet courses for industrial training and remote students, we view vicarious learning resources as providing useful additional learning materials, affective support through increasing the feeling of sharing in a learning community, and a means of more effective immersion into the language and practice of students' chosen areas.
This research was supported by the UK ESRC Cognitive Engineering programme and UK EPSRC Multimedia and Networking Applications programme. HCRC is an Interdisciplinary Research Centre funded by ESRC. Other members of the project were Jonathan Kilgour and Richard Tobin.
Clark, H. H. (1996). Using Language. Cambridge: University Press.
Cox, R., McKendree, J., Tobin, R., Lee, J. & Mayes. T. (In press) Vicarious learning from dialogue and discourse: A controlled comparison, Instructional Science.
Grice, H. P. (1975). Logic and Conversation. In P. Cole and J. L. Morgan (Eds.), Syntax and Semantics III: Speech Acts, New York: Academic Press, 41-58.
Lee, J., Dineen, F., McKendree, J. and Mayes, T. (1999). Vicarious Learning: cognitive and linguistic effects of observing peer discussions. Paper presented at AERA '99.
Lee, J., Dineen, F. and McKendree, J. (1998). Supporting student discussions: it isn't just talk, Education and Information Technologies, 3, 217-229.
Technology, Journal of IFIP TC3 Working Group, Kluwer Academic Publishers.
McKendree, J., Stenning, K., Mayes, T., Lee J., and Cox, R. (1998). Why Observing a Dialogue may Benefit Learning: The Vicarious Learner. Journal of Computer Assisted Learning, 14 (2), 110-119.
Schober, M.F. and Clark, H.H. (1989). Understanding by addressees and observers. Cognitive Psychology, 21, 11-232.
Sperber, D. and Wilson, D. (1986). Relevance. Cambridge, MA: Harvard University Press.
Stenning, K. (1978) Anaphora as an approach to pragmatics, in Halle, M., Bresnan, J. and Miller, G.A. (Eds.), Linguistic Theory and Psychological Reality, MIT Press, 162-200
Stenning, K. (1999) Representation and conceptualisation in educational communication. In van Someren, M., Reimann, P., Boshuizen, E.& de Jong, T. (eds.) Learning with multiple representations. Kluwer Academic Press, 321-334