374 Y. Yao designed in this way, an instructor no longer has to invest in as much time as is gener- ally assumed. This does not mean that the instructor should stop monitoring the dis- cussion process. Regular instructor visits of the discussion forum are still needed so that the instructor may help when a discussion gets stalled. This study also high- lighted the importance of interaction between students in online discussions [5]. Al- though there is a distinction between interaction and engagement in collective learn- ing, when interaction is made possible, there is more opportunity for students to engage in collective inquiry and benefit from it. The results of the study were based on the discussions from the first six weeks of two online classes. They need to be validated with studies that utilize data on a larger scale. In addition, the learning outcomes were examined through content analysis. Future research may focus on outcomes measured through the student perspectives, in terms of how much learning and what type of learning have resulted in association with the two discussion designs. The learning outcomes may also be measured in more objective terms, such as exam results. Besides student learning outcomes, a comparison of the two discussion designs may also need an examination of student satisfaction of the processes. References 1. Gold, S.: A Constructivist Approach to Online Training for Online Teachers. Journal of Asynchronous Learning Networks 5(1), 35–57 (2001) 2. Li, Q.: Knowledge Building Community: Keys for Using Online Forums. Tech. Trends: Linking Research & Practice to Improve Learning 48(4), 24–28 (2004) 3. Berge, Z.L.: Active, Interactive, and Reflective Elearning. The Quarterly Review of Dis- tance Education 3(2), 181–190 (2002) 4. Hemetsberger, A., Reinhardt, C.: Learning and Knowledge-Building in Open-Source Communities: a Social-Experiential Approach. Management Learning 37(2), 187–214 (2006) 5. Dennen, V.P., Wieland, K.: From Interaction to Intersubjectivity: Facilitating Online Group Discourse Processes. Distance Education 28(3), 281–297 (2007) 6. Baglione, S.L., Nastanski, M.: The Superiority of Online Discussion: Faculty Perceptions. The Quarterly Review of Distance Education 82(2), 139–150 (2007) 7. Pena-Shaff, J., Altman, W., Stephenson, H.: Asynchronous Online Discussions as a Tool for Learning: Students’ Attitudes, Expectations, and Perceptions. Journal of Interactive Learning Research 16(4), 409–430 (2005) 8. Bruyan, L.L.: Monitoring online communication: Can the Development of Convergence and Social Presence Indicate an Interactive Learning Environment? Distance Educa- tion 25(1), 67–81 (2004) 9. Dennen, V.P.: From Message Posting to Learning Dialogues: Factors Affecting Learner Participation in Asynchronous Discussion. Distance Education 26(1), 127–148 (2005) 10. Guldberg, K., Pilkington, R.: Tutor Roles in Facilitating Reflection on Practice through Online Discussion. Educational Technology & Society 10(1), 61–72 (2007) 11. Mazzolini, M., Maddison, S.: When to Jump in: The Role of the Instructor in Online Dis- cussion Forums. Computers & Education 49, 193–213 (2007) Comparing the Impact of Two Different Designs for Online Discussion 375 Appendix: Topics and Instructions for Discussion Fall 2007 Spring 2008 Topics Instructions for Discussions Topics Instructions for Discussions __________________________________________________________________________ About me Please tell us who you are, what High stakes What is your thought on high is your background, and what is testing stakes testing? Please also your favorite hobby. introduce yourself. Assessment What is an assessment topic Benefit of What do you think is the most topic you are most interested in and assessment important benefit for a teacher want to learn more about? Why? knowledge to know about assessment? Reliability In your opinion, how much Test What type of test reliability do and in what specific ways reliability you think is most useful to a should a teacher care about the teacher? Why? reliability of an assessment device? True score Give another name for "true score". True score In what way do you think the Explain in what way it resembles concept of true score is the concept of true score. Make helpful to a teacher? sure you don't use one someone has already mentioned. Validity Give an example of a test that you Test Why do you think a teacher example think is particularly valid or invalid. validity needs to understand the test Explain why you think it is valid or concept of validity? invalid. Test Suppose you developed a new test Test If you developed a test and validation that will be used as the final exam validation want to get a rough idea if the for a course you are teaching. test is valid, what may be a Briefly mention the subject area method you can use to find and the grade level of the test. this out without having to do Discuss what informal method any calculations? may be used to prove the validity of the test. Test bias Have you come across any test Biased Have you come across any that is biased?If so, what makes assessment assessment that you think is the test biased? biased? If so, in what way do you think it is biased? Test Do you perceive a need to Test Do you perceive a need to accom- use test accommodation for accom- use test accommodations for modation any students in your classes? modation any students in your class? If Why? If so, what test accom- so, what accommodations modation will you most likely will you most likely use? use? What student characteristics warrant accommodations? Cognitive Please construct a test question Cognitive Please think of a test level and specify the cognitive level level question or a performance of the question. Discuss if this activity and determine the is an appropriate level for your most likely cognitive level students. for the question or activity. Discuss if this is an appropriate level for your students. 376 Y. Yao Affective Please give an example of an Affective Please give an example of an or psycho- assessment item that may be or psycho- assessment item or activity motor used to assess student learning motor that may be used to assess domain in the affective or psychomotor assessment student learning in the domain. Discuss whether it is affective or psychomotor necessary to have such domain. Discuss if it is. assessment in your class. appropriate to have such assessment in your area of teaching Guideline Out of the various guidelines Guideline Out of the various guidelines for item for writing test items, which for selected for writing selected response construct- guideline is most helpful response test items, which one do you ion to you? Please use an example item find most helpful? Please to explain. use an example to explain. Type of Out of the various types of test Selected Out of the various types of test item items, which one do you think response selected response test items, will be the most useful to you item type which one do you think will in your area of teaching? Please be the most useful to your use an example to illustrate. area of teaching? Please use an example to explain. F. Li et al. (Eds.): ICWL 2008, LNCS 5145, pp. 377 – 389, 2008. © Springer-Verlag Berlin Heidelberg 2008 A Sociogram Analysis on Group Interaction in an Online Discussion Forum Jianhua Zhao School of Information Technology in Education, South China Normal University, Guangzhou, 510631, China jhuazhao@gmail.com Abstract. Group interaction reveals how group members communicate each other when they are working for a group task. The purposes for analysing group interaction are to know how students engaged in the group learning activities and what kinds of social networks are built. 23 students enrol a course and they are organised into 5 groups in this study. Each group has to complete two tasks in a group discussion forum which is offered by WebCL system (an online learning environment). The messages in the discussion forums posted by stu- dents in these 5 groups are analysed through sociogram and the results illustrate the characteristics of group interaction in this course. Keywords: Group Interaction, Online Learning Environment, a Sociogram Analysis, Online Discussion Forum. 1 Introduction Online discussion forum is an increasingly common use of new information and communication technologies in education [1] . Students are engaging in the discussion forums through communication each other, which can enhance learning outcomes, e.g. increasing motivation and engagement in the learning task, deeper levels of un- derstanding, increasing metacognition, the development of higher-order thinking skills and divergent thinking have been shown to result from the adoption of conver- sational modes of learning [2][3][4] . Numerous research have been explored how to ana- lyse discussion forums, such as Thomas examines students’ learning outcomes and the pattern of interaction within an online discussion forum and the findings demon- strate that the typical nonlinear branching structure of online discussion may be insuf- ficient for the realisation of truly conversational modes of learning [1] . Guzdial and Turns claim that simply making a discussion forum available does not mean that will be used effectively to enable learning [5] . Brace-Govan proposes a Moderators’ Matrix for giving moderators a framework through which to sort information about learning activity [6] . When students are organised into online discussion forums, they are nor- mally assigned into different groups as 4, 5 or 6 people. Webb’s series research find group interaction is positively related to achievement [7][8][9][10] , because interaction may help group members understand the material better, develop new perspectives, and construct more elaborate cognitive understanding [11] . We argue that the status of group interaction could help us understand students’ learning behaviour in this social 378 J. Zhao environment. In this paper, group interaction in an online discussion forum is going to be analysed through sociogram. The findings of this research would help us know more about group members’ learning behaviours when they participated in the discus- sion forums. 2 Method 2.1 Sociogram A sociogram is a diagrammatic representation of the valences and degrees of attrac- tiveness and acceptance of each individual ra ted according to the interpersonal inter- actions between and among members of a group [12] . It is a schematic rendering of communication patterns in a group [13] . In the diagram of a sociogram, the cycle points represent group members in a learning group. The “centre” is used for describing group task. When group member communicates with “centre”, the line will not have arrow on it. If an interaction is taken place between different group members, an ar- row will be added on the line and it represents the direction of the interaction. The number on the line is used for indicating the times of the interaction between group members. If the message is sent by one learner, the number will be added closer to her/him. 2.2 Participants 23 third-year undergraduate students who came from School of Information Technol- ogy in Education, South China Normal University participated in the research. They were organized into 5 groups. 3 groups included 5 students, and 2 groups consisted of 4 students. Most of them had experiences of online group learning. They enrolled in a course – “Computer in Education” running in a blended learning environment for 10 weeks. The students had been participated in 8 sessions face-to-face lecturing and 2 online group tasks. The group interaction taken place in those 5 groups when students engaged in 2 online group tasks was analysed through sociogram. 2.3 Online Learning Environment Online learning environment in this research is WebCL which is a web 2.0-based learning system and was developed by Educational Engineering Laboratory (EEL), Beijing Normal University (BNU), China. WebCL system includes six main modules, i.e. “my bookstore, my class, resources centre, communication centre, instructional management centre, and system management”. Many facilities are offered in WebCL system, such as managing groups, discussion forums, blogging, lecturing, tasks plan- ner, online chatting, managing learning resources, online survey, role playing, and publishing learning productions. Each group had its own discussion forum in WebCL and group members can post their contributions to their task in it. Group discussion forum is a group private space in which other person who does not belong to the group cannot join them. Before the products of their group work have been done, they cannot be opened to the public. The interface of group discussion forum is showed in Figure 1. . offered in WebCL system, such as managing groups, discussion forums, blogging, lecturing, tasks plan- ner, online chatting, managing learning resources, online survey, role playing, and publishing. increasing motivation and engagement in the learning task, deeper levels of un- derstanding, increasing metacognition, the development of higher-order thinking skills and divergent thinking have. group interaction in this course. Keywords: Group Interaction, Online Learning Environment, a Sociogram Analysis, Online Discussion Forum. 1 Introduction Online discussion forum is an increasingly