Analyzing Online Asynchronous Discussions with a Visual Data Anal

Lesley University DigitalCommons@Lesley Staff Scholarship Lesley University Faculty and Staff Scholarship Winter 11-2-2012 Analyzing Online Asynchronous Discussions with a Visual Data Analytic Tool (SNAPP) John McCormick Lesley University, jmccormi@lesley.edu Heather Tillberg-Webb Lesley University, heather.webb@lesley.edu Follow this and additional works at: https://digitalcommons.lesley.edu/staff_scholarship Part of the Educational Technology Commons, and the Online and Distance Education Commons Recommended Citation McCormick, John and Tillberg-Webb, Heather, "Analyzing Online Asynchronous Discussions with a Visual Data Analytic Tool (SNAPP)" (2012) Staff Scholarship https://digitalcommons.lesley.edu/staff_scholarship/2 This Conference Proceeding is brought to you for free and open access by the Lesley University Faculty and Staff Scholarship at DigitalCommons@Lesley It has been accepted for inclusion in Staff Scholarship by an authorized administrator of DigitalCommons@Lesley For more information, please contact digitalcommons@lesley.edu, cvrattos@lesley.edu DL6 - Analyzing Online Asynchronous Discussions with a Visual Data Analytic Tool (SNAPP) Schedule Information: Scheduled Time: Fri Nov 2012, 10:30 to 11:30am Building/Room: East- Suite Tower, Grand Ballroom C In Full Session: Posters: Friday- 10:30a Presenters/Authors: *Heather Tillberg-Webb (Lesley University) John McCormick (Lesley University) Short Description: In the age of globalization, learning analytics tools provide real-time visual displays to diagnose learning contexts SNAPP (Social Networks Adapting Pedagogical Practice) is a browser plug-in that generates network diagrams of asynchronous online discussions within a learning management system such as Blackboard Visual diagrams generated by SNAPP were used to study course discussions and recommend design interventions that structure effective communication in the online course environment Abstract: Introduction In the “age of globalization,” construction of effective learning environments takes into account the development of a community of inquiry and incorporates visual tools for quick evaluation of the vibrancy of the learning environment Discussion forums are an essential tool for co-construction of knowledge in a community of practice (Lave and Wenger, 1991) and for supporting a Community of Inquiry model (Garrison et al, 2003) For the purposes this session, effective online discussions are communications that build community and take place in forums designed for threaded, text-based interaction that help learners:  Attain course learning outcomes  Understand course content through collaborative knowledge-construction  Engage in learning via multiple pedagogical formats (debate, role play, etc.)  Forge connections with the instructor and peers Quickly evaluating the quality of interaction in discussions can be difficult using common learning management systems’ visual and numerical display and filtering tools At the same time, learning analytics tools are increasingly used to provide instructors with data to improve the instructional context for learners A team from the Australian Learning and Teaching Council (ALTC) developed a tool, SNAPP, which generates visual networked diagrams of discussion board interactions The report from this team by Dawson et al (2011) identified several diagram patterns that indicate lack of interaction and asserted a typical pattern that indicates a healthy learning community These models were created to assist in course design and facilitation interventions This study will incorporate the model developed by Dawson et al and apply their modeling to online discussions to evaluate the use of learning analytics to inform evaluation of the effectiveness of course design and facilitation Research questions  How discussions in online courses of this study compare with patterns identified by Dawson et al (2001)?  What other methods for determining discussion interaction quality can be used in conjunction with SNAPP data?  How does the SNAPP tool fit with a model for faculty development around online teaching and online course design? Literature Review Some incorporation of visual learning analytics tools to support instructors in understanding patterns in a course has been explored Analyses of the use of learning objects in courses with the LOCO-Analyst tool (Jovanovich, 2008 ) helped instructors determine their impact in coursework Using learning analytics for quickly grasping student interaction in online courses has been identified by instructors as very useful for managing successful distance courses (Mazza and Dimitrova, 2004) Defining quality in online discussions is a prerequisite to using data from learning analytics instruments to craft design and facilitation interventions (Spatariu et al, 2004) Methods suggested by Spatariu et al include levels of disagreement, argument structure analysis, and interaction-based coding, all intensive strategies for analyzing discussion content (2004) With these intense analyses, high-quality discussions can be identified as including higher levels of knowledge -construction (Wise, 2012) The originators of the SNAPP tool (Dawson et al, 2011) have identified models of networked visuals to diagnose performance in a an online discussion:  Facilitator interaction with high performing students  Participant isolation  Facilitator-centric  Non-interaction among groups  Bridging role individuals  Interaction indicating a learning community Dawson et al (2011) also cite recommendations for faculty professional development with the use of the SNAPP tool, in particular the need for assistance in recognizing participation patterns in visual data and intervention techniques with which to improve upon discussions identified as lacking in interaction This study incorporates the existing modeling strategies for online discussion with the SNAPP tool identified by Dawson et al (2011) and explore the relationship with other strategies for categorizing and analyzing discussions for quality and knowledge co-construction Data Collection and Analysis This research study employs action research methods to incorporate SNAPP into an analysis of several sections of online courses over three eight-week terms in Spring and Summer 2012 Each term, three online instructors were selected for professional development using SNAPP At regular intervals in the course, SNAPP visualizations were pulled of online discussions for analysis and comparison to Dawson et al’s modeling A post-facilitation review was used to debrief on all aspects of the online course delivery and the SNAPP visualizations were used to contextualize a conversation about what worked well in online discussions or to make recommendations for changes to the design or facilitation guidelines for the next offering of the course based on problematic patterns The poster session will highlight a specific case or two of online course discussion, the process of working with faculty using SNAPP as a tool to enhance professional development, and the resulting outcomes from the post-facilitation review Conclusion The use of learning analytics and visualization to provide data to improve teaching practice is an exploratory field and presents promise as a method for providing compelling evidence about patterns in a course Online discussions are a critical area for intervention, since so much “presence” in a course is wrapped up in this common activity Sharing cases derived from this study will identify a process to support the efficient identification of discussions needing improvement and a model of design intervention that can practically be applied by practicing instructional designers, faculty developers, and online instructors Works Cited Dawson, S., Bakharia, A., Lockyer, L Heathcote, E (2011) “Seeing” Networks: visualizing and evaluating student learning networks, Final Report 2011 Garrison R., Anderson., T., Archer, W (2003) A theory of critical inquiry in distance education, Handbook of Distance Education, pp 113-127 Jovanovich, J (2008) "Generating Context-related Feedback for Teachers," International Journal of Technology Enhanced Learning (IJTEL), Vol 1, No.1/2, 2008, pp 47 - 69 Lave, J and Wenger, E (1991) Situated Learning: Legitimate Peripheral Participation Mazza R., and Dimitrova, V (2004) Visualising student tracking data to support instructors in web-based distance education, Wide Web conference on Alternate track papers & posters New York, NY, USA: ACMPress, pp 154-161 Retrieved October 7, 2010 fromhttp://www.iw3c2.org/WWW2004/docs/2p154.pdf Spatariu, A., Hartley, K., and Bendixen, L (2004) Defining and measuring quality in online discussions, The Journal of Interactive Online Learning Volume 2, Number Wise, A., Ming Chiu, M (2012) Statistical discourse analysis of a role-based discussion forum: Patterns of knowledge construction  ... - Analyzing Online Asynchronous Discussions with a Visual Data Analytic Tool (SNAPP) Schedule Information: Scheduled Time: Fri Nov 2012, 10:30 to 11:30am Building/Room: East- Suite Tower, Grand... learning analytics tools provide real-time visual displays to diagnose learning contexts SNAPP (Social Networks Adapting Pedagogical Practice) is a browser plug-in that generates network diagrams... instructors with data to improve the instructional context for learners A team from the Australian Learning and Teaching Council (ALTC) developed a tool, SNAPP, which generates visual networked diagrams