Learning and Practicing Object-Oriented Programming Using a Collaborative Web-based IDE Vu Nguyen, Hai H Dang, Kha N Do, Thu D Tran Faculty of Information Technology University of Science, Vietnam National University – Ho Chi Minh city {nvu, dhhai, dnkha, tdthu}@fit.hcmus.edu.vn Abstract— Collaborative programming is an effective approach to software development, improving software quality, programmer’s satisfaction and shortening delivery time This study examines the application of a collaborative Web-based IDE named IDEOL to execute a four-week multi-submission programming assignment in an introductory object-oriented programming class Forty eight students forming 24 two-member groups in class used the IDE to interact and write source code required by the project All collaborative and programming activities performed by students were recorded by IDEOL The results of the study shows that students tend to postpone their programming work until the submission dates This study also provides an approach to designing and executing an extended programming exercises, which receives high student satisfaction Our results imply that IDEOL is a useful environment for students to collaborate, learn, and practice programming to improve their learning satisfaction In addition, as students tend to procrastinate, IDEOL is a useful tool to facilitate, monitor, and report student progress in extended programming exercises Keywords—Collaborative IDE; Web-based IDE; programming exercise; collaboration; interaction I INTRODUCTION Internet technologies play a key role in teaching and learning programming languages On the one hand, Internet technologies enhance collaboration and interaction between all participants involving in the education process They help improve learning performance and satisfaction [2][3][7] In collaborative environments, students share their perspectives, listen to the views of others, and observe what others are doing to generate knowledge and skills [12][8] On the other hand, the increased trends in distributed programming teams require instructors to prepare students necessary skills to collaborate effectively with members in such environments Collaborative programming has been shown to be an effective approach to accelerating problem-solving process, improving code design and shortening code length, increasing programmers’ satisfaction, which result in improving the programmer productivity and quality of source code delivered [4][5][6] Many Internet tools have been used to support collaboration and interaction in programming groups, ranging 978-1-4799-3922-0/14/$31.00 ©2014 IEEE from general-purpose products such as Skype, Google Chat, and Facebook to specifically tailored systems for development collaboration such as RECIPE [5], Collabode [13], Saros [14], and ATCoPE [15] We developed a collaborative web-based IDE called IDEOL to support teaching programming classes [1] Beside the essential features of a typical integrated development environment such as source code editing, compiling, building, and debugging, the system provides collaboration and interaction capabilities via Web This system allows students and instructors to communicate synchronously and asynchronously by employing social network capabilities such as commenting, liking, and sharing The system allows five most common modes of interaction, including student-student, student-instructor, student-content, instructor-instructor, and instructor-content [16] [17] One key feature of the system is the capability to record user activities, including date and time occurred, time spent on interaction, involving objects and participants, and other attributes of interaction On IDEOL, students are not constrained to posting questions or answers, but they can post any message or comment, which may be a question, an answer, a status, or simply a note on what they are doing or feeling In this study, we use IDEOL for implementing a four-week programming project in an object-oriented programming course All 48 second-year students from two classes taking the OOP course were assigned into 24 two-member groups, and each group was required to use IDEOL to perform the same project All groups worked on the exercise followed the same scenario: overall description of the work was given in the first week; assignments were posted at the beginning of each week; and each group was required to submit its results at the end of each week All collaborations were carried out on the IDEOL system, which records all transactions involved Our objective is to investigate the use of IDEOL to facilitate learning and practicing OOP through the multisubmission programming project We will describe how to use IDEOL to design and execute the project and how IDEOL Fig IDEOL Main Screen helps facilitate collaboration, interaction, and participation of students Through using IDEOL for the programming project, we will investigate student’s working and collaborating profiles on the system and student satisfaction with the system and the project Forty eight students forming 24 two-member groups in class used IDEOL to interact and write source code required by the project All collaborative and programming activities performed by students were recorded by the IDE The results of the study show that students tend to postpone their gramming work until the submission dates, but their ration activities were not delayed as much The results also show high student satisfaction when using the system to collaborate and complete the exercise In addition, this study provides an approach to designing and executing an extended programming exercise, which is useful to cope with student’s procrastination problem II IDEOL: A COLLABORATIVE ENVIRONMENT FOR LEARNING PROGRAMMING A The IDEOL architecture Fig shows the high level architecture of IDEOL In a traditional lab session, students and instructors can discuss issues directly at a computer running IDEOL The system provides core tools for a junior-level programming course, including source-code editing, compilation, execution and debugging In this face-to-face setting, learner-instructor and learner-learner interactions are facilitated through learner- content-instructor interactions Contents can be delivered in an integrated manner, embedded inside IDE tools Thus, interactions between learner-content-instructor through various tools are united within a single environment When out of class, students can interact with other students and instructors online The system supports all three types of interactions online through content sharing and tagging, realtime synchronous collaboration and discussion channels B IDEOL key features IDEOL provides a real-time interactive environment, with core functionalities of a traditional IDE combined with social media mechanism to foster interactions (see Fig and Fig 2) The system also provides data collection mechanisms for analytics A real-time interactive environment with awareness Team members can simultaneously work on a single or multiple source files and have changes by other members updated instantly through the editor Working statuses of members are updated concurrently: one can easily be notified which files the other members are viewing or editing; whether they are testing or debugging the application; or whether they are trying to solve an issue through the discussion board An integrated discussion channel with tagging (see Fig 1) The discussion board is organized into threads with each being an issue or a topic This feature allows users to tag files, specific lines of code, running and debugging results in a discussion Taggin g Discussion board Web browser Instructor Add new assignments Provide project template Monitor progress Post questions Discuss Answer questions etc Source code repository Build log HTTP/REST Web browser WebSocket Student Webserver Edit source code Build program Run tests etc Build automation User program Web browser Execution Student Output Watch edits Discuss Add new issue Debug etc Debug output Debugger Fig IDEOL Architecture Build automation, an interactive execution tool for console applications, and a debugger This feature uses the GNU Compiler Collection tools that allow users to obtain and examine outputs while compiling and debugging Activity tracking The system collects activity data through requests of services sent to servers Complex components at the client side can also track and frequently report non-serviced behaviors, for example, scrolling to view a document These data allow instructors to monitor students’ working progress and trace changes, so that they can be aware of each individual’s strengths and weaknesses and give appropriate adjustments III EXPERIMENT DESIGN In this section, we describe our IDEOL experiment through an extended exercise for an introductory OOP course A Research Questions In this study, we are interested using IDEOL to facilitate collaboration and collect data for analysis through the extended multi-submission programming exercise We investigate how often students collaborate, how much their satisfaction is when using IDEOL More specifically, our research questions are: RQ1: Given the exercise that requires them to submit work frequently, how often students collaborate and programming work via IDEOL? RQ2: How much are students satisfied with the design of the project, the IDEOL system, and collaboration? B The OOP Project The OOP course at our school includes 15 weeks of lectures and 12 weeks of practicing labs, hours each After introducing concepts of the OOP approach, the course guides students towards OO analysis and design, where students are required to practice OO modeling techniques In this experiment, we prepared a 4-week lab project that started at the 5th week of practicing labs (at which time the students had been through half of the course) The project is divided into smaller tasks with each taking approximately a day to complete in a week The full requirements were not provided to the students at the start of the project Instead, we introduced them as newly added requirements every week, to simulate changes and to challenge students to change the design accordingly This project can be seen as a scenario-based game-like group assignment Each week a group must complete the requirements of a task, and thus earns new knowledge At the end of the week, instructors would give comments and explanations on students’ work Students can then move on to the next task, with a sample solution to the previous task provided by the instructors Since each task depends on solutions to the previous tasks, each group must review their current solution against the solution provided by the instructors At the end of these iterations there can be extremely complicated solutions, or some simple yet delicate solutions to a complex problem This scenario-based project therefore requires a non-trivial problem In this experiment, we chose the problem of representing graphical objects and their persistence in the SVG XML-based format The project was organized as follows: • Lab week 1: introduction of two basic graphical objects, line and circle, and their properties according to the SVG 1.1 specification Each group was required to create a preliminary version of the UML class diagram for the objects • Lab week 2: introduction of other basic SVG graphical objects, including rectangle, ellipse, polygon and polyline Tasks included creating classes to represent these objects and their properties, and also creating a class hierarchy using inheritance and polymorphism Students were required to design the objects an UML class diagram with explanatory texts • Lab week 3: a sample class hierarchy design was provided Each group was required to compare this sample design with their own design, comment on weaknesses, strengths and trade-offs, and then revise their own design • Lab week 4: each group was required review its solution in terms of “the evil of duplication”, meaning each group had to check and identify duplication of code in creating objects and in parsing objects’ properties, and then update their design to resolve duplications All groups were required to submit their final solutions C Participants and Procedure A class for the OOP course with 48 second-year students majoring in computer science was chosen to conduct the experiment Each student was ranked by his or her previous achievements (e.g., grades in earlier semesters) and performance in an individual pre-test The students were then paired using their capability ranks to making sure that two students on each group have similar capability We ended up with levels, naming between A and E, to group the students http://www.w3.org/TR/SVG11/ Groups in the same rank are considered equally competent Rank A has groups, B with 2, C with 6, D with 8, and E with groups Pre-test An individual pre-test was conducted prior to the project to check students by their understanding of basic introductory materials The pre-test scores were used in part to rank and group students, as described above Tools and setup The system was put online at https://ideol.net for access at any time For each group an online project was created with the two members and one instructor Most of students had been exposed to the system in our earlier IDEOL experiments [1] We asked those who were not familiar with the system to explore and use it prior to the experiment Weekly tasks and lab sessions Each week’s tasks were given a few days before the lab session Students could discuss with their instructor online through the system The instructors gave brief comments on previous groups’ submissions when posting new materials each week There were weekly submissions during the experiment Post-tests Students and groups were required to complete individual and group post-tests Students were asked to demonstrate their knowledge and skills by working in the tests individually (for individual post-test) and in their groups (for group post-test) The individual post-test included theoretical questions while the group post-test did not Survey and feedback After the experiment, students were asked to complete a 7-point Likert scale survey and an open feedback form These questionnaires are used to gain information about student learning satisfaction and students’ comments on the scenario-based project and IDEOL D Metrics The independent variables measure two types of data, the time spent and participation via the system These independent variables, which are described in detail below, include the time spent on the system by each student, communicating participation, and working participation Time spent on the system (ToS) In general, this is the amount of time that a student uses the system, excluding idle time Idle time is identified as non-working time when there is completely no activity in a period that lasts more than minutes Communicating participation (PC) measures the degree of communication and collaboration between students and instructors through the system This measure is the weighted sum of communicating activities PC = Count of activities by type × Weight of activity type (1) Activities considered as communicating participation include creating a status or discussion thread (CC), updating a thread status, e.g., from ‘unsolved’ to ‘solved’ (CU), reading a status or thread (CV), posting a comment on a thread (CP), and rating a thread or comment (CR) The type-weight pairs are respectively (CC,1), (CU,1), (CV,1), (CP,2), and (CR,1) The weight of activity type is based on the level of contribution and the effort required to perform activities of the type The activity of posting a comment on a thread (CP) is assigned the weight of while the weight of the other activities is assigned because it requires at least twice as much the contribution and effort as the other activities As this metric only includes activities by students, the count excludes instructors’ activities Working participation or programming participation (PW) measures the participation of students by counting student’s programming activities performed on materials (e.g., the source code files and other project files) These activities include opening the project, creating a file, changing a file content, opening a file for reading, removing a file, starting a build session, starting an execution (program running) session, and starting a debug session All these activities are equally weighted The dependent variables measure student learning scores and learning satisfaction The student learning scores are the post-test scores achieved by individual students at the end of the project Learning satisfaction is measured using 7-point Likert scale questionnaire consisting of question items on the system and the project with interactions between students and instructors Each question is rated on a scale from or “strongly disagree” to or “strongly agree”; and the middle rating scale is 4, meaning “neither disagree nor agree” These questions were based on learning effectiveness satisfaction suggested by Sun and Cheng [9] and end-user satisfaction by Doll and Torkzadeh [10] and Davis [18] Examples below are two questions on the system and the project, respectively “The project design helped me learn easier” “Using IDEOL was an enjoyable experience for me” IV RESULTS All activities performed on the IDEOL user interface were recorded automatically by the system Using the activity type for identification, we computed the values of ToS, PW, and PC metrics Many activities such as mouse clicks and key presses were not counted, but they were used to identify time spent on the system and whether a session is active All except one of 24 groups participating in the project completed and submitted work required for each week This group did not participate in the fourth week and did not complete the final submission, and its group members did not Fig Distribution of Activities use the system much for compiling, running, and debugging source code Thus, we excluded this group and their members from the analysis In addition, two of 48 students did not complete the individual post-test required Because of the importance of the post-test score, they and their corresponding groups were not included in the analysis in this section The remaining data, which was collected from 42 students and 21 two-member groups, was used for analysis We measured the participation of students on the project using three metrics, time spent on the system (ToS), working participation (PW), and communicating participation (PC) ToS was determined by totaling the time between activities excluding the idle time that was longer than minutes PW was measured by counting programming activities, such as editing, compiling, running and debugging, that students performed on their source code PC was a count of communication activities, and computed using the formula (1) above A Working and Communicating Profiles Fig shows the distribution of activities performed by students on the system, with the y-axis representing the time spent on the system (ToS) measured in minutes, working participation (PW) and communicating participation (PC) activity counts and the x-axis representing the days during the 4-week project This distribution represents time, working, and communicating profiles of all students in completing the project The project has four weekly submissions at Days 7, 14, 21, and final submission at Day 28 As shown in the figure, the trends for ToS and PW are quite similar The ToS and PW values increase sharply in each of these submission days, meaning that students tend to more programming work on these days A closer look at the data, we found that 13 out of 21 groups spent 30% or more of their total project time on the submission days, with one group spending 2/3 of their total project time on the these days The PC trend is rather different with PW’s as students did more communication, e.g., posting and reading comments and threads, in a few days prior to the submission day Comparing the PC and PW trends, we can see that students tend to communicate across the weeks while they work on programming more on the submission days The profiles also indicate that few activities were taken on the first two to three days following each of the submission days although the requirements for the next submission were already available on those days The project did not require students to much programming in the first week, but required students to design and create UML diagrams for the project The profiles, however, show moderate programming and collaborating activities It is possible that students tried to explore and learn the functionality of the system The work on the second week was significantly higher than the rest, which reflects the week’s requirements with asking students to create classes, their properties, and relationships among them Other than this week, the workload seems to be balanced among the weeks B Student Learning Satisfaction Data on learning satisfaction was collected via the questionnaire consisting of 30 questions that were divided into two main categories, project satisfaction and IDEOL system satisfaction with 15 questions each The project satisfaction questions were used to measure students’ experiences on the schedule, design, and execution of the project The system satisfaction questions focused on how students thought about the IDEOL system, such as its utility, quality, ease of use, friendliness, and efficiency Of 30 questions in the questionnaire, there were six questions asking students about their experience in collaboration and interaction with their group members and instructors through the project execution and the system These questions asked whether students received sufficient information on time, whether the system allowed them to collaborate easily, whether they were overall satisfied with the teamwork and collaboration with other students and instructors We call this category of questions collaboration satisfaction TABLE I AVERAGE VALUES OF LEARNING SATISFACTION Category Average Stdev Project satisfaction 5.0 1.0 System satisfaction 4.5 1.2 Collaboration satisfaction 4.7 1.1 Overall learning satisfaction 4.8 1.1 Table I shows the average levels of overall learning satisfaction and its subcategories, satisfaction on the project, on the system, and on collaboration All of these levels are statistically significantly greater than the average level of 4.0 “neither disagree nor agree” (Student’s One-Sample T-test, p = 0.00 < 0.05) In fact, all of the questions received the level of 4.0 or higher except two questions on the stability and quality of the system with both being rated the average score of 3.8 This score reflects the fact that the system sometimes crashed and Internet connections were slow and not stable during the project Comparing the satisfaction levels of the categories, we found that the project satisfaction levels are statistically greater than those of system satisfaction (Student’s TwoSample T-test, p = 0.00 < 0.05) and collaboration satisfaction (p = 0.01 < 0.05) This result implies that students were more satisfied with the project design and execution than with the IDEOL system and collaboration via the system The satisfaction levels of overall learning, system, and collaboration are not statistically different although the average satisfaction level of collaboration is higher than that of the system V DISCUSSIONS In this section, we present discussions on the results shown in the preceding sections by addressing the research questions and providing our interpretations of the results as well as the limitations of the study RQ1: Given the exercise that requires them to submit work frequently, how often students communicate and programming work on IDEOL? To address this question, we generated the student programming and communicating profiles by presenting counts of student programming and communicating activities throughout four weeks of the project (see Fig 3) The project spanned in four weeks which each has a submission at the end of the week Although students work on the project throughout the days of four weeks, they only spent a significant amount of time on the system and performed many programming activities at each of the submission days Students also did little work on the first few days following the submission date The profiles show that students tend to postpone their programming work until the due date While this academic procrastination is not new in educational literature (e.g., [11]), the profiles provide insights into the procrastination phenomenon in the context of programming exercises From the profiles, we recommend that the extended programming exercise should be designed to include frequent deadlines that are distributed across the timeline This design requires students to better the programming work ahead of the final submission, and helps avoid the case where students made bad estimates and not have enough time to complete the programming exercises During the execution of programming exercises, the data captured by IDEOL provides us useful information to make corrective actions to balance workload for students and instructors throughout the course duration Instructors should adjust the exercise requirements and schedule when the workload reported by the system is too low or too high In this regard, IDEOL becomes a tool to not only report actual time and activities performed but also to gauge and monitor students’ workload RQ2: How much are students satisfied with the design of the project, the IDEOL system, and collaboration? We addressed this question through a survey of students at the end of the project The survey asked students to rate their levels of satisfaction on the design and execution of the project (project satisfaction), the functionality and characteristics of IDEOL (system satisfaction), and the collaboration experience using IDEOL The results shown in Section IV.B indicate that students were overall satisfied with the whole learning experience They were more satisfied with the project design and execution than the IDEOL system Examining further the survey result, we found that students found the system to be unstable and slow Internet connections causing frequent disconnections during their work Moreover, the system’s functionality was quite limited with lack of powerful capability of a typical desktop-based IDE such as real-time error feedback, code completion, and stack trace Although many of such limitations can be resolved in future versions of IDEOL, it is almost impossible to replicate powerful capabilities of popular desktop-based systems such Eclipse and Visual Studio because of limitations of the Web environment Students also rated high on their collaboration experience They found that the system allowed them to collaborate with the peer and instructors easily (satisfaction score = 5.1), and information concerning the project was provided on time (satisfaction score = 5.1) VI THREATS TO VALIDITY The study has several limitations that may present viable threats to validity of our results One concern is that the time and activities captured by IDEOL only reflect just a portion of all actual work done by students on the project We provided students IDEOL to collaborate and program the OOP exercise, but we did not prevent students from using other tools for collaboration While this design reflects the reality, using other more powerful programming tools or popular channels of communication besides IDEOL might alter overall student experience, and thus, affect student satisfaction Students could have used more powerful but less interactive IDEs such as Eclipse and Visual Studio to program and then pasted the source code to IDEOL Moreover, there are many activities that can be performed outside IDEOL such as face-to-face discussion, brainstorming, and drafting solutions on paper From IDEOL, we could not completely detect such activities from students who participated in the exercise Nonetheless, the large number of programming activities taken by students throughout the project via the system leads us to believe that a meaningful portion of work was done using the system, and thus, it is well reflected in the data used for the analysis The second possible threat to validity is that the idle time was set to minutes, which affected the calculation of time spent on the system If a session was idle for more than minutes, we considered that the corresponding student left the system and was no longer working on the exercise However, this assumption may not be true if during this period, the student still worked on the exercise except that he was thinking or working on paper This is a limitation of the current IDEOL, which will be investigated in future enhancements of the system VII CONCLUSIONS This paper reported our study in designing and executing the programming project in our introductory OOP course using IDEOL The project was performed by each of 24 twomember groups in four weeks All communication and programming activities, project requirements, design documents, source code, other materials, and submissions were captured by IDEOL This study reported several interesting results The data captured in IDEOL shows that students tend to postpone their programming work until the submission dates, which confirms the academic procrastination phenomenon known in educational literature To cope with this phenomenon in programming exercises, this study suggests that an extended programing exercise should include multiple deadlines to distribute work during the project IDEOL offers instructors capabilities to monitor and report students’ workload so that project assignments can be adjusted accordingly Moreover, this study provides an approach to designing and executing an extended programming exercise which uses IDEOL for both programming and facilitating collaboration among students and instructors Through a series of submission deadlines, the project requires students to plan their work effectively and thus helps them avoid overloading Additionally, our survey showed high student satisfaction with the design and execution of the programming exercise and the collaboration experience via IDEOL Students found easy to share resources and interact with peers and instructors via the system Although the study has several limitations as we described in the previous section, it suggests several implications First, a collaborative Web-based like IDEOL offers a useful integrated set of features for programming exercises in programming language courses It allows online collaborations and interactions between students and instructors to be taken easily while providing programming capabilities for students to learn and practice programming Via the system, students and instructors can share resources, ideas, and information about the programming exercises Students on the same team can work on the same programming tasks at the same time and make consensus decisions about what and how to When having issues or unsure how to do, students can post their questions or concerns on the embedded discussion board, and other students or instructors can provide feedback instantly or at a later time Instead of using various disintegrated tools for interacting, programming, and storing, students and instructors use the integrated IDEOL system for these purposes to have a consistent view of what they work on Second, the system allows instructors and researchers to automatically capture data from collaborations, interactions, programming activities, team work, and other kinds of activities performed on the system Such data is useful for improving teaching and student learning experience and for verifying learning and teaching approaches We plan to apply the system to experiment and evaluate the effectiveness of different team settings in programming assignments We will also investigate the effects of virtual social network interactions on students’ performance, satisfaction, and involvement in programming courses ACKNOWLEDGMENT This work is supported by Vietnam National University – Ho Chi Minh City (VNU-HCMC) under Grant No B2013-1801 REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] Tran, H., Dang, H., Do, K., Tran, T., and Nguyen, V 2013 “An interactive Web-based IDE towards teaching and learning in programming courses.” Proceedings of 2013 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), pp 439-444 Dillenbourg, P 1999 “Collaborative learning: Cognitive and computational approaches.” Advances in learning and instruction series, Elsevier Science & Technology Books Shaw, R S 2012 “The relationships among group size, participation, and performance of programming language learning supported with online forums.” Computers & Education, vol 62, pp 196-207 Nosek, J T 1998 “The case for collaborative programming.” Communications of ACM, vol 41, no 3, pp 105-108 Shen, H and Sun, C 2002 “Recipe: a web-based environment for supporting real-time collaborative programming.” Proceedings of Internaltional Conference on Networks, Parallel and Distributed Processing, pp 283-288 Williams, L., Kessler, R R., Cunningham, W., and Jeffries, R 2000 “Strengthening the case for pair programming.” IEEE Software, vol 17, no 4, pp 19-25 Hsu, M H., Chen, Y L., Chiu, C M., and Ju, T L 2007 “Exploring the antecedents of team performance in collaborative learning of computer software.” Computers & Education, vol 48, no 4, pp 700-718 Puntambekar, S 2006 “Analyzing collaborative interactions: divergence, shared understanding and construction of knowledge.” Computers & Education, vol 47, no 3, pp 332–351 [9] [10] [11] [12] [13] Sun, P C., and Cheng, H K 2007 “The design of instructional multimedia in e-learning: a media richness theory-based approach.” Computers & Education, vol 49, no 3, pp 662–676 Doll, W J., and Torkzadeh, G 1988 “The measurement of end-user computing satisfaction.” MIS Quarterly, 12(2), pp 259–274 Steel, P 2007 “The nature of procrastination: a meta-analytic and theoretical review of quintessential self-regulatory failure.” Psychological bulletin, 133(1), pp 65 Neo, M 2003 “Developing a collaborative learning environment using a web-based design.” Journal of Computer Assisted Learning, 19(4), 462–473 Goldman, M., Little, G., and Miller, R C 2011 “Real-time collaborative coding in a web IDE.” Proceedings of ACM Symposium on User interface Software and Technology, pp.155-164 [14] Salinger, S., Oezbek, C., Beecher, K., and Schenk, J 2010 “Saros: an eclipse plug-in for distributed party programming.” Proceedings of ICSE Workshop on Cooperative and Human Aspects of Softw Eng., pp 48-55 [15] Fan, H., Sun, C., and Shen, H 2012 “ATCoPE: any-time collaborative programming environment for seamless integration of real-time and non-real-time teamwork in software development.” Proceedings of the 17th ACM international conference on Supporting group work, pp 107116 [16] Moore, M 1989 “Three types of interaction.” American Journal of Distance Education, 3(2), 1-6 [17] Anderson, T., and Garrison, D R 1998 “Learning in a networked world: New roles and responsibilities.” Distance learners in higher education, 97-112 [18] Davis, F D 1989 “Perceive usefulness, perceived of ease of use, and end user acceptance of information technology.” MIS Quarterly (13:3), 319–340 ... collaboration satisfaction TABLE I AVERAGE VALUES OF LEARNING SATISFACTION Category Average Stdev Project satisfaction 5.0 1.0 System satisfaction 4.5 1.2 Collaboration satisfaction 4.7 1.1 Overall learning. .. learn and practice programming Via the system, students and instructors can share resources, ideas, and information about the programming exercises Students on the same team can work on the same programming. .. International Conference on Teaching, Assessment and Learning for Engineering (TALE), pp 439-444 Dillenbourg, P 1999 Collaborative learning: Cognitive and computational approaches.” Advances in learning