Academic rigour in science assessment tasks

ACADEMIC RIGOUR IN SCIENCE ASSESSMENT TASKS by Mary Lincoln DipT(BKTC), BEdSt, GradDipEd(TeachLib), GradDipLibSc, MAppSc(Lib&InfoMgt) This thesis is submitted in fulfilment of the requirements for the degree of Doctor of Philosophy (PhD) Centre for Learning Innovation Queensland University of Technology 2010 KEYWORDS Academic rigour, academic tasks, assessment tasks, alignment, assessment for learning, case study, creativity, higher order thinking, Information and Communication Technologies (ICTs), intellectual challenge, knowledge construction, middle years of schooling, prior knowledge, scaffolding, science, socio-cultural theory, student engagement, teachers‟ understandings i ii ABSTRACT The critical problem of student disengagement and underachievement in the middle years of schooling (Years – 9) has focussed attention on the quality of educational programs in schools, in Australia and elsewhere The loss of enthusiasm for science in the middle years is particularly problematic given the growing demand for science professionals Reshaping middle years programs has included an emphasis on integrating Information and Communication Technologies (ICTs) and improving assessment practices to engage students in higher cognitive processes and enhance academic rigour Understanding the nature of academic rigour and how to embed it in students‟ science assessment tasks that incorporate the use of ICTs could enable teachers to optimise the quality of the learning environment However, academic rigour is not clearly described or defined in the literature and there is little empirical evidence upon which researchers and teachers could draw to enhance understandings This study used a collective case study design to explore teachers' understandings of academic rigour within science assessment tasks The research design is based on a conceptual framework that is underpinned by socio-cultural theory Three methods were used to collect data from six middle years teachers and their students These methods were a survey, focus group discussion with teachers and a group of students and individual semi-structured interviews with teachers Findings of the case study revealed six criteria of academic rigour, namely, higher order thinking, alignment, building on prior knowledge, scaffolding, knowledge construction and creativity Results showed that the middle years teachers held rich understandings of academic rigour that led to effective utilisation of ICTs in science assessment tasks Findings also indicated that teachers could further enhance their understandings of academic rigour in some aspects of each of the criteria In particular, this study found that academic rigour could have been further optimised by: promoting more thoughtful discourse and interaction to foster higher order thinking; increasing alignment between curriculum, pedagogy, and assessment, and students‟ prior knowledge; placing greater emphasis on identifying, activating and building on prior knowledge; better differentiating the level of scaffolding provided iii and applying it more judiciously; fostering creativity throughout tasks; enhancing teachers‟ content knowledge and pedagogical content knowledge, and providing more in-depth coverage of fewer topics to support knowledge construction Key contributions of this study are a definition and a model which clarify the nature of academic rigour iv TABLE OF CONTENTS Keywords i Abstract iii Figures x Tables xi List of abbreviations xii Statement of originality xiii Acknowledgements xiv Refereed publications xv Chapter Introduction 1.1 Preamble 1.2 Chapter overview 1.3 Context of the study 1.3.1 International context 1.3.2 National context 1.3.3 Queensland 1.4 Significance 1.4.1 Inadequate intellectual demands in middle years programs 1.4.2 Limited research on academic rigour 1.4.3 ICTs with assessment for learning 1.4.4 Academic rigour and challenging tasks 1.4.5 Declining student interest in science 1.5 Purpose of the study 1.6 Conceptual framework and research design 10 1.7 Overview of the document 10 1.8 Chapter summary 12 Chapter Literature review 13 2.1 Introduction 13 2.2 Middle years of schooling 13 2.2.1 The middle years learner 14 2.2.2 Implementation and effectiveness of programs in America 16 2.2.3 Implementation and effectiveness of programs in Australia 18 2.2.4 Summary 25 2.3 Academic rigour 26 2.3.1 Studies of academic rigour 26 2.3.2 Higher order thinking 31 2.3.3 Alignment 38 2.3.4 Building on prior knowledge 42 2.3.5 Scaffolding 45 2.3.6 Knowledge construction 50 2.3.7 Summary 57 2.4 Assessment 57 2.4.1 Assessment for Learning 57 2.4.2 Summary 63 2.5 Information and communication technologies 64 2.5.1 Teacher use of information and communication technologies 64 2.5.2 Use of information and communication technologies for learning 67 2.5.3 Use of information and communication technologies for assessment 71 2.5.4 Summary 73 v 2.6 Conceptual framework 73 2.7 Chapter summary 75 Chapter Research design and methods 77 3.1 Introduction 77 3.2 Case study design 77 3.2.1 Appropriateness of case study design for this study 77 3.2.2 Research questions 80 3.2.3 The participants 80 3.2.4 The professional development program 82 3.2.5 The science assessment task 85 3.2.6 Stages of the study 87 3.2.7 The role of the researcher 88 3.2.8 Map of academic rigour 90 3.3 Data collection 91 3.3.1 Surveys 92 3.3.2 Semi-structured interviews 92 3.3.3 Focus groups 95 3.4 Data analysis 100 3.4.1 Three-level process of data analysis 100 3.4.2 Use of criteria for data analysis 101 3.4.3 Cross-case analysis 102 3.5 How data analysis was conducted for each source of data 103 3.5.1 Data analysis of survey 103 3.5.2 Data analysis for concept map and definition 104 3.5.3 Data analysis to address RQ1 105 3.5.4 Data analysis to address RQ2 106 3.5.5 Data analysis of the student focus group discussion 107 3.5.6 Data analysis of the teacher focus group discussion 107 3.6 Issues of quality 110 3.6.1 Trustworthiness 110 3.6.2 Compliance with ethical standards 112 3.7 Chapter summary 114 Chapter Question 1: teachers‟ perceptions 115 4.1 Introduction 115 4.2 Teachers’ group definition and concept map of academic rigour 115 4.2.1 Group definition of academic rigour 116 4.2.2 Concept map of academic rigour 117 4.3 Criteria One: Higher order thinking 120 4.3.1 Higher order thinking for academic rigour 120 4.3.2 Affordances of ICTs for fostering higher order thinking 123 4.3.3 Socio-cultural contexts for fostering thinking with ICTs 127 4.4 Criteria Two: Alignment 129 4.4.1 Alignment helps establish academic rigour 130 4.4.2 Alignment is strengthened through explicitness in task documentation 135 4.4.3 Alignment supports more rigorous use of ICTs in assessment tasks 138 4.4.4 Alignment helps create a challenging but supportive environment 139 4.5 Criteria Three: Building on prior knowledge 145 4.5.1 Building on prior knowledge for academic rigour 145 4.5.2 Identifying and activating prior knowledge 151 4.5.3 Building on different types of prior knowledge 154 vi 4.6 Criteria Four: Scaffolding 157 4.6.1 Relevance of scaffolding for academic rigour 157 4.6.2 The role of the teacher in scaffolding 162 4.6.3 Types of scaffolding 167 4.6.4 ICTs and scaffolding 175 4.7 Criteria Five: Knowledge construction 180 4.7.1 Approaches to knowledge construction in science 180 4.7.2 Active engagement in constructing scientific knowledge 186 4.7.3 Opportunities to be creative in science 188 4.8 Perceptions of academic rigour 191 4.8.1 Using ICTs in assessment 191 4.8.2 Summary of teachers‟ perceptions 192 4.9 Chapter summary 198 Chapter Discussion of results of teachers‟ perceptions 199 5.1 Introduction 199 5.2 Teachers’ perceptions of higher order thinking 199 5.3 Teachers’ perceptions of alignment 202 5.4 Teachers’ perceptions of building on prior knowledge 210 5.5 Teachers’ perceptions of scaffolding 213 5.6 Teachers’ perceptions of knowledge construction 217 5.7 Chapter summary 220 Chapter Results: Collective explication view 221 6.1 Introduction 221 6.2 The case of Alice: Collective explication view 221 6.3 Task design: Building on prior knowledge 224 6.3.1 The importance of building on prior knowledge 224 6.3.2 Intentionally activating prior knowledge 225 6.3.3 Challenging and extending the students 226 6.3.4 Building on various types of prior knowledge 228 6.3.5 Making learning meaningful 229 6.4 Task set-up: Alignment 231 6.4.1 Aligning teaching and learning experiences 231 6.4.2 The task structure 232 6.4.3 Providing a task example 232 6.4.4 Task explicitness 233 6.4.5 Aligned use of ICTs 234 6.4.6 Some alignment to prior knowledge 235 6.4.7 Establishing a positive climate 235 6.5 Task implementation: Scaffolding 236 6.5.1 Scaffolding contributed to academic rigour 236 6.5.2 Collaboration in scaffolding 237 6.5.3 Scaffolding ICTs use 238 6.5.4 Regulating the level of scaffolding 239 6.6 Task completion: Higher order thinking 241 6.6.1 Evidence of higher order thinking 241 6.6.2 Time and effort 241 6.6.3 The use of ICTs fostered higher order thinking 242 6.6.4 ICTs provide a shared focus for collaboration and thinking 243 6.6.5 Limited emphasis on in-depth knowledge of few topics 244 6.7 Task completion: Knowledge construction 245 vii 6.7.1 The identification and explanation of scientific concepts 245 6.7.2 Engaging and actively involving students 247 6.7.3 Evidence of creativity in presenting science knowledge 248 6.8 Chapter summary 250 Chapter Results: Concept application view 251 7.1 Introduction 251 7.2 The case of Emily: Concept application view 251 7.3 Task design: Building on prior knowledge 253 7.3.1 The importance of building on prior knowledge 253 7.3.2 Intentionally activating prior knowledge 255 7.3.3 Challenging and extending the students 255 7.3.4 Making learning meaningful 256 7.3.5 Building on various types of prior knowledge 257 7.4 Task set-up: Scaffolding 259 7.4.1 Scaffolding contributed to academic rigour 259 7.4.2 Role of collaboration in scaffolding 260 7.4.3 Scaffolding use of ICTs 260 7.4.4 Regulating the level of scaffolding 261 7.5 Task implementation: Knowledge construction 262 7.5.1 Knowledge construction contributes to academic rigour 263 7.5.2 The application of scientific concepts 264 7.5.3 Engaging and actively involving students 265 7.5.4 Opportunities to be creative in science 266 7.6 Task completion: Higher order thinking 267 7.6.1 Evidence of higher order thinking 267 7.6.2 Higher order thinking is fostered through effort 268 7.6.3 Higher order thinking with ICTs 269 7.6.4 Limited emphasis on collaboration 269 7.7 Task completion: Alignment 270 7.7.1 Alignment assisted in establishing academic rigour 271 7.7.2 Alignment helped foster a positive climate 271 7.7.3 Task clarity, explicitness and front-ending assessment 273 7.7.4 The aligned use of ICTs 273 7.8 Chapter summary 274 Chapter Discussion of results of case studies 277 8.1 Introduction 277 8.2 Teachers’ enactment of higher order thinking 277 8.2.1 Reflections on higher order thinking 282 8.3 Teachers’ enactment of alignment 282 8.3.1 Reflections on alignment 286 8.4 Teachers’ enactment of building on prior knowledge 287 8.4.1 Reflection on building on prior knowledge 291 8.5 Teachers’ enactment of scaffolding 292 8.5.1 Reflections on scaffolding 296 8.6 Teachers’ enactment of knowledge construction 296 8.6.1 Reflections on knowledge construction 301 8.7 Creativity for academic rigour 302 8.8 Summary of case study findings 303 8.9 Chapter summary 307 Chapter Conclusion and implications 309 viii ... Limited research on academic rigour 1.4.3 ICTs with assessment for learning 1.4.4 Academic rigour and challenging tasks 1.4.5 Declining student interest in science ... role in shaping their practice, this study investigated middle years teachers‟ understandings of academic rigour in the context of incorporating ICTs in middle years science assessment tasks. ..KEYWORDS Academic rigour, academic tasks, assessment tasks, alignment, assessment for learning, case study, creativity, higher order thinking, Information and Communication Technologies (ICTs), intellectual