Acquiring Search Expertise: Learning Experiences and Threshold Concepts by Virginia M Tucker BA (Stanford University), MLS (University of California at Berkeley) A thesis presented to Queensland University of Technology Faculty of Science and Engineering School of Information Systems in partial fulfilment of the requirements for the degree of Doctor of Philosophy Principal Supervisor: Prof Christine Bruce, Queensland University of Technology Associate Supervisors: Prof Judith Weedman, San José State University Prof Sylvia Edwards, Queensland University of Technology 2012 Acquiring Search Expertise Virginia Tucker PhD Thesis ©2012 Virginia M Tucker ii Acquiring Search Expertise Virginia Tucker PhD Thesis Keywords online searching; online searchers; search expertise; novice searchers; expert searchers; experienced searchers; threshold concepts; library science instruction; information science instruction; MLIS curriculum; iterative search iii Acquiring Search Expertise Virginia Tucker PhD Thesis Abstract Expert searchers engage with information as information brokers, researchers, reference librarians, information architects, faculty who teach advanced search, and in a variety of other information-intensive professions Their experiences are characterized by a profound understanding of information concepts and skills and they have an agile ability to apply this knowledge to interacting with and having an impact on the information environment This study explored the learning experiences of searchers to understand the acquisition of search expertise The research question was: What can be learned about becoming an expert searcher from the learning experiences of proficient novice searchers and highly experienced searchers? The key objectives were: (1) to explore the existence of threshold concepts in search expertise; (2) to improve our understanding of how search expertise is acquired and how novice searchers, intent on becoming experts, can learn to search in more expertlike ways The participant sample drew from two population groups: (1) highly experienced searchers with a minimum of 20 years of relevant professional experience, including LIS faculty who teach advanced search, information brokers, and search engine developers (11 subjects); and (2) MLIS students who had completed coursework in information retrieval and online searching and demonstrated exceptional ability (9 subjects) Using these two groups allowed a nuanced understanding of the experience of learning to search in expertlike ways, with data from those who search at a very high level as well as those who may be actively developing expertise The study used semi-structured interviews, search tasks with think-aloud narratives, and talk-after protocols Searches were screen-captured with simultaneous audio-recording of the thinkaloud narrative Data were coded and analyzed using NVivo9 and manually Grounded theory allowed categories and themes to emerge from the data Categories represented conceptual knowledge and attributes of expert searchers In accord with grounded theory method, once theoretical saturation was achieved, during the final stage of analysis the data were viewed through lenses of existing theoretical frameworks For this study, threshold concept theory (Meyer & Land, 2003) was used to explore which concepts might be threshold concepts Threshold concepts have been used to explore transformative learning portals in subjects ranging from economics to mathematics A threshold concept has five defining characteristics: transformative (causing a shift in perception), iv Acquiring Search Expertise Virginia Tucker PhD Thesis irreversible (unlikely to be forgotten), integrative (unifying separate concepts), troublesome (initially counter-intuitive), and may be bounded Themes that emerged provided evidence of four concepts which had the characteristics of threshold concepts These were: information environment: the total information environment is perceived and understood; information structures: content, index structures, and retrieval algorithms are understood; information vocabularies: fluency in search behaviors related to language, including natural language, controlled vocabulary, and finesse using proximity, truncation, and other language-based tools The fourth threshold concept was concept fusion, the integration of the other three threshold concepts and further defined by three properties: visioning (anticipating next moves), being light on one’s ‘search feet’ (dancing property), and profound ontological shift (identity as searcher) In addition to the threshold concepts, findings were reported that were not concept-based, including praxes and traits of expert searchers A model of search expertise is proposed with the four threshold concepts at its core that also integrates the traits and praxes elicited from the study, attributes which are likewise long recognized in LIS research as present in professional searchers The research provides a deeper understanding of the transformative learning experiences involved in the acquisition of search expertise It adds to our understanding of search expertise in the context of today's information environment and has implications for teaching advanced search, for research more broadly within library and information science, and for methodologies used to explore threshold concepts v Acquiring Search Expertise Virginia Tucker PhD Thesis Table of Contents Keywords iii Abstract _ iv Table of Contents _ vi Definitions _ xi List of Figures xiii List of Tables xiv Statement of Original Authorship _ xv Acknowledgments xvi Dedication xvi Chapter - Introduction: Background, Research Problem, & Context Background Research Problem _ Purpose _ Research Question _ Aims & Objectives _ 2 3 Research Context Research Interest Research Plan Overview Assessing the Plan 11 Limitations 16 Thesis Overview 16 Chapter Abstracts _ 17 Chapter - Literature Review: The Scene, Backdrop, & Framework _ 23 Introduction _ 23 The Scene: Online Searching The Professional Searcher Confounding Factors Subject Knowledge vs Generic Knowledge _ Operationalist vs Conceptualist Searching _ Search Engine Rapport _ LIS Education in Online Searching Web-based Searching vi 27 29 39 40 46 48 49 54 Acquiring Search Expertise Virginia Tucker PhD Thesis The Backdrop: Novices and Experts Novice-Expert Research Reflection as a Learning Practice of Experts Expertise at Learning 61 61 68 73 The Framework: Threshold Concept Theory and Learning Defining Learning _ Threshold Concept Theory Characteristics of a threshold concept Ambiguity and ontological shift Threshold concepts and curriculum _ Related learning constructs _ Investigating threshold concepts _ 77 77 79 81 87 88 95 97 Summary 100 Chapter - Research Design: Methodology, Data Collection, & Analysis _ 103 Introduction 103 Methodology _ Grounded theory: Introduction _ Approaches to grounded theory Doing grounded theory _ Position of literature review _ Suitability for this study _ Evaluating the study’s theory _ 103 105 106 111 116 119 121 Data Collection _ Participant recruitment _ Pilot study implications _ Interviews Data collection instruments Interview questions Interview venues and timeframes _ Search tasks Protocols: Think aloud and Talk after Researcher as observer _ 122 122 124 127 131 132 135 136 137 139 Data Analysis _ Overlapping phases of analysis _ Cycling through and unpacking the data Researcher as instrument _ 141 141 142 145 Materials and Costs 146 Ethics Clearance _ 147 vii Acquiring Search Expertise Virginia Tucker PhD Thesis Chapter - Results: Emergence of Threshold Concepts, Praxes, & Traits 149 Introduction 149 Participant Demographics _ Subject matter backgrounds of participants _ Teaching background of HE participants Search engine & database experience Participant Profiles _ 150 153 155 157 158 Results by Core Category _ Introduction Category A: Broad view _ A1 Environment, sources, and provider practices A2 Integration, synthesizing, connection making _ A3 Identity motifs _ A4 Rapport motifs _ A5 Information structures _ A6 Lifelong learning, transference of learning _ A7 Codes in multiple categories: Context & meaning _ Category B: Subject domain & databases _ B1 Knowing databases generally _ B2 Combining and synthesizing B3 Using outlier sources B4 Specific subject domains _ Category C: Nature of learning C1 Collaborative learning _ C2 Teaching as learning experience _ C3 Realization of own expertise C4 Reflection, sense making, ah-ha _ Category D: Qualities & approaches _ D1 Qualities, traits, attitudes D2 Approaches, tactics _ D3 Praxis of visioning Category E: Tools/search knowledge _ E1 Language-based tools E2 Structure-based strategies E3 Styles of searching 158 158 160 161 164 166 167 170 171 172 173 174 175 176 177 178 179 181 182 183 185 186 190 195 198 200 203 205 viii Acquiring Search Expertise Virginia Tucker PhD Thesis Category F: Work-related experiences _ F1 Search intermediary experiences _ F2 Teaching experiences F3 Product development experiences _ 207 209 210 211 Thematic Results Introduction Theoretical Lens _ Summary 213 213 213 218 Chapter - Discussion & Conclusions: A Model of Search Expertise 219 Introduction 219 Preparation 221 Threshold Concepts Introduction Threshold concept: Information environment _ Threshold concept: Information structures Threshold concept: Information vocabularies Threshold Concept: Concept fusion Visioning _ Dancing Profound Ontological Shift _ Visualizing the Threshold Concepts 225 225 229 233 236 238 239 241 242 243 Praxes and Traits 245 Unified Model of Research in Search Expertise 248 Extending the Net Lenses Model _ 249 Framing an Integrated Model of Search Expertise 253 Evaluating the Model _ 254 Questions Raised 256 Chapter 6: Summary, Implications, & Future Directions _ 257 Introduction 257 Research Objectives Fulfilled _ 257 Implications & Contributions Implications for LIS education Contribution to Models of Search Experience Contributions to Threshold Concept Theory _ Contribution to Methodology Limitations of Study ix 260 260 262 265 267 268 Acquiring Search Expertise Virginia Tucker PhD Thesis Future Research Directions Explore threshold concepts for non-LIS searchers Explore threshold concepts for other areas within LIS education _ Design and test new advanced search course Study lifelong learning patterns of search professionals Explore implications for search interface design 268 269 269 270 270 271 In Closing 272 References _ 273 Appendices _295 Appendix A Data Collection Instruments _ 296 A.1 Researcher Checklist A.2 Novice-Student (NS) - Information & Consent Form A.3 Novice-Student (NS) - Pre-interview email A.4 Novice-Student (NS) - Interview guide for researcher A.5 Novice-Student (NS) - Search tasks script for participant A.6 Experienced Subject (HE) - Information & Consent Form A.7 Experienced Subject (HE) - Pre-interview email A.8 Experienced Subject (HE) - Questionnaire A.9 Experienced Subject (HE) - Interview guide for researcher A.10 Experienced Subject (HE) - Search tasks script for participant Appendix B Participant Information B.1 Participant Profiles B.2 Participant Data _ 309 Appendix C Survey of MLIS Searching Courses _ 318 C.1 Schools Surveyed: 25 Largest in United States C.2 Respondent Schools and Course Names C.3 Survey Questionnaire x ... existence of threshold concepts in search expertise; Chapter 1: Introduction Acquiring Search Expertise Virginia Tucker PhD Thesis to improve our understanding of how search expertise is acquired and. .. the learning experiences of searchers to understand the acquisition of search expertise The research question was: What can be learned about becoming an expert searcher from the learning experiences. .. novice searchers and highly experienced searchers? The key objectives were: (1) to explore the existence of threshold concepts in search expertise; (2) to improve our understanding of how search expertise