Mixed Methods in Educational Research -The Third Research Paradigm Handout

The Third Research Paradigm: Mixed Methods in Educational Research ELC 2019 Belleview, WA Karen Huhn, PT, PhD Husson University Patricia Quinn McGinnis, PT, PhD, FNAP Stockton University Susan Wainwright, PT, PhD Thomas Jefferson University Session Overview OBJECTIVES: Examine the unique contributions and value of mixed methods research designs in physical therapy education research Link educational research questions to mixed methods analysis techniques Identify considerations for developing an educational research agenda Discuss how to establish a mixed method research team Explore resources within the APTE, APTA, and ACAPT to support educational researchers Why Mixed Methods Research?   Multi-pronged strategies needed to address prevention & intervention for public health problems Health sciences research – complex problems:    Genetic / physiologic causes Behavioral determinants Societal / environmental factors  Mixed method can be broader and more complete  Consider ICF Model as framework:   Quant – body structure/function & activity limitations Qual – participation, social & environmental contexts Why Mixed Methods Research? Quantitative Research Examines relationships among variables using numerical data under standardized conditions; tests theories Mixed Method Research Combines methods and capitalizes on the strengths of each; provides better understanding of research problem Qualitative Research Examines role of context and experiences to understand meaning of phenomenon or process; generates new theories Best Practice Guidelines  Best Practices for Mixed Methods Research in the Health Sciences     NIH – OBSSR (2011); Creswell, Plano Clark, Klassen, & Smith http://obssr.od.nih.gov/scientific_areas/methodology/mixed_methods _research/index.aspx For researchers seeking to incorporate mixed methods research into their applications for NIH research, fellowship, career & training grants Includes criteria for evaluating merits of mixed methods research applications Designing Mixed Methods Studies  Study aims & research questions     Specify methods Indicate the type of mixed methods design:     Present rationale for combining qualitative and quantitative strategies Distinct vs integrated aims Concurrent Sequential Embedded Integration of mixed methods during:    Data collection Data analysis Data interpretation Box Table of Strategy, Sample, Goals, and Analysis Gallo, J.J (2003-2007) Strategy Sample Goal Analysis Structured, standardized interviews Stratified random sample (based on depressive symptoms) of older adult patients from non- academic primary care settings Assess depressive symptom patterns and correlates Multivariate regression models Semistructured interviews Purposive: 50 African American and 50 white adults from Spectrum sample (who may or may not be depressed) Identify an explanatory model for depression Grounded theory Free listing and pile sorts First 25 African American and 25 white adults selected above for semi-structured interviews for free listing Second 25 African American and 25 white adults selected above for pile sorts Identify the domain of depression and its characteristics Cultural consensus analysis Ethnographic discoursecentered analysis Purposive: Another 15 African American and 15 white adults who are depressed based on survey of depression responses from the Spectrum sample Identify social meaning Discourse analysis Gallo, J.J (2003-2007) The sociocultural context of depression in late life Research grants funded by the National Institute of Mental Health (R01MH67077, R01MH62210, and R01MH62210-01S1) Design & Integration of Quantitative & Qualitative Data Merge Concurrent data Qualitative Data Results Quantitative data Connect Sequential data (exploratory/explanatory) Qualitatve data Quantitative data Results Embed the data Results Quantitative Qualitative data Adapted from Creswell & Plano Clark, 2007 Additional Considerations  Data analysis / interpretation   Software considerations Dissemination   Word limits Integration Summary for the Review Process      Justify why mixed methods research is the best option Use appropriate methodology for each technique Integrate!! Have a clear & strong theoretical framework Include team members with backgrounds in each method 10 Developing an Educational Research Agenda  Focus on a significant question related to teaching and learning     Recommendation 13 (from PTE-21): Create learning environments in academic and clinical settings that provide opportunities for situated learning experiences that are intentional, sequenced, and occur frequently across the curriculum Seek a theoretical understanding of the question through research/reading Identify a conceptual framework Identify collaborators 22 Developing an Educational Research Agenda   Break the "questions" into parts Design and construct a study     Quantitative Component? Quantitative Component? Evaluate the findings Use your findings to address the same or another piece of the "question" 23 Developing an Educational Research Agenda  Have multiple studies in various stages Reading and Research Design/IRB Data Collection Data Analysis Writing •Study •Study •Study •Study •Study 24 Teamwork: Forming the Mixed Methods Research Team The nature of the research question shapes the nature and experience levels of the research team compiled to address them: questions driving the research initiative determine the expertise required Level One Level Two Multidisciplinary Researchers work in parallel or sequentially from a disciplinaryspecific base to address common problems Interdisciplinary Researchers work jointly but from disciplinary-specific basis to address a common problem Transdisciplinary Level Three Researchers work jointly using a shared conceptual framework drawing together disciplinary-specific theories, concepts, and approaches to address common problems Rosenfield’s (1992) Taxonomy of Three Levels of Collaboration Teamwork: Forming the Mixed Methods Research Team ■ Successful mixed methods teams include breadth and depth of expertise: Address prioritization early  Requires planning & integration of both methodological and disciplinary expertise  Breadth of disciplines relevant to key questions; conceptual & analytic depth to answer them ■ A mixed methods team is more than a team of individual researchers with distinct methodological traditions and skills:  Should transcend distinct methodological & epistemological differences  create processes for integration throughout; ideally engaging multiple members of the research team ■ The relationship of each member of the research team to mixed methods research: It is not necessary for everyone to hold expertise in all methods employed ■ The capacity of a mixed methods research team takes time to develop: Perspective and approach adopted by core team members; evolves over time ■ A team that has a history of successful collaboration is in a strong position to demonstrate capacity for mixed methods research Leading and Guiding a Mixed Methods Research Team ■ The skills and outlook of the PI or research leader are critical:  A broad perspective on the utility of different methodologies  An ability to support and acknowledge different team member contributions  An effort to maintain continual dialogue about issues in working together  A sensitivity to workloads of team members that may pose challenges to working on the project  Support for educating team members in different methodologies, when needed ■ An effective leader should be experienced and interested in qualitative, quantitative, and mixed methods research, as well as a person who is able to bridge differences and create synergy  Will integrate team members who hold distinct methodological positions / expertise and those who cross methodological / disciplinary boundaries ■ ■ A shared vision and defined roles within the mixed methods team is critical A mixed methods team can be structured a variety of ways: a linear relationship, a spoked wheel Different team perspectives need to be recognized and honored ■ Building Infrastructure and Resources ■ All research initiatives benefit from being conducted in a wellresourced and dynamic research environment:  Mixed methods research likely to require wider array of computer software needs than a single method study  Mixed methods teams will benefit from being able to call upon other experienced and informed colleagues at their own institutions  Effective mixed methods research teams require collective as well as individualized workspace  Flexibility for staffing support ■ Team members can profit from mixed methods training and workshops ■ Mixed methods teams may benefit from the ability to work together virtually Training: Mixed Methods Research Team ■ Engaging in mixed methods research can serve as a learning opportunity for team members:  Collaborative educational opportunities → build methodological capacity throughout team  Productivity and effective collaborations largely defined by explicit, shared understanding of each member’s expectations and goals ■ Mixed methods research likely to require additional training of some / all of the research team ■ Appropriate and effective training for mixed methods research need not occur in a traditional classroom setting ■ Mixed methods analysis benefits from team members’ comfort with multiple types of data, analysis methods & approaches ■ Mixed methods research can benefit from adopting training and materials from other scholarly disciplines ■ Building mixed methods expertise should explicitly prioritize collective capacity Strategies to Develop Partnerships Within and Across Institutions   Network at conferences Seek others outside of your profession, who possess specific skill sets that differ from your strengths   i.e Grit Study Explore the literature for researchers who have similar interests; reach out 30 Guiding Questions   How can you apply mixed method techniques to your RQ? What opportunities exist at your institution? 31 Resources to Support Educational Researchers  Grants       GAMER PODs APTE grants Foundation grants MERC AERA 32  SIGs/Consortia     Scholarship of Education SIG CE SIG CR Consortia Journals        Journal of Physical Therapy Education The Journal of Clinical Education in Physical Therapy Academic Medicine Medical Teacher Advances in Health Science Education Health Professions Education Education in the Health Professions 33 Questions???  Thank You!!  Huhnk@husson.edu Patricia.McGinnis@stockton.edu Susan.Wainwright@jefferson.edu   34 REFERENCES Almalki, S (2016) Integrating Quantitative and Qualitative Data in Mixed Methods Research—Challenges and Benefits Journal of Education and Learning, 5(3) doi:10.5539/jel.v5n3p288 Creswell JW Research Design: Qualitative, Quantitative, and Mixed Methods Approaches 2nd Edition Sage Publications; 2003 Creswell J, Plano-Clark V Designing and Conducting Mixed Methods Research Thousand Oaks, CA: Sage Publications; 2007 Creswell JW, Carroll Klassen A, Plano Clark VL, Clegg Smith K Best Practices for Mixed Methods Research in the Health Sciences Office of Behavioral and Social Sciences Research Available at https://obssr.od.nih.gov/wp-content/uploads/2018/01/Best-Practices-forMixed-Methods-Research-in-the-Health-Sciences-2018-01-25.pdf Accessed 9/30/19 Curry L.A, O’Cathain,A, Plano Clark VL, Aroni, R, Fetters M, Berg D The role of group dynamics in mixed methods health sciences research teams Journal of Mixed Methods Research August 29, 2011; pp 5–20 DeJonckheere M, Lindquist-Grantz R, Toraman S, Haddad, K, Vaughn LM Intersection of Mixed Methods and Community-Based Participatory Research: A Methodological Review Journal of Mixed Methods Research 2018.1-22 https://doi.org/10.1177/1558689818778469 Jensen GM, Hack LM, Nordstrom T, Gwyer J, Mostrom E National Study of Excellence and Innovation in Physical Therapist Education: Part 2—A Call to Reform Physical Therapy 2017 97(9):875–888 https://doi.org/10.1093/ptj/pzx062 Jensen GM, Nordstrom T, Segal RL, McCallum C, Graham C, Greenfield B Education Research in Physical Therapy: Visions of the Possible, Physical Therapy, Volume 96, Issue 12, December 2016, Pages 1874–1884, https://doi.org/10.2522/ptj.20160159 Mayoh J, Onwuegbuzie AJ Toward a Conceptualization of Mixed Methods Phenomenological Research Journal of Mixed Methods Research 2015 9;1:91–107 https://doi.org/10.1177/1558689813505358 Mertens DM Publishing Mixed Methods Research Journal of Mixed Methods Research 2011;5(1):3–6 Moseholm E, Fetters MD Conceptual models to guide integration during analysis in convergent mixed methods studies Methodological Innovations 2017 10(2) 1–11 https://doi.org/10.1177/2059799117703118 Rosenfield P The potential of transdisiplinary research for sustaining and extending linkages between the health and social sciences Social Science and Medicine 1992;35(11):1343-1357 Saint Arnault D, Fetters MD RO1 Funding for Mixed Methods Research: Lessons Learned From the “Mixed- Method Analysis of Japanese Depression” Project Journal of Mixed Methods Research 2011;5(4):309–329 Sendall MC, McCosker LK, Brodie A, Hill M, Crane P Participatory action research, mixed methods, and research teams: learning from philosophically juxtaposed methodologies for optimal research outcomes BMC Medical Research Methodology 2018;18(1):167 https://doi.org/10.1186/s12874-018-0636-1 Shulha LM, Wilson RJ Collaborative mixed methods research In A Tashakkori & C Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp 639-669) Thousand Oaks: Sage Publications, 2003 35 Address the shortage of qualified faculty and of academic leadership Recommendation 1: Cultivate shared values of excellence, trust, respect, and collaboration Recommendation 5: Develop shared leadership models that facilitate innovation and excellence Recommendation 6: Foster innovation and risk taking to drive the shared vision Create strong, equal academ- ic-practice partnerships that foster excellence Recommendation 7: Build fair, creative, and responsible partnerships between academic and clinical faculty Recommendation 8: Make clinical faculty full partners with the academic program Recommendation 10: Require early authentic clinical experiences that provide for teaching and learning in the context of practice Infuse the learning sciences into the preparation of academic, clinical, residency, and fellowship faculty Recommendation 12: Implement faculty development programs grounded in the learning sciences that facilitate an understanding of, and competence in, the pedagogy of learning for and through practice Recommendation 13: Create learning environments in academic and clinical settings that provide oppor- tunities for situated learning experiences that are intentional, sequenced, and occur frequently across the curriculum Recommendation 16: Foster opportunities for physical therapist educational researchers to generate an evidence-based approach to teaching and learning Develop a continuum of profession- al performance expectations that are grounded in key competencies and support excellence in learner development Recommendation 14: Require academic programs to participate in residency education to enable reciprocal teaching and learning between professional and postprofessional learners in communities of practice Recommendation 15: Establish a comprehensive, longitudinal approach for standardization of perfor- mance-based learning outcomes grounded in foundational domains of professional competence Recommendation 25: Create a national data set that includes essential metrics of performance outcomes, structures, and processes to guide future evidencebased change Focus curriculum content on soci- etal needs for physical therapist practice Recommendation 2: Demonstrate learner-centered teaching and patient-centered care in all settings Recommendation 9: Make the profession’s signature pedagogy, the human body as teacher, visible in all environments and available for further investigation Recommendation 11: Develop consistent, shared language about the multifactorial movement system that can be used across academic and clinical settings Recommendation 17: Develop a comprehensive, longitudinal approach to teaching, learning and assessment of clinical reasoning abilities Devote significant resources to enhance the diversity in the pro- fession Recommendation 22: Academic institutions must take a leadership role to create more diverse and inclusive learning and practice environments in order for the profession to have a positive impact on addressing the social determinants of health Educate students to become moral agents as health care practitioners Recommendation 18: Develop a strong sense of the moral foundations that underpin and are inseparable from practice in all physical therapists so that they develop the moral courage and ability to meet patients’ needs There must be faculty who have expertise in the moral foundations of practice and who collaborate with other faculty members to integrate the moral foundation of practice into learning throughout the curriculum Recommendation 19: Prepare learners who act as moral agents and exhibit moral courage in addressing substandard practices Recommendation 20: Act on our individual and collective responsibilities to society, or we jeopardize our status as a profession Recommendation 21: Recognize the unique responsibility as academic programs to partner with the community in developing and implementing programs that place positive health outcomes to the community as their primary focus Achieve control of fiscal resources for physical therapist education Recommendation 3: Develop leaders who can leverage resources to succeed in a rapidly changing health care system Recommendation 4: Develop strong leaders with a compelling vision Recommendation 23: Focus attention across the profession on improved graduates’ outcomes, the need to reduce the cost of education to students and society, and the acquisition of resources to support physical therapist education Recommendation 24: Stop expending resources to identify a narrow set of specific academic organizational structures or curricular models for physical therapist education; expend resources to identify the best way to achieve excellence reflected in the findings of this study and recommendations Recommendation 26: Assure that academic programs have control of their financial resources, and that they develop economic models for revenue generation through multiple means, and move toward larger programs as one means to increase the range and depth of faculty expertise and other necessary resources Recommendation 27: Develop strategies so that academic programs become respected, valued partners within their organizations and have influence over their resources 36 ... _research/ index.aspx For researchers seeking to incorporate mixed methods research into their applications for NIH research, fellowship, career & training grants Includes criteria for evaluating merits... work together virtually Training: Mixed Methods Research Team ■ Engaging in mixed methods research can serve as a learning opportunity for team members:  Collaborative educational opportunities... evaluating merits of mixed methods research applications Designing Mixed Methods Studies  Study aims & research questions     Specify methods Indicate the type of mixed methods design:  