Exploring the Roles of Faculty Supervision: Improving Qualitative Doctoral Dissertation Methodology

Exploring the Roles of Faculty Supervision: Improving Qualitative Doctoral Dissertation Methodology Dan Kaczynski, PhD dkaczyns@uwf.edu Research Problem More qualitative dissertations Increasing emphasis on quality Shifting supervisory roles Understand assessment practices Strengthen qualitative research skills Develop future researchers Open Discussion  Do you use technology in your research? Open Discussion  How should we explore the tensions within and between:   Assessing Quality Adoption of QDAS Qualitative Software NVivo, MAXqda, Atlas ti, QDA Miner, Qualrus, Transana Kaczynski (2004) http://www.aare.edu.au/04pap/kac041065.pdf What is Good Qualitative Research? What does good work look like? Identify indicators of quality in a thesis: Identify common errors in a thesis:             What is Quality? The researchers logic of justification “Flaws in the logic of justification can potentially occur anywhere in the inquiry process The nature of such flaws and where they occur can jeopardize the soundness of a study in one or more ways.” (Piantanida & Garman 1999, p 147) Types of Quality Criteria Philosophical Criteria (Lincoln, 1995; Lincoln & Guba, 1985) Procedural (Creswell, 1998) Criteria Philosophical Criteria (Lincoln & Guba, 1985)  Credibility  Is the work authentic?  Transferability  Will the work fit outside this situation?  Dependability  Is the researcher consistent?  Confirmability  Are interpretations defensible? Procedural Criteria  Quality of methods (open-ended interviews)  Quality of data (verbatim long transcripts)  Quality of data analysis (comprehensive data treatment) (Silverman, 2004 [Sacks, 1984]) Standardized Procedural Criteria (controversial checklists or guidelines) Does the title reflect the study focus?  Is the problem socially important?  Is the literature review comprehensive?  Has study conformed to ethics standards?  Are issues of sampling discussed?  Did findings answer the questions?  Was study written convincingly?  Are issues of trustworthiness addressed?  Quantitative and Qualitative Criteria for Assessing Research Quality and Rigor Quantitative term Qualitative term Strategy employed Internal validity Credibility Prolonged External validity Transferability Provide Reliability Dependability Create Objectivity Confirmability Triangulation engagement in field Use of peer debriefing Triangulation Member checks Time sampling thick description Purposive sampling an audit trail Code-recode strategy Triangulation Peer examination Practice reflexivity Anfara, V A., Brown, K M., & Mangione, T L (2002) p 30 Transparent Assessment  Explore rich diversity of meanings  Sensitized appreciation of worth  Deeper assessment of analysis  Multiple paths to look inside  Transparency strengthens credibility Data Collection Stage Survey Research Supervisory Ability Quality Factors Technology Resources Professional Development Stage Interviews/ Documents Perceptions Processes Assessment Practices Stage Document Review Findings: Knowledge, Ability, and Confidence Supervising Serving on a committee Judging quality Satisfactory M SD or Higher 73.9% 3.48 1.28 91.3% 4.17 94 91.3% 4.09 95 Findings: Technology Tools Used in Assessment 52% (None) 30% (Not applicable) 17% NVivo 4% InfoRapid Findings: Resources Consulted for Expertise 83% Others 70% Publications 30% Conference workshops 22% Campus workshops 13% Continuing education Findings: Conceptualizations of Quality (cont.) AI: AI Alternative interpretations CE: Consideration of ethical issues AG: Ability to generalize findings HC: Hierarchical code structure MC: Member-checking MS: MS Memos AT: Methodological audit trail PD: Peer debriefing PF: Prolonged field engagement AS: Qualitative data analysis software RO: RO Researcher objectivity SS: SS Sampling strategies SD: SD Self-disclosure SC: Social context TO: TO Theoretical orientation TN: Triangulation VY: VY Validity Stage Findings: Critical Needs  Building – – – Moving beyond superficial assessment Significance of researcher transparency Teaching students to self-assess  Building – – – knowledge and skills a community of practice Strengthening qualitative research skills Sharing assessment strategies Engaging in professional development Stage Findings: Role of Technology Study Findings  Highly favorable attitudes toward qualitative research  Diverse conceptualizations of quality  Need for alternative assessment frameworks  Need and desire to strengthen knowledge and skills  Need for professional development Future Challenges  Progressing research methods Mixed → Blended → Integrated  Emerging research innovations mainstream adoption of QDAS  Positioning standards quality research Future Research Questions  What does it mean to disclose or conceal the role of technology?  Does nondisclosure of analysis software imply the presumption that the use of technology is ubiquitous and commonly accepted?  Does nondisclosure of QDAS suggest a student’s fear of the supervisor’s acceptance or sanctioning?  In what ways and under what conditions does a technological tool become a barrier to the learning process for the teacher and the learner?