R Prepared for the United States Army Approved for public release; distribution unlimited Donna O. Farley Georges Vernez Will Nicholas Elaine S. Quiter George J. Dydek Suzanne Pieklik Shan Cretin A RROYO C ENTER C ENTER FOR M ILITARY HEALTH P OLICY R ESEARCH Evaluation of the Low Back Pain Practice Guideline Implementation in the Army Medical Department The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. R ® is a registered trademark. © Copyright 2004 RAND Corporation All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from RAND. Published 2004 by the RAND Corporation 1700 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138 1200 South Hayes Street, Arlington, VA 22202-5050 201 North Craig Street, Suite 202, Pittsburgh, PA 15213-1516 RAND URL: http://www.rand.org/ To order RAND documents or to obtain additional information, contact Distribution Services: Telephone: (310) 451-7002; Fax: (310) 451-6915; Email: order@rand.org Library of Congress Cataloging-in-Publication Data Evaluation of the Low Back Pain Practice Guideline Implementation in the Army Medical Department / Donna Farley [et al.]. p. cm. “MR-1758.” Includes bibliographical references. ISBN 0-8330-3474-X (Paperback) 1. Backache—Treatment—Evaluation. I. Farley, Donna. RD771.B217E94 2003 355.3'45'0973—dc22 2003020092 The research described in this report was sponsored by the United States Army under Contract No. DASW01-01-C-0003. iii PREFACE The RAND Corporation has been working with the Army Medical Department on a project entitled “Implementing Clinical Practice Guidelines in the Army Medical System.” This project assisted the Army Medical Department in developing and testing methods to effectively implement clinical practice guidelines in the Army treat- ment facilities to achieve consistent and quality clinical care prac- tices across the Army health system. Three sequential demonstra- tions were conducted to test and refine implementation methods before embarking on full implementation of practice guidelines across the Army health system. The three guidelines were those for primary care management of low back pain, asthma, and diabetes. This report presents the final results of the evaluation that RAND conducted as part of the demonstration for the practice guideline for low back pain, which was conducted in 1999 and 2000. The evalua- tion included both (1) a process evaluation of the experiences of the participating military treatment facilities and (2) a quantitative eval- uation to assess effects on processes of care associated with the in- troduction of best practices recommended by the practice guideline. In this report, we present and synthesize the findings from these two evaluation components with the goal of providing as complete a picture as possible of variations across facilities in relevant practices, the extent to which the demonstration sites changed their practices, and measurable effects these actions had on utilization of services and medications. This report is the first of three final reports being generated in this project. It will be followed by similar reports from the demonstrations for the asthma and diabetes practice guidelines. This report will be of interest to personnel in the military health ser- iv Evaluation of the Low Back Pain Practice Guideline Implementation vices as well as to other organizations pursuing strategies for imple- menting best practices. This research was sponsored by the U.S. Army Surgeon General. It was conducted jointly in the Manpower and Training Program of the RAND Arroyo Center, a federally funded research and development center sponsored by the United States Army, and in RAND Health’s Center for Military Health Policy Research. RAND Arroyo Center and RAND Health’s Center for Military Health Policy Research are part of the RAND Corporation. For more information on RAND Arroyo Center, contact the Director of Operations (telephone 310-393-0411, extension 6419; FAX 310- 451-6952; e-mail Marcy_Agmon@rand.org), or visit the Arroyo Center's Web site at http://www.rand.org/ard/. v CONTENTS Preface iii Figures ix Tables xi Summary xiii Acknowledgments xxix Acronyms and Abbreviations xxxi Chapter One INTRODUCTION 1 The DoD/VA Guideline Adaptation Process 3 Overview of the Practice Guideline for Low Back Pain 4 Expected Effects on Health Care Practices 4 A Systems Approach to Implementation 8 Basic Implementation Strategy 9 Six Critical Success Factors 10 The AMEDD/RAND Guideline Implementation Project 11 The Demonstration Sites 13 The RAND Evaluation 15 The Process Evaluation 15 Analysis of Guideline Effects 16 Chapter Two METHODS AND DATA 17 Process Evaluation Methods 18 Outcome Evaluation Methods 19 Choice of Demonstration and Control Groups 20 vi Evaluation of the Low Back Pain Practice Guideline Implementation Data Sources 21 The Low Back Pain Population 21 Indicators for Demonstration Effects 22 Definition of Key Variables 24 Analysis Methods 26 Chapter Three BASELINE PERFORMANCE OF THE STUDY SITES 29 Distributions of MTFs on Low Back Pain Measures 30 Discussion 34 Chapter Four INFRASTRUCTURE FOR GUIDELINE IMPLEMENTATION 37 MEDCOM Support 37 The Kickoff Conference 38 The Low Back Pain Toolkit 39 Information Exchange 43 Structure and Support at the MTFs 45 Command Support and Accountability 45 The Champions 46 The Facilitators 47 The Implementation Teams 47 Lessons Learned 48 MEDCOM Support 48 Support at the MTF 50 Chapter Five IMPLEMENTATION ACTIONS BY THE DEMONSTRATION SITES 53 The MTF Environment 54 MTF Service Capabilities 54 Climate for Guideline Implementation 55 Implementation Activities and Progress 57 Implementation Strategies 58 The Implementation Process and Activities 61 Lessons Learned 66 Flexibility Versus Consistency 67 Monitoring and Accountability 67 Coding and Data Retrieval 68 Ongoing Provider/Staff Education 68 Patient Education 68 Contents vii Defining New Procedures and Responsibility for Them 69 Integrating New Practices 69 Chapter Six EFFECTS OF GUIDELINE IMPLEMENTATION 71 Provider Knowledge and Acceptance of the Guideline 71 Provider Knowledge and Views of the Low Back Pain Guideline 71 Effects of the Guideline on Providers’ Behavior 73 Reported Changes in Clinical Practices 73 Primary Care Services 74 Change in Patterns of Referrals 75 Change in Prescription of Pharmaceuticals 75 Staff Perceptions of Patient Satisfaction 76 Analysis of Effects on Clinical Practices 76 The Study Population 77 Measures and Methods 78 Referrals to Physical Therapy or Manipulation 79 Follow-Up Primary Care Visits 82 Referrals to Specialty Care 84 Prescription of Muscle Relaxants 87 Prescription of Narcotics 90 Prescription of High-Cost NSAIDs 92 Discussion 95 Chapter Seven LESSONS FROM THE LOW BACK PAIN DEMONSTRATION 97 Performance on Six Critical Success Factors 97 Some Perspectives for the Treatment Facilities 101 The Corporate Perspective 102 Appendix A. EVALUATION METHODOLOGY 107 B. REPORTS FROM THE FINAL ROUND OF SITE VISITS 117 C. MULTIVARIATE ANALYSES OF LOW BACK PAIN METRICS 153 References 163 ix FIGURES 1.1. Matrix of Implementation Outcomes 10 1.2. Diagram of the Demonstration Project 12 1.3. Guideline Implementation Process 12 3.1. Baseline Percentages of Acute Low Back Pain Patients Referred for Physical Therapy or Manipulation Services Within Six Weeks of Initial Low Back Pain Encounter 31 3.2. Baseline Average Number of Primary Care Visits for Acute Low Back Pain Patients Within Six Weeks of Initial Low Back Pain Encounter 32 3.3. Baseline Percentages of Acute Low Back Pain Patients Referred for Specialty Care Services Within Six Weeks of Initial Low Back Pain Encounter 33 3.4. Baseline Percentages of Acute Low Back Pain Patients Prescribed Muscle Relaxant Medications Within Six Weeks of Initial Low Back Pain Encounter 34 3.5. Baseline Percentages of Acute Low Back Pain Patients Prescribed Narcotic Medications Within Six Weeks of Initial Low Back Pain Encounter 35 3.6. Baseline Use of High-Cost NSAIDs by Acute Low Back Pain Patients as a Percentage of All NSAIDs Used 36 6.1. Trends in Percentage of Acute Low Back Pain Patients Referred for Physical Therapy or Manipulation Care, Demonstration and Control Sites 81 6.2. Trends in Percentage of Acute Low Back Pain Patients Referred for Physical Therapy or Manipulation Care, Individual Demonstration Sites 81 x Evaluation of the Low Back Pain Practice Guideline Implementation 6.3. Trends in the Number of Follow-Up Primary Care Visits Per Patient for Acute Low Back Pain Patients, Demonstration and Control Sites 83 6.4. Trends in the Number of Follow-Up Primary Care Visits Per Patient for Acute Low Back Pain Patients, by Demonstration MTF 83 6.5. Distribution of Specialty Referrals for Acute Low Back Pain Patients by Type of Specialty, Demonstration MTFs 85 6.6. Distribution of Specialty Referrals for Acute Low Back Pain Patients by Type of Specialty, Control MTFs 86 6.7. Trends in the Percentage of Acute Low Back Pain Patients Referred for Specialty Care, Demonstration and Control Sites 86 6.8. Trends in the Percentage of Acute Low Back Pain Patients Referred for Specialty Care, by Demonstration Site 87 6.9. Trends in Distributions of Specialty Referrals for Acute Low Back Pain Patients by Type of Specialty, Demonstration Site D 88 6.10. Percentage of Acute Low Back Pain Patients Prescribed Muscle Relaxants, at Demonstration and Control MTFs 89 6.11. Percentage of Acute Low Back Pain Patients Prescribed Muscle Relaxants, by Demonstration MTF 90 6.12. Percentage of Acute Low Back Pain Patients Prescribed Narcotics, for Demonstration and Control MTFs 91 6.13. Percentage of Acute Low Back Pain Patients Prescribed Narcotics, by Demonstration MTF 92 6.14. High-Cost NSAIDs Prescribed for Acute Low Back Pain Patients as a Percentage of All NSAIDs Prescribed, Demonstration and Control MTFs 94 6.15. High-Cost NSAIDs Prescribed for Acute Low Back Pain Patients as a Percentage of All NSAIDs Prescribed, by Demonstration MTF 94 A.1. A System View of Guideline Implementation 108 [...]... judgment of the expert panel 4 Evaluation of the Low Back Pain Practice Guideline Implementation OVERVIEW OF THE PRACTICE GUIDELINE FOR LOW BACK PAIN The principal emphasis of the DoD/VA practice guideline for primary care management of low back pain is on acute low back pain, which is defined as low back pain occurring during the first six weeks after the initial onset of pain (VHA/DoD, 1999) Five key guideline. .. documented in the written practice guideline (VHA/DoD, 1999) The Practice Guideline for Low Back Pain The principal emphasis of the DoD/VA low back pain practice guideline is on acute low back pain, which is defined as low back pain occurring during the first six weeks after the initial onset of pain Five key guideline elements were identified by the expert panel responsible for the low back pain guideline. .. monitoring of progress in achieving best practices for each condition addressed by a guideline This report presents the results from our evaluation of the implementation of the low back pain guideline in the Great Plains Region demonstration These findings build on and extend the results of our process evaluation of the first three months of activity for the low back pain demonstration.1 The remainder of this... EFFECTS ON HEALTH CARE PRACTICES The emphasis of the low back pain guideline on conservative treatment for patients with acute low back pain (the first six weeks following the initial low back pain visit) should be the primary driver of any changes in clinical practices that might be observed as the MTFs implemented the guideline For chronic low back pain patients (those who still have pain after six weeks),... Elements of the DoD/VA Practice Guideline for Low Back Pain 1.2 Profiles of the Military Treatment Facilities Participating in the Low Back Pain Guideline Demonstration 2.1 Guideline Introduced (April 1999) 2.2 Indicators Used to Measure Effects on Service Utilization Related to Implementation of the DoD/VA Low Back Pain Practice Guideline 3.1 Interpretation of MTF Baseline Performance on the Low. .. during the early weeks of low back pain Therefore, we expect to observe the hypothesized changes in clinical practices only in those MTFs that proactively implemented 2 The guideline leaves the actual timing of specialty referrals to the judgment of the clinician, depending on the severity of pain and presence of other symptoms during the conservative treatment period Summary xvii the new practices,... earlier process evaluation findings for activities and progress xiii xiv Evaluation of the Low Back Pain Practice Guideline Implementation during the first three months the demonstration MTFs worked with the low back pain demonstration.1 Specific components of RAND’s evaluation for each demonstration included the following: • Process evaluation documented the implementation activities of participating... summarizes the process DoD and the VA used to establish practice guidelines and MEDCOM’s approach to implementing the guidelines in the Army environment Chapter Two describes the methods and data used for the evaluation Chapter Three reports the benchmarking of baseline performance of the nine MTFs in the study on each of the six measures (see Table 3.1) of low back pain services used to assess the effects of. .. clinical practice guideline All the demonstrations worked with practice guidelines that were established collaboratively by the Departments of Veterans Affairs (VA) and Defense (DoD) In the first demonstration, four MTFs in the Great Plains Region implemented the practice guideline for low back pain Next, the practice guideline for asthma was implemented by four MTFs in the Southeast Region Last, the practice. .. guideline, but few of the champions received tangible support for their activi- xxiv Evaluation of the Low Back Pain Practice Guideline Implementation ties (other than attendance at the kickoff conference) Most of them had to perform the implementation work in addition to their regular workload In most of the MTFs, a facilitator designated by the MTF commander provided staff support to the champion, and . have low back pain (defined by the guideline as xvi Evaluation of the Low Back Pain Practice Guideline Implementation chronic low back pain) . 2 To the. the written practice guideline (VHA/DoD, 1999). The Practice Guideline for Low Back Pain The principal emphasis of the DoD/VA low back pain practice guide- line