ALQualityAssuranceGuideDecember2011

ALABAMA DEPARTMENT OF HUMAN RESOURCES FAMILY SERVICES DIVISION QUALITY ASSURANCE GUIDE OFFICE OF QUALITY ASSURANCE REVISED DECEMBER 2011 QUALITY ASSURANCE GUIDE Family Services Division Office of Quality Assurance Alabama Department of Human Resources TABLE OF CONTENTS Overview of Quality Assurance Introduction Background Purpose of the QA Guide Purpose of Quality Assurance Mission of Quality Assurance Goals and Approach Overview of Quality Assurance Functions Long-term trends Quality of services and outcomes Systemic issues County Improvement Plans County Feedback Loop Organization and Structure of QA State Quality Assurance Office County Quality Assurance Staff State Quality Assurance Committee County Quality Assurance Committees County QA Operations 16 Necessary Components of Successful County QA Operations Develop a Plan for Reorganizing the County QA System (if needed) Designate a County QA Coordinator (if needed) Recruiting QA Committee Members Train and Support the QA Committee in its Work Schedule and Maintain Regular QA Committee Meetings Maintain Goals and Time Frames to Complete the Required Number of QSR’s Regular Review of Data Related to Selected Indicators Regular Review of Information from Satisfaction Surveys Conduct Ongoing Periodic Stakeholder Interviews Submission of QA Reports Promoting Continuous QA Committee Functions and Activities Components of Quality Assurance Monitoring Outcome Domains 22 Quality Service Reviews (QSRs) Case Review Instruments Case Reviewer Certification Case Reviewers Case Selection Process Number of Cases to be Reviewed Inclusion of Case Reviews Conducted By the Office of Quality Assurance Types of Cases to be Reviewed Frequency of Case Reviews Recommendations Feedback Loop Monitoring Systemic Issues Core Systemic Issues County-Specific Systemic Issues 28 Information Used to Evaluate Systemic Issues Satisfaction Surveys Stakeholder Interviews Special Studies When to Conduct Special Studies Topics of Special Studies Design of Special Studies Interfacing Special Studies with Required Case Reviews Reporting the Findings of Special Studies 28 Onsite Reviews 32 County Improvement Plan 33 Reports Preparing and Distributing Reports Content of Reports 34 Maintaining QA Information 35 Technical Assistance Assistance in Maintaining QA Activities Assistance in Improving Practice 36 Appendices 38 Overview of Quality Assurance Introduction Background Quality assurance efforts in child and family services have traditionally focused on determining state compliance with requirements contained in federal statute and regulations which, if not met, resulted in the loss of federal funds to the state Quality assurance reviews have also most often been directed primarily toward foster care services, rather than examining the full continuum of services needed and used by children and families Two primary factors influenced the development of a comprehensive quality assurance system for child and family services by the Alabama Department of Human Resources First, provisions in the R.C vs Walley Consent Decree and related court-approved documents required the Department to implement a quality assurance system that included several specific components and functions of the system Second, the federal government’s process for reviewing state child and family services has undergone significant revision in response to legislation passed by Congress in 1993 (Public Law 103-432) The federal child and family service reviews examine outcomes of services for children and families, rather than focusing entirely on procedural requirements The Department has developed an approach to quality assurance that meets the requirements of the Consent Decree, is consistent with the federal review process, and is flexible enough to meet the review needs the Department has for internal information Purpose of the Quality Assurance Guide The Quality Assurance Guide serves as the Department’s policy on quality assurance functions It is intended to provide a comprehensive description of the mission, structure and functions of the quality assurance (QA) system that reviews child and family services administered by the Alabama Department of Human Resources The Guide is a primary source of information to County Departments of Human Resources and state and county quality assurance committees on the operation of quality assurance functions Information in the Guide will assist counties in sustaining a best practice orientation to management and practice Purpose of Quality Assurance Mission of Quality Assurance The QA system monitors, evaluates and provides feedback to the Department on the performance of the best practices in the service delivery system, and whether services provided are of sufficient intensity, scope and quality to meet the individual needs of children and their families The QA system is intended to support social workers, supervisors and management at every level within the Department, as well as to support the development, implementation and refinement of the service delivery system Quality assurance provides more than an audit function In addition to examining and assessing the components of best case practice, QA identifies needs and recommends corrective actions necessary to improve services, capacity, outcomes and conformity with federal, state and departmental program requirements It also confirms strengths, identifies successful strategies, and recommends ways in which effective practice and/or system performance can be replicated and/or improved It helps identify and provide necessary training, consultation, and technical assistance to DHR staff and technical providers, as well as reviewing for the implementation and effects of corrective actions where needed Goals and Approach The goals of Quality Assurance include the following: • To provide a permanent structure for independent, objective evaluations of the quality of services and outcomes for children and families; • To increase the capacity of the Department to deliver improved services through the use of QA evaluations; • To improve the outcomes for children and families served by the Department through the use of QA evaluations; • To provide a point in time assessment of each county system on a rotating schedule The approach the QA system uses in working toward these goals includes the following features: • Review for outcomes experienced by children and families who receive the Department’s services, particularly the outcome areas of safety, permanency and child well being; • Review for the adequacy of major systemic factors that affect the Department’s capacity to deliver services that will lead to improved outcomes for children and families; • Review programs for consistency with applicable federal, state, and departmental policies; • Review for the strengths of the service delivery system and the barriers to more effective performance; • Provide information on system and outcome areas needing improvement and identify barriers to improved service delivery; • Involve communities as partners with the Department in evaluating best case practice In striving for the goals noted above, the QA system gathers and uses several types of information: • Quantitative and factual data are used to describe activities, service capacity, and other relevant measurable factors These data enable QA to address questions such as, “how many?” “how often?” and “at what level?” The answers to these questions enable the QA system to establish baselines, track progress over time and monitor trends • Qualitative and outcome information is used to evaluate the functioning of children and families in light of services delivered This information enables QA to address questions such as, “how well?”, “how comprehensive?” and “what are the needs?” • Information obtained from community stakeholder interviews is used primarily to evaluate the systemic issues that affect the Department’s capacity to provide services that will lead to desirable outcomes for children and families • Information related to compliance with applicable standards, regulations, policies and laws is used to review Department and service provider functions in order to determine conformity with federal, state, and departmental program requirements Overview of Quality Assurance Functions There are three components of the QA system: (1) county quality assurance systems, including a QA coordinator and county QA committees, (2) the Office of Quality Assurance in the State Department of Human Resources, and (3) the State QA Committee Figure illustrates the interrelationship of the three components (Pg 8) The reporting and review of each county’s service delivery system by QA cover five distinct areas of information: • Long-term trends In reviewing for trends, the primary source of information is quantitative data The Office of Data Analysis provides much of the data needed by county QA committees to examine practice and outcomes, formatted from data already submitted by the county department The data fall into the three domains of safety, permanency and child well being This information is not only used to measure progress against national standards over time, but to target areas where more intense review is needed • Quality of services and outcomes In reviewing this area, the information gathered and evaluated is primarily qualitative, coming from case reviews that include interviews with children and families served by the Department, along with their social workers, service providers and foster parents This area also includes reviewing for child and family satisfaction with the services they have received Counties conduct a specified number of case reviews annually, using a standardized protocol Special studies of targeted populations, issues or procedures may also fall into this category • Systemic issues In addition to reviewing for outcomes, the QA system reviews systemic issues that impact the Department’s capacity to deliver services that promote successful outcomes The primary sources of information on systemic issues are data produced by county departments and interviews with community stakeholders and Department staff that have knowledge of the child and family service delivery system in the county State QA reviews for a set of core systemic issues in all counties, and county QA committees may review for additional countyspecific issues as needed • County improvement plans Following on-site QA reviews, each county develops a county improvement plan based on the results of the review The county facilitates the plan with assistance from the Office of Child Welfare Consultation and the Office of Quality Assurance The Office of Child Welfare Consultation is responsible for supporting the county in development of the plan The Office of Quality Assurance is responsible for supporting assessment and monitoring of the outcomes from the county improvement plan • County feedback loop The county will report its improvement plan and progress or lack of progress in the Biannual Quality Assurance Report Once the plan is achieved the county will submit a report regarding the completed steps of the plan and the outcome improvements to the Office of Quality Assurance The plan will be evaluated and submitted to the Family Services Division and the Office of Field Administration for concurrence that the county has successfully complied with its improvement plan Figure Quality Assurance Functions Office of Quality Assurance • • Provides training and technical assistance Produces and distributes data to County Departments and QA committees County Departments County QA Committees • • • • • • Review data Select case samples for reviews Conduct case reviews and stakeholder interviews Conduct special studies Make recommendations on cases reviewed or systemic issues Approve QA reports Feedback Loop • • • • Receives State and county QA reports Initiates special studies Advises Office of Quality Assurance on QA functioning Makes recommendations Assists county QA committees in carrying out their functions • • State QA Committee • • • • • • • Feedback Loop • Review data Produce additional data Provide information to/support QA committee Review and respond to recommendations from QA committee Issue biannual QA reports Office of Quality Assurance Receives county QA reports Conducts on-site reviews Supports QA functions in counties through consultation and technical support Provides information to State QA committee and supports committee’s functions Organization and Structure of Quality Assurance The Department’s QA system is comprised of the Office of Quality Assurance in the Family Services Division; a state QA Committee that includes representatives of the Department and stakeholders representing other interests and entities in the state; a QA coordinator in each county department; and a local QA committee in each county consisting of representatives of the county department and community stakeholders Statewide QA review functions are performed by the Office of Quality Assurance in the State Department of Human Resources and the State QA Committee These review functions are based on aggregate data and other information that reflect general functioning in key outcome and systemic areas County QA staff and committees perform reviews of county performance and outcomes for the families who receive the Department’s services A more detailed description of the functions of each QA entity is as follows: State Quality Assurance Office The Office of Quality Assurance includes the QA manager and QA staff in the Family Services Division The primary functions of state QA staff are directed toward consultation, training and technical assistance to county departments and conducting on-site reviews of county departments Staff also evaluates data on an on-going basis; engages in special studies and monitors county improvement plans More specific functions of the Office of Quality Assurance include the following: • Develop and maintain the components of the Department’s quality assurance system, including designing and maintaining the procedures, instruments, reporting formats, and materials necessary for the efficient operation of state and county QA functions; • Assist counties in developing and maintaining local quality assurance functions by providing orientation, training, technical assistance and if necessary, mediation for county staff and QA committee members; • Support on-going monitoring of aggregate statewide data on child safety and permanency and provide follow-up as needed with county QA systems; • Analyze data needed to assess the status of child protection and permanency for children in foster care, including assisting county QA staff and committees in interpreting and reporting relevant data; • Serve on child death review committees to examine system performance to strengthen the child welfare system and produce better outcomes; • Issue reports that reflect activity, progress and barriers in key child welfare domains on an annual or as needed basis as well as reports of on-site QA reviews; • Provide information to the Department’s administration and program divisions relevant to outcomes of services, best practices and identified needs for program development or corrective action; • Serve as liaison with the State QA Committee through supporting its functions; • Conduct special studies of various issues related to the services, activities and outcomes of the Department’s child and family services program, including initiating studies and participating in studies initiated by State or county QA committees; • Work in partnership with counties and the Office of Child Welfare Consultation to support development and monitoring of the county improvement plan (CIP) County Quality Assurance Staff Each county department operates its own quality assurance system, which conducts reviews of and assesses county activities County QA systems are comprised of a QA coordinator and a county quality assurance committee Counties with populations larger than 80,000 have a full-time DHR staff person designated as QA coordinator Smaller counties will have a designated QA coordinator who performs this function on at least a half-time basis in the county department Other staff of the county department will assist as needed in QA activities, including attending QA meetings, participating in reviews, tracking information needed for QA purposes, responding to recommendations on cases reviewed by the committee and so forth Among the specific functions of county QA systems are the following: • Direct and monitor the county’s own QA activities; • Provide logistical and staff support for state-directed data collection and analysis, case reviews, special studies, and site visits as needed; • Routinely collect and evaluate information concerning the outcomes for children and families and the county department’s capacity to deliver services consistent with the goals and mission of the Department; 10 While the goal is to achieve a strength in every best practice indicator, the county should set realistic goals that are achievable during the time frames for the CIP The CIP should be designed to lead to measurable changes in the indicators Careful consideration should be given to whether the strategies designed will lead to the desired outcomes The CIP should address the improvements in the day-to-day practice of child welfare Focusing on casework practice rather than policy and procedure allows the county to achieve lasting improvements in child welfare C MEASUREMENT OF THE COUNTY IMPROVEMENT PLAN PROCESS The approach that counties take in measuring their progress in implementing their CIP is very important to their success in achieving the goals of the CIP and to improving performance on subsequent on-site reviews The CIP should include provisions for evaluating progress toward overall goal achievement and implementation For each best practice indicator the CIP must include specific information for determining whether the county has reached the goals of the CIP For example, if data are to be used to evaluate the county’s success in reaching agreed upon levels of improvement, the CIP should specify the data to be used, and the specific percentages of improvement to be achieved If qualitative case reviews are used to evaluate progress, the CIP should include specific information about the review process used and the level of achievement that will be considered as meeting the goals in the CIP Some outcomes and benchmarks are better captured through quantitative measures, such as data elements, while others may be better measured by looking at the accomplishment of key steps In most situations, a mix of measures that inform will best serve in documenting desired improvements A county should make use of the data available on BOE, any local data collected, their local QA Committee, as well as Family Services consultants assigned to the County D STRATEGIES FOR DEVELOPING THE COUNTY IMPROVEMENT PLAN Counties should use the expertise they have gained in preparing for on-site reviews to develop their CIP by focusing on their self-assessment, the areas identified in the review that need strengthening, as well as data and reviews completed by the local QA Committee Counties should strive to engage as many partners as possible in the process of completing and monitoring the CIP Counties should also work in collaboration with Family Services Division to support, design and monitor the CIP Some ways to ensure a successful CIP: • Ensure that county leadership is involved in the CIP process and understands the importance of establishing a vision and providing direction to staff assigned to CIP development • Help all involved parties to view the CIP as a process designed to create lasting systemic change while addressing the immediate needs of children and families • Although there is no time frame for completing CIP work, each county should strive to complete work as soon as possible The QA review cycle has each county receiving a 99 review every three years and the goal of each CIP is to have the county reflecting a strength on every best practice indicator E COUNTY IMPROVEMENT PLAN FORMAT The CIP should be based on either the county self assessment or the most recent on-site review, whichever is more current If a self-assessment is the basis for the CIP, the best practice indicators should be rated as “Achieved” or “Partially Achieved” and supporting information should be identified under each indicator to justify the rating The areas to be addressed should then be identified and steps, person’s responsible, and time frames should be developed which will enable the county to reach or maintain “Achieved” on all best practice indicators F COUNTY IMPROVEMENT PLAN REPORTS The report of progress on the CIP will be reported biannually in the Quality Assurance Report The report is described in this guide on page 83, and any questions should be directed to the county’s QA Consultant The report will contain discussion and analysis of the CIP Was it developed, reviewed, or updated during the reporting period? Summarize the goals of the plan and what was done to address those goals during the reporting period 100 COUNTY SELF ASSESSMENT SAFETY Evidence information gathered at intake is sufficient to make required decisions and to assign appropriate response times _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: Face-to-face contacts with children in CAN and Prevention assessments are initiated promptly within time frames appropriate to the priority level of the report _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: CAN and Prevention assessments are thoroughly assessed, completed within designated time frames with appropriate dispositions and required notifications, and any pending assessments are in appropriate pending status _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: Evidence children are determined to be safe, safety is continuously assessed, appropriate intervention decisions are being made, and child safety is generally perceived in a positive manner _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: Evidence effective safety plans are in place for children when safety threats are present and current policy is followed in the development, implementations and documentation of safety plans _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 101 Evidence children are safely maintained in their own homes whenever possible and appropriate through the provision of supports and services _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: There is no evidence of children being subject to repeat maltreatment _ACHIEVED _PARTIALLY ACHIEVED PERMANENCY Evidence children in foster care are in placements that are in the best interest of the child and consistent with achieving the child’s permanency goal(s) _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: Evidence foster care placements are supported with services to minimize the likelihood of placement disruptions _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 10 Evidence children in foster care are in stable placements and any planned changes in the placement are in the best interest of the child _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 11 There is no notable pattern of children re-entering care within twelve months of a prior discharge from care _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 102 12 There is no notable pattern of children remaining in foster care beyond the length of time needed to achieve permanency _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 13 Permanency goals are established timely, are the most appropriate permanency goals for the child, and there is consistency between the permanency goal identified in the ISP and case activity _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 14 Evidence visits between children in foster care and their families are occurring in accordance with Visiting Policy and connections are being appropriately maintained _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 15 Sibling group placements conform to Sibling Placement Policy _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 16 Locations of children’s placements conform to Close Proximity Policy _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 17 Evidence the department maintained the child’s connections to his or her neighborhood, community, faith, extended family, tribe, school, and friends _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 103 18 Evidence foster and adoptive placements are being made in compliance with the requirements of the federal Multi-Ethnic Placement Act, including stipulations concerning placements of children and the recruitment, training and approval of foster and adoptive parents _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 19 There is a functioning case review system in the county that meets the requirements of P.L 96-272 and ASFA for periodic case reviews and permanency hearings _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: CHILD WELL-BEING EDUCATIONAL SUCCESS 20 Evidence the educational needs of children are being met appropriately _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 21 Evidence school personnel are routinely involved in ISP meetings/planning _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 22 Evidence department personnel are routinely involved in IEP meetings/planning _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 23 Evidence educational advocacy is implemented when needed on behalf of children in the Department’s care or responsibility _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 104 EMOTIONAL WELL-BEING 24 Evidence the emotional/behavioral needs of children are being met appropriately _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 25 Evidence the frequency and quality of visits between caseworkers and the child/family are sufficient to ensure safety, permanency, well-being and promote achievement of case goals _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: PHYSICAL WELL-BEING 26 Evidence the physical needs of children are being met appropriately _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: COMMUNITY COLLABORATION 27 The Department is actively involved in the mandatory interagency forums (e.g., multi-disciplinary teams, county multi-needs facilitation teams, Children’s Policy Council) _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 28 Evidence the Department collaborates with community stakeholders and services are coordinated appropriately to meet the needs of the children and families it serves _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 29 Evidence the department’s staff are responsive and available when needed or called upon _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 105 SERVICE ARRAY AND RESOURCE AVAILABILITY 30 The Department demonstrates the capacity to individualize services to children and families _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 31 There is a sufficient service array available to meet the needs of children and families _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 32 Evidence all youth ages fourteen and older are receiving services in accordance with Independent Living Services Policy _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 33 There is an adequate number of approved in-county foster family homes which meet policy requirements in order to place children according to their needs and that serves all major population groups in the county’s foster care population (e.g., racial/ethnic, age, gender, emotional/behavioral/physical issues) _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 34 Workers have access to flex funds which are used appropriately to meet the needs of children and families _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 106 ASSESSMENT/ISP 35 Evidence of thorough initial and ongoing assessments of strengths, needs, current status, protective capacity, safety threats, preferences and progress of families (children, parents, foster parents, absent parents, etc.) _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 36 Evidence that the ISP process is serving as a functional tool to guide practice and service delivery (e.g., engagement of families, involvement of all relevant stakeholders, major decisions made in context of ISP, document reflects practice, correct identification of needs, appropriate goals, steps/services match needs, lead to goal attainment, and are monitored regularly) _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: QUALITY ASSURANCE 37 There is evidence of a functioning county QA committee that includes members who routinely review the required number of cases, conduct yearly stakeholder interviews, complete special studies, and make recommendations to the county department _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 38 There is evidence the recommendations of the county QA committee are considered by the County Department and that there is a feedback loop back to the committee to indicate how the recommendations were addressed _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 39 There is a designated QA coordinator who performs the functions of a coordinator as described in the QA Guide and whose duties and responsibilities not exceed caseload standards _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 107 40 The QA committee has established leadership from within the committee (not department staff) as described in the QA Guide _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 41 The county QA system participates fully in the QA reporting process _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 42 There is a process in place to evaluate child, family and stakeholder satisfaction and to address concerns identified _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: SUPERVISION 43 There is evidence supervisors are managing the work in their units satisfactorily, by routinely reviewing all cases carried by staff members, monitoring for quality of work, and by providing appropriate feedback, modeling and coaching to staff _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 44 Supervisors conduct regular, individual supervisory conferences with staff members in their units _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: STAFFING & CASELOADS 45 There is no evidence of a pattern of caseloads falling outside established standards for a period longer than six months _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 108 46 The supervisor-to-worker ratio is within the Department’s established standards _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: STAFF & PROVIDER TRAINING 47 There is evidence that staff have completed the department’s required child welfare training curricula and are receiving on-going training opportunities to enhance and build their abilities to work with children and families _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: 48 The Department has the capacity to provide on-going foster and adoptive parent training with an adequate number of trained co-leaders and evidence of on-going training _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: INFORMATION SYSTEM CAPACITY 49 There is evidence that tracking systems are initiated, updated, and maintained and management utilizes the information to assess, plan and monitor _ ACHIEVED _ PARTIALLY ACHIEVED BASIS: TOTALS: ACHIEVED: PARTIALLY ACHIEVED: 109 State Onsite Reviews Processes and Procedures Periodic onsite reviews will be conducted in each county on a rotating basis State QA staff will lead and participate in onsite reviews with the remainder of the review team being comprised of trained adjunct reviewers The current length of onsite reviews varies from three days in small counties to five days in the largest counties of the state Those selected to participate in onsite reviews as either adjunct reviewers or shadows will be informed prior to the review the length of that particular review and when each review team member will be expected to report to the county Each onsite review will consist of stakeholder interviews conducted by lead and co-facilitators, case reviews or QSRs in which participants are interviewed, and safety, permanency and resource assessments Generally each case reviewer will be assigned two cases for review with interviews for each respective review scheduled on consecutive days There will be a minimum of two days onsite for conducting review activities according to the respective roles assigned for that review Review activities may also take place prior arriving onsite through the review of case material and other information available through FACTS and BOE Individual counties will determine if an introductory meeting with their staff and the review team will occur prior to the beginning of review activities Review team members will be informed of the expectation of attending any introductory meeting Review activities will be conducted daily according to the role of the team member Nightly debriefing sessions will be scheduled dependent upon the number of QSRs conducted for the size county being reviewed These debriefing sessions are designed to promote consistency in reviewer ratings A final debriefing session will be held to gather information from all review team members pertinent to the indicators of best practice The final debriefing session is designed for all team members to contribute to and participate in the determination of the status of each best practice indicator at the time of the onsite review Onsite review team members will be informed of the scheduled time to begin each debriefing session A brief exit conference will be held with the county staff to provide an overview of the results of the onsite review Review team members are excused at the conclusion of the exit conference Time frames to be considered in the review will be reported to team members while on site For data purposes, the reporting period is generally the four quarters prior to the review For QSR purposes, team members will be informed when the county received notification of the cases selected for review Generally ratings for those cases should take into account the previous 30 days for child and family status and the previous 90 days for system performance from the date the county became aware of the case selection Any marked changes taking place during the time period from case selection to the onsite review may be noted in the case write up Adjunct reviewers will be informed onsite of the due date for their completed work products from their respective roles Contact persons for review team members will be identified on site These persons will generally be the lead facilitator and the county QA coordinator, but other contact persons may be identified as well It is generally more appropriate to contact the identified review team member with concerns or questions about ratings, safety concerns identified while on site, debriefing issues, etc while it is generally more appropriate to contact the identified county contact person with concerns or questions about directions, interview schedules, missed appointments, etc Any state QA person who is on sight may be contacted for guidance if needed Safety concerns should be promptly reported to a state QA team member if the identified review contact member is not available Each review team member is expected to provide the information pertinent to their role concisely as possible during debriefings Debriefings are designed to focus on practice and system performance as it relates to the information gained in all roles on the review team Important details about cases should be reflected in the completed work products rather than in the debriefing process It is as important to identify strengths as it is to identify needs in both the debriefing process as well as in the individual work products Feedback and recommendations from team members to county staff in their respective roles will take place through the completed work products Feedback may be given to any team member while on site or at the receipt of work product in order to clarify ratings, supports for recommendations, or best practices, issues or 110 concerns It is essential that the information provided in work products be consistent with the information provided at the time of the debriefing Shadows may be invited to participate in onsite reviews to either complete their training as an adjunct reviewer or to experience the process prior to an onsite review being conducted in their county Shadows are generally paired with persons completing QSRs Additional persons will be invited to observe debriefing sessions These will generally include the county director, the county QA coordinator, the county QA committee chair, the county OCWC consultant, and the county DAS, although others may be present as well All persons present for debriefings will be required to sign a confidentiality agreement After the onsite review is conducted, each review team member is responsible for the completion of their respective work products The final report should be completed within a 30 day time frame and provided to the county shortly after completion The final report is the basis for the county to develop and/or revise their county improvement plan (CIP) The CIP and resulting progress will be reported on in the Biannual Report 111 State Onsite Reviews Adjunct Reviewer Responsibilities/Expectations Participation in state QA reviews requires a significant commitment of time and energy Adjunct reviewers can expect to work 10 to 12 hour days and typically will not have time during the onsite review to conduct other work, or consult frequently with their county office At the same time, the work can be quite rewarding in terms of contacts that occur with children, families and other relevant persons It is also rewarding in the sense of being part of a team that is able to provide to county staff affirmation of strengths, along with recommendations on how areas that need strengthening can be addressed Finally, serving as a member of a review team can be a productive learning experience in terms of gathering ideas from county staff and other members of the review team The general expectations and responsibilities of adjunct reviewers include the following: • Remain present throughout the entire review and participate in all scheduled review activities, including the introductory meeting, the de-briefing sessions, the exit conference and any other scheduled events (unless exceptions have been worked out in advance, or in the case of an emergency); • Conduct all assigned activities associated with your role on the review team, including the completion of all work products within the identified time frame; • Prepare for and participate in the review team de-briefings by presenting information gained from the work done in your review team role and discussing other issues with the review team members; • Present information as concisely as possible, sharing only information relevant for each rating item for QSRs and for the indicators of best practice in the final debriefing; • Review the pre-assessment and other materials provided to the review team prior to the final debriefing and be prepared to address any of the indicators of best practice from the information obtained from their role in the review; • Provide feedback to other review team members in a way that is strengths-based and helpful, and also be open to receiving input/feedback from others; • Abide by the agreement for confidentiality that each review team establishes, in terms of the information obtained and discussed in the de-briefing sessions; • As needed, assist in the review of the draft report of the findings of the onsite review and provide comments/suggestions to the person responsible for finalizing the report; • Remain available for telephone consultation after the review for purposes of giving/receiving feedback regarding the clarification of information provided (during the review, or in the written materials) and/or suggestions regarding the completion of the written documents 112 State Onsite Reviews Consistency Measures for Adjunct Reviewers An important aspect of conducting state QA reviews is that of promoting consistency among reviewers This is a challenge for state QA staff as we go from county to county, and the challenge can intensify for reviewers who have experienced/conducted reviews on a less frequent basis In order to promote consistency among adjunct reviewers several steps will be put in place (some of which also help promote consistency among state QA staff) Any exceptions to the measures identified below would occur by way of consensus among state QA staff These steps include: • • • • • • • • Adjunct reviewers will have participated in protocol training prior to serving as a reviewer; Adjunct reviewers will have participated as a shadow in a state QA review prior to serving as a reviewer; Adjunct reviewers will have an identified state QA staff person to whom they can ask questions or seek guidance while onsite for the state QA review; Adjunct reviewers will participate in the de-briefing sessions where discussions on ratings occur, feedback is given and rating decisions are made; Generally the guiding principle on rating changes is that the reviewer, after hearing discussion and input on an item, makes the call on what rating they will assign Although this will continue to be the general rule, there could be rare occasions where, based on the information provided, a rating is changed in order for consistency to be maintained Should this action be anticipated, the reviewer will be involved in the discussion; Adjunct reviewers will submit their written materials within the specified time frame to the lead facilitator or other identified state QA team member who will in turn contact them if any additional information or clarification is needed; The adjunct reviewer will participate in any ongoing training sessions that are held; The adjunct reviewer will participate in any activities that are designed to support their continued development in the adjunct reviewer role 113