qualityagingmatrixpresentation_srcconference.pptx

FROM OUTPUTS TO OUTCOMES: THE QUALITY AGING MATRIX Presentation prepared for the 2017 Annual Conference of the Southeast Michigan Senior Regional Collaborative, Four Points Sheraton Hotel, Novi, Michigan, September 26, 2017 Thomas B Jankowski, PhD, Associate Director for Research, Institute of Gerontology, Wayne State University t.jankowski@wayne.edu COLLABORATIVE APPROACH Our mission is to enhance the quality of life for all older adults residing in Southeast Michigan Our purpose is to develop a regional framework for advocacy, awareness, and action in support of issues affecting the safety and quality of life of older adults in our area Capacity Building (Organizational & Professional) Advocacy & Awareness Resource Development Shared/Stren gthened Systems WHY MEASURE OUTCOMES? The difference between inputs, outputs, and outcomes: Inputs = Resources, money, expertise, staff Outputs = Units of service, contact hours, people served Outcomes = Impact of programs on the lives of those served Inputs Outcomes: • Lend meaning to our work • Affect human lives Outputs Outcomes WHY MEASURE OUTCOMES? • Funders increasingly require it • Complete program evaluation is not possible without it • Traditional measures of input and output not allow us to evaluate program effectiveness in a systematic way • Logic model approach to evaluation requires objective outcome measure, rather than assumption about outcomes Resources Services Impact Do the services we provide have the impact we have always assumed they do? Are we doing what we intend to be doing? Are we really helping the people we are trying to help? SELF SUFFICIENCY MATRIX Origins and Background • ROMA - Results Oriented Management and Accountability • • • Government Performance and Results Act of 1993 Community Services Network’s Monitoring & Assessment Task Force Used by agencies receiving Community Services Block Grant funds • Outgrowth of ROMA efforts was Self Sufficiency Matrix • Used to measure progress of homeless working-age families with children toward self sufficiency • • • Local United Way chapters United Methodist Outreach Ministries Task forces: • Boulder County, Colorado, Snohomish County, Washington, Santa Clara County, California, and Maricopa County, Arizona • Different versions vary, but share similar characteristics SELF SUFFICIENCY MATRIX Measurement Characteristics • Common form of an outcome matrix • Consists of 10-25 related domains • Each domain has its own scale, but with a common metric • Assumes interval-level measurement to enable summation & calculations of central tendency (means, medians, and modes) • All use 5-point scale marked by transitive semantic differentials • UWSEM SSM • 19 domains with five categories or levels: In Crisis – Immediate unmet need Vulnerable – Immediate need met with emergency assistance Sufficiency – Needs met with significant permanent assistance Self-sufficiency – Client resources and limited assistance Sustainable self-sufficiency – Client meets all needs with own resources and plans for future with no assistance SELF SUFFICIENCY MATRIX Why a New Measure was Needed • Self sufficiency is a worthy goal for working-age people without disabilities • • UWSEM measure based on their funding priority Most programs funded serve families with children • Not appropriate for older and disabled population • • Sustainable self sufficiency not a realistic goal for those we serve Most will continue to rely on some kinds of assistance • Metrics based on unrealistic goals will show lack of progress • SSM applied to clients who are in later life or who live with a disability will make services appear ineffective • Our goal: Find an outcome matrix more appropriate to our programs, services, and clients OUTCOME MEASUREMENT TOOLS SEMISRC Data Committee Development Efforts • Extensive research was undertaken to find a better tool • Consultation with researchers, providers, quality improvement organizations • University of Michigan SOSW Curtis Research Center • Wayne State University Institute of Gerontology • Michigan Peer Review Organization • Other tools found • Health status, patient activation, client satisfaction, several different versions of self sufficiency matrix • No global measures of well being for those aging in community • Collaborative outcome tool development was necessary QUALITY AGING MATRIX Collaborative Development • SEMISRC Data Committee began work in 2013 • Stages of development, testing, piloting, user review, revision involved dozens of people over more than years • Recently completed our version revision of the QAM based on feedback from users • Basic issues faced in this process: • What concept to measure? What is our standard? • What domains we measure? • How we measure it in the most valid and reliable way possible? • How we make the measurement and recording of the Quality Aging Matrix data easy for users and standard across agencies? QUALITY AGING MATRIX Collaborative Development • The concept: quality of life, or quality aging • Categories within each domain: • In Crisis – Immediate unmet with no access to resources • Vulnerable – Immediate need partially met with only temporary, unreliable, and/or insufficient resources • Stable – Need is minimally met and minimally sufficient, with resources available for the long term • Safe – Need is fully met and sufficient resources are in place permanently • Thriving – Need is fully met according to the individual’s tastes and preferences and permanent resources are plentiful enough to support choice beyond basic needs QUALITY AGING MATRIX Example Domain: Nutrition QUALITY AGING MATRIX Example Domain: Physical Health QUALITY AGING MATRIX Example Domain: Physical Health QUALITY AGING MATRIX Implementation • Original pilot organizations: • • • • • • • • Jewish Family Service of Metropolitan Detroit Alzheimer’s Association Adult Well-Being Services Oakland Family Services OLHSA, A Community Action Agency Community Living Services Matrix Human Services Neighborhood Service Organization • Looking for more agencies willing to pilot the new version of the QAM • IT Partner: ardentCause QUALITY AGING MATRIX Implementation • Partnership with ardentCause • User friendly, cost effective data management solution • Web based; no need for software purchases or updates • Multiple views of data: Individual, Organizational, programmatic, regional aggregate (Collective impact) • Data can be extracted from existing data systems or imported manually • Data can be used for reporting to funders, program evaluation, advocacy efforts, grant proposals, etc • Switching to new platform: DataSnapz DATASNAPZ EXAMPLE: Data Entry DATASNAPZ EXAMPLE: Data Entry DATASNAPZ EXAMPLE: Data Entry DATASNAPZ EXAMPLE: Data Entry DATASNAPZ EXAMPLE: Data Entry DATASNAPZ EXAMPLE: Dashboard DATASNAPZ EXAMPLE: Dashboard DATASNAPZ EXAMPLE: Reports QUALITY AGING MATRIX Cost vs Benefit Summary • Yes, there are costs associated with using the QAM: • Staff time for training and incorporating QAM record-keeping into case management process • Interface and database hosting (currently free for SRC members) • We believe the benefits outweigh the costs: • • • • • Data to track the global well-being of individual clients Data to assess the needs of your client population Data to evaluate the effectiveness of programs Data to convince funders of program effectiveness User-friendly, efficient, and convenient data collection method that allows for use by individual agencies as well as the demonstration of collective impact FROM OUTPUTS TO OUTCOMES: THE QUALITY AGING MATRIX Thank you! Questions? Thomas B Jankowski, PhD, Associate Director for Research, Institute of Gerontology, Wayne State University t.jankowski@wayne.edu