Effects of Payment for Performance on accountability mechanisms Evidence from Pwani, Tanzania Accepted Manuscript Effects of Payment for Performance on accountability mechanisms Evidence from Pwani, T[.]
Accepted Manuscript Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania Iddy Mayumana, Jo Borghi, Laura Anselmi, Masuma Mamdani, Siri Lange PII: S0277-9536(17)30111-9 DOI: 10.1016/j.socscimed.2017.02.022 Reference: SSM 11077 To appear in: Social Science & Medicine Received Date: 25 June 2016 Revised Date: 13 February 2017 Accepted Date: 13 February 2017 Please cite this article as: Mayumana, I., Borghi, J., Anselmi, L., Mamdani, M., Lange, S., Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania, Social Science & Medicine (2017), doi: 10.1016/j.socscimed.2017.02.022 This is a PDF file of an unedited manuscript that has been accepted for publication As a service to our customers we are providing this early version of the manuscript The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain ACCEPTED MANUSCRIPT Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania Iddy Mayumana1, Jo Borghi2, Laura Anselmi3, Masuma Mamdani1, Siri Lange4 Ifakara Health Institute, P.O Box 78 373, Dar es Salaam, Tanzania imayumana@ihi.or.tz RI PT London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK Josephine.Borghi@lshtm.ac.uk Manchester Centre for Health Economics, Institute of Population Health, Faculty of Medical and Human, Sciences, University of Manchester, Room 4.311, Jean McFarlane Building, Oxford Road, Manchester M13 9PL, UK laura.anselmi@manchester.ac.uk Chr Michelsen Institute, Norway, P.O.Box 6033, N-5892 Bergen, Norway siri.lange@cmi.no SC AC C EP TE D M AN U Corresponding author: Siri Lange (PhD), siri.lange@cmi.no ACCEPTED MANUSCRIPT Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania Payment for Performance (P4P) aims to improve provider motivation to perform better, but little is known about the effects of P4P on accountability mechanisms We examined the effect of P4P in Tanzania on internal and external accountability mechanisms We carried out 93 individual in-depth interviews, group interviews and 19 Focus Group Discussions in five intervention districts in three rounds of data collection between 2011 – 2013 We carried out surveys in 150 health facilities across Pwani region and four control districts, and interviewed 200 health workers, before the scheme was introduced and 13 months later We 10 examined the effects of P4P on internal accountability mechanisms including management 11 changes, supervision, and priority setting, and external accountability mechanisms including 12 provider responsiveness to patients, and engagement with Health Facility Governing 13 Committees P4P had some positive effect on internal accountability, with increased 14 timeliness of supervision and the provision of feedback during supervision, but a lack of 15 effect on supervision intensity P4P reduced the interruption of service delivery due to 16 broken equipment as well as drug stock-outs due to increased financial autonomy and 17 responsiveness from managers Management practices became less hierarchical, with less 18 emphasis on bureaucratic procedures Effects on external accountability were mixed, health 19 workers treated pregnant women more kindly, but outreach activities did not increase 20 Facilities were more likely to have committees but their role was largely limited P4P 21 resulted in improvements in internal accountability measures through improved relations AC C EP TE D M AN U SC RI PT 1 ACCEPTED MANUSCRIPT and communication between stakeholders that were incentivised at different levels of the 23 system and enhanced provider autonomy over funds P4P had more limited effect on 24 external accountability, though attitudes towards patients appeared to improve, community 25 engagement through health facility governing committees remained limited Implementers 26 should examine the lines of accountability when setting incentives and deciding who to 27 incentivise in P4P schemes 28 Key words: Pay for performance, P4P, Performance-based financing, PBF, results-based 29 financing, RBF, accountability, Tanzania 30 Introduction 31 Since the 1990s, a variety of accountability mechanisms like user committees, suggestion 32 boxes, performance appraisal of health workers and maternal death audits have been 33 introduced in low income countries to enhance health services, but these initiatives often 34 not function adequately (Fox, 2015; McCoy, Hall, & Ridge, 2012) 35 Payment for Performance (P4P), also called Performance-based financing (PBF), has in 36 recent years been widely promoted in low income countries to improve providers’ 37 motivation and accountability to deliver better care (Meessen, Soucat, & Sekabaraga, 2011; 38 Witter et al., 2013) by paying bonuses based on the achievement of pre-specified 39 performance targets (Ireland, Paulb, & Dujardina, 2011; Meessen, et al., 2011; Njuki et al., 40 2012) AC C EP TE D M AN U SC RI PT 22 ACCEPTED MANUSCRIPT While there is a growing body of evidence evaluating the impact of P4P, the focus has been 42 primarily on service outcomes (Basinga et al., 2011; Bonfrer, Van de Poel, & Van 43 Doorslaer, 2014) Recent studies have paid more attention to context and the processes by 44 which these outcomes are or are not achieved, and the effects of P4P on people within the 45 health system, their relationships and the work environment (Bertone, Lagarde, & Witter, 46 2016; Bertone & Meessen, 2012; Bhatnagar & George, 2016; Huillery & Seban, 2014; 47 Lohmann, Houlfort, & De Allegri, 2016; Paul, Sossouhounto, & Eclou, 2014; Renmans, 48 Holvoet, Orach, & Criel, 2016; Ssengooba, McPake, & Palmer, 2012) However, a review 49 of P4P studies concludes that the findings are often contradictive, that context and design 50 matter, and that the exact mechanisms that P4P trigger remain unknown (Renmans, et al., 51 2016) This paper seeks to contribute to this emerging field by assessing whether and to 52 what degree accountability processes were enhanced by the Tanzanian P4P scheme 53 The theory of change of P4P: accountability measures and assumed 54 pathways of change 55 In this study we differentiate between internal accountability, mechanisms that are aimed at 56 relations within and between different levels of the health system; and external 57 accountability, aimed at relations between health providers and clients (Cleary, Molyneux, 58 & Gilson, 2013) P4P may improve internal accountability through more supportive 59 supervision linked to the verification of performance data, by strengthening relations 60 between managers and providers through joint incentives, and encouraging providers to 61 place demands on higher levels (Meessen, et al., 2011) P4P may affect external AC C EP TE D M AN U SC RI PT 41 ACCEPTED MANUSCRIPT accountability by encouraging provider responsiveness to users (Meessen, et al., 2011), to 63 attract clients to meet targets (Meessen, Kashala, & Musango, 2007), increasing outreach, 64 and the financial autonomy linked to P4P may stimulate health facility governing 65 committees that were otherwise inactive (Falisse, Meessen, Ndayishimiye, & Bossuyt, 66 2012) A complete overview of accountability mechanisms, and specific examples identified 67 within the Tanzanian context, together with the assumed pathway of change is provided in 68 Table Proposed indicators to measure each mechanism are also described here and 69 presented in the Data Collection section 70 Study setting 71 In January 2011, the Government of Tanzania in collaboration with Clinton Health Access 72 Initiative (CHAI) introduced a P4P scheme in Pwani Region, funded by the Government of 73 Norway The scheme provided incentive payments in six monthly payment cycles to all 74 health facilities in the region offering maternal and child health services based on their 75 achievement of pre-defined maternal and child health performance targets (Binyaruka et al., 76 2015; MoHSW, 2012) 70-75% of the bonus payments went to staff, approximately 10 77 percent of their salaries (Binyaruka, et al., 2015:3-4) The rest was earmarked for facility 78 improvement (MoHSW, 2012) The decision for how facility funds were to be spent was to 79 be made by health workers and health facility governing committees (HFGC), comprised of 80 facility in-charge and community members (URT, 2001) though the community members 81 were not eligible for bonus payments AC C EP TE D M AN U SC RI PT 62 ACCEPTED MANUSCRIPT Managers at Council and Regional levels received payments based on the achievement of 83 facilities in their district/region and additional targets linked to drug stock-outs in their 84 district/region 85 To keep track of facility and district/regional performance, the Pilot Management Team 86 (PMT), comprised of MOHSW and CHAI staff, issued score cards indicating their 87 achievement per indicator, bonus earned, bonus distribution between facility and health 88 workers, the number of health workers eligible, and the next targets The implementation of 89 P4P was accompanied by the introduction of an electronic District Health Information 90 System (DHIS) used to track performance indicators In each cycle the PMT and district 91 managers organized two day performance feedback meetings with providers 92 The P4P programme had a positive effects on two of the eight service delivery indicators: an 93 eight percent increase in institutional deliveries and a ten percent increase in the provision of 94 anti-malarials during pregnancy (Binyaruka, et al., 2015) 95 Data Collection 96 This study used a mix methods design Qualitative data was collected in five of the seven 97 intervention districts selected to represent peri-urban (Kibaha town and Bagamoyo), rural 98 (Mkuranga and Kisarawe) and remote settings (Mafia island) Fifteen health facilities were 99 purposively selected to represent variations in level of care and ownership: thirteen were 100 public, one was private, and one faith-based The data collection took place over the period 101 December 2011-March 2013, covering various programme stages (Figure 1) AC C EP TE D M AN U SC RI PT 82 102 ACCEPTED MANUSCRIPT In-depth interviews were carried out with health workers, managers at council level and 104 national level and stakeholders Group interviews were conducted with regional managers 105 and health facility committee members from three government facilities A total of 93 106 individual in-depth interviews (IDIs), group interviews and 18 Focus Group Discussions 107 (FGDs) were conducted by four social scientists working in pairs The interviews were 108 recorded digitally and subsequently transcribed and translated into English Observations of 109 performance feedback meetings and data verification activities were also done 110 Quantitative data collection was done in January 2012 and thirteen months later Health 111 facility and health worker surveys were carried out before and after the implementation of 112 P4P in all seven intervention districts in Pwani region and four comparison districts (Kilwa, 113 Mvomero, Morogoro town and Morogoro rural) A total of 150 facilities, 75 in the 114 intervention and 75 in the comparison group were sampled, representing 46% of all eligible 115 facilities in Pwani and 34% of all facilities in the comparison districts In each facility one 116 or two health workers delivering reproductive and child health services picked at random 117 from those on duty were also interviewed Facilities were randomly sampled amongst those 118 where P4P was implemented and matching comparison facilities were selected based on 119 provider type, ownership, and case load (Borghi et al., 2013) 120 Data Analysis 121 We used the Cleary et al framework to define internal and external accountability We then 122 identified relevant themes within the qualitative data, and indicators within the surveys 123 (Table 1) Verbatim transcriptions of qualitative data were first read to get an overall AC C EP TE D M AN U SC RI PT 103 ACCEPTED MANUSCRIPT impression A coding system was developed To capture internal accountability, we focused 125 on organizational culture, management and supervision practices and relationships, financial 126 autonomy, how funds were used and allocated among providers, how decisions about fund 127 use were taken (and by whom), and evidence of changes in prioritization practices in order 128 to achieve targets In relation to external accountability, we looked for evidence of changing 129 practices towards patients, the role of committees, and measures taken to increase outreach 130 services The data was managed and coded using NVivo 10 software 131 The quantitative indicators measured are summarised in Table We used a difference-in- 132 difference linear regression model to isolate the effect of P4P on the outcomes of interest, as 133 shown in Equation 134 Equation 1: M AN U SC RI PT 124 TE D 135 In all models, we included facility fixed effects ( 137 time invariant characteristics and year fixed effects ( 138 of at baseline and at endline, with health worker outcomes clustered at the facility level 139 The effect of P4P on outcomes is estimated as 140 precision of our results to removing the facility fixed effects from the model, using non- 141 linear (logit) models for binary outcomes, and, clustering standard errors at the district level 142 (Cameron & Miller, 2015) To adjust for multiple outcome testing, we applied a Bonferroni 143 correction which accounts for possible correlation between outcomes (Bonfrer et al., 2014) to control for facility-level unobserved a dummy variable taking the value We confirmed the robustness and AC C EP 136 ACCEPTED MANUSCRIPT The lagged dependent variable approach has been proposed as an alternative approach to 145 difference in differences It maximises statistical power and, when trends are not parallel, 146 produces unbiased results As we were unable to test whether the pre-intervention trends in 147 the specific outcomes considered in this paper were parallel, we applied a lagged dependent 148 variable approach as a further robustness check (D McKenzie, 2012; Ozler, 2015) 149 All analyses were carried out at the health facility level To generate health facility values 150 for indictors collected at the health worker level, the maximum value reported at a given 151 facility was selected for supervision outcomes and the mean scores were estimated 152 averaging indicators of satisfaction with community relations across health workers in the 153 same facility (David McKenzie, 2012) 154 Results 155 We present the P4P scheme’s effects on internal and external accountability mechanisms 156 The findings are summarised in table 157 Internal accountability 158 Content and Frequency of Supervision 159 There was no effect of P4P on the number of supervision visits by managers However, 160 there was a reduction of 17% (SE: 7.1) in the number of facilities reporting that supervision 161 happened less than once per quarter (see Table 1) - the recommended frequency for 162 supervision visits Managers indicated that they could not increase the intensity of visits due 163 to a lack of funds for fuel and allowances, and the council cars were often not available: AC C EP TE D M AN U SC RI PT 144 ... Corresponding author: Siri Lange (PhD), siri.lange@cmi.no ACCEPTED MANUSCRIPT Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania Payment for Performance. ..ACCEPTED MANUSCRIPT Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania Iddy Mayumana1, Jo Borghi2, Laura Anselmi3,... provider motivation to perform better, but little is known about the effects of P4P on accountability mechanisms We examined the effect of P4P in Tanzania on internal and external accountability