cm_011404_item_5_report-regulatory-framework-finalv1-2

Item Paper No: CM/01/13/04 Developing a strategic framework to guide the Care Quality Commission’s programme of evaluation REPORT Kieran Walshe and Denham Phipps January 2013 Item Paper No: CM/01/13/04 Item Paper No: CM/01/13/04 Executive summary 1 Introduction Background and context Analysis of the regulatory model Learning from research Comparisons with other regulators Using existing data Structure of the report 2 The regulatory model: an analysis Introduction Regulatory mission and purpose Registration Standard setting Information gathering and risk assessment Inspection and reporting Enforcement Information provision Reviewing the regulatory model Differentiation in regulatory design Introduction Learning from research Comparisons with other regulators Using existing data Conclusions 23 Regulatory standard setting Introduction Learning from research Comparisons with other regulators Using existing data Conclusions 33 Risk based regulatory approaches Introduction Learning from research Comparisons with other regulators Using existing data Conclusions 43 Item Paper No: CM/01/13/04 The competencies of the regulatory workforce Introduction Learning from research Comparisons with other regulators Using existing data Conclusions 55 Conclusions 66 References 68 Item Paper No: CM/01/13/04 Item Paper No: CM/01/13/04 Executive summary This report sets out to show how evidence and research can be used by CQC to evaluate how well its current regulatory arrangements in health and social care work, and plan future changes in those arrangements to improve their effectiveness and efficiency We are very thankful for the help, advice and support we received from many CQC staff in its preparation Of course, we remain responsible for its content and conclusions, and for any errors or omissions We develop a “logic model” of the current regulatory arrangements which tries to map out how they are intended to work, and what assumptions are made or consequences flow from those arrangements We conclude that as currently configured, CQC is a “safety-net” regulator, focused on dealing with poor performance but with limited capacity or capability to drive or support wider performance improvement We note that even in those terms the current regulatory model has some inherent problems, but that the direction of future strategy signals a shift towards a more ambitious interpretation of CQC’s remit and purpose, which has important implications for the regulatory design/arrangements We then discuss four areas – differentiation in regulatory design, standard setting, risk-based regulation, and the regulatory or inspection workforce – and look at what evidence is available from several sources about how current regulatory arrangements work We conclude:     CQC’s generic regulatory model is unusual (compared with other regulators) and hard to make work There are good reasons to consider greater differentiation between sectors (such as NHS/healthcare and adult social care) and within large sectors like adult social care This might mean the development of more specific and tailored standards and guidance, and greater specialisation among the inspection workforce CQC’s current standards are largely construed as an essential or minimal level of performance CQC could within its existing legislative and regulatory framework create a more differentiated, demanding and service specific set of standards, and it could consider making more use of standards developed in or with the sectors it regulates CQC had used a risk-based model of regulation, in which it adjusted its use of regulatory interventions like inspection with providers based on an assessment of risk and performance, but has recently returned to a universal schedule of annual inspection in most sectors We find that even modestly proportionate or risk-based regulation requires a strong and stable database of performance data which has clear predictive validity, and a graduated range of regulatory interventions short of full inspection CQC’s use of a generic inspection workforce is, as far as we can see, not emulated by other regulators and is problematic for several reasons Regulatory staff need content expertise in the area they regulate, methods expertise in the regulatory system, and interpersonal or behavioural expertise in dealing with people and organisations in sometimes difficult and contested circumstances Specialisation has many advantages, and investment in the development of the inspection workforce is worthwhile We conclude that CQC could make more use of evaluations than it has in the past both when introducing innovations in its regulatory arrangements so that they are properly tested and in the routine working of its regulatory arrangements so that it has ongoing evidence of their effectiveness and impact Item Paper No: CM/01/13/04 Chapter Introduction Background and context This report is the result of a short research project commissioned by the Care Quality Commission (CQC) from the University of Manchester Its aim was firstly, to help CQC to bring evidence to bear on examining and exploring the effectiveness and efficiency of its current regulatory arrangements; and secondly to help CQC to develop its own internal capacity to undertaken and use research and evaluation so that, going forward, it can make better use of opportunities to test, trial and assess regulatory changes and innovations, improve its efficiency and effectiveness, and gain a more robust understanding of its impact on the quality of health and social care in England Analysis of the regulatory model Our first step was to develop and test a “logic model” which mapped out the underlying “programme theory” for each major component of the regulatory arrangements In this approach we drew both on established methods for programme theory explication (see for example Bickman 1987; Rogers 2008) and recent developments in realist or theory driven evaluation (see Pawson and Tilley 1997; Pawson and Manzano-Santaella 2012; Marchal et al 2012) The purpose here was to make explicit the assumptions or presumed mechanisms by which these regulatory components bring about change in regulated organisations In our experience, there may be multiple and sometimes contradictory theories in use, and part of the value of mapping out the programme theory is to express these alternatives in forms which then make them testable, and which bring to the surface areas of contestation or inconsistency We developed the logic model around the main statutory functions of CQC – registration, compliance, enforcement, and information provision For each function, we used documents and interviews with CQC staff to explore the mechanisms at work, and to try to understand the regulatory model and the choices and consequences it represents We were acutely conscious that this was a time of change within CQC, and the organisation had in September 2012 published a strategic review on which it was consulting stakeholders That strategic review signalled important, even fundamental changes to the regulatory model, and so in our work while we focused on mapping the current regulatory model we tried to take explicit account of the likely future direction as well Item Paper No: CM/01/13/04 As part of this work, we sought to establish what were the areas where research and evaluation might make an important difference to CQC’s decision making – where there was significant uncertainty about how (or how well) the regulatory model worked, and where change seemed likely On that basis we chose four topics for further research:  Generic versus differentiated regulatory standards and processes – whether regulators use the same methods, standards and processes for organisations in different sectors or of different types or whether and how they differentiate and use different standards or processes for different organisations  How regulatory standards are set and measured – issues like whether standards are minimal, median or maximal, how they are measured, whether compliance/achievement is measured dichotomously or on a scale, how compliance is defined and how a threshold for acceptable performance is set, and whether different standards are used/applied for different organisations  Risk based or proportionate regulation – to what extent regulators try to make the regulatory process responsive to organisational performance, and so focus more attention on organisations which perform less well or represent greater risk, and how this is done  The regulatory or inspection workforce – what competencies are required, what kind of people are used to undertake regulation and inspection, how are they recruited and trained/developed, how is their performance appraised In each of these four areas, we set about drawing together what was already known from the research literature; comparing practice at CQC with four other regulators; and exploring what was known or might be found from using existing data sources within CQC Learning from research We looked for existing research evidence in the four topic areas using a wide range of bibliographic databases The general strategy for the literature review was to identify relevant literature in public administration, healthcare, management and safety science, and where appropriate, governmental policy documents The researchers searched the MEDLINE (1946-2012), EMBASE (1980-2012), ABI Inform (1971-2012), ASSIA (1987-2012), British Nursing Index (1993-2012), Social Services Abstracts (1979-2012), and HMIC (19792012) databases For the regulatory workforce topic, PsycInfo (1806-2012) was also consulted The keywords used for each topic were as follows:  Differentiation of models: regulat* model* Item Paper No: CM/01/13/04    Regulatory standard setting: regulat* AND standard*; ("nursing home" or "care home") and regulation Risk-based regulation: risk based regulat* Regulatory workforce: regulat* AND inspect* AND (train* OR compet*) In addition to the search of academic databases, public domain repositories (for example, government and NHS websites, and relevant professional organisations) were consulted Also, the authors’ previous work on related topics, and the reference lists of retrieved articles, were consulted for relevant material Comparisons with other regulators We identified four other regulatory agencies with whom to compare CQC in the four topic areas We chose two healthcare regulators – the Joint Commission for the Accreditation of Healthcare Organisations in the United States and the Inspectie voor de Gezondheidszorg (the Healthcare Inspectorate) in the Netherlands We chose two UK-based non-healthcare regulators – the Office for Standards in Education, Children’s Services and Skills (OFSTED) which regulates children’s services, particularly schools, and the Homes and Communities Agency which regulates social housing providers Our aim was to provide a range of examples of regulatory policy practice, in both the health and social care setting and elsewhere Some important characteristics of the comparator regulators – their overall remit/purpose, their size and scale, and who and what they are responsible for regulating – are set out in table 1.1 We gathered data about the comparator regulators through a review of published and unpublished documents, and interviews with a member of staff from each regulator The interpretation of the comparator regulators’ arrangements contained in this report is, of course, ours rather than an official statement of the agencies concerned The table illustrates that the four regulators vary in some important ways, though they have much in common Their statements of regulatory remit/purpose – taken from their own documents such as annual reports – generally all focus on performance improvement though some are more ambitious than others They range in scale from about 120 staff to 1,400 or more, and in annual turnover from £48 million (€55 million) to £167 million pa Most though not all are public/state organisations (JCAHO is a private not-for-profit foundation) All regulate thousands of organisations, and all but HCA regulate across multiple sectors or service areas Item Paper No: CM/01/13/04 Table 1.1 Comparing CQC with four other regulators Name Care Quality Commission Joint Commission for the Accreditation of Healthcare Organizations Inspectie voor de Gezondheidszorg (Dutch Healthcare Inspectorate) Abbreviation Sector CQC Health and social care England Non Departmental Public Body JCAHO Healthcare IGZ Healthcare USA Private not for profit foundation Formal regulatory remit/ purpose to “protect and promote” the health, safety and welfare of service users and the “general purpose of encouraging the improvement of health and social care services” Annual turnover £149 million “To continuously improve health care for the public, in collaboration with other stakeholders, by evaluating health care organizations and inspiring them to excel in providing safe and effective care of the highest quality and value.” $165 million+ Netherlands Part of Ministry for Health, Welfare and Sport, but partially independent “Promotes public health through effective enforcement of the quality of health services, prevention measures and medical products.” No of staff 1,885 Type of organisations regulated No of organisations regulated Country Organisational form Office for Standards in Education, Children’s Services and Skills OFSTED Childrens services England Non ministerial government department Homes and Communities Agency “inspects and regulates to achieve excellence in the care of children and young people, and in education and skills for learners of all ages, thereby raising standards and improving lives” “focus of our activity is on governance, financial viability and value for money as the basis for robust economic regulation; maintaining lender confidence and protecting taxpayers” €55 million £167 million 800 500 Health and social care providers Hospitals, long term care, behavioural healthcare, labs, homecare, ambulatory care Curative healthcare, long term care, public health, pharmaceuticals and medical devices, health professionals 22k in health, social care 20k 3k organisations, 800k professionals 1,400 plus contracted staff Schools (maintained and independent), further education, adult learning, early years/ childcare, childrens homes, childrens social care, adoption/ fostering 23k schools and about 100k other providers £55 million (whole of HCA – regulation function not separated) 120 in regulation function Registered social landlords – incl LAs, housing associations and for-profit providers HCA Social housing England Non Departmental Public Body 1,500 though 400 large RSLs are 90% of provision Item Paper No: CM/01/13/04 The point to be made here is that there are different “ways and means” by which inspectors can encourage high standards amongst the organisations being regulated (Lynxwiler et al., 1983; Almond, 2006) Inspectors could adopt a dispassionate, distant, formal approach, or they could adopt a personal, involved and informal approach; either or both may be felt to be appropriate depending on the inspector and the regulated organisation A case in point is the studies of Lynxweiler et al (1983) and Nielsen (2006), both described earlier The challenge for a regulatory organisation is how to reconcile detailed regulations and standardised inspections, which may be required to make inspections consistent and defensible, with allowing inspectors to use their own judgement and initiative in the pursuit of their aims (e.g May and Wood, 2003; Bruhn and Frick, 2011) We turn now to the issue of what knowledge and skills regulatory staff require in the regulatory domain or content area they regulate Some have argued that the movement of personnel between a regulated industry and its regulator leads to the latter’s activities favouring the industry’s interests over those of other stakeholders (such as the general public) – what Meghani and Kuzma (2010) label the “revolving door effect” Others have asserted that inspectors need content expertise in order to make informed and valid judgements, and to have the necessary credibility and authority with regulated organisations For example, if health and social care inspection were seen as a branch of healthcare practice, then that would suggest that inspectors should come from health and social care organisations, if not from the health and social care professions This would carry the advantage of inspectors having domain knowledge and credibility with providers, which might help to gain their cooperation (Bohigas et al., 1998) On the other hand, domain knowledge may be less relevant if the task of inspectors is simply to “check off” organisations against a checklist of basic standards, and there is also the possibility that inspectors that are closely associated with the practitioners and health or social care organisations being regulated may be insufficiently detached to apply formal enforcement measures when necessary (in other words, they fail to exercise impersonal authority) If health and social care inspection were seen as a lay activity, then inspectors could come from outside health and social care organisations This would be likely to result in inspectors who were more detached from providers, relied more on formal methods and rules, and were less embedded within the practice community and so would need to work harder to establish a working relationship with those parties that they are regulating (ref needed) Plebani (2001) argues that the decision as to whether to take inspectors from within or from outside healthcare is ultimately dictated by the purposes of the regulatory scheme If the emphasis of the scheme is on quality improvement, education or the facilitation of selfregulation, then he suggests that inspectors should come from the healthcare professions as they will need domain knowledge He suggests that currently practising healthcare 59 Item Paper No: CM/01/13/04 professionals (brought in on a part-time or voluntary basis) would be of particular value because they have a first-hand understanding of contemporary practice However, there would need to be measures in place to prevent a conflict of interest between their inspection work and their “day jobs” But he argues that where the inspection work is more administrative in nature – for example, registration or compliance – then the work would lend itself to being performed by full-time staff Whether these staff need to be healthcare professionals or not depends on the required trade-off between technical knowledge and independence One option could be to deploy a team of part-time or voluntary inspectors from the professions that is supervised by a full-time lead inspector Plebani (2001) and Bohigas et al (1998) describe the inspector training regimes of several healthcare accreditation programmes These typically include initial training of between and 15 days, followed by continuation training of between and 25 days per year (the number of days varies between organisations, and often assumes a certain level of knowledge on the part of those recruited as assessors) The training includes both classroom teaching and the observation of assessments Some of the accreditation programmes also require assessors (where they are not full-time employees) to commit a minimum number of days per year to assessment work The International Atomic Energy Authority (2001) recommends that, in order to obtain the workforces that they need, regulatory organisations use job analysis and training needs analysis to distil relevant staff competencies This information can then be used to inform staff selection, training and development systems By way of illustration, the IAEA’s report provides examples from organisations in the UK and internationally The examples are from the nuclear sector, but the principles are relevant to regulators in other settings Finally, it should be noted that while this chapter deals primarily with the competencies of the inspection workforce, the activity of inspectors takes place in and is mediated by the organisational context Hutter (1989) and May (1993) argue that regulatory style is influenced by organisational, political and social factors These include the agenda set by the regulator through organisational policies and strategies and the nature and frequency of interaction between regulators and regulated organisations Rothstein (2003) cautions about the risk of “institutional attenuation” – a lack of awareness on the part of the regulatory workforce about the risks that they should be controlling He suggests three organisational factors which might lead to such attenuation, to which we have added a fourth:  Responsibility for controlling risks is divided between different people or parts of the regulatory agency, which leads to either “too many cooks” (all with their own agendas) or “no cook at all”  Incentives and rewards for the regulatory workforce are misaligned with regulatory requirements (e.g inspectors receive no credit for investigating certain hazards 60 Item Paper No: CM/01/13/04   despite their apparent importance, or are given targets for activity which perversely lead them to focus on low-risk but easy regulatory tasks) The organisational culture more broadly discourages inspectors from investigating hazards or problems properly (for example, certain beliefs about sources of risk may be promulgated, or the organisation shows a preference for “doing the doable” rather than “doing what needs to be done”) The organisation focuses on rule-based regulation in search of consistency and certainty of process, and leaves insufficient space for regulatory staff to exercise discretion and use their judgement in assessing and managing risk Comparisons with other regulators Table 6.2 below sets out a structured summary of the way that our four comparator regulators – the Joint Commission for the Accreditation of Healthcare Organisations, the Dutch Healthcare Inspectorate, OFSTED and the Homes and Communities Agency – recruit and develop the people who act as their inspection workforce There is considerable commonality of approach across the four regulators in the area of inspector expertise All require the people they recruit as inspectors to have substantial and senior experience in the content area they are regulating The two healthcare regulators use senior medical and nursing staff, complemented by senior healthcare managers OFSTED requires school inspectors to have substantial teaching and senior leadership experience – lead inspectors in particular are likely to have been headteachers in one or more schools Most HCA regulatory engagement staff have a background in social housing, and the financial analysts who work with them on economic regulation are qualified accountants The argument for using such staff in inspection was generally that they needed to be able to make – and stand by – complex professional judgements about performance, and to bring a depth of knowledge and understanding to what they observed or found during inspections It was also argued that regulated organisations would not respect or give credence to inspectors who did not have strong content expertise This makes the inspection workforce an expensive resource – all reported providing remuneration commensurate with market rates for the senior leaders and professionals they wished to recruit 61 Item Paper No: CM/01/13/04 Table 6.2 Comparison of approaches to the inspection workforce JCAHO Senior people with 10-15 yrs experience in field – CEO, board director, medical director, chief nurse etc IGZ 10-15 years in the field as doctor or specialist OFSTED Senior leadership experience in one or more schools – usually at head level for lead inspector What are the main roles and responsibilities of inspectors? Undertaking surveys and writing reports Undertaking inspections and writing reports What kind of people are recruited to roles as inspectors? Most surveyors part-time – have substantive ongoing posts in healthcare Full-time field directors oversee surveyors work Surveyors trained in classroom for days with days prep Surveyors then go on precepted surveys, last of which is an evaluation of their performance All inspectors employed by IGZ They have all worked in the sectors they regulate How is workload allocated to inspectors and what is a typical workload? Varies widely – part-time inspectors commit 25% of their time minimum Varies by sector Hospitals – caseload or orgs Nursing homes – caseload 50-100 Undertaking inspections, and writing reports HMIs oversight of decisions and QA of inspectors and inspections Most inspectors employed by inspection providers not OFSTED Small number of HMIs employed by OFSTED Inspectors undergo training by their inspection provider – assessed month course, with face to face and online learning Followed by mentored inspections then sign-off by an HMI Varies very widely and depends on how many days they are contracted for How is performance of inspectors assessed? Ongoing performance monitored by field directors Annual evaluations use peer ratings, onsite evaluations, provider evaluations and survey data CPD, oversight of inspections, checking of decisions like enforcement What competencies are required of those who undertake regulation/ inspection? How are inspectors trained and developed? IGZ has own academy Inspectors one year initial training (1 day pw) under supervision of senior inspector plus ongoing CPD HMI observe insoections, feedback from school after each inspection, oversight of decisions and evidence HCA Most staff in regulatory engagement have background in social housing Financial analysts are qualified accountants Analysts and engagement staff work together with portfolio of RSLs – reviewing data and engaging with RSLs Very little recruitment since recent reorganisations All regulatory staff employed by HCA No formal inhouse training programme in HCA – noted above very little recruitment in last years More senior staff get more complex RSLs Typical portfolio of 5-10 RSLs per member of staff Through normal staff line management arrangements However, there was more variation in the way that people were recruited to the inspection workforce, and the level of commitment they were expected to have Here there seem to 62 Item Paper No: CM/01/13/04 be two competing concerns On the one hand, inspectors need to be part of the regulatory agency, have a good understanding of its processes and procedures, and have some critical distance from the organisations they regulate – all of which suggests a full-time employed inspector model On the other hand, inspectors need current, up to date content expertise, and credibility with regulated organisations – which suggests the use of inspectors who still work in the sector JCAHO, having experimented with different models over the years, now has a mostly part-time inspection workforce (between 25% and 50% of their time spent working for JCAHO) most of whom have ongoing senior roles in the healthcare sector, but has full-time field directors who oversee surveyors work In contrast, IGZ has moved to a fully employed inspector workforce, and has tried to make the inspector role an attractive one which can form part of a medical or nursing professional’s career development OFSTED uses a hybrid model – it has a relatively small number of employed, full-time inspectors (Her Majesty’s Inspectors or HMI), with a much larger number of “additional inspectors” who are employed or contracted by the inspection providers who most inspections on behalf of OFSTED Traditionally most additional inspectors were retired senior school leaders, but OFSTED is now aiming to have at least half of the additional inspectors working part-time, and still holding a senior role in a school HMI inspectors undertake quality control and assurance of inspectors and inspections, and directly oversee critical inspection decisions HCA has a relatively small number of regulatory staff, who are all employed directly by the organisation The issues of inspector expertise and employment status/level of commitment affect how the comparator regulators use their inspection workforce, and particularly the issues of workload and specialisation For example, IGZ inspectors specialise in one or more of its regulated areas (acute care, nursing homes, homecare, etc) and there are also inspectors who specialise in functions like undertaking investigations or developing thematic inspections Although they are all full-time, as a result inspectors carry very different workloads – an acute care inspector might have a caseload of 5-6 acute hospitals while a nursing home inspector might have a caseload of 50-100 nursing homes JCAHO and OFSTED, who both use part-time inspectors, require them to commit to undertaking a certain volume of inspections each year, usually expressed in terms of a commitment to working days In both cases, full-time regulatory agency staff oversee the performance of part-time inspectors and quality assure their work All the comparator regulators apart from HCA have selection and training programmes for new inspectors, which generally combine some classroom/distance learning instruction and engagement with some inspection observation and mentored inspection activity Some of these are formal educational programmes with summative assessments (for example OFSTED’s provides 30 credits towards a postgraduate qualification) They also generally have continuing professional development requirements for inspectors, designed to maintain their skills and to update them on new regulatory developments HCA – which has 63 Item Paper No: CM/01/13/04 been through a number of reorganisations and staff reduction – has a relatively small number of regulatory staff and does not currently run a formal training programme In conclusion, one of the senior regulatory agency staff we spoke to said that there were three essential components to the regulatory regime – the standards, the inspection process, and the inspectors themselves – but that in their view the latter was both the most important component and the one which was most difficult to get right Using existing data CQC does not have any routine data available about the 900+ people who work as compliance inspectors and compliance managers, and this means we are not able to provide any example analyses of available data This is unfortunate, because the inspection data set does contain a field identifying the inspector (though we think this is the lead inspector only – and if more than one inspector takes part in an inspection, it would not identify other inspectors) This means that we could use that data to explore many of the issues to with inspector expertise, training, development and performance which have been raised in this chapter In addition, CQC has two other relevant data sets, though they are much more recent developments First, it has started to collect activity data from inspectors, which could be used to examine differences in workload and productivity Second, it is starting to routinely survey providers after inspection, and this data could be used to examine providers’ views of inspector performance There are four areas in which information about the inspection workforce is needed, which could be collected through a survey relatively easily:  Education, professional qualifications and training – education level and area, whether they have a professional qualification in health and social care, and if so what qualifications, and what other training (non-qualification based) in relation to health and social care services have they undertaken  Sector expertise – whether they have worked in one or more of the sectors that CQC regulates (NHS, independent healthcare, adult social care, general practice, dentistry etc) and if so for how long, in what role (seniority and responsibility) and how long ago  Regulatory expertise – how long they have worked for CQC and/or its predecessor health and social care regulators, and in what role (seniority and responsibility)  Regulatory training and development – what training and continuing professional development in health and social care regulation have they had during their time working for CQC 64 Item Paper No: CM/01/13/04 This would then allow us to examine empirically some important and interesting areas For example, we could find out to what extent inspectors already “specialise” in particular sectors or carry a mixed portfolio of providers, and we could see how often inspections are undertaken by inspectors who have relevant content/sector expertise We could also look at how much longitudinality there is in the inspection process, and whether providers deal with the same inspector each time there is a regulatory intervention or with different ones Perhaps more centrally, we could start to examine whether the educational/professional background, content expertise and regulatory expertise of the inspector is associated with any differences in inspection processes or judgements We might hypothesise that inspectors with more experience and expertise would reach more valid judgements, but we could test empirically whether they find more or less compliance We could also explore whether there are relationships between inspector characteristics and the views of providers on the quality of inspection We could also use this data set, alongside the inspector activity data, to start to examine the variations in inspector productivity and, for example, what impact greater inspector specialisation is likely to have on inspection time taken and workload Conclusions It is striking that neither the research literature nor the comparisons with other regulators provide much if any support for the use of a generic inspection workforce, of the kind that CQC has sought to create over the last three years, even with the current regulatory model It is also notable that the content of this chapter suggests that the changes signalled in CQC’s recent strategic review require substantial changes to the inspection workforce The consensus – from the literature and from practice in other regulators – seems to be that the quality of the inspection workforce – in three main domains, of content expertise, regulatory expertise, and interpersonal effectiveness – is very important For the regulatory agency to have credibility and authority in the sectors it regulates, and for people to give credence to its judgements and assessments, it seems necessary to have a high quality inspection workforce which performs consistently well This is not a straightforward undertaking Firstly, it is expensive, as experienced and senior inspectors need commensurate remuneration to attract them Second, it requires careful initial selection/screening and training and development Third, it demands ongoing performance appraisal and development 65 Item Paper No: CM/01/13/04 Chapter Conclusions This report is intended to help CQC to bring research and evidence to bear on its remit and purpose – the regulation of health and social care in England We have used logic modelling in chapter to set out an explicit description of the current regulatory model and to examine the underlying assumptions and likely consequences of key decisions in that regulatory design We have then explored four areas in more depth and detail – differentiation in regulatory design, standard setting, risk-based regulation, and the regulatory or inspection workforce In each we have tried to bring together what is known about the area from a number of sources – research, the practice of other regulators and CQC’s own practice – both to review the current regulatory model and to help inform decisions about its future The purpose of this concluding chapter is not to repeat or summarise the findings from chapters to 6, but to draw some conclusions about what they tell us about CQC’s current regulatory model, and how it might change CQC has already signalled some quite fundamental changes in its current strategic review, and other developments arising from the national health reform agenda and from the Francis Inquiry report are likely to bring further change We draw three main conclusions First, it seems that the current regulatory model, which was designed largely to achieve “safety net” regulation in health and social care, is probably not sustainable and requires change and development There seems to be an absence of good evidence to support the current regulatory model, and to back up important changes to that model which have been made over time It may be that we are unaware of the full background to, for example, the decision to return to 12 monthly inspections of all regulated organisations in most sectors; or the decision to inspect against a limited subset of the essential standards rather than against all the standards at each inspection, but it appears to us that those changes were made without strong supporting evidence in advance, and without much subsequent evaluation of their effects or impact This has left CQC in a difficult position when, for example, explaining the rationale behind decisions, justifying necessary levels of investment in the regulatory process, or demonstrating value for money Second, given that changes to the regulatory model seem both desirable and inevitable, our most important recommendation is that those changes should be well grounded in an empirical analysis of current practice and wider regulatory knowledge; that their intended mechanisms should be spelt out explicitly so that they can be tested and challenged; and that they should be implemented experimentally so that they can be evaluated properly 66 Item Paper No: CM/01/13/04 before they are used more widely We think there is great potential to design and execute evaluations alongside the piloting and implementation of such initiatives, and the size of CQC’s regulatory programme and the range of sectors which it regulates means evaluations at scale could be undertaken relatively quickly We think this would require CQC to develop or allocate more internal capacity in research and evaluation, and to create a culture of decision-making in which empirical evidence is more central Third, because the nature of regulation is complex and the interaction between regulatory agencies, regulated organisations and other interests is a dynamic and evolving one, we would predict that the effectiveness and impact of regulatory initiatives will vary over time, and between sectors and organisations This does not mean that research is not needed or helpful, but it does mean that it should not be assumed that interventions which work well during evaluations will continue to work in the same way when they are used in practice There is a need to design the regulatory process itself to be “autoevaluative” – so that the data routinely collected through the regulatory processes like compliance and enforcement provides an ongoing evaluation of impact and effectiveness This means, for example, routinely evaluating inspections through post-inspection provider surveys, which CQC is starting to do; and routinely evaluating the impact of follow-up inspections in cases of noncompliance, which CQC has also begun to But taking this further means aiming to ensure that each component of the regulatory model has data to demonstrate the validity and reliability of measurement and the impact or effectiveness of intervention on the quality of health and social care 67 Item Paper No: CM/01/13/04 References Adil, M (2008) Risk-based regulatory system and its effective use in health and social care Journal of the Royal Society for the Promotion of Health, 128, 196-201 Allsop, J & Jones, K (2006) Quality assurance in medical regulation in an international context Unpublished report, University of Lincoln Almond, P (2006) An inspector’s eye view: the prospective enforcement of work-related fatality cases British Journal of Criminology, 46, 893-916 Ashcroft, D.M., Quinlan, P., & Blenkinsopp, A (2005) Prospective study of the incidence, nature and causes of dispensing errors in community pharmacies Pharmacoepidemiology and Drug Safety, 14, 327-332 Ayres, I., & Braithwaite, J (1992) Responsive Regulation: Transcending the deregulation debate Oxford: Oxford University Press Bardach, E., & Kagan, R.A (1982) Going By The Book Philadelphia: Temple University Press Bardsley, M., Spiegelhalter, D.J., Blunt, I., Chitnis, X., Roberts, A., & Bharania, S (2009) Using routine intelligence to target inspection of healthcare providers in England Quality and Safety in Health Care, 18, 189-194 Bartlett, H.P., & Phillips, D.R (1996) Policy issues in the private health sector: examples from long-term care in the UK Social Science and Medicine, 43, 731-737 Bevan, G., & Hood, C (2006) Have targets improved performance in the English NHS? British Medical Journal, 332, 419-422 Black, J., & Baldwin, R (2010) Really responsive risk-based regulation Law & Policy, 32, 181213 Black, J., & Baldwin, R (2012) When risk-based regulation aims low: approaches and challenges Regulation & Governance, 6, 2-22 Bohigas, L., Brooks, T., Donahue, T., Donaldson, B., Heidemann, E., Shaw, C., & Smith, D (1998) A comparative analysis of surveyors from six hospital accreditation programmes and a consideration of the related management issues International Journal for Quality in Health Care, 10, 7-13 Boyle, T.A., Scobie, A.C., MacKinnon, N.J., & Mahaffey, T (2012) Implications of process characteristics on quality-related event reporting in community pharmacy Research in Social and Administrative Pharmacy, 8, 76-86 Braye, S & Preston-Shoot, M (1999) Accountability, administrative law and social work practice: redressing or reinforcing the power imbalance? Journal of Social Welfare and Family Law, 21, 235-256 Brennan, T.A (1998) The role of regulation in quality improvement Millbank Quarterly, 76, 709-731 Brown, M.K (1981) Working the Street: Police discretion and the dilemmas of reform New York: Russell Sage 68 Item Paper No: CM/01/13/04 Bruhn, A., & Frick, K (2011) Why it was so difficult to develop new methods to inspect work organization and psychosocial risks in Sweden Safety Science, 49, 575-581 Campbell, A.C., Foggin, T.M., Elliott, C.T., & Kosatsky, T (2011) Health promotion as practiced by public health inspectors: the BC experience Canadian Journal of Public Health, 102(6), 432-436 Carroll, M.M (1995) Is our role changing? Professional Safety, 40(4), Cohen, M.R., Smetzer, J.L., Westphal, J.E., Comden, S.C., & Horn, D.M (2012) Risk models to improve safety of dispensing high-alert medications in community pharmacies Journal of the American Pharmaceutical Association, 52, 584-602 Cornock, M (2012) Not another missed opportunity: regulation of health care professionals Journal of Commonwealth Law and Legal Education, 8(1), 1-8 Currie, G., Humphreys, M., Waring, J., & Rowley, E (2009) Narratives of professional regulation and patient safety: the case of medical devices in anaesthetics Health, Risk & Society, 11(2), 117-135 Daníelsson, J (2003) On the feasibility of risk regulation CESifo Economic Studies, 49, 157179 Day, P., & Klein, R (1987) The regulation of nursing homes: a comparative perspective Millbank Quarterly, 65, 303-347 Dodds, A., & Kodate, N (2011) Accountability, organisational learning and risks to patient safety in England: conflict or compromise? Health, Risk & Society, 13(4), 327-346 Elvey, R.E., et al (in press) Who you think you are? Pharmacists’ perceptions of their professional identity International Journal of Pharmacy Practice Emery, R.J., Charlton, M.A., & Mathis J.L (2000) Estimating the administrative cost of regulatory noncompliance: a pilot method for quantifying the value of prevention Radiation Protection Journal, 78 (Suppl 2), S40-S47 Flin, R., O’Connor, P., & Crichton, M (2008) Safety at the Sharp End Aldershot: Ashgate Fogarty, G.J & McKeon, C.M (2006) Patient safety during medication administration: The influence of organizational and individual variables on unsafe work practices and medication errors Ergonomics, 49, 444-456 Foley, M., Fan, Z.J., Rauser, E., & Silverstein, B (2012) The impact of regulatory enforcement and consultation visits on workers’ compensation claims incidence rates and costs, 19992008 American Journal of Industrial Medicine, 55, 976-990 Gormley, W.T (1998) Regulatory enforcement styles Political Research Quarterly, 51(2), 363-383 Hampton, P (2005) Reducing administrative burdens: effective inspection and enforcement Norwich: HMSO Health Foundation (in press) Using safety cases in industry and healthcare London: Health Foundation Healthcare Commission (2007) Is anyone listening? A report on complaints handling in the NHS London: Healthcare Commission 69 Item Paper No: CM/01/13/04 Hirsch, D (2009) Does litigation against doctors and hospitals improve quality? In J Healy & P Dugdale (Eds.), Patient Safety First: Responsive regulation in health care (pp 254-272) Crows Nest, NSW: Allen & Unwin Hoel, H., & Einarsen, S (2010) Shortcomings of antibullying regulations: the case of Sweden European Journal of Work & Organizational Psychology, 19, 30-50 Hofstede, G (1994) Cultures and Organizations HarperCollins Business Hood, C (1991) A public management for all seasons? Public Administration, 69, 3-19 Hutter, B.M (1989) Variations in regulatory enforcement styles Law & Policy, 11(2), 153174 Hutter, B.M (1997) Compliance: Regulation and enforcement Oxford: Clarendon Press International Atomic Energy Agency (2001) Training the staff of the regulatory body for nuclear facilities: a competency framework Report IAEA-TECDC-1254 Vienna: IAEA James, K.L., Barlow, D., McArtney, R., Hiom, S., Roberts, D., & Whittlesea, C (2009) Incidence, type and causes of dispensing errors: a literature review International Journal of Pharmacy Practice, 17, 9-30 Lindøe, P.H., Engen, O.A., & Olsen, O.E (2011) Responses to accidents in different industrial sectors Safety Science, 49, 90-97 Lloyd-Bostock, S., & Hutter, B.M (2008) Reforming regulation of the medical profession: the risks of risk-based approaches Health, Risk & Society, 10, 69-83 Lloyd-Bostock, S (2010) The creation of risk-related information: the UK General Medical Council’s electronic database Journal of Health Organization and Management, 24, 584596 Lodge, M (2011) Risk, regulation and crisis: comparing national responses in food safety regulation Journal of Public Policy, 31, 25-50 Lynxwiler, J., Shover, N., & Clelland, D.A (1983) The organization and impact of inspector discretion in a regulatory bureaucracy Social Problems, 30, 425-436 Macrae, C (2008) Learning from patient safety incidents: creating participative risk regulation in healthcare Health, Risk & Society, 10(1), 53-67 Macrae, C (2009) Making risks visible: identifying and interpreting threats to airline flight safety Journal of Occupational and Organizational Psychology, 82, 273-293 Macrory, R.B (2006) Regulatory justice: making sanctions effective London: Better Regulation Executive Marshall, B.K., Picou, J.S., & Schlichtmann, J.R (2004) Technological disasters, litigation stress, and the use of alternative dispute resolution mechanisms Law & Policy, 26, 289-307 Masso, M., & Eagar, K (2009) Do public inquiries improve health care? In J Healy & P Dugdale (Eds.), Patient Safety First: Responsive regulation in health care (pp 318-340) Crows Nest, NSW: Allen & Unwin Griffiths, H (2004) From CHI to CHAI: what a difference an ‘A’ makes The Psychiatrist, 28, 235-237 May, P.J (1993) Mandate design and implementation: enhancing implementation efforts and shaping regulatory styles Journal of Policy Analysis and Management, 12(4), 634-663 70 Item Paper No: CM/01/13/04 May, P.J., & Wood, R.S (2003) At the regulatory front lines: inspectors’ enforcement styles and regulatory compliance Journal of Public Administration Research and Theory, 13(2), 117-139 Meghani, Z., & Kuzma, J (2011) The “revolving door” between regulatory agencies and industry: a problem that requires reconceptualising objectivity Journal of Agricultural and Environmental Ethics, 24, 575-599 Miceli, M.P., Near, J.P., Rehg, M.T., & Van Scotter, J.R (2012) Predicting employee reactions to perceived organizational wrongdoing: demoralization, justice, proactive personality, and whistle-blowing Human Relations, 65, 923-954 Nebeker, J.R et al (2007) Developing indicators of inpatient adverse drug events through non-linear analysis using administrative data Medical Care, 45 (Suppl 2), S81-S88 Netten, A., Forder, J., & Knight, J (1999) Costs for regulating care homes for adults PSSRU Discussion Paper 1496/2 Canterbury: Personal Social Services Research Unit Nichols, E., & Wildavsky, A (1987) Nuclear power regulation: seeking safety, doing harm? Regulation, 11(1), 45-53 Nielsen, V.L (2006) Are regulators responsive? Law & Policy, 28, 395-416 Ojo, M (2010) The growing importance of risk in financial regulation Journal of Risk Finance, 11, 249-267 Papadakis, M.A., Arnold, G.K., Blank, L.L., Holmboe, E.S., & Lipner, R.S (2008) Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards Annals of Internal Medicine, 148, 869-891 Papadakis, M.A., Teherani, A., Banach, M.A., Knettler, T.R., Rattner, S.L., Stern, D.T et al (2005) Disciplinary action by medical boards and prior behavior in medical school New England Journal of Medicine, 353, 2673-2682 Phipps, D.L., Noyce, P.R., Walshe, K., Parker, D., & Ashcroft, D.M (2010) Risk assessment in pharmacy practice Report to the RPSGB University of Manchester Phipps, D.L., Noyce, P.R., Walshe, K., Parker, D., & Ashcroft, D.M (2011a) Risk-based regulation of healthcare professionals: what are the implications for pharmacists? Health, Risk & Society, 13(3), 277-292 Phipps, D.L., Noyce, P.R., Walshe, K., Parker, D., & Ashcroft, D.M (2011b) Pharmacists subjected to disciplinary action: characteristics and risk factors International Journal of Pharmacy Practice, 19, 367-373 Plebani, M (2001) Role of inspectors in external review mechanisms: criteria for selection, training and appraisal Clinica Chimica Acta, 309, 147-154 Power, I (2012) Inspection: are we content with simple compliance or we want real school improvement? Accessed from http://www.hmc.org.uk/hmc-blog/inspection-are-wecontent-with-simple-compliance-or-do-we-want-real-school-improvement/ on 24th Oct 2012 Prosser T (1999) Theorising utility regulation Modern Law Review 62(2):196-217 Rosen, R., & Dewar, S (2004) On Being A Doctor: Redefining medical professionalism for better patient care London: King’s Fund 71 Item Paper No: CM/01/13/04 Ross, S., & Hannan, M (2007) Money laundering regulation and risk-based decision-making Journal of Money Laundering Control, 10, 106-115 Rothstein, H.F (2003) Neglected risk regulation: the institutional attenuation phenomenon Health, Risk & Society, 5(1), 85-102 Rudkin, D (2009) Healthcare regulators and risk: assessing the evidence Clinical Risk, 15, 61-63 Smith, A F., Goodwin, D., Mort, M., & Pope, C (2006) Adverse events in anaesthetic practice: qualitative study of definition, discussion and reporting British Journal of Anaesthesia, 96(6), 715-721 Strong, D.E (2011) Access to enforcement and disciplinary data: information practices of state health professional regulatory boards of dentistry, medicine and nursing Journal of Health and Human Services Administration, 33, 534-570 Sujan, M.A., Koornneef, F., Chozos, N., Pozzi, S., & Kelly, T (in press) Safety cases for medical devices and health IT: involving healthcare organisations in the assurance of safety Health Informatics Sutherland, K., & Leatherman, S (2006) Regulation and quality improvement: a review of the evidence London: The Health Foundation Uzzi, B., & Spiro, J (2005) Collaboration and creativity: the small world problem American Journal of Sociology, 111, 447-504 Walshe, K., & Shortell, S.M (2004) Social regulation of healthcare organizations in the United States: developing a framework for evaluation Health Services Management Research, 17, 79-99 Waring, J (2007) Adaptive regulation or governmentality: patient safety and the changing regulation of medicine Sociology of Health and Illness, 29(2), 163-179 Wenger, E (1998) Communities of Practice: Learning, meaning, and identity Cambridge: Cambridge University Press Wiener, J.M (2003) An assessment of strategies for improving quality of care in nursing homes The Gerontologist, 43 (Special issue II), 19-27 Wiig, S., & Lindøe, P.H (2009) Patient safety in the interface between hospital and risk regulator Journal of Risk Research, 12(3-4), 411-426 Williams, S.D., Phipps, D.L., & Ashcroft, D.M (2013) Understanding the attitudes of hospital pharmacists to reporting medication incidents: a qualitative study Research in Social and Administrative Pharmacy, 9, 80-9 Wilpert, B (2008) Regulatory styles and their consequences for safety Safety Science, 46, 371-375 Bibliography Dewing, I.P., & Russell, P.O (2012) Auditors as regulatory actors: the role of auditors in banking regulation in Switzerland European Accounting Review, 21(1), 1-28 Hunter, S., & Waterman, R.W (1992) Determining an agency’s regulatory style: how does the EPA water office enforce the law? The Western Political Quarterly, 45(2), 403-417 72 Item Paper No: CM/01/13/04 Hughes, G., Mears, R., & Winch, C (1997) An inspector calls? Regulation and accountability in three public services Policy and Politics, 25(3), 299-313 Johnson, K (2012) Reducing administration errors: a case study Nursing & Residential Care, 14(6), 296-299 Kagan, R.A (1999a) Adversarial legalism: tamed or still wild? NYU Journal of Legislation & Public Policy, 179, 217-245 Kagan, R.A (1999b) Trying to have it both ways: local discretion, central control, and adversarial legalism in American environmental regulation Ecology Law Quarterly, 25, 718732 Lindøe, P.H., & Olsen, O.E (2009) Conflicting goals and mixed roles in risk regulation: a case study of the Norwegian Petroleum Directorate Journal of Risk Research, 12(3-4), 427-441 Putnam, M., Tang, F., Brooks-Danso, A., Pickard, J., & Morrow-Howell, N (2007) Professionals’ beliefs about nursing home regulations in Missouri Journal of Applied Gerontology, 26, 290-304 Tuijn, S.M., Robben, P.B.M., Janseens, F.J.G., & van den Bergh, H et al (2011) Evaluating instruments for regulation of health care in the Netherlands Journal of Evaluation in Clinical Practice, 17, 411-419 73