Harry Hatry, one of the best known experts on PM, indicates that a performance measurement system can be said to be fully implemented when it is taken for granted and its data are used regularly to help make program and policy changes that lead to improved services (Hatry, 1999). In short, the policy and management system relies on good PM data to make a wide variety of decisions, a ranging from resource allocation to program and policy evaluation. What follows is a short list of the most critical ‘‘lessons learned’’ about how to successfully implement a PM system. They are derived from the literature and the south Florida survey.
20.3.1 Picking the Right PM Requires the Active Involvement of Stakeholders in the Development, Reviewing, and Revision of Measures
This also means that the stakeholders need to also be actively involved in the interpretation of findings and the identification of implications. Poister and Steib (2005) found the most support and involvement for strategic planning and PM (i.e., in communities who had completed strategic plans or their development was underway) from the following stakeholders:
City Managers or Chief Administrative Officers — 97%
Department Heads — 93%
City Councils — 80%
Mayors — 78%
Citizens and citizen groups — 62%
Lower Level Government Employees — 46%
The south Florida survey asked a somewhat different question but the rank order of support and involvement of stakeholders paralleled the 2005 survey; that is, top administrators were the most supportive, then came elective officials and citizen groups, and finally lower level government employees. Therein lies the dilemma for leadership in the development
Enhancing the Utilization of Performance Measures in Local Government g 571
of PM — measures will be needed that satisfy different kinds of audiences.
Also it is natural that the top administrators will be more committed and that more effort will need to be devoted to external stakeholders and elected officials. A variety of stakeholders will have to ‘‘buy into’’ the development and use of PM.
Another way of stating the dilemma faced in the development of feasible PM is to think about how to build collaborations around outcomes. How can successful collaborations be built across organizations and sectors so that a selected number of PMs can reflect the consensus of the community (Newcomer, K. et al., 2002). An example will demonstrate the challenge to building collaborations around outcomes. Let’s take the law enforcement field again. As indicated earlier, citizens are often enamored with response time because they feel comfort when police cars come quickly and are visible at the scene of the crime, regardless of whether response time has any impact on whether the case is solved. Professionals in the police department, however, prefer clearance rates. They feel that actually closing cases is the bottom line in crime fighting. Of course, budget and fiscal people will most certainly ask for the cost of a call for service. We also know that most elective officials like the reduction in crime rates as a measure (Dluhy et al., 2000).
So what can be done? The answer is to design a decision making pro- cess to develop the PMs from the beginning which includes representa- tives from all relevant audiences. You cannot simply rely on top-level management to develop and implement indicators, you will need to design a collaborative development process. One popular approach is to use a Community Advisory Task Force made up of citizens, community groups, elective officials, top-level administrators, and rank and file employees.
This Task Force should hold community forums, run focus groups, seek community input, encourage input from the bureaucracy, and allow elective officials to participate as well as to endorse the measures adopted. The Asheville-Buncombe Community in North Carolina about ten years ago began developing and is now using indicators which were developed and then sanctioned by a cross cutting Community Advisory Board (Asheville-Buncombe Vision, 2005).
While there are certainly other collaborative approaches that can be used, the main ‘‘lesson learned’’ about stakeholder involvement is that the development of performance measures are not to be left to expert consul- tants or professional staff as a bureaucratic exercise. Real ‘‘results-oriented management’’ requires widespread stakeholder involvement and the build- ing of consensus around a small set of outcome measures. In the example above, it may be necessary to include response time, clearance rates, crime rates, and the cost of calls for service in the final measurement paradigm adopted. Consensus in the use of outcome measures may simply mean
having enough measures to satisfy different constituencies but the critical message is — do not exclude the outcome preferences of important stake- holders. PM is not a narrow bureaucratic responsibility, it is a community based effort where important stakeholders each have their expectations included in the group of final measures used, even if that means that 5–10 measures per service area ultimately get adopted and used. Enhancing utilization means involving key stakeholders and expanding the ‘‘by in’’
process across the community and the bureaucracy.
20.3.2 Long-term Success for PM Requires That the Concepts of PM be Fully Integrated into the Organizational Culture and the Community.
PM Needs to be Institutionalized in Government and the Community
Long-term success also means penetrating all levels of the organization, not just the upper levels and spreading the concepts into different sectors of the community. Accordingly, here is a list of activities, which will support the integration of PM into the bureaucracy:
Require the use of PM in budget proposal justifications Provide training in PM for all levels of the organization Use PM in contracts for service
Use PM in annual performance appraisals for employees Connect pay to performance
Use PM in program evaluations
Incorporate PMs into the Strategic Planning or other planning processes
In addition, the following activities will support the integration of PM into the community:
Annual or quarterly reports to the public on the results of performance measures
Community forums discussing PM
Maintenance of interactive government web-sites where the mea- sures are posted
Press releases and news conferences where results are presented and discussed
Sponsor PM conferences and seminars for community leaders and citizens
More generally keep the data in front of the media using a comprehensive public relations strategy
Enhancing the Utilization of Performance Measures in Local Government g 573
This type of comprehensive integration of PM into the bureaucracy and the community requires what Carter calls the need for a Champion (Wholey, 1994). Champions are leaders inside or outside of government who take on a single mission like the development and implementation of PMs in government and the community. From the south Florida survey, participants agreed that it usually takes around three years to develop and fully implement a performance measurement system and that a champion with visibility and credibility is absolutely essential. PM needs leadership and people need to see and hear about the vision of where the govern- ment and the community will be after a PM system is fully implemented (Dluhy et al., 2000).
20.3.3 Successful Utilization of PM also Requires Clear and Persuasive Presentations of Results to
Different Audiences
Harry Hatry (1999) devotes more than half of his popular book on PM to how practitioners can specifically present PM in a way that varied audiences can understand the data. It is also important for PM advocates to provide the public and other audiences with the reasons or explanations for changes or trends in the data. When presenting the numbers, advocates of PM need to make sure to also include brief but succinct explanations for changes and trends in the data. Some of the other important tips on data presentation are (Hatry, 1999):
Use simple line graphs and trend lines to portray data changes over timeUse bar graphs and pie charts to show differences in percentages and
proportions
Use color coded GIS (government information systems) maps and breakouts for neighborhoods, jurisdictions, and regions to dramatize differences in service measures
When comparing oneself to other jurisdictions or communities normalize or standardize the data wherever possible
Use average performance levels when comparing a large number of jurisdictions and then indicate how close to the average your jurisdiction is (or is not)
Another aspect of good data presentation is to make sure the data used is credible and of the highest quality. One suggestion here is to regularly involve experts from a local or regional university, think tank, or well- respected consulting firm to join in the development and reporting of
the information. These research oriented experts can be part of the larger Community Advisory Task Force mentioned earlier or they can be assem- bled regularly to provide technical advice on the measures used and their reporting. Working with these type of expert organizations also allows the government and the community to supplement census and other government-collected data with other data, since larger universities and expert organizations are very likely to have other applied research capabi- lities like survey research. Involvement of experts will give further legitimacy to PM and enhance the use of it in the community.
Finally, with data presentation, it is important to be extremely cautious when using PM data to benchmark your government or community with another government or community. In this context, benchmarking means looking at others providing the same service and comparing you with them. That is, how are you doing compared to your peers ? The trick, of course, is to be able to select the appropriate peers for comparison and to make sure the service measures used are indeed comparable. A number of years ago, a colleague and myself were engaged in research comparing fiscal stress and economic development strategies between Miami and other cities around the country (Dluhy and Frank, 2002). Some colleagues argued that Miami should be compared to other large cities in Florida, others said the comparison should be made with cities around the country and still others said cities in Latin America since Miami’s population is so heavily Hispanic.
After much debate, a decision was made to compare Miami with other cities in the U.S. that were about the same population size since the size factor allowed many of the peer differences to wash out. Miami wound up being compared to places like Newark, St. Louis, and Cleveland rather than N.Y., Los Angeles, or Chicago or to much smaller cities in Florida.
When the analysis was finally completed, it was interesting that cities of about the same size only differed slightly on things like poverty, educational levels, and spending and service levels and Miami did not look too bad at all. However, had Miami been compared to Florida cities, it would have looked very troubled using almost any indicator. The lesson is to carefully and cautiously pick peers when comparisons are being made. You could make a big political mistake if you compare an apple with an orange.
20.3.4 Successful Implementation of PM Requires Leadership to Build Trust and Credibility in the Community
Building trust and credibility in any government enterprise is a necessity.
Whether it is the PM champion or a group of top administrators, or the City or County Manager or an elective official who is leading the effort
Enhancing the Utilization of Performance Measures in Local Government g 575
for institutionalizing PM, the broader community will demand that those leading the efforts arenot using PM for personal or political reasons. Many in the community want to trust the data, accept the interpretation of the trends taking place, and support the efforts aimed at improving or at least maintaining the desirable outcomes. With this as a back-drop, when measuring an outcome you should not avoid measuring the hard ones or choosing measures that have long term meaning. It would be easy, for example, to measure easy things with no meaning like response time in law enforcement, the number of re-zoning applications approved in planning, or the number of homeless people in shelters on the coldest day of the year. It would be more difficult to identify the intended or preferred outcome of interest that a consensus of stakeholders believe makes a real, long term difference. So, to build credibility and trust we need to work hard at getting at the right thing. In the examples above, clearance rates for criminal cases, the impact of the re-zoning application on traffic flows and infra-structure, and the percentage of homeless who return to the labor force are a lot better indicators to use.
Trust and credibility are also hurt when we misuse data, exaggerate its importance, or select measures that make the government service look good. To avoid these kind of pitfalls, expert stakeholders must be teamed with more political and organizational stakeholders when the measures are developed. It is human nature for department heads, CEOs, and elective officials to look for the easy way out, which in this case means selecting measures that they know in advance will give a positive spin on the service in question. We may not want to hear that response to burglar alarms are expensive to respond to and that 95% of them are false. Many citizens would rather hear that the average response time to the alarm is 8 minutes. In the long run, you have to pick the best measure not the expedient one.
Another dimension of this discussion of choosing the most appropriate measures involves whether you select indicators that are the easiest to influence. For example, a look at the federal mandates under the ‘‘ No Child Left Behind’’ Legislation is revealing. As we know, students in certain grades are regularly tested to see if their test scores in reading, science, and math are improving. Of course, as a result of the legislation, the teachers are now teaching to the tests so that the school is not penalized in the budgetary process for slumping scores. What is being avoided is measuring the longer-term effects of instruction and types of instruction on employ- ment, wages, career development, addiction, violence and incarceration, etc. The test scores are easier to measure and influence than the other outcomes mentioned above but the longer-term measures are more revealing and salient.
PM systems can provide the best benchmarks for how things have progressed over time, how a service or government compares to other
services and jurisdictions in other communities, and how expectations for performance can be developed for the future. Without an open and credible process for developing a set of indicators, citizens will eventually increase their cynicism and lower their levels of trust in ‘‘results-oriented management’’. Political trust in the U.S. has suffered over the last few decades but now the ‘‘results-oriented management’’ approach offers an alternative to this historical lack of trust and cynicism in government and government processes.