There is clearly a need for monitoring road safety engineering measures. Monitoring may be simply defined as the systematic collection of data about performance of road safety treatments after their implementation. Post-implementation monitoring is essential to ascertain the effects (both positive and negative) of a treatment. It is also important to monitor a scheme to assess whether or not it might have led to an increase in road collisions. It can also be considered to be a professional responsibil- ity to share the results of experience with peers, so that knowledge and skills can be mutually developed.
Ward and Allsop (1982) suggested that road safety schemes potentially affect the following parameters, and therefore some or all of them need to be monitored:
• The number and type of road collisions
• The severity of road collisions
• The distribution of road collisions
• Traffic flows and travel times
• Turning movements and delays at intersections
• Access times and distances within residential areas
• Routes taken by motorists, cyclists, and pedestrians
• Operation of buses
One challenge of monitoring collisions alone is that because collisions are rela- tively rare events, it may take a long time for a statistically reliable sample to accrue.
Engineering 265
This can be overcome by using proxy measures or indirect measures such as insur- ance company claims.
According to Ogden (1996), the essence of monitoring is to measure for each of the performance indicators what is actually happening in the real world and then, in evaluation phase, attempt to compare that with what we expect would have happened if the treatment had not been introduced. There are several experimental design challenges in doing this:
• There may be changes in the road environment such as a change in speed limit, change in traffic flow, and change in land use. All these are possible over a 3- to 5-year time period and virtually certain over an area or route.
• Road collisions are rare and randomly occurring events. There will be fluctuations year by year that might have nothing to do with the treat- ment being analyzed. Data for short time periods are highly unreliable.
These random year-by-year fluctuations, which not necessarily biasing the result of a monitoring exercise, introduce variability that must be accounted for in the statistical analysis. A particular problem is that of
“regression-to-mean.”
• It is necessary to monitor all significant factors that would possibly affect the outcome; otherwise, the outcome could be wrongly attributed to the treatment. If the variation in the treatment (e.g., the speed limit) varies sys- tematically with another variable (e.g., design standard), it may not be pos- sible to isolate the effects of one from the other. However, if only one is measured, it is likely that all of the change will be attributed to it.
• If the two variables that are systematically related are in fact both measured, then it will not be possible to reliably isolate their independent effects. This is particularly a problem if multiple linear regression techniques are used, since these require that the various independent variables are not correlated with one another.
• Statistical correlation does not necessarily imply logical correlation. For example, Haight and Olsen (1981) quoted in a case where the law giving pedestrians the right-of-way over vehicles was considerably strengthened in 1977, and the number of pedestrian deaths dropped from 365 in 1977 to 283 in 1983. However, the new law was not enforced and thus had no effect on behavior, so the improvement in the pedestrian situation must have been due to some other factors. This underlies the important of ensuring a link- age between the treatment being monitored and the change in the perfor- mance measure.
• Seasonal factors must be taken into account. Some factors may vary diur- nally (natural light, street lighting), and others will vary seasonally (rain, hours of daylight, traffic flow). The selection of factors such as control sites and before-and-after periods must take these variations into account. It would be incorrect to compare summer collision record with a winter col- lision record.
• Collision reporting levels may also change over time, and there may be inconsistencies in the data that would need to be considered. For example,
definitions attached to specific pieces of data may change over time, or the requirement to report collisions may have changed.
• There may be a long-term trend in collision occurrence, and thus changes over time in the number or rate of collisions at a site may merely reflect global trends.
The simplest method is to compare the collision record at the site before and after the implementation of the change. This according to critics is the least satisfactory method because of the lack of control of extraneous factors. For example, during the decade of the 1980s, several countries experienced a very substantial reduction in total casualty collisions. If a treatment installed in the middle of the decade was evaluated using, for example, 3- to 5-year before-and-after periods, it would quite possibly have shown significant reduction in the “after” period compared with the “before” period.
However, in reality, this may have mere reflected nationwide trends and had very little to do with the conditions at the site. Nevertheless, this method involves the following:
• Determining in advance the relevant objectives and corresponding evalua- tion criteria (Table 14.6).
• Monitoring the site or area to obtain numerical values of these criteria before and after the treatment.
TABLE 14.6
Statistical Tests or Procedures for Different Designs and Criteria
Evaluation Design Criterion Tests or Procedures
Before-and-after Frequencies χ2 for Poisson
Paired t-test
Rates Paired t-test
Proportions Z-test for proportions
Variances F-test
Distribution shifts Ridit
Kolmogorov–Smirnov Before-and-after with
randomized controls, comparison groups, or with correction for
“regression-to-mean”
Frequencies χ2 for Poisson frequency
Paired t-test for before/after within group t-test for group vs. group
Analysis of covariance Median test (categorical data) Mann–Whitney (categorical data)
Proportions Z-test for proportions
Rates Paired t-test for before/after within group t-test for group vs. group
Analysis of covariance
Variances F-test
Distribution shifts F-test
Kolmogorov–Smirnov
Engineering 267
• Comparing the before and after results.
• Considering where there are any plausible explanations for the changes and correcting for them if possible.
This process highlights the importance of determining in advance what the evalua- tion criteria are to be. While unexpected results might appear and the data should be examined carefully, the prime criterion is whether the treatment has had the desired effect or not. To this end, it is necessary to distinguish collision by type, and possibly time of day, or weather.
A major drawback of before-and-after studies approach is that it takes no account of trends of changes across the network as a whole. This can be overcome through the use of control sites. There are two variations of this method: the first using control groups that are randomly determined, and the second selecting comparison groups.
The first method involves a controlled experiment whereby several candidate sites for a particular treatment are identified in advance. They are then randomly split into two groups; all the sites in the first group are treated and no sites in the second group are treated. Their purpose therefore is attempting to make the control and treatment groups equal on all factors except the execution of the treatment. This method has a significant power as an investigative tool. However, it has limited validity for most applications faced by a road safety engineer, because there will rarely be the oppor- tunity to conduct a controlled experiment of this nature.
Therefore, the second methodology is of much more relevance. The process involves the following:
• Determining in advance the relevant objectives (e.g., collision types intended to be effected) and corresponding evaluation criteria.
• Identifying a control site or set of control sites, where no remedial works have been or are intended to be introduced.
• Monitoring both the treated sites and the control sites to obtain numerical values of these criteria before the treatment and after the treatment.
• Comparing the before and after results at both the treated and control sites.
• Considering whether there are other plausible explanations for the changes and correcting them if possible.
Selection of control sites is very important, and ideally they would be randomly selected. However, this is rarely possible, unless a large number of control sites can be identified, and a random selection made from these. The control sites should sat- isfy the following criteria:
• Be similar to the treated sites in general characteristics.
• Be geographically close.
• Have the same or similar traffic flows.
• Not be affected by the treatment at the test site.
• Not be treated in any way themselves for the period of the before-and-after study.
• Have collision records or other data that are consistent in collection criteria and coding.
Typical control sites include an adjacent section of rural highway or nearby network of urban streets.