Embedded Files
Performance measurement is a common term in many fields of engineering. What it means is to quantitatively evaluate (measure) the effectiveness or the quality of an action. You say something has a good quality, say your fridge, or your car, but how good is it? That is how does it perform and if you are able to give me a numerical value that shows its quality, then voila, you have a performance measure. Here we are concerned on how good is the safety by defining a measure to give us an idea on the quality of safety.
The National Highway Traffic Safety Administration (NHTSA) and the Governors Highway Safety Association (GHSA) have determined a set of safety performance measures that can be used by State and federal agencies. These measures can be used for screening purposes or for evaluating the effectiveness of crash countermeasures. It must be noted that these are only a sample (the most common) measures that are used and one can come up with any other performance measure as appropriate for a given condition. For example, I may say that the number of fatalities of children under 10 years old is a good performance measure for such or such geographical location.
The following list presents 5 of the 10 top safety performance measures in the core outcomes of the study by NHTSA and GHSA: They are named as C-#. For a list of all the measures and the study report click here (last accessed 02/09/2015).
C-1) Number of Traffic Fatalities
C-2) Number of Serious Injuries
C-3) Number of Traffic Fatalities per VMT (Vehicle Miles Traveled)
C-4) Number of Unrestrained Occupant Fatalities
C-5) Number of Impaired Driving Fatalities
Safety performance measures can also be used for defining reasonable safety targets in the future. For example, a State DOT does an analysis of their statewide safety performance and determines their target on the number of fatalities for 2017 (given that we are in 2015). Each performance target determined should be justifiable based on the safety data. A standard goal statement can be written as e.g. "Reduce number of traffic fatalities by 15% from 1,000 (average value over 2012-2014) to 850 by 2016. In this example, a 3-year average is used as the base and one year in the future, 2016, as the target year.
This is done by several methods that here I will try to demonstrate how each of them work. Assuming the performance measure C-1, the number of fatalities the following approaches can be taken.
Linear Trend Analysis
This method is simply about finding a linear trend of the number of fatalities over the past years and projecting it into the future and see what would be the expected value for any specific year. This is the general scheme of the method which sounds pretty straight forward. There are some customs that are adopted by highway agencies that are worth mentioning. First is that how many years in the past we are going to look at the data for developing the fatality trend? Some use three years, some use five years and I still do not know why not more than five years!!
Second is that whether to use one-year number of fatalities as our data points or an average of several years? Again, some use 3-year moving average and some use 5-year moving average. Depending on what the policy of the agency is and the quality and availability of data either one can be used. Let's assume that I generated the following chart from my data.
The chart includes the diagram for all of the accustomed conditions mentioned above (click on chart to see full size).
As it can be seen the annual data points have much more variability compared to the moving averages. The reason that the moving averages are used as data points in this approach is to alleviate those variations that can represent random noise. The true trend underneath the actual variations of fatalities (is assumed to) have a function. By looking at these moving averages we can have a better idea of how the crash statistics are changing over years.Using any one of the above diagrams we can fit a linear trend to the data points and use its equation to predict (our targets) for the future.
This same process can be done for the other sets of data points (also with more years of data, depending on the policy) and fit the linear trend, determine the changing (decrease/increase) rate of the chosen performance measure and forecast its value for future as the target value. Here is an Excel file I created for this approach that has everything ready and available. the user only needs to input the yearly data for the performance measure. The actual and moving average regression for different periods are presented and only waiting for the required data. For the passing word use "gishar" and make sure that the macro is enabled.
Note. Moving average method does not account for a nearby record to make a higher influence on the curve than a record that is located further away. If this higher influence of few records is important to be visible in the moving average curve, the "Nadaraya-Watson Kernel regression" can be used which is also a non-parametric method (a moving average smoothing method). For further information on this method, one can study chapter 4 of the book "The Art of Regression" written by Ezra Hauer.
Alternate Baseline Analysis
In this approach, the average percent change in the performance measure of several recent years (α years after the last year of the base period) is determined with respect to a β-year period. The resulting average percent change provides an estimate for the potential change in the target year given that the data for the baseline is available. The usual values that are used for the α and β are 3, and 5 respectively. Here is an example, Let's assume we want to use this approach to determine the target value of the number of fatalities in 2016. We have a 5-year period baseline and a 3 year temporal difference between the last year of the baseline and the comparing year. So for this case, we need to have the data for 2016-3 = 2013 and four years before that. In order to find the average percent change of the past years we can use the data for the years of 2012, 2011, and 2010 and compare their values with their corresponding 5-year baseline periods. That is, for example, if the average value of the annual number of fatalities for the years of 2005-2009 was 10 and three years after that in 2012 the annual number of fatalities was 9, there was a 10% reduction. similarly, we compare the average value for the period 2004-2008 with 2009 and see that there is 12% reduction, and so on. After all these calculations, the average values of the reductions will be determined and used for the target value. Let's say the average value of reduction from our calculation was found to be 11.76%. Now we determine the average annual fatality of 2009-2013 and apply the 11.76% reduction to it to find the target number of fatality for the year 2016 (three years after 2013). Once the approach is understood, it will be very easy to implement. Also, for this approach I created an Excel file that is ready to use. The user only needs to input the yearly data for the performance measure. Also, the α and β values can be changed and see how the target values change. please use "gishar" for the passing code and make sure that the macro is enabled.
Judgement Call
Well after you have the above approaches, you will come to an idea about your target value of the performance measure in a specific year in the future. You may still forecast the target value based on your on engineering judgement and not necessarily consider one of the calculated values as the unchangeable true value to be used as a reachable target.
For example, the next graph shows the trend line fitted to the data for the 3-year moving average diagram, extended for two more years. Now that we have the equation for this trend line, we can use that function to interpret the safety changes and statistics and also determine the number of fatalities for any year in the future. This chart for example, shows that the number of fatalities has a decreasing rate of about 8.5 deaths per year and a projection of the trend for 2015 is determined by solving the equation which results in 188 fatality. One can then say that this value (the target) is about 14% lower fatality compared to the last five year average (2008-2012) fatalities.
Page updated
Google Sites
Report abuse