Before-After Study

A before-after study considers a performance function (e.g. fatality rate, travel time reliability, etc.) of one or several sites before and after the implementation of a treatment. A before-after study, for example, in traffic safety (e.g. to develop a crash modification factor) compares the number of crashes occurred before and after a treatment is applied.

Developing CMF using a before-after study

Some points should be considered when using a before-after study for developing CMF:

• • Large sample size helps reduce the standard error of the estimate and therefore, more reliable CMF can be developed. If there is not enough data to proceed with this method, a cross-sectional method may be used.

  • Different types of biases should be kept in mind. Sometimes, the changes in crash observation after the treatment is not only because of the treatment. It maybe because of other factors such as change in traffic, effect of other countermeasures, time trends, change in the process of crash reporting (for example, the definition of minor injury crashes changes based on state enforcement law), or it may only be a result of regression to the mean (RTM) phenomenon. RTM is a common statistical phenomenon in simple before-after studies (also called naive study) which may lure an engineer into a seemingly (biased) pleasant or unpleasant result.

• It also should be noted that an appropriate functional form helps determine the true (reliable) effect of treatment.

A before-after study is usually preferred to a cross-sectional study, given that sufficient before and after data exists; but a simple before-after study (naive method) does not consider any other affecting factors into the CMF development process. In order to count for this downfall of the naive method, Empirical Bayesian (EB) method can be used which is a little more complex but more robust comparatively. 

Adjustment by adding a comparison group

Sometimes, in order to account for the effects of other factors than the treatment in a before-after study, a comparison group can be used in the analyses. The comparison group is a group of sites with similar characteristics were no treatments were applied –and not affected by the treatment at other sites. The ratio of observed crash count at comparison group in after treatment period to before treatment period. Since there is no treatment applied in the comparison group this ratio captures the effects of other factors such as time, traffic etc. This ration is multiplied to the observed crash count at treated sites in after treatment period to adjust for those other factors and remove their potential biasing effect. Now, the unbiased after treatment count versus the before treatment count is can be used to develop a more reliable CMF.

Given the above, however, still using a comparison group would not account for the RTM bias, unless the observed crash count on comparison sites in the before treatment condition are also matching that of the treatment sites. This is a hard challenge which is not easily feasible. If the RTM could also be addressed, then this adjustment to the before-after study can be suitably used instead of an EB method. 

[To be continued]