Browse Projects > Detailed View

Determining and Communicating Reliability of Crash Prediction Models


During initial implementation of the Highway Safety Manual (HSM) (1), the state of the art of safety analysis has progressed and more has been learned about the impact on accuracy of assumptions made during the development of crash prediction models using HSM procedures. It also appears that practitioners have a significant need to better understand and communicate the reliability of crash prediction models in terms of the implications on engineering practices and decisions. This reliability includes general model accuracy, in addition to accuracy when models are executed at or outside the appropriate design thresholds (i.e. e=8%, very low average annual daily traffic (AADT) values, etc.). In addition, it is important to understand how models that over- or under-predict crashes in comparison to observed crashes will impact the results used for decisions. Case studies presented at various conferences, including the Transportation Research Board (TRB) Annual Meeting, and through other initiatives (2) demonstrate that practitioners are utilizing the models in ways not recommended in the HSM and are also displaying crash prediction results without properly understanding the model reliability. Guidelines for determining the reliability of prediction models will aid practitioners in the appropriate development and application of the models. Understanding reliability of crash prediction results and being able to appropriately explain how results should be interpreted is critical to both performing a credible analysis and to ensuring the results are used correctly to support decisions. The guidelines developed through this research will help transportation agencies overcome barriers related to using statistical prediction methods that can aid engineering decision making. This topic was among the top ranked research needs identified at the 2013 Safety Effects of Geometric Design Decision Workshop, which was a joint meeting of TRB committees that focus on design, operational effects of design, safety performance, safety data, and asset management (AHB65, AFB10, ANB25, AHB70 and ANB20) and the AASHTO Technical Committee on Geometric Design. In addition to being of interest to both practitioners and researchers, a better understanding of the reliability of individual models and the general concepts of crash prediction reliability will support the AASHTO SCOHTS’ Strategic Plan (June 2011) by addressing Goal 2, related to the institutionalization and further development of the Highway Safety Manual, and Goal 4, Strategy 4 by developing tools that can better quantify changes in safety performance.


The primary objective is to develop guidelines for estimating, calculating, and reporting reliability of crash prediction models including crash modification factors/functions (CMFs), safety performance functions (SPFs), calibrations, and combinations thereof. In development of the guidelines, consideration should be given to the balance between improved model reliability and user friendliness.

Specific steps to achieving this objective include:

· Reach out to HSM practitioners, model developers, and end users, including decision makers and the public, to look at ways that model results are being presented and interpreted.

· Perform a literature review to create an understanding of issues that impact reliability (e.g., variance and functional form) of the existing crash prediction models, especially at the outer limits and ranges of the models. Models and model elements developed should be explored to include all known model forms and methods of determination found in the HSM or recommended in existing literature. In addition, existing literature should be used to supplement the understanding of current prediction model application.

· Highlight a number of case studies that demonstrate the issues with model reliability.

· Develop a method for calculating and expressing crash prediction model results that is user friendly and better expresses the model reliability. Examples include providing model elements as a range (i.e. CMF for paved shoulders ranges from 0.8 to 0.9), calculating model results as a range (i.e. total predicted crashes will likely be between 20 and 25 for the study period), and presenting results in terms of levels or ranges (similar to level of service for capacity).

· Apply recommended model structure to at least one current HSM model.

· Include a dissemination plan for providing outreach to the practitioner and research community.


The guidelines developed with this research will be able to be applied to existing crash prediction models and will serve to improve all future models and model elements. This could impact a range of documents including the HSM and all supporting documents that provide guidance on model development.

Related Research:

The HSM, published in 2010, provided models for predicting crashes for several common roadway facilities. The HSM provided methods for calibrating the models and some qualitative guidance on the model reliability but lacked methods to quantify the model variance and confidence. Subsequent research (3,4) furthered the reach of these models to a greater number of facilities but failed to address the issue of model reliability. Recent publications (5, 6) and ongoing research (7) have begun to address the issue and form some guidance on this issue. However, a more comprehensive study is still needed to further develop a method of how to calculate model reliability and promote a dialog regarding the most appropriate way to communicate model results.


The guidelines developed with this research will be able to be applied to existing crash prediction models and will serve to improve all future models and model elements. This could impact a range of documents including the HSM and all supporting documents that provide guidance on model development.


Crash prediction model results are currently being used to make project level and programmatic decisions without complete understanding of their reliability. Specific examples include lack of understanding of compounding errors, use of models at or near their known limits, poor understanding of model limitations and inclusion of CMFs not originally derived for the model. Continued use of models in this way may lead to suboptimal design of projects, degradation of credibility, and open concerns of liability and public trust. The guidelines developed with this research would not only address these concerns but also promote better informed engineering judgment in the model application, greater implementation of crash prediction models, and better acceptance of model results leading to improved decision making.

Sponsoring Committee:AKD10, Performance Effects on Geometric Design
Research Period:12 - 24 months
Research Priority:High
RNS Developer:Howard Lubliner, Jim Brewer, Mark Doctor, Patrick Hasson, Raghavan Srinivasan
Source Info:Problem statement developed as a result of the Safety Effects of Geometric Design Decisions Workshop at the 2013 mid-year meeting of TRB Committees AFB10 (Geometric Design), AHB65 (Operational Effects of Geometrics), ANB25 (Highway Safety Performance), AHB70 (Access Management), and ANB20 (Safety Data, Analysis, and Evaluation), in conjunction with the AASHTO Technical Committee on Geometric Design. Problem Statement # 2015-G-21 on the NCHRP/AASHTO SCOR list.
Date Posted:01/09/2014
Date Modified:01/28/2014
Index Terms:Crash prediction models, Highway safety, Highway Safety Manual, Reliability, State of the art, Traffic crashes, Guidelines, Annual average daily traffic,
Cosponsoring Committees:ACP60, Access Management; ACS20, Safety Performance and Analysis
Operations and Traffic Management
Safety and Human Factors

Please click here if you wish to share information or are aware of any research underway that addresses issues in this research needs statement. The information may be helpful to the sponsoring committee in keeping the statement up-to-date.