US Department of Transportation

Product: A Systems–Based Performance Measurement Framework for Highway Capacity Decision Making (C02)

Executive Summary

The Texas Department of Transportation (TxDOT) began implementing a performance–based planning process to meet legislative requirements and the agency's planning and programming goals, and to track its performance in meeting its goals and objectives. TxDOT developed a decision framework and the Performance–Based Planning Tool to facilitate the implementation of the agency' planning process. A key challenge was ensuring that the importance of border transportation projects, essential for facilitating international trade flows, was communicated and prioritized in statewide project portfolios.

TxDOT used A Systems–Based Performance Measurement Framework for Highway Capacity Decision Making (C02) to guide the piloting of the Performance–Based Planning Tool in prioritizing border transportation projects. Key components of this effort were identifying border transportation project criteria in consultation with state and local transportation planners and evaluating the relative priorities of border projects given the agency's goals.

The guidance offered by the SHRP2 product informed the piloting of the Performance–Based Planning Tool. TxDOT also identified areas in need of improvement with regard to the prioritization of border transportation projects in consultation with border stakeholders.

Agency's Challenge

During the 84th Legislative Session in 2015, the Texas Legislature passed Texas House Bill 20. In December 2015, Congress passed a new five–year federal funding and authorization bill (the Fixing America's Surface Transportation Act) that required state departments of transportation to move toward performance–based planning and programming. Performance–based planning and programming allows transportation agencies to allocate funding and prioritize projects to ensure that the agency meets its goals and objectives for the State's transportation system. TxDOT therefore initiated the development of a performance–based planning process and tool.

A key challenge was ensuring that Texas's border transportation needs were communicated and prioritized with the Performance–Based Planning Tool in the development of the agency's statewide project portfolio. Texas–Mexico trade amounted to approximately $187 billion in 2017 — 33 percent of all U.S.‑Mexico trade. The state's border transportation infrastructure, including the 13 commercial vehicle border crossings, rail and highway corridors serving the crossings, airports, and marine ports, is essential to facilitating U.S.–Mexico trade. For this reason, ensuring border transportation projects are developed remains a priority for TxDOT.

There are, however, a number of specific challenges in prioritizing border projects in Texas, including:

Defining border projects: There is no consistent definition for a border project. However, most stakeholders consulted agreed that not all planned projects within a geographic boundary from the Texas–Mexico border constitute a border project.

Understanding overlapping priority designations: While Texas has several corridor designations, it is not clear how border projects fit into them. Further, it is not clear in every case how one designation differs from another, complicating prioritization based on these designations.

Using consistent terminology: The terminology used in project prioritization frameworks is different in each planning and programming document. TxDOT's Performance–Based Planning Tool uses portfolio goals, parent criteria and subcriteria. The 2017 Unified Transportation Program and the Texas–Mexico border master plans (BMPs) talk about criteria categories and criteria. The use of consistent terminology will be a step forward in guiding the development of project portfolios.

Identifying border criteria and criteria weights: The challenge in this study was to get border stakeholders to identify a small number of criteria that distinguish border projects. There was great variation in the input stakeholders provided.

Overcoming data challenges: To ensure objectivity, each criterion must be quantified using the same data sources and methodology. In practice, collecting available data for all projects can be a major challenge. Furthermore, border networks vary in density and utility to Texas–Mexico trade, which can complicate the scoring process. Additionally, while data availability influences the priorities assigned to planned projects, some data do not become available until the project is in the construction phase.

Product Implementation

TxDOT developed its Performance–Based Planning Tool based on national best practices. A Systems–Based Performance Measurement Framework for Highway Capacity Decision Making provided examples of existing frameworks that use performance measures and targets to link agency goals/objectives to specific resource allocations. This product was consulted in the evaluation and piloting of the Performance–Based Planning Tool.

A Systems–Based Performance Measurement Framework for Highway Capacity Decision Making also provided guidelines for collaborative decision making. The Collaborative Decision–Making Framework (CDMF) is useful in that it provides the overall context for the Performance Measurement Framework. The CDMF identifies key decision points in the project development and planning process that could be improved with a more collaborative approach. The Performance Measurement Framework helps to support the CDMF by identifying relevant performance factors and measures to consider at each of the planning and project development phases. This framework was reviewed prior to piloting the Performance–Based Planning Tool for prioritizing border projects.

TxDOT's Performance–Based Planning Tool is designed to prioritize projects and optimize resources. The tool uses existing data and a consensus–based approach to set targets. These targets are designed to be consistent with expectations for system performance, abide by fiscal constraints, and guide investment and resource allocation as directed by House Bill 20.

In addition, TxDOT has developed a number of modal plans (e.g., Texas–Mexico BMPs and the Texas Freight Mobility Plan) to prioritize a specific class of projects of economic and strategic importance. The challenge is that there is very limited, if any, consistent dedicated funding available to fund these projects. The concern was then that the importance of border projects will not necessarily be communicated in the criteria that TxDOT uses to develop its statewide project portfolio and that high–priority border projects would therefore not be appropriately ranked in TxDOT's programming documents.

This study specifically evaluated how the rankings of border projects change if the unique attributes of border projects (aspects that distinguish border projects) are considered in TxDOT's Performance–Based Planning Tool. In the absence of dedicated funding for border projects, the criteria included in TxDOT's Performance–Based Planning Tool were modified (i.e., criteria were added and weights modified) to distinguish border projects and explicitly consider the importance of border projects in TxDOT's project portfolio development. The results of the analyses demonstrate the importance of identifying measurable criteria that capture the unique attributes of border projects in developing statewide project portfolios.

Stakeholder Collaboration

Developing the Performance–Based Planning Tool required input from stakeholders throughout the process — from determining measures and criteria to establishing priorities.

The portfolio goals used in TxDOT's Performance–Based Planning Tool were based on national and state–mandated planning guidelines including safety, preservation, mobility, economic development and the environment. TxDOT used a pair–wise comparison to determine the relative importance of each of the portfolio goals to the overall goal of selecting the best project portfolio. Stakeholders — Metropolitan Planning Organization (MPO) directors and TxDOT administration, district engineers, and division directors — anonymously provided their input. TxDOT then pooled this input to yield collective inputs and translate them into portfolio goal weights.

The next step was for subject matter experts to identify and determine the importance of different criteria for each TxDOT portfolio goal. At this level of decision making, the expert knowledge of subject matter experts was used, through group consensus, to determine how a variety of different criteria contribute to each portfolio goal independent of all other portfolio goals and in combination with one or more of the other goals.

Stakeholder input was also important to understand the criteria that distinguish border transportation projects from other transportation projects. The agency conducted outreach that included:

• A Webex meeting conducted with each of the TxDOT border districts.
• A web–based survey that was shared with TxDOT's border districts.
• A series of in–person meetings with TxDOT's border districts and Texas's border MPOs.

The goal of the outreach efforts was to request their feedback on communicating border project priorities and the identification of unique border criteria.

Key Outcomes

• Piloted the Performance–Based Planning Tool (Figure 1) using guidance from A Systems–Based Performance Measurement Framework for Highway Capacity Decision Making .
• Adapted and added criteria to distinguish border transportation projects in the statewide project portfolio.
• Ran different scenarios to compare border project rankings given changes in the criteria used. The following changes were made:
  • One of the environmental criteria was modified to consider whether the planned route diverts hazmat/nonradioactive materials.
  • The intermodal connector criterion was modified to also consider a land port of entry (POE).
  • The number of POEs served was added.
  • A criterion to quantify whether the project improves accessibility/traffic flow to and from a POE was added.

Lessons Learned

TxDOT has several planning documents and modal plans that identify transportation needs and prioritize projects for near– and long–term implementation. For border projects to be prioritized, they need to have a clear and consistent definition. As TxDOT embarks on the development of a Texas–Mexico BMP, the definition of a border project will be especially important.

In addition to defining border projects, it is also important to identify unique criteria with which to prioritize these projects. The study team found that it was difficult to identify criteria that distinguish border projects. This was due, in part, to the lack of available data. While certain criteria could have potentially distinguished border projects, these criteria could not be measured.

Most of the border stakeholders agreed that the economic impact of border projects is a defining characteristic. However, there are no clear criteria that would adequately reflect their importance. Pragmatism should be exercised in identifying meaningful criteria that have data available to quantify those criteria. Caution should be exercised not to include redundant criteria.

Further data challenges include using data sources and methodology consistently. In practice, collecting data for all border projects will be a challenge.

It is recommended that TxDOT's Performance–Based Planning Tool be used to prioritize planned projects in future modal studies. TxDOT's Performance–Based Planning Tool provides a transparent and objective framework for project prioritization and selection in the agency's planning (including the BMP planning effort) and programming documents. Given available data, the use of TxDOT's Performance–Based Planning Tool for developing the BMP will provide a relatively seamless mechanism for incorporating border projects in TxDOT's programming documents.

Next Steps

Work with MPOs to Develop Border Project Criteria

The metrics used in TxDOT's Performance–Based Planning Tool and by the MPOs to score and prioritize projects differ substantially. MPO metrics are often qualitative, while the Performance–Based Planning Tool uses quantitative metrics. Furthermore, while the Performance–Based Planning Tool and the Metropolitan Transportation Plan and Transportation Improvement Program criteria measure some of the same goals, they do it differently. Working with the MPOs to develop metrics and criteria would help incorporate regional priorities into statewide planning portfolios and more closely align the state and regional priorities.

Host Workshops

To develop a tool that is useful across various planning organizations, it is important to obtain input from the intended users. Hosting workshops with the planning professionals who could benefit from using the Performance–Based Planning Tool would help improve the usability of the software. These workshops would help explore the benefits and challenges of the Performance–Based Planning Tool. The input received during these workshops could lead to improvements in the software and increased use among planners.

Provide Support

The automation and data–driven approach offered by the Performance–Based Planning Tool can result in time savings and increased transparency. However, MPOs already find themselves stretched for resources, making learning and incorporating new software and processes challenging. With training and support, the burden on MPOs could be reduced. Further, proper training will ensure the tool is used to its full potential.

Connections to PlanWorks

This project did not use PlanWorks. However, elements of PlanWorks are relevant to the concluded effort. Specifically, the Stakeholder Collaboration Application, the Performance Measure Application, and the supporting strategies portion of the Stakeholder Collaboration Assessment could have provided additional insight.

For More Information