ITSWC Data Analysis

V2x Enterprise Monitor

The ITS World Congress in Detroit last month was an opportunity for eTrans a U.S. DOT Affiliated Test Bed Member to showcase our cloud-based services.  eTrans is realizing the future of transportation built on cloud-based services.  Affiliated Test Bed membership and our active participation in the Southeastern Michigan V2X architecture gave us a front-row seat. We want to be the first to showcase the benefits and advantages of sending data to the cloud and making it available to a diverse set of public and private users.

At ITS-WC, the USDOT made available several vehicles to demonstrate the benefit of V2V and V2I technology and cloud-based data. As part of this effort, the vehicles passed vehicle information, essentially copies of Basic Safety Messages (BSMs) up into the cloud to a data warehouse. Interested parties could then subscribe to the data warehouse and get access to the data. eTrans’ vision is to leverage this capability, which they demonstrated at ITS-WC.

Data Analysis and Refinement Process demonstrated

eTrans specializes in software, data quality assurance and security. We have developed a suite of tools to gather and analyze this vehicle data with a focus on security. Over the course of about the five days of ITS-WC, almost 1.5 million records of BSM type messages were collected. During the show, the progress of these vehicles were tracked in real-time. The following picture, one of the key outputs of the USDOT Affiliated Test Bed program, shows the flow of data during the event.

Transportation Data Flow

The three steps for the data, as shown above, to access eTrans were as follows:

  1. Each vehicle stored BSM information and passed it to the Connected Vehicle Wide Area Information Distributor
  2. From there it went to the Southeast Michigan Situation Data Clearing House
  3. In the third step, it flowed to eTrans cloud based system which have previously subscribed to receive that type of information.

Once eTrans received the information, it would begin to process it and display it. The following picture are examples of eTrans displays that are used to monitor and process the information in real-time.

eTrans2020 Displays

For eTrans though this is just the first step.  As you would expect the raw data collected, as is, is just a starting point. We analyze the collected data to find anomalies with the data and potential issues with how the information is processed.  This is the guide to help us find the records or series of records where the information just doesn’t line up or warrants further investigation, what you might call a refinement process.  This initial step helps to identify:

  • Missing fields
  • Records out of sequence (where the timestamps go in reverse)
  • Records where the vehicle doesn’t know its position or where it reports a clearly incorrect position
  • Records coming too close together or too far apart
  • Anomalies with the reported speed or with a comparison of the reported speed with a calculated speed
  • Anomalies with the rate of change in elevation
  • Anomalies with reported and calculated heading.

The second step is to “qualify the data”, that is use a set of data analysis tools to understand the root cause of data issues or errors. Data needs to be “qualified”, that is either have any issues resolved or to flag the data for further investigation and refinement (e.g. generally additional data is needed).

What we have learned so far

eTrans preliminary results have shown:

  • There are significant variations in how the individual vehicles operate.
  • The vehicles seem to “drift” at times, report incorrect data and then return.
  • In general, there isn’t an obvious pattern to the reporting timing
  • The issues tend to be inter-related

The figure below is a sample output from this process.


eTrans will provide the USDOT team a detailed presentation. Those interested in seeing the full report, please contact John Estrada at

What’s Next?

eTrans is also working on additional metrics to assess the ”quality” of their process.  Ultimately, they will score records as to data quality. As the data records are received, decisions therefore can be made on the usefulness of the data and the robustness of the security.

Security by design is an important design principle that needs to be evaluated. Leveraging cloud-based services is a new arena of development for the transportation industry and therefore has new challenges, namely security. We welcome the opportunities to be on “bleeding edge” of development.  We try out and test our new ideas at the various test beds, which gives us access to the emerging infrastructure.

1 reply
  1. says:

    We are confident motor carriers, drivers and also partners will see immediate and quantifiable benefits through readily available data and analytics around that information.

Comments are closed.