World State Encoder for Driving Scenario Mining

Finding specific driving scenarios in recorded vehicle data is difficult when each new scenario requires hand-written rules. I led development of a World State Encoder to test whether classifiers trained on temporal vehicle and surrounding-agent data could identify scenarios in historical driving logs.

I adapted the existing ML Comfort Metrics infrastructure into a general scenario-mining pipeline, covering data acquisition, feature generation, ROCKET temporal representations, XGBoost training, and scenario-level evaluation. Before selecting that approach, I evaluated zero-shot time-series foundation models and found that ROCKET with XGBoost remained the stronger baseline for our use case. I designed the encoder’s architecture and feature selection, implemented the pipeline, trained models, and evaluated it across seven driving-scenario datasets. I also extended the encoder to represent multiple nearby agents, testing different ways to select, encode, and combine them.

The work produced a functioning telemetry-based scenario-mining proof of concept and a reusable training pipeline in Metrics Engine. Multi-agent experiments improved some cases, but performance on difficult ego-centric scenarios remained insufficient. Those results prompted investigation of richer learned representations, while the earlier pipeline work began moving into the team’s Omnitag search infrastructure. The encoder established a tested direction for scenario mining without resolving every scenario class.