Grid
Analytics
A real-time fault-analysis platform for power-grid operators — parsing IEEE C37.111 oscillography data, classifying protection events at sub-5ms latency, and storing years of signal history queryable in milliseconds.
IEEE C37.111 Oscillography at Scale
COMTRADE files from protection relays are dense binary formats with sub-millisecond sample rates. Parsing thousands of fault records per hour requires both correctness and raw throughput — a Python-based pipeline was the original bottleneck.
Sub-50ms Fault Classification
Grid operators need fault classification results within 50ms of data arrival. The incumbent pipeline was exceeding 800ms p99 and dropping events during high-load periods — rendering it useless for live response.
Long-Term Time-Series Storage
Millions of signal samples accumulate daily. Standard relational databases couldn't answer retrospective queries across multi-year windows without multi-minute query times, blocking the analysis workflows grid engineers depended on.
Rust Signal Processing Engine
Rewrote the COMTRADE parser and FFT pipeline in Rust. The engine runs wavelet decomposition and frequency-domain feature extraction at native speed, achieving sub-5ms processing for a typical 30-cycle fault record — a 160× reduction from the Python baseline.
Kafka-Backed Streaming Architecture
Deployed a Kafka cluster to decouple data ingestion from processing. Each relay posts records to a dedicated partition; consumers process in parallel with backpressure handled gracefully under burst conditions. No event drops under the target ingest rate.
TimescaleDB for Signal History
Migrated signal storage to TimescaleDB with automatic chunk compression and continuous aggregate materialized views. Retrospective queries that previously took 4 minutes now complete in under 200ms across multi-year windows.
React Operations Dashboard
Built a real-time operations dashboard for grid engineers — live fault event feed, waveform visualization, relay health status, and exportable fault reports — backed by the streaming API over WebSockets.
- Rust
- nom parser
- rustfft
- Wavelet crate
- C++ adapters
- Apache Kafka 3.x
- Kafka Streams
- AWS MSK
- TimescaleDB
- PostgreSQL 16
- S3 + Parquet
- React 19
- WebSockets
- Prometheus
- Grafana
- OpenTelemetry
Building something similar?
We've solved these problems before. Let's talk about yours.