Real-time voting platform — race condition at scale
2021 – 2026 · NewCombinPrimary backend engineer on a high-traffic survey and voting platform built for major US broadcast networks. During live broadcast events — the exact moment traffic peaks — votes were silently disappearing.
What broke
A legacy DynamoDB-to-MySQL migration had left a race condition in the vote-write path. Under the concurrent write volume of a live broadcast, writes overlapped and votes were lost with no error, no alert — just a count that didn't add up after the show aired.
What I did
- Traced the loss to the migration's overlapping writes and rebuilt the path around Redis-based vote buffering, collapsing concurrent increments into atomic operations before they ever reached MySQL.
- Optimized the critical queries in the read/aggregation path and configured CPU-based EC2 autoscaling to absorb live-broadcast traffic spikes.
- Validated the fix with JMeter load tests at 30,000 votes/minute — zero data loss, a 5x increase in processing capacity over the previous ceiling.
- Removed the dead code left over from the original migration once the new path proved out in production, closing off a source of latent bugs.
- Built and extended the platform's social distribution integrations (Instagram, TikTok, Facebook, Vimeo) for direct video/image publishing from its media management platform.