How Reco's AI Rewrite Saved $500K/Year (And What It Really Took)
Everyone's talking about AI rewriting code, painting pictures of junior devs cranking out entire systems in an afternoon. Reco's story about `gnata` – a pure-Go JSONata 2.x implementation – seems to fit that narrative perfectly: 7 hours, $400 in tokens, $500K/year saved. This AI rewrite JSONata project is a headline grabber, sure. But if you've been in the trenches long enough, you know the devil is always in the details. This isn't just about an AI spitting out Go code; it's about a team that understood their problem, knew their tools, and did the hard engineering work around the AI.
The Old System: A Mess of Architectural Debt
The old system was a mess, a classic case of architectural debt accumulating. Reco was running `jsonata-js` in Node.js pods, hundreds of them, called via RPC from their Go services. That meant a constant drain of $300K a year just for compute, plus a 150-microsecond RPC latency hit on every single evaluation. That's 150 microseconds before the actual work even started. And the Kubernetes IP allocation limits? That's a clear sign you've scaled a bad pattern to its breaking point. This costly setup highlighted the urgent need for a solution like the AI rewrite JSONata project.
The Motivation for `gnata`: Cutting the Fat
So, the motivation for `gnata` was clear: cut the RPC, cut the JavaScript, cut the Node.js, and bring the evaluation logic directly into the Go services. This is where the AI came in. They ported the official `jsonata-js` test suite to Go – 1,778 tests – and then used an AI model to implement the evaluator until all those tests passed. That's a smart move. Using a comprehensive, existing test suite as the specification for an AI-driven port is the only way to ensure correctness without manually writing thousands of assertions. The AI generated 13,000 lines of Go code, and it cost them $400 in tokens. This initial AI rewrite JSONata effort was impressive, no doubt, setting the stage for significant savings.
Beyond the "Day" of AI Work: The Real Engineering Effort
Here's the thing, though: the "day" of AI work was just the start. The real engineering effort kicked in after that. The initial AI rewrite JSONata provided a strong foundation, but the subsequent optimization and validation required human ingenuity and deep technical understanding. This project wasn't just about the AI; it was about leveraging it effectively.
The `gnata` Architecture: Performance Gains Beyond the AI Rewrite JSONata
The `gnata` architecture itself is where the true performance gains come from, not just the language change. They built a two-tier evaluation system:
- Fast Path: This handles simple expressions – field lookups, comparisons, 21 built-in functions on pure paths. It evaluates directly against raw JSON bytes. No JSON parsing, zero heap allocations for simple cases like `account.status = "active"`. This is critical. Eliminating parsing overhead and allocations for the common case is how you get 1,000x speedups.
- Full Path: For everything else, it's a complete parser and evaluator, but it only parses the necessary subtrees of the JSON document. This is still 25-90x faster than the old RPC path, even with full parsing.
On top of that, they added a `StreamEvaluator`. This is for when you have multiple expressions to run against similar events. It merges all the field paths into a single scan, reading the raw event bytes only once. Think about that: one pass over the data for N expressions. That's efficiency. Evaluation plans are cached immutably, accessed via a lock-free hot path. This isn't just a straight port; it's a re-architecture for performance within the Go ecosystem, building on the initial AI rewrite JSONata foundation. The strategic implementation of `gnata` truly maximized the benefits of the AI rewrite JSONata approach.
The Rollout Process: Validating the AI Rewrite
The rollout process tells the real story of the "day's work":
The "day" was just step 2. The total project duration for the $500K savings was "under 2 weeks." That includes days of human code review, QA, and a shadow mode deployment processing billions of events to ensure correctness. They even found bugs in the reference `jsonata-js` implementation. And the AI code review? It flagged concurrency issues and cosmetic nitpicks in the AI-generated code. That's not a "set it and forget it" scenario; that's a skilled team using AI as a force multiplier, then rigorously validating its output. The success of the AI rewrite JSONata project hinged on this meticulous validation and ongoing human oversight.
Further Refactoring: Unlocking More Savings
The additional $200K in savings came from a further refactor of the rule engine internals, enabled by `gnata`'s performance. This wasn't AI-generated; it was engineers leveraging the new capabilities to optimize their system, replacing inefficient goroutine-based concurrency with just-in-time batching and short-lived caches. This demonstrates how the initial AI rewrite JSONata opened doors for deeper, human-driven optimizations, proving its long-term value.
The Real Takeaway: Engineers + AI, Not AI Replacing Engineers
So, what's the takeaway? This isn't a story about AI replacing engineers. It's a story about engineers using AI as a powerful tool to accelerate the initial, tedious parts of a complex porting project. The real win here is the engineering discipline – the comprehensive test suite, the shadow mode validation, the deep understanding of performance bottlenecks, and the subsequent architectural optimizations. The AI didn't save $500K/year; a smart engineering team, assisted by AI, did. Don't confuse a fast start with a finished product. The heavy lifting, the stability, the reliability – that still comes from battle-hardened engineers. The successful AI rewrite JSONata project at Reco is a testament to this synergy, showcasing the true potential when AI and human expertise combine.
