6.8x Faster, AI That Helped, and Why VBA Needs to Die

I spent a weekend doing something deeply specific, and I need to tell someone about it. This is about v-star — my actuarial engine in Go — and what happened in a few days of focused work. Some of it is technical. Some of it is philosophical. All of it is true enough.

The Performance Story First

I improved CSV parsing throughput by 6.8x. Let me say that again because I still don't fully believe it: from roughly 1.5 million rows per second to about 10.7 million. The parallel version hits 11 million. Don't tell anyone but I got a server and it hit 38 million.

For context: a "row" is one record in a CSV file. One policy. One customer. One calculation waiting to happen. When you're processing millions of policies, going from 1.5M to 10.7M rows per second changes everything.

Four optimizations got us there:

Streaming processor — Read and process one line at a time instead of loading the entire file into memory. Less RAM, faster results.

Pre-allocated slices — Count the fields first, create exactly the right-sized container. No wasted memory, no resizing overhead.

String interning — Go's unsafe package creates views into existing memory instead of copying bytes to new strings. Same data, no copy. Massive savings at millions of rows.

sync.Pool — Reuse temporary arrays instead of creating new ones for every row. Less garbage collection pressure.

I also ran benchmarks against Python with Pandas and Polars. I beat Pandas handily. Polars is a hit-or-miss — sometimes I win, sometimes I lose. It's early to say. The point isn't that Python is bad. The point is what you can achieve when you optimize carefully and leverage Go's strengths.

AI Actually Helped

I was skeptical about using AI for performance optimization. Code generation, explanations, debugging — sure. But optimization felt like it required human intuition, profiler data, and hours of staring at flame graphs.

Then I fed the CSV parser code to an AI and asked it to find unnecessary memory allocations. And it found things. A lot of things.

It found byte-to-string conversions in tight loops creating garbage on every iteration. It noticed the sync.Pool was missing entirely. It spotted append operations without capacity pre-allocation. It pattern-matched across the codebase and connected dots I would have found eventually — but not quickly.

Is this cheating? It's called embracing the times. Using AI to find performance issues is the same as using a profiler — a tool that surfaces information you'd find eventually, but faster. The expertise is in knowing what to fix and why it matters. The AI just helped me find it.

After implementing the fixes: 6.8x faster. That's not hallucination. That's measurable, reproducible, objective improvement.

The Other Stuff That Happened

I also added four core actuarial packages in that same stretch: interest rate calculations, mortality tables, annuity valuations, and reserve calculations. The mathematical backbone of any actuarial tool. Fixed a bug in CSV parsing where px columns weren't being detected correctly — small detail, but parsing errors flow through everything downstream. Added a --table flag to the CLI. Updated the README, ROADMAP, and LICENSE for v0.2.0.

Version numbers feel ambitious for something with maybe a dozen users total. But there's something satisfying about formalizing progress. It forces you to look at what you've done and call it "a release."

Why VBA Needs to Die (And Why It Won't)

Let me talk about something that actually matters.

Visual Basic for Applications is a disaster. It's the programming language embedded in Microsoft Excel. It's been around since 1993. Over 30 years of accumulated technical debt in every insurance company, pension fund, and actuarial firm on the planet. And most actuarial work is still done in VBA and Excel. Not because it's good. Because it's what everyone knows.

The specific problems:

No type safety. Pass a string where a number belongs and Excel will try its best. Sometimes it works. Sometimes it returns "type mismatch" at 2 AM before a deadline. Sometimes it silently does the wrong thing.

No version control. Excel files don't diff well. VBA code stored in Excel modules is invisible to Git. You can't review changes, branch, or merge. Every "improvement" is a new file.

Single-threaded. Want to process a million policies? Hope you like waiting.

Fragile. Move a cell? Formula breaks. Rename a sheet? Formula breaks. Change a column order? Formula breaks. The dependencies are invisible and impossible to track.

"It works on my machine." Different Excel versions, regional settings, security settings — code that runs perfectly on your machine fails on your client's. No way to test until production. This is the exact reason I wrote v-star in Go with no dependencies.

VBA persists because the institutional knowledge is decades deep, regulators accept it, universities still teach it, and there's no catastrophic failure to force change. When VBA breaks, it breaks quietly. Wrong numbers get caught in review. Nobody dies.

That's why I'm building v-star. Not to replace every spreadsheet — that's unrealistic. But to show that another way exists. Code that's version-controlled, typed, concurrent, and fast. It's possible. You just have to choose to do it.

Who This Is Actually For

I've been honest with myself about this. v-star probably has a tiny audience. Actuarial science is a niche field. High-performance actuarial software in Go is a niche within a niche.

Maybe it's for students learning actuarial math who want to see how formulas translate to real code. Maybe it's for actuaries tired of waiting 20 minutes for a valuation. Maybe it's just for me — a project that scratches an itch and teaches me Go deeply.

The goal has shifted as I've worked on it. It's no longer about replacing Excel models. It's simpler than that: make something that exists. Something that's fast. Something that proves you can build actuarial software without VBA, without the traditional ecosystem. Something that demonstrates it's possible, even if nobody ever uses it.

There's a strange satisfaction in that. Building something just to prove it can be built. Learning things you'd never learn from tutorials because you're solving problems that actually matter.

Repo is at github.com/lubasinkal/v-star. MIT licensed. Use it if it helps.