LLM bill parsing, from prompts to evals

Ivy Energy · solar billing · tech lead and de facto PM

The problem

Ivy Energy does solar billing. I owned the billing pipeline end to end: utility data intake, energy accounting, and bill generation. As we grew, input data quality became the main limit on billing accuracy and on expanding nationwide. Utility bills are complex and wildly inconsistent from one utility to the next, and every number we pulled from them ended up on a bill someone paid.

What I built

I saw the need and took the project from vision to production as both PM and tech lead. It started as solo R&D: an LLM + OCR pipeline to extract charges, rates, and assistance programs from those bills. I drove the product planning and led it into production.

Making it trustworthy

The first version ran on prompts and schemas defined in a UI. I moved it to labeled eval sets, so every change was measured against known-correct bills, and then to DSPy optimization on top of those evals.

The result was consistently more accurate than our legacy vendor on disputed bills: the hard cases, where getting it wrong cost real money and trust.

What I took from it

The model was rarely the bottleneck. Measurement was. Once there was a labeled set that reflected the real distribution of bills, improvements became straightforward to make and easy to trust. I've carried that into my current work on coding-agent harnesses, where evals do the same job.