Is it normal for AI-written CadQuery models to print with every hole slightly off?

You're missing the easy win here, and it's not your measuring. The AI is inventing hole positions from its training data, and for a generic ESP32 dev board there are half a dozen layouts with diferent spacing, so it's basially guessing. Never let it guess. Either find the mechanical drawing for your exact board (most vendors publish one with hole positions) or measure it once and hard-code the numbers as named variables at the top of the script. Then every revision uses the same constants and you're only fixing one thing at a time. For measuring standoff holes with cheap calipers: don't try to hit the center of the hole, that's where people go wrong. Measure from the edge of the board to the near edge of the hole, then add half the hole diameter. Do that from two adjacent edges and you've got your X and Y for that hole. Center-to-center between two holes is even easier, just jaw to jaw across the outside and subtract the diameter once. Even a $15 caliper is good to a tenth of a millimeter, way tighter than your printer. Before you print another full lid, slice just the flat base with the holes and print only that, or even a single layer of it. Two minutes of filament instead of hours, and you can drop the actual board on it to check alignment. Iterate on that until it's dead on, then add walls and the rest. Also worth asking the AI to generate a top-down 2D view of the model with dimension lines, or export the model and open it in a free CAD package to measure the hole spacing directly instead of eyeballing the print. One more gotcha: check whether the holes are positioned relative to the box center or a corner in the code. If the AI mixed those up between revisions, that alone explains a consistent 1mm-ish offset that moves around between attempts.

Another way out of the loop: don't print holes at all on early versions. Print the lid with just small pilot dimples or nothing, then hold the board on top, mark through the holes with a fine marker, and drill. I've done that on a couple of cases and it beats chasing half a millimeter through code. A hand drill with a bit matched to your screw size is plenty accurate for a box like this.

Once you've got a lid that actually fits, you can measure the drilled positions and feed those numbers back into the script so the final printed version comes out right.

Been doing boxes since I drew them on graph paper and cut them out of ABS sheet, and the measuring part never changes. One thing that saved me back then and still works: lay the board on a piece of paper, poke a pin through the holes, then measure pin to pin. Takes the fumbling out of it and your hands stop getting in the way. Also, keep a notebook of the numbers once you get them right. Half my old builds live on because I wrote the dims down on the inside of a folder somewhere, not because I remembered anything.

Been in a similar loop, except my problem ended up being hole size, not position. Made the holes the same size as the screw threads so nothing whould actually fit, even when the spots were right. Give them a little extra room, the screw head covers the slop anyway and the board can shift a hair if it needs to. Also, check the sliced model in your slicer preview before committing to a print. You can usally see the spacing there and catch a dumb number mistake in seconds. Saved me more than once when I fat-fingered a variable and didn't notice for three prints.

I had the same kind of loop but with a bracket for a doorbell chime, and what finally clicked for me was asking the AI to print out the coordinates it used as a comment list at the end. Seeing them written out made it obvious one pair of numbers was just slightly wrong, instead of me squinting at a failed print trying to figure out which hole moved.

Sort of related question though, is it normal for these tools to accept a photo of the board and mark the hole spots on it? I tried that once and got positions that looked plausible but I have no idea if I can trust it or if that's just a prettier way of guessing.