Physical keypad or touchscreen for a DIY calculator - which way would you go?

Buttons, all the way. I built something similar last winter with a 4x5 pad off a cheap scientific calculator housing, and half the fun was that clicky feel. I'd sit there doing quick math just becuase pressing the keys was satisfying. A touch screen never gave me that, and I found myself fat-fingered on it more than I expected, expecially without looking at the screen. The pin worry is smaller than it seems if you wire the pad as a matrix. My 20 keys only needed 9 connections, and the display took most of the rest. If you're realy squeezed, one of those I2C port expander chips gets you a whole extra bank of pins off two wires, though honestly I didn't need it.

I did try the hybrid thing you're describing, sort of by accident. First version had three physical keys for plus, minus, and equals, and everythign else was drawn on a touchscreen overlay. It was annoying in a way that's hard to describe. My thumb would hover over the buttons while my eyes were on the screen, and I kept tapping the wrong virtual key anyway. Ripped it out and went full physical.

One thing I'd suggest since your layout is fixed anyway: pick a key layout copied from a calculator you alredy like using. I modeled mine after an old Casio I've had since school and never once thought about where a key was. That muscle memory is the whole point.

I went touch on mine, mostly for a boring reason: the pins. My little build uses a small screen that shares its wires with the input side, so the whole thing costs me a handful of lines instead of the eight or so a matrix woud want. That freed up enugh to add a battery indicator and a little status LED, which I never would've had room for otherwise. And like you said, being able to reshuffle the layout whenever I get a new idea has kept the project fun instead of frozen.

The feel issue is real though, so here's what worked for me. I mounted the screen behind a piece of thin acrylic with a printed label sheet underneath, and the sheet has just enough flex that pressing gives a little click against the case. Not a real key, obviously, but way better than tapping bare glass, and my fat-finger rate dropped a lot once there was a firm surface to press against. If you ever want physical keys without burning pins, there are cheap touch controller chips that talk over just two wires and handle a whole pad for you. Or you coud try the ESP32's own touch sensing with foil pads under a faceplate, which costs basicly nothing to experiment with.

Honestly, give touch a weekend trial. Worst case you're out an afternoon and you go back to keys with no regrets.

I'm pretty new to these little boards so the wiring talk is already over my head, but the deciding factor for me might be speed. When I tap numbers on my phone I hit the wrong one maybe one time in five, so I can't imagine doing actual math on a screen. Still, touch does seem easier to set up for someone like me who keeps mixing up which wire goes where. Does tapping the screen react right away or is there a small lag before the number shows up?

I'd push back on the idea that copying an existing calculator's layout is the whole point. Mine has a custom arrangement because I kept hitting multiply when I wanted plus, and after maybe a week of use my hands stopped caring. Muscle memory builds around whatever you give it, it doesn't have to match some old Casio.

Also not sure the expander chip advice holds up in practice. I tried running my keypad through one and the scanning got laggy enough that fast typing dropped keypresses. Went back to burning the pins directly and it's been rock solid since. Depends on how you code the reading loop I guess, but it wasn't the free lunch it sounds like.