Why does my homemade USB device work for a few seconds then freeze?

You're right that it's a clock problem, but it's probably not the code. The pattern you're describing, where it runs fine for a few seconds and then gets sluggish and dies, is exactly what happens when a clone board has a ceramic resonator instead of a proper crystal. Look at the little silver metal can near the USB chip versus a small tan blob about the size of a grain of rice. A lot of cheap clones ship with the blob, and those drift as they warm up. Bit-banged USB needs the timing to stay within a couple percent at all times, so a board that passes the handshake at room temp will slowly fall out of spec as it heats, wich matches your symptoms perfectly. It also explains why swapping boards changed the runtime, since each resonator drifts a bit differently. Two ways out of it. The easy one is to sort thru your boards and find one with an actual crystal, they do exist on some clones, and the timing will hold. The other one is to use the clock calibration feature built into the V-USB library. You have to enable it in the config file, it's the option about measuring the frame length, and then the code periodically re-tunes the internal oscillator against the incoming USB frames every millisecond. I've run boards on junky resonators that way and they stayed connected for hours instead of seconds. One other thing worth checking before you blame the chip entirely. If both your boards are being fed through that cut cable and a couple of diodes, a sagging supply can look a lot like this too. But the warm-up behavior and the board-to-board variance point strongly at the clock, so I'd start there.

If none of your nanos have a crystal, it might honestly be cheaper to grab a couple that do than to spend another weekend fighting calibration.

Been down this exact rabbit hole on a similar setup, and the power angle in that last paragraph is the one I'd chase first. When you feed the chip through a cut cable with diodes in series you're dropping roughly 0.7 volts per diode, and on mine I measured the rail sitting at 4.1 volts. It ran fine cold, then the regulator warmth pushed it just under the brownout threshold and everything went sideways. Lifted one diode out of the chain and the problem vanished overnight. Cheap to test too, just probe the supply pins after a few minutes of running and watch wheather the number is sliding.

Easy way to confirm the heat theory before you buy anything: hold a cold soda can against the chip for a few seconds when it starts acting up. If it snaps back to normal, that settles it. I did the same trick back when I was bit-banging PS/2 keyboards off a bare chip, saved me a lot of guessing. One thing in your favor is you allready stripped the slow calls out. Just make sure you never turn off interrupts for more than a few cycles anywhere in your code, the USB side needs constant attention or the host gives up on it. And for what it's worth, folks were soldering straight onto cables long before these little boards existed. Your weekends are not wasted, you're just a component swap away.

This thread is making me look at my own boards differently. I use one of the cheap clones for a temperture logger in the garage and it has never hiccuped, but I guess logging once a second is way more forgiving than your setup. Now I'm curious whether mine has the tan blob or the metal can, will have a look with a magnifier tonight.

Question for you since I'm still pretty new here: when you enable that calibration option in the config, does it just work after a recompile, or are there other settings that have to match your board? I've broken sketches before by changing one line I didn't fully understand.

I had a receiver doing the same warm-up fade a couple years back and chasing code settings got me nowhere. What actually fixed it was pulling a crystal out of a dead computer mouse in my junk drawer and swapping it onto the board where the tan blob sat. Ten minutes with a soldering iron and the thing ran all weekend after that. If you've got old broken peripherals lying around, there's ussually a usable one inside, and it beats waiting on parts in the mail.

Update: tried the cold soda can trick and the thing snapped back to life instantly, so it's overheating, not the clock. Turns out my soldering job on the cable was dumping way too much current through the little board. Cleaned that up and it's been running solid for an hour now. Weekend not wasted after all.