What battery is an R4 supposed to be?

That R4 marking is almost certainly a mold number, not a battery code. I've pulled apart a bunch of those cheap light-up resin pieces over the years and the letters stamped inside the door are usally just the part identifier from the factory that made the housing. Same mold gets used across a whole product line, so it tells you nothing about what goes in the compartment.

What you're describing, one spring contact and a slot about the width of a watch battery, is the classic setup for stacked LR44 button cells. Tons of these decorations run on a shrink-wrapped stack of three of them, sometimes four, and the single spring presses the whole column against the contact. The molded slot is sized for the stack, which is why it looks a bit longer than a single cell. If you can, measure the opening. An LR44 is about 11.6mm across, so a stack of three is going to be roughly three cells tall plus a hair for the wrap. Since yours is five or six years old, the original cells are almost certainly dead and may have leaked a bit, so clean the contacts with a cotton swab and some vinegar or isopropyl alcohol before you install anything new. Corrosion on that single spring contact is the number one reason these things refuse to light up even with fresh batteries.

You can buy a three-pack of LR44s and just drop them in one at a time, they'll stack fine against the spring. Polarity-wise, button cells go in with the flat positive side facing whichever direction the contact points, and if you look inside the compartment there's usually a tiny stamped diagram showing the orientation. If it doesn't light, flip the stack, no harm done. One tip: if you want them to last more than one season, grab the 357 silver oxide version instead of the alkaline LR44s. Same size, same voltage, way better shelf life, which matters when the decoration sits in a box for eleven months of the year.

I had the same mystery with one of those little snow globes last year, the markings inside the door meant nothing at all. I just took the whole thing to the grocery store battery aisle and held the coin cells up against the opening til one matched. Took two minutes. The stack of little round ones was the winner for mine too. If you open it up and find white crusty stuff on the metal piece, does that mean it's dead for good or can it still be saved?

One thing I'd add is to check whether the original stack had a spacer in it. I've opened a few of these where the factory used two real cells plus a dummy steel disc to fill the column, and if you toss the spacer the spring sits slack and the contact pressure gets flaky. If you find a plain metal disc with no markings down in the compartment, that's what it is, just put it back in the stack. Also worth wiggling the spring a bit, on older ones they lose tension and a fresh stack just sits there doing nothing until you bend it back toward the contact.

Agree with the point about the spring needing a snug column. Mine had the same single-spring setup and the loose cells would rattle and cut out whenever the thing got bumped, even with fresh ones in. I wrapped the stack in a single layer of thin electrical tape and it's been solid ever since. Just leave the ends of the tape exposed so the flat sides still touch the contacts, and don't wrap so much that the column won't fit back under the spring. Also check the spring itself for tension, a couple of mine had gone soft from years of sitting compressed in storage.

One thing I learned the hard way swapping cells in these holiday doodads: don't measure the old dead ones if you're checking sizes. They swell up a little as they die and you'll end up buying the next size up by mistake. Lay a fresh cell on a piece of paper, trace around it with a pencil, and hold that up to the opening instead. Same trick I used back in the day for figuring out bulb sizes on those old battery operated window candles. Works fine and costs nothing.