Would that be a 'sidactor'? This variant works for AC limiting, should be OK with a polyfuse in series about not hurting a 24 supply
It has to be depowered completely to reset it, so the polyfuse will stay warm until unplugged. The breakover is built-in, rather coarse, so one hopes your nominal 12V box is OK with 15...