bit flips?

I did wonder, a few days ago, how a software update could fix this problem.

It doesn't make a lot of sense. Critical data should be error-detection/correction coded as it is being stored, and checked and checked for errors - and if possible - corrected to eliminate the errors when it is read out.

We were proposing to do that in our shaped-beam electron beam microfabricator back in 1986, and it wouldn't have been a big deal back then. With big FPGA's it would be trivial.

I have written a VHDL library with bits, bit_vectors, signed, unsigned etc that look like standard_logic types but are triple module redundant with voting internally (stuff for the ISS). ISS is still relatively harmless; there are some people living next door after all.

Most of the work was to prevent the optimizer to reduce this to a single version. Throwing 2/3 plus voting away is a huge temptation for the optimizer, and it does not give in easily. I solved that with signals like "always_high_but_don't_tell_the_optimizer" that came from an external pin with an external pull up etc. Maybe I should rewrite that in present-day VHDL for open source.

We used ST radiation hardened voltage regulators that still could go berserk for some ms after a SingleEventUpset if it hit the regulation amplifier. At least they would not latch up. The proposed cure was huge output capacitors that would limit the voltage rise for some ms of berserk mode.

BTW the caps from the qualified parts list were tantalums, but with abt. 6 times the volume of similar commercial ones.

Ah, and I was not qualified to solder my own boards.

Cheers, Gerhard

It probably is being ecc checked. However, an energetic neutron or a cascade of charged particles might cause multiple bit errors which would not necessarily get detected or corrected. The fix could be as simple as doing memory scrubs far more frequently than usual to reduce the probability of multi-bit errors accumulating to the point where they are not detected.

John

If does if the driver software for the whatever sensor was affected makes naive assumptions that the answer it gets is always correct. Simply ignoring any single reading that is physically unreasonable and reading it again ASAP will fix most radiation or cosmic ray induced read faults at altitude.

They will have ECC in all the main systems and know how bad things are from the frequency of soft correctable errors. I expect they advise flying lower at a threshold where detected double errors start to rise.

What surprised me was more that it wasn't a trans polar flight that got badly affected rather than one over the continental USA. I used to do those a lot and we sometimes did have to fly lower and slower at sunspot max. Mainly for the cabin crew's benefit so that they wouldn't accumulate a radiation dose that limited the routes they could fly on.

It is pretty much solar maximum now so it is a severe test of fly by wire electronics and sensors tolerance of energetic particles.

The Airbus software was hardened against bit flipping errors for years and years. End October they rolled out a routine update but it missed out the checksum or redundancy checks - remarkably soon afterwards a plane experienced a glitch so they are for time being reinstalling the previous software version. Beware software updates!

That's pretty improbable. If there's a big enough burst of radiation to completely wipe out the error correction, you still get error detection.

They are always detected, and if there are mostly single bit errors they get detected and corrected whenever the memory is read.

Dynamic RAM has to be refreshed pretty frequently, and modern processors seem to be designed to cope with that. A "memory scrub" is not something I've ever had to design in.

More likely it was a software bug. After all, there was a software fix.

John Larkin Highland Tech Glen Canyon Design Center Lunatic Fringe Electronics

Some PC BIOSes have it as an option. John

There is a 20 years old xilinx app note on this.

Cheers