Re: "RESET"

Even the initial microprocessors have a reset pin. When they are powered up, the status of the electronics is unknown, so a small time after power up, the line is triggered by a timer (555 or whatever).

Then, there are many designs where you can not pull power, because there is an unreachable battery.

Then, it is impossible to guarantee that the device will never find itself in a pickle. No matter how fantastic the designers are.

Sometimes it is the user that is lacking. Such as forgetting a password. Maybe putting the device in the wrong mode and can not remember how to get it out of that mode.

With the many lines of code it is impossiable to check out every possiale combination of things that could ge wrong in a reasonable time.

While not really much of a chance but in the eairly days bits could be randomally changed by stray radiation.

Ususally the "reset" button is really "factory reset", which is intended to revert persistent state back to know values. A typical use for this, that absolutely must not be done on a power cycle, is setting a login password to a known value ("1234", or some more secure unique string printed on a label if the designer is sensible).

As a convenience (for who?), a quick press of the reset button usually just does a POR, and you need to hold the button for some length of time to reset the configuration. I think this has just become a convention that every device seems to follow.

Yes, a lot of complex devices that really should be able to run forever sometimes can't, and lock up. I blame lack of understanding of the whole system by the programmers (and it is usually software that is at fault, rarely the hardware). It's going to get worse with completely clueless people throwing together something that barely works with the help of automated idiots.

I write microcontroller code in assembler that runs 24x7x365x∞ (that's "infinity", for the UTF8-challenged :)

Nowadays 'reset' often means 'reset to factory settings' rather than 'reboot'. The factory settings reset is needed because maybe you forgot the password and have no other way to reconfigure the thing. Or you need to make it go back into the initial pairing mode so you can attach it to another network/etc.

Having a physical factory reset button means that somebody with physical access can always regain access to it. You can also use it to verify destructive actions (eg 'to wipe all the data, now hold the button') so that they can't be done remotely.

In practical terms such a button might just be a GPIO rather than wired to a reset line, and the software pays attention to it at certain times such as during boot.

For more developer-focused devices, a true reset button is also better than yanking the power cord which can cause wear on the connectors. So when version 497 of your code crashes you can hit the button and upload v498.

Theo

There are things like routers and mobile phones where there is a very significant difference between power on/off and a hard factory reset. The former recovers it form having crashed internally whilst the latter trashes all previous settings into oblivion.

Not always available - there are quite a few different levels of reset too. I recall one particularly annoying one on an early Android device that require holding the on/off button and volume down in for 4 minutes. It did do a hard factory reset on about the fifth attempt. The previous four having failed because my fingers slipped. One minor annoyance was that the very hard reset put it into Chinese language mode.

ISTR the ordinary soft factory reset was about 10s holding the magic buttons in (but didn't work on this unit).

In an ideal world yes. But I have seen such devices in a state where the only process still running was the one pressing the dead man's handle to say that everything is OK. Many routers tend to go haywire after a continuous uptime of about 2 or 3 months having fragmented their stack.

It could also be to reset to a known state. Several of the more annoying gadgets have one time programmability from a PC but to make them secure the moment you make them active all further communication is impossible.

The only way to reprogram is factory reset and start again from scratch. User interface designed by someone who really enjoyed the maze in Zork. (some TV tuning menus fall into this category)

...

Think phones.

Should.

Actually, nobody warranties this. Oh, wait, I remember a company that did warranty it, for limited sized code, and hugely expensive. They did the core software on ATMs or the servers behind them, I don't remember.

...

And today. It is even a bigger chance, with lower logic voltages. Chips are not shielded.

Some designs preserve the status on power cycle. Intentionally.

...

Wow! :-o

Exactly. I recall a customer wanting us to verify all possible paths through a bit of air traffic control radar software, about 100,000 lines of plain C. Roughly one in five executable line was an IF statement, which is 20,000 IF statements. So there are 2^20000 =

The testing campaign will have only scratched the surface when the Sun runs out of hydrogen and goes supernova. Tomorrow's problem.

Joe

The problem is that you have no way to know which cases are irrelevant. And practical hardware will have many things able to retain state.

It is not usually the expected that causes trouble: It ain't what you don't know that matters, it what you know that ain't so that's the problem.

So assume only 1000 IF statements, so it's 2^1000 or 10^600 or so. You'll still run out of lifetime.

You do all the tests for required behavior - does it meet stated requirements.

Then you random probe it for weeks and see what goes Bang!

One form of this is Fuzzing.

.

Joe

Executing each path in the code at least once is a much more tractable problem. McCabes CCI metric will tell you how many test vectors it will take for a given complexity of code. And it should be done.

A sufficiently high CCI index for a routine also means that such code is highly unlikely to be correct.

I recall one instance on a mainframe (brand withheld to protect the guilty) where a rogue program that ran continuously was slowly using up IO handles repeatedly opening the tracker ball interface each time a new user accessed it (and never letting go).

One day after a particularly long uptime it completely ran out of IO handles. Guess what the first thing the error handler tried to do?

Yup! It tried to obtain a new IO handle to report the error!

You can't out test every possible combination of events but you can make sure that the code paths when executed in at least one scenario don't do anything horribly bad. A lot of faults can lurk in the rarely used error recovery code that only gets used after something else has gone wrong.

Ariane 5 was an example of that sort of thing. comp.risks is littered with them.

A concept you are looking for here is "cyclomatic complexity" :

Conditionals are not independent (in most cases) - thus the number of paths through code is not simply two to the power of the number of if statements.

(McCabe cyclomatic complexity measures are not the only way to look at this, and like anything else, the measure has its advantages and disadvantages, and is not suitable for everything. But it's a reasonable place to start.)

The way you handle complexity in software is exactly the same as any other complexity - you break things down into manageable parts. For software, that can be libraries, modules, files, classes, functions, and blocks. You specify things from the top down, and test them from the bottom up. How much testing you do, and how you do it, is going to depend on the application - an air traffic control system will need more thorough testing than a mobile phone game.

When you are looking at a function, the cyclomatic complexity can be calculated by tools. It will then give you a good idea of how much testing you need for the function. Too high a complexity, and you will never be able to test the function to get a solid idea of its correctness, as you would need too many test cases for practicality. (You may still be able to use other analysis methods.) The complexity can also give a good indication of how easy it is for humans to understand the function and judge its correctness. (Again, no one measure gives a complete picture.)

Other tools that can be useful in testing are code coverage tools - you can check that your test setups check all paths through the code.

But remember that testing cannot prove the absence of bugs - only their presence. And it only works on the assumption that the hardware is correct - even when the software is perfect, you might still need that reset button or a watchdog!

True, but it's the bounding case.

I recall those days. Some managers thought that if they decreed that no module could have a complexity (computed in various ways) exceeding some arbitrary limit. The problem was that real-world problems are vastly more complex, causing atomization of the inherent complexity into a bazillion tiny modules, hiding the structure and imposing large added processing overheads from traversing all those inter-module interfaces.

A parallel panacea was to mandate that the flowchart of be strictly planar, with no crossovers allowed. The hope was that with such a restriction it would be possible to create code correctness verification software. This also failed, because it's actually impossible to do (see the halting problem) and because the attempt is actually computationally intractable with non-toy examples.

The problem with any generalisation and rules is that they are sometimes inappropriate. /Most/ functions, modules, pages of schematic diagram, or whatever, should have a low complexity however you compute it. But there are always some that are exceptions, where the code is clearer despite being "complex" according to the metrics you use.

A false alarm for a code coverage report would mean code that is not reported as hit, but actually /is/ hit when the code is run. How does that come about?

But it is certainly true that any kind of automatic testing or verification is only going to get you so far - false hits or missed cases are inevitable.

And it's still true - modern systems give more scope for hardware issues than simpler systems (as well as more scope for subtle software bugs). A cosmic ray in the wrong place can render all your software verification void.

No, all of the complexity metrics were blown away by practical software running on practical hardware. Very few modules were that simple, because too many too small modules carry large inter-module interface overheads.

The code coverage vendors hold the details close, so we usually don't know how hits are declared, and probably never will.

The one that I did manage to obtain the details turned out to be looking for certain combinations of certain words and arrangements. It had zero understanding of what the code did, never mind why.

Maybe modern AI will do better, but may be too expensive to make business sense.

Yes, in fact the false hits dominate by a large factor, and the main expense in using such tools is the human effort needed to extract those few true hits.

I must say that there was much worry about cosmic ray hits back in the day, but they never turned out to matter in practice, except in space systems.

The dominant source of errors turned out to be electrical cross-talk and interference in the backplanes, and meta-stability in interfaces between logic clock domains in the larger hardware system.

I vaguely recall doing an analysis on this issue, some decades ago.

Joe

That changes nothing of the principles.

You aim for low and controlled complexity, at all levels, so that you can realistically test, verify, and check the code and systems at the different levels. (Checking can be automatic, manual, human code reviews, code coverage tools, etc., - usually in combination.) Any part with particularly high complexity is going to take more specialised testing and checking - that costs more time and money, and is higher risk. Sometimes it is still the right choice, because alternatives are worse (such as the "too many small modules" issues you mention) or because there are clear and reliable ways to test dues to particular patterns (as you might get in a very large "dispatch" function).

You don't just throw your hands in the air and say it's better with spaghetti in a module than spaghetti between modules, and therefore you can ignore complexity! I don't believe that is what you are actually doing, but it sounds a little like that.

Do the gcc and gcov developers hold their details secret? I'm sure there are many good reasons for picking different code coverage tools, and I'm not suggesting that gcov is in any way the "best" (for many reasons, code coverage tools would be of very limited use for most of my work). And there are all sorts of different coverage metrics. But it would surprise me if major vendors keep information about the prime purpose of the tool a secret. Who would buy a coverage tool that doesn't tell you what it measures?

I am not sure what kind of tool you are referring to here. Code coverage tools track metrics about the functions, blocks and code lines that are run. Different tools (or options) track different metrics - counts, times, or just "at least once". They might track things at different levels. Some are intrusive and accurate, others are non-intrusive but statistical based. If you are wanting to use code coverage tools in combination with branch testing, you just want to know that during your test suite runs, every branch is tested at least once in each direction.

We can pretty much guarantee that commercial vendors will add claims of AI to their tools and charge more for them. Whether or not they will be better for it, is another matter.

I would expect AI to be more useful in the context of static error checkers, simulators, and fuzz testers rather than code coverage at run-time.

Just to be clear - are you using non-intrusive statistical code coverage tools (i.e., a background thread, timer, etc., that samples the program counter of running code? Or are you using a tool that does instrumentation when compiling? I'm trying to get an understanding of the kinds of "false hits" you are seeing.

I guess there are many factors for that. If something weird happens, and you have no explanation and it never happens again, then you can easily say it was probably a cosmic ray - without any direct evidence. It is also the case that many systems or subsystems are tolerant of an occasional single-event upset - be it from cosmic rays or anything else. If you have ECC memory, or other kinds of redundancy or error checking, rare errors there are not an issue. So many types of memory, buses, and communication protocols are effectively immune to such things. However, critical parts of the system will still be vulnerable to hardware glitches. It is not without justification that safety-critical electronics often has two cores running in lockstep or other types of redundancy.

Sure. Cosmic rays were only an example (pulled out of thin air :.) ). Glitches on power lines, unlucky coincidences on bit patterns, production flaws or ESD damage eroding electrical tolerances - there are lots of possibilities. I'm not trying to suggest relative likelihoods here, as that will be highly variable.

I recall something of the opposite - a long time ago, we had to add a variety of "safety" features to a product to fulfil a customer's safety / reliability checklist, without regard to how realistic the failure scenarios were and without spending time and money on analysis. The result was, IMHO, lower reliability because it was more likely for the extra monitoring and checking hardware and software to fail than for the original functional stuff to fail. Many of these extra checks were in themselves impossible to test.

In theory, sure. In practice, it didn't help enough to make it worthwhile.

Peer review of the code works better, because no pattern scanning tool can tell spaghetti from inherent complexity.

And this goes double for operating system kernel code, which violate essentially all of the coding standards developed for user-level application code.

I was dealing with a proprietary code coverage tool that management was quite enamored with and so was pressuring us to use. But we had only a sales brochure to go from, and I point-blank refused to use it without knowing what it did and how. This caused a copy of the requirements document of the scanner to appear.

I don't think gcov existed then. We used gcc, so the software folk would have used it were it both available and mature enough.

It's the proprietary coverage tool mentioned above.

Yep. The longer-term outlook is that this proprietary tool is just one of a stable of coverage tools that are used to generate lists for manual disposition by the authors of the code being scanned.

Yes. Don't forget Quantum.

Why? I would think that a LLM could follow the thread far better than any static checker.

The US financial firm Morgan Stanley is using AI to analyze and summarize nine million lines of code (in languages such as COBOL) for re-implementation in modern languages. This from The Wall Street Journal, 3 June 2025 issue:

"Morgan Stanley is now aiming artificial intelligence at one of enterprise software's biggest pain points, and one it said Big Tech hasn't quite nailed yet: helping rewrite old, outdated code into modern coding languages.

In January, the company rolled out a tool known as DevGen.AI, built in-house on OpenAI's GPT models. It can translate legacy code from languages like COBOL into plain English specs that developers can then use to rewrite it. 

So far this year it's reviewed nine million lines of code, saving developers 280,000 hours, said Mike Pizzi, Morgan Stanley's global head of technology and operations."

What happened is that semiconductor technology progressed to the point that the amount of charge (or whatever) that distinguished symbols became very small and thus vulnerable to random errors, for which an error-correcting code had to be built in. At this point, cosmic rays were lost in the random noise, so to speak. So ECC is now inherent, not an extra-cost bolt-on.

And this is still true.

Yes. I recall directly testing the issue with ECC as implemented in early DEC VAX computers, in the early 1980s. We had a customer who specified ECC, so we had ECC. And soon discovered that the computer was more reliable with ECC disabled than enabled. That was the end of ECC.

Joe

OK.

That's certainly true in some cases. It surprises me a little that your experience was so much like that, but of course experiences differ. My experience (and I freely admit I haven't used complexity analysis tools much) is that most functions can be relatively low complexity - the inherently high complexity stuff is only a small proportion of the code. In one situation where this was not the case, I asked the programmer to re-structure the whole thing - the code was badly designed from the start and had become an incomprehensible mess. Peer review did not help, because the peer (me) couldn't figure out what was going on in the code.

However, it is entirely true that some code will be marked as very high complexity by tools and yet easily and simply understood by human reviewers. If that is happening a lot in a code base, automatic tools (at least the ones you are trying) are not going to be much use.

Different code has different needs and standards, yes.

No software tool can fix management problems :-(

Fair enough. I haven't done anything significant with gcov, so I can't say how good it might be. (It is very difficult to use tools that write data to files when you are working on small microcontrollers with no filesystem and at most a small RTOS.)

We are already into post-quantum algorithms, at least in some fields!

I mean that I think there is more potential for adding useful AI algorithms to static checkers and simulators than there is for using AI algorithms in run-time code coverage tools. But that's just a guess, not backed up by any evidence.

I can see AI being a help here - just as many existing tools can be helpful for figuring out what old code does. I am not holding my breath waiting for AI to manage such conversions on its own.

For very dense and small feature size electronics, that is mostly true - though even then there are parts that are vulnerable. It's just that those parts are a tiny proportion of the die size, compared to memory arrays, and the like.

Quis custodiet ipsos custodes?

Sometimes these fault monitors and error checking systems are just kicking the can further down the road, and not actually improving anything.