Hi, We currently do most (all) our embedeed FW testing on either real HW (if available), using FPGA, or using HW simulator and Verilog models. We are looking at performing more testing on the "Host" rather than the target, and since this is new to our group, I am looking for suggestions and comments. BTW, we use C, currently we don't use an RTOS, but going forward, this is something that we will be looking at Specifically: Do you use a Unit Test Framework, and does it have any features specific to embedded systems? Thanks, Steven
I use a unit test framework (cppunit), but it gets used on all of the code that doesn't depend on hardware, or hardware behavior. So there's lots of code that gets "unit tested" by carefully going through and making sure the system works correctly.
I've never tried it, but I've heard good things about c-unit (cunit?).
The challenging if you're going to test anything that depends on the hardware is making a good behavioral model of that hardware: if your code needs to wiggle pin A, see a number appear in register B, then wiggle pin C, you're going to have to do a lot of work in the hardware abstraction
-- and then your unit tests are only as valid as your hardware simulation. That, in a nutshell, is why I don't do more testing on the hardware.
--
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com
I've used different frameworks for different languages. For C, I've mostly used DejaGNU which is more of a regression test system than about unit testing. Some approaches to unit testing tie in with OOP and therefore don't really fit C that well.
I think test-driven development isn't a good match for C if that's what you are considering. TDD emerged from languages like Smalltalk which had no static type system but lots of runtime error checks. So most coding errors would cause a runtime exception that would crash the program, and you'd write enough unit tests to exercise all parts of the code, which would usually trigger most of the errors. C has slightly more compile time checks but no runtime checks, so errors that get through the compiler also have a good chance of not getting noticed by simple testing (another aspect of TDD philosophy is to write lots of small tests that are very simple).
If you're doing something serious in C, you have to rely more heavily on clean design, careful code reviews, and (lately) fancy static analysis.
I don't see why you claim that; the problem isn't in the way the program is structured, it is in the way the execution environment allows silent but deadly errors to go unnoticed. And that's just as possible with C++ (ugh) or Objective-C (i.e. a respectable combination of C and Smalltalk)
... but which is strongly typed, unlike C. In Smalltalk and Java you simply can't pretend that a Car is a Person (i.e cast a Car into a Person).
TDD became really practical with languages like Java, which has both strong run-time typing and a decent type system.
I don't know where you get that idea. For a start, in Smalltalk the concept if a "program" is a bit vague - you have your code being edited and compiled in an "image", and executed by a VM.
Unless there's a VM error, the nearest to a "crash" is either - a "does not understand" message being caught and handled by the code in the image being executed by the VM, or - your code simply not working as expected
In other words, the VM+image continues operating normally, without error - completely unlike C, of course.
Smalltalk is neither better nor worse than any other language in that respect, since it is dictated by budget and skill.
While both are true, the parentheses aren't related to the preceding sentence, of course.
And most especially a heavily constrained subset of the language and library that don't prevent static analysis. Same is true for ADA.
I'm talking about the pattern where you want to test some code that talks to the external world, but instead of making i/o calls directly, it interacts with an "external world" object that you pass into the code (Java applets always work that way). So you can implement both a live object that does the real i/o, and a test or mock object that supplies canned "external" data and checks the responses. Yes of course you can use this approach in C with some contortions, but it's not natural C style.
Nah, it originated in dynamically typed languages and works quite well in them (Python, Ruby, etc. have popular TDD frameworks).
Yes, that would count, normal execution of the function under test is stopped.
The relation is that traditional TDD doesn't attempt to be exhaustive. The programmer writes tests to exercises typical cases and also the corner cases that come to mind. I don't think JUnit does any automatic fuzz testing. Haskell's QuickCheck does that (I don't know if it was the first) and it has propagated to some other languages like Erlang.
True, especially in critical systems that must always work (give right answers) for every input, which is probably a more significant issue in embedded systems than in (say) typical web applications or compilers.
More usual programs must never give a wrong answer, but if some weird unexpected input ends up making the program fail by printing "error, please contact support", that's just an annoyance rather than a disaster. Simply using memory-safe languages transforms a lot of "potentially disastrous wrong answer" cases into "annoying error message" cases, which is a big help with such programs.
Constraining C is of some help with static analysis but it only goes so far. Here is an interesting blog post about why static analysis missed the Heartbleed bug, and another about an improvement that the Coverity guys are trying as a result:
I first did that, in C, in (gulp) 1982. It made the embedded system very easy to debug by running it in a test harness on a PDP11 instead of the embedded Z80.
It is no more natural or unnatural than any other model (either implicit or explicit) in C or any other language. All programs contain many different models; the better programs make the models explicit.
TDD was popularised there, but it didn't originate there. I've come across many ad-hoc TDD-style frameworks, starting with those for pure hardware.
There are significant qualitative differences. In particular you can easily see, inspect and debug the path and objects that lead up to the well-contained explicit message.
Contrariwise, in C you are lucky if you detect that something happened at some time in the past, but all useful evidence has not been retained.
Of course. Embedded environments are second-rate by comparison.
available), using FPGA, or using HW simulator and Verilog models.
arget, and since this is new to our group, I am looking for suggestions and comments.
something that we will be looking at
ures specific to embedded systems?
Hi Steven,
I would suggest the first step for you is changing you software architectu re slightly. In any embedded system there is the software that deal with th e bit twiddling of the hardware and the application logic that makes the pr oduct what it really is. Separating those two aspects is important. So I su ggest you design with a hardware abstraction layer and application layer. A ll the application layer code can be tested on a host. The hardware layer c ode can also be unit tested if the hardware can be separately simulated (e. g., the communications hardware in the device does not share bits in regist ers with the motor control).
You mentioned that you do't yet use an RTOS. Are your programs written in t he Grand DO_ALL loop? Long term, I think you will find an RTOS can really s implify your development.
For many embedded applications a "full blown" HAL would be overkill. Usually all that is needed is a clear distinction between the objective/intent (e.g. indicate "error") and mechanism (e.g. set bit 5 high to turn on LED). If there is a single lightweight mechanism coupling all the mechanisms and objectives, then it will help when replacing the test harness on the host with the real logic, and replacing the test harness on the target with the application.
That's a classic build-vs-buy dilemma, and whether it is worth it largely depends on what protocol stacks come with the RTOS.
If you can cast your application into the form of FSMs that receive[1] and generate[2] events, then an RTOS is very likely to be helpful -- and it will greatly ease your debugging and integration.
But you knew that, I expect.
[1] e.g. motor too hot, button pressed etc [2] e.g. turn led on, warn operator, switch fan on
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.