opinions on automated unit testing

Graeme Prentice · 2005-04-19T12:46:10+00:00

I'm looking for comments on automated unit testing of embedded Csoftware where the automated testing is run on a PC, not in the target.Any guesses as to what percentage of embedded C apps are tested withautomated testing?Any guesses as to what percentage of embedded C apps are tested with awell-defined,documented test spec and procedure as opposed toundocumented ad-hoc testing?For a new application, does the use of automated testing typically meana longer development time up to release one of the product andthereafter save time, or is development time up to release one typicallyreduced or not much different?Are the tests performed by the automated testing normally not repeatedby manual testing in the target?If substantial automated testing is done, is there normally also a lotof manual unit testing required in the target, to test things thatcouldn't be tested in the host environment?How do you verify that the automated testing is doing its job correctly- is there any automated means of doing this?Is it common to spend a lot of time writing code to emulate the targetin the host?Thanks for any comments.Graeme

Apr 19, 2005 24 Replies

Rate this thread:

MetalHead 21 years ago

In my previous job, we developed a new GPS receiver architecture and algorithms to take advantage of it. We wrote a simulation of the hardware in C, and had sampled data files from a GPS RF front end. These were used to develop the control software and advance the receiver architecture design.

It was extremely slow, but worked well. The target was not a PC, so we had a number of conditionals to hide the compiler differences and cover some of the shortcomings of the hardware simulation. All told it was pretty successful.

In the past, I have developed algorithms on the PC that I later ported to an embedded platform. It's pretty effective, especially if the target hardware is non-existent, hard to access or flakey.

Bob

Vote

Robert Kaiser 21 years ago

That would be done with a code coverage analysis. (i.e. instrument the code under test, then run the test, see if that leaves any white spots in the code under test, i.e. code/conditions that have not been exercised by the test.)

There are some nice tools available for doing this, but given their typical price tag, i guess not many people actually use them, unless they are forced to (e.g. by DO-178B).

Rob

Robert Kaiser email: rkaiser AT sysgo DOT com SYSGO AG http://www.elinos.com Klein-Winternheim / Germany http://www.sysgo.com

Vote

Not Really Me 21 years ago

No argument there. You are correct, it is mandatory in some industries. We do testing for safety-critical embedded software so I see those industries all the time. Even then customers send us software they think is ready to go into very critical systems that scares me half to death.

In the others I stand by my estimates.

Scott

Vote

Everett M. Greene 21 years ago

The College of Engineering has grown that much in the years since I was there?!?

There are (semi-)automated testing tools available and used for areas such as mathematical processing. There is no excuse for some sort of testing of math libraries, for instance, but there aren't very many people developing those.

A good illustration of the value of testing that I saw many years ago is the exercising of all execution paths in code. It was amazing the number of bugs uncovered by even that crude testing...

Vote

Paul E. Bennett 21 years ago

Testing with 100% code path coverage is always worthwhile and is easily accomplished in incremental development regimes (where you build the system bottom up and prove the lower layer is good before using it). Combine the incremental approach, unit testing and a decent Configuration Management System and you begin to get the ability to be fully confident in applying certification to your software.

******************************************************************** Paul E. Bennett .................... Forth based HIDECS Consultancy ..... Mob: +44 (0)7811-639972 Tel: +44 (0)1235-811095 Going Forth Safely ....EBA. http://www.electric-boat-association.org.uk/********************************************************************

Vote

Join the Discussion

Have something to add? Share your thoughts — no account required.

Didn't find your answer?

Ask the community — no account required