Validation for Toolchain GNU Linux embedded?

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
We're looking at using a Motorola PPC866CZP processor and the Toolchain GNU
Linux Embedded development environment for a safety-critical application.

As you know, in safety-critical applications, the issue of "compiler
validation" pops up. You have to demonstrate that not only the application,
but also the development tools are validated.

I'm a complete newcomer to the Linux embedded arena, so I don't know how
things stand here. Does somebody have some experience in this area? Does
somebody know of validation suites that have been run on the development
environment that can be used as evidence of a validated environment?

Thanks for any ideas,

John



Re: Validation for Toolchain GNU Linux embedded?

Quoted text here. Click to load it

It's not actually clear if you mean that you'll be using the GNU tools
and some embedded OS, or using the GNU tools and Linux as your OS.

Either way, I'm not sure if this is a good idea.  In fact, I am sure
that it is a bad idea; certainly it would be bad to use Linux for this
sort of application, and my experiences with gcc would make me never
think to use it for safety-critical work, either.


Quoted text here. Click to load it

It may be that some older versions of the plain C compiler from gcc
are well tested on certain platforms; there's no reason in theory that
it couldn't be done.  So you might be able to find a vendor of a
certified gcc-based toolchain.  But for this application it's hard to
recommend.


Quoted text here. Click to load it

Linux isn't much used for for safety critical work.  Frankly it's
pretty borderline for telco; large amounts of system and app level
redundancy are needed to achieve meaningful 9's.

--
Grant Taylor
Embedded Linux Consultant
We've slightly trimmed the long signature. Click to see the full one.
Re: Validation for Toolchain GNU Linux embedded?

Quoted text here. Click to load it

It's the latter: Linux as the OS.

Quoted text here. Click to load it

Wow - that's a warning to take to heart. Sounds like you've had some
convincing experience there.

Quoted text here. Click to load it

That's interesting to hear that it's even borderline for telco. I'm going to
have to go back to the guys and get a better handle on exactly how
safety-critical their application is, and report on what you said. Thanks,

John



Re: Validation for Toolchain GNU Linux embedded?

Quoted text here. Click to load it

Hmm, then you'd better be *darned* sure how well the kernel is
expected to work.

Any variations on the themes "the kernel must have been meaningfully
audited" or "the kernel must have proven reliability measured thusly"
will tend to disqualify Linux.  There are, for example, no Linux
design documents against which to audit the implementation; likewise
there are few useful reliability studies that have been done.  Even
the anecdotal standard "it's used in this similar product" may be a
hard one to reach.

Quoted text here. Click to load it


Yes.  BTW I am aware of gcc's test suite, which I think Dan mentioned.
It's a good thing that they have one, but it does not catch all bugs,
and -- especially on non-x86 platforms -- all caught bugs are not
easily fixed, to say nothing of uncaught bugs.  I'm not certain what
industry you are in, but regardless it is unlikely that the existence
of a test suite alone would suffice to make your lawyer happy :(

Experience-wise, I've never worked with a version of gcc that didn't
have a) known bugs and b) unknown "mystery" bugs.  Whether you can
find a version known to be safe enough for your task remains to be
seen.  I'd certainly be surprised if Montavista were able to provide
one; at most they'll say "it passes the test suite" and sell you a
service contract.  Personally I'd put more faith in service backed by
the guys over at Cygnus/Red Hat, but six of one...

Quoted text here. Click to load it


The standards for telco work are expressed as a number like "five
nines", meaning 99.999% uptime.  For certain data applications, the
standard is rather less than for traditional voice; service
interruption during reboot, for example, is grudgingly tolerated.

Be that as it may, the requirements do mean that you must build
redundant hardware.  That done, you of course end up with a matching
redundant software architecture, which allows you to tolerate a
certain degree of uncertainty in some software components.

Or at least that's the justification for not instantly disqualifying
Linux.  In reality market forces push Linux, because many engineers
are familiar with it, and it's provided for most silicon these days
(as opposed to the BSDs, which are not nearly as popular).  From a
purely technical standpoint, OSE, QNX-N, or similar would be better;
networking boxes do have realtime requirements that are much simpler
to meet with anything resembling an RTOS.

In terms of Linux software reliability, there are some very
rudimentary test suites, but they are run only as an afterthought.
Main-stable-branch "Linus" or "Marcelo" kernels, even on x86, are
something of a joke; whole subsystems are sometimes unusably buggy.

In the x86 server world this is OK, as companies like Red Hat, SuSE,
etc all do extensive testing and bugfixing on their kernel branches.
Unfortunately, for embedded purposes, and on non-x86 platforms, it is
very difficult to find a post-QA kernel version you can start from.
Even those organizations that do extensive testing as part of
preparing a Linux BSP do not manage to test nearly as well as the
"mainstream" x86 folks.

Please don't be put off of embedded Linux for everything -- it's a
fine OS for plenty of non-safety, non-realtime embedded tasks.  It has
vastly broader functionality and driver availability than most
commercial embedded offerings; it's easy to work with; and most
importantly, customers seem to like the results.

Quoted text here. Click to load it

Good plan...

--
Grant Taylor
Embedded Linux Consultant
We've slightly trimmed the long signature. Click to see the full one.
Re: Validation for Toolchain GNU Linux embedded?
Grant,

Quoted text here. Click to load it

I think this is going to be the key to the way forward here: getting an idea
of the dimensions of what the team wants to do in terms of safety-critical
and real-time. If we can constrict those dimensions sufficiently, it may
work out okay, since as you note, not only the customers like the results,
but the developers like using it as a platform.

Quoted text here. Click to load it

Thanks again,

John



Re: Validation for Toolchain GNU Linux embedded?
Quoted text here. Click to load it

Yes.  The list includes:
* the gcc regression test suite
* the glibc regression test suite
* the LTP regression test suite
You can also run test suites for large C++ frameworks, e.g. the ace/tao
regression test suite, as an additional test of the C++ compiler and
operating system.

See http://kegel.com/crosstool/ for notes on how to build and test
the gcc/glibc toolchain.

Alternately, you can buy a pre-validated linux / gcc / glibc combination
from e.g. Montavista.
- Dan

Re: Validation for Toolchain GNU Linux embedded?

Quoted text here. Click to load it
GNU
application.
application,

Very interesting! Thanks, Dan, I'll check out all of this. Great stuff.

John



Site Timeline