Just to be clear - this thread was about validating components such as an RTOS for safety work. *Nothing* I have written in this thread is related to language compliance testing for compilers.
If you want to continue the tit for tat with Chris about compilers please go back to the original thread, rename the subject of your posts, or start a new thread.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
No but that does not make it secret. The methods certainly are NOT secret.
Many highly qualified and experienced people have in seen the source code of both the main test suites. All the commercial compiler companies for a start. All of the test and validation houses have.
It is just that the source (and IP is not FOSS)
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
They are NOT SECRET. ALL the main compilers and test suites houses use them their methods are public.
The reasonable back up is most of the worlds serious compilers vendors and test houses, many consultants who themselves are qualified and experienced (the one I use us UKAS qualified). Most of the tool vendors who do static analysis use these test suites.
There is a vast amount of qualified and experienced backup Even some GCC companies use these test suites. The fact that the rabble don't get the source code is neither here not there.
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
No one has. There are two main forms of testing. Both use back up documentation. Richard is referring to tests that require producing and certifying a document that details a series of measurements resulting from a series of prescribed procedures following a publicly available standards document. Many standards tests use these procedures, FCC EMI standards testing is another example. Individuals who work on standards documents generally assign copyright for their work so that international standards organizations (ISO for example) and national standards organizations (ANSI and CSA in United States and Canada) can distribute and sell the standards to support administration and support of standards.
Compiler language testing document the execution of standardized programs. The compiler testsuites were developed by companies that specialize in designing tests that check the validity of tools against standards documents. These tests are supplied in source form to testsuite subscribers. During compiler development these tests are closely scrutinized for errors if they appear to have a difference of interpretation. Language testsuites are developed by companies that specialize in a testing methodology and government agencies (NIST and the military for example). These companies provide many support services for there products including a separate informed opinion on the meaning of standards documents. Each language tends to have its own testing traditions that are respected by its users. Fortran and Ada test suites are very different from C.
Regards
-- Walter Banks Byte Craft Limited
Exactly. And any such validation is worthless without exposing the validation process in detail. It has to do with components, whether compilers, other software, hardware, or whatever.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
... snip ...
But it is. Who are you to define 'rabble' anyhow? If that rabble is intended to accept the fact that 'This thing was tested and passed' they are entitled to know the details of the testing. The pure existence of a hiding methodology makes the whole process highly suspicious.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
... snip ...
And Chris is referring to some nebulous tests that are not publicized, cannot be tested and criticized, under threat of copyright suits, and should be accepted by the docile 'rabble'.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
... snip ...
And that gives those firms the right to say "We have applied our secret tests to it, and consider it accurate". No more.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
... snip ...
It still is. Compilers are just a typical example of a component.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
[trying out my new Motzarella account - thanks for the tip]
I agree compilers are a component, but I'm still talking about something different. I'm talking about testing components to international safety standards, as published by, ratified by, inspected by and audited by 'official' bodies. Chris is talking about language compliance testing, which is not 'governed' in the same way. In any case a compiler can be
100% compliant with a standard (if you can tie the standard down 100%), and still produce bum code, (as discussed in the ARM IDE thread) so being language compliant makes little difference to how you would validate the use of a compiler in a particular application - but PLEASE lets no go there again - not in this thread anyway.-- Regards, Richard. + http://www.FreeRTOS.org & http://www.FreeRTOS.org/shop 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
Well, I think everyone has expressed their opinion, at least once, so I see no need to continue it. :-)
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
There is a small difference between uC/OS on one side and SafeRTOS and SCIOPTA on the other: Those two are certified. That is, you may take it as black box plus you can rely on a certified scheduling. Whereas with the validation suite, I have to take the RTOS as well as my code to certify it.
As of IEC61508, the enviromnent must be fixed for a certification which includes tools such as compiler,assembler and linker.
But a re-certification with different tools (if they are valid for safety related systems) does not cost as much as the original one. Mainly because all the paperwork and test-suites are already done.
-- 42Bastian Do not email to bastian42@yahoo.com, it's a spam-only account :-) Use @monlynx.de instead !
Despite the legal issues: You may not alter the certified version. At least for Sciopta it is checksummed and a bit changed voids the certificate.
-- 42Bastian Do not email to bastian42@yahoo.com, it's a spam-only account :-) Use @monlynx.de instead !
The methods of testing an RTOS (which what the OT was all about) are public (as the standard defines it). But the test program are not public to you if you are not a customer.
But, and this is important, a customer of a certified software gets more than you will ever get from some kind of OpenSource(TM) software:
- Test suits
- Paperwork
- design information
- source code
Just take a single simple Linux driver and try to make a full boundary test.
-- 42Bastian Do not email to bastian42@yahoo.com, it's a spam-only account :-) Use @monlynx.de instead !
I think you misread the relevant posts.
w..
The method is not AFAIK hidden. Just the source code.
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
No. Very clearly defined tests.
Very well known
Have been tested and criticized
All Sw has a license particularly FOSS
Yes.
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Those tests are NOT secret. They are just not FOSS
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.