snip
Disclaimer: We produce the Validation Suites for uCOS, so my opinion may not be completely unbiased.
You should also be aware, that although FreeRTOS has been ported to a number of processors, according to their web site SafeRTOS has only been certified on one. If you are not using that specific chip/compiler combination your expense is likely to be much higher.
Scott Validated Software
I will have a look at this, it seems very missleading or out of date. Our own original component certification by TÜV was performed on one particular configuration baseline, using a particular compiler, and a particular processor [the only way it can be done]. It does not, or at least is not supposed to, mean this is the only configuration baseline that is available.
Not so, as far as I know the target platfrom is not a price differentiator, but then I'm not on the commercial side so please don't quote me on that!
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
... snip ...
And I have my doubts. Where is the source of this test suite? To what independant international standard does it adhere? If this stuff is available, why can't the users adapt it to other systems, and publish their work? Etc. etc.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
You don't need the source.
IEC61508
You don't understand safety critical work do you? The legalities involved mean you can't tinker with the source without a full and complete re-test. This isn't a hobby or a bit of fun. Lives depend on it.
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
As I already stated - that is not the case on both counts.
I think a little confusion is creeping into these threads. I cannot comment on this question properly because I don't really understand what you are asking. The tests are very specific to the product (not generic), and cost a whole load of money to produce. We are not talking about an open source code base here.
Again, I don't understand this comment so might be ansering the wrong questions.
*If* this is referring to the test suite that comes with a validated RTOS (be it SafeRTOS, Validated Software or whatever) then you most definately do need the source code to be able to qualify the product in your own environment, on your hardware, with your compiler, demonstrate that the required coverage is obtained, etc.......and we provide the source code to paying customer.They can.
Because we would sue you and your childeren would go unfed ;o) I really think we are talking at cross purposes though, so maybe not.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
... snip ...
I have difficulties reading and evaluating the tests without it.
You don't seem to understand reading. Who mentioned 'tinkering'?
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
... snip ...
Oh? You would consider publishing something like:
In version 123.456, lines 789 thru 792:
delete "in blah blah blah" and substitute "in blah gubris" and add "otherwise frabble" to meet standard provision 6.3.4.5.
a copyright violation? I suspect you would have trouble finding a legal firm to represent you.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
Firstly, this was not supposed to be a serious comment, and second I still don't understand where you are coming from.
If I was to licensed you (with usual clauses about no publication or re-distribution etc.) under NDA some proprietary source code that included a test suite and you saw fit to publish this, yes I think it would be very easy to find somebody to represent me (or whoever owned the software, not actually me in this case). I'm sure you would agree with this, which just proves my other point that I think we are talking about different things.
If on the other hand you are saying that somebody could take open source code and publish a load of patches to 'upgrade' the code to be compliant with some safety related standard or other then, I would not have a problem with that. Although I would say it would be of little value for actual use in a highly safety critical environment as the source code is probably 5% of the evidence required, and 0% of the evidence required unless you have complete lifecycle evidence to go with it.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
In message , FreeRTOS.org writes
Source code is less than 5% of the evidence required and 0% unless you have the complete life cycle evidence to go with it....
That is what I have been trying to say.
Life cycle evidence is an important part of things. In a commercial compiler and RTOS companies they have full control and full history at a level you don't normally get in Open Source Projects.
61508 requires a coding subset, coding standard and static analysis for using C, never mind the other testing and proofs.Also changes, any changes, require FULL regression testing. This would be a full build test suite, language compliance, test of language extensions, maths tests etc.
The people I know who do who want they are doing have described the GCC "test suite" as a "ragtag group of tests" and "very weak" compared to most commercial compiler build testing. Added to which most GCC compilers never get anywhere hear a recognised Language test suite. .
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
I think we are mixing threads up here - the "ARM IDE" thread was on this topic.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
Because, as far as I am concerned, a secret method of validating something is totally worthless. This has the same value as my entering a Usenet message saying that Microsofts OS code is absolutely perfect. I don't think anyone would be foolish enough to accept such a pronouncement. However, if I included a set of source code tests, and their results, so that anyone could duplicate them, and stated exactly what I considered those tests proved, at least some would check that out. If they disagreed they would be able to state the disagreement, and possibly the cure.
Please don't remove attributes for material you quote. I have restored them above, I hope without error.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
What "secret method"?
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
There is absolutely nothing secret about a process that is published in an international standard and has many sections on exactly the techniques and measures you have to follow!
Software is never perfect and this claim is never made - be it safety related or not - its about providing the evidence that gives you a very high level of confidence.
Of course not. See we agree on some things.
This is exactly what is done, its just that you have to pay to get them, but we are not talking about a mass market product its very specialist.
The tests are independently generated, independently reviewed, independently audited, and sometimes have to be signed of by government accredited bodies. Lots of people see them. Anybody can buy them.
However, source code tests by themselves are meaningless unless you have the rest of the lifecycle evidence so you know what you are testing for/against. You could test your code for twenty years and still fail an audit within
30seconds when the first question is asked "please demonstrate your requirements tracing methods".I don't like the word 'prove', but the point of the tests is published in the standard. Again, nothing secret.
Believe me - they have been checked out by plenty of people.
Yes. That is the point of independent review and independent audit. Agreeing again.
With respect - I think you have some fundamental misconceptions about how these things work. When I get on an aeroplane I have confidence that the landing gear will work because I know the process the developer has had to go through to get them passed as fit to fly on that aircraft. I don't need to personally see the stress analysis that was performed to have that confidence. However, if I was to take that landing gear and integrate it onto another plane then I would need the entire lifecycle and design data so I could ensure the component was suitable for the new integration platform. I would not expect that data to be publically available though.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
Come on now Chris - you know I get very uncomfortable when we start agreeing with each other.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
In message , FreeRTOS.org writes
BTW both the main test suites are not "secret" it is just that you can't publish openly the *results* of the tests for the normal commercial reasons.
Exactly they are completely open. You do have to pay for them and adhere to the license (just like you do with FOSS)
Which is why it is difficult to do validation for a GCC compiler compared to a compiler from say Byte craft where they have all the documentation from day 1 and control the process. Also all the developers are known
Lots. All with relevant qualifications and experience. The provenance of Plum-Hall and Perennial can not be questioned.
The problem is, as usual, the FOSS people have set up a new model for working and are compiling when the rest of us won't play. However they won't play with the model the real world uses.
Good example.
-- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Oh? Are you now suddenly claiming that the source code of such test suites is publically available and criticizable?
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
... snip ...
I see no sign of testing code and methods in the ISO C standard. So, if you can't publish that code, or even the results of the tests, I see no value whatsoever in the tests. As I said before, it is obvious, according to the XYZ tests, that Microsoft software is uniformly correct and bug free. I have no need to expose the guts of the XYZ tests, nor the actual results, according to you, yet I should be taken at my word. Do you detect anything imbecelic in that statement?
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
No Chuck, what he's trying to make you understand is that in the present World we have a lot of things that are "public" but that you have to pay for.
And my point is that such 'secret' tests are useless. You can get away with this for something like an editor, where the failures are fairly evident. But you can't just say "I have tested this" without some reasonable backup.
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]:
Try the download section.
Just to be clear - this thread was about validating components such as an RTOS for safety work. *Nothing* I have written in this thread is related to language compliance testing for compilers.
If you want to continue the ti_t for tat with Chris about compilers please go back to the original thread, rename the subject of your posts, or start a new thread.
-- Regards, Richard. + http://www.FreeRTOS.org Designed for Microcontrollers 17 official architecture ports, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.