FreeRTOS / SafeRTOS in a Medical Device

Our team is working on the early stages of development of a class 2 medical device using a low-end ARM7 controller.

Our plan is to do much of the preliminary development using FreeRTOS and then migrate to SafeRTOS after we have proved feasibility, on the theory that SafeRTOS will be easier to get past the FDA.

Today, I got a brief look at what SafeRTOS costs ... something on the order of $60,000 (!!!!) including the documentation and test suites to support FDA approval (I think they call it their "Design Assurance Pack" or something close to that).

I've done plenty of work on medical devices in the past and we didn't pay that kind of money for VxWorks or LynxOS... unfortunately those OSs aren't available for low end microcontrollers like the ARM7.

This is my first time with FreeRTOS or SafeRTOS. The whole FreeRTOS / SafeRTOS migration path seemed eminently reasonable to me... do most of the development with minimal risk and then when you've proven feasibility, migrate to something that will be easier to get approved.

But, I had no idea that the "certifiable" version was so wildly expensive, and we are definitely going to need to come up with some alternatives.

Fact of the matter is that the operating system code is a small percentage of the overall code, and we're going to have to validate the overall software product, so since we have the source code for the operating system, it seems like it'd be a relatively minor amount of extra work to validate that, too.

Have any of you been this route with FreeRTOS / SafeRTOS? How did you handle it? And what alternatives do we have?

Reply to
C. J. Clegg
Loading thread data ...

First question:- Do you really need an OS for the project?

I have worked with some devices that had to have CE (Medical Devices) and FDA approval and we did the whole thing in Forth. However, I think that if you are doing the bare metal as well as the software from there up you should have no problems no matter what language you use if you approach the task the right way. Of course, if you need an RTOS then you should have one team that concentrates on proving that it is good. For that you need to understand everything about the OS and why certain methods are used. It needs to undergo static analysis and dynamic testing.

Second question:- Do you have a really sound development process that gives you the evidence for what was done at each stage of the development? The companies that look over the work you do for certification need to see a full evidence stream for the quality and soundness of the system development and thus the system itself. I would go with simpler processes that let such evidence fall out of the operation of the process naturally. You need to ensure you create the right documentation at each stage and in the right order. You need to do risk assessments for the system and prove the dependability of the resultant design through adequate review and testing. Make sure your process gets the bugs out early (before you start in on the detail design preferably).

--
********************************************************************
Paul E. Bennett...............
 Click to see the full signature
Reply to
Paul E. Bennett

First off full disclosure - I am connected with the SafeRTOS support team (and obviously FreeRTOS).

There is a technical note on upgrading Free->Safe which can be obtained from WITTENSTEIN high integrity systems.

The cost comes from the design assurance package, in which there are many hours of work invested, I will ellaborate more in a reply to Paul E Bennetts post (the first reply to your post). Compared to doing the work yourself, its a bargain ;o)

You not only get the paperwork, but all the test suites that can be run on your own hardware, in your own environment, making them valid for your system. The tests are also designed to provide some compiler validation (for the application, not the compiler itself) to remove another obsticle for safety related developments. See

formatting link
.

I think also included is some on site engineering time, but you would have to check that with the vendors.

The design assurance package can be used as a template for validating the rest of the code. Also, the use of a kernel means that the rest of your code can be smaller, simpler and more modular (the timing information is abstracted away by the kernel). This is not intended to sound like an advert for SafeRTOS - the same would be true with any kernel.

--
Regards,
Richard.
 Click to see the full signature
Reply to
FreeRTOS.org

Full disclosure up front as before....

And this is the point of the design assurance packaged. All this has been done for you, and certified by a very credible and completely independent third party as fulfilling the requirements for safety related systems. This means you can integrate the design assurance package into your application evidence with a high level of confidence that it is also 'certifiable' in your context.

Again, the design assurance package (DAP) was developed using such a process and contains all the required evidence.

Not sure as to what the reference pont for the 'simpler' is, but agree in principal.

Done and included in the DAP in such a way that it should assist with the application level assessments.

Done and included in the DAP.

Done and included in the DAP.

[this is where a lot of the cost is, reviews take engineering hours]

Done and included in the DAP.

Getting all the bugs out before starting the detailed design, now that would be nice. I'm assuming your talking about the system engineering bugs ;o)

--
Regards,
Richard.
 Click to see the full signature
Reply to
FreeRTOS.org

In message , C. J. Clegg writes

Scandalous....

I can sell you the Sciopta RTOS that is also tested for Safety critical use (SIL3 ) at a bargain price of about 59,000 Euro :-) see

formatting link

Both SafeRTOS and Sciopta are less expensive AFAIK than the Green Hills Integrity and other certified RTOS

As Richard has pointed out elsewhere there is a LOT of labour intensive work involved in validating something fro safety critical use. More to the point this work has to be carried out by suitably qualified and experienced people. Lives depend on it.

The cost of SafeRTOS is about right and not over priced.

That's life. BTW what were VxWorks and LynxOS certified for?

This is how Sciopta works They have a less expensive ( about 2K5 USD) not critical version with the same API and a more expensive certified version.

BTW if you saw all the costs involved with certification you would see you are getting a good deal at 60K USD

Don't do the project.

You can't do safety critical on the cheap.

The SafeRTOS is not exorbitantly priced. As a competitor I can telly you it is quite reasonable.

--
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
 Click to see the full signature
Reply to
Chris H

I guess you have to ask yourself: how much would it cost me to do al this certification myself, and I bet you'll end up with a figure that's much higher than $60k.

Reply to
Bresco

Exactly (BTW: I think the single project license is actually $45k).

--
Regards,
Richard.
 Click to see the full signature
Reply to
FreeRTOS.org

Can you tell us what a FDA approved version of VxWorks or LynxOS costs then? I bet it's very similar in pricepoint to FreeRTOS. I've used another RTOS years ago and its price was also in the $40k range back then, probably more now

You can also try using Linux with the RT patch, which will give you response times (latencey) in the 100uS range. That should be anough for most applications and it's totally free. Although it won't be FDA certifiied.. And then there's eCos, the open-source RTOS which also runs on ARM7, again not FDA approved, but I bet ya that someone has made a medical appliance with eCos already.

Reply to
Bresco

In message , Bresco writes

There is the problem... what will it cost to get Linux certified? :-)

What cost for certifying that?

What I can tell you is that Sciopta and I would think SafeRTOS have priced their certified RTOS assuming multiple sales and have spread the cost. As the non certified freeRTOS and Sciopta RTOS are zero and less than 3K USD I think you can work out that it is likely to cost you a LOT more than 60K USD to certify an RTOS.

Remember not only is there the cost of certification you need to have ALL the documentation, specification, history etc not to mention testing and show the development procedure was up to spec.... That takes time and effort. More so for Linux (showing development procedures and testing)

I think you may find that it will be more expensive to try and do a certified Linux system.

--
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
 Click to see the full signature
Reply to
Chris H

I'm not advocating neither Linux w/ RT patch nor eCos, just summing up the possibilities. For non-medical applications these two will suffice in 99% of all cases.

Reply to
Bresco

Thanks to everyone for your responses in this thread. If no one minds, I will combine my responses to all, in this post.

First of all, a bit of history... I have done a lot of medical device development work but not in the last five years or so. Hence I may have lost track a bit, regarding the current thinking of the FDA on this topic, although I am well familiar with the FDA guidances including the one on OTS software, and they don't seem to have changed markedly in the last several years.

Back in the mid-late 1990's, I worked on a Class 2 medical device that was based (I swear I'm not making this up) on MS-DOS 5.0. The company had no trouble getting that past FDA. They had trouble getting other aspects of the development past the FDA, but not the operating system. That device, as well as the device I'm currently working on, was "substantially equivalent" to other devices on the market, and so the companies involved did seek and are seeking approved via the 510k route and not the PMA route.

Other medical device projects I have worked on used VxWorks, LynxOS, and home-grown solutions. None were "certified", per se, by FDA. However we were able to "validate for intended use" by testing and by showing that they were mature products in the industry, and that their manufacturers had effective support and defect-repair systems in place. We were never asked for the full documentation and design controls package, including design history file, for any of the commercial operating systems, and of course we would never have been able to provide it had we been asked.

Perhaps all of this ties in with what one means by "safety critical". This current project is to develop a Class 2 device, which will be presented for approval via the 510k route, and we are going to argue that the software is of a Moderate Level of Concern (we may not get away with that one and I'm certainly going to encourage the team lead to run that one past the FDA sooner rather than later). So perhaps it doesn't rise to the definition of "safety-critical" that would trigger the need for all the design controls documentation of the OS (and would probably trigger the need for a PMA as well).

Perhaps in the last five years or so, the FDA has solidified its expectations regarding OTS software, and we certainly expect to work with them to clarify those expectations and make sure we meet them. But, as I said, in the past none of the products I have worked on, using home-grown, VxWorks, or LynxOS, have had any particular trouble getting the OS's past FDA review, even if the OS's were not "certified". Anyway, back then, there really wasn't any such thing as a "certified" VxWorks... is there such a beast today? And, the version of LynxOS we used was the plain vanilla one, not the LynxOS-178 or even the -SE.

(Neither VxWorks nor LynxOS will run on the planned platform, so all of the VxWorks / LynxOS discussion is moot, but I just wanted to point out that in past projects, those OS's weren't "certified" per se, and we didn't have any particular problems due to that.)

Now, on to some of the points that you all have generously raised...

As a practical matter, yes. We have done some architecture design and we expect the software to run in at least a half dozen separate tasks at different priorities. It's always possible to roll one's own, but as we all know, it makes no sense when commercial products are available at low cost. For example, OpenRTOS at I think something like $4000 or $5000 a license would be a no-brainer for this product. SafeRTOS at $60,000, amortized over the quantities this company is talking about, would require some serious thought.

That's a separate issue that is being addressed. Suffice to say that it's an area that is a work in progress. However, it needs to be there for all of the design aspects of the product, not just the OS.

We are in the process now of establishing the policies and procedures that control this. Certainly everything you suggest is part of the plan. But, all of this needs to apply to all of the software, not just the small percentage that is the OS. See below...

This product is intended to grow into a family of related products, so we'll likely need the unlimited-projects license.

Several of you have pointed out, rightfully so, that the kind of validation required for the operating system of a truly "safety-critical" product (depending on how one defines the term) is likely to cost more than $60,000. That's certainly true enough. However, ALL of the software for this system is going to have to undergo an appropriate level of validation, and the operating system is a small percentage of the overall code base. Therefore, I claim that we could apply the same validation procedures to FreeRTOS (since we have the source code) as to the rest of the software in the product, and the only difference is that we wouldn't have the full development life cycle documentation for the OS. That wouldn't have been any particular problem "back in the day" for this class of medical device. Maybe FDA has gotten more strict about that stuff in the last few years.

You all have pointed out the advantages of the SafeRTOS Design Assurance Package, mainly that it provides all of the validation evidence, suitable for truly safety-critical systems, for a one-time payment. My question is, do we need it at that level? This isn't a DO-178B Level A device like a fly-by-wire system (but, does SafeRTOS's "certification" really rise to that level of demonstrable reliability in the eyes of the FAA, which is considerably stricter about such things than the FDA?).

In conclusion, I would like to apologize for characterizing the SafeRTOS cost as "wildly expensive". I know that sounds like I was criticizing, and that wasn't the intent. I can see that, for a truly safety-critical system, it's a good deal. I'm just not at all convinced (yet) that we'll need it at that level.

Reply to
C. J. Clegg

Hi C. J. - could you send me a message to r (_dot_) barry {at} freertos.org - I would be interested in having a FreeRTOS.org related chat. I know this is not the done thing in these groups and am not suggesting that we don't continue this interesting thread here too. I also pledge not to pass your email onto any of the commercial guys unless I have your permission to first :o)

--
Regards,
Richard.
 Click to see the full signature
Reply to
FreeRTOS.org

snipped-for-privacy@4ax.com...

Interesting discussion - worth noting I also downloaded the SafeRTOS pricing and it is only $45k for a project license........

Reply to
dw4780

I could do that... but there really isn't anything I can contribute privately that I can't contribute publicly, and there's nothing I can say that I wouldn't want the commercial guys to see.

I'm all for giving the widest possible audience the benefit of these discussions, to the extent that they are interested...

Reply to
C. J. Clegg

...but there are some things I can contribute privately that I cannot on a public forum.

--
Regards,
Richard.
 Click to see the full signature
Reply to
FreeRTOS.org

In message , Bresco writes

You mean non-safey-critical

there are many systems that require validation other than medical systems.

--
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
 Click to see the full signature
Reply to
Chris H

In message , FreeRTOS.org writes

Of course if this was FOSS..... :-)

(I'll get me coat :-)

--
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
 Click to see the full signature
Reply to
Chris H

Yes it would be very nice if everyone else would do just that. The majority of the bugs in a system happen in the requirements specification stage. Getting all of those out before starting detailed design will always improve the technical detailed design. Getting correct designs has always been and will always be by having a good process. There is no magic language, no magic OS that does it all for you and you have to expend some verification effort even if you are buying in a SafetyRTOS type product.

--
********************************************************************
Paul E. Bennett...............
 Click to see the full signature
Reply to
Paul E. Bennett

There is a lot of data to back this up. Incidentally as Paul knows on Safety critical systems about 50% of the problems come from incorrect specifications. Less than 14% from the implementation phase regardless of language used.

In a good, well defined, process the choice of language has minimal impact.

So very true. I have seen data to conform this. However many will always try and subvert this. In fact some of the systems currently in vogue run counter to this. They will, eventually, learn the hard way

If you buy in a validated RTOS and use validated compilers you STILL have to have a good process for the rest of the project. You need to put all the rest of the code though a rigorous process. It is just that the RTOS will have been done for you. Saving much time

Using certified compilers likewise saves you a hell of a lot of time and effort. The cost of the compiler fades in to insignificance compered to the other costs.

--
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
 Click to see the full signature
Reply to
Chris H
[%X]

So long as that makes sense to your product then fine. There have been times when I have seen people have terrific problems with an OS on a project when it would have been easier without. My own foray into Medical Systems was on Anaesthesia Ventilators.

When you get your process together don't forget to test it out and make sure it catches problems before they get too far. Do regular audits of the process and ensure everyone in the team is following it.

Definitely.

If the RTOS is really such a small part then it does not seem like it will be much more burden on your efforts with the rest of the system. Hope you have a team for the OS reverse engineering effort as this may be the only way to be certain it really is sound. I presume that FreeRTOS might be in something like C (don't know for sure as never used it - but is a reasonable guess I think) in which case getting the code through a LINT or similar tool configured for MISRA-C rules might help you out a bit. Don't forget in all this effort that the "MK1 eyeball" is also a very good tool (ie: spending some time just looking at the code - asking oneself "is it elegant?" "does it logically scan?" "does the coded function meet with the requirements?" and a host of other related questions).

I have seen projects come unstuck when the developers assumed the product would not be safety critical but in use it turns out should have been a SIL1 system. At least you have some appreciation of the kind of Hazards and Risks involved (from experience). This is why all system developers should start with a full list of the Hazards that might be faced by their system implementation and design out the risk level accordingly (fully documenting as you go). I tend to begin by assuming I will develop for SIL4 until proven otherwise.

[%X]
--
********************************************************************
Paul E. Bennett...............
 Click to see the full signature
Reply to
Paul E. Bennett

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.