Embedded Linux Vs. Real time Linux

With any type of Linux you need to write a device driver to handle a hardware interrupt (but a device driver is just a kind of program :) )

Same is true for DMA, as well, even if no interrupt is used.

You need to decide if you need hard realtime (a (maybe even some 100 ms) timeout needs to be met in any case) or a soft realtime specification (e.g.: as an average, every years once a 100 msec timeout is allowed to be missed). Kernel 2.6.x offers quite good soft realtime behavior. for hard realtime, standard Linux is not applicable. Here you need commercial stuff or add-ins (like RTAI).

If you use Linux, you need to do everything "the Linux way". There are lot's of specs that help and restrict.

I once heard about "realtime Ethernet", but I don't know much about this. Linux Realtime solutions (e.g. RTAI or PikeOS) provide a basic OS and Linux runs as a low priority task of same. So all realtime stuff has priority above all Linux stuff (including Ethernet).

-Michael

"Morten M. Jørgensen" skrev i meddelandet news: snipped-for-privacy@k2g2000hse.googlegroups.com...

Why bother with Montavista, if you just want ordinary Linux?

You are likely to run with a MMU enabled. Your code and data needs to be in the virtual address space of a Linux process.

Anything is possible, given the right interrupt controller, but that does not mean that it is supported by Linux. On the AT91, you could will get a vectorized interrupt but I assume all vectors are the same in Linux. Doing something else will require modifying the linux kernel.

The key problem is probably the transfer of data between your interrupt and the linux kernel.

You use a fast enoúgh processor, or rely on the TCP/IP protocol to resend data that gets lost because you do not have time to process it. At 1 Gbit, you hopefully have a DMA which can store multiple packets without CPU intervention.

Next to the commercial versions there are also some open source versions, like Xenomai or RTAI. These have the same concept: a small realtime kernel, and Linux as a low priority task of this kernel. I don't have experience with the commercial version, only with Xenomai, so the answers below relate to this, but are probably also true for other realtime versions of Linux.

Yes, this is possible, even when Linux is processing another interrupt the realtime interrupt can interrupt this. An interrupt can be processed by Xenomai, or by Linux, or even by both. When leaving the ISR you can indicate if the interrupt was handled. If not the interrupt will be serviced by Linux.

Real time Linux allows your user level applications to run in real time, even under stress conditions. Using a very simple application it is possible to test this: wait for timer interrupt, read timestamp, calculate difference with previous timestamp. If you use a 10 msec timer interrupt you would expect the difference to be 10 msec. Under low/no load this is true: the variation is within 20 usec. Under load however the variation can be a number of msecs, which is not real time behavior. When using Xenomai (or other RT-Linux versions) this variation is still around 20 usec.

Xenomai is part of the Linux kernel, so the drivers exist within the kernel. This means that you have access to all the resources of the kernel. You are also using the same address scheme as Linux. If you want you can even write multimode drivers: both RT and non-RT drivers combined into one driver. Your applications are also running as normal Linux application, only with some restrictions: as long as you don't do standard Linux system calls your application works as a realtime application. You can also have a thread in an application run as real time thread, while others are running as Linux threads. As these are thread they share the same address space, so you have access to the same memory, and can use this to communicate between RT and non-RT.

The interrupt routine will be called, but the processing of the data will be suspended until no more realtime tasks are running. This means that if the buffer between the interrupt routine and driver is not large enough you might loose data.

Kind regards, Johan Borkhuis

Hey Ulf,

We are considering montavista because of the support that they supply

- we would like to use a kernel port that is fully supported. We are a small R&D and a even smaller SW team, therefore we would like to dodge the hard-core linux problems and concentrate on application development.

But still working as a totally independent part - not being a part of the Linux system in any way?

The right interrupt controller? In my world I need to have a possibility to mask the interrupt that I want to interrupt the Linux? Regarding the transfer between the ISR and the kernel - how can I do this when MMU is enabled and without needing to modify the kernel?

Best Regards

MMJ

Why using a separate kernel for hard realtime, when you can get a whole hard realtime Linux kernel? Give RT Preeempt a try. This adds fully hard realtime feature to the Linux kernel (and most of it is already mainline). And: You can use the hard realtime feature also in userspace, to program hard realtime application with POSIX-API only. Refer:

With RT Preempt there is no need for such tricks. You can use all resources from Linux, your can write your drivers like all other drivers, you need no special knowledge about drivers in a separate RT kernel.

JB

RTAI is depreciated. Don't use it for new projects.

JB

I don't know about that (I don't follow RTAI closely), but as far as I can see there is still development going on (a new test release was released this year), and the mailing list is quite active as well.

Kind regards, Johan Borkhuis

Hi Michael,

Hehe...as I said I'm ultra noob into this brave new world :-D But is the device driver not a part of the Linux envrioment with defined ways of communications and scheduling together with the Linux kernel? What I am talking about is some totally independant code that is able to interrupt all Linux processes and complete it's own task before re-enabling the Linux execution.

But how is the Linux/device driver interaction defined? Will a Linux device driver not make use of Linux system calls in some extent?

Take for examble my DMA interrupt. When this interrupt is raised I need to service the DMA and moveing some data from a circular buffer into the application area where the data will be processed and then trasmitted via ethernet. I cannot afford to miss data from the DMA since this will cause huge problems for our system.

So as I see it (and do correct me if i'm the slightest wrong with this) we need either:

- Some independant code outside the Linux enviroment to service the DMA ISR. The DMA interrupt will happen without Linux knowing and Linux will be 100% interrupted and resceduled after ISR completion.

- A Real Time Linux system having a real time extension in the kernel including a defined interface between the real time part and the kernel.

- Third option?

What would be the Linux way of the problem above?

That is exactly what we are considering - but will this be possible with a standard 2.6 kernel ?

-- MMJ

Development for current kernels?

JB

I did see references to 2.6.19 and 2.6.20 kernels, so that seems pretty current.

Kind regards, Johan Borkhuis

I don't understand why you would need to do this.

You can however do whatever you like inside a Kernel device driver. At least on a single processor system it should still be possible to disable Interrupts alltogether and running your code exclusively.

The generic Linux way would be to implement an Interrupt service routine in your device driver which will afterwords transfer your Data to userland.

If speed is important you might be able to use mmap instead to prevent the data from getting transmitted between kernel and userland too often.

Again, I don't see any reason why this code need to run outside the Linux kernel enviroment.

Again, I don't see a reason, why this should be necessary

This is possible to do with the standard Linux Kernel API.

Some Real Time Linux systems (e.g. RTAI) do it this way.

An ordinary Linux Kernel with Realtime Preemption patches will however provide ordinary Unix _Userland Applications_ with Realtime capabilities

Sven

Hello Juergen,

That sound really interesting. Exactly what I am looking for - a complete solution that allows for hard RT performance without forcing us to do experiments trying to integrate outside RT code. I'll look into that for sure

-- MMJ

If you have a prioritized interrupt controller like the AT91 AIC, then you can have one level of interrupts which will enter the Linux Interrupt routine, and other (higher) levels could be used to generate interrupt vectors to high priority interrupts, including the FIQ (Fast Interrupt).

You will have to have your code in the linux kernel address space, but there is no need to link it with the kernel.

Use of FIQ, will require you to have your non-linux code right after the exception vector table.

Also for PowerPC ? that's interesting. Last time I tried (2.6.19) things didn't seem to build/work for our MPC82xx based PowerPC boards.

I you don't mind me asking, what kind of PowerPC CPUs/board you did successfully apply the RT patch for ? with what kernel version ?

Also for PowerPC ? that's interesting. Last time I tried (2.6.19) things didn't seem to build/work for our MPC82xx based PowerPC boards.

I you don't mind me asking, what kind of PowerPC CPUs/board you did successfully apply the RT patch for ? with what kernel version ?

Freescale's MPC5200B with a 2.6.23.1 and currently 2.6.24.rc5. But the

JB

Of course. But that is more of a benefit than a restriction.

The standard Linux Kernel will disable the interrupt now and then, so the interrupt will not be recognized by the processor. If your hardware allows multiple interrupt priorities, you could modify the Kernel to leave one level enabled for your DMA system, but if you do so, there is no decent possibility (other then a hardware FIFO or similar) to have your system communicate with the Linux system (to pass the acquired data), as you can't construct a non-spinning semaphore.

For any decently constructed hardware that creates or needs data bytes in a speed that is slow enough for moving the stream via Ethernet, with a decent processor a DMA enabled device driver should be by far appropriate (otherwise the Ethernet driver would fail).

If the hardware does not give the processor enough time to allow for some decent latency between two consecutive DMA actions, the hardware is erroneously constructed. It should contain some kind of FIFO, double buffers or a similar construct. With that the processor can react on the end of the previous DMA block while the next is running.

-Michael

Indeed it is - I my self would like to everything the right way keeping us from adding outside code and doing kernel modifications.

So what you are saying is that the Linux kernel WILL disable global interrupts from time to time. But I guees the interrupt still will be pending when the interrupts are reenabled by the kernel ? This causes a delay on the service time of the DMA and worst case is that the buffer overflows. Which parameter decides how long the kernel wil disable interrupts - CPU load? Is it posiible to make any estimation of this time?

I'll pass this on to the HW team :-D

-- MMJ

AFAIK, this is correct (maybe depending on the CPU architecture). You need to inspect the Kernel sources to be sure.

Right.

As Linux is not meant to be a realtime OS there are no specs on this. But of course the time is rather short. But If you can live with that delay I'm quite sure that doing a standard interrupt driven Linux device driver will do the work, too.

What kind of hardware do you (intend to) have ? If you do this with an FPGA, maybe the hardware can be done/changed appropriately. Anyway if designing hardware that is to communicate with a processor, the appropriate design rules should be followed. :)

-Michael