1. Home
  2. Embedded Linux
  3. x86-linux sources of latency

x86-linux sources of latency

Hi If you're not going to read much, please begin from VVV below

I'm working on a port of currently modula2-based EDM CNC system to Linux base. Since this is a port, there's a need to create the same environment as in original system, which is relatively simple all based on i8254 timer (it's a cool Russian soft, really). So as for today I've made the following (it's all about linux-2.6.5 kernel):

1) The scheduler is now working from rtc's IRQ8 (at 1024Hz) => i8254's channel #0 is free and available 2) local_irq_disable() etc. are reimplemented to mask interrupts (all but IRQ0) in interrupt controller, so cli/sti sections do not disable timer's interrupts (this involves several changes in i386 arch files) Ant, as it seems, at this point the system responds quite satisfactory. The top criteria is not that hi, stable 1Khz interrupts (say, +-0.25ms) is enough. My test suite is just fun (copied from old system's crush-test). I establish 1-6 Khz IRQ0 and change speaker's gate bit so it gives sound of given frequency, then I do different actions such as walking through X, syncing disks etc and if things go bad, I here how the sound clicks (even short 1-period holes are noticable). Original 2.6.5, ofcourse, is bad-bad, but with my patches I just couldn't break the sound. Going to make more statistics via Pentium's time stamp counter.

So. Thanks for reading. I see, that some or all of this is just what RTAI and RTLinux do (never looked into), but it's way more interesting to self-code it and, the main, I get exactly what I need, no more, no less. Maybe this is interesting for someone else? If so, great!

VVV

Here's what I really need and haven't yet found.

- Information on sources of hardware latencies in PC's, mainly x86 (I excpect there are I/O issues on buses' level, though I don't know the arch well enough /yet/).

- Are there other real-time constraints in mainstream Linux kernel than interrupt disabling?

- As I know, current real-time Linux solutions give 1 ms granularity (am I wrong? sorry then). What stops developers from going farther?

Thank you and with best regards, Dmitry M. Shatrov

Reply to
Dmitry M. Shatrov
Loading thread data ...

Hmm. That sounds like an interesting approach. I assume you are also enabling kernel preemption?

Well - contrary to most others - I still think I get better performance from 2.4.x with both low latency and preemption patches. What I'm measuring is what shows up at the application level which may be an apples to oranges comparison to what you measure.

If your patches are available - I'd like to review them as well. We don't use either RTAI nor RTLinux since we don't want to run our real time applications (several millions of lines of code) in kernel mode (ours is a rehost from a large SGI system).

I don't certainly know enough details, but am aware of some issues related to the PCI bus. For example, some of the host bridge settings will affect both the latency and throughput of operations to PCI devices. We use a shared memory (like) interface - Scalable Coherent Interface - where we map large (100's of megabytes) remote memory regions through a PCI card. By making host bridge configuration changes

- we can improve the performance to copy data from one machine to another by 20% or more (our best is about 100 Mbyte/sec on a 32 bit 33 Mhz PCI bus). Getting the host bridge really screwed up causes the performance to drop to perhaps 5 Mbyte/sec.

This will vary by host bridge as well. Our particular application will use a loopback (a CPU accesses the PCI card, then the card access local RAM) on occasion and that has been known to break host bridges as well (the Serverworks 64 bit PCI host bridge was broken about 2 years ago - haven't tested a more recent version). We had to use VIA chip sets to get our application to work OK.

When you get to try to run a real time task (as an application), there are lots of other activities that can preempt the real time task. For example - the buffering in the ext3 file system (about 2.6.2 time period) was giving us > 15 msec latencies when doing something as simple as: cat /dev/mem > /tmp/mj.dat I has gotten better more recently but I'm still seeing 3 msec or more latencies with test programs.

We set HZ to 2400 to be a harmonic frequency of 100 Hz, 80 Hz, 60 Hz, 50 Hz, .... The system appears to run fine (on a 1 Ghz Pentium III, dual CPU) without a lot of extra overhead. We've been doing this for a couple years now. We did find some broken drivers in 2.4.x which required patches or other fixes. Our Gbit Ethernet card (Acenic) for example would run OK at HZ=1200 but not HZ=2400 until relatively recently. YMMV.

--Mark

Reply to
Mark Johnson
[...]

RTAI allows user space realtime. It's even the default/advised way in the 3.X releases.

[...]

You're definitely wrong about this in the context of RTLinux and RTAI; both have nanosecond timer resolutions.

Herman

--
  K.U.Leuven, Mechanical Eng.,  Mechatronics & Robotics Research Group
     Tel: +32 16 322480
Reply to
Herman Bruyninckx

Yes, I do enable it. Preemption does good things for userspace and nothing bad for me.

Hm. The aims are a bit different. The case with real-time tasks and user-space processes is that there is no way to make them both hard real-time and fully functional. For example, we periodically call our rt-process. We may once (it's even normal) wake it up during a disk interrupt.. so we can't do anything with filesystems; so on, we may got woken up in our own (!) sleeping system call.. so we can't call anything that sleeps.. And we come up with that yes, it is a user-space process, but the program is limited to some very restrictive API closely related to a 'real-time solution' being used. I read that RTAI already allows such processes and that they are strongly restricted in API, though really user-space. And we are just as restricted while handling an interrupt that is desynchronized with any kernel activities. Extereme care is needed for not to break things.

They are, I'll just make what is called "a pacth" and post it here in a day.

As I understand, you use now 2.4 + low_latency + preempt? It's a combination that turns Linux into a soft-rt system (as far as I know, it all began from the wish-letter from linux audio developers) and it is really better than ordinary 2.6. What I am going to do is to split into two parts: the first one makes step-commands in userspace, it's fifo'ed with the second one, which is in kernel-space and realtime. If such splitting is acceptable for you, consider trying it.

This test makes nothing to me. Anyway, I meausure it in an interrupt and you do in userspace.. apples-oranges, really

Well, as I mean hart-r.t., I assume that _nothing_ can preempt my task and that it's guaranteed that the task will be up and running when I want it. I've just made a test (TimeStampCounter-based) with 1200Hz IRQ0's (950-Mhz PC), it showed that there are stable periodic ticks in a

10-microsecond fidelity (+-5). Interrupts on 50KHz also went fine, still with that 5-microsecond drift.
Reply to
Dmitry M. Shatrov

Just as vanilla 2.6 does. Timer resolution an scheduling granularity are different, please don't mess them.

Dmitry

Reply to
Dmitry M. Shatrov

snip

Beware of system management mode. In this mode the O/S isn't aware anything happened. In the bios setup make sure "enable legacy mode USB" is not enabled ( we had problem with that when I worked at QNX software systems)

Good luck Pat

Reply to
Pat Ford

But I have had "hard real-time" capabilities as a user-space process before. On SGI/Irix for example, you get this by using isolated and restricted (must be both) processors. On a 28 CPU machine, dedicating a CPU for non real-time tasks is OK, on a cluster of 2 CPU machines, It seems a waste of 50% of my CPU capability to do that.

I also worked (many years ago) with a Honeywell DPS 6 system where the tasks were run using the hardware's 32 (or 64?) level priority scheme. Very nice for a hard real time system. You could put drivers below the priority level of real time (user-space) tasks if you needed to, something you can't do with Linux or a stock Unix system.

Sorry - our application can't do that. It must be able to: - talk to the network [works with other real time systems, interface with separate machines running the GUI display] - read / write the disk [simulating a real world hard disk drive] - get memory [but over 95% of that is allocated before starting real-time] and so on. No restricted API's allowed.

[we seem to agree we have an apples to oranges comparison] --Mark
Reply to
Mark Johnson
[ ... ]

If you have hard realtime demands with full functionality, you'll need an OS that was designed for it from the start, I'm afraid. If you want to stay POSIX/UN*Xish, my (biased) opinion is that LynxOS will probably do what you want. There are a couple of other OSes out there that also claim POSIX and realtime, but the other ones I've worked with (eCos and VxWorks) aren't very POSIX.

Caution: I'm a LynuxWorks shareholder, and worked on LynxOS for 8 years, so, yes, I'm biased.

--
Steve Watt KD6GGD  PP-ASEL-IA          ICBM: 121W 56' 57.8" / 37N 20' 14.9"
 Internet: steve @ Watt.COM                         Whois: SW32
   Free time?  There's no such thing.  It just comes in varying prices...
Reply to
Steve Watt

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.