Question on PCI-express verssus Standard PCI performance

One lane PCIe 1.x should be able to turn a word read around in about

250ns assuming not too much else is going on. Of course an excessive number of switches (or slow switches) or slow hardware on either end are obviously possible issues. But PCIe is certainly much faster than 3-4us to read a word.

I have no actual experience of experimenting with this, however, I have been interested in a latency sensitive device that may potentially use PCI-E so have been looking around for answers.

Have a look at this write up, of a comparison of HyperTransport and PCI-E. The authors claim around 250 nano-seconds (page 9) to read the first byte:

It would be interesting to hear what is causing you to see 3-4 us? That would kill off my potential project, so I am hoping to be able to match the results in the above paper.

Could there be some inaccuracy in your measurements; how do you measure the latency?

Rupert

When designing with PCI or PCIe you should really try to avoid reads as much as possible. What do you need it for anyway? In a multitasking operating system you are going to have microseconds of jitter on the software side in kernel mode and tens of miliseconds in user mode anyway. So I am wondering what the scenario is that benefits from sub us latency for software reads?

Kolja cronologic.de

Generally speaking PCI Express much more prone to latency than convertional PCI because packets have to be constructed, passed through a structure of nodes, and checked at most levels. Data checking isn't completed, and onward transmission, until last data arrives and CRCs are checked.

If you do a "read" this will have a packet outgoing and one coming back so doubly worse. If you can do a DMA like operation where data is sent from the data source and then interrupt your system to use the data in memory.

The latency will also vary from system to system because rooting structures differ between motherboards. The amount of other things going on will also affect latency as different things contend for the data pipes. Generally speaking if you are trying to do anything real time it is something of a nightmare if you are planning using the host motherboard processor for control functions.

You can try and make the latency smaller by using smaller packet sizes and this sometimes helps. Ultimately if there is a real time element to this then putting the processing and/or control on your card is probably best for performance and accuracy.

John Adair Home of Raggedstone2. The Spartan-6 PCIe Development Board.

In the paper I posted a link to, I think the times are for an interrupt, or for DMA, not a software initiated "read". Thanks for explaining the difference.

Rupert

Is it possible that time-stamping the data would disconnect you somewhat from the latency problem? Usually data can't be processed and presented real-time at those speeds anyway..