1. Home
  2. Embedded Programming
  3. DMA issue while recording data on SATA HD

DMA issue while recording data on SATA HD

Hi there, Before I post my problem, let me present the overall picture. We have developed a realtime embedded data acquisition system based on A/D, FPGA, SATA controller, SATA HDD, microcontroller and Ethernet. The purpose of the system is to record seismic data from the sensors, digitize them and write them on the SATA HDD. Later, the recorded data can be retrieved through Ethernet. We have used Spartan 3 FPGA, Silicon Image SATA controller (Sil3512) and SATA HDD. There is no OS in our system, we have developed our own simple FAT, the FPGA acts as the host device which talks to the SATA controller through a dedicated PCI interface. There is no PCI slot on our system, we have used opencores PCI bridge on the FPGA which talks to the SATA controller through its PCI interface. There are two ping pong buffers in the FPGA, each 512 Kbytes. The digitized data is stored in these buffers in a ping pong fashion at 36 Mbytes/s. The FPGA programs the SATA controller DMA. The data is read out from the buffers at 66 MHz and transported on the PCI bus at 33 MHz towards SATA controller when the SATA controller initiates the read DMA. The system works perfect when the incoming data rate is 24 Mbytes/sec. But as we increase the data rate to 36 Mbytes/s (which is what we require), we see loss of data occurring. Here is the algorithm for data recording:

  1. Continue filling buffer 1 and buffer 2 with digitized data in a round robin fashion.
  2. As soon as buffer 1 is full, program the DMA for SATA controller so that it comes and reads out the filled buffer while buffer 2 is being filled.
  3. Wait for the DMA done from the SATA controller from the first DMA.
  4. On finding DMA done, wait for buffer2 to be filled.
  5. As soon as buffer 2 is filled, program the DMA for SATA controller so that it comes and reads out the filled buffer 2 while buffer 1 is being filled.
  6. Wait for the DMA done from the SATA controller from the second DMA.
  7. On finding DMA done, wait for buffer1 to be filled.
  8. Repeat steps 2-7 till the data acquisition is stop.

The data loss occurs due to delay in step 3 and step 6 where the FPGA waits for the DMA done signal from the SATA controller. At times, the SATA controller takes longer than anticipated and due to this, data buffers overflow. 36Mbytes/s should not be an aggressive data rate for the SATA controller + SATA HDD specially when the PCI link is dedicated only between the host and the SATA controller.. We have tried out different numbers with PCI bridge configuration and SATA controller PCI configuration but things have not improved. We are using UDMA 6 mode and 80 GB 7400 RPM SATA HD. Anyone out there to guide us how to handle this problem? Let me know if more information is required. Thanks in advance.

Best Regards

Reply to
maverick
Loading thread data ...

Op Tue, 27 May 2008 11:54:29 +0200 schreef maverick :

Indeed also because most modern disks have an internal buffer sized in the order of megabytes. Maybe you (or the DMA controller) are waiting for the physical disk write to complete, before continuing?

--
Gemaakt met Opera's revolutionaire e-mailprogramma:  
http://www.opera.com/mail/
Reply to
Boudewijn Dijkstra

=A0

he =A0

m/mail/

But these hard disks guarantee around 60-65 MBytes/s sustained throughput. That should include all these timings !

Reply to
maverick

te:

he =EF=BF=BD

ng for the =EF=BF=BD

Can you merge a number of successive writes which you now do as separate ATA write commands in a single one? That is, the drive will think you write say 20 buffers worth of length, you will only reprogram the DMAC on the fly. If the problem persists, the issue is likely something other than what Boudewijn suggessts. If this solves the problem, he is right (my guess is he is right, throughput and latency can differ dramatically).

Didi

------------------------------------------------------ Dimiter Popoff Transgalactic Instruments

formatting link

------------------------------------------------------

formatting link

Original message:

formatting link

Reply to
Didi

Are you writing to the inner or outer sectors on the HDD? This makes a large difference. For example the HDD that's in my PC right now will do 57 MB/s on the outer sectors but only 30 MB/s on the inner sectors.

If you can temporarily remove the HDD from your device and plug it into a PC, try benchmarking it with a program like "HD Tune" to get an idea of its actual performance.

If you need a sustained 36 MB/s on the entire surface of the disk, there are

10000 rpm SATA drives. You might also consider going larger than 80GB even if you don't need that much room since (all things being equal) a 500GB 3.5" drive at 7200RPM will usually be faster than an 80GB 3.5" drive at 7200RPM.

If that's still not good enough, just use two hard drives (if you have enough room in your enclosure) and alternate the writes.

--Tom.

Reply to
Tom

n a

ller so

t DMA.

roller

nd DMA.

As I seen the problem occurs with increased data rate then it might be also problem of the internal SATA controller design. I know from past it might be a problem of internal architecture. Some SoC vendors are pushing to save a cost for die. Usual way to write/read data to/from SATA controller (DMASATA transfer) is through internal FIFO in SATA controller. DMA writes/reads to/from FIFO of the SATA controller. If FIFO is too small (and I already saw such designs) then it easily causes stop of the data transfer while writing because FIFO gets easily empty if DMA is not capable to refil FIFO fast enough.I saw e.g. design of FIFO from CEVA in CPU from Freescale with only 64bytes

- compare with size of one physical sector of 512 bytes. So it's a problem of the arbitration between CPU and DMA while accessing SDR/ DDR. Size of FIFO just boosts problem. Every such stopped transfer (I think mechanism called HOLD, HLDA) requires retransmittion for few bytes. This definitely results in significant performance penalty. The same may happen while reading data, just a problem of full FIFO. Also make sure that filesystem adds some overhead, specially journaling filesystem, but basically any filesystem, because data is not sequential. More security/safety results in more overhead. Last but not least: do not forget about fragmentation. Fragmented HDD even with very simple filesystem gives you another significant penalty. You should do regular defragmentation. To do perfect measurement you would need SATA analyzer.

Reply to
martin.nkm

n a

ller so

t DMA.

roller

nd DMA.

Maverick, I forgot to write: In the case of free space in the cache of the HDD (memory which physicaly resides on HDD, usualy 8 or 16MB) you should forget about speed of 30-60MBytes/sec (sustained data rate), but you have to take into account speed of SATA interface -

150/300MBytes/sec, depending which SATA you have. In that moment you write into the cache at the full speed of SATA interface. Then wrong setup of arbitration between CPU and DMA while accessing SDR/DDR together with too small SATA controller FIFO may show up.
Reply to
martin.nkm

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.