How to debug DMA problems

This question probably has to do with the fact that in many architectures the DMA can access the memory only by pages. If the DMA overruns the page border, it will wrap to the beginning of the same page and corrupt it.

Vladimir Vassilevsky

DSP and Mixed Signal Design Consultant

The first thing I would do is scrutinize the documentation for the DMA and understand any quirks. Yes, you can use a logic analyzer (or scope with logic probes) to see the bus activity. You can also do experiments with changing the block length, setup technique, etc. The important point is to be able to change code and see the results quickly.

Software for embedded system IS a major topic in this newgroup.

I realize software is a common topic in this newsgroup, but because of the name it implies a focus on hardware. I'm aware of comp.realtime and comp.programming but wanted to make sure I wasn't missing a more appropriate newsgroup.

And then there are cache coherency issues, where memory ranges used for DMA buffers must be flushed on "transmit" operations and invalidated on "receive" transfers. There are also issues with DMA buffers, cache line boundaries, and the potential for corrupting non-DMA data adjacent to DMA buffers.

The question is how do you debug problems involving DMA, not what are the issues. I am well aware of the issues but if you see that a particular DMA transfer is not behaving as expected, how do you start debugging? I have not heard of a single reply to this point yet.

How to debug depends on the issues. If you don't have a cache, then you don't have to worry about debugging cache coherency issues. Many DMA problems aren't even related to the DMA itself, but are just common software bugs that happen to involve some DMA actions. For instance, you could free() a network packet that's still in a queue somewhere, and is later used for some DMA transfer, possibly corrupting something else. Those bugs can be hard to find because the code involving the DMA and the DMA controller itself are working fine, and the actual mistake could have been made a minute earlier. There is not a universal method to catch all bugs involving DMA. It all depends on the way DMA is used in a system.

The last time I debugged some DMA problems, I modified the code that set up the DMA controller to verify the base address/length were both sane, and printed both of them to a remote log. As soon as a problem was detected in the memory contents, I would examine the past DMA transactions involving that particular area of memory. If I suspected a more complex interaction between modules, I would also add appropriate logging in other places.

... snip ...

You look at the source code, and think hard about corner timing issues. When you form a theory you create a jury rig to test it or to make the fault repeat. Practice cursing.

The first thing to determine is: is it a hardware or software issue ?. If you are bringing up a new piece of hardware for the first time, the hardware dma mechanisms need to be verified using software test harness etc. You use the test harness code to stimulate the hardware using random and other data patterns, together with scope and logic analysers to measure the results. Once you are sure that the hardware works, meets timing constraints etc, only then are you ready to start writng the application code. By this time, you are pretty sure the hardware is ok, so any faults can be found through standard debugging techniques.

DMA is largely a *hardware process* and programming should not be difficult if the hardware works and the programmer has a good understanding of the underlying transfer mechanisms and programming requirements...

Chris