Virtex 4 FIFO16 blocks - Corruption ?

Hi,

We're faced with a strange problem ... While investigating a bug in one design, we could only observe that behavior on real board and not in simulation.

Using chipscope, we finally traced down the problem by monitoring both write and read port of a FIFO16 configured as 18x1024, using the same rd/wr clocks. That fifo was used in a "weird" way, by setting a ALMOSTFULL threshold very high (but still within spec), so that it turn on very quicly. And what we observed is that we push a data with some parity bits (which are not 'true' parity but some critical control), we continue to push, the almost full goes up (normal), and we still push (we still have plenty of room) and at the same time we re-read but slower (not at each clock cycle) and when we finally re-read the data where the parity bit was set, the data (15:0) are there but the parity bit is not, it's just 0 ...

The chipscope 'probes' were tied directly to the fifo signals, no logic in between. That fifo is supposed to cross clock domains but for debugging, we just sent the same clock everywhere. And the behavior of the surrounding logic is consitent with that bit being missed.

Instead of using ALMOSTFULL set to a very high value, we used not ALMOSTEMPTY (here since we're debugging with just 1 clock domain, it's ok), and there it looks like we never observe such a miss.

Has someone ever observed such a behavior ?

Sylvain

Reply to
Sylvain Munaut
Loading thread data ...

Have you got any resolution on this? Have you opened a case with Xilinx? What does Xilinx have to say about it?

I am aware that some people have had problems with the FIFO16 not working correctly. I had an issue with trying to use the FIFO as a synchronous fifo (it is async, so there is a possibility with some ambiguity on the flag latency when both clocks are the same). I have asked Xilinx repeatedly to document this behavior prominently in the user guide, but so far they have only quietly acknowledged that the user has to be careful if read and write clocks are the same.

That said, your problem is different than the one I experienced and appears to be a more serious problem in the FIFO16 logic. You are not the first person I've heard state they had problems with the fifo16 async behavior. There may be some issues with the flag logic for asynchronous use as well.

I do find it interesting that Altera was forthcoming with their recent problems with dual port memories. I hope that Xilinx is equally forthcoming if there is indeed a problem with the FIFO16 logic.

Reply to
Ray Andraka

Ray,

The bug for use of the async FIFO synchronously has been acknolwedged, and we apologize for not getting it out there more prominently. But:

In our defense, it is unusual (or at least, so far we think it is unusual) where the read and write clocks are tied directly together (why use a FIFO at all? I guess it is a really useful structure, so even when used this way it is too useful to ignore....?).

The solution is to not source the two clocks from the same source directly, but place a small delay in one, or the other.

The problem does not exist in the asynchronous case, as it takes two subsequent clock cycles on BOTH clocks (at exactly the wrong times) to cause the problem. As long as the probability of two adjacent clock cyles not coming in on both clocks exactly the same just as you are getting full (or is it empty? I'm not the expert on this), it works fine.

Sometimes with problems like this (that are difficult to even cause) it doesn't make sense to put up a billboard that it is an issue, as then everyone comes down with the disease (mass hypochrondira) when they don't really have the problem.

Now, if the feature is just plain broke, then it is a different story, and we will end the pain as soon as we are sure it is just plain broke.

No one is intentionally hiding anything, but we are judiciously placing (obscure) bug information only with the hotline and support community, rather than broadcasting it across the entire user community publicy.

If, for any reason, you feel that you have caught the disease (have a bug we haven't shared universally), the entry of a webcase will get you the help you need, as the hotline will search for all such issues. If yours is there, then we will immediately share with you the solution.

These are known as "internal answers" and it isn't that we don't want to share them, we just don't think they are likely issues for everyone. Better to talk to you and find out what the problem is, first.

If these internal answers are made external, we imagine there would be thousands of designers running down debug paths that are so obscure, there is almost no chance they will find this as their problem. Then we get a bad reputation, and the hotline is overwhelmed with folks who all think they have this obscure problem!

I hope folks will appreciate that sometimes telling every strange and obscure story causes more trouble than selectively understanding each issue that arises, and dealing with it directly.

Support: it is an art.

Austin

Ray Andraka wrote:

Reply to
Austin Lesea

Austin,

You are kidding as far as the usefulness of a synchronous fifo (one which has both sides clocked by the same clock), right? This is a rather common structure in pipelined designs, it is an elastic buffer. Useful, for example, for processing bursty data at a more relaxed rate than the data is presented. I'd be hard pressed to find one of my designs that does NOT have a synchronous FIFO in it. The solution with the "small" delay is fine if you are not pushing the performance envelope, but it will destroy timing closure in designs that are. For example, I have a floating point FFT design with a target clock rate of

400 MHz in an SX55-10 part... basically running at the DSP48/memory speed. It has synchronous FIFOs in it, and there is no room in the timing for adding small delays to clocks. This is a real limitation to the FIFO16 design, and has cost me several weeks of debug and redesign time to find and work around it. It should be prominently highlighted in the user guide under the section that describes the used of the FIFO16. I am sure other users are going to encounter the same issue. No one looks at the answers database until they have a problem and have identified the source of the problem. The synchronous FIFO issue could easily be considered a limitation rather than an outright bug, but it does have to be made clear to the user before he does the design, not when trying to figure out why it isn't working. By keeping it close to your chest as an internal answer, I suspect you'll wind up generating a heck of a lot more hotline cases than if you put it in black and white right in the user's guide that this is the way the FIFO16's work and that these are the things you need to do to work around the limitation if the clocks are the same on both sides. BTW, I don't think this is an "obscure" issue either, as anyone attempting to use the FIFO16 as a synchronous FIFO is going to encounter it.

The flip answers regarding the synchronous FIFO (things like such a structure is not useful, and just add delays to the clock when I've explained that it is not a viable solution for maximum performance designs), combined with the reluctance to make it clear to users that this is a limitation of the FIFO16 design, makes it appear that either Xilinx doesn't understand the issue or that they are trying to sweep it under the rug. I presume and hope it is the former, although neither is a particularly good outcome.

I am reluctant to enter a webcase on an issue such as this unless it has become critical for the project. Invariably, the result of entering a webcase is my having to generate and submit testcases to prove the problem, and often having to come up with my own work-around because the fix won't be available until the next major release. Nobody pays me for the time spent doing testcases to ferret out the source of a bug in the software or silicon. There have been months recently where I've spent more than a quarter of my time identifying and generating test cases for problems in the tools (not just Xilinx). Naturally, I'd like to avoid that as much as practical.

Regarding the asynchronous FIFO behavior, I don't have any direct experience with the FIFO16 behaving badly as an async FIFO, But I haven't used it in that mode in a design that has made it to testing. Silvain's description does sound as though the FIFO may be misbehaving, and it jives with things I've heard from others. This is why I asked him if he had opened a case with Xilinx and what the resolution of that case was. It is important to know if there is a potential problem so that I can avoid it during the design rather than discover it during integration. I am currently working on a design that has several async FIFO16's in it, and would like to believe that they will work for me, however these rumblings have me concerned, hence my asking Sylvain about his resolution. So far, the work arounds I am aware of have used the coregen FIFO instead of the FIFO16, which does not have the same clock performance as the FIFO16.

I didn't intend to kick over the beehive here, I was only trying to collect more data so that I might avoid a problem in my own design if it does exist.

Aust> Ray,

Reply to
Ray Andraka

My colleague had some contact with our distributor but afaik, no news yet.

Looking at the xilinx answer record , I saw that fifo usign FIFO16 blocks generated with an old version of fifogenerator could show some datacorruption problem and that usage of the new one is recommanded ... but I didn't use coregenerator, i instanciated FIFO16 directly (coregen doesn't have first word fall thru anyway ...)

I haven't opened a webcase myself yet, ... often before "bothering" xilinx peoples, I want to be sure ;p I've tried to reproduce the problem with a far simpler design but so far no luck ... (even in the full design it's quite "rare" but 1 times suffice to lock it ...)

What exactly is the problem if the clocks are the same ? (what behaviour could happen ?)

Well, here we use the fifo synchronously ... They are meant in the future to be used asynchronously but for testing, we've put everything at the same clock. But other part in the design will always use them synchronously so I must get it working in both mode ...

Sylvain

Reply to
Sylvain Munaut

Where can I get detailled infos about it ? (to be sure not to run into it, or at least that it doesn't cause trouble in my design ?)

Sylvain

Reply to
Sylvain Munaut

Sylvain,

I expect the fastest way is to open a webcase requesting the information.

As I already stated, if both read and write clocks are from the same BUFG net, then this may (will) probably be an issue at some process/voltage/temperature corner (hence the indsidiousness of the issue).

A quick fix is to drive one of the clocks from the other edge (one rising, one falling) which may require another BUFG resource (in order to be sure the delay doesn't put you right back where you started).

It is my understanding that a macro will be created to instantiate the sync FIFO with the required offset delay automatically in the best way we can (probably using fabric resources, like a LUT, doubles, hexes, etc.).

The issue as I was told is that at the critical instant, the almost full/almost empty flag assertions will be correct, but if the event occurs again on the very next clock cycle, the flag will reset to 0, which will not be correct (as the FIFO is still almost full, or almost empty if nothing was done to read anything out, or write anything in on that cycle).

There may be other simpler solutions (that we haven't thought of yet).

Again, the jury is out on the async case....

Aust> Aust>

Reply to
Austin Lesea

This is exactly what I mean by the problem being hidden. I searched the answers database for FIFO16, and did not turn up anything regarding the known synchronous behavior problem, nor any async problems. It may still only be in the internal database, if it is even there. In debugging stuff like this, I've always assumed the silicon is good and that any problems are a result of the design until I can prove otherwise. As a result, you don't suspect the FIFO itself as being the problem. That can lead to a tremendous amount of debugging effort before finding out there is a problem or unpublished limitation with the silicon. Considering how much time I spent fiddling with this problem, I suspect there are literally thousands of manhours put into debugging the same problem in different projects simply because Xilinx doesn't want to advertise a limitation with their design.

The problem with the synchronous usage is that the flag circuit is an async design. When the clock is the same to both sides, and a read and write are done on the same clock cycle, the flag circuit displays a one clock jitter in the timing of the flag outputs, such that the word written in at the same time the last one is read out may or may not make the fifo show empty. If empty does get set, it then takes something like 3 clocks to go away, so you wind up with a non-deterministic behavior. It is an artifact of using an async flag circuit.

BTW, finding stuff in the answers database is a lot like finding a needle in a haystack, provided you even know what you are looking for.

Reply to
Ray Andraka

Ray -

I've got to agree with you that finding stuff in the answer data base is hit-n-miss at best. I use XIlinx parts in my designs and I like the parts, but, the support from the web site leaves a lot to be desired. Thank goodness Peter and Austin pay attention to this group!

I often think that web masters should be forced to sit with users for a while so they end up understanding how slow and poor the user experience is.

My most recent frustration was trying to find information on a DCM bug requiring the bitgen centered option. Good luck finding info on it.

John Providenza.

Reply to
johnp

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.