Sync + FIFO

It makes no sense to say that you are going to "synchronize the data bus with some register stages". What will invariably happen when you try to do that is that some of the portions of the data bus will occasionally exit the last stage on different clocks. Each individual bit of the bus may be synchronized correctly, but the bus as a whole will not, without taking additional measures.

Why do you need to synchronize the data coming out of the fifo? As you pointed out, the read clock is different from the write clock. So the work of synchronizing is already done for you. This is in fact a very good way of synchronizing a bus.

Just use an "asynchronous" FIFO design which is ideally suited for this application. It uses a dual-ported RAM to separate the two clock domains (write and read). And it can also handle the special cases of EMPTY and FULL. That's where the two clock domains meet and where any FIFO design gets tricky, We tested our Virtex-4 hard FIFO design (in every BlockRAM) at 200 MHz write and ~500 MHz read clocking for many hundred hours without any problems. Peter Alfke, Xilinx

While the idea of using core made by the FPGA companies like Xilinx Altera etc and while those core if done properly will give not only good performance but also minimum use of resources I believe in this case it is a fast solution which will work but a bad one.

Base on the way the question was put I believe the person want to have asynchronous FIFO however have no knowledge how to do it.

In such case, to use something which was done by other, without the understanding as how exactly the implementation is done and as such what the pro and con, mean maybe solving the problem but without getting the knowledge and understanding.

This is something like telling a new Math student who need to solve 1+1 or 5x7 not how to solve those equations but rather to buy a calculator ...

It might be a better solution if Xilinx have an App note as how to do it and provide it and only after this Eng learn and understand the App note and know how to make Async FIFO by himself, than he should use Xilinx Core or Altera or anyone else for that matter.

Have fun.

Berty, I disagree. While it is good for every young engineer to learn the basic skills, designing an asynchronous FIFO is far from "basic". Using a dual-ported RAM makes most of the design trivial, but the EMPTY and FULL detection and arbitration at high asynchronous clock rates is far from simple. I have a few patents and several magazine articles, and Clifford Cummings has published extensively. It is still considered wizardry, and grown men can get into deep arguments, since this involves Gray counter decoding and metastability. It is much safer to use a proven circuit designed by experts. It may sometimes be overkill, but it works, and lets you spend your energy on the really important and unique parts of your design. Peter Alfke, Xilinx Applications

Hi Peter,

As commercially biased as I am, I have to agree with you.

Designing a good dual-clock FIFOs can take many weeks of devising algorithms, testing them, throwing them out of the (for stress relief, preferrably closed) window etc. When I designed my first cross-clock bus it was on OTP parts, so I truly learned to simulate before burning, and even then it took me six or seven failing designs (equalling 3 fully equipped worth about $800 each) before getting it Right.

If you have three or four weeks of spare time and a powerful computer to research the subject, then by all means, please do find an efficient way to build a dual-clock FIFO construct we haven't thought of. Otherwise, please use $VENDOR's implementation. Project pressure usually is way too high to waste on a FIFO.

Best regards,

Ben

Excellent post IMO. We (who spend weeks coding complex designs) are _so_ happy that gifted engineers took the pain to build asynchronous stuffs that _work_, guaranteed :-) Clock domain crossing isn't usually a problem thanks to these little "cores" that we use everyday without thinking.

Interestingly, it seems that, today, fewer and fewer engineers are even curious to know how modern FFT cores are implemented internally !

Well, I believe there is still room for creativity and clever engineering, but the level of abstraction is just moving up one other notch. The challenge also moved towards verification, but I think there is no verification methodology which will ever fix a poor design.

But thanks again anyway to the creators of the async Fifos !!!

Bert Cuzeau

Thank you for your answers. The external data stream I am talking about comes form an USB transceiver which sends the data synchronous to 60MHz clk which I can use in my FPGA as FIFO write clock. Under this assumption of synchronous data stream the portions of the data bus will NOT occasionally exit the last stage on different clocks, will they ?

My idea was to use additional register stages to improve the performance of the data flow.

Rgds André

That should work fine (assuming of course the FIFOs are designed right). In this case, the FIFOs are doing the synchronizing, not the registers.

Peter, While I understand your point I'm afraid this is why you see so many Eng that know just about nothing except copy paste of IP module. Of course Async FIFO is not as simple as shift register. And of course it involve some thinking and I would strongly recommend anyone who want to design such thing or for that matter any new design to have DR of more experience Eng to see how he did what he did and see how to improve or fix the wrong. I'm fully aware of the Empty Full having designed several FIFO of all type and flavor. Except for the obvious of the advantage of knowing what you do and example as for why you should know to design by yourself can be that Some FIFO depend on the implementation when they have one last entry will toggle the empty while other will not. Sometime this toggle can be more useful however if you know Zilch about how the FIFO was design you can do nothing and have to adapt yourself to what ever the core give even if this is not the best for the design you do. And talking about synconizers and Gray counter etc while to use them correctly is important this is not rocket science, Sure to give complete and full explanation of what Metastable is and the effect of it in clear way and not just using "wave hand" explanation can be challenging but the actual implementation once you understand the meaning of it is not so difficult that one have to pass it aside and use other proven code. I guess it all boil down to are you an Eng who want to be a copy/paste one or are you an Eng who want to know how to do thing and yes ONCE you know use other if they make sense, but even for this to tell if it make sense you need to understand and not just be another copy/paste-Eng as more and more I for one encounter.

And all those who might give example on how they saved money, time etc by using other FIFO and not learning how to do it the right way are just an example as why you SHOULD learn and not just be copy paste one and use this as example to why to use other code.

Back to Math using your own logic is equivalent to say to Eng you should learn how to do 1+1 however to do integral of X^2 from 0 to 2 is to complex so use calculator, I do hope university will not go with this logic and those that do well maybe from there we get all the copy-paste Eng's.

Remember that any minute you "Waste" today for learn how to do it will pay thousands time in your future, when you have design which are not simple and there is no IP and you need to draw from your own experience, which if it involve only/mainly copy-paste without the knowledge mean you will never become ASIC leader or Architect of new complex designs and you will stay basic simple Designer, as no knowledge mean poor capability.

Interesting post too... however... I sure am always extremely frustrated when I see people coding in HDL and not knowing the basics, calling their VHDL code a "program", not knowing what crossing clock domains is about, etc !

But on the other hand, we are in a competing world, and as a Project Manager I would be VERY unhappy to see one engineer spend a week trying to write his own dual clock Fifo and SOMEWHAT scared about the result (through a synthesis tool in particular). As of what he would learn in the process (no pun intended) I'm not very sure (verification of such a beast is not obvious either). And the customer wouldn't see any advantage vs a design using ready-made Fifo, but he WILL see a tangible advantage if we spent an extra week implementing new innovative features (for the same budget).

But I sure taught my team all the many tricks I use when designing with FSMs. We designed our own FFTs, they used to be superior to the vendors' in many situations, but that's less and less the case, so we drop this know-how (for FPGA) and we more and more use the standard megafunction.

I would say that there is room in-between these extremes (knowing nothing and knowing everything)

In our courses, we do our best to teach really solid foundations and stay focused on the issues that will make a difference between good and poor design. To me it's safe enough if the engineer clearly understands the issues, & knows the principles of what the cores are made of and why, and knows which to use in each situation. I'm not sure it's so important anymore to fully understand the RISC instruction set in the context of SOPC. It's a long time I didn't code in assembly language.

My opinion indeed... Thanks anyway for this exchange of views.

Bert Cuzeau

is

And where do you stop? By your own logic, they can't just stop at doing integrals. You need to know how to derive all the integral short-cuts that you use, for fear that someday while waiting for a bus, someone holds a gun to your head and makes you re-derive the integral of sin(x)dx. But you can't stop there either - you have to move on to differential calculus and partial differential equations. But that's not all, then you need to... (etc)

Moving back to the electronics world, how about other components in the FPGA (or an ASIC)? Before I can use the PLL or CDR within the Rocket I/O, should I understand it so well that I could design one myself? What about the DCM? Do I need to be able to design my own latch-up resistant, ESD protected, discrete input and output buffers before I can use an ASIC vendors' IO cell? What about the optical transceiver that I connected to the Rocket I/O - do I need to know exactly how the pin diodes and laser drivers work before I can use an optical transceiver?

Everyone is going to draw the line in different spots, based upon their needs. Not everyone wants or needs to specialize in FIFO design. If you can do it, great - it gives you a slight competitive advantage for a few jobs out there (or turn it into a life-long job, as Peter has). But that alone does not indicate

Having said all this, however, I have to agree with some of the underlying points upon which you've placed your FIFO example... that engineers need a firm understanding of the basics so that when they are called upon to design something, they know where to start. Even more importantly, they understand the limits of their knowledge, and if necessary, know how to educate *themselves* further on the topic, without much outside help. That's the true sign of an engineer - not if they can design an async FIFO.

Have fun,

Marc

Well, I did design FIFOs,and in particular the asynchronous arbiter in the Virtex-4 BlockRAM FIFO ( and the test methodology for it), and I measured asynchronous behavior and metastable recovery. But these were major efforts, and were based on many decades of design experience. Xilinx can afford such exploration since we expect that many of our customers will eventually benefit from it... It's just not what you should entrust a junior engineer to do. Let them grow up with synchronous logic, and carefully graduate to data transfer across clock boundaries later. When you cannot simulate, you really need to be both experienced and confident Peter Alfke

I think you highly under estimate the average engineer. I believe this is because you only deal with people that can't get their designs to work. There are a lot more out there that never need help and you never hear from them. I would hate to think what engineers are designing logic that can't design an async FIFO(actually I know one, see bleow). My experience has shown that cut and paste does not provide the best performance, it may provide the best time to market as pointed out. When performance is more important than time to market you design everything yourself and highly optimize it. Apparently you only work with some below average junior engineers. We did have a senior senior engineer that couldn't design his own async FIFO, just goes to show there are below average people at all levels. That is the same type of mentality that would not hire a person with less than 5 years experience. I hope the people with that mentality end up with senior senior on their team.

Peter Alfke wrote:

Well, Bryan, here is a test for a smart engineer: Assume a 1K-address deep FIFO ( data width does not matter) implemented in a dual-ported RAM. Design only the EMPTY-flag detect circuit that operates reliably at totally asynchronous write and read clock frequencies of >300 MHz, and show a test circuit that proves that this operation is reliable. (Hint: The Virtex-4 BlockRAM does this job at up to 500 MHz worst case). Peter Alfke

Bryan -

Maybe I'm jaded since I'm a consultant, but a lot of engineers just don't understand subtle issues like clock domain crossing. I've had to clean up so many bugs in this area that I can't count them all. Just when you think you've seen every way to send signals between clock domains incorrectly, a new way is invented.

Same comments on signal integrity issues. Too many designers still think the world is only 1's and 0's. Reflections? Crosstalk? What are those?

Sorry for the minor rant, but if Xilinx can make async fifos easier, maybe the average designer can make reliable designs.

Yes, for custom case, you can may be able to create a smaller, more optimized design for a specific application, but for some engineers there effort may be better spent on the core of their design rather than re-creating a low level design block.

Peter - thanks for all the great input you have in this newsgroup!

John P

We are a Design house plus we train about 300 engineers per year, and I have exactly the _same_ experience a Bryan.

And what I see in the newsgroup doesn't tell me that every engineer writing VHDL does it with a clear understanding of these issues and of their solutions. I can only say for France, but here, the language and Electronic Design is very poorly taught even in the most prestigious Engineering schools (they don't even teach what Static Timing Analysis is !). We train lots of young engineers, and they have all the same feedback and stories.

In our consulting business we sometimes do kind of "PostMortem" on designs that fail miserably, and the issues end up having a very basic cause. Sort of 80-20 here too : 80% of the failures have a very low complexity reason and could be avoided by simple rules. The reason of failures (respins) in Asic design as reported year after year by Synopsys seem to say as much (with better figures, since beginners are not allowed here).

Another point I see is personal satisfaction. It's rewarding to create a complex working system in a small FPGA in a few days or weeks. It's no so satisfying to laboriously recreate an inferior version of something done by others and freely available. That's why, in our Educational Kit, I try to set up fancy exercises and stay away from Trafic Light Controllers. I think it's more fun to drive a thermometer chip, an LCD,or read a Smart Card, or create a UART, and not much more complex.

Bert Cuzeau

As a student that is about to enter the workforce in the US, I find that I am lacking many of the skills that were mentioned in this thread. I know what clock domain crossing is and I know that you can do it with an asynch fifo, but I wouldn't know where to begin in designing one. This kind of thing is simply not taught in school. Static timing analysis is also not taught. Honestly strong VHDL isn't taught either (and Verilog is not taught at all).

So my question to all of those seasoned gurus out there is where did you learn things like clock domain crossing and static timing analysis? Did you folks learn these things the hard way (screwing up and having to pay for it)? Did you take a course? Did you guess your way through it? I would love to learn these things, but frankly, I don't have a clue where to start.

You may be asking why a person like me could even think of entering into a marketplace without those skills, but I do have my other qualities and my university has a very strong analog circuits program. Also I have taught myself Verilog and have learned a lot of state machine things and synthesis things.

Hopefully those in the know can provide information for all of those students out there like me that don't know about advanced topics and don't have a resource for learning them and don't know where to begin looking for a resource.

And my input into this thread. I would say that the Xilinx built in async FIFOs are going to be faster than hand designed. The reason is simply that they have dedicated logic. If they are well tested (and I'm willing to believe they are) and they're faster, then it's hard to argue for spinning your own. The knowledge may be important (and I think it is), but if there's a faster, already tested implementation that is free, my pocket book and my time card are telling me that's the way to go.

Thanks for the input, Arlen

Arlen -

A good place to start learning about clock crossing and async fifos is by looking at app notes from companies like Xilinx. Simply realizing the problem exists and realizing it is important is a great first step.

Issues in clock crossing are flip-flop metastability, passing busses of data between domains, and Req/Ack handshaking between domains.

Metastability is discussed in many papers that you can look for on the web. Basically if you violate the setup/hold time on a flip-flop, it's going to be angry for a while, ie, potentially in a non 1/0 state. I t may hover at an inbetween voltage level, it many oscillate a bit, it may take longer to settle, etc. You r logic has to deal with this. Traditionally, a 2 stage flip-flop chain is considered good enough for this. This solution depends on clock and data rates!

Passing data/address/counters between domains is trickier. Assume you send an 8 bit value between domains. Because of delays in the gates and in routing, each bit of the bus arrives at the target flip-flops at a **slightly** different time. Because of this difference, when the targe clock tries to grab the data, it may latch some old data and some new data. In a perfect world, all the incoming bits would arrive at the exact same time and the target flip-flops would grab ALL of either old or new data. In the real world, the target flip flops will grab SOME new data and some old. So, if you are passing fifo address pointers between domains, you have potential problems in getting correct values at each clock edge. Solution: look into Gray coded values to pass back and forth. Since only one bit of a gray counter changes at a time, there is no problem with the receiving flip flops.

Sending Req/Ack handshake between domains is another area that can be trickier than one would like. There are efficient ways to do this, I'd suggest some Googling as part of your education.

Thought experiments in this area of domain crossing are very educational.

Clock crossing is one area where you MUST be paranoid. The problems in general are hard or impossible to simulate and hard to find in the real world. Since they are semi-random, you can spend a lot of effort tracking down a problem that only happens once an hour.

I hope this helps!

John P

I am not going to tell you that designing an async FIFO is hard, I hope this doesn't affect your wizard status.