Survey: simulator usage

Apologies for the (slightly) commercial nature of this post.

Those of you with long memories may recall coding devices in ABEL. One nice thing about ABEL was that you could write very simple vector files to simulate your device, where a vector was something like

[C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]

ie. setup some inputs, apply a clock pulse, and check some outputs. I've written some software over the years that lets me do the same thing in VHDL, with various extensions, and I use it to test most of my RTL code. It's simple, you don't need to write or know *any* VHDL to use it, and it gives you a pass/fail very quickly, for a module or a whole device.

I'm thinking about brushing this up a bit, adding Verilog support, and flogging it for maybe 100 - 300 USD a go. To use it, you obviously still need a simulator - the software currently produces VHDL-only output, and uses your simulator to simulate your chip using the auto-generated verification code.

This brings me to my problem. I can make the software a lot more sophisticated if I can generate C code, as well as (or instead of) the VHDL or Verilog. There are some testbenchy things which are just very difficult to do in pure VHDL or Verilog. *But*, most of the potential users of this software will be FPGA coders with a cheap simulator that doesn't support a C-language interface (ModelSim PE/VHDL on Windows, for example, doesn't do this, and presumably the FPGA-specific simulators don't do this either).

What I'd really like to find out, if you can spare the time and this might be of interest to you, is:

  • What simulator do you use?

  • Is your RTL code in Verilog/VHDL/both?

  • Does your simulator have a C-language interface? From Verilog, or VHDL, or both?

  • If your simulator doesn't support C, would you be willing to upgrade it to use a product of this sort? Or would you prefer to get pure VHDL or Verilog out of this software, even if it means reduced vector file functionality?

As a bonus, if you add the line "this is a great idea and I claim my

50% discount", then you can have 50% off the (initial) purchase price, if I ever get around to doing this.

You can reply here or directly to me at 'unet+50' 'at' 'riverside-machines' 'dot' 'com'.

Thanks -

Evan

Reply to
Evan Lavelle
Loading thread data ...

What about supporting Tcl, most if not all simulators support this language. You can force and examine signal using Modelsim's Tcl.

Such as?

You can get a SystemC license for Modelsim PE which IMHO is much much easier to use than the FLI/PLI.

Modelsim SE.

VHDL.

Yes, SE comes with the FLI which I have used in the past until I discovered SystemC :-)

No since I don't understand the advantages of your product, also changing PE to SE or adding a SystemC license to PE is not particular cheap and it might be more cost effective to spend some more time on your testbench :-)

Hans

formatting link

Reply to
Hans
Reply to
Stephen Williams

So this requires you generate all outputs ? If the sim does fail, what format is the failure report, and how does the user relate that back to a given line of test source code ?

Interesting idea. What about doing a simple VHDL/Verilog/Demo-C version that is free, and a smarter version that is $$ ?

A problem with this type of tool, is explaining to each user how its feature set can offer more to a given task, so the free/simple versions ( and good examples) do that.

For those who want it, I think most? FPGA vendors have (free) waveform entry for simulation entry, so that is one competition point.

There is also this work by Jan Decaluwe, that uses Python for both verilog generation, and testbench generation.

formatting link
formatting link

-jg

Reply to
Jim Granville

If you really mean such a simple format, my answer is: don't bother, that way is obsolete for anything but the most trivial designs. Given the amount of testing and the complexity of many interfaces/busses today, I need to write my test vectors at a higher level than applying a bitvector per clock cycle.

In order to do that, I normally create a library of high-level functions in a scripting language like Perl or Tcl, which would do "the dirty work" for me. Normally I would make the script library interface directly to a VHDL/Verilog testbench through TEXTIO files. With your extention, I would still have to make the script library, but just have a "backend" to that emits files compatible with your format. All I'd save is to write my VHDL testbench for the interface (which may could have serious performance impacts, since you make the simulator stop for each clock cycle and is a real no-no for performance), and frankly, writing the VHDL testbench isn't the problem.

Affirmative (at least for the XE version).

Modelsim PE / XE

VHDL, but we have to deal with Verilog as well due to backannotated GL sims.

Considering that the list price of Modelsim SE is 170K DKK vs 70K DKK for a PE license (both floating licenses), not bloody likely.

With a little imagination, you can do a surprising amount intelligent work with "dumb" TEXTIO vector files, by carefully dividing the work between script library (vector generation) text files, and the testbench itself.

Kai

--
Kai Harrekilde-Petersen
Reply to
Kai Harrekilde-Petersen

This sort of test-vector entry is basically unmaintainable, and to be honest, I haven't written this type of go/no-go test in ages (maybe since the last time I used ABEL).

I prefer to write test benches at a higher level. I like to have models of the things to which the DUT connects drive the DUT. For example, if my FPGA is a NAND flash controller that sits between a microprocessor bus and the NAND flash devices, I'll use a bus-functional model of the micro and the flash, and my test bench will mostly consist of MicroWriteBus() and MicroReadBus() "commands."

I can't go back to the simple drive-a-vector test.

-a

Reply to
Andy Peters

NC-Verilog

Verilog

I'm sure it does

Frankly, as an ASIC designer. I probably wouldn't use this language. In the future, we plan to use System Verilog.

Reply to
bronzefury

Only if you actually need to test them. If a given vector file wants to check a specific port, but that port is a don't care for most of the vectors, then you just enter a '-' for that port in the expected outputs. You could also specify the expected outputs algorithmically, if possible: you can declare variables and do arithmetic on them, and use them as module inputs or test values.

Here's an example from a chip I've just done. Line 392 of one of the vector files looks like (note that I've put in a line break to try to avoid wrapping problems on your newsreader):

[11C 1 0011 0 5A5A5A5A xxxxxxx xxxxxxx zzzzzzzz zzzzzzzz] -> [-0 -------- ---- EE FF 011 5A5A5A5A 011 5A5A5A5A ------- -- --]

If I change the last '5A5A5A5A' on the expected outputs to '5B5A5A5A', then ModelSim reports:

# ** Note: L2HDB error: line 392 (expected 0x5b5a5a5a, got 0x5a5a5a5a) # Time: 4381 ns Iteration: 0 Instance: /testbench # ** Note: simulation complete: 305 out of 306 vectors passed. # Time: 7007400 ps Iteration: 0 Instance: /testbench

The pin's identified, the line of the vector file, and the time.

Hmmm... maybe a 'linear' version (ie. no smarts) for free, and looping/variables/procedures/macros etc. for $$? My recollection is that ABEL just gave you the dumb version, but it seemed useful at the time.

Yes, explaining that this is (hopefully) useful is difficult. I went through this with a friend recently, who never writes testbenches, and who spent years on ABEL. He wasn't impressed... :(

I've never got this. Does anyone actually do that?

Yes, I know of Jan's stuff, but hadn't realised he also does TB generation - I'll have a look at that.

Evan

Reply to
Evan Lavelle

Yes, makes sense; I've got no problem with high-level testbenches. If a device or module has lots of buried state, and it's not obvious when any outputs or observable points should change as a result of an input change, or even *what* that change should be, then a test vector approach is pretty useless. If your problem is, for example, apply an opcode to a CPU and check that a memory location changes at some arbitrary later time, then you need to be a lot smarter. But, horses for courses. Consider:

- Your complex DUT is probably built of simpler modules. If a module was just an FSM, a register block, a decoder, or a bus interface, for example, and you could test the module simply by writing

20/50/100/whatever vectors, would it be worthwhile? Or do you just stitch the whole thing together and test the top level?

- Many complete chips are surprisingly simple (seen from the outside, anyway). I first used this code for a (large) DSP chain for baseband processing. I got MATLAB data for the inputs, with expected data for the outputs. Once I'd automated the test, verification was trivial. Seven years on, and I've just finished part of a Spartan-3. Nothing fancy - 25K marketing-gates - it just translates accesses between two different buses connected to 15 external chips, handles bursts, has a register block with control and status ops, does some address mapping between the various chips, and so on. I did a basic test in a day by writing 200-odd vectors, and I had a complete-ish test of 752 vectors with a bit more work. Whatever way I'd tested it, these 752 vectors would have existed somewhere, if only implicitly in the VHDL code, or even in the documentation. Why not make the vectors themselves the test? The rest of it is just unnecessary verbiage that you have to repeat again, and again, for each new module and device you do.

- What are MicroWriteBus() and MicroReadBus()? I can do macros and pass parameters to the macros; you can call the macros from wherever you want in the vector file. I can also do basic C-like control structures - looping, branching based on tested values, and so on.

Ok, is it worth any more than $0 now? :)

Evan

Reply to
Evan Lavelle

Ok, here's a really trivial example. You've just coded a D-type F/F - how do you test it?

- Option 1: code a testbench module, instantiate your D-type, provide a clock and some way to apply inputs (in this case, probably just explicitly driving the pins sequentially as required), check that the output is as required, inform the user that the test passed or failed. If it failed, tell the user where. Not difficult, but tedious. Also much more tedious if you need to do a timing sim, and you have to ensure that the inputs are driven only for the required setup and hold times, and the outputs only change between tCOmin and tCOmax. It's also not productive; this is all code you've written a thousand times before. And it may have a bug in it, and the testbench itself must be tested before you can use it to test your DUT.

- Option 2:

([CLK, RST, D] -> [Q, QB]) [x, 0, x] -> [0, 1]; async reset [C, 1, 1] -> [1, 0]; clock 1 [C, 1, 0] -> [0, 1]; clock 0

As I said, it's trivial. You don't need to know any VHDL or Verilog, and you don't have to write a test program. All that tedious stuff has been done for you. Of course, You're not going to test a D-type, but there are certainly far more complex cases where this is useful. And there are more complex cases where this isn't useful; see my reply to Andy.

Evan

Reply to
Evan Lavelle

Why using a code generator when you can write such bit vector tests within VHDL itself?

constant rom_test: unsigned(63 downto 0) := x"cf000000e4ec2933";

rom := rom_test; for j in 1 to 64 loop wait until ds_wire = '0'; wait until ds_wire = 'Z'; if rom(0) = '0' then ds_wire

Reply to
Frank Buss

ABEL is still supplied with some tool flows. CUPL is also still usefull for CPLD end of the design, and it is both free, and has some looping/macros in the functional Sim engine.

CUPL is functional-sim only, so has no timing smarts itself, but you CAN take the fitter (verilog/vhdl) output, and run a testbench on that.

CPLD tend to be less of a timing puzzle than FPGAs so this extra flow is not used much.

What CUPL _does_ offer, is test vectors into the JED file, which is good for smaller devices that go via device programmers.

CUPL also allows a '*' in the vector input, which means CUPL generates (fills-in) the H.L.Z in the output, and that can speed the test process.

If you then want to lock in all test values, you can paste the SimOut values back into the SimIN file, but I tend to scan the SimOUT carefully, and then rely on the same results being generated from the same stimulus. (These are stable tools.)

Here is a simple example: SimIN

0C1* ** **** *** **** 111111 ****** ****** ***** C * SimOUT 0C1L LL LHHH LHL ZZZZ 111111 LLLLLL LLLLLL LLHHH C L

I have a friend who thinks this is great. He does a lot of BUS interface work. Personally, I prefer using a text editor, and command line tools. I can see the appeal of waveform entry, but the custom-file risk of that outweighs the benefits, for me.

-jg

Reply to
Jim Granville

Shorthand notation for "generic test-bench procedure/task call that performs a read or write on my microprocessor's external bus, from the micro's point-of-view."

-a

Reply to
Andy Peters

In a word, no.

Why go to the trouble of learning a new language to try to do things like macros, loops, random stimulus, etc. when you have the power of the VHDL language at your disposal in a VHDL testbench? Now, if you have vectors from an external model/simulation, those can be applied with text-io relatively easily from within a vhdl testbench that will run on any vhdl simulator.

My "unit level tests" are usually at a high enough level that I need a lot more capability than is available in any vector based scripting language.

Andy

Reply to
Andy

Generalising the question of this discussion I would like to know what does the group here think that code generators are worth?

Say Someone comes out with a code generator which requires some input data say, D_in and generates a bunch of output code say, C_out.

Given that C_out is something that you already write for your design wether the code generator exists or not, when will the code generator look like a fesible option?

Given the following man days(Including coding, verfying and debugging time) for writing D_in and generating C_out VS manual writing of C_out, what is the price you will be willing to pay for the code generator in each case?

--------------------------------------------- Development time in Mandays

--------------------------------------------- D_in : Manual C_out

---------------------------------------------

1 : 10 1 : 100 1 : 1000

If such a code generator appears > Evan Lavelle wrote:

Reply to
Thomas

Personally, I would avoid code generators to generate RTL for design. Coding RTL is not a bottleneck for me, even on multi-million gate designs. Some IP is provided by 3rd-party, other IP our developers code. It is fairly easy to write Verilog once you have tried and true coding style. Why would I want to learn a proprietary language to generate a standard language? (and get locked-in?). In addition, I know exactly what hardware I'll generate when I code. For logic circuits that are used often, the HDL addresses that by allowing designers to create modules or primitives. Who knows what a code generator will output. I also wouldn't want to be locked in to a proprietary.

System Verilog addresses many of the my department's needs.

Even if a big name EDA company were to come out with one, I doubt this type of tool would be purchased.

I think if you want to attack something in the area of chip design, you've got to address the parts in the design flow that addresses a company's development bottlenecks such as verification or physical design.

Reply to
bronzefury

I think you may be in even more trouble than me...

You need to be specific about what you're generating. A human is a code generator; are you replacing humans? Are you automating something that's already done? How many people do it? How much time do they spend on it? Can you do it better than a human? Or just faster? Or just cheaper? Or is it too difficult for a human to do at all?

Are you generating IP? If so, you're in trouble. People don't like buying IP, and they'll only buy it from someone they already trust. The IP itself is unimportant, anyway: what matters is how well it has been verified, and whether you can prove it. You're selling a verification plan and coverage metrics, not HDL code.

Who are you targetting? Is it FPGA users? If so, you're shafted. If not, you've got no market. Google for anyone who's tried to sell IP for FPGAs; I can't believe anyone's ever made money doing this. If the users like it, the FPGA vendors will do it themselves and sell it for

*zero*. Remember, the vendors are selling silicon; they need to bundle software and IP for free to get market share. If you don't understand this, think Microsoft and Netscape, WMP, etc.

Finally, remember that the EDA market is tiny, and 3 mid-sized fish are desperately trying to make a living in it, primarily by eating all the smaller fish. If the FPGA vendors don't get you, then the EDA vendors will. The men's underwear market is bigger than EDA. Perhaps you should be designing a better pair of Y-fronts...

:)

Evan

Reply to
Evan Lavelle

There are different levels of generated code. e.g.

1> Schematic to code generation e.g. FSM, block diagram,module integration. 2> Translating code from one format to another. 3> Template based code generators etc. While some are simple (have Coding RTL is not a bottleneck for me, even on multi-million gate designs.

Applying the 80-20 rule, I would say that 80% of any code is bug free but identifying the other 20% can take up 80% of my development time. so would it not be in my interest to reduce the amount of hand written code by generating code wherever possible? Also quite a few bugs are because of copy, paste, modify errors whihc a template based code generator can easily fix.

If we are generating RTL the generated code will be verilog or VHDL and if it is a testcase it will be e,c,SystemVerilog or SystemC and if at any time we want to "break out" of the code generator "lock-in" we can always take the generated code and continue from there.

Note: I dont have a code generator to sell. But have seen various reusable code generators written by my team mates which have saved weeks and weeks of coding+debugging effort and was wondering what will it cost in the market. Regards Thomas.

Reply to
Thomas

We also had some inhouse people working on these kinds of tools. Some issues we ran into are that code would be written incorrectly or a bug found, then not only would the RTL have to change, the code generator would have to change as well. Which means you're debugging two things and requiring more people and time to develop a product. To make a long story short, for us, it was a big waste of time because the time they spent on it would have been better spent verifying the 20% that you speak of, understanding performance bottlenecks, planning for future product, training, or fine tuning an architecture.

I really think code generators are a tough sell. I can't see the value proposition. Coding just isn't on the top ten list of headaches for me or any of the designers that I work with. Again, verifying takes up most of the development effort.

From your resp>> Personally, I would avoid code generators to generate RTL for design.

Reply to
bronzefury

I am looking at a problem set where We tend to do the same task over and over again for each project. After doing it a few times we know all the possible permutations and combinations and which combination to apply for which problem but it still consumes most of our coding time. Some of the tasks in the problem set may require one person to be dedicated for it throughout the project cycle.

The generated code could be similar to what is written by a human and only reduces the development and debug time. For e.g. it may do the same job with the benifit of giving 10X to 1000X improvement in the coding+debugging time.

As an example think about writing y=a+b; and using a tool for synthesis and layout. VS using a layout editor to design the ckt one CMOS transistor at a time followed by spice simulations. If this were the 70's will you pay for the current crop of synthesis+layout tools?

While my company is currently working on a set of IP's for its client. My question was more towards the market fesibility of code generators beacause every year EE Times covers a couple of developers who have tried to sell their solution to specific problems in IC design which could be replicated in a a few hundred lines of perl code.

What I am looking at is a set of tools which help in taking the project from Spec to samples to Product faster than the current manual process would allow. So It could involve

  1. The methodology.
  2. The code generators to generate verbose codes from a brief high level input.
  3. Resolving any other bottlenecks.

Going with the Microsoft analogy below. I am looking at something similar to windows apps development using visual C++ where you fireup the relevent wizard click a few buttons and are presented with a template based code framework runnning in a few tens of thousands of lines in which you insert your alogrithm and other code fragments.

Note:I know about the code bloat issue for MS VC which a tool for HW design cannot afford.

The audience for such tools could be both FPGA and ASIC users. I know about the risk any successful product will have from the established EDA vendors copying it and offering it for free. so my question of wether your manager will buy it from a no name developer or wait for a Known EDA Vendor to come up with a me too product.

I dont think any developer will regret beig eaten by a big name EDA vendor if the price is right :) Also I think EDA vendors are currently more interested in me having a bigger code base to verify so that meeting coverage goals will require more run time or more simulators.

After a decade in the Industry I think I am too old to change professions. I will make sure that my kid keeps his options open ;) Regards Thomas

Reply to
Thomas

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.