Survey: simulator usage

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Apologies for the (slightly) commercial nature of this post.

Those of you with long memories may recall coding devices in ABEL. One
nice thing about ABEL was that you could write very simple vector
files to simulate your device, where a vector was something like

[C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]

ie. setup some inputs, apply a clock pulse, and check some outputs.
I've written some software over the years that lets me do the same
thing in VHDL, with various extensions, and I use it to test most of
my RTL code. It's simple, you don't need to write or know *any* VHDL
to use it, and it gives you a pass/fail very quickly, for a module or
a whole device.

I'm thinking about brushing this up a bit, adding Verilog support, and
flogging it for maybe 100 - 300 USD a go. To use it, you obviously
still need a simulator - the software currently produces VHDL-only
output, and uses your simulator to simulate your chip using the
auto-generated verification code.

This brings me to my problem. I can make the software a lot more
sophisticated if I can generate C code, as well as (or instead of) the
VHDL or Verilog. There are some testbenchy things which are just very
difficult to do in pure VHDL or Verilog. *But*, most of the potential
users of this software will be FPGA coders with a cheap simulator that
doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
for example, doesn't do this, and presumably the FPGA-specific
simulators don't do this either).

What I'd really like to find out, if you can spare the time and this
might be of interest to you, is:

* What simulator do you use?

* Is your RTL code in Verilog/VHDL/both?

* Does your simulator have a C-language interface? From Verilog, or
VHDL, or both?

* If your simulator doesn't support C, would you be willing to upgrade
it to use a product of this sort? Or would you prefer to get pure VHDL
or Verilog out of this software, even if it means reduced vector file
functionality?

As a bonus, if you add the line "this is a great idea and I claim my
50% discount", then you can have 50% off the (initial) purchase price,
if I ever get around to doing this.

You can reply here or directly to me at 'unet+50' 'at'
'riverside-machines' 'dot' 'com'.

Thanks -

Evan

Re: Survey: simulator usage

..

Quoted text here. Click to load it

What about supporting Tcl, most if not all simulators support this language.
You can force and examine signal using Modelsim's Tcl.

Quoted text here. Click to load it

Such as?

Quoted text here. Click to load it

You can get a SystemC license for Modelsim PE which IMHO is much much easier
to use than the FLI/PLI.

Quoted text here. Click to load it

Modelsim SE.


VHDL.


Yes, SE comes with the FLI which I have used in the past until I discovered
SystemC :-)

Quoted text here. Click to load it

No since I don't understand the advantages of your product, also changing PE
to SE or adding a SystemC license to PE is not particular cheap and it might
be more cost effective to spend some more time on your testbench :-)

Hans
www.ht-lab.com




Re: Survey: simulator usage

Quoted text here. Click to load it

Ok, here's a really trivial example. You've just coded a D-type F/F -
how do you test it?

- Option 1: code a testbench module, instantiate your D-type, provide
a clock and some way to apply inputs (in this case, probably just
explicitly driving the pins sequentially as required), check that the
output is as required, inform the user that the test passed or failed.
If it failed, tell the user where. Not difficult, but tedious. Also
much more tedious if you need to do a timing sim, and you have to
ensure that the inputs are driven only for the required setup and hold
times, and the outputs only change between tCOmin and tCOmax. It's
also not productive; this is all code you've written a thousand times
before. And it may have a bug in it, and the testbench itself must be
tested before you can use it to test your DUT.

- Option 2:

([CLK, RST, D] -> [Q, QB])
[x, 0, x] -> [0, 1]; async reset
[C, 1, 1] -> [1, 0]; clock 1
[C, 1, 0] -> [0, 1]; clock 0

As I said, it's trivial. You don't need to know any VHDL or Verilog,
and you don't have to write a test program. All that tedious stuff has
been done for you. Of course, You're not going to test a D-type, but
there are certainly far more complex cases where this is useful. And
there are more complex cases where this isn't useful; see my reply to
Andy.

Evan


Re: Survey: simulator usage
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Evan Lavelle wrote:

Quoted text here. Click to load it

Cheap/free simulators *should* support some sort of C interface.
Icarus Verilog supports VPI and I'd be astonished if the bundled
Verilog compilers w/ FPGA vendor tools doesn't also support VPI.

- --
Steve Williams                "The woods are lovely, dark and deep.
steve at icarus.com           But I have promises to keep,
http://www.icarus.com and lines to code before I sleep,
http://www.picturel.com And lines to code before I sleep."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFFQjburPt1Sc2b3ikRAlWSAJ4z5s8Hn9mQ4B0U8IevKZzmOsCkDQCg500z
EziQS9sPImvMVTE3FP0T3f0=
=tv0x
-----END PGP SIGNATURE-----

Re: Survey: simulator usage
Quoted text here. Click to load it

So this requires you generate all outputs ?
If the sim does fail, what format is the failure report, and
how does the user relate that back to a given line of test source code ?

Quoted text here. Click to load it

Interesting idea.
  What about doing a simple VHDL/Verilog/Demo-C version that is free, and
a smarter version that is $$ ?

  A problem with this type of tool, is explaining to each user how
its feature set can offer more to a given task, so the free/simple
versions ( and good examples) do that.

  For those who want it, I think most? FPGA vendors have (free) waveform
entry for simulation entry, so that is one competition point.

  There is also this work by Jan Decaluwe, that uses Python for
both verilog generation, and testbench generation.
http://myhdl.jandecaluwe.com/doku.php/start
http://myhdl.jandecaluwe.com/doku.php/cookbook:stopwatch

-jg


Re: Survey: simulator usage
On Sat, 28 Oct 2006 08:18:51 +1300, Jim Granville

Quoted text here. Click to load it

Only if you actually need to test them. If a given vector file wants
to check a specific port, but that port is a don't care for most of
the vectors, then you just enter a '-' for that port in the expected
outputs. You could also specify the expected outputs algorithmically,
if possible: you can declare variables and do arithmetic on them, and
use them as module inputs or test values.

Quoted text here. Click to load it

Here's an example from a chip I've just done. Line 392 of one of the
vector files looks like (note that I've put in a line break to try to
avoid wrapping problems on your newsreader):

[11C 1 0011 0 5A5A5A5A xxxxxxx xxxxxxx zzzzzzzz zzzzzzzz] ->
[-0 -------- ---- EE FF 011 5A5A5A5A 011 5A5A5A5A ------- -- --]

If I change the last '5A5A5A5A' on the expected outputs to '5B5A5A5A',
then ModelSim reports:

# ** Note: L2HDB error: line 392 (expected 0x5b5a5a5a, got 0x5a5a5a5a)
#    Time: 4381 ns  Iteration: 0  Instance: /testbench
# ** Note: simulation complete: 305 out of 306 vectors passed.
#    Time: 7007400 ps  Iteration: 0  Instance: /testbench

The pin's identified, the line of the vector file, and the time.

Quoted text here. Click to load it

Hmmm... maybe a 'linear' version (ie. no smarts) for free, and
looping/variables/procedures/macros etc. for $$? My recollection is
that ABEL just gave you the dumb version, but it seemed useful at the
time.

Yes, explaining that this is (hopefully) useful is difficult. I went
through this with a friend recently, who never writes testbenches, and
who spent years on ABEL. He wasn't impressed...  :(

Quoted text here. Click to load it

I've never got this. Does anyone actually do that?

Quoted text here. Click to load it

Yes, I know of Jan's stuff, but hadn't realised he also does TB
generation - I'll have a look at that.

Evan


Re: Survey: simulator usage
Quoted text here. Click to load it

ABEL is still supplied with some tool flows.
CUPL is also still usefull for CPLD end of the design, and it is both
free, and has some looping/macros in the functional Sim engine.

CUPL is functional-sim only, so has no timing smarts itself, but you CAN
take the fitter (verilog/vhdl) output, and run a testbench on that.

CPLD tend to be less of a timing puzzle than FPGAs so this extra flow is
not used much.

What CUPL _does_ offer, is test vectors into the JED file, which is
good for smaller devices that go via device programmers.

CUPL also allows a '*' in the vector input, which means CUPL
generates (fills-in) the H.L.Z in the output, and that can
speed the test process.

If you then want to lock in all test values, you can paste the
SimOut values back into the SimIN file, but I tend to scan the
SimOUT carefully, and then rely on the same results being generated
from the same stimulus. (These are stable tools.)

Here is a simple example:
SimIN
0C1* ** **** *** **** 111111 ****** ****** ***** C *
SimOUT
0C1L LL  LHHH  LHL ZZZZ  111111  LLLLLL  LLLLLL  LLHHH  C L


Quoted text here. Click to load it

  I have a friend who thinks this is great. He does a lot of BUS
interface work.
  Personally, I prefer using a text editor, and command line tools.
  I can see the appeal of waveform entry, but the custom-file risk of
that outweighs the benefits, for me.

-jg


Re: Survey: simulator usage

Quoted text here. Click to load it

If you really mean such a simple format, my answer is: don't bother,
that way is obsolete for anything but the most trivial designs.  Given
the amount of testing and the complexity of many interfaces/busses
today, I need to write my test vectors at a higher level than applying
a bitvector per clock cycle.

In order to do that, I normally create a library of high-level
functions in a scripting language like Perl or Tcl, which would do
"the dirty work" for me.  Normally I would make the script library
interface directly to a VHDL/Verilog testbench through TEXTIO files.
With your extention, I would still have to make the script library,
but just have a "backend" to that emits files compatible with your
format.  All I'd save is to write my VHDL testbench for the interface
(which may could have serious performance impacts, since you make the
simulator stop for each clock cycle and is a real no-no for
performance), and frankly, writing the VHDL testbench isn't the
problem.

Quoted text here. Click to load it

Affirmative (at least for the XE version).

Quoted text here. Click to load it

Modelsim PE / XE

Quoted text here. Click to load it

VHDL, but we have to deal with Verilog as well due to backannotated GL
sims.

Quoted text here. Click to load it

Considering that the list price of Modelsim SE is 170K DKK vs 70K DKK
for a PE license (both floating licenses), not bloody likely.

Quoted text here. Click to load it

With a little imagination, you can do a surprising amount intelligent
work with "dumb" TEXTIO vector files, by carefully dividing the work
between script library (vector generation) text files, and the
testbench itself.


Kai
--
Kai Harrekilde-Petersen <khp(at)harrekilde(dot)dk>

Re: Survey: simulator usage
Quoted text here. Click to load it

This sort of test-vector entry is basically unmaintainable, and to be
honest, I haven't written this type of go/no-go test in ages (maybe
since the last time I used ABEL).

I prefer to write test benches at a higher level.  I like to have
models of the things to which the DUT connects drive the DUT.  For
example, if my FPGA is a NAND flash controller that sits between a
microprocessor bus and the NAND flash devices, I'll use a
bus-functional model of the micro and the flash, and my test bench will
mostly consist of MicroWriteBus() and MicroReadBus() "commands."

I can't go back to the simple drive-a-vector test.

-a


Re: Survey: simulator usage

Quoted text here. Click to load it

Yes, makes sense; I've got no problem with high-level testbenches. If
a device or module has lots of buried state, and it's not obvious when
any outputs or observable points should change as a result of an input
change, or even *what* that change should be, then a test vector
approach is pretty useless. If your problem is, for example, apply an
opcode to a CPU and check that a memory location changes at some
arbitrary later time, then you need to be a lot smarter. But, horses
for courses. Consider:

- Your complex DUT is probably built of simpler modules. If a module
was just an FSM, a register block, a decoder, or a bus interface, for
example, and you could test the module simply by writing
20/50/100/whatever vectors, would it be worthwhile? Or do you just
stitch the whole thing together and test the top level?

- Many complete chips are surprisingly simple (seen from the outside,
anyway). I first used this code for a (large) DSP chain for baseband
processing. I got MATLAB data for the inputs, with expected data for
the outputs. Once I'd automated the test, verification was trivial.
Seven years on, and I've just finished part of a Spartan-3. Nothing
fancy - 25K marketing-gates - it just translates accesses between two
different buses connected to 15 external chips, handles bursts, has a
register block with control and status ops, does some address mapping
between the various chips, and so on. I did a basic test in a day by
writing 200-odd vectors, and I had a complete-ish test of 752 vectors
with a bit more work. Whatever way I'd tested it, these 752 vectors
would have existed somewhere, if only implicitly in the VHDL code, or
even in the documentation. Why not make the vectors themselves the
test? The rest of it is just unnecessary verbiage that you have to
repeat again, and again, for each new module and device you do.

- What are MicroWriteBus() and MicroReadBus()? I can do macros and
pass parameters to the macros; you can call the macros from wherever
you want in the vector file. I can also do basic C-like control
structures - looping, branching based on tested values, and so on.

Ok, is it worth any more than $0 now? :)

Evan


Re: Survey: simulator usage

Quoted text here. Click to load it

Shorthand notation for "generic test-bench procedure/task call that
performs a read or write on my microprocessor's external bus, from the
micro's point-of-view."

-a


Re: Survey: simulator usage

Quoted text here. Click to load it

In a word, no.

Why go to the trouble of learning a new language to try to do things
like macros, loops, random stimulus, etc. when you have the power of
the VHDL language at your disposal in a VHDL testbench?  Now, if you
have vectors from an external model/simulation, those can be applied
with text-io relatively easily from within a vhdl testbench that will
run on any vhdl simulator.

My "unit level tests" are usually at a high enough level that I need a
lot more capability than is available in any vector based scripting
language.

Andy


Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)
Generalising the question of this discussion I would like to know what
does the group here think that code generators are worth?

Say Someone comes out with a code generator which requires some  input
data say, D_in and generates a bunch of output code say, C_out.

 Given that C_out is something that you already write for your design
wether the code generator exists or not, when will the code generator
look like a fesible option?

Given the following man days(Including coding, verfying and debugging
time) for writing  D_in and generating C_out VS manual writing of
C_out, what is the price you will be willing to pay for the code
generator in each case?
---------------------------------------------
Development time in Mandays
---------------------------------------------
D_in :      Manual C_out
---------------------------------------------
1     :      10
1     :      100
1     :      1000

If such a code generator appears on the market will your management
wait until a bigname EDA comes out with a me too product or will they
buy it from a noname developer?
Regards
Thomas

Andy wrote:
Quoted text here. Click to load it


Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

Personally, I would avoid code generators to generate RTL for design.
Coding RTL is not a bottleneck for me, even on multi-million gate designs.
Some IP is provided by 3rd-party, other IP our developers code.  It is
fairly easy to write Verilog once you have tried and true coding style.  Why
would I want to learn a proprietary  language to generate a standard
language?  (and get locked-in?).  In addition, I know exactly what hardware
I'll generate when I code.  For logic circuits that are used often, the HDL
addresses that by allowing designers to create modules or primitives.  Who
knows what a code generator will output.  I also wouldn't want to be locked
in to a proprietary.

System Verilog addresses many of the my department's needs.

Even if a big name EDA company were to come out with one, I doubt this type
of tool would be purchased.

I think if you want to attack something in the area of chip design, you've
got to address the parts in the design flow that addresses a company's
development bottlenecks such as verification or physical design.


Quoted text here. Click to load it



Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)
Quoted text here. Click to load it
There are different levels of generated code. e.g.
1> Schematic to code generation e.g. FSM, block diagram,module
integration.
2> Translating code from one format to another.
3> Template based code generators etc.
While some are simple (have <10x improvement in total coding time)
others are complex and can result in saving days or even weeks of
coding time. While I personally wont care for the first case the second
case is what interests me.
Quoted text here. Click to load it
Applying the 80-20 rule, I would say that 80% of any code is bug free
but identifying the other 20% can take up 80% of my development time.
so would it not be in my interest to reduce the amount of hand written
code by generating code wherever possible? Also quite a few bugs are
because of copy, paste, modify errors whihc a template based code
generator can easily fix.
Quoted text here. Click to load it
If we are generating RTL the generated code will be verilog or VHDL and
if it is a testcase it will be e,c,SystemVerilog or SystemC and if at
any time we want to "break out" of the code generator "lock-in" we can
always take the generated code and continue from there.

Note: I dont have a code generator to sell. But have seen various
reusable code generators written by my team mates which  have saved
weeks and weeks of coding+debugging effort and was wondering what will
it cost in the market.
Regards
Thomas.
Quoted text here. Click to load it


Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

We also had some inhouse people working on these kinds of tools.  Some
issues we ran into are that code would be written incorrectly or a bug
found, then not only would the RTL have to change, the code generator would
have to change as well.   Which means you're debugging two things and
requiring more people and time to develop a product.  To make a long story
short, for us, it was a big waste of time because the time they spent on it
would have been better spent verifying the 20% that you speak of,
understanding performance bottlenecks, planning for future product,
training, or fine tuning an architecture.

I really think code generators are a tough sell.  I can't see the value
proposition.  Coding just isn't on the top ten list of headaches for me or
any of the designers that I work with.  Again, verifying takes up most of
the development effort.

From your response, it sounds like you've benefited.  Maybe you know more
than I do and your experience was much more enjoyable.  If you believe in
it... put it on the market

Quoted text here. Click to load it



Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

Quoted text here. Click to load it

I think you may be in even more trouble than me...

You need to be specific about what you're generating. A human is a
code generator; are you replacing humans? Are you automating something
that's already done? How many people do it? How much time do they
spend on it? Can you do it better than a human? Or just faster? Or
just cheaper? Or is it too difficult for a human to do at all?

Are you generating IP? If so, you're in trouble. People don't like
buying IP, and they'll only buy it from someone they already trust.
The IP itself is unimportant, anyway: what matters is how well it has
been verified, and whether you can prove it. You're selling a
verification plan and coverage metrics, not HDL code.

Who are you targetting? Is it FPGA users? If so, you're shafted. If
not, you've got no market. Google for anyone who's tried to sell IP
for FPGAs; I can't believe anyone's ever made money doing this. If the
users like it, the FPGA vendors will do it themselves and sell it for
*zero*. Remember, the vendors are selling silicon; they need to bundle
software and IP for free to get market share. If you don't understand
this, think Microsoft and Netscape, WMP, etc.

Finally, remember that the EDA market is tiny, and 3 mid-sized fish
are desperately trying to make a living in it, primarily by eating all
the smaller fish. If the FPGA vendors don't get you, then the EDA
vendors will. The men's underwear market is bigger than EDA. Perhaps
you should be designing a better pair of Y-fronts...

:)

Evan

Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)
Quoted text here. Click to load it

I am looking at a problem set where
We tend to do the same task over and over again for each project. After
doing it a few times we know all the possible permutations and
combinations and which combination to apply for which problem but it
still consumes most of our coding time. Some of the tasks in the
problem set may require one person to be dedicated for it throughout
the project cycle.

The generated code could be similar to what is written by a human and
only reduces the development and debug time. For e.g. it may do the
same job with the benifit of giving 10X to 1000X improvement in the
coding+debugging time.

As an example think about writing
y=a+b; and using a tool for synthesis and layout. VS using a layout
editor to design the ckt one CMOS transistor at a time followed by
spice simulations. If this were the 70's will you pay for the current
crop of synthesis+layout tools?

Quoted text here. Click to load it
While my company is currently working on a set of IP's for its client.
My question was more towards the market fesibility of code generators
beacause every year EE Times covers a couple of developers who have
tried to sell their solution to specific problems in IC design  which
could be replicated in a a few hundred lines of perl code.
Quoted text here. Click to load it
What I am looking at is a set of tools which help in taking the project
from Spec to samples to Product faster than the current manual process
would allow. So It could involve
1. The methodology.
2. The code generators to generate verbose codes from a brief high
level input.
3. Resolving any other bottlenecks.

Going with the Microsoft analogy below. I am looking at something
similar to windows apps development using visual C++ where you fireup
the relevent wizard click a few buttons and are presented with a
template based code framework runnning in a few tens of thousands of
lines in which you insert your alogrithm and other code fragments.

Note:I know about the code bloat issue for MS VC which a tool for HW
design cannot afford.

Quoted text here. Click to load it

The audience for such tools could be both FPGA and ASIC users. I know
about the risk any successful product will have from the established
EDA vendors copying it and offering it for free. so my question of
wether your manager will buy it from a no name developer or wait for a
Known EDA Vendor to come up with a me too product.
Quoted text here. Click to load it
I dont think any developer will regret beig eaten by a big name EDA
vendor if the price is right :)
Also I think EDA vendors are currently more interested in me having a
bigger code base to verify so that meeting coverage goals will require
more run time or more simulators.

Quoted text here. Click to load it
After a decade in the Industry I think I am too old to change
professions. I will make sure that my kid keeps his options open ;)
Regards
Thomas
Quoted text here. Click to load it


Re: Survey: simulator usage


Quoted text here. Click to load it

NC-Verilog

Quoted text here. Click to load it

Verilog


I'm sure it does

Quoted text here. Click to load it

Frankly, as an ASIC designer.  I probably wouldn't use this language.  In
the future, we plan to use System Verilog.

Quoted text here. Click to load it



Re: Survey: simulator usage

Quoted text here. Click to load it

Why using a code generator when you can write such bit vector tests within
VHDL itself?

constant rom_test: unsigned(63 downto 0) := x"cf000000e4ec2933";

rom := rom_test;
for j in 1 to 64 loop
  wait until ds_wire = '0';
  wait until ds_wire = 'Z';
  if rom(0) = '0' then
    ds_wire <= '0';
    wait for 30 us;
    ds_wire <= 'Z';
    wait for 1 us;
  end if;
  rom := shift_right(rom, 1);
end loop;
  
-- wait until latched
wait until data_valid = '1';

-- check, if read process was successful
assert rom_test(55 downto 8) = ds_wire_rom report "ROM read error" severity
failure;

You can refactor the for-loop to a function or procedure, if you need it
multiple times.

--
Frank Buss, fb@frank-buss.de
http://www.frank-buss.de , http://www.it4-systems.de

Site Timeline