Scoreboard and Checker in Testbench?

Hi Davy,

The AVM cookbook from Mentor describes these terms, you can download a free copy from:

Hans

Hi Hans,

This cookbook is so good, thanks a lot!

Best regards, Davy

Can I use base class mentioned in Mentor's AVM cookbook in other simulator like Cadence's NCSim?

Best regards, Davy

Hi Davy,

Sorry, can't answer your question since I don't know SystemVerilog and I use Modelsim (VHDL+SystemC). I believe the examples are written in generic SystemVerilog/SystemC so I would suggest to just try it out,

Hans.

Generally speaking, a scoreboard is a mechanism for tracking which tests and which requirements have been run/verified. For example, a testbench may have multiple tests which verify multiple requirements (not often in 1:1 correspondence either). A portion of the overall testbench is to capture and record the success/failure of each test, and/or requirement verified. This is the purpose of the scoreboard.

Another aspect of good verification is to segregate interface specification verification from performance specification verification. For example you could create a protocol checker that monitors an interface and verifies that each transaction on that interface is per the protocol on the interface. But to keep it portable, that protocol checker would not verify specific contents of transactions, that is done elsewhere (so the content checker can be portable across different interfaces too).

Andy

Davy wrote:

Davy, A checker is used to check whether a given transaction has taken place correctly . This may include data correctness and correct signalling order. A Scoreboard is used to keep track of how many transactions were initiated, how many finished and how many are pending and wether a given transaction passed or failed. Regards

-- We are now hiring in India check our requirements at

IMO, There is lack for clear definition of verification terms. For me, both of them are verification components which check specific design properties for all simulation times and all testcases. They correspond to the "safety" properties in formal verification: during system execution, something wrong will never happen. IN AVM Cookbook, there is definition for the scoreboard as a transaction-level checking component. There are no defition of checker in the cookbook, yet there is an example of so-called "Assertion-based checker". From this example, we can find out that assertion-based checker checks lower-level interface properties. Also, it has to be implemented using assertions, but implementation details are not important here.

So, to find the differences between scoreboard and checker we have to understand the meaning of transaction: "The transaction is quantum of activity that occurs in design bounded by time" "A transaction is a single transfer of control or data between 2 entities" "A transaction is a function call"

First definition is too broad. By this definition, scoreboard is the same as assertion-based checker, since both of them deal with quantums of design activity bounded by time. Second definition is different from the first one and too restrictive. What does it mean "single transfer"? Why between only two entities? Third definition is the one I cannot understand. Does it mean that "transaction-based" component must contain functions?

Regards,

-Alex

Alex, I think you need to be careful not to confuse the generic use of terms like entities (things) in the AVM cookbook with VHDL key words.

I agree it is difficult to tie down a definition of a transaction, as transactions can be different things to different people eg a h/w designer may want to identify individual transfers of data on a bus as a transaction, while someone viewing the design at a higher level is more interested in frames or packets, while at a system level it may be a message between s/w on different processors. All are valid transactions.

In terms of the AVM, the basic TLM communication mechanism is based on the semantics of the OSCI TLM, implemented in SystemC and/or SystemVerilog.

This defines the use of put(), get() and peek() function/task calls to transfer a transaction from one component to another.

Both these languages support a component calling the tasks/functions of another component (using classes and/or interfaces). Thus my monitor can call the write() function of my scoreboard without needing to what it does. This allows me to change the scoreboard without effecting the monitor, provided the new scoreboard also implements a function called write(). This is the basis for verification component reuse in the AVM (and other transaction based verification methodologies.)

So in the AVM, a transaction is most commonly just a function/task call between verification components, thus the third definition.

I hope this help clarify things

- Nigel

This may be not a problem : I am pure Verilog coder ;)

That's sound like a problem. Transaction definition is too broad to be useful. To clarify things, different companies developed their own understanding of transaction. (see link to the article below).

OSCI definition of transaction also seems too broad to be useful:

"OSCI seems to have the most liberal interpretation. OSCI includes several levels of abstraction under TLM, including Programmer's View (PV), which contains no timing; Programmer's View with Timing (PVT), which adds timed protocols and can analyze latency or throughput; and Cycle Accurate, which is accurate to the clock edge but does not model internal registers.

considered TLM, said Pat Sheridan, OSCI executive director and the director of marketing at CoWare Inc. But the way most users think of TLM appears to include just the untimed (PV) and cycle-approximate (PVT) styles of modeling."

Which means that it is possible to use these convenient functions for transaction communication. But, it is also possible to connect bus master with the bus slave - they will also communicate with transactions through the signal-level interface. And this is transaction-level modeling too.

There may be different methods for transaction communication for reuse. For example, there are strong arguments against AVM method, where monitor code has to be modified in order to communicate with specific checking component. Since monitor is protocol-specific (design-independent), it is usualy more reusable then design-specific checker/scoreboard. Then, it may be a better idea for monitor to simply "present" transaction without calling some external functions. Then, it will be the task of checking component to grab needed data from needed monitors when they are signalling about "transaction completion". This make monitors to be independent from the "external world" and thus highly reusable between the projects.

In any case, functions/tasks presented by OSCI do not clarify the meaning of transaction, but rather provide some implementational details.

I agree. In this case, AVM may contain only third transaction definition. This will greatly clarify the meaning of transaction in the context of AVM.

Regards,

-Alex

I actually think the broad scope of TLM is one of its key strengths. Being able to apply the same techniques across a wide range of problems means the same methodology can be used at multiple abstraction levels. Transactors are used to communicate across these abstraction boundaries (not just the TLMsignal boundary) enabling modelling to occur at the highest abstraction level appropriate resulting in smaller/faster code.

A quick point on AVM monitors. The AVM provides a capability called an analysis port (which is an implementation of the OO design observer design pattern).

An analysis port provides a mechanism for a monitor to broadcast transactions to zero or more subscribers (listeners) via a single write() function call.

The subscribers need to implement a write() function, each of which can do different things (eg record functional coverage, update a scoreboard etc). Thus a monitor does not need to know in what context it will be used as it is only in the top level environment of the simulation where analysis components are registered onto a particular analysis port.

This makes AVM monitors extremely generic, being able to be used in module, chip and system verification and across multiple designs using the same interface.

Regards

- Nigel

Sorry, Nigel, I don't agree with this. Broad definitions lead to ambiguity. Everything which is not formalized is ambiguis. Ambiguity of the basic terms leads to ambiguity of the whole system, based on these terms.

We all know how difficult is to work with ambiguis specifications. The problem is that we believe that we understand correctly the meaning of some term, and don't do any efforts to clarify things further. Then, we spend significant amount of time and energy in wrong direction.

As verification engineers, we have to formalize design specification documents in verification plans, thus removing ambiguity from them. However, we have ambiguity in our own definitions, which leads to many communication problems. Transaction definition is one example. Here is another one: what is the difference between assertions, assertion monitors, asertion-based checkers, non-assertion-based checkers and scoreboards? All of them validate the correctness od some design properties during all simulation time and independently of stimulus.

Thanks for clarification. In this case, the monitors are not "transaction senders" but rather "transaction presenters", which really make them highly reusable. I am also using similar concept to build Verilog testbenches for more than 4 years.

Regards,

-Alex

Sorry, Nigel, I don't agree with this. Broad definitions lead to ambiguity. Everything which is not formalized is ambiguis. Ambiguity of the basic terms leads to ambiguity of the whole system, based on these terms.

We all know how difficult is to work with ambiguis specifications. The problem is that we believe that we understand correctly the meaning of some term, and don't do any efforts to clarify things further. Then, we spend significant amount of time and energy in wrong direction.

As verification engineers, we have to formalize design specification documents in verification plans, thus removing ambiguity from them. However, we have ambiguity in our own definitions, which leads to many communication problems. Transaction definition is one example. Here is another one: what is the difference between assertions, assertion monitors, asertion-based checkers, non-assertion-based checkers and scoreboards? All of them validate the correctness od some design properties during all simulation time and independently of stimulus.

Thanks for clarification. In this case, the monitors are not "transaction senders" but rather "transaction presenters", which really make them highly reusable. I am also using similar concept to build Verilog testbenches.

Regards,

-Alex

Alex,

I think this definition issue depends on the perspective you are looking from ;)

I agree that within a verification team, you should have a clear, common understanding of what each component should and shouldn't do. However, what may be suitable for one team may be too loose OR too restrictive for another team. Thus a methodology that can accommodate most user requirements is one that can be broadly adopted. However that doesn't preclude it being used in a tightly defined manner.

I guess where we differ is who should be responsible for defining these precise definitions - the user or the tool/methodology provider ?

Regards

- Nigel

In this context, I am not speaking about "what each component should and shouldn't do". I am speaking about terminology and basic definitions. The whole industry will betefit from speaking the same language. Imagine, for example, job interview question: "What is scoreboard?". Or : "What does transaction mean?". The applicant is lucky, if his/her understanding of scoreboard/transaction matches the understanding of the interviewer.

For me, there is only one answer: methodology provider. Methodiology is a building, and definitions/concepts are the building blocks. If the shape of building block changes (at least, slightly), the shape of the whole buiding changes significantly. Then, user definitions are not always complementary - they can also contradict. The attempt of "being nice to everyone by making loose definitions" may reduce the technical value of the whole methodology, developed by EDA company.

Regards,

-Alex

Scoreboard is mainly used to check the functional property correctness of a DUT. eg. whether a certain functional algorithm of the DUT works correctly.

Checker is used to verify the protocol on the DUT interface output. ie. whether the timing of certain transactions are correct or not. Mainly it is used to verify whether the output signals of the DUT confiorm to a specific protocol eg. Ambd AXI.

Hope this is a clear definition.

I would have checkers in my testbench to flag an error on protocol error before the scoreboard checks the actual output of the DUT.

Don