C++TESK Getting Started¶

Introduction¶

Hardware verification is usually understood as the process of checking behavior of hardware on conformity to its specification. Such a process can be done formally by means of, e.g., model checking, automatic theorem proving, etc. Also, verification can be done by means of simulation of separated hardware modules with the help of simulator.

Accounting the complexity of hardware models under verification, the task of automation should have usually been solved before the actual verification. The more processes will be done automatically and the less manual labor will be needed, the more effective check will be made. Without touching upon the formal verification methods, in this course we will focus only on simulation-based verification. Moreover, we will further speak only about one of the existing verification tool, created in the Institute for system programming of RAS. The tool''s capabilities allow speaking about it as a powerful and quite modern solution. So, we will speak about using C++TESK Testing ToolKit (or C++TESK for short).

C++TESK implements simulation based approach to verification. The main element of the tool is its core library, implemented in programming languages C and C++. All core components are arranged in one package and are available at http://forge.ispras.ru/projects/cpptesk-toolkit/files. The tool is designed for creating test systems using C++ for different models of synchronous hardware at different levels of abstraction. Test systems are created using any means, provided by C++, basing on the approach, macros and classes defined by C++TESK.

When creating test systems for simulation based verification, three main tasks are usually solved. The first one is test sequence construction, the second one is checking of behavior correctness, and the third one is test completeness estimation. C++TESK allows construction test sequences of two types: selection random stimulus set from the previously described stimuli at each simulation cycle or selective choice of stimuli based on techniques of exploration of implicitly defined FSMs. Checking of behavior correctness is made at each simulation cycle by means of executable reference model, created by verification engineer at some level of abstraction. External model (e.g., system simulator) can also be used. Test completeness is determined either by the number of testing cycles for randomly selected stimuli, or on the basis of the information about completeness of FSM exploration.

Common scheme of test system is represented in figure 1.

Figure1. Generalized structure of test system

Let us shortly describe elements represented at the common scheme.

1. Stimulus generator is a component making test sequence (stimuli sequence). It is adjusted by test scenarios;
2. Test oracle is a component receiving data flows from stimulus generator and target component, sending stimulus flow from generator to target component, estimating correctness of target system behavior;
3. Target system is a hardware model, developed at one of hardware description languages (here, in Verilog), receiving stimulus flow and responding to it by reactions which should be checked;
4. Coverage tracker is a component grabbing information about reference model functional coverage, which in general affects stimulus generator work (e.g., by information of reached coverage);
5. Test report generator is a component making reports (test traces) with information of traversed transitions, reached coverage, found errors, etc.

The following tasks will be overviewed below.

• Analysis of documentation for development of C++TESK test systems;
• Development of test oracles including reference models and their adapters;
• Definition of test coverage;
• Setting up stimulus generator;
• Verification itself.

Documentation analysis for development of test systems¶

Under verification we mean a process of checking observed behavior against specification. In the other words, verification is an establishing of the correspondence between target system behavior and its specification. Therefore, we should have not only the target system, but its specification too, being a document written at the beginning of target system development, slightly modified during development process, and containing information of target system functionality.

In reality specification is often very poor or even absent, and target system developers might have written only lists of input and output interface signals. In this case to conduct verification is difficult as to speak about bug in target system is possible only having information about correct behavior. Verification engineers have to interview target system developers, making list of requirements which are obligatory for target system. When list of requirements (specification) is obtained, verification can be started.

Practice shows that specification (especially, cycle accurate) is convenient to represent in the following ways.
1. by means of block diagrams (see figure 2).

Figure 2. Example of block diagram for single operation

Block diagram allows describing reference model behavior with any proximity to cycle accurateness. Figure 2 contains abstract representation of operation IO-WRITE (write via IO channel), starting from one-cycle request by the 20th interface. At the following cycle, device starts to receive data, taking 17, 19, 33 or 37 cycles depending on data length and presence of mask among sent data. This block diagram is insufficient for usage in reference model development and is supplied with chart of cycle accurate operation description, which should be described below. Operation finishes after receiving and saving of all data.

The following notations are used in block diagrams.

• Oval symbols mean points of operation start and stop; start point contains mnemonic designation of operation to be executed in this control flow path;
• Rectangle symbols mean blocks taking exactly one cycle for execution at current abstraction level. Process of reference model functioning is always described inside of such blocks. Block is allowed to be supplied with information about real execution time, other auxiliary information facilitating binding the scheme to reference model.
• Rhombus symbols mean branches in control flow. The symbol should contain condition of branching inside. There should be only one ingoing path and two outgoing paths;
• Forking of control flow is represented by bold point with two or more outgoing paths;
• Merging of several control flow paths is made also in bold point, meaning synchronization of control flow paths (waiting for the slowest one) to proceed to the outgoing path.

2. by means of charts of cycle accurate operation description (see table 1).

Table 1. Example of chart of cycle accurate operation description

Stimulus	Branch 0	Microoperation 0	Microoperation 1	Branch 1	Branch 2	Microoperation 2	Microoperation 3
PRE: 1) only one operation of the type may be executed simultaneously 2) val_wr_data_buff_nreg_to_IO(o)=1	BRANCH: if val_mask=1 (in stimulus) microoperation 0 is started, else microoperation 1 is started.	PRE: -	PRE: -	BRANCH: if wr64=0 (in stim.), operation is done, else branch 2 is started.	BRANCH: if val_mask=1 (in stimulus),  microoperation 2 is started, else micro operation 3 is started.	PRE: -	PRE: -
Set stimulus parameters: val_wr_data_from_IO (i)=1 (strobe) wr_data_from_IO [15:0](i)=[0-7] Stimulus takes one cycle.		wr_data_from_IO [15:0] (i) contains mask. It is written into buffer(21) (with capacity of 4 32- or 64-bytes words). (repeat once).	wr_data_from_IO [15:0] (i) contains 2 data bytes. Data are written into buffer(21). (repeat 16 times).			wr_data_from_IO [15:0] (i) contains mask for the higher part of 64-byte transmission, being written in buffer(21). (repeat once).	wr_data_from_IO [15:0] (i) contains 2 bytes of the second part of 64-byte transmission. Data are written in buffer(21). (repeat 16 times).
		POST: next cycle after val signal wr_data_buff_nreg_to_IO (o) (being written item in (21)) changes to the number of the following free item or to 8 if buffer (21) is full	POST: next cycle after val signal wr_data_buff_nreg_to_IO (o) (being written item in (21)) changes to the number of the following free item or to 8 if buffer (21) is full			POST: -	POST: -
INPUT: val_wr_data_from_IO(i) - strobe wr_data_from_IO[15:0](i) includes - [15:3] - reserved; - [2] A5 – write to the high 32-bytes of 64-byte item flag; - [1] wr64 – 64-byte transmission flag; - [0] val_mask – mask flag.	INPUT: wr_data_from_IO [15:0] - mask	INPUT: wr_data_from_IO [15:0] - data			INPUT: wr_data_from_IO [15:0] - mask	INPUT: wr_data_from_IO [15:0] - data