Verification Horizons
 |
The advent of new technologies — such as constrained-random data generation, assertion-based verification, coverage-driven verification, formal model checking and Intelligent Testbench Automation to name a few — have changed the way we see functional verification productivity. An advanced verification process enables users to manage the application of these new technologies in a complementary way, providing confidence that the myriad corner cases of today's increasingly complex designs have been covered. The Verification Horizons publication expands upon verification topics to provide concepts, values, methodologies and examples to assist with the understanding of what these advanced functional verification technologies can do and how to most effectively apply them.
November 2011 | Volume 7, Issue 3
Verification Horizons Complete Issue:
Verification Horizons Articles:
With Vision and Planning, You Can Actually Enjoy Upgrading Your Process. Seriously.
by Tom Fitzpatrick, Editor and Verification Methodologist, Mentor Graphics Corporation
We're finally doing it. My wife and I have been in our house for 13½ years and we're finally redoing our kitchen, which my wife has wanted to do for about 11 years now. This is kind of a big step for us, as those of you who have gone through the exercise can attest, I'm sure. When we had the house built, we chose to limit our budget to keep "expensive" from turning into "exorbitant," and we've been living with formica countertops and a linoleum floor ever since. It's rather exciting seeing the first steps of converting these into granite and hardwood. As you might imagine, I find myself thinking of how this relates to upgrading a verification methodology because I'm sure that, by now, you know that that's how my mind works.
Graph-Based IP Verification in an ARM SoC Environment
by Andreas Meyer, Verification Technologist, Mentor Graphics Corporation
The use of graph-based verification methods for block designs has been shown to provide a significant time and cost reduction when compared to more traditional constrained random techniques. By filling the random constraint space in a controlled random fashion, the full set of constraints can be filled significantly faster than a random constraint solver will. Possibly more importantly, the ability to describe a graph in canonical form is likely to be easier to modify and maintain over the life of an IP block than constraint statements. Combining graphs with a UVM environment, it is possible to extend block-level verification components into a SoC-level test environment with little additional verification development.
Use Scripting Language in Combination with the Verification Testbench to Define Powerful Test Cases
by Franz Pammer, Infineon
This article focuses on providing a methodology to make the process of writing test cases easier for verifying mixed analog digital designs. It explains the advantage of using a scripting language together with a Hardware Description Language (HDL) environment. It shows a way to give the HDL engineers the possibility to use constrained randomized test cases without having too much knowledge about the implemented testbench. Two possible scenarios for defining your test cases will be demonstrated. One can be used for the traditional VHDL testbench and another for the SystemVerilog (SV) Open Verification Methodology (OVM) environment. For both cases a scripting language will be used to write test cases. At the end you will understand that with the mentioned methodology also design engineers will fully benefit of verifying their design.
STMicroelectronics Engineers Find Mixed Analog-Digital Verification Success with OVM
by Alberto Allara, Matteo Barbati, and Fabio Brognara, STMicroelectronic
By now it's a cliché to speak of the rise of digital technology. Follow technology coverage in the media for any length of time and it doesn't take long to note the tacit assumption that nearly anything with an on/off switch will eventually communicate with the world at-large exclusively in strings of 0s and 1s. Of course, as long as the electronics industry is built on harnessing the laws of physics, the importance of analog signals will never go away. Nature speaks in waveforms, not regimented bitstreams. So the challenge, and one that must be repeatedly solved by those building ever more complex semiconductor devices, is how to verify what's happening at the analog-digital interface. One recent solution comes from a trio of engineers at STMicroelectronics in Milan, Italy working on verifying an important bit of intellectual property (IP) for an ST hard disk component. The team's approach was to combine OVM methodology, Mentor Graphics Questa ADMS simulator and lots of collaboration among engineers at both companies.
Polymorphic Interfaces: An Alternative for SystemVerilog Interfaces
by Shashi Bhutada, Mentor Graphics Corporation
The SystemVerilog language is increasing in adoption for advanced verification techniques. This enables the creation of dynamic test environments using SystemVerilog classes among other things. The SystemVerilog virtual interface has been the primary method for connecting class-based SystemVerilog testbenches with a VHDL or Verilog DUT, but this construct has certain limitations, especially when the interface is parameterized. In this article we will discuss some of these limitations and demonstrate an alternative approach called as the Polymorphic Interface. The recommended approach can also work generically wherever one uses virtual interfaces and not just parameterized interfaces. For the SystemVerilog testbench, we will use OVM infrastructure, but this same discussion applies to UVM testbenches. This article assumes SystemVerilog OVM/UVM class and interface syntactical understanding.
Layering in UVM
Extracted from the UVM/OVM Online Methodology Cookbook
Many protocols have a hierarchical definition. For example, PCI express, USB 3.0, and MIPI LLI all have a Transaction Layer, a Transport Layer, and a Physical Layer. Sometimes we want to create a protocol independent layer on top of a standard protocol so that we can create protocol independent verification components ( for example TLM 2.0 GP over AMBA AHB ). All these cases require that we deconstruct sequence items of the higher level protocol into sequence items of the lower level protocol in order to stimulate the bus and that we reconstruct higher level sequence items from the lower level sequence items in the analysis datapath.
Successful Adoption of OVM and What We Learned Along the Way
by Mike Bartley, Founder, TVS and Steve Holloway, Senior Verification Engineer, Dialog Semiconductor
OVM has gained a lot of momentum in the market to become the dominant verification "methodology" and the indications are that UVM will gain even greater adoption. OVM is built around SystemVerilog and provide libraries that allow the user to build advanced verification test benches more quickly. There is extensive documentation, training and support for how to best develop such test benches and to encourage test bench re-use. However, there is less advice on how to adapt your verification processes on your project to best use OVM and even less advice on how to do this for company wide adoption. In this article we discuss the experiences of the authors of a company-wide adoption of OVM. We consider the original motivations for that adoption and the expected benefits, and the actual results achieved and problems that have been overcome. The aim of the article is to give advice to other companies considering adopting OVM.
Process Management: Are You Driving in the Dark with Faulty Headlights?
by Darron May, Manager of Verification Analysis Solutions, Mentor Graphics Corporation
If your car's headlights were faulty, would you even think about leaving home in the dark bound for a distant destination to which you've never been? Even if you were armed with a map, a detailed set of instructions and a good flashlight, the blackness on the other side of your windshield would still make it difficult to navigate. And even with reliable headlights and maps, you'll invariably still encounter obstacles and detours. These hassles are nowadays mitigated somewhat by car navigation systems that tell us where we are, how far we have travelled and, so we can consider alternate routes and estimate how long it will take to get to our final destination, how bad the traffic is ahead. Verification of SOCs and electronic systems is certainly a little more complex than road navigation. However, the process of planning what you are verifying and then constantly measuring where you are in the process is equally important whether your final destination is a swanky new hotel a few hours away or a successful tape-out by the end of the year.
June 2011 | Volume 7, Issue 2
Verification Horizons Complete Issue:
Verification Horizons Articles:
How Do You Know You Have the Right Tool for the Right Job?
by Tom Fitzpatrick, Editor and Verification Methodologist, Mentor Graphics Corporation
As I write this, spring appears to have finally arrived here in New England – about a month and a half later than the calendar says it should have. As much as I love warm spring weather, though, it means that I now have to deal with my lawn again. I know that many people actually enjoy working on the lawn, but as far as I'm concerned, the greatest advance in lawn-care technology happened last year when my son became old enough to drive the lawn mower. If you've ever seen a 13-year old boy driving a lawn tractor, you'll understand my characterizing him as "constrained-random" when it comes to getting the lawn cut. I handle the "directed testing" by taking care of the edging and hard-to-reach spots, and together we manage to get the lawn done in considerably less time than it used to take me alone.
First Principles: Why Bother with This Methodology Stuff, Anyway?
by Joshua Rensch, Verification Lead, Lockheed Martin and Tom Fitzpatrick, Verification Methodologist, Mentor Graphics Corporation
Many of us are so used to the idea of "verification methodology," including constrained random and functional coverage, that we sometimes lose sight of the fact that there is still a large section of the industry to whom these are new concepts. Every once in a while, it’s a good idea to go back to "first principles" and understand how we got where we are and why things like the OVM and UVM are so popular. Both authors have found ourselves in this situation of trying to explain these ideas to colleagues and we thought it might be helpful to document some of the discussions we’ve had. If you’re new to the idea of object-oriented testbenches in SystemVerilog and maybe are wondering what all the fuss about UVM at shows like DAC and DVCon is all about, or if you’re getting ready to take that plunge, we think these ideas might help you "begin with the end in mind." If you’re an "expert" at this stuff, we hope that this dialog will help you take a step back and appreciate how far we’ve come as an industry and remember not to get too hung up on the whiz-bang features of a methodology but to keep in mind the ultimate goal, which is to make sure that our chips are going to work properly.
Online UVM/OVM Methodology Cookbook: Registers/Overview
by Mark Peryer, Verification Methodologist, Mentor Graphics Corporation
The UVM register model provides a way of tracking the register content of a DUT and a convenience layer for accessing register and memory locations within the DUT. The register model abstraction reflects the structure of a hardware-software register specification, since that is the common reference specification for hardware design and verification engineers, and it is also used by software engineers developing firmware layer software. It is very important that all three groups reference a common specification and it is crucial that the design is verified against an accurate model. The UVM register model is designed to faciliate productive verification of programmable hardware. When used effectively, it raises the level of stimulus abstraction and makes the resultant stimulus code straight-forward to reuse, either when there is a change in the DUT register address map, or when the DUT block is reused as a sub-component.
A Methodology for Hardware-Assisted Acceleration of OVM and UVM Testbenches
by Hans van der Schoot, Anoop Saha, Ankit Garg, Krishnamurthy Suresh, Emulation Division, Mentor Graphics Corporation
A methodology is presented for writing modern SystemVerilog testbenches that can be used not only for software simulation, but especially for hardware-assisted acceleration. The methodology is founded on a transaction-based co-emulation approach and enables truly single source, fully IEEE 1800 SystemVerilog compliant, transaction-level testbenches that work for both simulation and acceleration. Substantial run-time improvements are possible in acceleration mode and without sacrificing simulator verification capabilities and integrations including SystemVerilog coverage-driven, constrained-random and assertion-based techniques as well as prevalent verification methodologies like OVM or UVM.
Combining Algebraic Constraints with Graph-based Intelligent Testbench Automation
by Mike Andrews, Verification Technologist, Mentor Graphics
The Questa® inFact intelligent testbench automation tool is already proven to help verification teams dramatically accelerate the time it takes to reach their coverage goals. It does this by intelligently traversing a graph-based description of the test sequences and allowing the user to prioritize the input combinations required to meet the testbench coverage metrics while still delivering those sequences in a pseudo-random order to the device under test (DUT). The rule language, an extended Backus Naur Format (BNF) that is used to describe the graph structure, has recently been enhanced to add two powerful new features. Algebraic constraints can now be included to define relationships between the fields of the stimulus description (such as the fields of an OVM/UVM sequence item). Also, external testbench values can now be imported into the graph, allowing for the definition of relationships between Questa inFact-generated field values and externally selected values. The Questa inFact algorithms can now target cross combinations of fields that are under its control with fields that are outside of Questa inFact's control. This article describes these powerful new capabilities in more detail with some simple application examples.
Data Management: Is There Such a Thing as an Optimized Unified Coverage Database?
by Darron May, Manager of Verification Analysis Solutions and Gabriel Chidolue, Verification Technologist, Mentor Graphics Corporation
With the sheer volumes of data that are produced from today's verification environments there is a real need for solutions that deliver both the highest capacities along with the performance to enable the data to be accessed and analyzed in a timely manner. There is no one single coverage metric that can be used to measure functional verification completeness and today’s complex systems demand multiple verification methods. This means there is a requirement not only to unify different coverage metrics’ but also to unify data from multiple tools and verification engines. Data management forms the foundation of any verification environment.
A Unified Verification Flow Using Assertion Synthesis Technology
by Yuan Lu, Nextop Software Inc., and Ping Yeung, Mentor Graphics Corporation
As SOC integration complexity grows tremendously in the last decade, traditional blackbox checker based verification methodology fails to keep up to provide enough observability needed. Assertion-based verification (ABV) methodology is widely recognized as a solution to this problem. ABV is a methodology in which designers use assertions to capture specific internal design intent or interface specification and, either through simulation, formal verification, or emulation of these assertions, verify that the design correctly implements that intent. Assertions actively monitor a design (or testbench) to ensure correct functional behavior. They detect design errors at their source, greatly increasing observability and decreasing debugging.
Benchmarking Functional Verification
by Mike Bartley and Mike Benjamin, Test and Verification Solutions
Over the years there have been numerous attempts to develop benchmarking methodologies. One of the most widely used is the Capability Maturity Model (CMMI) developed by the Software Engineering Institute at Carnegie Mellon University. Although aimed at software engineering it provides a framework that is widely applicable to most business activities. However, whilst we have drawn considerable inspiration from CMMI, it has a number of serious limitations when trying to use it to benchmark a highly specific activity such as functional verification.
Universal Verification Methodology (UVM)-based SystemVerilog Testbench for VITAL Models
by Tanja Cotra, Program Manager, HDL Design House
With the increasing number of different VITAL model families, there is a need to develop a base Verification Environment (VE) which can be reused with each new VITAL model family. UVM methodology applied to the SystemVerilog Testbench for the VITAL models should provide a unique VE. The reusability of such UVM VE is the key benefit compared to the standard approach (direct testing) of VITAL models verification. Also, it incorporates the rule of "4 Cs" (Con-figuration, Constraints, Checkers and Coverage). Thus, instead of writing specific tests for each DUT feature, a single test can be randomized and run as part of regression which speeds up the collection of functional coverage. The results show that UVM VE testbench, with respect to a standard direct test bench, requires nearly equal time to develop. In return it provides re-usability and much faster verification of each new VITAL model. The changes one needs to do are mainly related to the test where the appropriate configuration must be applied.
Efficient Failure Triage with Automated Debug: a Case Study
by Sean Safarpour, Evean Qin, and Mustafa Abbas, Vennsa Technologies Inc.
Functional debug is a dreadful yet necessary part of today's verification effort. At the 2010 Microprocessor Test and Verification Workshop experts agreed that debug consumes approximately one third of the design development time. Typically, debugging occurs in two steps, triage and root cause analysis. The triage step is applied when bugs are first discovered through debugging regression failures. This occurs at the chip, system or sub-system level, where a verification engineer will group different failures together and perform an initial investigation to determine which engineers should deal with the bug. We refer to this as the triage step due to its similarity to how hospital emergency rooms assess incoming patients and determine the next step for treatment. Once the bug has been passed on, the root cause analysis step begins. Here, the engineer responsible for the bug will determine its full cause and how to fix it. Both triage and root cause analysis must be performed accurately and efficiently to result in an effective debug process. This article focuses on the often neglected pain of triage. It presents a case study where a UVM verification environment is enhanced with powerful automation tools to improve the overall debug effort. More specifically, the use of Vennsa's OnPoint and Mentor Graphics' Questa suite of tools can distinguish between multiple error sources in the design and the testbench as well as reduce the debugging time by combining different failures of the same error source.
Are OVM & UVM Macros Evil? A Cost-Benefit Analysis
by Adam Erickson, Mentor Graphics Corporation
Are macros evil? Well, yes and no. Macros are an unavoidable and integral part of any piece of software, and the Open Verification Methodology (OVM) and Universal Verification Methodology (UVM) libraries are no exception. Macros should be employed sparingly to ease repetitive typing of small bits of code, to hide implementation differences or limitations among the vendors’ simulators, or to ensure correct operation of critical features. Although the benefits of the OVM and UVM macros may be obvious and immediate, benchmarks and recurring support issues have exposed their hidden costs. Some macros expand into large blocks of complex code that end up hurting performance and productivity, while others unnecessarily obscure and limit usage of otherwise simple, flexible APIs.The ovm_field macros in particular have long-term costs that far exceed their short-term benefit. While they save you the one-time cost of writing implementations, their run-time performance and debug costs are incurred over and over again. Consider the extent of reuse across thousands of simulation runs, across projects, and, for VIP, across the industry. These costs increase disproportionately with increased reuse, which runs counter to the goals of reuse. In most cases, it takes a short amount of time and far fewer lines of code to replace a macro with a "direct" implementation. Testbenches would be smaller and run faster with much less code to learn and debug. The costs are fixed and up-front, and the performance and productivity benefits increase with reuse.
February 2011 | Volume 7, Issue 1
Verification Horizons Complete Issue: