Speaking of Verification

A conversation with Harry Foster ...

by Peggy Aycinena

There were at least 3 panels last week at DesignCon that put verification front and center. The panels included:

* "Taking the Pain out of Verification: Exploring What Has to Change & Why"

Moderated by Gabe Moretti

Panelists included: Harry Foster (Jasper Design), Kevin Normoyle (Azul Systems), John Goodenough (ARM), Andrew Piziali (Verisity), and Harry Stiumer (Sun).

* "Design Verification: From Specification to Closure"

Moderated by Sergio Camerlo

Panelists included: Dhrumil Gandhi (ARM), Yaron Kashai (Verisity), Sudhakar Sabada (LSI), Joe Sawicki (Mentor Graphics), and Ronnie Vasishta (eASIC).

* "Are We Spending our Verification Resources Wisely?"

Moderated by Brian Bailey

Panelists included: Peter Becker (Software Prototype Technology), Janick Bergeron (Synopsys), Ira Chayut (nVidia), Sunil Kakkar (Freescale), and Michael McKinney (TI).

*****************************

As a student of verification, I diligently attended each panel, diligently jotted down notes and comments from each panel, and then conducted the following interview based on those notes.

This interview is a lengthy discussion with Harry Foster, Chief Methodologist at Jasper Design Systems. Prior to Jasper, Harry was Chief Architect at Verplex Systems, and prior to that – a senior member of the CAD Technical Staff at Hewlett-Packard. Harry is author of several books on verification: "Assertion-Based Design" and "Principles of Verifiable RTL Design." He has another book coming out in March, and is writing yet another book even as we speak. Harry has also been active in the standards area. He created the Accellera Open Verification Library (OVL) assertion monitor standard and is currently serving as Chairman of the Accellera Formal Verification Technical Committee.

As with several other people listed above, Harry Foster's name is a household word in the world of verification. So I was delighted to have him field these questions, gleaned from the panels at DesignCon. I was also delighted with the ease and good humor that he brought to the conversation.

*****************************

Harry Foster: I agree that we're still talking about things that we've been talking about for some time. As I said in my opening statement on Gabe's panel – today's verification methods are based on yesterday's best practices. We're so hung up on things like – "If only I could speed up my simulation, or if only I could think of all of the different scenarios for my system, I could fully verify it." But yesterday's solutions don't scale. It doesn't matter that we've debated the same things over and over again, the traditional technology just doesn't scale.

I come from a background in both design and developing tools. When I was at HP, our goal was to get to the lab as soon a possible with the design, and we always scheduled in a re-spin in the development cycle. We wanted to find out what was going on – and the design cycle was 3 to 5 years.

[In that timeframe], EDA technology was for high-end systems and high-end chips, but I've seen a shift over the past couple of years. It's no longer the high-end type of design that's driving EDA, it's the consumer space. The markets are changing at a much quicker pace than we're were used to in the past, and because in this kind of dynamic market, feature changes are being introduced late in the design flow.

Traditional simulation approaches don't deal well with late-date changes in specifications. So, meeting the demands of customers in the shifting consumer market is adding to the verification challenge. Today, the entire industry is very dynamic in its demands. [And of course], the short design cycles can't afford re-spins.

Harry Foster: Actually, that's kind of interesting. If you look at the investment in verification versus the design investment needed to develop a product – in general, in terms of the money and the bodies involved, at the back-end, you're using about $5 million in tools and approximately 6 people to do the place and route. There's a lot of automation there.

But if you look at the front-end, you've got about $1 million in tools and approximately 18 people. The investment in people is there because it's still a very manual process that goes back to writing testbenches. You've got to sit there and think of all possible scenarios, and then write correct and adequate testbenches to provide the necessary coverage. We're throwing a lot of bodies at this process because we're lacking the automation.

Of course, even if we could fully automate the definition of coverage, it would be impossible to fully execute. As I said on Gabe's panel, for a 32-bit comparator, you would need 600,000 years to simulate the design – even if you could automate the process of writing the testbenches. I think that when you've got a problem as complex as this, instead of trying to scale existing solutions, you've got to pull the problem apart so that it's not so huge.

Harry Foster: The process is based on using traditional simulation – again, yesterday's best practices. We start with the architectural specification, and that specification itself is a problem. For a lot of people, it's a document that's actually written after the design is done. [Anyway], we can create the specification in C, SystemC, or some ESL-type model in order to validate the architecture.

The next step is to take the architectural specification and go down to a micro-architectural specification. Sometimes, people call that spec the design specification. Essentially, we take the algorithms and figure out how to partition them for implementation. The micro-architectural specification has to consider the performance – for instance, the latency of registers needs to be evaluated.

Now the partitioned pieces are picked up by the designers, and by the verification engineers. And this is where the great divide takes place – which is not a bad idea. The verification engineer looks at the specification, and comes up with a list of things that they need to check. They think about how to generate stimulus for the testbench simulation – random or direct. They ask, "How do we know that we've generated all possible scenarios?" Then, they think about coverage. The state of the art is at the functional coverage level, and maybe at the transaction level. Once all the functional coverage is defined, then they start simulation. Meanwhile, the designer is also working from the specification. He's done his interpretation of the spec, and gone off with the implementation.

The idea here is that both the designer and the verification engineer will then come back together for the verification process. The RTL is checked out – the verification guys check it out – and their interpretation of the spec versus the interpretation of the designer is compared. Hopefully, that process will uncover any problems in the specification.

So, we simulate and we look at the functional coverage. There has been some progress in automation in this area, with some of the tools attempting to create vectors to hit the functional coverage. You keep going until you've hit all of your functional coverage points. Of course, the coverage is only a measure of the quality of the vectors applied to the design.

Then you tape out.

Harry Foster: On Gabe Moretti's panel, he asked each of us, "How do you measure a successful verification process?" I said the goal of design development is to fabricate something that can be sold, so the company can make a profit. The reality is, the only way you can measure a good verification process is to measure the time to profit.

The product lifecycle today for a cell phone, for instance, is 6 months. If you slip the development schedule by 6 weeks, you've lost 25 percent of the profits. It's true that it's not a linear curve – there's more profit at the beginning of the lifecycle – but in any case, the reason I use time to profit as a measure of success is because that's what really matters. You need a predictable development schedule that produces a quality product, and to do that you need a verification strategy that will result in [an optimal] time to profit.

Harry Foster: The reality is that at the end of the lifecycle of your product – not at the end of the development cycle – somebody else has produced something that's wowing the customer, so clearly, the end of the lifecycle of your product is a lot easier to pin point than the end date for the development cycle. But, you still have to establish a development schedule and try to meet it.

Harry Foster: It all comes back to the fact that time to profit is the only thing that matters. That's what you have to look at to know if the customer is being successful. However, the more important question is whether the customer is willing to embrace new ideas. Are they willing to move forward, or are they still focused on yesterday's best practices?

Traditionally, engineering managers have been conservative. They make only incremental changes between projects. That's not surprising; you don't want to have a lot of new unknowns on a project. But at the same time, you certainly have to do a good post mortum on each project, and then look at what's now available [to improve on the process]. You need to be willing to invest in trying new ideas.

When we're talking to customers, we've got to talk to both the engineers and the people who control the schedule. If you just talk to the designers, they've resigned themselves to living in a narrow box. Frequently, they don't have the necessary vision to see beyond that. That's not a negative comment, it's just that the engineers have got so much work to do. I don't know any designer who wants to do something extra – they're already so overloaded. So you've got to talk to the people who own the development schedule. Once you've gotten by those people, the designers will follow along.

Harry Foster: I'm a little cautious about that strong of a statement. One of the problems is the separation between the two roles. The reality is that designers are actually part of the verification process. There are some instances where up to 35 percent of the designer's time is pulled into verification. That makes the whole product development schedule, or predictability, go out the window. Debugging is actually the biggest bottleneck in the flow, but managers don't adequately prepare for it. Meanwhile, verification engineers are in short supply because of the way we do verification, verification being an extremely manual process.

The reality is that designers want their designs to work, but they've lacked the tools to insure they can do that. The problem again is that the verification processes that people are using today are the traditional verification processes, and are based on yesterday's ideas.

Harry Foster: I can't imagine anybody doing work where they don't have pride in what they're doing.

Harry Foster: I think it's a fairly simplistic to think that it's just the smaller geometeries that are the problem. Today we're doing block-based designs, but trying to use traditional simulation-based approaches to verify those designs. However, that's impossible because you can't test every single scenario on a system built up of IP blocks using traditional simulation scenarios.

Certainly today, the IP vendors have a lot of responsibility for delivering high quality IP. But, I believe there's a false sense of security out there that if you just buy IP, it will work. You've got two issues here – compliance and interoperability. The verification process today has to take into account both of these things. Compliance – as in does the IP comply with the specifications – and interoperability – can the IP blocks work together in the system? I can't just put the IP block into the system without worrying about its interoperability with the other blocks. Then to verify the system, I have to think of all possible scenarios to validate the design.

There are other factors contributing to the crisis as well. One of our customers is a large consumer product company in Japan who has been designing a memory controller for use in a number of different products. This controller has to support multiple configurations, which is a big problem [from a verification point of view]. You can't predict what every scenario will be for a design when it's going to be implemented in a variety of configurations. You've got to find a different strategy.

These are the situations that attract me to formal verification. I only have to state what the system has to do, and formal verification calculates mathematically all of the different scenarios required to execute that specification.

Harry Foster: When I say verification – and it applies to timing, physical, functional, etc. – in general, verification is a process of ensuring that the implementation satisfies the specification. And usually, that's a lot easier said than done.

Functional verification is a process of ensuring the implementation satisfies the specification. But we're being specific – it's the functionality of the design, rather than the timing for instance, that we're looking at.

Formal verification is a form of functional verification. It's a systematic process that's mathematically precise in being sure that the implementation satisfies the specification. Conceptually, formal verification reverse engineers a design with respect to the specification. It tries to find all sequences that lead to the implementation. There are no test vectors involved. The beauty of formal verification is that I don't have to think up all possible scenarios. I only have to tell the tool what the system needs to, and it reverse engineers the design to determine which sequences would break the implementation.

There's an analogy in designing a car. If one of the requirements in my car is to have a fuse that won't blow up under any situation – then I could start the car, take it to 37 miles per hour, roll down the window, play a CD and see if the fuse would blow. If it doesn’t, then I could take the car to 45 miles per hour, and if the fuse then blows I would have a scenario that [invalidates] the design. But with this strategy, I'm forced to think of all possible input scenarios to check if everything is true. However, you can't ever think of all input scenarios. Using formal verification, this wouldn't be necessary.

Harry Foster: Well, the first generation formal tools couldn't handle the required capacity. Plus, people got into the space way too soon. The water was tainted by the first-generation tools that couldn't deliver. For a comparison, you have to look back at the early days back in 1993 when Chrysalis came out with equivalence checking. That technology had a lot of capacity issues as well, and it wasn't until 1998 or '99 when Verplex and Cadence were able to [capitalize on the technology]. It took about 5 years for equivalence checking to be [fully accepted]. Today, we're in the 1995 to '97 timeframe with respect to formal verification [being fully accepted].

Harry Foster: Perhaps the VCs didn’t do their homework before investing in all of those companies in the late '90s. By 2000 or 2001, it was crazy – people were throwing money at anything. There were a lot of tools developed in that period, particularly in formal verification, but they were just too early in the market. There was an inherent limitation in the algorithms in the technology. All of those little start-ups were just trying to tweak the algorithms, not change them. But there were problems with capacity.

There's a good analogy in physical design. In place and route, if you attempt to throw the whole chip at the place and route tools, it just blows up. You have to floorplan first with these big chips, and only then does the algorithm for place and route become tractable.

Similarly, in the early years of verification – tweaking the algorithms wasn't enough. Now, we're doing something different – we're adding something else on top of those algorithms to address the capacity requirements for today's chips. What we've done here at Jasper is to create an interactive formal verification tool. We've got a way to partition the design into tractable sizes instead of throwing the whole chip at the formal engine.

Harry Foster: Formal verification of the whole chip is a huge problem, comparable to doing place and route on the whole chip. Currently it just doesn’t work, but I can envision a time when the tools will be here. I've seen a lot of radical changes in the technology over the last 15 to 20 years. I, myself, have worked in timing, in place and route, in synthesis, and so forth, and I've seen a lot of breakthroughs. I'm encouraged that a lot more breakthroughs are going to happen, but we can't just sit around and wait for those things. We have to make them happen.

Harry Foster: Previously, the solutions were more painful than the problems, but now we're at a point where the problems are becoming more painful than the solutions. That's always the time when new ideas are embraced.

Harry Foster: In the past couple of years, there have been various breakthroughs. Prior to 2 or 3 years ago, we didn't have a common way of specifying design intent. You could use pure Verilog, for instance. But today, there's a standard for that process and it really helps. Now I can state the intent of my design in a way that multiple tools can understand. Specification is actually something that I'm focusing on – the new ways that people can make the specifications, so that the design is more verifiable.

There have been other breakthroughs, as well. As I mentioned, we're moving beyond just tweaking the verification algorithms – we're moving to the next level where multiple engines can automatically partition the design.

I think that people are seeing that simulation is running out of gas. In my mind, the old way fell apart starting in the mid 1990's. For now, however, they're still throwing larger and larger pieces of the design at system-level simulation – in fact, I understand that only about one third of the industry is actually doing comprehensive block-level simulation. Clearly, this has to change.

We need to start using a combination of formal verification and traditional techniques. What I'm proposing is a focus at the block level, to use formal verification on the blocks, and then integrate the blocks into the system for a system-level simulation. I have evidence from customers that [this strategy] is successful. It's actually a new way of doing design and verification. Clearly, it's finally time to start moving ahead.

****************************

Editor's Note: The conversation about verification continues this coming week at DVCon 2005 in Silicon Valley. If you're interested in the issues and controversies, you should be attending.

****************************

February 11, 2005

Peggy Aycinena owns and operates EDA Confidential. She can be reached at peggy@aycinena.com

Copyright (c) 2005, Peggy Aycinena. All rights reserved.