Verification and Testing

Verification is the golden seal that validates an implemented system according to its specification. When it comes to silicon intellectual property (IP), where ASIC fabrication costs millions of dollars, companies can only afford a few fabrication runs. Test-driven development is the only way to verify that the design works according to its specifications.

Often times, verification infrastructure may extend beyond validation and can offer insights for product development, as illustrated in the following case study:


Case Study

OrganizationNGCodec Inc
Business UnitAlgorithm Verification
Design ObjectiveDetermine video-quality impact of proposed video compression algorithm alternatives.
Deliverables
  • Command-line utility to orchestrate thousands of tests using a concise syntax
  • Command-line utility to compute objective quality metrics on output products
  • Command-line utility to aggregate specific metrics from
  • Directory structure specification for output products and metric reports
Implementation
  1. Orchestrate encoding operations using a number of raw videos, sweeping parameters of interest to the study
  2. Score outputs according to visual-quality assessment tools (PSNR, SSIM, VMAF)
  3. Aggregate scores to prepare concise charts for review by algorithm developers, to determine if there are actionable patterns
Results
  • Promising insights for algorithm optimization.
  • Increased test coverage, found defects in corner cases during parameter sweeps.