Most of our techniques treat tested software as a black box, using an executable implementation instead of an abstract algorithm description. We use the theoretical relationship between algorithm stability, problem condition, and errors in computed results to determine to what extent algorithm instability, rather than problem ill-conditioning, is responsible for inaccurate results. Using a robust optimization method to search for examples of extreme numerical instability, we can assess how stable or unstable a piece of numerical software is. If examples of instability are found, we turn our attention inside the code to localize and correct the cause of the instability.
Our approach avoids the reliability obstacles of traditional analysis and testing, and is applicable to virtually any type of numerical software that can be put in executable form.