Software Testing
Software Testing
Carnegie Mellon University
18-849b Dependable Embedded Systems
Spring 1999
Authors: Jiantao Pan mailto:%20jpan@cmu.edu Abstract:
Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. [Hetzel88] Although crucial to software quality and widely deployed by programmers and testers, software testing still remains an art, due to limited understanding of the principles of software. The difficulty in software testing stems from the complexity of software: we can not completely test a program with moderate complexity. Testing is more than just debugging. The purpose of testing can be quality assurance, verification and validation, or reliability estimation. Testing can be used as a generic metric as well. Correctness testing and reliability testing are two major areas of testing. Software testing is a trade-off between budget, time and quality.
Contents: * Introduction * Key Concepts * Taxonomy * Testing automation * When to stop testing? * Alternatives to testing * Available tools, techniques, and metrics * Relationship to other topics * Conclusions * Annotated Reference List & Further Reading
Introduction
Software Testing is the process of executing a program or system with the intent of finding errors. [Myers79] Or, it involves any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. [Hetzel88] Software is not unlike other physical processes where inputs are received and outputs are produced. Where software differs is in the manner in which it fails. Most physical systems fail in a fixed (and reasonably small) set of ways. By contrast, software can fail in many bizarre ways. Detecting all of the different failure modes for software is generally infeasible. [Rstcorp]
Unlike most physical systems, most of
Carnegie Mellon University
18-849b Dependable Embedded Systems
Spring 1999
Authors: Jiantao Pan mailto:%20jpan@cmu.edu Abstract:
Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. [Hetzel88] Although crucial to software quality and widely deployed by programmers and testers, software testing still remains an art, due to limited understanding of the principles of software. The difficulty in software testing stems from the complexity of software: we can not completely test a program with moderate complexity. Testing is more than just debugging. The purpose of testing can be quality assurance, verification and validation, or reliability estimation. Testing can be used as a generic metric as well. Correctness testing and reliability testing are two major areas of testing. Software testing is a trade-off between budget, time and quality.
Contents: * Introduction * Key Concepts * Taxonomy * Testing automation * When to stop testing? * Alternatives to testing * Available tools, techniques, and metrics * Relationship to other topics * Conclusions * Annotated Reference List & Further Reading
Introduction
Software Testing is the process of executing a program or system with the intent of finding errors. [Myers79] Or, it involves any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. [Hetzel88] Software is not unlike other physical processes where inputs are received and outputs are produced. Where software differs is in the manner in which it fails. Most physical systems fail in a fixed (and reasonably small) set of ways. By contrast, software can fail in many bizarre ways. Detecting all of the different failure modes for software is generally infeasible. [Rstcorp]
Unlike most physical systems, most of
References: * [Basili85] Victor R. Basili, Richard W. Selby, Jr. "Comparing the Effectiveness of Software Testing Strategies", Technical Report, Department of Computer Science, University of Maryland, College Park, 1985 * [Beizer90] Boris Beizer, Software Testing Techniques. Second edition. 1990 * A very comprehensive book on the testing techniques In-depth discussion about functional testing throughout all product life-cycle. * [IEEE90] IEEE Standard Glossary of Software Engineering Terminology (IEEE Std 610.12-1990), IEEE Computer Soc., Dec * Introduction of the tools BoundsChecker by NuMega * [Parrington89] Norman Parrington and Marc Roper, Understanding Software Testing, Published by John Willey & Sons, 1989