Mutation testing

Mutation testing (sometimes also called mutation analysys) is a method of software testing, which involves modifying program's source code in small ways.[1] These, so-called mutations, are based on well-defined mutation operators that either mimic typical user mistakes (such as using the wrong operator or variable name) or force the creation of valuable tests (such as driving each expression to zero). The purpose is to help the tester develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution.

Pioneered in the 1970s, mutation testing was originally intended to locate and expose weaknesses in test suites. The theory was that if a mutation was introduced without the behavior (generally output) of the program being affected, this indicated either that the code that had been mutated was never executed (redundant code) or that the testing suite was unable to locate the fault. In order for this to function at any scale, a large number of mutations had to be introduced into a large program, leading to the compilation and execution of an extremely large number of copies of the program. This problem of the expense of mutation testing, has reduced its practical use as a method of software testing.

Mutation testing was originally proposed by Dick Lipton as a student, and first developed and published by DeMillo, Lipton and Sayward. The first implementation of a mutation testing tool was by Timothy Budd as part of his PhD work (titled Mutation Analysis) in 1980 from Yale University.

Recently, with the availability of massive computing power, there has been a resurgence of mutation analysis within computer science community, and work has been done to define methods of applying mutation testing to object oriented programming languages and non-procedural languages such as XML, SMV, and finite state machines.

Mutation testing overview
Mutation testing is done by selecting a set of mutation operators and then applying them to the source program one at a time for each applicable piece of the source code. The result of applying of one mutation operator to the program is called a mutant. If the test suite is able to detect the change (i.e. one of tests fails), then mutant is said to be killed.

For example, let's consider the following C++ code fragment:

if (a && b)
    c = 1;
else
    c = 0;
The condition mutation operator would replace '&&' with '||' and produce the following mutant:

if (a || b)
    c = 1;
else
    c = 0;
Now, for the test to kill this mutant, the following condition should be met:

Test input data should cause different program states for mutant and original program. For example, a test with a=1 and b=0 would do this.
The value of 'c' should be propagated to the program's output and checked by the test.
Weak mutation testing (or weak mutation coverage) requires that only the first condition is satisfied. Strong mutation testing requires that both conditions are satisfied. Strong mutation is more poweful, since it ensures that the test suite can really catch the problems. Weak mutation is closely related to code coverage methods. It requires much less computing power to ensure that the test suite satisfies weak mutation testing than strong mutation testing.

Mutation operators
  This short section requires expansion.

A variety of mutation operators was explored by researchers. There are mutation operators for arithmetic and boolean operations, for concurrent constructions[2], complex objects like containers[3], etc.

References
^ A Practical System for Mutation Testing: Help for the Common Programmer by A. Jefferson Offutt.
^ Mutation Operators for Concurrent Java (J2SE 5.0) by Jeremy S. Bradbury, James R. Cordy, Juergen Dingel.
^ Mutation of Java Objects by Roger T. Alexander, James M. Bieman, Sudipto Ghosh, Bixia Ji.

External links
Mutation testing online an open community which brings together the Hardware and Software research communities studying mutation testing.
Mutation testing list of tools and publications by Jeff Offutt.
Mutation testing tools
An Overview of Mutation Testing by Konstantinos Adamopoulos.

本文来自CSDN博客，转载请标明出处：http://blog.csdn.net/dawnwang/archive/2007/09/08/1777496.aspx