Test Case Generation, UML, and Eclipse（转载）

Test Case Generation, UML, and Eclipse

Encouraging good practices to ensure software quality

(Page 1 of 2)

Luis Fernández-Sanz and Pedro J. Lara-Bercial

Standard notations and open environments make useful tools for software quality-assurance techniques possible.

Luis is an assistant professor of computer science at Universidad deAlcala. Pedro is director of the Computer Systems Department at theUniversidad Europea de Madrid. They can be contacted at [e-mail] and [e-mail], respectively.

 

---

 

 

Automatic test case generation has long been on developers' wishlists,and a promising approach to implementing it is the use of statediagrams to describe interactions. The idea is that diagrams can guidethe generation of test cases based on the assumption that user activitywith the program is modeled by the diagram.

 

The approach we propose in this article is to link requirementsexpressed in UML format with a test-generation method we call "AQUABUS"(esp.uem.es/aquabus/index.htm). To do so, we use UML models stored in an Eclipse project to create a plug-in called "AGAPE" (also available at esp.uem.es/aquabus/index.htm) that automatically generates test cases from software specifications based on use cases and activity diagrams.

 

Our approach with AQUABUS is to extend an activity diagram to includeinformation about the use and severity of possible failures for eachelement as a starting point for the generation of test cases. AQUABUScreates a list of paths from the initial state to the ending one, thenranks them in order of priority using the information about probabilityof use and cost/severity of possible failures.

 

This approach is in sync with commonly used procedures for generatingtest cases based on the UI. However, this isn't practical if you can'tsupport some type of tool or automated environment. So in this article,we present a solution that uses Eclipse to develop solutions thatintegrate UML modeling with test-generation aids.

AGAPE: A Test Case Generation Tool

 

AGAPE is a test generation plug-in for Eclipse. It is based on the EclipseUML (www.omondo.com) UML modeling plug-in and the Eclipse Modeling Framework (www.eclipse.org/modeling/emf)for diagramming. AGAPE uses the XMI definition of the activity diagramsassociated with use cases as an input so you can complement the diagramwith additional information about:

 

• Input data in specific states where users have to enter information in the system.
• Probability of "execution" for each transition (arrow) in the diagram.
• Severity (cost of experiencing an error) associated with the execution of an activity of the diagram.

 

 

AGAPE provides the following functions when using UML models to design tests:

 

• Modeling UML activity diagrams within the standardstructure of an Eclipse project, allowing that analysts allocateprobability and/or cost information to diagram elements to determinerisk level associated to each path (scenario of the use case); that is,to each of the test cases that may be generated from the diagram usingthe AQUABUS algorithm.
• Graphical design and management ofdiagram elements, allowing the insertion of specific states (see Figure1) like any other CASE tool. It lets you save diagrams in XMI format,as in Listing One.
• Management of data input activities bothgraphically (Figure 1) and via dialogs (Figure 2). The form isavailable for each specific activity related to data input to allowentering value ranges and equivalence classes both for valid andinvalid data. Of course, there are also specific fields in the form forentering probability of use and cost of possible failures for eachoption.
• Test case generation—the essential objective ofAGAPE. Once the activity diagram is specified and modeled, the toolgenerates paths from the beginning to the end of the use case. Each ofthese paths corresponds to a test case that can be stored as a textfile (in XML format) and imported by existing automatic test executiontools. Listing Two shows the test cases generated from Figure 1 wherethere are only three paths because we have only established two data tobe tested in the activity AD1. In general, data input activities shouldinclude several equivalent classes (valid and invalid values) that arerepresented as different paths in the diagram.

 

 

[Click image to view at full size]

 

Figure 1: Activity diagram modeled with AGAPE.

 

[Click image to view at full size]

 

Figure 2: Form for data input activities.

 

<?xml version='1.0' encoding='iso-8859-1'?>
<activityDiagram>
  <start-point id="w11190188197910" itemName="start-point" x="387" y="14"/>
  <activity id="w11190188197911" itemName="A1" x="361" y="50"/>
  <decision id="w11190188197913" itemName="decision0" x="386" y="114"/>
  <activityData id="w11190188197918" itemName="AD1" x="74" y="166" w="336" h="150">
     <data id="w11190188197919" itemName="d1" x="0" y="0">
        <value id="w111901881979110" itemName="d1@value0" x="123" y="258" value="0" type="Valid"/>
     </data>
     <data id="w111901881979114" itemName="d2" x="0" y="0">
         <value id="w111901881979115" itemName="d2@value2" x="210" y="246" value="-1" type="Valid"/>
     </data>
</activityData>
<activity id="w11190188197916" itemName="A2" x="528" y="208"/>
<end-point id="w111901881979121" itemName="end-point" x="384" y="375"/>
<connections>
   <sourceConnection id="w11190188197914" source="w11190188197913" target="w11190188197916" p="0.5" c="3"/>
   <sourceConnection id="w11190188197912" source="w11190188197911" target="w11190188197913" p="" c=""/>
   <sourceConnection id="w111901881979119" source="w11190188197918" target="w111901881979121" p="" c=""/>
   <sourceConnection id="w111901881979120" source="w11190188197918" target="w111901881979115" p="0.7" c="3"/>
   <sourceConnection id="w111901881979117" source="w11190188197918" target="w111901881979110" p="0.5" c="4"/>
   <sourceConnection id="w111901881979116" source="w111901881979115" target="w11190188197918" p="" c=""/>
   <sourceConnection id="w11195300692600" source="w11190188197910" target="w11190188197911" p="" c=""/>
   <sourceConnection id="w111901881979111" source="w111901881979110" target="w11190188197918" p="" c=""/>
   <sourceConnection id="w11190188197917" source="w11190188197916" target="w111901881979121" p="" c=""/>
   <sourceConnection id="w11190188197915" source="w11190188197913" target="w11190188197918" p="0.5" c="5"/>
</connections>
</activityDiagram>

Listing One

 

Figure 3, an overview of the design, illustrates the classes that letAGAPE model the extended activity diagrams with the requiredcharacteristics to comply with the properties of the Eclipse GraphicalEditing Framework (www.eclipse.org/gef) GEF and Eclipse Modeling Framework. The basic element is ActivityDiagram broken down into elements Children that may be linked (Connections) among them (each link with probability of use and severity of possible failures).

 

[Click image to view at full size]

 

Figure 3: Class diagram referred to the part of AGAPE in charge of modeling extended diagrams.

 

 

<?xml version='1.0' encoding='iso-8859-1'?>
<testCases>
   <path id='0'>
      <children itemName='start-point'/>
      <children itemName='A1' p='' c=''/>
      <children itemName='decision0' p='' c=''/>
      <children itemName='A2' p='0.5' c='3'/>
      <children itemName='end-point'/>
  </path>
  <path id='1'>
      <children itemName='start-point'/>
      <children itemName='A1' p='' c=''/>
         <children itemName='decision0' p='' c=''/>
         <children itemName='AD1' p='0.5' c='5'>
            <value dataValue='d1=0' p='0.5' c='4'/>
         </children>
         <children itemName='end-point'/>
   </path>
   <path id='2'>
      <children itemName='start-point'/>
      <children itemName='A1' p='' c=''/>
      <children itemName='decision0' p='' c=''/>
      <children itemName='AD1' p='0.5' c='5'>
          <value dataValue='d2=-1' p='0.3' c='3'/>
      </children>
      <children itemName='end-point'/>
   </path>
</testCases>

Benefits

 

To determine the benefits of AQUABUS and AGAPE thanks to the support oftwo projects (TIN2007-67843-C06-01 and TIN2005-24792-E funded by theSpanish Ministry of Science), we collected performance data from morethan 100 project leaders, analysts, programmers, testers, teachers,consultants, and the like, based on the following:

 

• We published on a web site a simple data management application for managing a DVD collection.
• Participants tested it, recording the test cases they did.
• Recorded test cases from the participants were compared with the set of test cases generated with AGAPE (AQUABUS).
• Participantshad access to the results of their tests and the comparison with theAGAPE test cases. They were asked to give their opinion via aquestionnaire designed to analyze their evaluation, both of the methodand the plug-in. They were also asked to give a priority mark to eachtest case according to their idea of importance for the softwareapplication.

 

 

Given that the system recorded tests proposed and time devoted to thetask (mean time of 21 minutes), poor-quality results (short time, fewtest cases) were discarded, leaving only 72 valid test sets. Althoughwe collected lots of interesting information, for the purposes of thisarticle we highlighted the following results:

 

• None of the participants included any test case ordifferent scenario (path of activity diagram) that was not covered bythe tests design generated by the algorithm.
• Only oneparticipant reached more than 75 percent of the total existing paths;more than 50 percent did not reach the 50 percent of thepaths/scenarios. Test cases designed by 50 (70 percent) of theparticipants hardly surpassed the threshold of 25 percent of totalscenarios.
• Testers who used activity diagrams as reference designed 70 percent more test cases than those who did not.
• Amongthe 10 most tested paths by all participants, only one (in eighthplace) appears in the 10 top priority paths for them. Within the listof the 10 least tested paths, three of the top priority paths areincluded. Even more, at least 50 percent of the testing effort wasdedicated to test something previously tested.
• Regardingthe usefulness of the method and plug-in, 77 percent of the testerssaid that AQUABUS would be really cost-effective for theirorganizations, although activity diagrams are needed as part ofsoftware specifications. Seventy percent of the participants believethe priority allocated to the different scenarios following the AQUABUSmethod would be cost-effective.

 

 

Conclusion

 

To encourage good practices for controlling software quality,developers clearly need integrated environments that take advantage ofthe cross relationships between analysis and modeling. Althoughexisting tools support certain quality assurance activities,integration isn't always easy because of problems with communicationbetween tools, or the lack of techniques to exploit models for qualitycontrol. However, standard notation like UML and open environments likeEclipse make possible useful tools that go one step further in makingaffordable and convincing implementations of software quality assurancetechniques. The AGAPE plug-in lets you concentrate on requirements andmodels, and with a simple click lets you generate a whole set of testcases with a priority rank.