Cruise cotnrol for Tofino UI automation

Requirements

Software/component

Source

comments

cruisecontrol-bin-2.8.3.zip

http://sourceforge.net/projects/cruisecontrol/files/CruiseControl/2.8.3/

An open source CI sever, can be upgraded on demand.

JDK6_lv.zip

http://www.cr173.com/soft/61653.html

Green JDK, can be used directly without installing, can  be upgraded on demand.

VS2012.4 RC Agents ENU.iso

http://www.microsoft.com/en-us/download/details.aspx?id=40338

The latest VS2012 test agent (till 2013/10/17) which  contains the MSTEST component, Coded UI test cases can be run against it on  IE9 and above version;

The command to install test agent: vstf_testagent.exe /Q /S /Passive /Full;

The test agent should be updated the first time when  new version is published,  otherwise  tests can run into unexpected failures.

e.g.: see this email -                                                                 

 

Set up Cruisecontrol

1.      Download JDK

Install it and set the JAVA_HOME environment variable.

Sample: jdk-7u40-windows-x64.exe /quiet /passive

Or you can use the green JDK above.

2.      Download and configure Cruisecontrol

This version of CruiseControl comes with adefault sample project – connectfour

Sampe CC config file - cruisecontrol-bin-2.8.3\config.xml

Ant build file: cruisecontrol-bin-2.8.3\projects\connectfour\build.xml

 

The CC entry file to start CC is“cruisecontrol-bin-2.8.3\cruisecontrol.bat” in windows OS, it sets the default settings for CC andstart it there.

The web port for CC dashboard is set to 8080 inthat entry file, but 8080 is usually the default port for TFS server, so it’ssuggested to use another port for CC’s dashboard to avoid the possibleBindException like below:

 

3.      Start Cruisecontrol

Now you can run CC entry file“cruisecontrol-bin-2.8.3\cruisecontrol.bat” to start CC and view the build status on its dashboard:http://localhost:9090/dashboard

Or the build result on its report portal:http://localhost:9090/cruisecontrol

 

Advancedconfigurations

1.      Customize the CC entry file

Because the test environment is not always theway we think it should be, it’s advisable to create a custom CC entry fileinstead of directly calling cruisecontrol.bat.

Here is one I created– startCC.bat(put it the same path hierarchy as “cruisecontrol-bin-2.8.3”)

@echo off

echo startCC.bat call cruisecontrol.bat for build

call CheckJDK.bat

set here=%~dp0

set CCDIR=%here%cruisecontrol-bin-2.8.3

for /f "tokens=* delims= " %%i in('RegistryQuery.exe"'HKEY_CURRENT_USER\\Software\\Microsoft\\VisualStudio\\11.0_Config''InstallDir'"') do (

set VS2012_InstalledPath=%%i

)

REM echo VS2012_InstalledPath=%VS2012_InstalledPath%

set MSTEST=%VS2012_InstalledPath%

set ANT_HOME=%CCDIR%/apache-ant-1.7.0

set PATH=%JAVA_HOME%\bin;%ANT_HOME%\bin;%PATH%

cd %CCDIR%

del /F "*.ser"

"%CCDIR%/cruisecontrol.bat" -jmxport 9009-webport 9090 -configfile "config - cruisecontrol.xml" -webapppath %CCDIR%/webapps/cruisecontrol-dashboard %CCDIR%/webapps/dashboard

 

 

2.      Set up the trigger for build

A new CC entry configurationfile "config - cruisecontrol.xml" is defined as below:

<cruisecontrol>

 

        <!-- Load environment variables -->

   <property environment="env"toupper="true"/>

        <property name="cc.dir"value="${env.CCDIR}" />

        

        <system>

                  <configuration>

                           <threads count="4" />

                  </configuration>

        </system>      

       <!-- Here the CC configuration file has to be putunderneath the root directory of project ‘Tofino-Automation-Win64’, or else the“ignoreFiles “ property of CC node “modificationset “ would never works -->

   <include.projectsfile="projects\Tofino-Automation-Win64\config.xml" />

        

        <!-- Sample project - connectfour -->

   <include.projectsfile="config.xml" />

</cruisecontrol>

 

CC configuration file forproject “Tofino-Automation-Win64”:

<cruisecontrol>     

        <property name="PROJECT_NAME"value="Tofino-Automation-Win64" />

   <propertyname="CRUISE_WORKSPACE" value="${cc.dir}\projects\${PROJECT_NAME}"/>

 

         <projectname="${PROJECT_NAME}">

                  <listeners>

                            <currentbuildstatuslistenerfile="logs/${project.name}/status.txt"/>

                  </listeners>

                   

                  <modificationset quietperiod="30"ignoreFiles="**/bin/*.log,**/bin/TestResults/*.*">

           <!-- touch any file in thisproject to trigger a build -->

           <filesystemfolder="${CRUISE_WORKSPACE}"/>     

       </modificationset>

                   

                  <schedule interval="3600">

                           <antbuildfile="${CRUISE_WORKSPACE}/build.xml"/>

                  </schedule>

                   

                  <log>

                           <mergedir="${CRUISE_WORKSPACE}\bin\TestResults"pattern="*.trx"/>

                  </log>

                  

                  <publishers>

           <htmlemailmailhost="smtp.gmail.com" mailport="465"

                                              username="**"password="**"

                                              usessl="true"

                                              returnaddress="**"               

                                              subjectprefix="Mstestreport(${env.COMPUTERNAME}): "

                                              skipusers="true"

                                              spamwhilebroken="true"

                                              charset="utf-8"              

                                              xsldir="${cc.dir}\webapps\cruisecontrol\xsl"

                                              css="${cc.dir}\webapps\cruisecontrol\css\cruisecontrol.css"

                                              xslfile="${cc.dir}\webapps\cruisecontrol\xsl\testdetails- email.xsl"

                                              logdir="${cc.dir}\logs\${project.name}">                                     

                                              <alwaysaddress="[e-mail]" />

                                    </htmlemail>

       </publishers>

                  

         </project>

</cruisecontrol>

 

With the modificationsetsetting above, any update to the files within the root directory of project"Tofino-Automation-Win64" but the files specified byignoreFiles would be monitored andtrigger a new round build, the attached CC log file proves this.

 

 

3.      Customize the XSL files to parse the test logsproduced by VS2012 MSTEST

Because VS2012 MSTESTdefines a different scheme for its generated test report, the default XSL filesof CC intended to parse the test logs in the format of old schemes can’t beapplied to test logs produced by VS2012 MSTEST.

For example, the structureof the xml log files for CC’s sample project(connectfour) has a node type of testsuite in it

 

 

But for VS2012 MSTEST, it isanother scheme for the xml test logs.

 

The change of the scheme ofVS2012 MSTEST makes the default CC XSL files incompatible with the test logsproduced by VS2012 MSTEST, we have to modify the XSL files which are under the directory:cruisecontrol-bin-2.8.3\webapps\cruisecontrol\xsl, so that the test results onCC portal and in email notifications can be legitimately displayed.

 

Here are some snapshotsrepresenting how the CC’s test result looks like on CC’s portal and emailnotifications for different projects(JAVA, .NET). 

Tofino-Automation-Win64(.NET)build result:

Tofino-Automation-Win64(.NET)test result:

 

connectfour (JAVA) buildresult:

 

connectfour (JAVA) buildresult:

 

 

Email notification: Tofino-Automation-Win64(.NET)test result

 

4.      Sent email notification

As don’t know the SMPT(mail)sever in the MS corpnet, using GMAIL server as an example.

<publishers>

           <htmlemail mailhost="smtp.gmail.com"mailport="465"

                                              username="**"password="**"

                                              usessl="true"

                                              returnaddress="**"               

                                              subjectprefix="Mstestreport(${env.COMPUTERNAME}): "

                                              skipusers="true"

                                              spamwhilebroken="true"

                                              charset="utf-8"              

                                              xsldir="${cc.dir}\webapps\cruisecontrol\xsl"

                                              css="${cc.dir}\webapps\cruisecontrol\css\cruisecontrol.css"

                                              xslfile="${cc.dir}\webapps\cruisecontrol\xsl\testdetails- email.xsl"

                                              logdir="${cc.dir}\logs\${project.name}">                                     

                                              <alwaysaddress="[e-mail]" />

                                    </htmlemail>

       </publishers>

 

Reference articles

Cruisecontrolintroduction:

http://cruisecontrol.sourceforge.net/main/configxml.html

CruisecontrolQ&A:

http://cruisecontrol.1086188.n5.nabble.com/

Overview ofApache Ant Tasks

http://ant.apache.org/manual/tasksoverview.html#compile

Xsl file to parse trx result fromVS2010:

http://www.telerik.com/automated-testing-tools/community/forums/test-studio-express/general-discussions/display-report-automation-test-on-cruise-control.aspx

XSLTsyntax:

http://www.w3schools.com/xsl/xsl_w3celementref.asp

XPathsyntax

http://www.w3schools.com/xpath/xpath_syntax.asp

How to redirect log4net’s logs?

http://logging.apache.org/log4net/release/config-examples.html