将Visual Studio的单元测试结果生成HTML报表



Visual Studio单元测试结束后我们可以查看单元测试结果，同时会生成相关的trx文件。如果还查看了单元测试的代码覆盖率，那还会生成对应的codecoverage文件。如果我们需要在集成测试环境执行相关测试并查看测试结果及代码覆盖情况，我们是这样做的：

步骤

一、使用VSTest.Console.exe执行单元测试

之前使用mstest执行相关单元测试时指定.runsettings文件作为用于生成代码覆盖率的配置文件时，mstest在命令行工作执行时总是报不能成功解析.runsettings的错误。因此我们使用VSTest.Console.exe执行单元测试。

准备相关.runsettings文件

相关.runsettings文件格式定义请参照这里

在CI相关MSBuild文件的某个target中定义以下命令行配置

<Exec

    Command="vstest.console.exe DeviceActivator.Test\bin\Release\DeviceActivator.Test.dll /InIsolation /EnableCodeCoverage /logger:trx /Settings:unittest.runsettings"

    WorkingDirectory="$(Workspace)"

    IgnoreExitCode="True"

    />

具体的vstest.console.exe命令行说明，参照这里

在命令行窗口执行msbuild命令执行相关task

msbuild ci.msbuild.xml /t:RunTestsAndExtractResults /p:Configuration=Release,Platform=AnyCPU

二、将单元测试结束后生成的trx转换成可读的html文件

我们可以依赖第三方MSBuild task： trx2htm来达成我们的目标。我们定制过的trx2htm源码，可以在这里找到。

用使用这个task，我们可以在MSBuild文件中定义以下类似配置：

<Target Name="TestResultReportGenerator" DependsOnTargets="RunTestsAndExtractResults">

    <ItemGroup>

        <TrxFiles Include="$(SrcPath)\TestResults\*.trx" />

    </ItemGroup>

     <Trx2Html TrxFiles="@(TrxFiles)" OutputDirectory="$(SrcPath)\TestResults" />

</Target>

三、将Visual Studio代码覆盖率报表文件转换成可读的html文件集

首先Visual Studio生成的代码覆盖率报表文件是二进制格式的，我们需要先将二进制文件转换成xml格式。这个功能同样已有第三方MSBuild task：ConvertVSCoverageToXml实现,它的源码在github上。以下为如何在MSBuild文件中使用它的方式：

...

<ItemGroup>

    <CoverageFiles Include="$(SrcPath)\TestResults\**\*.coverage"/>

</ItemGroup>

<ConvertVSCoverageToXml

    CoverageFiles="@(CoverageFiles)"

    SymbolsDirectory="$(SrcPath)\DeviceActivator.Test\bin\Release"

    />

...

接着需要将生成的xml结果再转换成html文件集，这样网页中就可以像在Visual Studio中那样查找具体的代码覆盖情况。

要将微软专属的代码覆盖xml结果转换成html结果，我们可以使用ReportGenerator。 ReportGenerator可以将各种来源的代码覆盖率XML报表数据（微软、OpenCover、NCover、PartCover）转换成统一的html结果。

ReportGenerator的官网地址在这里，我们可以像以下方式那样使用它：

<Target Name="CoverageReportGenerator" DependsOnTargets="RunTestsAndExtractResults">

    <ItemGroup>

        <CoverageXMLFiles Include="$(SrcPath)\TestResults\**\*.xml" />

        <SourceDirectories Include="$(SrcPath)\DeviceActivator.Service" />

        <SourceDirectories Include="$(SrcPath)\DeviceActivator.Model" />

        <SourceDirectories Include="$(SrcPath)\DeviceActivator.Common" />

    </ItemGroup>

    <ReportGenerator ReportFiles="@(CoverageXMLFiles)" TargetDirectory="report" ReportTypes="Html;Latex" SourceDirectories="@(SourceDirectories)" HistoryDirectory="history"AssemblyFilters="+*" VerbosityLevel="Verbose" />

</Target>   

注意：CoverageReportGenerator的AssemblyFilters和ClassFilters的相关使用方式请仔细参看说明及源码。

知识点

MSTest命令行说明

https://msdn.microsoft.com/en-us/library/ms182489.aspx#runconfig

mstest.exe /testcontainer:Path\To\Tests.dll /resultsfile:TestResults.trx /runconfig:WithCoverage.runsettings

VSTest.Console命令行说明

VSTest.Console可以用于执行Unit Test以及Coded UI Test

https://msdn.microsoft.com/en-us/library/jj155800.aspx

VSTest.Console.exe文件目录

C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow

Smaple:

vstest.console.exe MyTestAssembly.dll /EnableCodeCoverage /Settings:CodeCoverage.runsettings /Logger:trx

CodeCoverage.exe使用说明

https://github.com/danielpalme/ReportGenerator/wiki/Visual-Studio-Coverage-Tools

CodeCoverage.exe is another coverage tool that comes with Visual Studio 2012/2013 (Premium and Ultimate). By default CodeCoverage.exe creates a  *.coverage  file. To generate a coverage report with ReportGenerator the file has to be converted to  *.xml  format.

To get the  *.xml  file you can use the following command:

"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" collect /output:DynamicCodeCoverage.coverage "PATH_OF_YOUR_EXECUTABLE_OR_DLL"

"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output:DynamicCodeCoverage.coveragexml DynamicCodeCoverage.coverage

Customizing Code Coverage Analysis

https://msdn.microsoft.com/en-us/library/jj159530.aspx

！！！VS2013 .runsettings详细说明

https://msdn.microsoft.com/library/jj635153.aspx

VS2013 .runsettings template

https://visualstudiogallery.msdn.microsoft.com/704ebd18-7d60-4341-9224-532f73229c74

!!Sample .runsettings file

<?xml version="1.0" encoding="utf-8"?>

<!-- File name extension must be .runsettings -->

<RunSettings>

  <DataCollectionRunSettings>

    <DataCollectors>

      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">

        <Configuration>

          <CodeCoverage>

<!--

Additional paths to search for .pdb (symbol) files. Symbols must be found for modules to be instrumented.

If .pdb files are in the same folder as the .dll or .exe files, they are automatically found. Otherwise, specify them here.

Note that searching for symbols increases code coverage runtime. So keep this small and local.

-->

<!--           

            <SymbolSearchPaths>             

                   <Path>C:\Users\User\Documents\Visual Studio 2012\Projects\ProjectX\bin\Debug</Path>

                   <Path>\\mybuildshare\builds\ProjectX</Path>

            </SymbolSearchPaths>

-->

<!--

About include/exclude lists:

Empty "Include" clauses imply all; empty "Exclude" clauses imply none.

Each element in the list is a regular expression (ECMAScript syntax). See http://msdn.microsoft.com/library/2k3te2cs.aspx.

An item must first match at least one entry in the include list to be included.

Included items must then not match any entries in the exclude list to remain included.

-->

            <!-- Match assembly file paths: -->

            <ModulePaths>

              <Include>

                <ModulePath>.*\.dll$</ModulePath>

                <ModulePath>.*\.exe$</ModulePath>

              </Include>

              <Exclude>

                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>

              </Exclude>

            </ModulePaths>

            <!-- Match fully qualified names of functions: -->

            <!-- (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.)  -->

            <Functions>

              <Exclude>

                <Function>^Fabrikam\.UnitTest\..*</Function>         

                <Function>^std::.*</Function>

                <Function>^ATL::.*</Function>

                <Function>.*::__GetTestMethodInfo.*</Function>

                <Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function>

                <Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>

              </Exclude>

            </Functions>

            <!-- Match attributes on any code element: -->

            <Attributes>

              <Exclude>

                <!—Don't forget "Attribute" at the end of the name -->

                <Attribute>^System\.Diagnostics\.DebuggerHiddenAttribute$</Attribute>

                <Attribute>^System\.Diagnostics\.DebuggerNonUserCodeAttribute$</Attribute>

                <Attribute>^System\.Runtime\.CompilerServices.CompilerGeneratedAttribute$</Attribute>

                <Attribute>^System\.CodeDom\.Compiler.GeneratedCodeAttribute$</Attribute>

                <Attribute>^System\.Diagnostics\.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>

              </Exclude>

            </Attributes>

            <!-- Match the path of the source files in which each method is defined: -->

            <Sources>

              <Exclude>

                <Source>.*\\atlmfc\\.*</Source>

                <Source>.*\\vctools\\.*</Source>

                <Source>.*\\public\\sdk\\.*</Source>

                <Source>.*\\microsoft sdks\\.*</Source>

                <Source>.*\\vc\\include\\.*</Source>

              </Exclude>

            </Sources>

            <!-- Match the company name property in the assembly: -->

            <CompanyNames>

              <Exclude>

                <CompanyName>.*microsoft.*</CompanyName>

              </Exclude>

            </CompanyNames>

            <!-- Match the public key token of a signed assembly: -->

            <PublicKeyTokens>

              <!-- Exclude Visual Studio extensions: -->

              <Exclude>

                <PublicKeyToken>^B77A5C561934E089$</PublicKeyToken>

                <PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>

                <PublicKeyToken>^31BF3856AD364E35$</PublicKeyToken>

                <PublicKeyToken>^89845DCD8080CC91$</PublicKeyToken>

                <PublicKeyToken>^71E9BCE111E9429C$</PublicKeyToken>

                <PublicKeyToken>^8F50407C4E9E73B6$</PublicKeyToken>

                <PublicKeyToken>^E361AF139669C375$</PublicKeyToken>

              </Exclude>

            </PublicKeyTokens>

            <!-- We recommend you do not change the following values: -->

            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>

            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>

            <CollectFromChildProcesses>True</CollectFromChildProcesses>

            <CollectAspDotNet>False</CollectAspDotNet>

          </CodeCoverage>

        </Configuration>

      </DataCollector>

    </DataCollectors>

  </DataCollectionRunSettings>

</RunSettings>

如何在.runsettings文件中引用.testsettings文件

http://stackoverflow.com/questions/12928436/are-there-any-xsd-schemas-available-for-the-runsettings-file-in-visual-studio-2

<RunSettings>

    <MSTest>

        <SettingsFile>my.testsettings</SettingsFile>

        <ForcedLegacyMode>true</ForcedLegacyMode>

    </MSTest>

</RunSettings>

Trx2Htm

将MSTest生成的单元测试结果trx文件转换成可读的html格式的MSBuild Task

https://github.com/ridomin/RidoTasks

CI.MSBuild.Tasks

将VSTest.Console, MSTest生成的代码覆盖率数据文件转换成XML格式的MSBuild定制化Task。通过该Task生成的XML再传入ReportGenerator生成可读报表。

https://github.com/gredman/CI.MSBuild.Tasks

ReportGenerator

将各种Code Coverage数据转换成可读性报表的工具

https://github.com/danielpalme/ReportGenerator

使用该Task的build文件Sample连接

https://github.com/gredman/CI.MSBuild.Tasks/blob/master/Example.msbuild

CI.MSBuild.Tasks需要依赖VistualStudio的私有dll，因此发布的时候需要将以下两个dll放在CI.MSBuild.Tasks.dll相同目录下。

• Microsoft.VisualStudio.Coverage.Analysis.dll
• Microsoft.VisualStudio.Coverage.Symbols.dl

此外Microsoft.VisualStudio.Coverage.Analysis.dll只有x86平台版本，因此在编译CI.MSBuild.Tasks项目的时候需要指定平台为x86

Microsoft.VisualStudio.Coverage.Analysis类说明

https://msdn.microsoft.com/en-us/library/microsoft.visualstudio.coverage.analysis.coverageds.aspx

Test Settings for Visual Studio Tests

MSTest专用的testsettings选项相关的配置文件，以下链接对testsettings进行了详细的说明，同时也提到了其与.runsettings file的区别。

当然MSTest也可以指定runconfig选项对测试进行定制化，比如说定制化单元测试中的代码覆盖率相关配置

https://msdn.microsoft.com/en-us/library/ee256991.aspx

VS2010->VS2012 testsettings相关配置的变化

https://msdn.microsoft.com/en-us/library/jj155802.aspx

Configure unit tests by using a .runsettings file

VSTest.Console可以通过指定.runsettings文件可以定制化单元测试，以下链接.runningsettings进行了详细的说明

https://msdn.microsoft.com/en-us/library/jj635153.aspx

Jenkins相关插件

Jenkins MSTest Plugin

https://wiki.jenkins-ci.org/display/JENKINS/MSTest+Plugin

Jenkins MSTestRunner Plugin

https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin

Jenkins VSTestRunner Plugin

https://wiki.jenkins-ci.org/display/JENKINS/VsTestRunner+Plugin

其他连接：

MSTest生成的Code Coverage文件转换成XML后自己处理成可读报表的博文

http://blogs.msdn.com/b/ms_joc/archive/2005/11/22/495996.aspx

MSBuild使用ConvertVSCoverageToXml Task的Sample

http://megakemp.com/2008/10/09/visual-studio-code-coverage-reports-with-msbuild/

其他人使用MSTest生成Code Coverage数据遇到的问题

https://social.msdn.microsoft.com/Forums/en-US/af1eb48b-b03d-41c3-8456-3124950cb003/code-coverage-results-when-using-mstest-command-line?forum=vststest

MSBuild相关博文

http://www.infoq.com/articles/MSBuild-1

http://www.infoq.com/articles/MSBuild-2