Integrating PsUnit and MSBuild

The last month I have been digging into, adding new and refactoring the Powershell scripts we use in RemoteX Applications. We primarily use PowerShell to configure and deploy our product.

More often or too many times – with our latest releases – our fellow collegues at Customer Operations have been telling us “The xyz powershell script doesn’t work anymore, what have you done to it???”

I could have answered: “- We don’t have automated tests on those scripts, so what do you expect? Manual testing? Pfffff…”, but for obvious reasons I didn’t

So it was time to add some automated tests for those scripts so I could get this guy at Customer Operations back to work.

All compiled code (C#) is automatically tested in our Continuous Integration builds when someone checks in, but until now we haven’t had automated tests for our PowerShell scripts. With “automated” I mean that a test runner executes a test suite during a build.

In order to test our PowerShell scripts I have started using PsUnit. Since we are using Team Foundation Server and MSBuild for our CI builds, I needed to integrate PsUnit with MSBuild somehow.

There are several ways to invoke Powershell from an MSBuild script – we have chosen the Exec task which invokes powershell.exe.

First of all I created a separate build target which invokes PsUnit:

<![CDATA[
<PropertyGroup>
<CoreTestDependsOn>$(CoreTestDependsOn);RunPsUnit</CoreTestDependsOn>
<RunPsUnitDependsOn>CoreRunPsUnit</RunPsUnitDependsOn>
</PropertyGroup>

<Target Name="RunPsUnit" DependsOnTargets="$(RunPsUnitDependsOn)"/>
<Target Name="CoreRunPsUnit" Condition=" '$(RunTest)'!='false' ">
<Exec Command="$(PowerShell) -Command "&{ .\runpsunit.ps1 -PsUnitTestFile %(PsUnitTest.FullPath) }""/>
</Target>
]]>
Using this batch target, all that needs to be done is to add PsUnitTest items to the build script. Just like the way we set up TestContainer items for use with MSTest:
<![CDATA[
<ItemGroup>
<PsUnitTest Include="$(SolutionRoot)\Scripts\Tests\*.Test.ps1"/>
<PsUnitTest Include="$(SolutionRoot)\References\PsUnit\*.Test.ps1"/>
</ItemGroup>
]]>
So far so good, but when tests are failing, the build is successful. This is because the Exec task only reports a build error when the executed command returns an exit code other than zero (0×0). So I started looking into the test runner of PsUnit (PsUnit.Run.ps1) and discovered that it didn’t send any output nor produced some other exit code than 0×0. The only output PsUnit will send you is some nice colorful information written to the PowerShell host.

So I ended up with the following:

1. this tiny tweak at the end of PsUnit.Run.ps1:

<![CDATA[
$TestResults
]]>
This makes PsUnit.Run.ps1 send the test results along the pipeline.

2. A small wrapper for the test runner which returns some exit code other than 0×0 to the Exec task in MSBuild:

<![CDATA[
$results = Invoke-Expression "$psUnitRun -PsUnitTestFile $PsUnitTestFile"
$results | % {
if( $_.Result -eq "FAIL" ) {
throw "Test run failed: $($_.Test) - $($_.Reason)"
}
}
]]>
This makes the Exec task report a build error if we find any failing test in the test result from PsUnit – and last but not least – the name of the failing test and the reason ends up nicely in the MSBuild log file.

So now when somebody checks in a change to compiled code, which affects the result of our setup scripts, we will be able to detect errors ASAP! Yay!

Written by anderssonjohan

November 11, 2009 at 19:50

Posted in scripting

Tagged with msbuild, powershell, testing

Mr Andersson