Hello, I'm trying to achive the following:
Run MSTest with multiple containers and get the test result in Build result. (ex: 11 Tests, 3 failed)
The point is the current MSTest task only accept one container, I want to run all ùy test assemblies in one row to generate only one testresult file.
Also, I dont want to explicit specify all containers because I want to avoid to reconfigure Bamboo task each time a new test assembly is added to solution.
So, my current approach is to execute MSTest from a custom MSBuild task that will look for *.Test.dll assemblies and generate a command line with multiple "/container" parameters. This is running ok but when a test fail, the MSBuild task fail (exit code = 1, the MSTest exit code actually) and Bamboo assume build is failing and dont show test results information (ex 11 Test executed, 3 failed)
So, any one can give a hint on it ?
I came up with a solution to this in the end
I created an MSBuild task to get all the names of the tests, run mstest and save the output to a file
Then use the test parser to pick up the file and parse the tests
here is the build XML which i saved as a .proj file in the troot of the repository (but coudl go anywhere althought you might have to change the include mask in the build file)
<Project ToolsVersion="4.0" DefaultTarget="RunTests" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <PropertyGroup> <results_file>./TestResults/results.trx</results_file> </PropertyGroup> <PropertyGroup> <MsTestExePath Condition="'$(MsTestExePath)'==''">C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe</MsTestExePath> </PropertyGroup> <ItemGroup> <TestAssemblies Include=".\**\bin\**\*.Tests.dll" /> </ItemGroup> <ItemGroup> <ResultsFile Include="$(results_file)" /> </ItemGroup> <Target Name="RunTests"> <Message Text="Found test assemblies: @(TestAssemblies)" /> <MakeDir Directories="./TestResults" /> <CallTarget Targets="RunMsTest" /> </Target> <Target Name="RunMsTest" > <Message Text="Running to results file: %(ResultsFile.Identity)" /> <PropertyGroup> <MsTestCommand>"$(MsTestExePath)" @(TestAssemblies ->'/testcontainer:&quot;%(RecursiveDir)%(Filename)%(Extension)&quot; ', ' ') /resultsfile:TestResults\%(ResultsFile.Identity)</MsTestCommand> </PropertyGroup> <Message Text="Running command $(MsTestCommand)" /> <Exec Condition=" '@(TestAssemblies)' != ''" Command="$(MsTestCommand)" ContinueOnError="true"> <Output TaskParameter="ExitCode" ItemName="ErrorCode"/> </Exec> <Message Text="Tests complete" /> <Message Text="The exit code is $(ErrorCode)"/> <Error Text="Error while executing MSTest" Condition="'$(ErrorCode)' != ''" /> <OnError ExecuteTargets="MessageErrorHandler"/> </Target> <Target Name="MessageErrorHandler"> <Message Text="An error has occurred while executing MSTest"/> </Target> </Project>
what i then did was create an msbuild task in bamboo and set it to build the proj file. then set the following in the options line
this tells it which file name to use.
I then created an mstest parser task to pick up the file
took a little trial and error but works great. Even spits out the test results into the log..
Im trying to overcome the same issue. Did you get anywhere?
The only thing i can think of is to runa script to identify all *.test.dll and put them into a bamboo run config variable separated by /testcontainer
Then run a custom task to start mstest with the previous string and output to a path using variables
Then run the test parser...
I currently have the problem that MSTest parser does not seem to work correctly. Of an expected number of around 1300 tests only 650 tests are shown in the bamboo status page of the build. The approach for generating the TRX file is similar to the one mentioned above.
The genenerated TRX file contains all expected tests as well as the correct results, hence the test runner does his jobs. I've looked into the implementation of MSTestXmlTestResultsParser, which should be responsible for parsing trx files. MSTestXmlTestResultsParser does not perform any filtering, hence the problem must be somewhere else.
First I suspected that duplicate testnames are filtered, but that does not seem to be the case. The trx contains indeed duplicate test names, but after filtering duplicates there are only 617 left and not 650. Does anybody have an idea?
Atlassian Summit is an excellent opportunity for in-person support, training, and networking.Learn more
👋Community members! Downtime happens. And great incident response takes a village. Teams like Support, Dev, SRE, Ops, IT, and Marketing have to come together to resolve the problem while keep...
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG
You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs