Maybe I'm just missing some pieces to the puzzle here, but I feel like Bamboo is a moderate let-down in terms of what we want it to do. This may be because we're not using it as it is intended, or maybe it is due to something I am missing, but let me explain.
First off, we are not using Bamboo to test Java code, or to do unit testing, etc. All we are using it for is to checkout the sourcecode, call msbuild to build it, then kick off a perl test harness script, which we simply use to run our compiled application with different combinations of command line arguments (we're using perl instead of batch because batch is god-awful). The perl script then collects the app's exit code, and any error and warning messages from the application's log file. The end result is that for each loop in the perl script, we have a specific combination of arguments that we kick off our application with, and a summary log containing not just errors, but warnings, notes, and asserts.
Originally, I was just outputting all this data to a different summary log file on the system that is running the remote agent. I wanted to show this stuff in the Bamboo summary log window, but couldn't figure out how it was done.
Recently, we found that we could use JUnit Parser (among others) as a subsequent task in bamboo, as long as our perl script sent its error summary to a JUnit compliant file. So that is what we did, and on a basic level, Bamboo is doing what we need. What is really puzzling is why Bamboo seems so limited on what info it can display in its build summary window. All we can get the JUnit xml files to display are the number of errors, and a brief error summary. There is no concept or warnings, or any other "non-error" notes that can be displayed, unless I am missing something obvious? I mean, there's not even a summary of how many tests were performed, or how many tests passed... just the number of errors. (See attached screen shot for an example of this). In this screenshot, you see a grand total of one error, and a very brief summary. Now, I realize that bamboo wasn't the one running the tests, it was the perl script, but I feel like that has got to be a way to pass that information back into bamboo. Am I wrong?
Does anyone have any insight on this? Yes, we can certainly just use it as is, and when ever we see an error in the summary log, we can navigate to the perl script's output log and manually pull it up, but with a tool like Bamboo, the whole point is to be able to display errors, warnings, notes, etc, about your test runs (not just errors!), but I'm not finding a way to do this. I've looked through every plugin and am not seeing one that takes care of this.
Do I have to write my own plugin to get this functionality?
Thanks!
Scott
