Selenium and Bamboo

[Bernard O'Flynn]

I've seen a couple of people posting about this on StackExchange and I have similar questions and thought I'd post it here.

We want to run selenium html tests in our bamboo build and integrate the results with Bamboo.

I know that Atlassian uses Selenium to test Jira, etc. so I was hoping that there was a plug-in or some automated way to integrate Bamboo and Selenium

In particular taking the html result file from Selenium and displaying it in Bamboo.

Does anyone know how we can do this? Apparently Bamboo requires its results in jUnit format?

[Gary Weaver]

You could call Selenium RC with Junit: http://www.seleniumwiki.com/tag/junit/

That should produce results in Junit format.

[Bernard O'Flynn]

we're already adding the html result as an artifact, but I would have hoped that we could integrate it into Bamboo like other build servers allow you to do.

A lot of our tests are in the html format as they were done by business testers. My understanding of Selenium is that to call it using jUnit you would need to convert them all to java so that's not an option. Please correct me if I'm wrong?

[Gary Weaver]

You might want to remove this answer and instead add comments to each of the answers individually to keep comments about answers together.

[Bernard O'Flynn]

we're already adding the html result as an artifact, but I would have hoped that we could integrate it into Bamboo like other build servers allow you to do.

A lot of our tests are in the html format as they were done by business testers. My understanding of Selenium is that to call it using jUnit you would need to convert them all to java so that's not an option. Please correct me if I'm wrong?

[Bernard O'Flynn]

Good point - I've deleted my post and added things as comments instead...

we're already adding the html result as an artifact, but I would have hoped that we could integrate it into Bamboo like other build servers allow you to do.

A lot of our tests are in the html format as they were done by business testers. My understanding of Selenium is that to call it using jUnit you would need to convert them all to java so that's not an option. Please correct me if I'm wrong?

[Joe Clark]

This is the approach we take at Atlassian - most of our Selenium tests are invoked via JUnit and/or TestNG.

We don't have (m)any business users who write acceptance tests/functional tests.

[Gary Weaver]

You could convert selenium tests to junit, even though you said you didn't want to:

• In IDE use File -> Export Test Case As
• Batch conversion: http://stackoverflow.com/questions/5951136/automated-way-of-converting-selenium-html-tests-to-junit and plugin at https://addons.mozilla.org/en-US/firefox/addon/test-suite-batch-converter-sel/?src=api

Atlassian's Nick Menere and Scott Farquhar in Selenium - Is It Worth the Pain said that Atlassian uses Bamboo + Cargo + Selenium for its tests. More here: Confluence has a Selenium Build and there are more posts on Selenium in Atlassian Developer Blogs

There was also a talk about Selenium and Bamboo in Atlassian Summit 2010 by Dyn Inc.

You could also use the ant task or maven plugin, but I don't know how results would be integrated into bamboo results:

• http://mojo.codehaus.org/selenium-maven-plugin/
• http://release.seleniumhq.org/selenium-remote-control/0.9.0/doc/server/org/openqa/selenium/server/ant/SeleneseAntTask.html

You could probably also do something like this calling as bamboo script, but I don't know how results would be integrated into bamboo results:

• http://stackoverflow.com/questions/1231975/i-want-to-run-selenium-test-case-file-from-command-line

[Bernard O'Flynn]

We're a .Net team so while we understand the syntax of Java, converting all the html tests that were created by our QA team to Java isn't really an option.

The ideal solution is to either conver the html test result file which is just html into the jUnit format OR enable Bamboo to read the html test result file.

It seems to me that Bamboo has a bit of a gap here compared to other CI products (like CC.Net) which allow easy plugging in of results files to builds.

[Gary Weaver]

For .Net, NUnit works with Bamboo and Selenium RC can be used with NUnit:

• http://www.theautomatedtester.co.uk/tutorials/selenium/selenium_two_csharp_nunit.htm
• http://lostechies.com/christaylor/2009/02/10/how-to-configure-selenium-rc-for-use-in-c-nunit-tests/

[Gary Weaver]

Also create a ticket and tell Atlassian what you'd like them to do to Bamboo to make it easier and post the link here so others can vote on it.

[Gary Weaver]

And if you wanted to write an NUnit test that started Selenium tests via command-line, you could use Process in .Net. In JUnit/java you could use Runtime.getRuntime().exec("my command"). Also, here is a note about running Selenium from command-line in Windows.

[Bernard O'Flynn]

Thanks for all these. We've looked at all of that and it still means that our QA team (mostly business type users) would need to learn c# - its just not going to happen.

So the end result is that Bamboo (like cc.net and TeamCity and Hudson all do) should allow you to configure it so that it can display html result files.

I've opened a request for this https://jira.atlassian.com/browse/BAM-9179

Please vote for it.

[Gary Weaver]

Sure thing, and it would help my karma if you voted my answer/comments up. Thanks!

[Stefan Kohler]

Would the Sauce OnDemand plugin do what you want? https://plugins.atlassian.com/plugin/details/30134

As last resort, you can always add the HTML file as an artifact to your build, that way it isn't lost after a build and easy accessable for viewing.

[Bernard O'Flynn]

The Sauce OnDemand plugin isn't free and runs tests in the cloud which isn't an option for us.

[EddieW]

Will NUnit - http://www.nunit.org/ not meet your needs?

Along with Bamboo's provided .NET plugin you should be able to export Selenium test cases to an NUnit format and run those right in bamboo along side your unit tests. It is exactly as a Java project would go the route, but using .NET.

http://codebetter.com/jeremymiller/2006/05/15/automated-web-testing-with-selenium-driven-by-net/

[Gary Weaver]

Eddie, appreciate your answer. Just to note, if you down mod other people's answers like mine and Stefan's for this question, please provide a reason why in the comments so that we and others know what was wrong with the answer. Note that everything including changes to karma are public, also.