C#.NET NUnit/Selenium Integration Testing on full AWS deployment

[Jeremy Roelfs]

We have a multi-service product built in C#.NET. Bamboo builds the individual services, then after they are successfully built, QA manually runs CodeDeploy in AWS.

I was recently tasked with building a smoke/integration testing framework and updating our current workflow to a CI system that implements the testing on each commit. We are already 50% of the way there with Bamboo.

I built a testing framework using C#/Selenium/NUnit(Page Object Model) I have a separate testing solution with it's own Bitbucket repo.

What tools/technologies do I need to use in Bamboo to trigger a Deployment, get a response from AWS that deployment is successful, then trigger a Task to run my C#Selenium solution against that instance. Finally, returning successful or fail(new issue) to the developers via JIRA?

I have watch so many videos and tutorials today... but there is not a single concise way to achieve the above. I'm sort of lost with all of the plugins.

[Steffen Opel _Utoolity_ · Accepted solution]

There are probably several ways to compose a CI pipeline for this scenario - here's what I would do (from the top of my head):

1. Use a Repository triggers the build when changes are committed trigger with a Bamboo build project for your service (in place already)
2. Use an After successful build plan trigger with a Bamboo deployment project - here you have two options for the AWS CodeDeploy task:
   
   1. You may get by with the built-in Bamboo AWS CodeDeploy task
   2. You could also look into Utoolity's (commercial) AWS CodeDeploy Deployment task, which is more versatile, but possibly more involved depending on your scenario - see How do the Tasks for AWS CodeDeploy tasks differ from the built-in Bamboo CodeDeploy task? for a comparison
3. Use Atlassian Lab's After Deployment trigger with another Bamboo build project that executes the integration tests.
4. As for "returning successful or fail", the common approach is to just rely on the build status automatically reported in the JIRA development panel and require the developers to manually create a new issue (if that's your policy) - if you also want to automate that, you could create a JIRA issue via the REST API, and again you have two options:
   1. Either just use a Bamboo Script task and a CLI tool like curl or HTTPie
   2. Or use the Invoke Function action in Utoolity's AWS Lambda Function task, or the Invoke Lambda Function action in Utoolity's Automate with AWS task

Unfortunately we haven't yet documented how to create a JIRA issue from a Lambda function via REST - let me know if you want to get down that path and I'll try to get someone to look into those overdue docs ;)

[Jeremy Roelfs]

Thank you! That is exactly what I needed. 

[Jeremy Roelfs]

Could you do me a favor and break down #3 a little more? What tools would you recommend to invoke the integration tests from my Testing Solution against the "Deployed Instance" from #2?

I see I have Selenium Grid options: GridLastic and Charlotte. But they are geared towards JUnit. I don't even know if Charlotte can utilize the C#/Selenium/Nunit testing framework we have created.

GridLastic looks like a good solution. But, it seems very difficult to set it up strictly as a Bamboo Cloud option. *And please keep in mind this is all cloud based. GridLastic seems to work ON the cloud, but ran from a hub/node on a Bamboo Server locally. That isn't really what we are looking for.

[Jeremy Roelfs]

And I need to correct that. We are not running Bamboo Cloud. We are runng Bamboo on an EC2 instance.

[Steffen Opel _Utoolity_]

@Jeremy Roelfs - I'm not well versed in UI testing, sorry.  Accordingly, I recommend to either ask a new question here, or reach out to one of the two vendors that seem to support UI test integration with a Bamboo add-on directly:

• Charlotte - ask a tagged question here or submit a support request to them
• X-Ray - ask a tagged question here or submit a support request to them

I have used neither yet, but X-Ray notably lists NUnit in their Marketplace listing (also requires their JIRA add-on it seems, might be a good thing for your scenario of course).

[Jeremy Roelfs]

Awesome, will do. Thank  you very much for your help!