Acceptance Test Driven Development with Jira

Hi there,

We would like to do Acceptance Test Driven Development (ATDD), that is to say, write user stories in a precise format (given / when / then, or equivalent), parseable by a tool to run automated tests (against source code through "fixtures", or against GUI through GUI driver tools like Selenium).

We already tried Fitnesse, Cucumber, etc. but as we're using Jira, we would prefer to have a unique repository / tool for managing our stories.

Is there any Jira plugin that allow to do ATDD, or to integrate with third party ATDD tool?

Thanks,

Maxime

5 answers

This widget could not be displayed.

Hi Maxime,

I suggest Xray for JIRA: https://marketplace.atlassian.com/plugins/com.xpandit.plugins.xray

It's a test management tool supporting both manual and automated tests (using Cucumber) that you can link to requirements (in this case user Stories). You can create automated cucumber tests and generate .feature files to execute externally to JIRA. We are working on a REST API for exporting Cucumber .feature files and importing Test Execution results into JIRA. This will allow you to use CI platforms like Jenkins to execute your tests and have the report and requirement coverage in JIRA.

Best Regards,

Bruno Conde.

@Bruno Conde did your team figure out an API to import test execution results into JIRA?

@Bruno conde Hi Bruno, We already have tests BDD test running in Cucumber framework, any thing on the API for importing test results back in JIRA? Went through a lot of documentation online but no luck 

This widget could not be displayed.

I am not sure whether you need a plugin for this as this is more of a practice that needs to be followed in the project. This is what we used to follow. We used to have an issuetype called testcase and with some custom fields like 'pre-condition', 'post-condition' to fully describe the testcase. So for every userstory there will be a set of testcases which are linked to the userstory using the Jira linking function. The link that was used was 'is tested by' even though the type of link has no relevance apart from correcting displaying the relation while viewing the story.

The testcase issue type also was associated with a workflow which moves from states New => Under Testing => Success/Failure

Additionally, our CI system automatically transistions the testcase to success when the testcase passes in the CI and this becomes the closure DONE criteria for the userstory.

Hope this helps.

how do you link the testcase in jira and the testcase run in ci server ? using jira api?

Yes, access jira using SOAP interface.

This widget could not be displayed.

Our documentation is a bit outdated, but there's Using JIRA for Test Case Management. Check that out and its sub-pages. In general, search Google (and our resources, like confluence.atlassian.com) for 'test case management'. There's also a good thread over on Stack Overflow.

ATDD is about test automation. You define user stories as test scenario, and you make them runnable on your system. It's like unit tests understandable and writable by business analysts.

Greenpepper, which is based on Confluence is a tool of that kind. As confluence and jira are integrated, this could be a solution, but a complicated and expensive one (2 new tools and licenses), considering a jira plugin could do the work.

We have built such a JIRA Plugin (Behave for JIRA) and its now available in the marketplace https://marketplace.atlassian.com/plugins/com.hindsighttesting.behave.jira

It allows "Scenarios" to be added to user stories or any JIRA issue as Acceptance Tests. The scenarios are written in Cucumber's Gherkin format and a maven plugin transforms the relevant scenarios into features files during the build. Cucumber can then run the generate feature files to execute the tests.

This widget could not be displayed.

I have tried "Behave for JIRA" but there is some limitation in the features. For example, users can not search the created features. And it seems to be hard to organize the created features as well. And what are the 'tags' for?

@Alan Parkinson, do you have some guide or solution for those?

This widget could not be displayed.

Maxime - We're going to try this as well and we've looked at Behave but aren't sure we like it. The biggest limitation was being able to group and categorise stories/acceptance critiera.

Instead we're going to try using Bob Swifts CLI tool to export BDD/ATDD scripts from the description fields of JIRA User Stories to text files. We will then try and pump them through Cucumber bolted onto a Selenium driver and into our application.

Wish us luck!

@Nick

how did it go with the bob swift cli tool ? I might think to implement the same thing, for example launching jenkins, which retrieves the user stories from jira and then execute cucumber to launch selenium driver.

Nikhil I'm New Here Apr 12, 2018

Hi @Maxime Lemaniss1@Bruno Conde @kamalpreet kaur@Evan Walker Wil@Emidio Stani

Actually I am totally new for XRAY- Cucumber integration. Basically I have one cucumber bdd .feature file written in Intellij IDE. At the same time I have corresponding step definition too which is working fine locally. Now as per my requirement, I need to trigger this feature from Xray-Jira but don't know how to do that, I mean which api to call, how to call that api, where to call that api etc etc.

I tried to find some good document on google but could not find anything fruitful which can help me step by step setup.

Can anyone please help me in this ?

Thanks.

Regards,

Nikhil Agrawal

Hi Nikhil,

 

You can define cucumber tests directly in Xray.  For this you can just create a new Test issue and choose the Cucumber type. 

A cucumber test in xray only contains the scenario definition. You should create a Test issue per scenario.

The implementation part, or the code that will parse the scenario and execute it's steps is not meant to be stored in Jira. You can use your favorite code repository. 

Xray also provides an API that allow you to import your cucumber feature files. This will automatically create Tests, and other entities in Jira:

https://confluence.xpand-addons.com/display/XRAY/Importing+Cucumber+Tests+-+REST

This is just for defining/importing ccuumber tests in Xray. Regarding the import of execution results Xray also provides a REST API that allows you to simply take your cucumber results json file and import it directly to Jira. This will update/create new Test Executions in Xray with the results. Remember that it is not the purpose of Xray/Jira to execute cucumber tests. Xray just stores the execution results and outputs nice reports.

It is possible to configure builds in a CI tool that will:

1. Export the cucumber definitions from Xray issues:

2. Checkout your implementation from the repository and merge it with the feature files exported in step 1.

3. Execute the tests with cucumber

4. Import the execution results into Xray.

 

Xray provides plugins for Jenkins and Bamboo to easy the configuration of these tasks.

It is also possible to import the cucumber execution results right from the UI (from a Test Execution or a Test Run).

Please check the following links for importing execution results into Xray:

https://confluence.xpand-addons.com/display/XRAY/Import+Execution+Results+-+REST

 https://confluence.xpand-addons.com/display/XRAY/Continuous+Integration+Tools

https://confluence.xpand-addons.com/display/XRAY/Integration+with+Bamboo

https://confluence.xpand-addons.com/display/XRAY/Integration+with+Jenkins

https://confluence.xpand-addons.com/display/XRAY/Exporting+Cucumber+Tests+-+REST

 

Hope this helps,

Regards,

Bruno Conde

Nikhil I'm New Here Apr 13, 2018

Thanks a lot @Bruno Conde. This information really helped me a lot. Just want to confirm one thing, do we have any plugin of Jira-XRAY with Bitbucket code repository or it would be handled other way round, I mean plugins needs to be installed in Jenkins (CI tool).

What I understood is, we do not have any bitbucket or jenkins plugins that can be configured in XRay rather we can install these XRay and Bitbucket plugins in Jenkins. Correct me if I am wrong ?

Suggest an answer

Log in or Sign up to answer
Atlassian Summit 2018

Meet the community IRL

Atlassian Summit is an excellent opportunity for in-person support, training, and networking.

Learn more
Community showcase
Posted Aug 06, 2018 in Jira Service Desk

A is for Activate: Share your top Jira Service Desk onboarding tips for new users!

Hi, everyone! Molly here from the Jira Service Desk Product Marketing Team :).  In the spirit of this month's  august-challenge, we're sourcing stories of Jira Service Desk activation fro...

535 views 23 15
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you