How to write integration tests for Atlassian Jira (jira-func-test-plugin)

[#1]

Hello!

There are three ways to write integration tests in Jira:

1. Wired tests - these tests can check if your exported services in your plugin work correctly.

2. Integration tests written with the help of jira-func-test-plugin and jira-testkit-plugin - 

these tests check the web-UI of your plugin, using HTTP requests and responses to assert that the web-UI is exhibiting the correct behaviour.

3. Page objects tests - these tests also check the web-Ui by using HTTP requests and responses but with the Selenium web driver under the hood.

As it is written by Atlassian the third way is more preferable than the second way. But in this article we will try to create integration tests with the second way: integration tests with the jira-func-test-plugin and jira-testkit-plugin.

The source code of the plugin in this article you can find here.

What is an integration test?

The main difference of an integration test from a unit test is that an integration test runs on a Jira instance, while a unit test runs without an available Jira instance.

That is why we need a Jira instance to be up to run our tests. It is not required to run a Jira instance from the same plugin, where your integration tests are. You can run integration tests on a remote instance of Jira. But first we will run our tests on a Jira instance from the same plugin, where the integration tests are. Later we will see, how to run the same tests on a remote instance.

Create jira plugin

You create a Jira plugin with the atlas-create-jira-plugin command.

Add dependencies to the pom.xml file

You should add the following dependencies to your plugin:

<dependency>
    <groupId>com.atlassian.jira.tests</groupId>
    <artifactId>jira-testkit-client</artifactId>
    <version>${jira.version}</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.atlassian.jira</groupId>
    <artifactId>jira-func-tests</artifactId>
    <version>${jira.version}</version>
    <scope>test</scope>
</dependency>

These dependencies will let you write integration tests in your plugin.

Add required plugins to the Jira instance

There should be the jira-func-test-plugin and jira-testkit-plugin plugins installed in the Jira instance, which is going to be under tests, otherwise integration tests will not work. That is why we should add the following plugins to the pom.xml file into the configuration section of the maven-jira-plugin:

<pluginArtifacts>
    <pluginArtifact>
        <groupId>com.atlassian.jira</groupId>
        <artifactId>jira-func-test-plugin</artifactId>
        <version>${jira.version}</version>
    </pluginArtifact>
    <pluginArtifact>
        <groupId>com.atlassian.jira.tests</groupId>
        <artifactId>jira-testkit-plugin</artifactId>
        <version>7.12.3</version>
    </pluginArtifact>
</pluginArtifacts>

Write a test

Let's create the ExampleTest.java file in the src/test/java/it/ru/matveev/alexey/jira/integrationtests/selenium/ folder:

package it.ru.matveev.alexey.jira.integrationtests.selenium;

import com.atlassian.jira.functest.framework.BaseJiraFuncTest;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class ExampleTest extends BaseJiraFuncTest {

    @Before
    public void setup() {
        backdoor.project().addProject("testproject", "TP", "admin");
        backdoor.issues().createIssue("TP", "test issue");
    }

    @Test
    public void goToIssueNavigator() {
        navigation.login("admin");
        navigation.issueNavigator().createSearch("project = TP");
        tester.assertTextPresent("TP-1");
    }

    @After
    public void cleanup() {
        backdoor.project().deleteProject("TP");
    }

By default any test must have the Test suffix in the class name so that it could be run as a test. 

This behaviour can be changed if we provide the functionalTestPattern parameter for the configuration part of the maven-jira-plugin.

In this test we want to check that the search in the Issue Navigator works. That is why in the setup method we create a project and an issue in this project. Then we create a query to select the created issue and check that the created issue were selected.Then in the cleanup method we delete our project.

Run the test

Now we can run this test with the atlas-integration-test command. It will start a Jira instance and execute our test.

Test class structure

There are a couple of classes from which your test class must be extended: BaseJiraEmailTest, BaseJiraFuncTest, BaseJiraRestTest. The most used class is the BaseJiraFuncTest. This class allows to test you any functionality in Jira.

This class has the following variables, which you can use in your test class:

backdoor - allows you to perform administrative functions in Jira like managing projects, indexes, permission schemes, users and so on.

navigation - lets you navigate to different screens in your Jira. Also it lets you login to Jira and logout. 

tester - lets you work with pages in Jira: set values for custom fields, click buttons, check if an element is present and so on.

It is important to know that the integration test engine tries to connect to Jira under the admin user with the password admin. Also when you use the navigation.login('username") to login to Jira. The password of the user must be equal to the username.

Run on a remote instance 

You should install the jira-func-test-plugin and jira-testkit-plugin to the remote instance and set parameters of your remote instance in the src/test/resources/localtest.properties file:

jira.protocol = http
jira.host = remote-instance
jira.port = 2991
jira.context = /jira
jira.edition = all

jira.xml.data.location = src/test/xml

Now you can run the atlas-integration-test command and your tests will be run on the remote instance.

Test examples

And the best part is that the developers of the jira-func-tests package inserted there many examples on how to write tests. You can find them in the webtests package.

That is all for tests. Hopefully this info was useful to you. Good luck with Jira testing!

[Immo Hüneke]

Good article. Unfortunately it doesn't work for me after I followed the tutorial published here:

https://developer.atlassian.com/server/framework/atlassian-sdk/run-wired-tests-with-the-plugin-test-console/

The developer toolbar no longer exists in JIRA 7.12.3. I was only able to get to the Atlassian plugin test console by entering its URL explicitly: http://localhost:2990/jira/plugins/servlet/it-test-console

And even then, it doesn't discover any of my wired tests. This could be connected with the different test runners and constructor method signatures used in the two tutorials.

[Immo Hüneke]

Made some progress on this issue - see also https://community.atlassian.com/t5/Questions/Jira-wired-plugin-tests-fail-java-lang-NoClassDefFoundError/qaq-p/933316

[Sebastian_Knopp]

Hi,

do you know if there is something similar for Bamboo Plugins? I searched for something like "bamboo-func-test-plugin" but didn't find anything. I want to integrate selenium into the tests of my plugins. Having those test classes would be pretty nice for that. Unfortunately i've no experience with selenium yet. I'm just looking for the best way to start.

Thanks for your help!

[uidev]

@Alexey Matveev Is there any such framework that gives you Jira "test fixtures" that conveniently create issues (in memory) for you to unit test your Java plugin backend against?

I am doing this manually but it is a lot of code to manage, as the interfaces for Jira objects are sometimes huge (50+ methods) that my mock test fixtures need to implement. 

The problem with integration testing is it is very slow and so I'm hoping to mock a Jira server reliably enough to quickly and efficiently run unit tests against, but I'm not finding any resources/helpers for this. Any thoughts?

[Slava Dobromyslov]

@uidev nope, there is not.

I see that your main concern here is "The problem with integration testing is it is very slow".

To deal with it you don't need to run your tests with `atlas-mvn jira:integration-test` every time and wait for Jira server restart all the time. There is a better approach:

1. Run your test Jira instance as:

atlas-mvn jira:debug

And make sure you have QuickReload enabled in your `jira-maven-plugin` configuration.

2. Connect your JetBrains Idea Remote JVM Debug to it to be able to set breakpoints and debug tests.

3. Run atlas-mvn package to rebuild your plugin and tests. In a few seconds QuickReload will automatically upload changes to the running Jira server.

4. Go to Jira server in your browser and open Dev Toolbox -> Plugin Test Console

5. Run tests and see the results or work with breakpoints in your IDE.

Other ways of running your tests fast are described in the tutorial: Run Wired Tests with the Plugin Test Console .

Creating test data is described in another tutorial Create test data and a test fixture 

IMPORTANT NOTE about running tests from the console: You MUST disable QuckReload and start Jira server without it to use the console command:

atlas-mvn jira:remote-test -Dserver=localhost -D"http.port"=2990 -D"context.path"=jira

 If you run it when Quick Reload is enabled you will get weird exceptions because remote-test will try uploading your plugin via UPM Rest API at the same time Quick Reload doing plugin update via its own API. 

[Slava Dobromyslov]

For those who read this tutorial and wonders what's going on with these different testing approaches and tools (Wired integration tests, Func/E2E/Acceptance tests, Page Objects, TestKit) in the new Jira 9.x I prepared the clarification post.