Extracting complete issue history via API

history.jpeg

There are a lot of things that can be said when you have a history of such information. Given that Jira is a great piece of software, it provides a means whereby there's data integrity when something is changed within a project. The issue history feature by far is one of the most vital features that Jira has to offer. The complexity involved in recording data and the changes that took place makes it an integral part that shouldn't be easily manipulated.

Hence, I believe one of the fundamental things to keep in mind while moving from server to cloud is the ability to retain issue history. However this is a very complex process to do and in most cases, you end up not transferring this data due to the complexity involved.

 

Alternatively, it doesn't mean you can't extract such information from either a server to cloud instance or vice versa or even a cloud to cloud instance of Jira. The way to go about it, is to use Jira's REST API and the end result is an output in a CSV file which can be viewed from any spreadsheet application.

issue.pngMost of the API related instructions here are based on python examples using a specific package called jiraone (can be installed using pip install jiraone). Which can simply output a report file in CSV format. Example for Jira server instance given below

from jiraone import LOGIN, PROJECT

user = "username"
password = "password"
link = "https://yourinstance.atlassian.net"
LOGIN.api = False  # comment out line, if you want to extract history from a cloud instance
LOGIN(user=user, password=password, url=link)

if __name__ == '__main__':
    # the output of the file would be absolute to the directory where this python file is being executed from
    jql = "project in (PYT) ORDER BY Rank DESC"  # A valid JQL query
    PROJECT.change_log(jql=jql)

With the above, you will get a complete extract of all issue history in CSV format within the specified project listed in the JQL variable.

24 comments

M Amine
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
June 9, 2021

Very interesting content

Tristan Munthree August 23, 2021

Hi Prince,

I have used this code and it runs well and gets the history of issues but at a certain point I get the error: "expecting value line 1 column 1 (char 0)" which is when the code stops running. Can I please get some help on how to go about fixing this error so that it does not interrupt the code from getting all of the issue history? Your assistance will be appreciated.

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 24, 2021

Hi @Tristan Munthree 

Please can you create an issue on the package in Github with the stack trace error, so I can trace where the error is coming from?

Tristan Munthree
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
August 24, 2021

Hi @Prince Nyeche 

I have just created an issue with attachments showing the error, thank you for your prompt response.

Like Prince Nyeche likes this
Miguel Angel Velazquez Solis March 10, 2022

Where should I run the script?

Can I do it directly on ScriptRunner?

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 11, 2022

No, this is a python script. You will need to install python on your device and run it from your terminal or if you have an IDE, you could run it from there too. 

Miguel Angel Velazquez Solis March 14, 2022

@Prince Nyeche 

Thanks for your comments, I have tested the script and it works but does it have export limits?
I have to export the information of more than 1000 records, is this possible?
What do I have to modify in the code or where can I consult more information to achieve this?

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 15, 2022

Yes, there's no export limit. You can go to the doc https://jiraone.readthedocs.io

Like # people like this
Miguel Angel Velazquez Solis June 1, 2022

Hi @Prince Nyeche

When I run the code I get the error below

UnicodeEncodeError: 'charmap' codec can't encode character '\x95' in position 1114: character maps to <undefined>

I checked the issue and this happens because the custom field (objective) has bullets as example below

image.png

Do you know how can I solve this?

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 3, 2022

Hey @Miguel Angel Velazquez Solis 

It's suppose to try to convert that using utf-8 character encoding as that's the default on unix-like systems. However, I think if you're using windows this has to be explicitly mentioned when writing to a file. I'll push an update about this problem. Please can you create an issue here with the error that you received and I'll follow up with a resolution in the next update.

Miguel Angel Velazquez Solis June 3, 2022

Hi @Prince Nyeche

Thank you so much 

I have already created the ticket in github.

Regards

Andrew Rainboldt June 16, 2022

Hi,

How are credentials passed to the Project object?

The code above uses the `token_session` to initialize the `auth_request` attribute of the LOGIN class. But inside the PROJECT class the LOGIN class isn't ever initialized and no parameters are passed from the instantiated LOGIN class to the environment or PROJECT class in any way. 

How is this supposed to work?

I'm getting this error:

JiraOneErrors: <JiraOneError: Authentication failed. Please check your credentials.>
 

Thanks,
Andrew

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 16, 2022

Hey Andrew,

Great question. To get the authentication to work, you need to instantiate the LOGIN  alias or class once hence the call

# previous login variables
LOGIN
(user=user, password=password, url=link)

Once that is done, you can access all the objects accessible to that class from anywhere within the code. The LOGIN alias is a global object for authentication that is used with the classmethod for Jira's API url called endpoint. So since the LOGIN performs authentication and HTTP requests, we can do something like after it's instantiated.

LOGIN.get()

And since this is global when combined with the classmethod for each endpoint alias or class. This passes the entire session plus the endpoint class to the HTTP methods extracting the API contents. The PROJECT alias mostly being static doesn't really need instantiation. However, note that it is already instantiated automatically hence it can be called directly. With the above when combined together, each session is passed sequentially and the session lives in memory until the script is terminated or you terminate the connection or whatever objects you've called. Now, coming to why you get the error, there could be a number of reasons but the only thing that can cause that error is either a wrong email address or token (if on Jira cloud) or a wrong username or password (if on Jira server or DC).

If you're on Jira server or DC and using PAT, I think this might not work as I didn't test such authentication mechanism. So it's best to stick with username and password. OAuth 2.0 works as well but it was only tested on Jira cloud, so its mechanism might not work for Jira server or DC. Let me know if that answered your question.

Like Andrew Rainboldt likes this
Andrew Rainboldt June 16, 2022

Hi, 

Thanks for the quick response. Now I see the way LOGIN is created, via this

LOGIN = InitProcess()  

 and then re-instantiated when called. 

Thanks for the explanation.

Regarding the error, I'm just trying to access my company jira server with username and password as indicated in the example. 

I'm running out of Jupyter, perhaps something odd is happening there, but l've switched to cmd and the same error is coming up

Thanks

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 16, 2022

Before calling the LOGIN alias since you're connected to a Jira server instance do the below

# previous login variables
LOGIN.api = False
LOGIN
(user=user, password=password, url=link)

As Jira server still uses v2 of Atlassian API. The attribute `LOGIN.api` changes the endpoints from api `v3` which works on cloud to `latest` which works on Jira server or DC. Probably that's why the auth keeps failing because it cannot connect at all.

Andrew Rainboldt June 24, 2022

Hey Prince,

thanks, authentication still doesn't work. It could be to do with the way my company manages accounts. 

 

Thanks anyway

Agnès Masson-Sibut
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
October 18, 2022

Hello there,

I managed to connect to my company Jira Cloud instance using the API token (https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/) but I still have the same error than Miguel:

UnicodeEncodeError: 'ascii' codec can't encode character '\u0117' in position 94: ordinal not in range(128)

Did you find any workaround for that @Miguel Angel Velazquez Solis ?

Thanks

Regards 

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
October 30, 2022

Hey, @Agnès Masson-Sibut  You should download the new version v0.7.3 which should fix the exception error of encoded characters.

Michael Walker
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
March 30, 2023

@Agnès Masson-Sibut can you put the code you used to connect with the api token?

Handy Rey
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
June 25, 2023

Hello @Prince Nyeche

Is there a way to query more specific filter?

someting like "project = <your_project> AND issuetype = Story AND status = Done order by created DESC". I've tried JQL syntax above, it returned an error.

 

Thank you.

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 26, 2023

Hey @Handy Rey 

Any valid JQL should simply just work, except it returned no issue. However, to target a specific field, you need to use the field_name argument. For example

# previous expression/statement
field = "Summary"
PROJECT
.change_log(jql=jql, field_name=field)

That way, you export only the values from that field if it exists as part of the history. The name of the field has to be an exact string of the field name, the way it is represented on Jira.

Alexander Scholz November 10, 2023

Hey @AlL

we were searching for a solution to exctract the full history of all issues via REST API to a Qlik Cloud project. 

 

Since there is no possibility, here is a ticket to implement the function. 

Ability to extract issue history of all issues via API 

Please vote. :-)

Thanks

Prince Nyeche
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
January 1, 2024

Hey folks, the latest version of jiraone comes with an asynchronous function to extract the issue history very fast. Example

# import statement
PROJECT.async_change_log(
               jql, folder="TEST", file="sample.csv", workers=20, flush=10
            )

An explanation of the arguments can be found here

Camilo A. Soto
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
March 26, 2024

Using atlassian-python-api==3.41.11 you can query the change logs 

from atlassian import ServiceDesk, Jira
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)

jira_url = ""
jira_token=""
jira = Jira(url=jira_url, token=jira_token, verify_ssl=False)
changes = jira.get_issue_changelog("1982769")
for history in changes["histories"]:
for item in history["items"]:
print(item)
print("========================")

 

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events