How to export all issues from Jira via API

[Prince Nyeche]

It's for the next release version.

[Madhav Dube]

Also, @Prince Nyeche I am seeing many duplicated fields. Not sure if it is because of REST API endpoint. One such duplicated field in the export is `comment` like I am seeing 10+ comment fields in the export. Is it expected?

[Prince Nyeche]

In the export, those fields are expected to be in multiple columns as each column will denote a different value for such field. E.g. fields such as attachments, comments, labels, components etc will likely be in multiple columns. 

[Prince Nyeche]

Hey folks,

I have updated this function to include other arguments with the latest v0.7.6. So it is now possible to export issues into JSON format and the CSV export now allows you to select certain fields to be included before export using the include_fields argument. In the final export, only those fields mentioned will show up in the result CSV file e.g.

# previous expression

fields = ["Summary", "Assignee", "Reporter", "Labels"]
issue_export(jql=jql, include_fields=fields)

If you want to include the current field shown in your advanced issue search via the UI, you can use the field_type argument to achieve that. This will export a CSV file using only the default fields as configured via the UI

# previous expression

issue_export(jql=jql, field_type="current")

For import of attachments via CSV which demands the addition of your credential to the attachment columns, it is possible to use this same method to append a user credential which is acceptable by the Jira external importer.

# previous expression

issue_export(jql=jql, allow_media=True)

Then making it easier, to just import the projects and their media content.

If you have other Jira or even non-Jira CSV files and you want to merge all those files into a single file, you can use the merge_files argument which accepts a list of file names.

# previous expression

files = ["file1.csv", "file2.csv", "file3.csv", "file4.csv"]
issue_export(merge_files=files)

If the files you're merging are non-Jira files, you can use the check_auth argument to prevent the method from checking for authentication.

# previous expression

files = ["file1.csv", "file2.csv", "file3.csv", "file4.csv"]
issue_export(merge_files=files, check_auth=False)

[jorizdg]

@Prince Nyeche Thanks for this. I get the following errors when I use the simple execution:

Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\jdeguzman\AppData\Local\Programs\Python\Python39\lib\threading.py", line 980, in _bootstrap_inner
self.run()
File "C:\Users\jdeguzman\AppData\Local\Programs\Python\Python39\lib\threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\venv\lib\site-packages\jiraone\reporting.py", line 2415, in map_field
mapper = field.get_field(fname)
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\venv\lib\site-packages\jiraone\access.py", line 3606, in get_field
"key": a["key"],
KeyError: 'key'

Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Users\jdeguzman\AppData\Local\Programs\Python\Python39\lib\threading.py", line 980, in _bootstrap_inner
self.run()
File "C:\Users\jdeguzman\AppData\Local\Programs\Python\Python39\lib\threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\venv\lib\site-packages\jiraone\reporting.py", line 2415, in map_field
mapper = field.get_field(fname)
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\venv\lib\site-packages\jiraone\access.py", line 3598, in get_field
"key": a["key"],
KeyError: 'key'

Traceback (most recent call last):
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\main.py", line 28, in <module>
issue_export(jql=jql)
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\venv\lib\site-packages\jiraone\reporting.py", line 2454, in export_issues
field_value_check(_field_names_) # We'll always check this
File "C:\Users\jdeguzman\PycharmProjects\Jira Export\venv\lib\site-packages\jiraone\reporting.py", line 2439, in field_value_check
raise JiraOneErrors(
jiraone.exceptions.JiraOneErrors: <JiraOneError: Unable to find initial field, probably such field "Sprint,Watchers,Reporter,Assignee" doesn't exist for fields argument>

I get the same error when I specify fields. Appreciate any guidance here. Thanks

[Prince Nyeche]

Hey @jorizdg 

Change the fields argument to an empty list

fields = []
jql = ...
issue_export(jql=jql, fields=fields)

I believe this issue occurs on the Jira Server/DC. If you're using that then you might want to do

fields = []
jql = ...
LOGIN.api = False
issue_export(jql=jql, fields=fields)

Let me know which one works for you.

[jorizdg]

@Prince Nyeche  Changing the argument to an empty list solved it. (I'm on Jira DC so I used the latter option). Thanks!

[jaych]

@Prince Nyeche this API has been great to use in python but I'm trying to integrate it with Microsoft Fabric. The notebook runs alone but one I try to automate the process I get an error saying "Error: Size of notebook content and exit value must be less than 8MB. Please try to write large outputs to files"

Is there a way to change where the files are being saved to? I want to save the files to a data lakehouse in fabric. Thanks in advance. 

[Prince Nyeche]

Hey @jaych if you are connected to any storage plugin within MS Fabric such as onelake, you can add a directory path or URI by using the folder argument and set a file name by using the final_file argument (optional). For example

issue_export(folder="<storage_path_URI_here>", final_file="<file_name>")

The concept here is that the folder argument expects a path-like object and it will attempt to link it to the filename before giving an output. As long as the path exists, it will generate and form a symlink to that path. Thereby allowing you to store the data in a different location other than the default which seems to be problematic in your notebook.

[jaych]

Thank you! That solved an issue I've had for months. I'm also getting a few errors after the report is complete. 
1)DtypeWarning: Columns have mixed types. Specify dtype option on import or set low_memory=False.

2)FutureWarning: The default value of regex will change from True to False in a future version. In addition, single character regular expressions will *not* be treated as literal strings when regex=True.

3)FutureWarning: The default value of regex will change from True to False in a future version. In addition, single character regular expressions will *not* be treated as literal strings when regex=True.

Are these errors I can correct on my end or does the API have to be updated? 

Thanks in advance. 

[Prince Nyeche]

I believe those warnings are from the notebook API being used. Probably there's a configuration within the application settings to turn warnings off but I'm unsure about that.

[Birgit Saalfeld]

Hello @Prince Nyeche

I need help with the following error message, since I don't have a clue, what I did wrong:

from jiraone import LOGIN, issue_export, USER
import json

file = "config.json"
config = json.load(open(file))
LOGIN(**config)

jql = "project=ITS"
issue_export(jql=jql)

Traceback (most recent call last):

Cell In[16], line 1
issue_export(jql=jql)

File ~\AppData\Local\anaconda3\lib\site-packages\jiraone\reporting.py:3365 in export_issues
field_value_check(_field_names_) # We'll always check this

File ~\AppData\Local\anaconda3\lib\site-packages\jiraone\reporting.py:3347 in field_value_check
raise JiraOneErrors(

JiraOneErrors: <JiraOneError: Unable to find initial field, probably such field "Watchers,Reporter,Assignee" doesn't exist for fields argument>

I am sure, that my authentication works, since print(LOGIN.get(endpoint.myself())) shows me 200 as response and I could successfully download all users (USER.get_all_users(file="user.csv", folder="USERS")). Furthermore, I also tried using the optional parameter "fields". Both: as empty element ([]) and with the columns you suggested earlier in the comments.

Setup: Access to Jira cloud with python 3.10.9 and jiraone 0.8.2 in Spyder, Anaconda.

Does anyone have any idea what I have to change to make the issue export call working?

Thanks a lot in advance!

### UPDATE ###

Since I am a python beginner, I oversaw that there is a difference between the function export_issue and the static function issue_export.

fields=[] # works fine
fields = ["key", "reporter", "issuetype", "status", "created"] # gives an error
export_file = "example_export.csv"
jql = "project=ITS"

issue_export(jql=jql,include_fields = fields, final_file=export_file, extension="csv", encoding="utf_8", delimit=";")

When I try to specify the columns, I end up with the following error:

issue_export(jql=jql,include_fields = fields, final_file=export_file, extension="csv", encoding="utf_8", delimit=";")
Traceback (most recent call last):

Cell In[100], line 1
issue_export(jql=jql,include_fields = fields, final_file=export_file, extension="csv", encoding="utf_8", delimit=";")

File ~\AppData\Local\anaconda3\lib\site-packages\jiraone\reporting.py:3370 in export_issues
field_value_check(

File ~\AppData\Local\anaconda3\lib\site-packages\jiraone\reporting.py:3347 in field_value_check
raise JiraOneErrors(

JiraOneErrors: <JiraOneError: Unable to find initial field, probably such field "key,reporter,issuetype,status,created" doesn't exist for include_fields argument>

[Prince Nyeche]

Hey @Birgit Saalfeld 

There's no difference between the method export_issues and issue_export as both reference the same object. However, when dealing with fields argument, the names of the fields are case-sensitive and have to be represented the way they appear on Jira.

For example, if you're looking for these fields

fields = ["key", "reporter", "issuetype", "status", "created"]

You have to call it this way

fields = ["Key", "Reporter", "Issue Type", "Status", "Created"]

An empty list of the fields argument will prevent the lookup of those fields. If your use case is to include certain fields, you have to use the exact name of the field the way they are represented on Jira. Subsequently, you could select the columns on your Jira UI advance search and use the field_type argument with the value "current" to export the same field columns as they appear on Jira UI.

[Birgit Saalfeld]

Thank you so much for your fast and helpful answer. Your example worked fine for my test project.
However, I am still facing problems regarding the naming.

Your sentence "However, when dealing with fields argument, the names of the fields are case-sensitive and have to be represented the way they appear on Jira." makes me curios.

When I apply my query against the Iira-Instance where I must export the issues from, it seems, that the don't have a "key"-Information. The following code snippet works without the "Key" argument.

jql = "project= KKN AND issuetype != Sub-task and createddate > '2024-03-22' ORDER BY key"
fields = ["Zusammenfassung", "Autor", "Sprint" ]#"cf[10064]"
issue_export(jql=jql,include_fields = fields, final_file=export_file, extension="csv", encoding="utf-8", delimit=";")

As soon as i add "Key" (or the listed alternatives) to the fields variable, the request fails with following message.

#Alternatives tried: "Issue Key", "Key", "key", "issue key", "Schluessel", "Vorgangsschlüssel", "Vorgangs-ID"
JiraOneErrors: <JiraOneError: Unable to find initial field, probably such field "Key" doesn't exist for include_fields argument>

What could be the reason?

[Prince Nyeche]

I'm not entirely sure why that happens for your environment but I suggest using the next best alternative which is the field_type argument that would give you exactly the export you need if the include_fields argument keeps throwing an error about the "Key" field not being available.

[Birgit Saalfeld]

Dear @Prince Nyeche 

the workaround with field_type="all" works fine. I still don't know, why the Jira-instance behaves in such a weird way. Thanks a lot.