How do I get comments and custom fields from all issues in my project using python?

[Noah F Wilson]

I am trying to use the Jira Rest API to gather data from Jira tickets in my project using python. I want to be able to collect the comments, custom fields, and summary/key.

Trying to use the following code:

import pandas as pd

import requests

from requests.auth import HTTPBasicAuth

import json

from concurrent.futures import ThreadPoolExecutor

pat = "ACCESS TOKEN"

email = "EMAIL"

url = "https://MY_COMPANY.atlassian.net/rest/api/3/jql"

auth = HTTPBasicAuth(email, pat)

headers = {

  "Accept": "application/json",

  "Content-Type": "application/json"

}

base_payload = {

  "expand": [

    "names",

    "schema",

    "operations"

  ],

  "fields": [

    "key",

    "comment"

  ],

  "fieldsByKeys": False,

  "jql": 'project = PROJECT NAME',

  "maxResults": 100,

  "startAt": 0

}

def fetch_issues_chunk(start_at):

    payload = base_payload.copy()

    payload["startAt"] = start_at

    payload = json.dumps(payload)

    response = requests.request("POST", url, data=payload, headers=headers, auth=auth)

    if response.status_code != 200:

        print(f"Request failed with status code: {response.status_code}, message: {response.text}")

        return []

    data = response.json()

    return data.get("issues", [])

def fetch_all_issues(url, headers, base_payload, auth):

    initial_response = requests.request("POST", url, data=json.dumps(base_payload), headers=headers, auth=auth)

    if initial_response.status_code != 200:

        print(f"Initial request failed with status code: {initial_response.status_code}, message: {initial_response.text}")

        return []

    initial_data = initial_response.json()

    total = initial_data.get("total", 0)

    issues = initial_data.get("issues", [])

    with ThreadPoolExecutor() as executor:

        futures = [executor.submit(fetch_issues_chunk, start_at) for start_at in range(100, total, 100)]

        for future in futures:

            issues.extend(future.result())

    return issues

# Fetch all issues

issues = fetch_all_issues(url, headers, base_payload, auth)

And this continues to return a 400 code for me [Initial request failed with status code: 400, message: {"errorMessages":["Invalid request payload. Refer to the REST API documentation and try again."]}] and I am not sure why.. TIA

[Sunny Ape · Accepted solution]

Hello @Noah F Wilson 

url = "https://MY_COMPANY.atlassian.net/rest/api/3/jql"

If you refer to the Jira Cloud REST API documentation, you will see there is no POST endpoint with the path /rest/api/3/jql

You probably should be using the Search for Issues using JQL (POST) endpoint which has the path /rest/api/3/search/jql

I recommend that you use an API test tool like Postman to validate your requests independently of your code.

[Noah F Wilson]

Ahhhh I am sorry this is a typo in my code, I am using /rest/api/3/search/jql and still getting this error. I will check the Postman tool to validate thanks!

[Sunny Ape]

When you get such error responses from a REST APII when making a complex request, break the request down into smaller pieces and solve the problem one step at time.

Start by making the simplest request possible to that API endpoint; declare just the jql parameter and its value only. If that request works, then start adding more parameters and their values, one by one, making sure each new combination works, until you encounter which one is causing the problem. Then, refer to the documentation to confirm:

1. The endpoint actually supports that particular parameter, and if so:
2. The value has been declared in the correct format required

Have fun

PS. In your Python code, I can see which of your request parameters is invalid and causing the error, but I would like you to try the problem solving process for yourself, rather than just give you the answer. I hope you understand my rational and take it in the manner intended.