It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Receiving Error 400 when creating Issue through Jira REST API , Occurs in Python but not Powershell Edited

Nick Boyd Apr 23, 2018

I'm encountering a really strange issue with the Jira Rest API.

Whever I try to create an issue using POST request to jira/rest/api/latest/issue/ I receive Error code 400 from Python 2.7 requests but it succeeds from Powershell's Invoke Web Request.

I've tried a number of troubleshooting suggestions from similar topics on this site already:

  • I've verified that the Authorisation is correct (it's also required for the GET requests which all work)
  • I'm testing with an account that has administrator level access to the Jira instance
  • I've stripped the json back to just the fields and format matching the REST API documentation here: jira-rest-api-examples/#creating-an-issue-examples
  • I've ensured that all the relevant session cookie and header data is stored and added to the follw-up requests
  • I've verified through Issue/createmeta that I have the capabilities to create that issue type (and as I pointed out earlier - it works in Powershell)
  • I've tried using Issuetype name and id as well as project key and id as the identifier, neither one changes anything
  • I've even tried both omitting and including the trailing slash on the /issue path in case that was important

Json Body (Raw):

{"fields": {"issuetype": {"id": "10702"}, "project": {"id": "10061"}, "description": "Execution for Issue: SDBX-859", "summary": "Execution for Issue: SDBX-859"}}

(Formatted for Legibility):

{
    "fields": {
        "issuetype": {
            "id": "10702"
        },
        "project": {
            "id": "10061"
        },
        "description": "Execution for Issue: SDBX-859",
        "summary": "Execution for Issue: SDBX-859"
    }
}

 

The process flow starts with this class:

class Migrator(object):
RestURLs = {
"projects": "api/latest/project",
"issuetype": "api/latest/issuetype",
"fields": "api/latest/field",
"tests": "api/latest/search?maxResults={limit}&expand=meta&jql=IssueType='{testType}'+and+project={projectKey}",
"zSteps": "zapi/latest/teststep/{issueId}",
"zExecutions": "zapi/latest/zql/executeSearch?zqlQuery=project={projectKey}",
"zCycles": "zapi/latest/cycle?projectId={projectId}",
"issue": "api/latest/issue/{issueKey}",
"xSteps": "raven/1.0/api/test/{issueKey}/step/{stepId}",
"xSet": "raven/1.0/api/testset/{issueKey}/test",
"xExecution": "raven/1.0/api/testexec/{issueKey}/test"
}

CustomFields = {
"Zephyr Teststep": "",
"Manual Test Steps": "",
"Test Type": ""
}

IssueNames = {
"zephyr":"Zephyr - Test",
"xray":"Test",
"set":"Test Set",
"execution":"Test Execution"
}
IssueTypes = {}

def __init__(self):
self.results = []
print("new Migrator initialised")
self.restHandler = RestHandler()
self.baseURL = "http://127.0.0.1/jira/rest/"
self.authentication = ""
self.commonHeaders = {}
self.projectList = []
self.project = None
self.testList = []
self.executionList = {}
self.versionList = set()
self.cycleList = {}
self.setList = []

def connect(self, username, password, serverUrl="http://127.0.0.1"):
# 1 - connect to jira
if serverUrl[-1] != '/':
serverUrl += '/'
self.baseURL = str.format("{0}jira/rest/", serverUrl)
self.authentication = "Basic " + base64.b64encode(username + ":" + password)
self.commonHeaders = {"Authorization": self.authentication}

print("Connecting to Server: " + self.baseURL)
headers = self.commonHeaders
projList = self.restHandler.perform(method=HTTP.GET,url=self.baseURL,path=Migrator.RestURLs["projects"],headers=headers)

# 2 - populate projects list
for projDict in projList:
self.projectList.append(Project().fromDict(projDict))

from this method:

    def migrateExecutions(self, project):
print "working..."
for execution in self.executionList:
# Restricting it only to my issues for testing...
if execution.assigneeUserName == "boydnic":
headers = self.commonHeaders
execData = {"fields":{}}
execData["fields"]["issuetype"] = {"id":self.IssueTypes[self.IssueNames["execution"]].id}
execData["fields"]["project"] = {"id":project.id}
# execData["fields"]["reporter"] = {"name": userName}
# execData["fields"]["assignee"] = {"name": execution.assigneeUserName}
execData["fields"]["summary"] = "Execution for Issue: " + execution.issueKey
execData["fields"]["description"] = execution.comment if execution.comment else execData["fields"]["summary"]

xrayExec = self.createIssue(execData)
self.results.append(self.restHandler.perform(method=HTTP.POST, url=self.baseURL,
path=self.RestURLs["xExecution"], urlData={"issueKey":xrayExec.key},
headers=headers, body={"add":[execution.issueKey]}))

to this Method:

def createIssue(self, issueTemplate):
result = self.restHandler.perform(method=HTTP.POST, url=self.baseURL, path=Migrator.RestURLs["issue"], urlData={"issueKey":""}, headers=self.commonHeaders, body=issueTemplate)
issue = Issue()
issue.id = result["id"]
issue.key = result["key"]
issue.self = result["self"]
print("Created Issue: "+issue.key)
return issue

Which itself calls this class:

class RestHandler(object):
def __init__(self):
self.headerStore = {'X-CITNET-USER':"",
'X-ASEN':"",
'X-ASESSIONID':"",
'X-AUSERNAME':""}
self.cookieJar = requests.cookies.RequestsCookieJar()

def perform(self, method, url, path, headers={}, urlData={"projectId": "", "projectKey": "", "issueId": "", "issueKey": ""},
formData=dict(), body=""):
resultData = "{}"
path = url + path.format(**urlData)
body = body if isinstance(body, str) else json.dumps(body)
if self.headerStore:
headers.update(self.headerStore)
jar = self.cookieJar
print(str(method))
print(path)
if method is HTTP.GET:
resultData = requests.get(path, headers=headers, cookies = jar)
elif method is HTTP.POST:
print (body)
path = path.rstrip('/')
resultData = requests.post(path, json=body, headers=headers, cookies = jar)
elif method is HTTP.PUT:
print (body)
resultData = requests.put(path, json=body, headers=headers, cookies = jar)
elif method is HTTP.DELETE:
request = "DELETE request to " + path
else:
raise TypeError
print("\n\n===============================\nRest Call Debugging\n===============================")
print(resultData)
print(resultData.url)
print(resultData.status_code)
print(resultData.headers)
print(resultData.content)
print("\n\n===============================\n/Rest Call Debugging\n==============================")
if 199 < resultData.status_code < 300:
for hKey, hValue in resultData.headers.iteritems():
if hKey in self.headerStore.keys():
self.headerStore[hKey] = hValue
self.cookieJar.update(resultData.cookies)
print "testing breakpoint"
return json.loads(resultData.content)
else:
raise IOError(resultData.reason)

I'm getting incredibly frustrated by this, not least because it's blocking the completion of this migration script and seems to make no sense.

2 answers

1 accepted

0 votes
Answer accepted
Nick Boyd Apr 25, 2018

It turns out that the json= kwarg in the requests library, does a dumb dump of whatever you pass it. (similar to the the check I did earlier where I perform a json dumps() if the body var is not a string, but without the checking beforehand)

As a result, Requests was reserialising the data I'd already serialised, but the avaialable output in the IDE gave no indication of it.

Thanks to both Edwin and Warren for their help.

0 votes
Warren Levy Community Leader Apr 23, 2018

Hi Nick

I know you're saying that it works in PowerShell, but it would be useful if you gave us the JSON that you're using to create an issue (you can anonymise it if there's any sensitive data) and the full call i.e. beyond the api/latest/issue

Nick Boyd Apr 23, 2018

Hi Warren, thankfully any sensitive data is loaded at runtime so it's safe, I've posted the relevant sections of code here, I can post the entire script if necessary (only 548 lines currently) but most of it is just fluff for digesting Zephyr test cases/cycles etc from ZAPI.

Warren Levy Community Leader Apr 23, 2018

Okay, I've been comparing your JSON with one of mine which works (see below). The difference I note is that I use name for issuetype and key for project, as opposed to the id's

{
"fields": {
"project": {
"key": "DS"
},
"summary": "Fix invalid HTML or missing opening / closing tags",
"description": "Work through the latest Sitemorse list",
"issuetype": {
"name": "Task"
},
}
}

Possibly try those minor changes and see if it helps

Nick Boyd Apr 24, 2018

Hi Warren, I'm afraid I've already tried that (6th bullet point) without any success.

Also, according to the docs either combinations of (id/key) and (id/name) should work (indeed, it does over powershell) which is why I'm so stumped by this.

Warren Levy Community Leader Apr 24, 2018

Right, we need to take a step back here and think about it.

You're saying that the same JSON is used successfully in PowerShell, so let's assume that the problem isn't there. That means that somewhere in your python calling code, there's something wrong. I briefly looked through the code you've given, the line 

path = path.rstrip('/')

concerns me - it looks as though you're stripping out all /'s ??

Could you let me have all the output from your Rest call debugging section?

Nick Boyd Apr 24, 2018 • edited

Happily, Redacted a few bit of session info, but otherwise intact:

===============================
Rest Call Debugging
===============================
https://webgate.test.ec.europa.eu/CITnet/jira/rest/api/latest/issue
400
{'X-AUSERNAME': 'boydnic', 'X-AREQUESTID': '<redacted>', 'X-Content-Type-Options': 'nosniff', 'Transfer-Encoding': 'chunked', 'Set-Cookie': 'crowd.token_key=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/; HttpOnly, crowd.token_key=<redacted>; Path=/; HttpOnly, JSESSIONID=<redacted>; Path=/CITnet/jira; HttpOnly, atlassian.xsrf.token=<redacted>; Path=/CITnet/jira', 'X-Seraph-LoginReason': 'OUT, OK', 'X-ASEN': '<redacted>', 'X-CITNET-USER': 'boydnic', 'Connection': 'Keep-Alive', 'X-ASESSIONID': '<redacted>', 'Cache-Control': 'no-cache, no-store, no-transform, proxy-revalidate', 'Date': 'Tue, 24 Apr 2018 08:29:16 GMT', 'Server': 'Apache-Coyote/1.1', 'Content-Type': 'application/json;charset=UTF-8'}
{"errorMessages":["Can not instantiate value of type [simple type, class com.atlassian.jira.rest.v2.issue.IssueUpdateBean] from JSON String; no single-String constructor/factory method"]}


===============================
/Rest Call Debugging
==============================

 

It's intermingled (disentangled them for this message) with the Error stacktrace from Python that lays out the path of method calls I detailed above:

 File "C:/Users/BOYDnic/Documents/migrator/issueMigrator.py", line 330, in migrate
self.migrateExecutions(project)
File "C:/Users/BOYDnic/Documents/migrator/issueMigrator.py", line 475, in migrateExecutions
xrayExec = self.createIssue(execData)
File "C:/Users/BOYDnic/Documents/migrator/issueMigrator.py", line 334, in createIssue
result = self.restHandler.perform(method=HTTP.POST, url=self.baseURL, path=Migrator.RestURLs["issue"], urlData={"issueKey":""}, headers=self.commonHeaders, body=issueTemplate)
File "C:/Users/BOYDnic/Documents/migrator/issueMigrator.py", line 84, in perform
raise IOError(resultData.reason)
IOError: Bad Request

As for the rstrip(), it will only remove the last instances of the given string, so it this case, I'm using it to turn "/issue/" into "/issue" but I've tested it without too.

(some REST implementations don't like end points being treated like directories for POST/PUT/DELETE requests, I wondered if that might be the case here)

 

Warren Levy Community Leader Apr 24, 2018

Has this creating an issue ever worked for you using python? Does it work to get data out of Jira?

Are there any other mandatory fields when creating directly in Jira, which aren't in your JSON?

Here are some links I came across doing a Google search of your error message - they may trigger something with you

https://community.atlassian.com/t5/Answers-Developer-Questions/Odd-error-while-trying-to-create-issue-via-REST/qaq-p/504590

https://community.atlassian.com/t5/Answers-Developer-Questions/Jira-CREATE-issue-API-ERROR-Can-not-instantiate-value-of-type/qaq-p/563866

Nick Boyd Apr 24, 2018 • edited

Hi Warren,

Thanks for the links, I'd read them before posting this question, the first is missing the opening and closing braces on their JSON (the body string included above passes Doug Crockford's JSLint tool).

The second one seems to be an issue with reading JSON from a file (the only time I do that is for loading Config options at initialisation and hasn't encountered any issues yet) the body tag is serialized from a vanilla Python dict.

 

I also turned up this result from StackOverflow:
https://stackoverflow.com/questions/33285966/400-error-while-trying-to-post-to-jira-issue

They appear to be trying to chain creation and transitioning together however and this is currently just the raw creation stage.

In terms of missing fields - not that I know of.
I stripped it back while testing to just the basic fields mentioned in the REST documentation.

 

As for pulling data from Jira - no issue there so far, there's several calls which pull:

  • Project List
  • List of Issue Types
  • List of Custom Fields
  • Zephyr Test Issue List for given Project
  • Zephyr Test Cycle List for Project
  • Zephyr Test Execution List for Project

All of these respond with code 200 and relevant data, all running through the same RestHandler class (and instance)

Nick Boyd Apr 24, 2018

Hi Warren,

Thanks for the links, I'd read them before posting this question, the first is missing the opening and closing braces on their JSON (the body string included above passes Doug Crockford's JSLint tool).

The second one seems to be an issue with reading JSON from a file (the only time I do that is for loading Config options at initialisation and hasn't encountered any issues yet) the body tag is serialized from a vanilla Python dict.

 

I also turned up this result from StackOverflow:
https://stackoverflow.com/questions/33285966/400-error-while-trying-to-post-to-jira-issue

They appear to be trying to chain creation and transitioning together however and this is currently just the raw creation stage.

In terms of missing fields - not that I know of.
I stripped it back while testing to just the basic fields mentioned in the REST documentation.

 

As for pulling data from Jira - no issue there so far, there's several calls which pull:

  • Project List
  • List of Issue Types
  • List of Custom Fields
  • Zephyr Test Issue List for given Project
  • Zephyr Test Cycle List for Project
  • Zephyr Test Execution List for Project

All of these respond with code 200 and relevant data, all running through the same RestHandler class (and instance)

Warren Levy Community Leader Apr 24, 2018

I'm going to ask some of the other Community Champions to have a look at this

Nick Boyd Apr 24, 2018

Cheers, I'm going to post on Stack Overflow too, see if I can get a few more eyes on this problem.

I'll also see if I can get any more data on the successful requests (I tried proxying Powershell through ZAP, but it refused to accept a HTTPS proxy)

Nick Boyd Apr 24, 2018 • edited

Also, is it just me or are some messages disappearing from this Thread? I can't see the response to your rstrip() question anymore. if you have a copy in your email/inbox, feel free to repost it.

Warren Levy Community Leader Apr 24, 2018

No it's not just you, I can see all my replies, but most of yours aren't appearing. I mentioned this to the other Champions as well

Nick Boyd Apr 24, 2018

Cheers, that's good to know!

I've repeated the information here over at StackOverflow and taken advantage of their code formatting (the code block here don't seem to handle python comments well):

/questions/50003096/jira-post-put-rest-calls-return-error-400-from-python

Edwin Kyalangalilwa Community Leader Apr 24, 2018

Nick,

Have you tried the Python JIRA library?

new_issue = jira.create_issue(project='PROJ_key_or_id', summary='New summary',
description='Look into this one', issuetype={'name': 'Bug'})

print(new_issue)
Nick Boyd Apr 25, 2018 • edited

Hi Edwin,

Thanks for the link, I hadn't actually tried that, (I also need to interface with the Zephyr and Xray REST api extensions, so I opted for a consolidated rest handler)

I'm looking into it today but it looks like I'll need to keep my RestHandler for the Zephyr and Xray sections though.

Let me test the plugin URLs, if they fail, then it's probably a deeper problem with either my code or the server configuration than a library dedicated to the Jira core functionality can solve.

Nick Boyd Apr 25, 2018 • edited

Hi Edwin and Warren,

I've found the issue thanks to channelling the requests through the OWASP ZAP proxy.

It turns out that the json= kwarg in the requests library, does a dumb dump of whatever you pass it. (similar to the the check I did earlier where I perform a json dumps() if the body var is not a string, but without the checking beforehand)

As a result, Requests was reserialising the data I'd already serialised, but the avaialable output in the IDE gave no indication of it.

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Jira

Try Jira Cloud for Outlook: Organize your work without leaving your inbox

Hi Atlassian community, My name is Max and I work on the product integration team at Atlassian. I am pleased to announce the early access program for the Jira Cloud add-in for Outlook. This add-in...

2,684 views 6 15
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you