JsonParseException: Invalid UTF-8 middle byte 0xe4 on RestAPI create project

Question

Hi

we run a groovy script to create projects from a jira issue in a workflow post function via RestAPI "createshared". The script runs fine as long as the project name doesn't contain german special character (Umlauts). Then the following error is thrown:

Uncaught exception thrown by REST service: Invalid UTF-8 middle byte 0xe4

Here is my code snippet:

final String PROJECT_NAME_TO_CREATE = "ÄäÖöÜüß"
final String PROJECT_LEAD_USERNAME = issue.reporter.name //Reporter wird Project Lead

def cField5Object = customFieldManager.getCustomFieldObjectsByName('Produkt + Template')
def cField5 = cField5Object[0]
def cFieldValue5 = issue.getCustomFieldValue(cField5)
def String template = String.valueOf(cFieldValue5).drop(14);
final String PROJECT_TEMPLATE = template.substring(0,template.length() - 1)

// Projekt Beschreibung wird der Template Name eingefügt
final String PROJECT_DESCRIPTION = "Basierend auf Template: " + PROJECT_TEMPLATE

projectTemplateID = 20925
ProjectManager projectManager = ComponentAccessor.getProjectManager()
Project templateProjectId = projectManager.getProjectByCurrentKey(projectTemplateKey);

//StringEntity stringEntity = new StringEntity('{"key":"RYMG","name":"Rest Ye Merry Gentlemen","lead":"XV13499"}')
def propertyString = '{"key":"' + PROJECT_KEY_TO_CREATE + '","name":"' + PROJECT_NAME_TO_CREATE + '","lead":"' + PROJECT_LEAD_USERNAME + '","description":"' + PROJECT_DESCRIPTION + '"}'
StringEntity stringEntity = new StringEntity(propertyString)
log.warn("stringEntity: " + propertyString)
HttpPost httpPost = new HttpPost(JIRA_BASEURL + "/rest/project-templates/1.0/createshared/" + projectTemplateID)
log.warn("HttpPost = " + JIRA_BASEURL + "/rest/project-templates/1.0/createshared/" + projectTemplateID)
httpPost.setHeader(HttpHeaders.CONTENT_TYPE, "application/json")
httpPost.setEntity(stringEntity)

//httpPost.setHeader(HttpHeaders.AUTHORIZATION, "Basic dXNlcjpwd2Q=")
httpPost.setHeader(HttpHeaders.AUTHORIZATION, "Basic ${authString}")
HttpClient httpClient = HttpClientBuilder.create().build()
HttpResponse response = httpClient.execute(httpPost)

The RestAPI with curl directly from the commandline works perfect.

Any ideas

Peter

Peter Brodt · Answer

Hi Peter,

thanks so much for your help.  After implementing the json builder everything works fine.

I definitely will change the script in respect to your "TrustedRequestFactory" advice.

Yes, there are sureley smarter ways for extracting the project id. But it is a workflow for our end-users to create Jira project or Conflunec space generation tickets. The field in question is not a project picker but a hard coded cascading select list (product[Jira|Confluence] + Templates). Since there are only a couple of templates we allow, we did it the simple way.

The native project service was our first attempt. Unfortunatly we came across an error, that newly created projects sometimes were not visible. After editing the project details of a random project, they became visible. The error did not occur every time but much too often to be acceptable. Neither Adaptavist nor Atlassian were able to help. That's why we switched to REST.

Thanks again for your help. I really appreciate it (:-.

Peter

PD Sheehan · Answer

I was forgetting the newer syntax for getting a built-in script object:

import com.onresolve.scriptrunner.runner.ScriptRunnerImpl
import com.onresolve.scriptrunner.canned.jira.admin.CopyProject

def copyProject = ScriptRunnerImpl.scriptRunner.createBean(CopyProject)

PD Sheehan · Answer

I have several thoughts ... 1) If you are manipulating JSON in groovy, don't try to do it yourself. Use the built-in libraries for that import groovy.json.JsonBuilder def projectPayload = [key: PROJECT_KEY_TO_CREATE, name:PROJECT_NAME_TO_CREATE, lead:PROJECT_LEAD_USERNAME, description:PROJECT_DESCRIPTION] def projectPayloadJson = new JsonBuildere(projectPayload).toPrettyString() This should fix your immediate issue. Here is a sample console script that shows the difference: import groovy.json.JsonBuilder final String PROJECT_NAME_TO_CREATE = "ÄäÖöÜüß" String PROJECT_KEY_TO_CREATE = 'key' String PROJECT_LEAD_USERNAME = 'lead' String PROJECT_DESCRIPTION = 'desc' def projectPayload = [key: PROJECT_KEY_TO_CREATE, name: PROJECT_NAME_TO_CREATE, lead: PROJECT_LEAD_USERNAME, description: PROJECT_DESCRIPTION] def propertyString = '{"key":"' + PROJECT_KEY_TO_CREATE + '","name":"' + PROJECT_NAME_TO_CREATE + '","lead":"' + PROJECT_LEAD_USERNAME + '","description":"' + PROJECT_DESCRIPTION + '"}' log.info "raw object output: $projectPayload" log.info "proper json: ${new JsonBuilder(projectPayload).toString()}" log.info "manual json: $propertyString" 2) If you need to use the rest API, it's much better to use the "TrustedRequestFactory" import com.atlassian.jira.component.ComponentAccessor import com.atlassian.jira.config.properties.APKeys import com.atlassian.sal.api.net.Request import com.atlassian.sal.api.net.TrustedRequest import com.atlassian.sal.api.net.TrustedRequestFactory import com.onresolve.scriptrunner.runner.customisers.PluginModule import groovyx.net.http.URIBuilder @PluginModule TrustedRequestFactory trustedRequestFactory def runAsUser= ComponentAccessor.userManager.getUserByName('admin') def baseUrl = ComponentAccessor.applicationProperties.getString(APKeys.JIRA_BASEURL) def request = trustedRequestFactory.createTrustedRequest(Request.MethodType.POST, '/rest/project-templates/1.0/createshared/') as TrustedRequest request.addTrustedTokenAuthentication(new URIBuilder(baseUrl).host, runAsUser.name) def projectPayload = [key: PROJECT_KEY_TO_CREATE, name: PROJECT_NAME_TO_CREATE, lead: PROJECT_LEAD_USERNAME, description: PROJECT_DESCRIPTION] def projectPayloadJson = new JsonBuildere(projectPayload).toPrettyString() request.setRequestBody(projectPayloadJson, 'application/json') def response = request.execute() This way, you won't need to deal with protecting credentials 3) I'm not sure what type of custom field your "Produkt + Template" field is, but I'm sure there is a better way to extract a project key from the selected item. If your field is a "Project Picker" field, then the value of the getCustomFieldValue will be a project object. You could just get the key or id directly from it (or, if you want auto-completion and prevent static type checking error, you can type cast the result): import com.atlassian.jira.project.Project def selectedProject = issue.getCustomFieldValue(cf) as Project def selectedProejctKey = selectedProject.key def selectedProjectId = selectedProject.id 3) But really, you don't need to use REST API at all for this. you could either use the native ProjectService, or the built-in script "Copy Project. Here is a native example (for the console) using some of your variables and values: import com.atlassian.jira.component.ComponentAccessor import com.atlassian.jira.issue.Issue import com.atlassian.jira.project.Project import com.atlassian.jira.bc.project.ProjectCreationData import com.atlassian.jira.bc.project.ProjectService def projectService = ComponentAccessor.getComponent(ProjectService) def issue = ComponentAccessor.issueManager.getIssueObject('KEY-123') def runAsUser = ComponentAccessor.userManager.getUserByName('admin') def projectCf = ComponentAccessor.customFieldManager.getCustomFieldObjectsByName('Project Name')[0] def PROJECT_TEMPLATE = (issue.getCustomFieldValue(projectCf) as Project) String PROJECT_NAME_TO_CREATE = "ÄäÖöÜüß" String PROJECT_DESCRIPTION = "Basierend auf Template: " + PROJECT_TEMPLATE.key String PROJECT_KEY_TO_CREATE = 'KEY' def projBuilder = new ProjectCreationData.Builder() projBuilder. withProjectTemplateKey(PROJECT_TEMPLATE.key). withKey(PROJECT_KEY_TO_CREATE). withName(PROJECT_NAME_TO_CREATE). withDescription(PROJECT_DESCRIPTION). withLead(issue.reporter) def validationResult = projectService.validateCreateProjectBasedOnExistingProject(runAsUser, PROJECT_TEMPLATE.id,projBuilder.build()) assert validationResult.isValid(), validationResult.errorCollection projectService.createProject(validationResult) Here is an example with the CopyProject: import com.atlassian.jira.bc.project.ProjectCreationData import com.atlassian.jira.component.ComponentAccessor import com.onresolve.scriptrunner.canned.jira.admin.CopyProject import com.atlassian.jira.project.Project def projectManager = ComponentAccessor.projectManager def runasUser = ComponentAccessor.userManager.getUserByName('admin') //user with create project permission def issue = ComponentAccessor.issueManager.getIssueObject('KEY-123') def projectCf = ComponentAccessor.customFieldManager.getCustomFieldObjectsByName('Project Name')[0] def PROJECT_TEMPLATE = issue.getCustomFieldValue(projectCf) as Project String PROJECT_NAME_TO_CREATE = "ÄäÖöÜüß" String PROJECT_DESCRIPTION = "Basierend auf Template: " + PROJECT_TEMPLATE.key String PROJECT_KEY_TO_CREATE = 'KEY' def copyProject = new CopyProject() def inputs = [ (CopyProject.FIELD_SOURCE_PROJECT) : PROJECT_TEMPLATE.key, (CopyProject.FIELD_TARGET_PROJECT) : PROJECT_KEY_TO_CREATE, (CopyProject.FIELD_TARGET_PROJECT_NAME) : PROJECT_NAME_TO_CREATE, (CopyProject.FIELD_COPY_VERSIONS) : false, (CopyProject.FIELD_COPY_COMPONENTS) : false, (CopyProject.FIELD_COPY_ISSUES) : false, (CopyProject.FIELD_COPY_DASH_AND_FILTERS): false, ] def errorCollection = copyProject.doValidate(inputs as Map, false) assert !errorCollection.hasAnyErrors(), errorCollection ComponentAccessor.jiraAuthenticationContext.setLoggedInUser(runasUser) def output = copyProject.doScript(inputs as Map) log.info output //the CopyProject script can't set the Project Lead and Description during the copy, so we set them after def newProject = projectManager.getProjectObjByKeyIgnoreCase(PROJECT_KEY_TO_CREATE) projectManager.updateProject(newProject, newProject.name, PROJECT_DESCRIPTION, issue.reporter.name, "", newProject.assigneeType) I personally prefer the way the built-In CopyProject script handles custom fields contexts (the new project will share any contexts that the template project used, whereas the ProjectService method, the new project is not added to any context). Sorry... looks like I went a bit overboard with this answer. Hopefully, it can help some others too.