Create project with shared configuration with groovy and REST

[Peter Brodt]

Hi folks,

I try to automate our project generation and fail because I have absolutely no idea how to implement the following REST API call within a groovy script.

curl -D- -u User:Password -X POST --data @Data.json -H "Content-Type: application/json" https://myJiraBaseURL/jira/rest/project-templates/1.0/createshared/{20926}

Any help is welcome. Best thing would be when the current user is used for authentication.

We are on Jira Server 8.13.4

Peter

[Radek Dostál · Accepted solution]

This seems to work as a quick PoC:

import com.atlassian.jira.component.ComponentAccessor
import org.apache.http.HttpHeaders
import org.apache.http.HttpResponse
import org.apache.http.client.HttpClient
import org.apache.http.client.methods.HttpPost
import org.apache.http.entity.StringEntity
import org.apache.http.impl.client.HttpClientBuilder

final String JIRA_BASEURL = ComponentAccessor.getApplicationProperties().getString("jira.baseurl")
final int SOURCE_PROJECT_ID = 38700
StringEntity stringEntity = new StringEntity('{"key":"RYMG","name":"Rest Ye Merry Gentlemen","lead":"isthiskeyorusername"}')

HttpPost httpPost = new HttpPost(JIRA_BASEURL + "/rest/project-templates/1.0/createshared/" + SOURCE_PROJECT_ID)
httpPost.setHeader(HttpHeaders.CONTENT_TYPE, "application/json")
httpPost.setEntity(stringEntity)

httpPost.setHeader(HttpHeaders.AUTHORIZATION, "Basic dXNlcjpwd2Q=")

HttpClient httpClient = HttpClientBuilder.create().build()
HttpResponse response = httpClient.execute(httpPost)

return response.getStatusLine().getStatusCode()

As for authentication/authorization, unless a better dev corrects me, you must provide something, basic user:pwd, token, cookie, something. The POST itself doesn't know about your browser session, the code is executed inside JVM, not in the browser.

All that said, however, Java API already does provide the interface to create projects based on a shared configuration:

https://docs.atlassian.com/software/jira/docs/api/8.13.4/com/atlassian/jira/bc/project/ProjectService.html#validateCreateProjectBasedOnExistingProject-com.atlassian.jira.user.ApplicationUser-java.lang.Long-com.atlassian.jira.bc.project.ProjectCreationData-

Is there a reason why you run a groovy script to do an http post? If you want to make this use Java API I do have some old code somewhere I could share as an example. The main benefit is that it would not need any hardcoded user, since the thread executing that Java API code already carries the information about whoever is running that code.

[Peter Brodt]

Hi Radek,

thanks so much for your response. The script works like charm (:-.

To answer your question: I implemented the project creation with the Java API but ran into a problem which I couldn't resolve. 

Running this code worked fine in our test environment.

---------

final ProjectService.CreateProjectValidationResult projectValidationResult = projectService.validateCreateProjectBasedOnExistingProject(loggedInUser, fromProjID, creationData)
assert projectValidationResult.valid : projectValidationResult.errorCollection
projectService.createProject(projectValidationResult)

---------

In our production environment the code created the project but the following statements  which try to add users to project roles run into an error because the project cannot be found in approximatly 9 out of 10.

A reindex of the system helped but only for a short time. 

I then added a project reindex to the code, without any success

import com.atlassian.jira.bc.project.index.ProjectReindexService
ProjectReindexService projectReindexService = ComponentAccessor.getComponent(ProjectReindexService.class);

If you have an idea how we can solve this I'd be grateful.

Peter

[Radek Dostál]

In our production environment the code created the project but the following statements  which try to add users to project roles run into an error because the project cannot be found in approximatly 9 out of 10.

A reindex of the system helped but only for a short time. 

That sounds very strange, are you sure the "project" can't be found, as in the Project project? I can't really guess without seeing the code, maybe there is something amiss in there. Indexing should play no part in this - I would say maybe you're having a caching problem perhaps, but not indexes.

The code that I made a while back is in .java and already gets somewhat pre-validated input, so I tried to:

 - gut it out to my IDE

 - transform to groovy

 - change a.. few things, make it more console/post-function-ish

Namely:

 - I added a method to copy project role actors to that created project as well, which seems to be working as expected, maybe you can compare this with your implementation if there are any differences

 - I realized it does in fact still need a service account (to create and add the currentUser to admin role), sooo my bad, but at least you just need the account name, no password or things like that, just need to get the user by username so it's still better than hardcoding it with http post

 - this also takes care of custom field contexts, because out of box creating shared configuration only associates the same schemes, but it doesn't do anything about contexts, so, this code does

Overall I changed a lot of stuff and only did very brief testing (on 2-node data center), so of course it might have a flaw somewhere, but hopefully not.

If you'd send your current Java API script and the error you're getting I might have a guess or two what the matter is, here comes my version, even if you don't use it I'll store this in git anyway for later :o

Anyway the code is imperfect, some things can be optimized and refactored, but, but, it seems to do it's job.

import com.atlassian.jira.bc.project.ProjectCreationData
import com.atlassian.jira.bc.project.ProjectService
import com.atlassian.jira.bc.projectroles.ProjectRoleService
import com.atlassian.jira.component.ComponentAccessor
import com.atlassian.jira.issue.CustomFieldManager
import com.atlassian.jira.issue.context.JiraContextNode
import com.atlassian.jira.issue.context.ProjectContext
import com.atlassian.jira.issue.fields.CustomField
import com.atlassian.jira.issue.fields.config.FieldConfigScheme
import com.atlassian.jira.issue.fields.config.manager.FieldConfigSchemeManager
import com.atlassian.jira.project.Project
import com.atlassian.jira.project.ProjectManager
import com.atlassian.jira.security.roles.*
import com.atlassian.jira.user.ApplicationUser
import com.atlassian.jira.user.util.UserManager
import com.atlassian.jira.util.SimpleErrorCollection
import groovy.transform.Field

//Note on this - the API to create a project requires a user in it's method as part of it's validation
//We don't need password or anything, this is just an account which the project will be "created by"
//This is assuming that the real user triggering this code is not a global admin, but just a regular user,
//Thus we will need to temporarily.. borrow the admin a few times
@Field String ADMIN_SERVICE_ACCOUNT_USERNAME = "admin"
@Field ApplicationUser ADMIN_SERVICE_ACCOUNT = ComponentAccessor.getUserManager().getUserByName(ADMIN_SERVICE_ACCOUNT_USERNAME)
if (ADMIN_SERVICE_ACCOUNT == null)
    throw new Exception("CreateProject with shared configuration | Admin service account cannot be found!")

final String PROJECT_LEAD_USERNAME = ComponentAccessor.getJiraAuthenticationContext().getLoggedInUser().getName() //current user or maybe something dynamic instead?
final String PROJECT_TEMPLATE_KEY = "SOURCEKEY"

final String PROJECT_NAME_TO_CREATE = "New Project Name"
final String PROJECT_KEY_TO_CREATE = "NEWPKEY"


Project createdProject
try {
    createdProject = createProjectFromExistingTemplate(PROJECT_NAME_TO_CREATE, PROJECT_KEY_TO_CREATE, PROJECT_LEAD_USERNAME, PROJECT_TEMPLATE_KEY)
    log.info("Created a project from template " +
            "{Project Name:"+PROJECT_NAME_TO_CREATE+", " +
            "Project Key:"+PROJECT_KEY_TO_CREATE+", " +
            "Template Key:"+PROJECT_TEMPLATE_KEY+", +" +
            "Project Type:"+createdProject.getProjectTypeKey().getKey()+"}")
}
catch (Exception e) {
    throw new Exception("CreateProject with shared configuration | Exception: " + e.toString())
}

// **********************
//      FUNCTIONS       *
// **********************
Project createProjectFromExistingTemplate(String projectName, String projectKey, String projectLeadUsername, String templateProjectKey) throws IllegalStateException {

    UserManager userManager = ComponentAccessor.getUserManager()
    ProjectManager projectManager = ComponentAccessor.getProjectManager()
    ProjectService projectService = ComponentAccessor.getComponent(ProjectService)

    ApplicationUser projectLeadUser = userManager.getUserByName(projectLeadUsername);
    Project templateProject = projectManager.getProjectByCurrentKey(templateProjectKey);
    String projectType = templateProject.getProjectTypeKey().getKey();

    ProjectCreationData.Builder builder = new ProjectCreationData.Builder();
    ProjectCreationData projectCreationData = builder
            .withName(projectName)
            .withKey(projectKey)
            .withType(projectType)
            .withLead(projectLeadUser)
            .build();

    ProjectService.CreateProjectValidationResult validationResult = projectService.validateCreateProjectBasedOnExistingProject(ADMIN_SERVICE_ACCOUNT, templateProject.getId(), projectCreationData);

    if (!validationResult.isValid())
        throw new Exception("CreateProjectValidationResult: " + validationResult.getErrorCollection().getErrorMessages().toString())

    /*
    This is a workaround to avoid getting runtime exceptions when a non-administrator user attempts to create a Service Desk project
    See, the problem is that JSD implements some kind of a ProjectCreatedHandler, which will then use .getLoggedInUser() internally and inevitably fail
    So we have to replace the user for the JSD handler to do it's stuff, and then we can switch it back
    Not ideal and a little ugly but.. what else can we do
    I believe however that we don't have to do this for non-servicedesk projects, the API should be fine there
     */
    ApplicationUser currentlyLoggedInUser = ComponentAccessor.getJiraAuthenticationContext().getLoggedInUser();
    Project project = null
    try {
        ComponentAccessor.getJiraAuthenticationContext().setLoggedInUser(ADMIN_SERVICE_ACCOUNT)
        project = projectService.createProject(validationResult);
    }
    finally {
        ComponentAccessor.getJiraAuthenticationContext().setLoggedInUser(currentlyLoggedInUser)
    }

    //Add currentUser(projectLead) to Administrators role
    addUserToProjectRole(project, getAdministratorsProjectRole(), projectLeadUser.getKey())

    //Add currentUser(projectLead) to Service Desk Team role as well if this is SD project
    if (projectType.equals("service_desk")) addUserToProjectRole(project, getServiceDeskTeamRole(), projectLeadUsername)

    //Because a project created from a template has the same project category, either remove it, or comment this line out
    projectManager.setProjectCategory(project, null);

    //We also need to set up custom field contexts, because they would be missing otherwise
    grantCustomFieldContextsToProjectFromTemplate(project, templateProject);

    //Finally, we copy project role actors
    copyProjectRoleActorsFromSourceProjectToDestinationProject(templateProject, project)

    //And if everything went fine, we should be returning the project from where
    return project;
}

ProjectRole getServiceDeskTeamRole() {
    ProjectRoleManager projectRoleManager = ComponentAccessor.getComponent(ProjectRoleManager)
    return projectRoleManager.getProjectRole("Service Desk Team")
}

ProjectRole getAdministratorsProjectRole() {
    ProjectRoleManager projectRoleManager = ComponentAccessor.getComponent(ProjectRoleManager)
    return projectRoleManager.getProjectRole("Administrators")
}

void copyProjectRoleActorsFromSourceProjectToDestinationProject(Project source, Project destination) {
    ProjectRoleManager projectRoleManager = ComponentAccessor.getComponent(ProjectRoleManager)
    ProjectRoleService projectRoleService = ComponentAccessor.getComponent(ProjectRoleService)
    Collection<ProjectRole> projectRoles = projectRoleManager.getProjectRoles()
    ApplicationUser currentlyLoggedInUser = ComponentAccessor.getJiraAuthenticationContext().getLoggedInUser();

    SimpleErrorCollection simpleErrorCollection = new SimpleErrorCollection()

    for (ProjectRole projectRole : projectRoles) {
        ProjectRoleActors projectRoleActors = projectRoleManager.getProjectRoleActors(projectRole, source)
        Set<RoleActor> userRolesActors = projectRoleActors.getRoleActorsByType(ProjectRoleActor.USER_ROLE_ACTOR_TYPE)
        List<String> userRoleActors_parameters = userRolesActors*.getParameter() //groovy is weird

        //remove our current user (project lead) to avoid trying to add the user to Administrators role again (should otherwise throw an error in error collection)
        //same with Service Desk Team
        if (projectRole.getName().equals("Administrators") || projectRole.getName().equals("Service Desk Team"))
            userRoleActors_parameters.remove(currentlyLoggedInUser.getKey())

        Set<RoleActor> groupRoleActors = projectRoleActors.getRoleActorsByType(ProjectRoleActor.GROUP_ROLE_ACTOR_TYPE)
        List<String> groupRoleActors_parameters = groupRoleActors*.getParameter()

        if (userRoleActors_parameters.size() > 0)
            projectRoleService.addActorsToProjectRole(userRoleActors_parameters, projectRole, destination, ProjectRoleActor.USER_ROLE_ACTOR_TYPE, simpleErrorCollection)

        if (groupRoleActors_parameters.size() > 0)
            projectRoleService.addActorsToProjectRole(groupRoleActors_parameters, projectRole, destination, ProjectRoleActor.GROUP_ROLE_ACTOR_TYPE, simpleErrorCollection)
    }

    if (simpleErrorCollection.hasAnyErrors())
        log.error("copyProjectRoleActorsFromSourceProjectToDestinationProject("+source.getKey()+", "+destination.getKey()+"): " + simpleErrorCollection.getErrorMessages().toString())
}

void grantCustomFieldContextsToProjectFromTemplate(Project project, Project template) {
    CustomFieldManager customFieldManager = ComponentAccessor.getCustomFieldManager();
    FieldConfigSchemeManager fieldConfigSchemeManager = ComponentAccessor.getFieldConfigSchemeManager();

    List<CustomField> customFields = customFieldManager.getCustomFieldObjects();

    for (CustomField customField : customFields) {
        List<FieldConfigScheme> fieldConfigSchemes = customField.getConfigurationSchemes(); //This is to Humans contexts in UI as we know it

        fieldConfigSchemeFor: for (FieldConfigScheme fieldConfigScheme : fieldConfigSchemes) {
            if (!fieldConfigScheme.isAllProjects()) {
                List<JiraContextNode> jiraContextNodes = fieldConfigScheme.getContexts();   //This is to Jira individual mappings within a human-context
                for (JiraContextNode jiraContextNode : jiraContextNodes) {                  //Each context than has this JiraContextNode for each project, so if we find that it contains
                    if (jiraContextNode.getProjectId().equals(template.getId())) {          // the template, we know we can also add a new JiraContextNode for our created project
                        JiraContextNode newContextNode = new ProjectContext(project.getId());
                        List<JiraContextNode> updatedContextNodes = new ArrayList<>(jiraContextNodes);
                        updatedContextNodes.add(newContextNode);                             //Add our project's context node to existing ones
                        fieldConfigSchemeManager.updateFieldConfigScheme(fieldConfigScheme, updatedContextNodes, customField);
                        break fieldConfigSchemeFor
                    }
                }
            }
        }
    }
    customFieldManager.refresh(); //refresh the caches
}

void addUserToProjectRole(Project project, ProjectRole projectRole, String userkey) {
    ProjectRoleService projectRoleService = ComponentAccessor.getComponent(ProjectRoleService)

    List<String> userkeys = new LinkedList<>()
    userkeys.add(userkey)

    SimpleErrorCollection simpleErrorCollection = new SimpleErrorCollection();

    //Using deprecated method as opposed to
    // projectRoleService.addActorsToProjectRole(usernames, projectRole, project, ProjectRoleActor.USER_ROLE_ACTOR_TYPE, simpleErrorCollection);
    // because this will try to execute this under currentUser, who will be evaluated as the regular user and so it could error out, that is why we are using deprecated method
    // to execute this under adminAccount
    // If we previously added the user as project admin, then the official non-deprecated method should work just fine, this is more of a.. I'm not exactly sure about it approach
    // This is similar to how we create the project in ProjectService, but here we can still use deprecator method, in ProjectService we just can't
    projectRoleService.addActorsToProjectRole(ADMIN_SERVICE_ACCOUNT, userkeys, projectRole, project, ProjectRoleActor.USER_ROLE_ACTOR_TYPE, simpleErrorCollection);

    if (simpleErrorCollection.hasAnyErrors()) {
        log.error(simpleErrorCollection.getErrorMessages().toString())
    }
}

[Peter Brodt]

Hi Radek,

my code for creating the project is almost the same. The only difference I detected is that I used the project id instead of the key of the project template. Doesn't explain though why my code worked fine in our test environment and failed in production.

Anyway I adapted a couple of changes from your code and first tests were successful in both environments. I can't give a final statement yet because I am out of office right now. I will continue testing next week and give you a feedback.

Thanks again for your tremendous help and support. 

Peter

[RAJESH]

Hi Peter Brodt,

If you have time, please look on below query..

I. Can we create automatically when issue created for project creation with dynamic following values??

1. Project Name      2. Project Key     3.projectTypeKey(business/Software)

4. project Template Key         5. Description        6.  lead         7. url  

8. Assignee Type     9. Issue Security Scheme     10. permission Scheme

II. When I am using  below RESTAPI code through postman, the project creation is executing successfully.  How can Def same value groovy script.

{
"key": "RESTAPI",
"name": "RESTAPI",
"projectTypeKey": "business",
"projectTemplateKey": "com.atlassian.jira-core-project-templates:jira-core-project-management",
"description": "Example Project description",
"lead": "admin",
"url": "http://10.23.73.94:8080",
"assigneeType": "PROJECT_LEAD",
"issueSecurityScheme": 10000,
"permissionScheme": 10000,
"notificationScheme": 10000

}

Thanks in Advance!!!!!

Rajesh Kamistti

[Peter Brodt · Accepted solution]

the json looks like this:

{"key":"RYMG","name":"Rest Ye Merry Gentlemen","lead":"myID"}