Repository stored Java Specs (RSS) cannot updated the plan if it uses BitBucketServerRepository()

[Evgeny Prigorodov]

Bamboo server version 6.2.2

We are testing automatic publishing of Java Specs upon push into Bitbucket repository, a simple plan created with Repository Stored Specs tutorial.

It works fine when its checkout task uses GitRepository():

VcsRepository gitRepository() {
    return new GitRepository() 
        .name("Bitbucket/Project/repository") 
        .url("https://bitbucket.domain.tld/scm/prj/repository.git") 
        .authentication(new SharedCredentialsIdentifier("bamboo")) 
        .branch("master"); 
}

Spec update fails if the checkout task uses BitBucketServerRepository() configured via BitBucket Application Link:

VcsRepository bitBucketRepository() {
    return new BitbucketServerRepository()
        .name("Bitbucket/Project/repository")
        .server(new ApplicationLink().name("COMPANY BitBucket Production"))
        .projectKey("PRJ")
        .repositorySlug("repository")
        .sshPublicKey("<public SSH key here")
        .sshPrivateKey("<private SSH key here")
        .branch("master");
}

All checks in the troubleshooting guide are green.
The SSH keys were copied from Plan / Configure / View plan as Bamboo Specs and public key has been added to Bitbucket repository settings.

The very same Java specs with BitbucketServerRepository() works fine when being published manually from the developer workstation via `mvn -Ppublish-specs`.

Only automatic build and execution on the server side fails.

The Bamboo sends an error email which refers to the following update log:

... top part of successful build skipped ...
[INFO] --- bamboo-specs-runner:6.2.2:run (default-cli) @ bamboo-specs-generator ---
[INFO] Scanning /opt/bamboo/xml-data/build-dir/serverSide/REPOSITORY_STORED_SPECS/repository-5767172/checkout/bamboo-specs/target/classes for classes annotated with Bamboo plan annotation.
[INFO] Found @BambooSpec annotation on class com.my.company.PlanSpec
[INFO] Running class com.my.company.PlanSpec
2019-09-17 15:05:37,744 INFO [BambooServer] Publishing plan TEST-RPC
2019-09-17 15:05:37,884 INFO [BambooServer] Writing specs into /opt/bamboo/xml-data/build-dir/serverSide/REPOSITORY_STORED_SPECS/repository-5767172/output/plan/00000-plan-TEST-RPC.yaml
2019-09-17 15:05:37,896 INFO [BambooServer] Publishing plan permission for plan PlanIdentifier{projectKey=TEST, key=RPC, oid=null}
2019-09-17 15:05:37,912 INFO [BambooServer] Writing specs into /opt/bamboo/xml-data/build-dir/serverSide/REPOSITORY_STORED_SPECS/repository-5767172/output/plan-permission/00001-plan-permission-for-plan-PlanIdentifier-projectKey-TEST--key-RPC--oid-null-.yaml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.143 s
[INFO] Finished at: 2019-09-17T15:05:37+02:00
[INFO] Final Memory: 27M/290M
[INFO] ------------------------------------------------------------------------
You are not permitted to access this resource
com.atlassian.stash.rest.client.api.StashUnauthorizedRestException: You are not permitted to access this resource
at com.atlassian.stash.rest.client.core.StashClientImpl.createStashRestException(StashClientImpl.java:669)
at com.atlassian.stash.rest.client.core.StashClientImpl.lambda$doRestCall$10(StashClientImpl.java:655)
at com.atlassian.stash.rest.client.applinks.ApplinkHttpExecutor$StashApplinkResponseHandler.handle(ApplinkHttpExecutor.java:88)
at com.atlassian.applinks.oauth.auth.OAuthApplinksResponseHandler$1.handle(OAuthApplinksResponseHandler.java:120)
at com.atlassian.applinks.oauth.auth.OAuthApplinksResponseHandler.handle(OAuthApplinksResponseHandler.java:76)
at com.atlassian.plugins.rest.module.jersey.JerseyRequest$2.handle(JerseyRequest.java:134)
at com.atlassian.sal.core.net.HttpClientRequest.executeAndReturn(HttpClientRequest.java:104)
at com.atlassian.plugins.rest.module.jersey.JerseyRequest.executeAndReturn(JerseyRequest.java:131)
at com.atlassian.applinks.core.auth.ApplicationLinkRequestAdaptor.execute(ApplicationLinkRequestAdaptor.java:58)
at com.atlassian.applinks.oauth.auth.OAuthRequest.execute(OAuthRequest.java:58)
at com.atlassian.stash.rest.client.applinks.ApplinkHttpExecutor.execute(ApplinkHttpExecutor.java:55)
at com.atlassian.stash.rest.client.core.StashClientImpl.doRestCall(StashClientImpl.java:619)
at com.atlassian.stash.rest.client.core.StashClientImpl.doRestCall(StashClientImpl.java:606)
at com.atlassian.stash.rest.client.core.StashClientImpl.getRepository(StashClientImpl.java:156)
at com.atlassian.bamboo.plugins.stash.v2.exporter.BitbucketServerExporter.importLocationData(BitbucketServerExporter.java:217)
at com.atlassian.bamboo.plugins.stash.v2.exporter.BitbucketServerExporter.importLocationData(BitbucketServerExporter.java:67)
at com.atlassian.bamboo.configuration.external.exporters.RepositoryExporterImpl.importRepository(RepositoryExporterImpl.java:257)
at com.atlassian.bamboo.configuration.external.PlanConfigImportServiceImpl.lambda$validateAndCreateRepositoryObjects$15(PlanConfigImportServiceImpl.java:918)
at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:196)
at com.atlassian.bamboo.configuration.external.PlanConfigImportServiceImpl.validateAndCreateRepositoryObjects(PlanConfigImportServiceImpl.java:913)
at com.atlassian.bamboo.configuration.external.PlanConfigImportServiceImpl.lambda$modifyTopLevelPlanFunction$5(PlanConfigImportServiceImpl.java:529)
at com.atlassian.bamboo.core.ScopedExclusionServiceImpl.tryWithLock(ScopedExclusionServiceImpl.java:74)
at com.atlassian.bamboo.configuration.external.PlanConfigImportServiceImpl.modifyExistingTopLevelPlan(PlanConfigImportServiceImpl.java:421)
at com.atlassian.bamboo.configuration.external.PlanConfigImportServiceImpl.lambda$importPlan$0(PlanConfigImportServiceImpl.java:327)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
... rest of exception traceback skipped ...

The Bitbucket access log contains a 401 response:

<BAMBOO_IP>:58668,<PROXY_IP> | https | o@1AX074Cx905x49880x0 | - | 2019-09-17 15:05:38,333 | "GET /rest/api/1.0/projects/PRJ/repos/repository HTTP/1.1" | "" "Apache-HttpClient/4.5.3 (Java/1.8.0_151)" | 401 | 0 | 152 | - | 6 | - |

We have captured network dumps with REST requests sent from Bamboo to Bitbucket (full dumps are available on request).

In the case of server-side update Bamboo sends the following request and BitBucket responds with 401:

GET /rest/api/1.0/projects/PRJ/repos/repository?xoauth_requestor_id=SYSTEM HTTP/1.1

In case of manual publishing via `mvn -Ppublish-specs` the Bamboo sends impersonated request and Bitbucket returns 200 with JSON body:

GET /rest/api/1.0/projects/PRJ/repos/repository?xoauth_requestor_id=<developer_user_name> HTTP/1.1

Is it a bug or do we lack some permissions to be granted on the Bitbucket side?

Thank you.

UPDATE: creating a Bitbucket user with the name SYSTEM and granting it access to referred repository fixes the problem. This is acceptable workaround for us, but how was this supposed to work without hacks?

[Rafael Pinto Sperafico]

Hi @Evgeny Prigorodov ,

Thank you for raising this inquire and for all the troubleshooting and workaround you have suggested.

Please, notice there were some issues in Bamboo Java Specs related to repositories that have been addressed in a later Bamboo version than the one you are running, here is an example:

• https://jira.atlassian.com/browse/BAM-20012 - Cannot set up custom SSH keys in Bamboo Java Specs

When calling Bamboo Specs from within Bamboo, the communication between application happens through the user SYSTEM (an internal user used for events by the application link), therefore, the workaround you have suggested would be a way to surpass that, however inconvenient.

If possible, you could set up repositories in Bamboo as Linked Repositories (beforehand) and refer to them in your Bamboo Java Specs, e.g:

https://docs.atlassian.com/bamboo-specs-docs/6.2.2/#bitbucket-server-git-repository

Plan plan = new Plan(project, planName, planKey)
  .planRepositories(
    new BitbucketServerRepository()
      .name("my-bitbucket-repository")
      .server(new ApplicationLink()
        .name("bitbucket-server"))
      .projectKey("BBSPROJECT")
      .repositorySlug("my-repository-slug")
      .branch("master")
  );

Therefore, it would be best if you could upgrade Bamboo to a latest version, where issues related to Bamboo Java Specs and improvements on Bamboo YAML Specs have been made.

For instance, Bamboo v6.9 comes with Bamboo YAML Specs v2.0 as mentioned in Bamboo Release Notes (https://confluence.atlassian.com/bamboo/bamboo-6-9-release-notes-969521942.html). Even though, what you are trying to accomplish still unavailable in Bamboo as YAML, you will definitely enjoy the improvements made in Bamboo Java Specs.

Kind regards,
Rafael