AWS IRSA for K8s based runner

[vitalii.kostenko]

Hi I'm trying setup kubernetes based runners using https://community.atlassian.com/t5/Bitbucket-Pipelines-Runner/gh-p/bitbucket-runner-autoscaler-4k8s

I've successfully assigned AWS IAM service role to runner pod (both runner and docker containers) so I can see all the AWS specific env vars and may use aws cli if I jump intpo runner container OR docker container in the runner pod, how ever non of these variables are visible in the nested container (which is responsible for pipeline steps execution) when I'm running pipeline.

Is there any chance that I may proxy AWS config from runner pod to nested container in any way ?

[vitalii.kostenko]

@Marcos Sampaio regarding autoscaler - it does not seems to be supporting Arm runners so far. Fix is trivial but API spec for runners endpoint is not public. Is there any chance we may work on this ?

[vitalii.kostenko]

so to add arm64 we just need to add a proper label into the request.
but some how it can be done via the GUI but can't be done via API

[vitalii.kostenko]

when I'm trying to add "linux.arm64" label I have 

Status code: 400. {"key": "agent-service.request.bad-request", "message": "Only one platform label must be provided.", "arguments": {}}

the reason is folloving default config

DEFAULT_LABELS = frozenset({'self.hosted', 'linux', AUTOSCALER_RUNNER})

these are default labels which are mixed with custom labels provided in the config. As a result API server is confused which one to apply - linux OR linux.arm64

it's probably better to rework this part. as a hotfix we may replace this value in place before container starts with SED

[vitalii.kostenko]

@Edmund Munday @Liam Nunns @Marcos Sampaio  @Raul Gomis  @Oleksandr Kyrdan  @Igor Stoyanov 
May be You have any ideas as a maintainers ?

[vitalii.kostenko]

In this question I'm referring  to the following AWS feature to https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html

[Marcos Sampaio]

hi @vitalii.kostenko ,

Passing these variables to the build container is not possible at the moment.

However, not sure if it fits your use case but one thing that you could consider using is Bitbucket Pipelines OIDC feature:

https://support.atlassian.com/bitbucket-cloud/docs/deploy-on-aws-using-bitbucket-pipelines-openid-connect/

[vitalii.kostenko]

@Marcos Sampaio  that's exactly answer to my question. I understand why such architecture was suggested by Atlassian but I don't think any SecOps engineer will approve adding a external OIDC which is not controlled by corporate IT. So it does not seems be good way for me to go.