Bitbucket runner ERROR (Token failed verification: expired)

[황다솔]

Runner image configuration

image:
  repository: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "1"

Runner matching labels:
  - {skip}
Runner name: xxx-ruuner-01
Runner UUID: {...}
Runner labels: {skip}

Runner version:
 current: 1.466
 latest: 1.470

Recently we move runners to EKS from EC2, however now our bitbucket pipelines we are getting the following errors:

[2023-05-23 05:05:19,384] Getting environment variables for step.
[2023-05-23 05:05:19,888] Getting all artifacts for step.
[2023-05-23 05:05:19,891] Getting SSH private key.
[2023-05-23 05:05:19,892] Getting known hosts.
[2023-05-23 05:05:20,227] SSH private key not found
[2023-05-23 05:05:20,231] Setting up directories.
[2023-05-23 05:05:20,231] Starting log uploader.
[2023-05-23 05:05:20,239] Removing container ...(skip)_clone
[2023-05-23 05:05:20,244] Removing container ...(skip)_build
[2023-05-23 05:05:20,252] Setting up step timeout: PT2H
[2023-05-23 05:05:20,252] Starting websocket listening to STEP_COMPLETED events.
[2023-05-23 05:05:20,253] Checking for step completion every PT30S seconds.
[2023-05-23 05:05:20,560] Updating step progress to PULLING_IMAGES.
[2023-05-23 05:05:20,886] Pulling image public.ecr.aws/o4g5y6x4/sox/atlassian/public/pipelines-dvcstools:prod-stable.
[2023-05-23 05:05:20,887] Looking for auth in config for image Image{name=public.ecr.aws/o4g5y6x4/sox/atlassian/public/pipelines-dvcstools:prod-stable, runAsUser=None, auth=None} and found auth null
[2023-05-23 05:05:21,240] Appending log line to main log.
[2023-05-23 05:05:22,428] Pulling image {Our private ECR}/bitbucket-pipeline:latest.
[2023-05-23 05:05:22,550] Pulling image public.ecr.aws/o4g5y6x4/sox/atlassian/public/pipelines-docker-daemon:v20.10.23-multiarch-prod-stable.
[2023-05-23 05:05:22,550] Looking for auth in config for image Image{name=public.ecr.aws/o4g5y6x4/sox/atlassian/public/pipelines-docker-daemon:v20.10.23-multiarch-prod-stable, runAsUser=None, auth=None} and found auth null
[2023-05-23 05:05:24,067] Pulling image public.ecr.aws/o4g5y6x4/sox/atlassian/public/pipelines-auth-proxy:prod-stable.
[2023-05-23 05:05:24,068] Looking for auth in config for image Image{name=public.ecr.aws/o4g5y6x4/sox/atlassian/public/pipelines-auth-proxy:prod-stable, runAsUser=None, auth=None} and found auth null
[2023-05-23 05:05:25,553] Pulling image k8s-docker.packages.atlassian.com/pause:3.8.
[2023-05-23 05:05:25,553] Looking for auth in config for image Image{name=k8s-docker.packages.atlassian.com/pause:3.8, runAsUser=None, auth=None} and found auth null
[2023-05-23 05:05:27,506] An error occurred whilst pulling image.

com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"unknown: Token failed verification: expired"}

 at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:103)

 at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:32)

 at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)

 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)

 at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)

 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)

 at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)

 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)

 at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)

 at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)

 at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)

 at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)

 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)

 at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)

 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)

 at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)

 at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)

 at io.netty.channel.epoll.EpollDomainSocketChannel$EpollDomainUnsafe.epollInReady(EpollDomainSocketChannel.java:138)

 at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)

 at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)

 at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)

 at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

 at java.base/java.lang.Thread.run(Unknown Source)

Seems the issue is in the pulling image from docker. (k8s-docker.packages.atlassian.com/pause:3.8 or my private ecr image) Unsure what it is.

Other runners with the same settings are working, But only specific runner(mgmt-runner-05) is not working 

What's wrong with this runner?