Pipelines - Failing to pull and unpack docker image (rpc error)

Leonardo Gardel Valverde August 22, 2019

We have been using the same version of a docker hub repository as the image of our bitbucket pipelines for 1 year. Suddenly, it begun failing and we don't know the reason.

The code below is at the top of the bitbucket-pipelines.yml file

image:
name: juryvox/juryvox-react-test
username: $DOCKER_HUB_USERNAME
password: $DOCKER_HUB_PASSWORD
email: $DOCKER_HUB_EMAIL

 

The pipelines fails right after its beginning, with the following error.

 

rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/juryvox/juryvox-react-test:latest": failed to copy: httpReaderSeeker: failed open: unexpected status code http://localhost:5000/v2/juryvox/juryvox-react-test/blobs/sha256:80e2dc61b67de22acb940d4bea02b2cd973abc3e7fd2bfd2fe1a486ce2dfde6f: 500 Internal Server Error

 

I have already checked if I am able to pull the repository from dockerhub using the same username and password that are set as bitbucket's repository variables, and it worked. I don't understand why it became a problem without any change neither at the container repository, nor at the bitbucket-pipelines.yml.

 

What could be causing this error?

13 answers

2 votes
Patryk Kwiatkowski April 14, 2021

similar issue since today:

rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/library/maven:3.3.3-jdk-7": failed to unpack image on snapshotter overlayfs: failed to extract layer sha256:42755cf4ee95900a105b4e33452e787026ecdefffcc1992f961aa286dc3f7f95: failed to get reader from content store: content digest sha256:5c90d4a2d1a8dfffd05ff2dd659923f0ca2d843b5e45d030e17abbcd06a11b5b: not found

Patryk Kwiatkowski April 14, 2021

In our case we had to change docker image for maven from 3.3.3-jdk-7 to 3.3.9

2 votes
Antonio Jerez February 10, 2020

Hi there.

We are consistently experiencing this issue too, last time was a 7 hours window we couldn't deploy using any dockerhub image, but it randomly pops here and there...

Daniel Silva Reis August 7, 2020

Hello Antonio!

 

We are experiencing the same issue. Have you found any solution?

2 votes
Robert_Goldsmith November 13, 2019

This happens periodically for us and we have to wait about 10mins and then manually restart the pipeline. It's really annoying.

1 vote
Johnathan Mayke Melo Neto April 1, 2021

We're also experiencing this issue.

1 vote
Gerwin van de Steeg March 23, 2020

A similar problem just started intermittently happening for us using an AWS ECR as the backend.  Doesn't appear to be resolved.

1 vote
Michel Moreau February 19, 2020

It's been a 6 days window here... python:3.7.2 image

0 votes
Vinicius Valente April 14, 2021

I managed to solve it by changing the python version from 3.5.1 to 3.7.2

0 votes
Rohan Gore January 15, 2021

I am also facing the same issue since yesterday, though there are a couple of behaviours different in my case

1. Could not recall this happening before, have been running the same private image from Docker Hub for more than a year at least across different BB repos.

2. Restarting the pipeline does not make it work, and it is failing in all of these repos.

Note: Docker Hub Credentials are not changed. Able to login to Docker Hub using the same creds. Also used another pair of creds, still the same error!!

Rohan Gore January 17, 2021

In my case,  it was happening as we were using the legacy Docker Hub plan. The moment we switched to the new team license, everything started working.

0 votes
Matt Cavanagh January 14, 2021

We're also experiencing this issue periodically. It appears you also can't reliably re-run the pipeline for a period when this issue occurs.

Errors are similar to above, except this is one of Atlassian's docker images.

0 votes
nicolas_nitche December 24, 2020

I am getting a similar error:

rpc error: code = Unknown desc = failed to pull and unpack image "company.azurecr.io/custom-image:latest": failed to copy: read tcp XX.XX.XX.XX:XXXX->XX.XX.XX.XX:XXX: read: connection timed out


No solution has been found yet. Like Anton, I need to restart the pipeline when the error occur, and it works. 
Does anyone found a clean solution on that ?

0 votes
Anton Daneyko October 9, 2020

I am getting a similar error:

```

rpc error: code = Unknown desc = failed to pull and unpack image "eu.gcr.io/lofty-seer-161814/othoz-docker-base:4.0.0": failed to resolve reference "eu.gcr.io/lofty-seer-161814/othoz-docker-base:4.0.0": failed to authorize: failed to fetch oauth token: Get https://eu.gcr.io/v2/token?scope=repository%3Alofty-seer-161814%2Fothoz-docker-base%3Apull&service=eu.gcr.io: dial tcp 173.194.66.82:443: i/o timeout

```

it happens sporadically.  A restart goes OK once in a while.

0 votes
Le Phuong October 22, 2019

@Graham Gatus We also see the same issue intermittently. A re-run would fix it at the moment but we hope this would be resolved completely.

0 votes
Graham Gatus
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 29, 2019

@Leonardo Gardel Valverde are you still experiencing this issue? If so can you please raise a support case via https://support.atlassian.com so we can investigate further.

Leonardo Gardel Valverde August 30, 2019

I had to make my docker image public to make it work. I think there is some bug with the communication between the repository variables and the pipelines instance. The code I posted worked for a long time, until I update my docker hub password. Then, I updated the repository variables with the new value and it stopped working, and the problem was not the password, because I copied pasted it on docker hub and it worked.

I hope my problem may help your team to find the bug, but I'm no longer interested in the answer.

Sorry for posting my problem at the wrong place.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events