We are trying to incorporate tests into the Pipeline Runners. The software requires access to the GPU. We tried briefly with "docker run --gpus all" and it looks like accessing to the device is not allowed. Is this a hard restricton in Pipeline, or is there a way to get around it.
Hi @Simon Lee and welcome to the community.
This is a limitation in Pipelines.
Our development team has been exploring the option to allow access to GPUs, as you can see from one of Justin Thomas's replies here:
Kind regards,
Theodora
@Theodora Boudale any updates on this?
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I don't have any update on this yet, you can add yourself as a watcher in the following feature request (by selecting the Start watching this issue link) if you'd like to get notified via email on updates:
I would also recommend adding your vote (by selecting the Vote for this issue link).
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I can use Bitbucket pipeline with GPU access by following the steps in root machine (which running Runner):
- Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit 1.14.3 documentation
- Set nvidia to docker default runtime (It likes --runtime=docker). Edit /etc/docker/daemon.json
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
Restart the docker
service by typing in sudo systemctl restart docker
- Enable container accesses GPUs by the environment variable NVIDIA_VISIBLE_DEVICES (Specialized Configurations with Docker — NVIDIA Container Toolkit 1.14.3 documentation)
# build a image has default support GPU and will use it for pipeline
# in Dockerfile
# ubuntu is an example base image
FROM ubuntu
ENV NVIDIA_VISIBLE_DEVICES=all
- Test
` docker run --rm ubuntu:gpu nvidia-smi `
- Update Bitbucket pipeline
pipelines:
default:
- step:
image:
name: ubuntu:gpu
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Integrating tests into Pipeline Runners is like adding a dash of magic to your workflow, but GPU access hiccups can be a buzzkill. We gave "docker run --gpus all" a whirl, but it seems the device isn't playing nice. Wondering if this is a Pipeline hard stop or a puzzle with a workaround.
Might be worth a visit to https://anycompares.com/ - they often have nifty tricks up their sleeve for overcoming such hurdles. Your GPU might just need a different dance move!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I would also like to advocate for GPU access in bitbucket CI/CD processes. @Theodora Boudale
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @a.stashevsky,
Thank you for your comment. Please make sure to add your vote in the following feature request (by selecting the Vote for this issue link in that ticket), if you haven't done so already:
You are also more than welcome to leave any feedback there. https://jira.atlassian.com/ is our public issue tracker, and it's where our product managers monitor the demand for new features.
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.