Self Hosted Runner | Logs Cleanup

[cloud_admin]

Hi Team,

I am managing a Linux shell self-hosted runner and constantly need to expand the allocated disk space. I'm struggling to pinpoint which files or folders can be regularly cleaned up to ensure that I can redeploy existing pipelines while removing unnecessary files generated during the build process.

Can you provide guidance on identifying and managing these files effectively?

[Theodora Boudale]

Hi @cloud_admin and welcome to the community!

If the runner directory on this machine is for example /home/ubuntu/atlassian-bitbucket-pipelines-runner, then the runner is going to create directories during every build inside

/home/ubuntu/atlassian-bitbucket-pipelines-runner/tmp/b609961f-891e-c872-c36b-f3f2c315d186/

where b609961f-891e-c872-c36b-f3f2c315d186 will be the UUID of your own runner. The runner will try to clean these directories after every build. If it fails, you can also delete the directories inside this location, if no build is running at that moment.

The runner.log file is located in this directory as well, but I see that after deleting it, it is not recreated automatically on the next build. If an error occurs with the runner, the first thing we need to look into is the runner log. So, if you delete this file and also close the terminal session where the runner runs, there will be no logs to investigate what is happening in case of an error. I will ask the development team whether there is a way to work around this issue and I will get back to you. In the meantime, I would recommend not deleting this file for now.

Since this is a Linux Shell runner, it is also possible that the commands from your bitbucket-pipelines.yml file's script generate files outside the directory I mentioned earlier. If that is the case, the runner is not going to delete these files. You will need to check your build's commands and your project's configuration to see if the build generates files outside the directory I mentioned.

Kind regards,
Theodora

[cloud_admin]

Hello Theodora,

Upon inspection of the mentioned folder, only log files were discovered. It seems these files are being cleaned as part of the build process. However, I am still uncertain about the reason for the continual increase in file size. Any files generated during the pipeline are solely copied to the BITBUCKET_CLONE_DIR directory.  

What additional cleanup measures can be undertaken?

[Theodora Boudale]

Hi @cloud_admin,

I'm not sure which files you are referring to by "the continual increase in file size".

Have you checked which files take up a lot of space on that server and increase in size?

Is it the runner.log file? If not, is it any other runner-related file, and if so, which one?

Kind regards,
Theodora