Important! Windows Runner build can have a shorter path?

Valeriu Ghisoiu September 11, 2024

Hello Atlassian,

 

Can we overcome the long name folders where the build is cloned from the remote repo in the CI/CD pipelines? It makes my paths of the deepest files to have 259 paths in Windows.
The maximum is 260 (even if I enable the long paths is not working) (Windows limitations).

eg: C:\CICD\tmp\{runner-uuid}\8547119988257\build

 

Can we at least make the folders names shorter??? (I can use only 7 characters CICD\tmp :(. This is not ideal.

 

Need BIG HELP on this, I can't rename the files in submodules.

 

Warm regards,

Valeriu

1 answer

0 votes
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 13, 2024

Hi Valeriu,

The part after C:\CICD\tmp in the path is the UUID of the runner (just a heads up, I edited it in your post for privacy reasons). The part 8547119988257 is based on the EPOCH time when the build starts. These parts cannot be edited, however there are ways to resolve the issue.

You can configure Git on your Windows machine to allow paths of 4096 characters. You can do this by running the following command from Git Bash or another terminal application:

git config --global core.longpaths true

This config will apply to the currently logged in Windows user, since it has the --global flag.

Another option is to change the default 260 character limit in the Windows registry. You can read more info on how to do this here:

This knowledge base article is about Bamboo, but it concerns the same issue, so the info there should help with the issue on Bitbucket's Windows runner.

Please feel free to let me know if this helps or if you're still facing issues.

Kind regards,
Theodora

Valeriu Ghisoiu September 17, 2024

Hello Theodora,

 

Thank you very much for all the help and suggestions, and also for editing the sensitive data.

Luckily or Unfortunately everything that you proposed I already tried.

This is why I ended up here as a "last hope".

 

I added "git config --global core.longpaths true" in my yaml file
I changed the windows registry to use long paths and all solutions searched.

 

Unfortunately I found out that Microsoft Windows solved in 2024 only half of the issue, because they didn't update the API for File Manager (file explorer) to really use the long paths, so it is still limited to 260.
This means that powershell, cmd are using maximum 260.
This is the reason why I asked Atlassian if there is any possibility to shorten the path of the runner temporary file :(.

I only have 7 characters left up to 260 and is not enough.

Maybe there is a possibility that the runner is using something like Total Commander instead of Windows file Manager and that will solve the issue, because TC has the updated API of windows with longpaths.


I hope and wish so much to help me with this matter and also I think it is very important for Atlassian if can offer a help or work-around on this matter it can win a lot in front of other competitors.

Looking forward to your answer.

Kind regards,
Valeriu

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 18, 2024

Hi Valeriu,

I have tested the solution with

git config --global core.longpaths true

and it works well for me as long as the path is less than 4096 characters.

 

I would like to ask a few questions:

1. Is the longest path of your files less than 4096 characters or is it longer?

2. If you log in to this Windows machine with the Windows user you use to log in to start the runner, can you check the file .gitconfig in the home directory of this user? Does this file contain an entry like the following?

[core]
    longpaths = true

If not, can you execute the command git config --global core.longpaths true on PowerShell, outside of the runner?

Then ensure that the entry is in the .gitconfig file, and run another build. Do you see the same error that the filename is too long?

3. What version of Git and PowerShell are you using on this machine, and what is the version of Windows?

Kind regards,
Theodora

Valeriu Ghisoiu September 18, 2024

Hi Theodora,

 

Let's break it down.

1. Yes it is less, more exactly I would need 280-300 characters, I tried to minimize and it was working for 259 (but I don't want to use locations like C:\a\b\c That is impossible to work :(. I am working on windows 11 and 10, both didnt work

 

2. Yes it exists there already, (along side other core options or a path to OpenSSH - hope is not influencing). I ran the runner on 3 different computers with windows and enabled long paths in windows and git, none was working :((

3. 

- Git version 2.38.1.windows.1

- powershell 5 (the basic from windows 11/10)

- Windows 11 (and also tried on windows 10).

(Are the same versions for you too?)


Kind regards,

Valeriu

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 20, 2024

Hi Valeriu,

Thank you for the info.

I tested this on a Windows 10 machine, which happens to have the same version of Git and PowerShell as yours.

One additional thing to check is if you're using the latest version of the runner, which is 3.1.0 at the moment. If you do, I would suggest creating a ticket with the support team (since you have access to a workspace on a paid billing plan) for further investigation, so that we see if we can find a solution. The next thing I would want to check is your build logs, something that I cannot do without a support ticket.

You can create a ticket via https://support.atlassian.com/contact/#/, in "What can we help you with?" select "Technical issues and bugs" and then Bitbucket Cloud as product. When you are asked to provide the workspace URL, please make sure you enter the URL of the workspace that is on a paid billing plan to proceed with ticket creation.

Please provide in the support ticket description the URL of a failed build and mention that you have tried the core.longpaths setting in the git config and editing the Windows registry. The support ticket will be visible to you and Atlassian staff only, so anything you post there won't be publicly visible.

Please feel free to let me know if you have any questions.

Kind regards,
Theodora

Valeriu Ghisoiu September 20, 2024

Hi Theodora,

 

If there is not too long can you also break down again all the steps that you followed? Those 3 main ones I always followed on all my 3 computers and still the long path didn't worked even if everything was enable for long paths. Maybe I miss some small detail.

I am using the 3.1.0 runner, but I didn't adjust any setting to it, I use it basic as Bitbucket is suggesting me (the 4 commands).

 

Related to the ticket I will have to analyse, and then I come back. If it is private then maybe will be no issue, but I have to think. (There is no other alternative of private messages?)

 

Kind regards,

Valeriu

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 20, 2024

Hi Valeriu,

Of course. First, I created a repo and in it a folder with a very long name (and a file inside it) to reproduce the issue.

I created a Windows runner. I started PowerShell on my Windows machine as an Administrator (right click > Run as administrator), and then I used the 4 commands provided by Bitbucket to download and start the runner.

Then I set up Pipelines in the repo and ensured that the step of my pipeline would use the Windows runner. During the Build setup of my build I got the error "Filename too long".

Then, I opened a new PowerShell window on the Windows machine where the runner was running, and I ran the command

git config --global core.longpaths true

Afterwards, I double-checked the .gitconfig in the home directory of my Windows user to see that the following option was there

[core]
longpaths = true

Then, I reran the Pipelines build that failed with this error, and this time the Build setup completed successfully.


There is no option for private messaging, the support ticket is the only way we have for a conversation that is not public. If you don't want the support team to access your repo, you can say so in the ticket and provide any logs asked by support in the ticket. You can also request a video call with the team so that you have a screenshare and go through your configurations.

Kind regards,
Theodora

Valeriu Ghisoiu September 20, 2024

Hi Theodora,

 

Thank you for the brief of your steps.

It seems so simple, my repository has multiple submodules and those are having very long paths. 

I perform in my code script a "git submodule update --init --recursive" and then if I want to use the file in the very long path it says "file not found", or other cases "file read only", where again windows after 260 characters, the internet security lock the access to some files.

 

So basically I did the same as you, is just that my repo is more complex using submodules. And also needed to use and change the files in the long path.

 

Do you use in the yaml file any command that uses the \\?\C:\<folders> ? Because apparently to handle long paths in windows you have to use \\?\ prefix.

 

I like the possibility of a video call, but when I decide I will put also a ticket for support.

 

Kind regards,

Valeriu

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 23, 2024

Hi Valeriu,

Thank you for the info, I believe what you shared is important. Setting the git config option core.longpaths to true will prevent issues during a git clone. However, based on what you said, it looks like you are not receiving an error during clone in the Build Setup or during submodule update, but when running a different command. Is that correct?

My build is not doing anything with the files, I just executed some echo and dir commands. We'd need to know what command is failing to troubleshoot further. If you feel comfortable sharing here, I can look into it. Otherwise, you can share in a support ticket (if you'd like to create one).

Kind regards,
Theodora

Valeriu Ghisoiu September 23, 2024

Hi Theodora,

 

"based on what you said, it looks like you are not receiving an error during clone in the Build Setup or during submodule update, but when running a different command. Is that correct?" -> it is happening in both ways. This means that when the submodules are updated (in the pipe process) they can't be cloned/pulled in the long paths (which later I need to use them, but they are not there).

I think that the answer is Yes, because I run the command of "git module update" (init, recursive) using a py script, not in the yaml file directly (but I remember I tried also to update through the yaml and it still had issues by cloning everything but a couple of long path files. 

If I start a ticket will be a colleague of yours or you? I think it would be easier because you already know the problem :).

 

Kind regard,

Vale

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 23, 2024

Hi Valeriu,

It will be a colleague of mine, I don't work on support tickets. I can create the support ticket on your behalf if you like, so that I add a summary of what we have discussed here. Would you prefer that?

Kind regards,
Theodora

Valeriu Ghisoiu September 24, 2024

Hi Theodora,

 

If it is possible to do that it would be really kind and useful to include also a summary of what we have discussed yes, I would prefer that,

 

I very much appreciate it!

 

Kind regards,

Valeriu

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 25, 2024

Hi Valeriu,

Of course, I created a support ticket on your behalf. You should have received an email with a link to the ticket, and you can also see the ticket here (after you log in with the email address of your community account):

Please note:

  • It's important to know the exact commands you are executing in your script up until you get the error, and the output of these commands. If you don't feel comfortable sharing in the ticket the URL of a failed build with this issue, could you please post the commands from your yml file's script (if that command is a script, please share the script's commands) and the output in the build log?

    I have shared a summary of what we discussed in an internal comment in the ticket, but my colleague will also need the above info to investigate.

  • We provide 9/5 regional support. I created the ticket with a timezone based on your company's address that we have on file. You can see the ticket's timezone if you scroll down to the bottom, there is a field "Customer Timezone". If that doesn't correspond to your timezone, please let us know in a comment so that my colleague changes it and you receive support from a team closer to you.

Please feel free to reach out if you have any questions.

Kind regards,
Theodora

Valeriu Ghisoiu September 30, 2024

Hi Theodora,

 

I was away for the end of the last week.

Today and tomorrow I am on this matter and as soon as I analyse the situation and all the aspects that are needed, I will return to the ticket that you opened for me.

 

Thank you for your patience and keep in touch with your colleagues from support.

Kind regards,

Valeriu

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 30, 2024

Hi Valeriu,

Sure, that's not a problem. I let my colleague know and I see you also left a comment in the ticket.

Kind regards,
Theodora

Like Valeriu Ghisoiu likes this

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events