windows Clean working directory task configuration fails randomly

[timothy schmidt]

I seem to get random failures when running the first TASK of the plan that is supposed to clean up whatever is in the current REMOTE AGENT's working directory, which is an artifact directory with two multi level subdirectories with very few files, and only two of which are of any real size (7.5 Mb each).

When I look at the working directory after the failure, everything but the 2 top level directories has been removed, so it almost looks like a timing issue of some sort where there still is a file system lock at the time the TASK checks to see if it's been completed.

If/when the plan is re-run, the removal succeeds/completes.

Has anyone run into this type of issue with windows BAMBOO?

I suppose I could remove them myself, but I should think the plugin task would be able to handle this.

Thanks!

T.S.

SAMPLE ERROR LOG for reference :

...

22-Jan-2019 12:17:46 Build working directory is D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1
22-Jan-2019 12:17:46 Executing build RETOOLS - TOOLS TEST - Default Job #151 (RET-TT-JOB1-151)
22-Jan-2019 12:17:46 Starting task 'clean' of type 'com.atlassian.bamboo.plugins.bamboo-artifact-downloader-plugin:cleanWorkingDirectoryTask'
22-Jan-2019 12:17:46 Cleaning working directory 'D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1'
22-Jan-2019 12:17:46 Unable to clean working directory 'D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1' D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1\testartifacts
22-Jan-2019 12:17:46 Error occurred while running Task 'clean(6)' of type com.atlassian.bamboo.plugins.bamboo-artifact-downloader-plugin:cleanWorkingDirectoryTask.
22-Jan-2019 12:17:46 com.atlassian.bamboo.task.TaskException: Unable to clean working directory 'D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1' D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1\testartifacts
22-Jan-2019 12:17:46 at com.atlassian.bamboo.plugins.artifact.CleanWorkingDirectoryTask.cleanWorkingDir(CleanWorkingDirectoryTask.java:52)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.plugins.artifact.CleanWorkingDirectoryTask.execute(CleanWorkingDirectoryTask.java:26)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.task.TaskExecutorImpl.lambda$executeTasks$3(TaskExecutorImpl.java:319)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.task.TaskExecutorImpl.executeTaskWithPrePostActions(TaskExecutorImpl.java:252)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:319)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.task.TaskExecutorImpl.executePreparationTasks(TaskExecutorImpl.java:91)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:81)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.executeBuildPhase(DefaultBuildAgent.java:203)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:175)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.lambda$waitAndPerformBuild$0(BuildAgentControllerImpl.java:129)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:185)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:123)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:126)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
22-Jan-2019 12:17:46 at java.lang.Thread.run(Unknown Source)
22-Jan-2019 12:17:46 Caused by: java.nio.file.DirectoryNotEmptyException: D:\BAGENTS\AGENT3\xml-data\build-dir\RET-TT-JOB1\testartifacts
22-Jan-2019 12:17:46 at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
22-Jan-2019 12:17:46 at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
22-Jan-2019 12:17:46 at java.nio.file.Files.delete(Unknown Source)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooFiles.delete(BambooFiles.java:43)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooPathUtils.deleteNoThrow(BambooPathUtils.java:162)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooPathUtils.access$000(BambooPathUtils.java:38)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooPathUtils$1.postVisitDirectory(BambooPathUtils.java:121)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooPathUtils$1.postVisitDirectory(BambooPathUtils.java:91)
22-Jan-2019 12:17:46 at java.nio.file.Files.walkFileTree(Unknown Source)
22-Jan-2019 12:17:46 at java.nio.file.Files.walkFileTree(Unknown Source)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.utils.BambooPathUtils.cleanDirectory(BambooPathUtils.java:80)
22-Jan-2019 12:17:46 at com.atlassian.bamboo.plugins.artifact.CleanWorkingDirectoryTask.cleanWorkingDir(CleanWorkingDirectoryTask.java:48)
22-Jan-2019 12:17:46 ... 17 more
22-Jan-2019 12:17:46 Failed to prepare the build 'RETOOLS - TOOLS TEST - Default Job #151 (RET-TT-JOB1-151)'
22-Jan-2019 12:17:46 Running on server: post build plugin 'NCover Results Collector'
22-Jan-2019 12:17:46 Running on server: post build plugin 'Build Hanging Detection Configuration'
22-Jan-2019 12:17:46 Running on server: post build plugin 'Clover Delta Calculator'
22-Jan-2019 12:17:46 Running on server: post build plugin 'Maven Dependencies Postprocessor'
22-Jan-2019 12:17:46 All post build plugins have finished
22-Jan-2019 12:17:46 Generating build results summary...
22-Jan-2019 12:17:46 Saving build results to disk...
22-Jan-2019 12:17:46 Logging substituted variables...

[Daniel Santos]

Hi @timothy schmidt,

I saw a case where a build was calling an external tool that was not properly closed after finishing the build. The app was locking a file and it was breaking the next build right at the beginning, in the build working directory task.

• Did you try to manually remove the remaining folders when this error happened?
• Did you check the file/folder permissions from files left there?
• Are you calling any external tool that could potentially lock files from your build?

[timothy schmidt]

Manual removal works fine.

Re-running the plan cleans it up too.

File permissions are identical each run.

No external tools are "sitting in" dirs or "looking" at files. I'm too familiar with file/dir locks :)

I updated each hosts' much older JDK8 version to the latest update and restarted both server & agents.

So far the problems haven't shown up again. Not saying this was involved, but it's the only thing that I'm aware of that I changed.

[Daniel Santos]

Thank you for sharing. 

I'm afraid we will need to open a support ticket to investigate this one further. Do you want me to open a support ticket for you or should we wait to check if this problem will happen again?

I'm suggesting a support ticket so we can evaluate with more details your server and agent logs. We could also collect more details to try reproducing the same scenario.

[timothy schmidt]

Thanks Daniel,

Given that I've updated all the servers and the problem as not re-appeared with the same data set, I'm not sure this would be a useful endeavor.

But I appreciate the offer very much.

If this issue pops up again I'll take you up on it!

Tim

[Daniel Santos]

Thank you @timothy schmidt!

It makes sense. Let's see how the system will behave.
I hope the problem is solved already.

Have a good one!