Unable to create artifact from SSH task result

[Oleg Rybchenko]

I use SSH task to launch an application in Docker (I don't use Docker task because there's a proxy issue with this app and the only way to start it is to launch directly from server over SSH).

This application has it's working directory where .html reports appear. This working directory is attached to certain directory on sever: /app_job/reports
The absolute path is: /home/adm/app_job/reports

In my SSH task logs I see:
Build working directory is /opt/atlassian/bamboo/data/local-working-dir/136856233/PROJ-PLAN-JOB1

So I configure Artifacts for my JOB1 this way:
Name: app-job-report
Location: /app_job/reports
Copy pattern: *.html
Build dependency: REQUIRED

But the result is:
Unable to publish artifact [app-job-report]: the source directory /opt/atlassian/bamboo/data/local-working-dir/136856233/PROJ-PLAN-JOB1/app_job/reports does not exist.

How can I specify the correct folder to get the artifact? What am I doing wrong?

[Eduardo Alvarenga · Accepted solution]

Hello @Oleg Rybchenko ,

Welcome to Atlassian Community!

If you are running the docker container from a Script task, you can simplify the mount location invocation by running $(pwd) -- it will expand to the current folder:

In the Script Task:

$ mkdir -p app_jobs/report
$ docker run -v ./app_jobs/report:/home/adm/app_job/reports other_options ... container_name

So for example, if the build runs on /opt/atlassian/bamboo/data/local-working-dir/136856233/PROJ-PLAN-JOB1 and the artifact is saved on <build_dir>/app_job/report, configure the Job artifact as:

Name: app-job-report
Location: app_job/reports    <--- NOTICE no forward-slash (/) before the location
Copy pattern: *.html
Build dependency: REQUIRED

EDIT: As you have mentioned that you are running this over an SSH Task, you will also have to copy the artifacts back to the Bamboo Server to the Plan's build directory after they are produced so the Job's Artifact definition will be able to locate it. Ideally, you should have a Remote Agent that would run the docker operation locally and have access to the artifacts without making you have to use SSH and SCP tasks.

Sincerely,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful-- 

[Oleg Rybchenko]

Hello @Eduardo Alvarenga !
Thanks for your answer.

When I launch container the directory is already mounted as $(pwd)/app/wrk and output reports path is $(pwd)/app/wrk/app_job/reports.
This refers to absolute path on server home/adm/app_job/reports when container finish it's work.

SSH Task starts with:
cd app_jobs
docker run -v $(pwd)/app/wrk ....

So I thing the bamboo build directory absolute path is home/adm
and I can get any artifacts only from home/adm/app_job/reports because bamboo use SSH Task that collect artifacts as soon as this Task is finished (container work is finished by this time)

This fully relates to the case you've mentioned above:

Name: app-job-report
Location: app_job/reports  (without forward slash)
Copy pattern: *.html
Build dependency: REQUIRED

But in fact I still receive:
Unable to publish artifact [app-job-report]: the source directory /opt/atlassian/bamboo/data/local-working-dir/136856233/PROJ-PLAN-JOB1/app_job/reports does not exist.

[Eduardo Alvarenga]

Hello @Oleg Rybchenko

You are missing an essential piece of the Bamboo architecture here. You are running a command over an SSH task. Bamboo will expect that command to succeed and that's it. Bamboo's build context has no idea about the output/content generated by that command executed over SSH on the remote server.

To have access to the artifacts, you need to copy the files created on the remote SSH host back to your Bamboo Server. From the provided build_dir path I see you are running a Local Agent.

Then, once Bamboo can read the files on /opt/atlassian/bamboo/data/local-working-dir/136856233/PROJ-PLAN-JOB1/app_job/reports it will then be able to handle the artifacts correctly.

How to make the files available from the remote SSH host back to Bamboo will be up to you, I can think about things such as:

• An SSH task that will connect to the remote SSH server and run an "scp", "ftp", "rsync", etc, to copy the files back to the Bamboo server
• Share an NFS folder between Bamboo and the Remote SSH server and make Bamboo copy the generated files to its build_dir locally
• Install a remote Agent and run the docker builds from it. You can even install the agent on the same machine that you are SSHing into. This way the artifacts will be available immediately

Cheers,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful-- 

[Oleg Rybchenko]

Thanks a lot! I'll try this and get back with result.