Unit testing in pipelines when reportfile generated inside of container

Question

We trying to use pipeline reporting, the reportfile is generated inside of container.
Also we using aws. Command:

aws ssm send-command --document-name "AWS-RunShellScript"
--document-version "1" --targets '[{"Key":"tag:Name","Values":["dummy_app"]}]'
--parameters '{"workingDirectory":["/home/user/directory"],"executionTimeout":["3600"],"commands":[
"docker exec $(docker ps -q --filter name=app_1) phpunit --log-junit ./test-reports/output.xml",""
]}'
--timeout-seconds 600 --max-concurrency "50" --max-errors "0" --region eu-west-1

But the output is empty

How is possible to make it work?

Alex_il48ya · Answer

Thanks for your response.
Yes, output.xml contain failures. 
The image for command is: atlassian/pipelines-awscli:latest

Our test are data oriented , which means we use the target machine database data, to test functionalities. So those test need to run on the target machine. Our production machine and staging is on aws, we are using aws cli image to make the deployment on the new code updates on different ec2 instances.

We try to use the same logic on the testing, using aws run command service. Which indeed run the commands phpunit on target container, but don't receive the result output for bitbucket.

ktomk · Answer

The message you see in "Build teardown" is for a specific feature of Atlassian Bitbucket Pipelines Add-On, see Test Reporting in Pipelines.

As you have not shared in your question which image is in use in the pipeline you ask about, there is more guesswork to actually answer your question.

According to the referenced feature (or use case), I would first of all say, you can ignore the message. The pipeline should not become red as no test reports are found and therefore no failing tests should be reported.

If you are however concerned that this is not the case (e.g. you know about failing tests and have expected Bitbucket Pipelines to paint the build red), then please see the Configuring test reporting section in the hypertext document referenced above where to place/name the junit file to make it compatible.

If I see it right, you're running an additional docker container - correct me if I'm wrong - it might need additional configuration to have the xml file transparently placed into the pipelines container before the pipeline step ends (this is where the "Build teardown" kicks in).

I might be a bit far-off here as I do not know the `aws ssm` specifically and I do not have any understanding where the docker command it has in it's parameters is being executed. If it's not within the infrastructure of the Atlassian Bitbucket Pipelines Add-On, it would be more far-away and therefore I would say there is more to do to integrate the test report back into the pipeline step build results (e.g. this runs in the AWS account of your own).

At the end of the day, if the xml report file is placed in the pipelines container (with the right name and below the clone directory), build teardown will find it.

Feel free to comment with your questions. Depending on what you're trying to achieve, passing the file in the end (whether it be from the pipeline to your AWS account or the other direction) should do it. Also if you don't need to, please write as well.