It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Bamboo build log not in sync

Graeme Kidd Jan 25, 2012

Hi,

One of our deployment scripts uses Python which Bamboo is able to run no problem. The issue comes when the Build Log doesn't seem to record the output in the same logical order specified in the Python script.

For example when the Python script is run on the command line you might expect to see the following output:

sending incremental file list
graphics/
images/
files/
support/

SERVICE_NAME: apache
TYPE               : 10  WIN32_OWN_PROCESS 
STATE              : 1  STOPPED
WIN32_EXIT_CODE    : 0  (0x0)
SERVICE_EXIT_CODE  : 0  (0x0)
CHECKPOINT         : 0x0
WAIT_HINT          : 0x0

Instead the Bamboo Build log displays the following log:

sending incremental file list
graphics/
imaSERVICE_NAME: apache
TYPE               : 10  WIN32_OWN_PROCESS 
STATE              : 1  STOPPED
WIN32_EXIT_CODE    : 0  (0x0)
SERVICE_EXIT_CODE  : 0  (0x0)
CHECKPOINT         : 0x0
WAIT_HINT          : 0x0
ges/
files/
support/

Does this imply that the commands are not working in the same logical order compared to the command prompt or is it just the way Java reads the output stream?

Either way is it possible to improve the output of the log as knowing the sequence of events is an important tool in the debugging stage.

Thanks

1 answer

1 accepted

1 vote
Answer accepted
Przemek Bruski Atlassian Team Jan 25, 2012

Your script probably sends some of the output on stdout and the rest on stderr stream. These streams are separate, one is buffered and the other one is not. The fact that you are seeing them in certain order when you run the script on command line cannot be depended upon.

You can get the same result on command line and in Bamboo if you make the script output everything on stdout - ideally directly in the script, but if you can't, redirect the output.

Graeme Kidd Jan 25, 2012

Thanks for the tip which does seem possible in Python. At the moment I use one of the following commands:

# Basic call
subprocess.call(apache_cmd, stderr=subprocess.STDOUT)
# Advanced call to record output
p = subprocess.Popen(apache_cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
stdout, stderr = p.communicate()

The basic call on line 2 forces all STDOUT data to be handled by the parent’s file handler and all STDERR output should be captured by the same file handler as STDOUT. The same is happening on line 4 except STDOUT now goes to a new PIPE which Python can communicate with. Further explanation is given in the Python documentation:

17.1.1.1. Frequently Used Arguments

stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are <tt>PIPE</tt>, an existing file descriptor (a positive integer), an existing file object, and <tt>None</tt>. <tt>PIPE</tt> indicates that a new pipe to the child should be created. With the default settings of <tt>None</tt>, no redirection will occur; the child’s file handles will be inherited from the parent. Additionally, stderr can be <tt>STDOUT</tt>, which indicates that the stderr data from the child process should be captured into the same file handle as for stdout.

Unfortunately these settings don't seem to help so unless you have any other ideas I could take this question to a more Python oriented forum.

Thanks again

Przemek Bruski Atlassian Team Jan 25, 2012

Let's start with something simple, if you change your ptyhon script call to :

python script.py 2>&1

does it work?

Graeme Kidd Jan 26, 2012

The change has been applied in following way:

Beginning to execute external process for build 'Test Web Deploymet'
... running command line:
apache_deploy.py
--src
E:\bamboo-home\xml-data\build-dir\TESTDEPLOY\web\trunk\build\
--des
test-web
--rev
169
--typ
ful
2>&1
... in: E:\bamboo-home\xml-data\build-dir\TESTDEPLOY\web\etc\trunk\scripts

Still no luck

Przemek Bruski Atlassian Team Jan 27, 2012

And if you redirect:

script.py >logfile 2&1

Is the file in the proper order then?

Graeme Kidd Jan 29, 2012

Looks like simply outputting to a text file outside of Bamboo causes inconstancies. After looking into it further the issue was indeed due to buffering, after forcing stdin, stdout and stderr to be totally unbuffered using the –u flag: http://docs.python.org/using/cmdline.html#cmdoption-u the issue resolved itself.

Thanks again

Joshua Bouganim Jul 11, 2018

-u did it for me. Thanks!

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Opsgenie

Introducing Postmortems for Opsgenie

When incidents happen, it is crucial to diagnose and solve the problem as fast as possible. Equally important to this, teams need a way to reflect on why the incident occurred and how they can improv...

245 views 0 6
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you