Bamboo incorrectly reports error and fails build though task was successful

[Gretchen Jones]

I'm running psexec tasks from bamboo using a variety of approaches and all seem to take the output from psexec and report it in the log files as an error.

My command is using an executable (<drivepath>\psexec.exe) and the command is

e:\bb\build\tools\psexec.exe -h -n 15 \\mydev-web1 %windir%\system32\inetsrv\APPCMD recycle apppool contentplayer

The output looks like this:

command	17-Feb-2013 00:17:55	Beginning to execute external process for build 'mybuild - Recycle MyDev Web1 App Pools'\n ... running command line: \ne:\bb\build\tools\psexec.exe -h -n 15 \\mydev-web1 %windir%\system32\inetsrv\APPCMD recycle apppool defaultapppool\n ... in: E:\bb\xml-data\build-dir\ATI-NG-RDW1AP\n ... using extra environment variables: \nbamboo_BuildPoint=E:\bb\DPBT_NUR\nbamboo_versionNbr=FEB-V13.2\nbamboo_deployRootLoc=E:\BB\build\\nbamboo_FirstSVNVerNbr=5093\nbamboo_branchVariable=/trunk/\nbamboo_CABTarget=Staging\nbamboo_FirstProdSVNVerNbr=4998\nbamboo_labelVar=/trunk/\nbamboo_deployReleasesLoc=E:\bb\build\Releases\\nbamboo_tagVar=trunk\nbamboo_deployReleasesLoc2=E:\\bb\\build\\Releases\\\nbamboo_deployTargetLoc=\nbamboo_lastNextGenBuild=\nbamboo_Previous_build_end_version=5092\nbamboo_deployCopyFromBuild=E:\bb\DPBT_NUR\\nbamboo_ProdRelRoot=E:\bb\build\Releases\Prod\\nbamboo_ProdVerNbr=FEB-V13.2\nbamboo_bambooBuildDir=E:\bb\xml-data\build-dir\\nbamboo_deployTemplate=Full_Deploy_TEMPLATE_SPLIT\n
error	17-Feb-2013 00:17:55	
error	17-Feb-2013 00:17:55	PsExec v1.98 - Execute processes remotely
error	17-Feb-2013 00:17:55	Copyright (C) 2001-2010 Mark Russinovich
error	17-Feb-2013 00:17:55	Sysinternals - www.sysinternals.com
error	17-Feb-2013 00:17:55	
build	17-Feb-2013 00:17:56	"DefaultAppPool" successfully recycled
error	17-Feb-2013 00:17:56	Connecting to mydev-web1...
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	Starting PsExec service on mydev-web1...
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	Connecting with PsExec service on mydev-web1...
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	Starting C:\Windows\system32\inetsrv\APPCMD on mydev-web1...
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	
error	17-Feb-2013 00:17:56	C:\Windows\system32\inetsrv\APPCMD exited on mydev-web1 with error code 0.
simple	17-Feb-2013 00:17:56	Finished task 'recycle apppool defaultapppool mydev-web1'

Since I really don't know what criteria bamboo or the executable plugin uses to determine if something is an true "error" I can't see a way to adjust the script (or my command line) to account for it. Process hangs and must be job stopped in order to get on with my life. I had the same hung process issue running as a script but I was able to stop and rerun incomplete/failed tasks and it would eventually complete (when I ran it enough times).

Command runs fine from the command line in all cases. I've tried putting it inline script, in a cmd file, and now using the external command (the approach reflected in this output).

I just want to recycle the stupid app pools. I'd like to have a record of it for posterity. (I know it probably doesn't need to be done but it's become a rabbits foot for some of the team). Should I redirect stderr to stdout? does that control what bamboo recognizes as error?

Appreciate any suggestions.

[Eric Hood]

Regarding it locking up: If you're running psexec under a different acct, it may be hanging waiting for the EULA to be accepted, which you can't see.  Newer versions of psexec have a "-accepteula" parameter to fix this.

Regarding error: I've noticed that psexec redirects the output it generates to stderr.  Not sure why, maybe to separate it from whatever is being executed to use stdout.  It is confusing though.

[sdesalas]

Try running this on remote command line as system user:

e:\bb\build\tools\psexec.exe -s -i

instead of

e:\bb\build\tools\psexec.exe -h -n 15

[PiotrA]

Hello Gretchen,

I'm having a bit of problem understanding what is exactly the problem (seems like there are multiple issues you're trying to overcome), but I'll try to give some hints.

1) Bamboo fails the build when any of the build task fail. What does it mean "the task fails" - it's up to the task, but for example for script tasks it is the return code of the executed script. So if your script task doesn't return 0, the Bamboo will fail the build. I'm not sure how the logic is coded for "Command" task, or similar.

2) When parsing logs from the Script Tasks, Bamboo perceive all data pumped to stderr to be 'error messages'. AFAIK these won't fail the build, but will be explicitly displayed in the build result summary under "Build Error Messages" or similar.

I'm not sure what are those "hung" issues you're mentioning. Are your Bamboo builds hanging up?

So, if I were you I'd try to:

- use Script Task to run the PsExec thingy
- play with redirecting PsExec stderr to be stdout (if you don't want these messages to be seen in Bamboo build summary pages)
- play with script return code (for example "exit 0") - to make the Bamboo treat that script as always passing (and if that works - update the script to return different code when something goes wrong)

Is it helpful?

regards

[Gretchen Jones]

I have tried using the script task. It hangs bamboo (task icon still spinning, nothing happening) I'll try the redirection the issue isn't the messaging though I think that is peculiar that it reports as error since it should be stdout.

The exit code zero is the norm but it still hangs, I have seen some other codes but reviewing them provides no insight into the problem.

It's almost as though the "pipe" breaks losing connection to the task. But of course I can't tell from the output what is actually happening.

This task executes on a remote server and it's an important task that recycles the IIS Application pools on the remote server. (something that may be technically unnecessary, but is required by the dev team as part of the deployment task).

Thanks for your input.