Pipeline step results are truncated

Question

I am running a step with some "aws cli" commands to pull the values from existing AWS resources and then running "aws cloudformation deploy" a template passing those parameters value.

However, step logs in log viewer only displaying limited number of lines and i am unable to see the full log to understand or debug anything.

End of logs: Even after downloading the logs:

+ dwh_iam_role="$(aws cloudformation describe-stacks --stack-name ${PROJECT_SHORT_NAME}-${ENVIRONMENT}-infra --profile ${BITBUCKET_BRANCH} | grep -A1 OutputKey | grep -A1 DWHCopyExecutionRoleArn | grep OutputValue | tr -d '"'|cut -d':' -f 2- | sed s/"//g | sed -e 's/^[ 	]*//')"

+ dwh_elb_dns_name="$(aws cloudformation describe-stacks --stack-name ${PROJECT_SHORT_NAME}-${ENVIRONMENT}-infra --profile ${BITBUCKET_BRANCH} | grep -A1 OutputKey | grep -A1 DWHNetworkLoadBalancerDnsName | grep OutputValue | tr -d '"'|cut -d':' -f 2 | sed s/"//g | sed -e 's/^[ 	]*//')"

Anshul Shah · Answer

Nothing wrong with the script. Pipeline was successful.

It is a bug in bitbucket pipelines instead, where more number of commands sometimes does not log all commands at right place.

There is an issue opened with the respective team about it.

https://jira.atlassian.com/browse/BCLOUD-18574

Halyna Berezovska · Answer

@Anshul Shahif agter such text there are indeed no logs, it can be that something wrong with the script

dwh_iam_role="$(aws cloudformation describe-stacks --stack-name ${PROJECT_SHORT_NAME}-${ENVIRONMENT}-infra --profile ${BITBUCKET_BRANCH} | grep -A1 OutputKey | grep -A1 DWHCopyExecutionRoleArn | grep OutputValue | tr -d '"'|cut -d':' -f 2- | sed s/"//g | sed -e 's/^[ 	]*//')"

or

dwh_elb_dns_name="$(aws cloudformation describe-stacks --stack-name ${PROJECT_SHORT_NAME}-${ENVIRONMENT}-infra --profile ${BITBUCKET_BRANCH} | grep -A1 OutputKey | grep -A1 DWHNetworkLoadBalancerDnsName | grep OutputValue | tr -d '"'|cut -d':' -f 2 | sed s/"//g | sed -e 's/^[ 	]*//')"

Try to debug those scripts and see why they can fail (perhaps some keywords of grep command do not work. Also I recommend to use jq tool, since it parses json structure, that aws cli responds with).

Regards, Galyna