RPM built in a Docker container on Bamboo has different contents than RPM built by hand

[Jim Melton]

I am migrating my team's practices to the 21st century, so please don't bother with "why would you do that?" type comments.

I have a Makefile that builds all my products and creates an RPM for deployment. This Makefile is being run within a Docker container as a Bamboo job. I want to automatically deploy the resulting RPM to Artifactory whenever a build on the master branch is run.

The problem is that the RPM produced by the job does not have the same contents as if the same commands are executed by hand. That is, if I log into the Bamboo server and interactively run the commands in my Docker container, I get an RPM with these files

% rpm -qlp /path/to/proj-1.0-1234567.el7.x86_64.rpm
/etc/opt/proj/CSCI_A_Config_Dflt.txt
/etc/opt/proj/CSCI_B_Config_Dflt.txt
/etc/opt/proj/CSCI_C_Config_Dflt.txt
/etc/opt/proj/CSC_Config_Dflt.txt
/etc/systemd/system/proj.service
/opt/proj/bin/CSCI_A
/opt/proj/bin/CSCI_B
/opt/proj/bin/CSCI_C
/opt/proj/bin/CSC
/opt/proj/bin/run_PROJ.bash
/opt/proj/lib64/libmpsse.so

 The RPM produced by my Bamboo job has these contents:

/etc/opt/proj/CSCIproj-1.0
/etc/opt/proj/CSCIproj-1.0/binAproj-1.0
/etc/opt/proj/CSCIproj-1.0/binAproj-1.0/binConfigproj-1.0
/etc/opt/proj/CSCIproj-1.0/binAproj-1.0/binConfigproj-1.0/binDflt.txt
/etc/opt/proj/CSCIproj-1.0/binBproj-1.0
/etc/opt/proj/CSCIproj-1.0/binBproj-1.0/binConfigproj-1.0
/etc/opt/proj/CSCIproj-1.0/binBproj-1.0/binConfigproj-1.0/binDflt.txt
/etc/opt/proj/CSCIproj-1.0/binCproj-1.0
/etc/opt/proj/CSCIproj-1.0/binCproj-1.0/binConfigproj-1.0
/etc/opt/proj/CSCIproj-1.0/binCproj-1.0/binConfigproj-1.0/binDflt.txt
/etc/opt/proj/CSCproj-1.0
/etc/opt/proj/CSCproj-1.0/binConfigproj-1.0
/etc/opt/proj/CSCproj-1.0/binConfigproj-1.0/binDflt.txt
/etc/systemd/system/proj.service
/opt/proj/bin/CSCIproj-1.0
/opt/proj/bin/CSCIproj-1.0/binA
/opt/proj/bin/CSCIproj-1.0/binB
/opt/proj/bin/CSCIproj-1.0/binC
/opt/proj/bin/CSC
/opt/proj/bin/runproj-1.0
/opt/proj/bin/runproj-1.0/binPROJ.bash
/opt/proj/lib64/libmpsse.so

(1.0 is the version of my RPM).

It would appear that underscores are being replaced with proj-1.0/bin in all the filenames. Any idea what is causing this, and how to fix it? 

[Jim Melton · Accepted solution]

Well, the good news is that it has nothing to do with Bamboo. It was a stupid scripting mistake. My build script looked something like this:

#! /bin/bash

docker_image=my-dock
build_products=$(cd $(dirname $0); pwd)

# This will run as an arbitrary user. Make sure they can write here
chmod o+w $build_products

# Clean up previous runs
ls $build_products
echo "rm $build_products/*.rpm"
rm -f $build_products/*.rpm
ls $build_products

docker run --rm --network=host \
 -v ${build_products}:/data $docker_image \
/bin/bash -c "pwd; 
svn checkout --quiet http://my.svn.repo/trunk/ /usr/src/proj;
cd /usr/src/spds;
export build_type=Release
cd Build/Linux; make CFG=$build_type clean; make CFG=$build_type; make CFG=$build_type rpm;
cp ~/rpmbuild/RPMS/x86_64/*.rpm /data"

chmod o-w $build_products

 The error is in using double-quotes to enclose the script to run in the Docker container. It causes the $build_type expression to be evaluated before build_type is defined, within the script. Simply replacing the double-quotes with single quotes defers the evaluation of $build_type to after it has been set in the script and all is good.

Why? Because (for legacy reasons) the build products are written to _Debug or _Release directories, based on the value of the CFG macro in the makefile. When CFG=$build_type is evaluated before it is set, CFG is empty, so the build target is _ (not very useful).

In the rpm target in the Makefile, I create a tar file, transforming names from where they exist in the build tree to where I want them to be in the deployment tree, like this:

rpm: proj.spec
        @rpmdev-wipetree; rpmdev-setuptre
        tar -cvf ~/rpmbuild/SOURCES/$(RPM_TARGET).tar \
                --exclude='*~' \
                --show-transformed-names \
                --transform=s:etc/:$(RPM_TARGET)/etc/:g etc \
                --transform=s:bin/:$(RPM_TARGET)/bin/:g bin \
                --transform=s:Config/:$(RPM_TARGET)/etc/:g Config/* \
                --transform=s:_$(CFG):$(RPM_TARGET)/bin:g _$(CFG)/{CSC,CSCI_{A,B,C}} \
                --transform=s:usr/local:$(RPM_TARGET): /usr/local/lib64/libmpsse.so
         rpmbuild -bb --define '_targetVersion $(RPM_VERSION)' proj.spec

 Note particularly the second to last --transform clause. If $(CFG) evaluates to an empty string, it will replace every underscore with $(RPM_TARGET)/bin ... and no, that wasn't what I wanted :)

Sorry for the trouble.