Home directory missing?

[Chris]

I started a default instance of bamboo 3.0.4-studio-4-1308709939816 ami-0671896f and upgraded php and phpunit

Then followed the instructions http://confluence.atlassian.com/display/BAMBOO/Creating+a+Custom+Elastic+Image with some differences.

I skipped to step 4 by accessing the instance bamboo started as when I created an instance with the steps in Step 3 with the ami-0671896f, it never started because of missing keys/folders

Since bamboo binaries and user were already there I next skipped to step 5.4 and adjusted the keys to be my own then finally packaged the bundle

When starting the instance it starts to fail when it cant find /home/

Any idea why the new image wouldnt contain the /home/ directory when I created it from an image that had it?

Log dump: http://pastebin.com/QHux7ZzR

Starting ConsoleKit: [  OK  ]

Starting crond: [  OK  ]
[  OK  ]

Starting anacron: Could not initialise connection to hald.
Normally this means the HAL daemon (hald) is not running or not ready.
[  OK  ]

grep: /root/.ssh/authorized_keys: No such file or directory
Attempting ami-utils update from S3
--2011-12-06 11:30:17--  http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm
Resolving s3.amazonaws.com... 72.21.211.170
Connecting to s3.amazonaws.com|72.21.211.170|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 172568 (169K) [application/x-unknown-content-type]
Saving to: `ec2-ami-tools.noarch.rpm'


 0% [                                       ] 0           --.-K/s              
82% [===============================>       ] 141,907      683K/s              
100%[======================================>] 172,568      824K/s   in 0.2s    

2011-12-06 11:30:18 (824 KB/s) - `ec2-ami-tools.noarch.rpm' saved [172568/172568]

ec2: Retreived ec2-ami-tools from S3
	package ec2-ami-tools-1.4-0.2.noarch is already installed
ec2: Preparing...                ##################################################
ec2: ec2-ami-tools already up to date
ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 2048 b5:key_remove_by_me:01 /etc/ssh/ssh_host_key.pub
ec2: 2048 b4:key_remove_by_me:fd /etc/ssh/ssh_host_rsa_key.pub
/etc/ssh/ssh_host_dsa_key.pub: No such file or directory
ec2: -----END SSH HOST KEY FINGERPRINTS-----
mv: cannot stat `/home/*': No such file or directory
mv: cannot stat `/home/.*': No such file or directory
mount: mount point /home does not exist
Stopping crond: su: warning: cannot change directory to /home/bamboo: No such file or directory
[  OK  ]

Starting crond: [  OK  ]
[  OK  ]

tee: /home/bamboo/bamboo-elastic-agent.out: No such file or directory
0 [main] INFO com.atlassian.bamboo.agent.elastic.client.ElasticAgentBootstrap  - Using tunnnelling. Registering 'httpt' and 'httpst' protocols.

Fedora release 8 (Werewolf)
Kernel 2.6.21.7-2.fc8xen on an i686

ip-10-66-93-95 login: 1411 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Waiting for tunnel connection.
3308 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Accepted tunnel connection.
3310 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Waiting for tunnel connection.
3310 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Accepted tunnel connection.
3311 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Waiting for tunnel connection.
3503 [tunnelserver-26224-1-thread-3] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Waiting for local connection on port 46593.
3514 [tunnelserver-26224-1-thread-1] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Waiting for local connection on port 4527.
Agent bootstrap using baseUrl: httpt://127.0.0.1:46593/builds/agentServer/
3580 [main] INFO com.atlassian.bamboo.agent.elastic.client.ElasticAgentBootstrap  - Using tunnelling. Setting virtual host name to ourcompany.jira.com
Requesting fingerprintRequestUrl: httpt://127.0.0.1:46593/builds/agentServer/GetFingerprint.action?hostName=ip-10-66-93-95.ec2.internal&agentType=elastic&instanceId=i-1fdfd27c
5619 [tunnelserver-26224-1-thread-3] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Accepted local connection on port 46593.
5667 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Accepted tunnel connection.
5667 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Waiting for tunnel connection.
5745 [tunnelserver-26224-1-thread-5] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Waiting for local connection on port 46593.
5811 [Thread-2] INFO com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent  - com.atlassian.bamboo.agent.elastic.client.ElasticAgent starting.
5841 [tunnelserver-26224-1-thread-5] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Accepted local connection on port 46593.
5887 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Accepted tunnel connection.
5887 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Waiting for tunnel connection.
5958 [tunnelserver-26224-1-thread-7] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Waiting for local connection on port 46593.
6502 [tunnelserver-26224-1-thread-7] INFO com.sun.sungrid.service.tunnel.server.LocalAcceptor  - Accepted local connection on port 46593.
6552 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Accepted tunnel connection.
6552 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  - Waiting for tunnel connection.
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: atlassian-bamboo.log (Permission denied)
	at java.io.FileOutputStream.openAppend(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
	at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
	at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:194)
	at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
	at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
	at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
	at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
	at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:544)
	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:440)
	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:476)
	at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:354)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.configureLog4j(RemoteAgent.java:103)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:49)
	at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:20)
	at java.lang.Thread.run(Thread.java:619)
2011-12-06 11:30:30,358 INFO [tunnelserver-26224-1-thread-9] [LocalAcceptor] Waiting for local connection on port 46593.
2011-12-06 11:30:30,609 INFO [Thread-2] [RemoteAgent] Starting Bamboo Remote Agent...
2011-12-06 11:30:30,610 INFO [Thread-2] [RemoteAgent] Starting application context...
2011-12-06 11:30:35,324 INFO [Thread-2] [RemoteAgentHomeLocator] bamboo.home set to default location: "/home/bamboo/bamboo-agent-home" (Customise by specifying -Dbamboo.home=<directory> on the command line.)
2011-12-06 11:30:35,324 WARN [Thread-2] [AgentConfiguration] Application home does not exist. Creating directory: /home/bamboo/bamboo-agent-home
Exiting due to fatal exception.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentConfiguration' defined in class path resource [applicationContextRemoteAgent.xml]: Invocation of init method failed; nested exception is com.atlassian.bamboo.configuration.ConfigurationException: Could not make directory/ies: /home/bamboo/bamboo-agent-home
Caused by: com.atlassian.bamboo.configuration.ConfigurationException: Could not make directory/ies: /home/bamboo/bamboo-agent-home
	at com.atlassian.bamboo.v2.build.agent.remote.AgentConfiguration.createAppHomeIfNotExists(AgentConfiguration.java:212)
	at com.atlassian.bamboo.v2.build.agent.remote.AgentConfiguration.init(AgentConfiguration.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:190)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:53)
	at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:20)
	at java.lang.Thread.run(Thread.java:619)
2011-12-06 11:30:35,338 FATAL [Thread-2] [AgentBootstrap] Exiting due to fatal exception.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentConfiguration' defined in class path resource [applicationContextRemoteAgent.xml]: Invocation of init method failed; nested exception is com.atlassian.bamboo.configuration.ConfigurationException: Could not make directory/ies: /home/bamboo/bamboo-agent-home
Caused by: com.atlassian.bamboo.configuration.ConfigurationException: Could not make directory/ies: /home/bamboo/bamboo-agent-home
	at com.atlassian.bamboo.v2.build.agent.remote.AgentConfiguration.createAppHomeIfNotExists(AgentConfiguration.java:212)
	at com.atlassian.bamboo.v2.build.agent.remote.AgentConfiguration.init(AgentConfiguration.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:190)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:53)
	at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:20)
	at java.lang.Thread.run(Thread.java:619)

INIT: Switching to runlevel: 0

INIT: Sending processes the TERM signal


INIT: Sending processes the KILL signal

Stopping ConsoleKit: [  OK  ]

Stopping sshd: [  OK  ]

Stopping crond: [  OK  ]

Stopping system message bus: [  OK  ]

Shutting down kernel logger: [  OK  ]

Shutting down system logger: [  OK  ]

Shutting down interface eth0:  [  OK  ]

Shutting down loopback interface:  [  OK  ]

Starting killall:  [  OK  ]

Sending all processes the TERM signal... 
Sending all processes the KILL signal... 
Saving random seed:  
Syncing hardware clock to system time 
Turning off swap:  
Turning off quotas:  
Unmounting file systems:  
mount: /proc is busy
Halting system...
System halted.

[Przemek Bruski · Accepted solution]

Normally, /home is mounted on ephemeral storage. If you want to customise your image, you need to copy the /home to root partition before saving the image so that it's a part of your final image. Something like this should do the trick:

/etc/init.d/crond stop

agentCfgName=bamboo-agent.cfg.xml
agentCfgFile=/home/bamboo/bamboo-agent-home/${agentCfgName}

sed -n -e '/<agentDefinition/,/<.agentDefinition/!p' ${agentCfgFile} | grep -v '<agentUuid>' >/tmp/${agentCfgName}

echo Killing all Java processes...
killall java 2>/dev/null
sleep 2
killall -9 java 2>/dev/null


rootDevice=$(mount |grep " / " | sed "s/ .*//")

tempRootMountPoint=/tmp/rootDevice${RANDOM}
echo Mounting root device $rootDevice in ${tempRootMountPoint} to copy data from ephemeral storage...
mkdir ${tempRootMountPoint}
mount ${rootDevice} ${tempRootMountPoint}

echo Copying back /home from ephemeral storage to the root device...

cp -ax /home/ ${tempRootMountPoint}

mv /tmp/${agentCfgName} ${tempRootMountPoint}/${agentCfgFile}
chown -R bamboo:bamboo ${tempRootMountPoint}/home/bamboo/

This script will be disctributed with Bamboo at some point.

[Chris]

Before Przemek responded, running the ec2-bundle-vol cmd with --all included everything. I have not tried his solution as we got it running with that.

[fshehzad]

Hi Chris,

Are you able to solve this issue ? Below suggestion didnot work for me. it fails to create volume after i execute it.