How to import a confluence xml backup via a script

[Partha Kamal]

I have a sandbox instance that I would like to refresh everyday.

Can this be scripted and if so, how?

[twong_atlassian · Accepted solution]

I wouldn't recommend doing that. As per Production Backup Strategy and various KB articles like this one, XML backup is generally slower, and generally more problemmatic.

If you must use xml backup (maybe you're doing migration between differing database servers), your most likely bet is to create a script invoking the Confluence Remote API.

The most relevant call is :

String exportSite(String token, boolean exportAttachments) - exports a Confluence instance and returns a String holding the URL for the download. The boolean argument indicates whether or not attachments ought to be included in the export.

However, the actual import is much more difficult, considering that a fresh install won't have the remote api on startup, and there does not exist an importSite call. (It's hinted that importSpace() is implemented, but it isn't documented). A workaround here would be to start the sandbox instance with the -Dpassword.confirmation.disabled=true directive (source), then use curl to probe the restore url in the administration panel, posting the xml.

A better, more sane way of approaching this, is simply to scp the attachments folder between the instances, and scripting the db dump, transfer, and restore to the box. A Confluence instances can be replicated as a snapshot by transfering just the attachments, and the db to the correct places. Index will have to be rebuilt.

[Partha Kamal]

Good point.

Also, I could use hsql and I could script shutting down the db, removing the existing hsql db, and copying over backed up version, copying over a backed up confluence home directory and restart confluence. This would give me a refreshed sandbox.

I know that hsql is not recommended for production, but since this is sandbox that would be refreshed everyday, its could still be used.

[twong_atlassian]

That would require that your production instance be hsql, as there isn't a good way to convert another db to hypersonic.

Not ideal. Best is rsync + native db commandline awesomeness.

[Brett Ryan]

Using an MySQL database I do this quite easily though it does depend on your environment.

1. Perform a live DB dump of the production system you are refreshing from
2. Copy dump files to your test instance (an rsync should do it)
3. In your test intsance shut down confluence
4. Dump and Load the new database on your test instance
5. Have any scripts applied to update any changes across each

For reference the following is a backup script I use, keep in mind that I use a common tomcat environment using the confluence WAR distribution using a local tomcat context by use of CATALINA_HOME, this approach GREATLY eases what you're trying to do by isolating the confluence install away from the tomcat install allowing both to be upgraded independently, and the only confluence dependency of course is the WAR file to be built.

cd "`dirname \"$0\"`"
DIR="${PWD-`pwd`}"
cd ..
 
[ -x "$DIR/common.sh" ] || {
    echo "You must configure \`$DIR/common.sh' to set CATALINA_BASE."
    exit 1
}
. "$DIR/common.sh"
 
BACKUP_DIR='/data/backup/confluence'
MYSQL_USER='MY_USER'
MYSQL_PASS='MY_PASS
MYSQL_DB="confluencedb"
BACKUP_DAYS=2
 
MYSQLDUMP="/usr/bin/mysqldump"
FIND="/usr/bin/find"
GZIP="/bin/gzip"
TAR="/bin/tar"
 
# Remove old backups
"$FIND" "$BACKUP_DIR" -daystart -maxdepth 1 -type f -name 'confluence*' -mtime +$BACKUP_DAYS -delete
 
# Backup the database.
"$MYSQLDUMP" -u "$MYSQL_USER" --password="$MYSQL_PASS" "$MYSQL_DB" \
    | $GZIP --fast > "$BACKUP_DIR"/"confluence_db-`date +%Y-%m-%d_%H-%M`".sql.gz
 
DIR="`basename \"$CONFLUENCE_HOME\"`"
 
cd "$CONFLUENCE_HOME"/..
 
"$TAR" czf "$BACKUP_DIR"/"confluence_home-`date +%Y-%m-%d_%H-%M`".tar.gz "$DIR"
 
exit

For this to work I use a common.sh script to setup an environment for startup/shutdown/backup

CONFLUENCE_HOME="/data/confluence"
CONFLUENCE_INST="/appl/confluence"
CATALINA_HOME="/appl/tomcat/latest"
 
CATALINA_BASE="$CONFLUENCE_INST"
CATALINA_LOGDIR="$CONFLUENCE_HOME/logs"
CATALINA_TMPDIR="$CONFLUENCE_HOME/temp"
CATALINA_OUT="$CATALINA_LOGDIR/catalina.out"
CATALINA_PID="$CATALINA_BASE/catalina.pid"
 
#export LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
 
TMP="$CATALINA_TMPDIR"
TEMP="$TMP"
 
# Check to see if base environment looks okay.
[ -z "$CATALINA_BASE" ] && {
    echo "ERROR: CATALINA_BASE has not been set." >&2
    exit 2
} 
[ -d "$CATALINA_BASE" ] || {
    echo "ERROR: CATALINA_BASE ($CATALINA_BASE) does not exist." >&2
    exit 3
} 
[ -d "$CATALINA_TMPDIR" ] || mkdir -p "$CATALINA_TMPDIR" || {
    echo "ERROR: Couldn't create CATALINA_TMPDIR ($CATALINA_TMPDIR)" >&2
    exit 4
} 
[ -d "$CATALINA_LOGDIR" ] || mkdir -p "$CATALINA_LOGDIR" || {
    echo "ERROR: Couldn't create CATALINA_LOGDIR ($CATALINA_LOGDIR)." >&2
    exit 5
}
 
export CONFLUENCE_HOME CONFLUENCE_INST
export CATALINA_TMPDIR CATALINA_PID CATALINA_OUT CATALINA_HOME CATALINA_BASE
export TMP TEMP

[Brett Ryan]

My goodness that looks terrible, how do you format code in this new system?

[Brett Ryan]

Cheers laura.

[Laura Kolker]

@Brett That's been bugging me too, so I just posted this: https://answers.atlassian.com/questions/489/how-do-we-provide-code-blocks-in-these-questions

[twong_atlassian]

We're working on it! pokes Jeremy some more Maybe if I offer cake.