Can I copy Bamboo Artifacts from one instance to another ?

[yash_shanker24]

After exporting the backup, without Artifacts, and importing it into another Bamboo Instance, my plan is to copy the artifacts from file system from the old Bamboo instance to the new one.

I am not doing the full System Export [Overview > System > Export] including Artifacts, because my Bamboo instance is huge, and the export process is taking way too long when I try to export Artifacts with the System Export, and I'm talking weeks here. The last export process started 9 days ago, and its still in progress.

Is that possible ? If yes, how. If not, why and is there any kind of workaround ?

[Daniel Santos · Accepted solution]

Hi @yash_shanker24

Yes, you can add the artifacts later, but that is only possible if you use the option [x] Export Results when exporting the data.

I'm working on a different question from you:

•  Is my Bamboo Export Process still in progress ?

Let's focus our efforts on that one. Depending on what you are trying to achieve we could use the backup of your DB external tool and not Bamboo export. The Bamboo export is a slow process that translates your Bamboo data to an independent DB format and therefore requires a lot of processing time. Its performance is bad for instance that are too big.

The other option on migration and upgrades is using a backup from the external DB tool. That backup would be fast and would also preserve all the Bamboo information.

Let's talk about this process in the other thread.

[yash_shanker24]

Hey @Daniel Santos ,

I understand you, but I am actually performing two parallel export processes of the same Bamboo instance on two different clone servers.

One is a complete backup with all options checked, lets call it WA. And other with all options checked except artifacts, lets call it WoA. 

WA export process is in progress for the 9th day today. WoA export process completed in 1 day, and I had setup a new Bamboo instance, lets call it NewBamboo with upgraded Bamboo version and external MSSQL database, and I started the import of the WoA export file in that. And the export is in progress for the 8th day today.

Logs in the importing process in NewBamboo as of now are :

2019-10-17 18:24:55,075 INFO [http-nio-8085-exec-8] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 120000, used memory 1004 MB
2019-10-18 07:43:34,793 INFO [http-nio-8085-exec-8] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 130000, used memory 1014 MB

[yash_shanker24]

Hey @Daniel Santos 

Logs from just now

17-Oct-2019 10:57:07.879 WARNING [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadCompleted Thread [http-nio-8085-exec-23] (id=[41]) was previously reported to be stuck but has completed. It was active for approximately [63,825,226] milliseconds. There is/are still [1] thread(s) that are monitored by this Valve and may be stuck.
2019-10-17 11:31:53,757 INFO [http-nio-8085-exec-17] [AccessLogFilter] 0:0:0:0:0:0:0:1 GET http://localhost:8085/error/errorEvents.action?next=%2F 59036kb
2019-10-17 15:18:34,784 INFO [http-nio-8085-exec-25] [AccessLogFilter] 0:0:0:0:0:0:0:1 GET http://localhost:8085/error/errorEvents.action?next=%2F 41242kb
2019-10-17 17:24:34,488 INFO [http-nio-8085-exec-24] [AccessLogFilter] 0:0:0:0:0:0:0:1 GET http://localhost:8085/error/errorEvents.action?next=%2F 42990kb
2019-10-17 18:24:55,075 INFO [http-nio-8085-exec-8] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 120000, used memory 1004 MB
2019-10-18 07:43:34,793 INFO [http-nio-8085-exec-8] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 130000, used memory 1014 MB
2019-10-18 12:36:12,704 ERROR [http-nio-8085-exec-8] [TransactionInterceptor] Application exception overridden by rollback exception
java.lang.OutOfMemoryError: Java heap space
at java.util.AbstractCollection.toArray(Unknown Source)
at java.util.ArrayList.<init>(Unknown Source)
at org.springframework.transaction.support.TransactionSynchronizationManager.getSynchronizations(TransactionSynchronizationManager.java:321)
at com.atlassian.bamboo.hibernate.cachehooks.DirtiedHibernateSessionObjectRecorder.getTxSync(DirtiedHibernateSessionObjectRecorder.java:137)
at com.atlassian.bamboo.hibernate.cachehooks.DirtiedHibernateSessionObjectRecorder.scheduleInvalidationIfNeeded(DirtiedHibernateSessionObjectRecorder.java:71)
at com.atlassian.bamboo.hibernate.cachehooks.DirtiedHibernateSessionObjectRecorder.onSave(DirtiedHibernateSessionObjectRecorder.java:44)
at org.springframework.orm.hibernate.support.ChainedInterceptorSupport.onSave(ChainedInterceptorSupport.java:118)
at org.springframework.orm.hibernate.support.ChainedInterceptorSupport.onSave(ChainedInterceptorSupport.java:118)
at com.atlassian.bamboo.hibernate.ReadOnlyGetMethodEnforcer.onSave(ReadOnlyGetMethodEnforcer.java:88)
at org.springframework.orm.hibernate.support.ChainedInterceptorSupport.onSave(ChainedInterceptorSupport.java:118)
at com.atlassian.bamboo.persister.OidGenerationInterceptor.onSave(OidGenerationInterceptor.java:46)
at org.springframework.orm.hibernate.support.ChainedInterceptorSupport.onSave(ChainedInterceptorSupport.java:118)
at org.hibernate.event.internal.AbstractSaveEventListener.substituteValuesIfNecessary(AbstractSaveEventListener.java:419)
at org.hibernate.event.internal.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:275)
at org.hibernate.event.internal.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:201)
at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:144)
at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:192)
at org.hibernate.event.internal.DefaultSaveEventListener.saveWithGeneratedOrRequestedId(DefaultSaveEventListener.java:38)
at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:177)
at org.hibernate.event.internal.DefaultSaveEventListener.performSaveOrUpdate(DefaultSaveEventListener.java:32)
at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:73)
at org.hibernate.internal.SessionImpl.fireSave(SessionImpl.java:709)
at org.hibernate.internal.SessionImpl.save(SessionImpl.java:701)
at org.hibernate.internal.SessionImpl.save(SessionImpl.java:696)
at com.atlassian.bamboo.persistence3.HibernateDaoUtils.save(HibernateDaoUtils.java:170)
at com.atlassian.bamboo.migration.BambooStAXMappingHelperAbstractImpl.saveBambooObject(BambooStAXMappingHelperAbstractImpl.java:202)
at com.atlassian.bamboo.build.artifact.ArtifactLinkMapper.afterImportListItem(ArtifactLinkMapper.java:129)
at com.atlassian.bamboo.build.artifact.ArtifactLinkMapper.afterImportListItem(ArtifactLinkMapper.java:28)
at com.atlassian.bamboo.migration.BambooStAXListImportStrategy.importListItemXml(BambooStAXListImportStrategy.java:253)
at com.atlassian.bamboo.migration.BambooStAXListImportStrategy.access$100(BambooStAXListImportStrategy.java:21)
at com.atlassian.bamboo.migration.BambooStAXListImportStrategy$3$1.doInTransaction(BambooStAXListImportStrategy.java:97)
at com.atlassian.bamboo.migration.BambooStAXListImportStrategy$3$1.doInTransaction(BambooStAXListImportStrategy.java:89)
2019-10-18 12:36:12,720 ERROR [http-nio-8085-exec-8] [ExceptionMappingInterceptor] Method "execute" failed for object com.atlassian.bamboo.ww2.actions.setup.SetupImportDataAction@52ec9b18
ognl.MethodFailedException: Method "execute" failed for object com.atlassian.bamboo.ww2.actions.setup.SetupImportDataAction@52ec9b18 [java.lang.OutOfMemoryError: Java heap space]

I guess the import failed. Kindly instruct me further.
This import was running on a t3.large AWS Instance.

[yash_shanker24]

By default, the values of Xms and Xmx were 1024MB, I have increased them to 16GB and started the import process again.

[Daniel Santos]

Hi @yash_shanker24

Thank you for sharing that. Unfortunately, the import process is even slower than the export one. 

Short instructions on this one:

1. We don't need the export with artifacts and build logs
2. We should try to shrink this instance to the minimum size possible before moving forward
3. Yes, we will need more memory for the import and it will take a lot of time. 
   I'm considering that you are migrating from an embedded DB to an external one like shared in the other thread.

Please let's work only in the other thread from now. It will help us to keep ourselves organized.

This is the one we should talk from now on:

• https://community.atlassian.com/t5/Bamboo-questions/Is-my-Bamboo-Export-Process-still-in-progress/qaq-p/1204160

I'll send you another message there.

[Nithyanantham Lakshmanan]

@yash_shanker24 

We are also increased JVM memory like below and started importing but still going on last 15hrs.. We also don't how much time to complete this process.

actual zip file size is 104GB.

#

: ${JVM_MINIMUM_MEMORY:=12288m}

: ${JVM_MAXIMUM_MEMORY:=18432m}

#

server Logs:

2021-01-10 23:15:33,475 INFO [http-nio-8085-exec-2] [AccessLogFilter] 172.16.10.3 GET http://10.162.5.18:8085/error/errorEvents.action 12356183kb

2021-01-10 23:18:10,092 INFO [http-nio-8085-exec-3] [AccessLogFilter] 172.16.10.3 GET http://10.162.5.18:8085/error/errorEvents.action 12094037kb

2021-01-11 00:10:10,244 INFO [http-nio-8085-exec-3] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 160000, used memory 3548 MB

2021-01-11 01:46:42,410 INFO [http-nio-8085-exec-3] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 170000, used memory 6438 MB

2021-01-11 03:21:54,871 WARN [atlassian-scheduler-quartz2.local_Worker-4] [PacClientImpl] Update check request may take longer because of the number of apps

2021-01-11 03:26:17,336 INFO [http-nio-8085-exec-3] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 180000, used memory 5178 MB

2021-01-11 05:08:08,380 INFO [http-nio-8085-exec-3] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 190000, used memory 6192 MB

2021-01-11 06:53:17,820 INFO [http-nio-8085-exec-3] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 200000, used memory 4864 MB

2021-01-11 08:41:06,085 INFO [http-nio-8085-exec-3] [BambooStAXMappingHelperAbstractImpl] Importing artifactLinks.artifactLink: 210000, used memory 4507 MB