Stash: using / managing multiple repos simultaneously?

If I usually need to work with multiple Stash repos across multiple projects, what's the best way to manage this? For example, I need Jenkins to be able to clone 12 different repos to do a full system integration test. I came across Google's git-repo : http://code.google.com/p/git-repo/ , but was wondering what the "Atlassian" solution is.

1 answer

This widget could not be displayed.

Hi Brandon,

That's a tricky question, and it depends entirely on how you version your repositories.

Atlassian generally doesn't have this specific problem because we use Maven, which handles dependency management at the binary level. You would build one project and release it (when ready) to Maven, and another project would depend on that exact version. All of the required artifacts would then be downloaded as part of a build. The downside to this approach is that making sweaping changes across multiple projects can require a little more co-ordination.

Another option that is specific to Git would be to use submodules. Personally, I'm not a huge fan of them, but the nice thing is they are supported in Bamboo. When you configure a build you have the option of downloading all the submodules before building, which might be exactly what you want. Submodules aren't all that different from Maven in that you (usually) depend on a specific commit from a specific repository (with a hard-coded URL, which is the part I don't like). Every time you make a change you have to update the parent repository with the new commit. This is actually a good thing, which means your builds will be identical every time they run, but could be considered tedious. As of Git 1.8.2 they added support for submodules tracking branches.

Finally you could just use artifact passing in Bamboo. So one build generates an artifact from one repository, and is passed to the next build which uses another repository, etc. Or alternatively just add the 12 repositories in a build and have all the source in one place. The downsides to this are that there is no explicit versioning and so you will never be able to rebuild to get the exact same result.

I'm afraid I haven't used git-repo so I couldn't comment about whether it would be better than one of these options, although I believe it is similar to submodules.

Just to summarize, my personal recommandation is to consider using a proper dependency management tool. This is likely to be lanuage specific, so Maven in the case of Java, or Bundler/RubyGems for Ruby, Pip/Eggs for Python, etc. The list goes on. If this isn't an option, submodules might be an easy way to achieve a similar outcome.

I hope this helps?

Charles

We are working towards using Gradle & Nexus for builds, dependency management, artifact publishing, etc. However, for mostly non-technical reasons, we won't be able to migrate our large legacy project to this stack and/or use artifact delivery instead of source delivery. Our plan was once we had our Stash repos set up, we could work on refactoring each repo to support artifact-based development, while still supporting requirements for managing lots of parallel repos clones in a giant build area.

Hi Brandon,

Sounds like you're heading in the right direction regarding your migration plan. For now you'll either have to:

a. Leave the files in a single repository. This is the easiest approach, and is only a problem if the size of the actual git objects is too large. Git can handle fairly large histories, it's large binary files that are the bigger problem. You can then use something like git filter-branch to extract subrepositories when you move to Gradle. This is what I've done in the past, but it really depends on the size of the repository.

b. Use something like submodules and/or git-repo from above. Using the new branches feature in submodules may make your life easier to avoid having to bump the parent module on every change, but I have never used it in anger.

c. Just use bamboo to clone all the repositorires from a specific branch. That may make it interesting for developers to build on their local machine.

I hope I haven't missed anything.

Charles

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted Thursday in United States

Local Atlassian Research Workshop opportunity on Sep. 28th

We're looking for participants for another workshop at Atlassian! We need Jira admins who have interesting custom workflows, issue views, or boards. Think you have a story to sh...

46 views 0 0
View post

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you