We have Ansible playbooks, are you interested?

Hi there, Community! My name is Steve and I'm a senior developer on the DC deployments team at Atlassian. 

tl;dr of this post:

  • Atlassian has a repository of Ansible roles and playbooks that we use to support our CloudFormation templates and internal deployments; check them out here.

  • It's very-much "some assembly required" right now.

  • We think this might be the basis of something useful to the wider community,
    but we need your input to decide.

  • Read-on for more details and to have your say.

Background

For the last 18 months or so the Atlassian DC deployments and internal IT teams have been slowly building and expanding a curated set of Ansible roles and playbooks to automate deployment, configuration, and maintenance of our Data Center products. While these roles were originally intended to be used in conjunction with our AWS CloudFormation templates , we’ve deliberately built them to be as general as possible as we didn’t want to limit their use-case. The roles are also increasingly being used internally by our IT operations team.

We're now thinking that these roles may be the basis of a useful toolkit for Atlassian administrators such as yourselves - especially those running Data Center deployments at scale.

An example playbook

This is a slightly contrived example of a playbook that will configure nodes of a Jira Software cluster, with an existing Postgres database that is deployed elsewhere, and a shared home mounted on /media/atl:

---


# Invoked with:
#
# ansible-playbook -v -i inv/standalone_instances -e atl_db_root_password=XXX -e atl_jdbc_password=XXX standalone_jira_software_dc_node.yml

- hosts: jira_software_node
become: true

vars:
atl_proxy_name: "jira.internal.my-company.com"

atl_product_family: "jira"
atl_product_user: "jira"
atl_product_edition: "jira-software"
atl_product_version: "8.13.0"

atl_enable_clustering: true
atl_shared_mountpoint: "/media/atl"

atl_db_host: "jiradb.internal.my-company.com"
atl_jdbc_user: "jira"
atl_jdbc_db_name: "jira"

atl_jvm_heap: "12288m"
atl_use_system_jdk: true
atl_download_format: "tarball"
atl_systemd_service_name: "jira.service"

roles:
- role: linux_common
- role: database_init
- role: product_common
- role: product_install
- role: jira_config
- role: product_startup

As you can see, almost all of the functionality is embedded in the roles, with the rest of the playbook being parameters supplied to the roles (in the vars section). I say this is "slightly contrived" as often these variables would be provided at runtime via the environment, although checking your playbooks configuration into a repository and fetching it at runtime (AKA GitOps) is certainly a valid approach.

Technical details

The canonical use of these roles is to bring up new product nodes on-the-fly, with the runtime parameters being supplied by the environment and any configuration generated on first-run; cattle in the DevOps parlance. In the Cloudformation templates, we dump the stack details to a file and pull the variables in via group-vars. Depending on your infrastructure, these variables could come from a cloud configuration provider (such as etcd or Zookeeper), or just checked-in to Git.

This suite is intended to consist of many small, composable roles that can be combined together into playbooks. Wherever possible, roles are product-agnostic and platform-agnostic. To ensure quality and cross-platform compatibility Ansible's Molecule test-framework is used to test each role against multiple Linux distributions (currently Amazon Linux 2 and Ubuntu).

Some caveats and the current state of play

As mentioned at the start, this suite has primarily been used internally at Atlassian for our own purposes, and mostly on AWS. If you’re interested in trying the suite out in your infrastructure it's definitely “some assembly required”. But the repository is available to use or fork, and is under a permissive open-source license.

While we've tried to keep things general as possible, and produce tests for compatibility, it's inevitable that some assumptions are baked-in and there are still rough-edges:

  • The roles don't configure infrastructure; as far as possible, they are infrastructure agnostic. We made this choice deliberately, as the suite is intended to be used in conjunction with other infrastructure tools, such as CloudFormation or Terraform.

  • So far, our focus has been on working with AWS infrastructure, in particular CloudFormation and Amazon Linux 2. While we actively test against a non-RPM Linux distribution (Ubuntu) we know that there could be some blind-spots.

  • Speaking of Linux, we only support that; no other OSs are currently on our radar at this point.

  • Within the Linux sphere, we assume systemd, although other runtime configurations are possible.

  • We assume AdoptOpenJDK is available on the target platform.

Where you come in

So with all that said, what do you think? Is this something that would be useful to the Atlassian administrator community? What features are missing that you might want, and what forms of infrastructure would you like to see supported?

8 comments

Comment

Log in or Sign up to comment
Gonchik Tsymzhitov
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
January 16, 2021

@Steve Smith Thank you! 

At least to me will be helpful :) 

As I see some of part I can reuse it 

Like Steve Smith likes this
David Grizzell
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
January 18, 2021

We would be interested in Ansible playbooks. Would certainly help - particularly for partners doing a lot of DC deployments.

wkennedy January 19, 2021

This means I can stop maintaining mine - yay!

Rodney Nissen
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
January 19, 2021

@wkennedyI feel you.  I want to dig into this - mine supports an API jail with dedicated API nodes (via HAProxy).  But if I can adapt this to do the actual node setup, win-win

Like wkennedy likes this
Jared Dohrman {ANZ} January 19, 2021

I also found this little gem recently too: https://github.com/atlassian-labs/data-center-helm-charts 

Helpful if anyone is running or considering running on k8s

Stefano Coletta January 21, 2022

We are going exactly in this direction! It would be a great idea.

Gregg Lowrimore February 17, 2022

So I have adapted these wonder playbooks and roles, but Jira doesn't startup. I get the infamous "Startup check failed. Jira will be locked." error upon startup. Initial Googling shows it may be b/c the required plugins are missing. I look and I do NOT have any plugins in the plugins folder. Where in your Ansible repo do the plugins get downloaded/installed?

Gregg Lowrimore February 18, 2022

Also, at what point is the Postgresql database initialized with tables/views/data for Jira? I see the database_init role called and executed cleanly, but when I open the postgres database/schema I am not seeing anything in it.

 

One of the issues with Jira not starting up, is the potential inability to connect to the database. Hence why I am testing out the connectivity (which is fine BTW) and seeing nothing in the db/schema.

Any tips on troubleshooting why Jira won't start and only gives this cryptic error:
"[c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked." ??

BTW, does the database root user or jira user passwords need to be encrypted in the ansible vars? Or just cleartext?

TAGS
AUG Leaders

Atlassian Community Events