Ansible via Opsgenie Edge Connector
The purpose of this post is to guide you through the setup of OEC and Ansible so you can trigger custom actions in the form of playbooks.
Disclaimer: I performed all the actions for this guide on a CentOS 7.6 server, so make changes where needed for your distro.
I have also made Ansible roles that will install it and maintain the life-cycle of the OEC config for the scripts. I highly recommend going this route as it will allow you to have a self-healing OEC instance. Please note, you would need to run the role from either another system that has Ansible installed on it, or simply install Ansible on the OEC host and then run the it, this can be found here:
https://galaxy.ansible.com/phil_avery/og_oec
https://github.com/phil-avery/og-runner
Components Needed
Opsgenie Edge Connector
Opsgenie Edge Connector (OEC) is a lightweight application / service, this can be on Linux, Windows or a Docker container. It pulls the payload information down from Opsgenie and then executes a custom action in the form of scripts, these scripts can be any language that the OEC host can interpret. Thus no need to expose anything external including your Ansible Tower.
OEC supports environment variables, arguments, and flags that are passed to scripts. These can be set globally for all scripts or locally on a per script basis. Stderr and stdout options are also available.
For more information on Edge Connector, check out the documentation.
Ansible Options
You can use any form of Ansible, ( Tower, Runner and Core ) However; I've found that Ansible Tower or Ansible Runner give the best experience.
Ansible Tower
If you have an existing Ansible Tower setup you can easily pass onto the Ansible Tower server to run your Job Templates or Workflows.
Ansible Runner
Ansible runner has the ability to run playbooks from the command line, yet you can also force execution within a directory. This means that you can perform all Ansible custom actions inside a directory per Alert or Incident ID, which allows for a great workflow and auditability.
Ansible Core
If you need to run Ansible Core you can by simply installing Ansible on the same server as OEC, and then executing ansible-playbook along with any options. Please note that Ansible Core doesn’t give the ability to audit historical runs.
Setup Ansible ( Ansible Runner )
1.) Install epel
yum install -y epel-release
2.) Install Python Development
yum install -y python-devel
3.) Install python-pip
yum install python-pip
4.) Install Ansible and Ansible-runner
pip install ansible ansible-runner
5.) If you need any additional packages for Ansible now is a good time to get them, examples if you want to use VMWare or AWS for dynamic inventories then there are pip package dependencies for them
5.) Create a folder for the Ansible playbooks and runner
mkdir /var/ansible
mkdir /var/ansible/playbooks
mkdir /var/ansible/runner
6.) Ansible Runner gives us the ability to create an audit log of all jobs that are run by creating a directory and running them from there. This is ideal as you can have a folder created with the Opsgenie Alert ID for audit purposes. I am going to have mine set as /var/log/runner but feel free to change this as needed.
mkdir /var/log/runner
7.) Create the runner python script "runner.py" in /var/ansible/runner/
A copy of my script is located here
https://raw.githubusercontent.com/phil-avery/og-runner/master/files/runner.py
If you changed the path in step 6 above, you will need to change the line
ansible_dir = "/var/log/runner/" + alert_id
8.) Give the python script execute permissions
chmod 0755 /var/ansible/runner/runner.py
9.) Create some test playbooks to execute as custom actions
in /var/ansible/playbooks create some test playbooks you can bring down the ones from my repo
facts.yml
https://raw.githubusercontent.com/phil-avery/og-runner/master/files/facts.yml
This will collect service facts and some other system facts and present them back to Opsgenie as key value pairs.
Setup OEC
1.) Install OEC package
We have documented this on our docs page and stays current with any changes so no need to rewrite it here.
https://docs.opsgenie.com/docs/oec-installation
2.) Configure OEC
This is also documented here
https://docs.opsgenie.com/docs/oec-configuration
However, the following guide is what I find to be the best set up for our use case.
a.) Create OEC Config folder
mkdir /etc/opsgenie
b.) Set Permissions to /etc/opsgenie
chown opsgenie:opsgenie /etc/opsgenie
c.) Create OEC Environments file
vi /etc/opsgenie/oec.env
add in the following
OEC_CONF_SOURCE_TYPE=local
OEC_CONF_LOCAL_FILEPATH=/etc/opsgenie/oecconfig.yml
d.) Ensure the OEC Service file is present by enabling the service
systemctl enable oec
e.) Edit the OEC Service file so we can use the environment file rather than having them set as ENV variables
vi /etc/systemd/system/oec.service
Add in the EnvironmentFile line
[Unit]
Description=Opsgenie Edge Connector (OEC)
[Service]
Group=opsgenie
User=opsgenie
Type=simple
ExecStart=/usr/local/bin/OpsgenieEdgeConnector
EnvironmentFile=/etc/opsgenie/oec.env
[Install]
WantedBy=multi-user.target
f.) Create our OEC Config file ( Note. OEC can use either YAML or JSON, I prefer YAML but change to JSON if it helps you )
vi /etc/opsgenie/oecconfig.yml
This is the minimum needed for oec
---
appName: OEC-Example
apiKey: <OEC-Enabled-Integration-API>
baseUrl: https://api.opsgenie.com
actionMappings:
Custom:
sourceType: local
filepath: "/executable"
The appName is just a name and is beneficial if you end up having multiple OEC services on a single server.
apiKey will be the API for a OEC Enabled integration i.e Zabbix for monitoring, this is not the OEC Integration apiKey.
baseUrl is defaulted to US but note to change this is using EU or Sandbox
actionMappings. There are some default names that will auto trigger currently these are ( Create, Acknowlege, Close, Example if using Create than any alert that is created from the apiKey will trigger this action.
Custom actions need to executed within the portal, in the above example Custom, would require an action to be added to the Integration called Custom.
g:) After changing the oec configuration file, you must restart the service
systemctl restart oec
Was this helpful?
Thanks!
Philip Avery
4 comments