Use Jira to drive automated release processes

[#1]

Background

At the beginning of my company's establishment, due to the size of the team, the operation and maintenance work is still in the "slash and burn" mode, usually most of the release work is to use some simple scripts to publish through ssh or remote desktop. In this mode of operation, not only is the publishing efficiency low and the work intensity is very high, the release version record is also undocumented, and the resource utilization is unknown.

With the explosive growth of the company's business, the number of servers is increasing, the number of applications on the line is increasing, and the daily release volume is increasing sharply. It is often seen that the developmentline up to wait for the operation and maintenance to go online, or accompanied the development of staying up late to carry out the code on the line and so on.

This article describes the automated publishing system to solve these problems and improve the efficiency of the entire team. The goal is to program some periodic, repetitive, and regular work, reduce operation and maintenance costs, and improve operation and maintenance efficiency through standardization, automation, architecture, and process optimization.

Requirements

• Support for various language code distribution (IIS, Java, Nodejs, Python, Windows Service), etc.
• Support both Windows and Linux systems
• Supports multiple publishing requirements such as single release, one-click release, grayscale release, expansion release, and rollback release.
• Support multi-machine room and cloud deployment
• Processes are customizable and easy to modify
• Support RBAC rights management
• Support release history backtracking
  
  

Design

• The overall idea of using a micro-service architecture avoids high coupling and facilitates future expansion
• Each application only needs to maintain its own business logic
• Each application provides a standard, RESTful-compliant API interface
• Unified use of JSON data format for transmission
• Try to use an operation and maintenance tool that can support cross-platform support

System Structure

 

Jira

The Jira system is used to control the release process. We customized Jira Workflow and controlled permissions through the user Group and Role. Development engineers submit release task , develop leader approvals, test through SIT tests..., development, testing can complete the entire process from SIT to PROD release on Jira, and all operations are documented.

WebUI

WebUI allows users to create publishing tasks and provide more detailed data display than Jira system, which is convenient for development, testing, operation and maintenance for one-stop monitoring, tracking and problem handling.

CMDB

The CMDB provides data support for the publishing system, such as basic information of the application, deployment environment information, etc., which are obtained from the CMDB.

Configuration Management Center

The configuration management center is responsible for saving configuration information of various applications, such as database user names and passwords. When publishing, it is responsible for packaging and uploading the configuration to the FTP server.

Product release system

We have 4 deployment environments: SIT, UAT, Stage, PROD. The SIT and UAT environments are deployed by testers through Jenkins, and our release system is responsible for the deployment of the Stage and PROD environments. After the UAT environment test is passed, the tester uploads the UAT product into a zip package through Jenkins, and the release system driver configuration management center also uploads the configuration file into a zip package and uploads it to FTP; the release system is released from the FTP at the Stage release. Download the product package and configuration package, and after the stage is successfully released, merge the product package and configuration package into a zip package and upload it to FTP for PROD release.

To support various language code releases (IIS, Java, Nodejs, Python, Windows Service, etc.), we write Salt modules or scripts for each application type. The publishing system distinguishes which module is executed by the task type. For example, the IIS application calls the PowerShell script execution, and the Java, Python, Nodejs, Static, and other application calls are executed by the Salt custom module.

Use a common startup script to reduce maintenance costs. If necessary, you can also customize the startup script to meet your requirements. When publishing, it will take precedence to find the custom startup script under the specified path, otherwise it will run with the common startup script.

After the application releases the new version, the historical version of the product package is kept locally, which is used to support the version quick rollback without downloading the product package from FTP.

Features

We use the Scrum development model, a three-week iteration, and we also put our requirements in Jira for scheduling and tracking. The following is a description of each of our iterations.

Sprint 1

[Add] Release Mode Selection [Concurrent/Rolling]
[Add] Stage release permission transition to R & D Leader
[Add] Add the release unit "Main Task" "Publish Subtask"
[Add] Supports the release order strong association configuration between "release subtasks"

Sprint 2

[Add] R&D Leader self-executing production line release
[Add] Posting failed comments in JIRA with a bold red label
[Modify] Publish subtask: < Whether to configure packaging> The default is <Yes>, supplement the description of the field
[Remove] Remove the strong association function of the release order between the "publish subtasks", and the research and development leader independently determines the release order.

Sprint 3

[Add] Release permission to add Backup function, one site supports 2 research and development Leader release
[Add] Increase the Stage publishing permission check
[Add] The version number that prompts the user to roll back before the version is rolled back, allowing editing
[Add] Release platform to increase the completion time record of each link in the release process
[Remove] Remove the CMDB edit permission of the non-operation and maintenance team

Sprint4

[Add] Release failure reason classification
[Modify] version rollback permission adjustment to R&D Leader
[Add] Publish site whitelist check
[Add] Validity of Check version number before Stg release
[Add] Check whether there is a product package before Stg is released.
[Add] Publish results and database synchronization
[Modify] Non-IIS site supports ignition function

Sprint5

[Remove] Delete the "single node publishing" function
[Add] Added "Optional Node Publishing" function