Opiniated Deployment Pipelines

Sachin Nagalgawe
CI/CD Pipelines

Opiniated Deployment Pipelines

Problem Statement

Simplify the existing complicated app deployment process on a hybrid-cloud environment by bringing in opiniated pipelines to cater specifically to very specific use-cases for deploying applications. It needed to support Server-less (Cloud Foundry), Kubernetes as well as VM deploys.

Need

The IT department of the company used to develop variety of applications mainly Spring Boot Micro-services which each team was deploying in their own way. Our team found out that there is a minimum common denominator to the process involved in deploying the micro-services. Each of the developers were logging into the cloud and deploying their applications using their own process and method. This was causing issues when for a given micro-service, accountability had to be established, history of releases needed to be maintained, quality gates needed to be established in order to promote the code to higher environments.

The intent was to reduce the developer's effort and time with these deployment tasks and not add additional layer of complexity and cost of having a release management team to manage the releases. However, this application did cater to the release management team as well reducing their time and effort as well to deal with complex delepyment tools as opposed to minimalistic approach for deploying the micro-services.

Architecture

We wanted a pipeline approach for the deployment of these micro-services and we had to finalize a piepline tool which would be the backbone of our whole release/deployment pieplines. Had the following options to choose from:

  • Jenkins
  • Bamboo
  • Concourse

With its flexibility, ease of use, support for Dockers, Kubernetes and Shell Scripts we opted for Concourse as our underlying pipeline tool.

At a high-level, the solution had the following layers:

  • Configuration
  • Business
  • Execution
  • Cloud

Configuration

The configuration layer dealt with the per environment configuration needed for the app to be deployed based on the target cloud (Cloud Foundry, Kubernetes, VM). The expectation was that all configuration that was needed to deploy the app on a specific environment on a specific cloud was provided to the solution.

Business

This was the orchestration layer which dealt with orchestration of execution of the pipelines, maintaining history, authenticating, authorizing users to deploy, providing analytics.

Execution

The execution layer delat with the execution of the Concourse pipelines, managing its status and notifying the business layer for the same.

Cloud

The cloud layer is of the actual cloud layer with various environments. Depending on the micro-service, its business use-case, and the budget, a appropriate cloud was chosen. The solution supported:

  • Pivotal Cloud Foundry
  • Azure Kubernetes Service
  • Google Kubernetes Engine
  • AWS Elastic Kubernetes Service
  • Virtual Machines

Deployment Architecture Diagram

Benifits

Around 40% of the developer's time and effort was saved as opposed to the older method of deployments and around 20% of developer's overall time and efforts were saved, hence making this solution a success.