CI/CD Pipeline with BuildKite Tool

Author: Siva Kumar Ch

What is CI/CD?

CI/CD is a method to frequently deliver apps to customers by introducing automation into the stages of app development.

What is Continuous Integration?

Continuous Integration (CI) is a development practice where developers integrate code into a shared repository frequently, preferably several times a day. Each integration can then be verified by an automated build and automated tests

What is Continuous Deployment?

Continuous Deployment (CD) is closely related to Continuous Integration and refers to keeping your application deployable at any point or even automatically releasing to a test or production environment if the latest version passes all automated tests.

How CI/CD works with Buildkite Tool?

Buildkite is a platform for running fast, secure, and scalable continuous integration and deployment pipelines on your own infrastructure.

What is BuildKite Agent?

The buildkite agent is a small, reliable and cross-platform build runner that makes it easy to run automated builds on your own infrastructure. Its main responsibilities are polling buildkite.com for work, running build jobs, reporting back the status code and output log of the job, and uploading the job’s artifacts.

To work with Buildkite tool, login into https://buildkite.com/login with your credentials. If you don’t have access to Buildkite please create an account at https://buildkite.com/signup.

Requirements:

1.Buildkite Agent for automated builds

2.Maven for deployment 

3.Github for pushing the code in repository

Below are the steps to setup Buildkite Agent tool on local:

Step 1: You need to download the Buildkite Agent and setup on your Local

Step 2: Select the environment to set up your first agent

Step 3: Edit the <Root-Folder>/buildkite-agent.cfg file and add your token which is highlighted above on local buildkite agent 

example path: C:\buildkite-agent-windows-386-3.22.1\buildkite-agent.cfg

Manual Installation:

1. Download the latest Windows release from Buildkite Agent releases on GitHub
2. Extract the files to a directory of your choice (we recommend C:\buildkite-agent)
3. Edit buildkite-agent.cfg and add your token
4. Run buildkite-agent.exe start from a command prompt

Step 4: Generate SSH Key for your BuildKite Agent in cmd prompt with following command

  command:ssh-keygen -t rsa -b 4096 -C "<Email Address>"

In order to authenticate with GitHub to pull your project, you’ll need to generate an SSH key for the user running your Buildkite Agent (buildkite-agent by default).

https://buildkite.com/docs/agent/v3/ssh-keys#creating-a-single-ssh-key

Step 5: Start your Buildkite Agent from cmd prompt with the following command

            command: buildkite-agent start 

Create Repository in Github:

Login to https://github.com/ with your credentials. If you don’t have a github account, just create one at https://github.com/join?ref_cta=Sign+up

Create Pipeline for your project:

Step 1: Create a new pipeline on Buildkite 

Step 2: After the pipeline is created, you’ll be presented with instructions for setting up Webhooks on GitHub, so that Buildkite can automatically trigger builds every time code is pushed to your repository. Copy/Paste the generated Payload URL, since we’ll need it for the section below.

Prepare Github repository for Buildkite:

Step 1: Add Webhook URL

As per the Buildkite instructions, go to the Webhooks section of your GitHub repository Settings page and add the Payload URL generated by Buildkite.

Visit your Github repository’s Webhooks settings:

https://github.com/{Your Github username}/{your Github repository}/settings/hooks

Step 2: Add deploy key to repository

Visit the Deploy Keys section of your GitHub repository Settings page and add the SSH key we generated earlier.

Prepare your project for continuous deployment.

The pipeline is configured to export your package and run on GitHub commits and pull requests. All that is left is to configure Maven to use this environment variable, and publish to Cloudhub.

Step 1:  Add the .buildkite.settings.xml file in your project main and  Add <server></server> settings under <servers> in settings.xml (do this per id used below)

Create .buildkite folder in main project and under .buildkite folder create settings.xml (or)  copy the settings.xml file from maven(.m2) and paste in .buildkite folder.

Step 2: Add your Github repository to distributionManagement 

Add the repository that you created in Gihub to the distributionManagement section of your pom.xml. The id that we set here in distributionManagement has to match the id of <server></server> settings under <servers>  in our .buildkite.settings.xml file.

Step 3: Mule Maven Plugin for CloudHub Deployment

The deployment of Mule applications can be managed using Maven. The Mule Maven plugin allows integrating the packaging, testing, and deployment of Mule applications with the Maven lifecycle.

It helps in automating the deployment.

Add the plugin in pom.xml if not added already.

In the plugin, add the configuration tag to define parameters for CloudHub deployment.

To deploy the application, run the maven deploy phase and one argument -DmuleDeploy.

Command: mvn clean package deploy -DmuleDeploy

Step 4: Push Code from Github

Step 5: When you push new code to your project, or create a pull request, it will trigger your Buildkite Agent to pull the code and run the step you’ve defined in your pipeline.

Conclusion 

We have seen how to create a CICD pipeline with Buildkite Tool and deployment to Cloudhub.

References: https://buildkite.com/docs/tutorials/getting-started