Jenkins, Continuous Integration and You: How to Develop a CI Pipeline with Jenkins

Continuous Integration, or CI for short, is a development practice wherein developers can make changes to project code and have those changes automatically trigger a process which builds the project, runs any test suites and deploys the project into an environment. The process enables teams to rapidly test and develop ideas and bring innovation faster to the market. This approach allows teams to detect issues much earlier in the process than with traditional software development approaches.

With its roots in Oracle’s Hudson server, Jenkins is an open source integration server written in Java. The server can be extended through the use of plugins and is highly configurable. Automated tasks are defined on the server as jobs, and can be executed manually on the server itself, or triggered by external events, such as merging a new branch of code into a repository. Jobs can also be chained together to form a pipeline, taking a project all the way from code to deployment, and even monitoring of the deployed solution in some cases.

In this article, we’re going to look at how to set up a simple build job on a Jenkins server and look at some of the features available natively on the server to monitor and troubleshoot the build process. This article is intended as a primer on Jenkins for those who have not used it before, or have never leveraged it to build a complete CI pipeline.

Before We Get Started

This article assumes that you already have a Jenkins server installed on your local machine or on a server to which you have access. If you have not yet accomplished this, the Jenkins community and documentation can be an excellent source of information and resources to assist you. Jenkins is published under the MIT License and is available for download from their GitHub repository, or from the Jenkins website.

Within the Jenkins documentation, you’ll find a Guided Tour, which will walk you through setting up a pipeline on your Jenkins box. One of the advantages of taking this tour is that it will show you how to create a configuration file for your pipeline, which you can store in your code repository, side-by-side with your project. The one downside of the examples presented is that they are very generic. For a different perspective on Jenkins jobs, let’s look at creating a build pipeline manually through the Jenkins console.

Creating A Build Job

For this example, we’ll be using a project on GitHub that creates a Lambda function to be deployed on AWS. The project is Gradle-based and will be built with Java 8. The principles we’re using could be applied to other code repositories, build and deployment situations.

Log in to your Jenkins server, and select New Item from the navigation menu.

Choose a name for your project, select Freestyle project and then scroll down and click OK. I’ll be naming mine Build Example Lambda. When the new project screen appears, follow the following steps. Not all of these steps are necessary, but they’ll make maintaining your project easier.

• Enter a Description for your project and describe what this pipeline will be doing with it.
• Check Discard old builds, and select the Log Rotation Strategy with the Max # of builds to keep set to 10. These are the settings I use, but you may select different numbers. Having this option in place prevents old builds from taking too much space on your server.
• We’ll add a parameter for the branch to build, and default it to master. This will allow you to build and deploy from a different branch if the need arises.
  • Select This project is parameterized.
  • Click on Add Parameter and select String Parameter.
    • Name: BRANCH
    • Default Value: master
    • Description: The branch from which to pull. Defaults to master.
• Scroll down to Source Code Management. Select Git.
  • Enter the Repository URL. In my case, I entered https://github.com/echovue/Lambda_SQSMessageCreator.git
    You may also add credentials if your Git repository is secure, but setting that up is beyond the scope of this article.
  • For the Branch Specifier, we’ll use the parameter we set up previously. Parameters are added by enclosing the parameter name in curly braces and prefixing it with a dollar sign. Update this field to read */${BRANCH}

• For now, we’ll leave Build Triggers alone.
• Under Build Environment, select Delete workspace before build starts, to ensure that we are starting each build with a clean environment.
• Under Build, select Add build step, and select Invoke Gradle script.
  • When I want to build and deploy my project locally, I’ll enter ./gradlew build fatJar on the command line. To accomplish this as part of the Jenkins job, I’ll complete the following steps.
    • Select Use Gradle Wrapper
    • Check From Root Build Script Dir
    • For Tasks, enter build fatJar
• Finally, I want to save the Fat Jar which is created in the /build/libs folder of my project, as this is what I’ll be uploading to AWS in the next step.
  • Under Post-build Actions, Select Add post-build action and choose Archive the artifacts.
  • In files to archive, enter build/libs/AWSMessageCreator-all-*
  • Finally, click on Save.

Your job will now have been created. To run your job, simply click on the link to Build with Parameters. If the job completes successfully, you’ll have a jar file which can then be deployed to AWS Lambda. If the job fails, you can click on the job number, and then click on Console Output to troubleshoot your job.

Next Steps

If your Jenkins server is hosting on a network that is accessible from the network which hosts the code repository you’re using, you may be able to set up a webhook to trigger the build job when changes are merged into the master branch.

The next logical step is to automate the deployment of the new build to your environment if it builds successfully. Install the AWS Lambda Plugin and the Copy Artifact Plugin on your Jenkins server, and use it to create a job to deploy your Lambda to AWS, which copies the jar file we archived as part of the job we built above.

When the deployment job has been successfully created, open the build job, and click on the Configure option. Add a second Post-build action to Build other projects. Enter the name of the deployment project, and select Trigger only if build is stable.

At this point, the successful execution of the build job will automatically start the deployment job.

Congrats! You’ve now constructed a complete CI pipeline with Jenkins.