DevOps From the Trenches: Lessons learned from the DevOps community

Introduction: Transparency Across DevOps Culture, Process and Toolchain

With DevOps currently at the peak of the hype curve according to Gartner’s Hype Cycle for Enterprise Architecture, 2015, there’s little question that DevOps has become a highly charged, and some would argue, over-marketed term, giving rise to confusion and misunderstandings of what DevOps really is. Organizations in turn risk time, resources and money attempting to solve the wrong problem.

As you’ll hear in the book, DevOps is a strategy for developing what is becoming a universally recognized process for managing continuous delivery, but it is not that process. Continuous Delivery (CD) is the process whereby software is developed iteratively and delivered in stages along a deployment pipeline. The benefits of CD for larger IT organizations are clear and widely embraced.

As you’ll learn, DevOps is the culture, process and tools used in the CD cycle. From an agile perspective it facilitates teams to work harmoniously together with the goals of software quality and continuous improvement. The process, as just stated is continuous delivery, but the goals are similar: To deliver quality software more often while teams increase their velocity. That leaves tools.

The DevOps Toolchain

Tools used along the CD pipeline range from compilers, debuggers, and build tools to configuration management, monitoring and deployment tools. Depending on whether the platform is on-premise or in the cloud, tools can be categorized into team collaboration, application development, system level, and API-level tools. These tools include:

• Integrated Development Environments – This can be Visual Studio on local workstation or a hosted development environment like salesforce.com’s Force.com DE environment.
• Source control – Repositories like Artifactory and Github provide an accessible, controlled means for storing scripts, code and other key assets.
• Continuous integration servers – CI servers like Jenkins merge code and automate the build, test, and commit stages, providing early feedback.
• Configuration management – Tools like Puppet and Chef help maintain state and consistency between staging and production environments.
• Issue tracking – These tools can help improve responsiveness and visibility.
• Monitoring tools – Provide clear, shared responsibility for relevant parts of service health.
• Deployment tools – For building out environments and regularly updating systems.
• Collaboration and Planning – Tools like Kanban provide project management while offering transparency into the process.

Troubleshooting Code in a DevOps World

While increasing code coverage for unit tests and automated testing have greatly improved the quality of software, clean code doesn’t mean software always behaves as expected. A faulty algorithm or failure to account for unforeseen conditions can cause software to behave unpredictably. Within the CD pipeline, troubleshooting can be difficult, and in cases like debugging in a production environment it may not even be possible.

The good news is that virtually all of the tools across the CD pipeline emit machine data, which are typically captured in log files. At the applications level, logs capture the stream of events of your app’s running processes including events generated by the application server and libraries. System logs capture events such as restarting a crashed process, and API logs can provide useful information when deploying new code.

Sumo Logic provides developers and others on DevOps teams with visibility into critical issues across the DevOps tool chain using event logs. Gone are the days of setting breakpoints in a debugger and stepping through code. Once a collector is configured for your app, logs are fed through the Sumo Logic service. Sumo Logic delivers out-of-the box dashboards, reports, saved searches, and field extraction for popular data sources. When an app is installed in Sumo Logic, these pre-set searches and dashboards are customized with source configurations and populated in a folder selected by the user.

Through Sumo Logic’s LogReduce® pattern-matching algorithm selects patterns as well as outliers that help developers quickly spot patterns in event logs, CI server logs and troubleshoot their applications in real time. So they can spend less time troubleshooting and more time developing code.

Sumo Logic address five common use cases:

• Increase availability and performance. Sumo Logic enables issues to be identified before they impact the application and customer. Precise, proactive analytics quickly uncover hidden root causes across all layers of the application and infrastructure stack.
• Provide real time insights. With Sumo Logic DevOps teams can easily extract machine data insights to provide greater intelligence around their customers, products, and application usage. These insights provide a more accurate and complete analysis for business users.
• Accelerate cloud deployment. Sumo Logic enables DevOps to automate and speed the development and deployment process for cloud-based applications. Companies can rapidly detect, identify and resolve application issues.
• Decrease app time to market. With Sumo Logic, DevOps teams can implement a consistent release process resulting in on-time releases. They can easily identify application issues and configuration changes across development, test and deployment environments.
• Enforce compliance. Sumo Logic delivers a simple, proactive and automated process to audit and investigate operational, security and regulatory compliance incidents. All data is centralized, secured, and easily analyzed in real-time through a single, highly scalable solution.

Summary

The following articles in this eBook first appeared on devops.sumologic.com, and are written by members of the Sumo Logic DevOps community. This eBook looks not only at DevOps, but technologies like containerization that are changing how DevOps teams configure environments for deployment. The opinions are theirs: We hope you’ll find useful insights to problems DevOps teams are challenged with today.