Technical Debt and its Impact on DevOps

Well, yes and no, but largely no. It’s important to understand the ways in which DevOps is not the cure for technical debt — and particularly the ways in which DevOps may produce its own, characteristic forms of technical debt.

What is technical debt in DevOps?

First, a very quick refresher: In software development, technical debt is code that is substandard (in design, execution, or both), and which may cause problems in development, debugging, or operation. It may not be buggy (and often isn’t), but it’s poor design and/or execution makes it more likely that it will eventually result in some bugs. More than anything, it can increase the time, effort, and cost of development and release.

Technical debt is a tradeoff

It’s important to note that technical debt usually isn’t just a case of bad coding by bad programmers. Very often, it represents a tradeoff: a quick-and-dirty implementation of a new feature or bug fix in order to deliver the new release on time. If delivery on time sells enough software, the eventual cost of the quick-and-dirty code may be worth it. But there will be an eventual cost, and that cost is technical debt.

CI/CD Pipeline, DevOps, and Technical Debt

DevOps (in the context of CI and CD) is often pitched as the solution to the technical debt problem, and to some degree, that claim is accurate. Manual release and manual testing lend themselves far too easily to technical-debt workarounds: servers and test machines with one-of-a-kind, handcrafted environment settings, politely ignoring (or turning off) error messages, and special handling of special cases. The process of automating these tasks forces the DevOps implementation team to confront such workarounds, with a choice between incorporating them (and the technical debt that they represent) into the new, automated system, or eliminating them, along with the underlying debt.

But what about technical debt that doesn’t actually interact with the DevOps pipeline, or that only affects developers, without any real impact on QA, deployment, or operations? A self-encapsulated blob of bad code could run through the entire pipeline without even being noticed — but it could still cause problems whenever a developer has to deal with it.

DevOps Methodology and Technical Debt

The truth is that there isn’t any development methodology that can completely eliminate technical debt, if only because the causes of such debt often lie outside of the scope of such methodologies. If it is more cost-efficient based on market conditions for a software company to include third-rate-code in a new release, then it will ship third-rate-code, no matter what development methodology it uses. That methodology may play a role in determining whether the code is refactored sooner, or later, or not at all, but even then, if the cost of refactoring is greater than the cost of the technical debt, the inferior code may remain in place.

And DevOps itself isn’t immune to technical debt, either in terms of generating its own debt, or of being affected by existing debt. If process is code in DevOps, then the code that constitutes the processes can include technical debt: scripts for testing and deployment, or any code that governs the pipeline. And DevOps, with its emphasis on rapid response and continuous release, lends itself all too easily to the kind of quick-and-dirty solutions that “we’ll fix later.” Technical debt embedded in pipeline code may be as difficult to address and refactor as technical debt in application code, and its effects may be even more pronounced, since it involves the delivery process itself, and not just the product. The open-source utilities on which DevOps often depends also carry their own technical debt, and too many DevOps teams are lax (if not outright careless) about keeping up with the current versions of those utilities.

DevOps Implementation and Technical Debt

And typically, a DevOps implementation is not a clean slate. If an organization is moving from traditional methods of development and operations to DevOps, there are plenty of opportunities for existing practices to insert themselves into the mix — and those practices, along with the workarounds required to accommodate them, will all be technical debt. Not all of the debt that is embodied in the application code will be of the kind that can be easily eliminated — or that will travel through the pipeline without causing any problems. Any application interacts with something, and usually several things — users, databases, operating systems — and even in a virtualized or containerized environment, technical debt may affect those interactions. Automated testing isn’t really immune to technical debt, either; it may have to carry over some of the special-snowflake routines from manual testing, even against the better judgement of the DevOps team.

When existing technical debt does get carried over into DevOps, it may be very difficult to tackle, in part because the mere fact that it has been carried over means that it is deeply entrenched in the product, the organization, or both — or that the people in charge of the DevOps transition lack the insight or the leverage required to keep it from being carried over (which can represent a serious problem in itself). By its nature, carried-over technical debt will also be deeply embedded in the DevOps process, because it will be incorporated into that process from the start, and all later changes to that process may be affected by that debt.

It may be that DevOps’ real strength in relation to technical debt is its fundamental dynamism. DevOps assumes that nothing is static, that everything will change and should change, and it casts itself as the appropriate framework for change. This means that existing technical debt, whether in the application code or in the process, will (at least in theory) eventually be eliminated by the same dynamic process of self-renewal that is at the core of DevOps itself. But no DevOps team can afford to assume that it is free of technical debt.

*Figure 1. Roelbob, Jan, 2016. DevOps.com.

About the Author

Michael Churchman started as a scriptwriter, editor, and producer during the anything-goes early years of the game industry, working on the prototype for the ground-breaking laser-disc game Dragon’s Lair. He spent much of the 90s in the high-pressure bundled software industry, where near-continuous release cycles and automated deployment were already de facto standards; during that time he developed a semi-automated system for managing localization in over fifteen languages. For the past ten years, he has been involved in the analysis of software development processes and related engineering management issues.