You already know this: devops is worth your time and energy. Once you get it mostly right, devops processes and technologies help you get developers’ code from their laptops to the customer faster than anyone imagined even five years ago. There are lots of reasons for it, but speaking as a developer I think one important reason is that it automates routine, error prone steps in the development process.
Can that automation (and process discipline) apply to other parts of the software development process? I think so.
My goal in writing this post is to start a conversation about how that same devops mindset, both the processes and technologies, can apply not only to moving bits from a developer’s laptop to a production machine but how the discipline can apply to licensing third-party dependencies for developers.
One developer’s perspective on the value of the devops discipline
For me, writing strongly from the developer’s side as opposed to the operations side of the dev/ops “portmanteau” (seriously from the Wikipedia entry), devops’s value is that it gives me both processes and tooling to automate away the tedious and error-prone aspects of development to make the process as low-friction as possible.
As a developer, I know it takes a little discipline and requires me to try ensure that every git push my code has solid code. But when I do, then I’m confident not only in my code but the whole code base because I know the build will break if the unit tests fail. I’m even more confident about the build when integration tests run. Moving further up the release, the devops approach leads me to believe that the nodes that support my application in production have lots of fail over and resiliency so that the code I write for the customers stays available. Put simply, even though it makes me think a little up front, devops takes a lot of friction out of the process and technology of getting quality code from my laptop to the end user.
So it would be even more awesome if the process made adding new dependencies similarly frictionless.
There’s hidden friction in adding new third party dependencies. No, really.
Wait. How could adding a new dependency be any more fluid with modern dependency management tools like Bower, Ivy, or the eponymous Maven? I’m telling you there’s some kind of friction here? Yes, yes I am. The vast majority of developers think nothing of adding a new dependency (assuming they like the technology, of course). Seriously, when’s the last time you checked a dependency’s license before including it? I’m a lawyer and I’ve never been consistent about it.
But adding each new dependency does add friction. You’re bringing in new licenses, new source code from unknown contributors, and a new legal risk every time. (Don’t get me wrong, I’m very pro-open source.)
Nobody checks licenses, yet it’s the state of the art — a very immature state. I’d argue that the in licensing process we’re following is roughly at the maturity level that our industry was at before we ran automated unit tests. Remember when you’d write the code and then just see if it got past the QA team? If you were testing before deploying, it was all ad hoc –just like we’re doing now as an industry when it comes to bringing in external dependencies.
Software developers in almost all industries (some are pretty tight about this) do one of the following:
- Just include the dependency and never even considered whether there could be an issue or just hope there’s not going to be an issue.
- Build it, and then let the lawyers review the dependencies on their lawyer schedule sometime before go-live and simply run the risk that the engineers will have to scramble if the lawyers veto something.
- Ask (when we remember to) the lawyers to review dependencies on a case by case basis before adding the dependency, which can really slow down development and cause developer moral issues among your most talented developers.
- The engineering team maintains some kind of white or black list of licenses or specific dependencies that developers can or can’t use, which needs to be manually consulted each time (unless you’re using my Maven license-check plugin!).
Since every time a developer brings in a new third-party dependency, that developer binds the company to a new licensing agreement, all these options are bad ones — especially in light of the speed, assurance, and power of modern devops approaches.
In a best case scenario, the developers will have incorporated third-party dependencies that create no licensing/intellectual property risk, which is to say you’ll get lucky. In another scenario, the company will need to scramble to audit and prove that its intellectual property (remember, your IP in software is partly a compilation of other people’s property in those third-party dependencies) is clean if the company want to sell or cash out. In a worst case scenario, the company could sued for infringement. With all the outcomes, wouldn’t it have been better if you had the stability, speed, and assurance that a devops-driven discipline promises?
How could third-party dependency licensing be easier and more reliable with devops?
I think there are a lot of ways to solve this problem using devops processes and technologies. Let’s say you have a Jenkins build/delivery pipeline with these steps:
- A continuous integration step, where every developer check in gets built and deployed, assuming the build passes all tests and the code meets other requirements (like PMD, jslint, whatever).
- A release candidate step, in which release versions of the code are cut and deployed for developers and other interested parties to vet before sending to a quality assurance environment.
- A quality assurance/QA step in which the code is deployed by the QA team to a controlled environment for full testing
- A staging step in which the release candidate is staged for production (maybe in a blue-green style)
- A production/go live step, which should be self-explanatory.
To the extent that a development team makes conscious license-based choices about what third-party dependencies to incorporate, they too often do it too late in the process. Imagine that you have a solid release candidate of your application in staging when your in house counsel suddenly delivers the bad news that you can’t use an open source library because the community it came from has poor contributor licensing practices.
Now you have to scramble to replace the dependency and start the pipeline over again. In a best case scenario, you’ve only wasted time and the costs of rework. In a worst case scenario, it could cause delays significant enough to trigger breach of contract issues with your clients when you miss deployment dates.
In an ideal world, every check in would either raise flags or break the build if the developer incorporates an unknown third party artifact or one that carries a risky open source license with it (my license-check Maven plugin does this). But this introduces significant friction for developers, who you don’t necessarily want to distract with licensing issues; in a developer-centric world, we want them to focus primarily on picking code coding the best technologies they can and run with it. Let the managers and lawyers fuss about things like licensing.
One proposal for keeping third party license checks easy for developers while letting managers and lawyers making conscious choices
My proposed solution is to break the release step into two linked jobs (or new steps to the new Jenkins workflow plugin). First, there would be a job to raise flags for managers or even counsel when new dependencies are introduced. Ideally, this technology could review a number of factors — not just the license (or non-license) attached to the artifact in the Maven repository. Second, there would be some kind of human approval or checkpoint to allow the build to proceed.
I’ll readily concede that my proposal slows the development process a bit when developers are trying to cut a release and get their artifact ready to test; still I think using a bit of technology and a bit of human process right at that phase of the development cycle accomplishes goals that are hard to make play nicely:
- letting developers focus on technology without worrying too much about licensing and contractual issues,
- forcing product teams (and counsel) to make conscious decisions about in licensing choices, and
- catching problematic issues as early as possible in the process so engineers have as much time as possible to mitigate.