For something as popular as the burgeoning field of DevOps, it’s a conundrum that such an important topic is still ill-defined. One thing is starting to be agreed upon, and that is that DevOps is a methodology, not just a profession. Some people also call it a culture, a feedback loop consisting of software development and operations monitoring that a company must engage in if it is to successfully adapt and grow.
As recently as 10 years ago, development and operations were siloed in organizations, as if they were two different crops that a farmer wanted to keep separate but still available on the farm. The tech industry realized it was better to make the two tightly integrated, and the silos came down, for the better. Development pushed code into production, while the ops team kept it running. Meanwhile, QA would work with customers and developers to improve the product, and the DevOps cycle was born.
This is not how DevOps works, in case you’re wondering. (Though developers would like to think it is.)
It’s 2017 now, and everyone recognizes that the industry has shifted to this superior integrated mindset, but it’s still difficult to piece together a cohesive platform of tools that meet all your organization’s DevOps needs. Why? Because it’s so broad and all-encompassing. And no one company does it all.
Go ahead. Read a dozen different articles about DevOps, get a dozen different definitions. It makes sense, then, that you might need a dozen different tools or so to manage all your development, operations, QA, and security ops needs.
David Sayers, technical services director at Midvision, wrote over at DZone that his “suggestion is that we should try to narrow the definition of what DevOps is in an attempt to create some boundaries so that it can be considered more of a discipline than a utopian goal.”
Along those same lines, here’s our stab at creating some boundaries and reining in on the amazing but somewhat elusive DevOps phenomenon. We present a quick list of the most salient DevOps disciplines and a sampling of the most popular tools you need to build a functional DevOps platform for your needs. We try to call out each tool’s source code philosophy (open or closed) and pricing model to help you make an informed decision.
Configuration Management
When you need to ensure your infrastructure is configured properly and up to date, you need configuration management, or just config management for short. Config management is used for everything from setting up a startup’s server array to the military managing the lifecycle of its weapons systems.
Popular tools include Ansible (open source, free and paid tiers), Chef (open source and closed source, paid), Puppet (open source, free and paid options), and Vagrant (open source, free).
Artifacts
In addition to being Indiana Jones’ favorite things, artifacts refer to the various byproducts that result from software development: binary code, .jar files, processed .js, .css files, etc. Artifacts also include old code, documentation, models and diagrams, etc. They’re basically the leftovers or “artifacts” remaining from the development process. There are a lot of tools out there (called “artifact repositories”) to help manage these artifacts. A few include JFrog Artifactory (open source, paid), Apache’s Maven (open source, free), or just using Amazon AWS as a free artifact repository.
Provisioning
Whether you manage your own datacenter, launch hundreds of virtual machines, use Docker (open source, free and paid versions) to manage your containerized services, or manage a hybrid cloud, there have never been more options for provisioning. There’s AWS EC2 (closed source, paid), Google Compute Engine Instances (closed source, paid), Rackspace Cloud Servers (closed source, paid), or HashiCorp Nomad (open source. free).
Version Control
Though college professors don’t typically teach it (because it’s relatively new), git is a version control system created by Linus Torvalds, a name you might also recognize as the father of the Linux kernel. Git helps developers manage codebase changes, control who has access to push updates, and as a way to track down when bugs may have been introduced.
The most popular online version control systems are GitHub (closed source, free and paid), and BitBucket (closed source, free and paid).
Deployment & Testing
Perhaps the most exciting part of developing is deploying your code and testing it, whether it’s in a pre-release alpha state, a beta test, or in production. With all the aspects that go into deployment and testing, it can get overwhelming trying to manage it all. That’s where Jenkins (open source, free), CircleCI (closed source, free and paid), or Travis-CI (open source, free and paid), to name a few, come in hand. These types of tools help automate the process of viewing test results, seeing which changes in code are shippable, automatic notification to the dev/ops/QA teams, feedback collection, and more.
Monitoring
Monitoring: Ah, the DevOps pro’s boon! We’ve saved the best for last, because this category is a doozy. Monitoring, or keeping tabs on, well, everything, is the most splintered of the DevOps disciplines, and for intuitive reasons: There’s so much data to collect. Logs, application metrics, security incidents, networks, databases . You name it.
There are a ton of tools out there, each one for a different need: Datadog (closed source, free and paid) for infrastructure monitoring, Pingdom (closed source, paid) to check website and app performance, PagerDuty (closed source, paid) for incidents, or New Relic (closed source, free and paid) for application performance monitoring.
We can shamelessly add ourselves add to the list with Lumberjack (closed source, paid), our centralized log management tool, and Watchdog (closed source, free), our Essential Alerts™ tool that notifies you of crucial system metrics that impact uptime and availability.
The DevOps Tools Franken-Suite and the Future of DevOps Platforms
The Franken-Monitor, by Bernd Harzog at Virtualization Practice
You might be thinking, with all the logging tools out there, wouldn’t it be great if someone made a platform that had a tool for each DevOps discipline? A platform that helped DevOps live up to its mantra of being non-siloed and agile? After all, having to manage a dozen vendors or so for all your DevOps tools needs isn’t exactly efficient.
Currently the DevOps tools landscape is sort of like Frankenstein, a scientific experiment to patch together a platform out of disparate tools from various vendors. The result is often a monster for DevOps people — and for their budgets, vendor management, and security compliance. This is especially true for DevOps monitoring, as we pointed out above.
After all, having to pay for a tool for each need would be like having to pay for each individual ride at Disneyland. In 2017, our development/operations/QA/security is no longer siloed, but our tools are.
Lumberjack (paid) and Watchdog (free), our first two DevOps cloud monitoring products, integrate seamlessly for your logging and hardware alerts. In the future, the Blue Matador platform will encompass each of the DevOps disciplines mentioned above. Look for more tight integrations in the Blue Matador suite in the future.
In the meantime, we hope our infographic will help your team (or you, if you’re the sole engineer doing DevOps in your organization, which is a common scenario for small businesses) make sense of the DevOps Franken-suite that ends up happening when you piece together several different vendors and tools together.
