Open source tools perform a very specific task, and the source code is openly published for use or modification free of charge. I’ve written about DevOps multiple times on this blog. I reiterate the point that DevOps is not about specific tools. It’s a philosophy for building and improving software value streams, and there are three principles: flow, feedback, learning.
The philosophy is simple: Optimize for fast flow from development to production, integrate feedback from production into development, and continuously experiment to improve that process. These principles manifest themselves in software teams as continuous delivery (and hopefully deployment), highly integrated telemetry, and learning and experimentation drive the culture. That said, certain tools make achieving flow, feedback, and learning easier. You don’t have to shell out big bucks to third party vendors though. You can build a DevOps value stream with established open source tools.
Let’s start with the principle flow and what the open source community has to offer for supporting continuous delivery. In this article, we’ll cover the top 20 open source tools to achieve DevOps success. But to dive deeper in deployment pipelines and the role different tools, check out Cloud Academy’s DevOps – Continuous Integration and Continuous Delivery (CI/CD) Tools and Services Learning Path. 
Open Source Continuous Delivery
1. Gitlab is a great project for source control management, configuring continuous integration, and managing deployments. Gitlab offers a unified interface for continuous integration and deployment branded as “Auto DevOps.” Team members can trigger deploys or automatically created dedicated environments for a pull-request and see test results all within the same system.
2. Kubernetes and Docker are associated tools like docker-compose to make it easy to maintain development environments and work with any language or framework. Kubernetes is the go-to container orchestration platform today, so look here first for deploying containerized applications to production (and dev, test, staging, etc).
3. Spinnaker is designed for continuous delivery. Spinnaker removes grunt work from packaging and deploying applications. It has built-in support for continuous delivery practices like canary deployments, blue-green deploys, and even percentage based rollouts. Spinnaker abstracts away the underlying infrastructure so you can build a continuous delivery pipeline on AWS, GCP, or even on your own Kubernetes cluster.
Infrastructure-as-Code
The underlying infrastructure must be created and configured regardless of it being on a cloud provider or container orchestration. Infrastructure-as-code is the DevOps way.
4. Terraform (from Hashicorp) is the best tool for open source infrastructure-as-code. It supports AWS, GCP, Azure, Digital Open, and more using a declarative language. Terraform handles the underlying infrastructure such as EC2 instances, networking, and load balancers. It’s not intended to configure software running on that infrastructure. That’s where configuration management and immutable infrastructure tools have a role to play.
5. Packer (also from Hashicorp) is a tool for building immutable infrastructure. Packer can build Docker images, Amazon Machine Images, and other virtual machine formats. Its flexibility makes an easy choice for the “package” step in cloud based deployment processes. You can even integrate Packer and Spinnaker for golden image deployments.
6-9. Ansible, Chef, Puppet, and SaltStack are configuration management tools. Each vary slightly in design intended uses. They’re all intended to configure mutable state across your infrastructure. The odds are you’ll end up mixing Terraform, Ansible, and Packer for a complete infrastructure-as-code solution. Cloud Academy’s Cloud Configuration Management Tools Learning Path gives you an overview of configuration management, and then introduces you to three of the most common tools used today: Ansible, Puppet, and Chef. Cloud Academy’s Ansible Learning Path, developed in partnership with Ansible, teaches configuration management and application deployment. It demonstrates how Ansible’s flexibility can used to solve common DevOps problems.
Open Source Telemetry
The SDLC really starts when code enters production. The DevOps principle of feedback calls for using production telemetry to inform development work. Or in other words: use real time operational data such as time series data, logs, and alerts to understand the reality and act accordingly. The FOSS community supports multiple projects to bring telemetry into your day-to-day work.
10. Prometheus is a Cloud Native Computing Foundation (CNCF) project for managing time series data and alerts. It’s integrated into Kubernetes, another CNCF project, as well. In fact, many of the CNCF projects prefer Prometheus for metric data. Support is not limited to CNCF projects either. Prometheus is a strong choice for many different infrastructures because it uses an open API format, includes alert support, and integrates with many common components.
11. Statsd is a Prometheus alternative for time series data. Prometheus uses a pull approach. This is good for understanding if a monitored system is unavailable but requires registering new systems with Prometheus. Statsd on the other hand uses a push model. Any system can push data into a statsd server but data is sent over UDP. Statsd, unlike Prometheus, only support time series data, so you’ll need another tool to manage alerts.
12. Grafana is for data visualization. Projects like Prometheus and statsd only handle data collection. They rely on other tools for visualization. There is where Gafana comes in. Grafana is a flexible visualization system with integrations for popular data sources like Promotheus, Statsd, and AWS Cloudwatch. Grafana dashboards are just text files which makes it a natural fit for infrastructure-as-code practices.
13. The Elastic Stack is a complete solution for time series data and logs. The Elastic Stack uses ElasticSearch for time series data and log storage paired with Kibana for visualization. Log stash connects and transforms logs from various components like web server logs or redis server logs into a standard format.
14. Flutend is another CNCF telemetry project. It acts like a unified logging layer for ingress, transformation, and routing. Data steams may be forwarded to multiple sources like statsd for real time interactions or sent to S3 for archiving. Fluentd supports many data sources and data outputs. Projects like Fluentd are especially useful for connecting disparate to a standard set of upstream tools.
15. Jaeger is a distributed request tracing project compatible with Open Tracing. Traces track individual interactions within a system across all instrumented components with latency and other metadata. This is a must for micro service and other distributed architectures since engineers can pinpoint where, what, and when.
Expanding Out
The third way of DevOps calls for continuous improvement through experimentation and learning. Once the continuous delivery pipeline is established along with telemetry to improve velocity, quality, and customer satisfaction. Here are some projects that help teams improve different aspects of their process.
16. Chaos Monkey is a project by Netflix to introduce chaos into running systems. The idea is to introduce faults into system to increase reliability and durability. This is part of the principles of chaos engineering and further described in Release It! and Google’s Site Reliability Engineering book. The idea that willingly breaking your production environment may sound foreign but doing so will reveal unknowns and train teams to design away possible failure scenarios. You don’t have to go all in at once either. You can rules and restrictions so you don’t destroy your production environment until you’re ready.
17. Vault by Hashicorp is a tool for securing, storing, and controlling access to tokens, passwords, certificates, encryption keys and other sensitive data using a UI, CLI, or HTTP API. It’s great for info-sec minded teams looking for a better solution than text files or environment variables.
Building and deploying software
You’ll encounter some of these tools building and deploying software. This list isn’t exhaustive by any means.
18.Nomad is light-weight Kubernetes alternative.
19.GoCD is another deployment pipeline and CI option.
20.The serverless framework opens the door into an entirely new architecture. Just consider the list of CNCF projects. You’re likely to uncover tools for scenarios you never considered. DevOps-focused teams will assuredly use a mix of FOSS and proprietary software when building their systems. Engineers must understand how the different projects fit into their overall architecture and leverage them for best effect.
Also keep in mind these projects are not infrastructure specific. You can use them for your on-premises infrastructure, AWS, GCP, or Azure systems. Cloud Academy’s Terraform Learning Path teach students to achieve DevOps success with Terraform and infrastructure-as-code, covering AWS and Azure. Engineers can learn these tools and keep skills portable across different setups.
Don’t get lost in tooling though. You can achieve DevOps success irrespective to the underlying tools if the right culture is in place — check out the DevOps Playbook – Moving to a DevOps Culture. The secret is to build on the philosophy that values flow, feedback, and learning and realizes their practices via tools. Learn the ideas, build a culture, and the rest will sort itself out.
