Managing Kubernetes with GitOps in a multi-cluster, multi-cloud world

[weaveworks]

There are many general blog posts on Kubernetes out there and quite a few are written with a couple of assumptions in mind. First, that the author has assumed you’ll only have one cluster to contend with. Second, they might also assume that you are willing to create and deploy configuration files manually.

It’s easy to see why as an author, you’d make this assumption. It keeps things simple and allows you to focus on the principles you’re trying to explain. But the real world isn’t like that. In real life, you’ll probably be dealing with multiple clusters. They might even be running across more than one cloud or backend. And you can guarantee you’ll be under time and operational pressure that rules out manual and error prone configuration.

And that is one of the reasons we are big proponents of GitOps - it represents a great solution to the above mentioned problems.

GitOps in a nutshell

Rather than add to all these assumptions by assuming you know what GitOps is, here’s a quick overview:

• GitOps is an operating model for Kubernetes that involves storing all configuration code in Git, just as you would application code.
• Git thereby becomes the single source of truth for your application infrastructure. Software agents constantly monitor the running application for any divergence from the system described in Git. If they spot anything, they fire off alerts so you can fix it.
• As a result, developers and DevOps teams can work faster and more productively, bring new features to market faster and breaking things less often.
• If the worst happens, rollbacks are easy, as everything is recorded in Git.
• Even security and compliance are improved, thanks to RBAC and the constant generation of audit trails.

For in depth reading - visit our what you need to know about GitOps page.

Multiple clusters and multiple clouds

If you’re deploying anything at scale, you’re going to need some level of automation in order to achieve consistency, no matter how many clusters or clouds you need to manage. That’s where GitOps comes in. By using a GitOps workflow to manage your Kubernetes clusters, you can solve some of the most common challenges associated with multi-cluster environments. Let’s look at them in turn.

Diverse toolchains

One way to achieve consistency in large-scale environments is to mandate use of the same tools. But there’s a downside: Developers and IT professionals often want to choose the tools that work best for them.

Forcing a toolset upon all teams can adversely affect morale. Worse still, it can also deprive some teams of the ability to use the best tools for the job. Some may want to use one container registry, for example, while others prefer another. Some delivery pipelines may be best hosted in the cloud, while others work better on-premise.

With GitOps policy management, you don’t have to choose between consistency and freedom of tool choice. Instead, teams can use the tools of their choice to build their delivery pipelines, provided they use Git to store their cluster management data. When all configuration information is stored in Git, it can be pushed out in a consistent and scalable way, no matter how many clusters are in use, or which toolsets are behind your delivery chains.

Managing dozens of clusters

Another benefit is the ability to apply the same configurations to as many clusters as necessary. You can automate it too. There is no need to generate or apply custom YAML files for each cluster in your infrastructure. You can let tools do it for you so your team can focus instead on crafting the best configurations for your workloads. This approach not only makes managing dozens of clusters much more efficient, but it also increases security by reducing the risk of manual configuration mistakes.

Achieving consistency between local environments and the cloud

In many organizations today, application infrastructure includes a mix of on-premise resources and public clouds. 

Managing clusters consistently on heterogeneous infrastructure often involves the use of different tools for each part of the infrastructure – as many tools that work in one type of environment simply don’t work in others. Kops works only with certain clouds, for example. You might not even be able to use kubectl consistently, because some commands vary depending on the infrastructure and Kubernetes version.

With GitOps, you can avoid these constraints. You can write a single set of configuration files, tailoring them where necessary to address the nuances of different clouds or on-premise clusters, then deploy them to all parts of your infrastructure with a single tool and process. DataScan, a financial services company and customer of Weave Kubernetes Platform explains in this case study, how they operate secure on-premise production and cloud based development environments.

Streamlining access control across the organization

When manually managing multiple clusters spread over multiple clouds, access control and roles can get out of hand, resulting for example in a different set of roles and access-control policies for each cluster. This is a recipe for oversights and security holes.

With GitOps, it becomes feasible to define a single set of access-control policies and apply them to all clusters – not just because you can apply the configurations in an automated way, but also because centralized policy management minimizes the number of people with access to your clusters in the first place.

Deploying new clusters

Even with an automated tool, significant effort is required to configure and secure a Kubernetes cluster. It’s one reason clusters can end up with too many namespaces, rather than being broken into multiple clusters.

GitOps policy management solves this issue by removing the need to configure clusters manually, instead allowing engineers to apply pre-set configurations that are tailored for different types of workload (e.g. development, testing and production). In this way, GitOps lets you add new clusters just as easily as you can spin up a new virtual server in the cloud, or pull a container image.

Where to go next

GitOps is not a product. It’s an operations model – a combination of software and associated working practices you can adopt today. If you want to start with an open source project, we recommend Flux (a CNCF incubating project that was originally created by us) or join one of our Weave online user group sessions.

If you are looking for vendor support and an accelerated adoption, Weaveworks professional services team offers comprehensive GitOps discovery, design and deploy packages for any Kubernetes. We are happy to get you from zero to GitOps.

View the full article