The Road to Flux v2 - September update

[weaveworks]

This is the first blog post in a series which accompanies the development of the GitOps Toolkit on the road to delivering Flux v2 and more. We want to highlight what has been happening in development in the past weeks, who has contributed and what you can look forward to. And if you are curious, how to try it out for yourself.

It has been three weeks since the Flux community announced their plans for Flux v2 that has been rebuilt from scratch and based on top of the new GitOps Toolkit. If you haven’t heard the news yet, here’s what we are planning on doing:

1. The GitOps functionality in Flux and Helm Operator is now delivered by individual components, to enable and support far more use-cases and to simplify development.
2. Everything is driven by Custom Resources for more control and better observability.
3. We built the GitOps Toolkit from the ground up incorporating the most requested features:
   • Support for multiple source Git repositories
   • Operational insight through health checks, events and alerts
   • Multi-tenancy capabilities, such as applying each source repository with its own set of permissions

In April this year we started with early experimentation for building a proof-of-concept, which was first shown at the GitOps Days virtual event in May, followed by a request-for-comments period and roadmap discussions in June. Since then the Flux community has come a long way.

Our highlight in this episode: the road to Helm Operator v2

Since the announcement, the team has made great strides, especially in the area of Helm: around 90% of all the work related to building the “Helm Operator v2” is done. The Helm Controller now has support for values from Secret and ConfigMap resources, and support for strategies for handling different kinds of failure. Its API package is also available as an independent Go Module. Failure handling is a big one. It means that you can extensively configure how to automatically remediate an installation, test, or upgrade failure. This includes actions that should be performed to remediate (rollback or uninstall), the number of retries, and even if the release has failed and retries have expired, you can debug the issue more easily. In addition, it is possible to ignore test failures if necessary (as opposed to remediate after a failed “helm test”).

Also the API to support Helm releases, based on source API, has been discussed and designed. In particular, providing values from sources and support for Helm charts from Git. So what’s left for Helm Controller to be on par with today’s Helm Operator? Only implementing support for referring to an alternative chart values file. And of course lots of testing. Before we tell the world they can transition to Helm Operator v2, we need to hear from current users and integrate their feedback.

If you are using Helm Operator right now, it would be fantastic if you could test the Helm Controller: simply check out https://toolkit.fluxcd.io/guides/helmreleases/ and give us feedback on how the journey went for you (find contact info at the bottom of this blog post).

In other news

Here’s a recap of what happened in the project overall in the past three weeks:

• Support for HelmChart artifacts built from GitRepository sources.
• Various bug fixes, e.g. Notification Controller: fix to the Prometheus scraping endpoint.
• We were approached by members of the Grafana Tanka project who decided to call their binary “tk” so we decided to go with “gotk” from now on.
• In the future we will also be able to enable ARM support; one blocker for this is that Flux shelled out to the kustomize binary, which has been fixed now.
• Mozilla SOPS decryption support is coming!

More projects starting to look at using go-git-providers: eksctl, libgitops, and through libgitops: ignite, cluster-api-provider-existinginfra and wksctl.

Since Flux become a CNCF Sandbox project about a year ago, we formally started the process to review the project with the CNCF - the PR is worth a read if you are interested in where the project stands as a whole and what happened in the last year. The amount of care and contributions our large community of users and contributors has shown is somewhat mind-boggling. We are certainly very pleased with what we have achieved.

Development-wise the focus for the next few weeks is going to be:

• Figuring out the final form of Flux v2
• The next big milestone: Bringing the image automation on par with Flux v1
• Increasing general test coverage

If you want to help us shape the future of any of these, please reach out - we would love to hear from you.

Special thanks to our contributors - here’s what they landed:

• Martin H Berwanger improved the gotk install script and automated the upgrades of controller component dependencies in the Toollkit.
• Sean Eagan contributed the conditional remediation on failed actions to the helm-controller.
• Philip Laine implemented notifications for Github commits.
• We also had many folks jump into conceptional work in our Github Discussions:
  • Shane Sveller is looking into non-curl-bash installation methods
  • Moshe Immerman had a look at the image update automation design
  • Sean Eagan has been working on various parts of the Helm controller planning. Calle Petterson, Jonas Arneberg and Steve Hipwell as well.
  • Daniel Morgan in SOPS support and Philip Laine in e2e testing, exposing image fields in kustomization resources and a possible Job controller.

There is still lots where you can help shape the future of the GitOps Toolkit. Some of the important discussions are:

• What is the final form of Flux v2?
• Prometheus metrics for resource status
• Design for image update automation
• Reconciling errors vs failure to make progress

We are looking forward to working with you.

Here is how you can get involved:

• Join our upcoming dev meeting on Sept 10
• Talk to us in the #flux channel on CNCF Slack
• Join the planning discussions
• And if you are completely new to the GitOps Toolkit, take a look at our get Started guide and give us feedback

View the full article