Kustomize — Validate before it is too late

Introduction

In the Kubernetes ecosystem, when you start to think about customize your manifests, you might have 2 solutions: Helm or Kustomize.

This article is not about the pros and cons of each solution. Based on your needs and your approach, you will choose the right tool for your deployment.

Kustomize is a great tool for generating manifests depending on the target environment.

In this article, we will go through an use-case to be sure that the result of the “compilation” of Kustomize will be deploy-able inside your k8s cluster and catch errors (typo, k8s objects not following the API manifest, …) before a true deployment.

How frustrating it is when during the deployment in production, you notice a typo in your manifests and you didn’t detect it at an early stage (when you merged to master for example)?

Using Kustomize concepts, your project might follow the standard structure:

In order to apply the k8s objects in a target environment, you will execute the command:

In a CI/CD organization, depending on your deployment step, the CD pipeline will build the right manifests with kustomize and apply it in your kubernetes cluster.

As all the overlays could be different from an environment to another one, you are not 100% sure that your overlays are good/validate without a true deployment if you don’t check it before.

The idea is to detect at the earliest stage of your development that all the Kustomize configurations are verified.

A solution could be to test all the overlays when you do a Pull-Request to the master branch. According to that, you will be sure that the master branch will contains “build-able/deploy-able” manifests trough all your environments.

Without testing your overlays, when the CD will generate all the manifests and apply it to a cluster, the k8s api could reject it. In an “as code” mindset, you will have to fix it then do a new PR and finally trigger your CD pipeline.

What a wasting time!! Let’s validate everything at the beginning (the 1st PR).

Validate all the configurations as soon as possible

To commit validated overlays in your repository, let’s use the tool kubernetes-validate. (you could choose other tools as kubeval of course)

kubernetes-validate validates Kubernetes resource definitions against the declared Kubernetes schemas.

It is quite simple to use it. Please check the github project to get all the informations.

The idea of validating your overlays is to execute kubernetes-validate after kustomize build (logic, isn’t it?)

Add that step in your CI process and you will strengthen the codebase and your deployment process.

All resources can be found in github. Don’t hesitate to comment and give new inputs.