Why my ConfigMap data looks messy? - Kubernetes

The Problem

Have you seen this before? When you create a ConfigMap with a multiline data block!

Here is an example: When you create the following ConfigMap:

Continue Reading »

How to define GitHub Actions multiline environment variable or output - CI/CD

I'm not sure if that was a hack or undocumented feature, but I can find it now in the GitHub Actions docs.

But in the past, I needed to copy a short multiline file between GitHub Actions jobs, and I didn't want to bother with extra steps of stash/unstash stuff, so I found that you can define a multiline GitHub Actions variable!

It was as easy as this:

jobs:
  job1:
    runs-on: ubuntu-latest
    steps:
      - name: Set multiline value in bash
        run: |
          # The curly brackets are just Bash syntax to group commands
          # and are not mandatory.
          {
              echo 'JSON_RESPONSE<<EOF'
              cat my-file.json
              echo EOF
          } >> "$GITHUB_OUTPUT"

Of course, you need to be sure that the delimiter EOF doesn't occure within the value.

Then you can call that again as:

[...]
  job2:
    needs: job1
    runs-on: ubuntu-latest
    steps:
      - name: Get multiline value in bash
        run: |
          echo "${{ needs.job1.outputs.JSON_RESPONSE }}"

That's it! Enjoy! ♾️

Continue Reading »

Gomplate v4 is here! - Tools

This year, one of my nice discoveries was gomplate, a fast template renderer supporting many data sources and hundreds of functions.

And finally, gomplate v4 was released in June 2024. It has many excellent new features and additions (v3 was released in 2018!).

I will not really cover much here as it has extensive documentation. The most basic example could be:

echo 'Hello, {{ .Env.USER }}' | gomplate
Hello, ahmed

But I totally like the idea of the separation between data input (data source) and rendering, so I can generate the data with any tool and then just leave the rendering to gomplate like:

Template file:

{{- $release := ds "release" -}}
{{- with $release -}}
Version: .version
{{- end -}}

Rendering command:

RELEASE_DATA_JSON='{"version": "1.0.0"}'
gomplate \
    --config .gomplate.yaml \
    --file RELEASE.md.tpl \
    --datasource release=env:///RELEASE_DATA_JSON?type=application/json

Of course, that's still super basic, but the idea here is that I could generate the data of env var RELEASE_DATA_JSON or even generate a JSON file and read it in gomplate.

What's better? It's already available in asdf! Just add it directly or declaratively via asdf plugin manager.

asdf plugin-add gomplate

If you want a decent tool for templating, then gomplate is your tool to go.

Enjoy :-)

Continue Reading »

Do NOT use unpackaged Helm charts! - DevOps

TL;DR

Don't install/template from unpackaged Helm charts directly because Helm CLI has many bugs related to packaging and dependencies, especially bug no. helm/helm #11484 and you will get unexpected behavior.

Continue Reading »

Awesome Kustomize list - Kubernetes

After 3 years of using Kustomize intensively to manage Kubernetes manifests, I just created a list of "Awesome Kustomize" resources 🤩️

If you are using Kubernetes, this repo will be a pretty good starting point for Kustomize plugins, guides, tips and tricks, and more.

Kustomize is not a new tool, it started in 2018, and now it's built into "kubectl" (since v1.14), which means it's now considered the official method to deal with advanced Kubernetes scenarios (e.g., GitOps).

If you are SysOps, DevOps, SRE, or anyone working intensively with Kubernetes, you definitely should read more about Kustomize!

Also, if you just use "kubectl" or Helm, then consider using Kustomize. It could help you in many different ways (it's not a replacement for Helm, you can use both).

Follow the repo to get the updates 🚀️

https://github.com/aabouzaid/awesome-kustomize

Continue Reading »

Handling nullable nested values - Helm

I'd say this is one of the best tricks I've learned about Helm recently! For me, one of the most annoying things in Helm was that it's hard to handle nullable nested values!

For example, if you want to get the value of a nested key like

Continue Reading »

How to create Makefile targets with dynamic parameters and autocompletion - Make

Make is one of the oldest build automation tools ever (the original Make was created in 1976!). And since then, it got many implementations as BSD make, GNU make, and Microsoft nmake. It uses a declarative syntax, and sometimes that's the best and worst thing about it!

You probably used Make at least once. It's widely adopted in the tech industry, or as someone said, "Make is the second best tool to automate anything!". 😄️

After

Continue Reading »

2 ways to route Ingress traffic across namespaces - Kubernetes

The tech industry is full of workarounds, and you probably rely on one or more. There is no problem with that per se, but it's important to review your workarounds from time to time because there could be a new standard/intuitive way to make it.

The Problem

A couple of years ago I had a use case where a single domain has 2 sub-paths each of them using its own service in different namespaces. Let's see this example:

example.com/app => service "backend" in namespace "app"
example.com/blog => service "wordpress" in namespace "blog"

The problem was that the Ingress object is namespaced which means that it interacts with services within the same namespace. Also, only one ingress object per host/domain is allowed.

So at that time I found a generic solution which looks like a workaround. Actually, by thinking about it now, it was not a bad workaround. It depends on how you manage your infrastructure, and you can think about it as a centralized vs decentralized approach.

The Solution

So here are the 2 ways to route Ingress traffic across namespaces in Kubernetes. The 1st is a generic way that will work with any Ingress controller. The 2nd relies on the Ingress controller capabilities NGINX Ingress Controller by NGINX, Inc. (NOT Ingress-NGINX Controller by Kubernetes).

Option One: Generic method - ExternalName Service

This method relies on native Kubernetes ExternalName Service which is simply a DNS CNAME! This method is centralized where it uses the normal Ingress object in addition to ExternalName Service within the same namespace as a bridge to the services in any other namespace.

The following is an example of that setup with a single Ingress resource and 2 ExternalName services (3 endpoints which are /, /coffee, and /tea).

Config for shop.example.com including the 2 sub-paths /coffee and /tea in addition to the root /.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress
  namespace: shop
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - shop.example.com
    secretName: shop-secret
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /coffee
        pathType: Prefix
        backend:
          service:
            name: coffee-svc-bridge
            port:
              number: 80
      - path: /tea
        pathType: Prefix
        backend:
          service:
            name: tea-svc-bridge
            port:
              number: 80

The coffee-svc-bridge service in the shop namespace is a CNAME for the coffee-svc service in coffee namespace:

apiVersion: v1
kind: Service
metadata:
  name: coffee-svc-bridge
  namespace: shop
spec:
  type: ExternalName
  externalName: coffee-svc.coffee

The tea-svc-bridge service in the shop namespace is a CNAME for the tea-svc service in tea namespace:

apiVersion: v1
kind: Service
metadata:
  name: tea-svc-bridge
  namespace: shop
spec:
  type: ExternalName
  externalName: tea-svc.tea

As you see, the Ingress config comes in 1 part and is normal. And use the ExternalName services as a bridge to access the services in the other namespaces.

Option Two: Controller-specific method - Mergeable Ingress Resources

The other option is using controller-specific capabilities to achieve that goal. There are dozens of Ingress controllers for Kubernetes like the Ingress-NGINX (by Kubernetes project), NGINX Ingress Controller (by NGINX, Inc.), Traefik, HAProxy, Istio, and many more.

Here I will cover only NGINX Ingress Controller by NGINX, Inc., but the idea is the same, using the controller-specific features.

If you took a look at the official Nginx docs you will find the Cross-namespace Configuration page suggests using Mergeable Ingress Resources.

That approach relies on a simple idea, there is a single Ingress resource that has all configurations related to the host/domain and that resource is called "master", and any number of the Ingress resources handles the paths under that host/domain and each of these resources is called "minion".

Each one of the master or minion can or can not contain some Ingress annotations based on their role. Here I will use here the examples from the official documentation.

Config for shop.example.com like TLS and host-level annotations.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress-master
  namespace: shop
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "master"
spec:
  tls:
  - hosts:
    - shop.example.com
    secretName: shop-secret
  rules:
  - host: shop.example.com

Config for shop.example.com/coffee which is in the coffee namespace and routes the traffic of the coffee-svc service.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress-coffee-minion
  namespace: coffee
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "minion"
spec:
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /coffee
        pathType: Prefix
        backend:
          service:
            name: coffee-svc
            port:
              number: 80

Config for shop.example.com/tea which is in the tea namespace and routes the traffic of the tea-svc service.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress-tea-minion
  namespace: tea
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "minion"
spec:
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /tea
        pathType: Prefix
        backend:
          service:
            name: tea-svc
            port:
              number: 80

As you see, the Ingress config is split into 2 parts, the host/domain config, and the paths config. Each one of them could be in a different namespace and handles the services in that namespace.

Conclusion

Maybe the first approach looks like a workaround, but for many workloads could be better and easier to follow and digest. But in general, it's good to have different ways to use what's fit better.

Enjoy :-)

Continue Reading »

Apply Kustomize builtin transformers on a single resource - Kubernetes

Kustomize is a template-free declarative management tool for Kubernetes resources. Kustomize has 2 main concepts: Generators and Transformers. In short, the first able to create K8s manifests, and the second is able to manipulate K8s manifests. In this post, I'm interested in the Kustomize Transformers.

Continue Reading »

Delete a manifest from Kustomize base - Kubernetes

Kustomize is a template-free declarative management tool for Kubernetes resources. One of the important primitives of Kustomize is patchesStrategicMerge. It allows you to customize the base manifests according to

Continue Reading »

Use ConfigMap as a dynamic var source in Kustomize - Kubernetes

Update 25.07.2021: As mentioned before, the vars option will be deprecated in favor of replacements.

---

Kustomize is a declarative configuration tool for Kubernetes. And unlike other tools like Helm, it's "template-free", in a simple put, it's like merging dictionaries (Python) or maps (Java).

A pure declarative style could be annoying sometimes! But actually, Kustomize has a feature called

Continue Reading »

RabbitMQ vs Apache Kafka explained - General

Usually, I share my stuff here in this blog, but from time to time I share some external resources.

A couple of years ago I was

Continue Reading »

Automating my laptop system setup - Ansible

As I always say, I like to avoid manual boring work, and manual repetitive work is always boring for me! So I used to automate my system I use on my laptop with most of stuff I used to use.

Although I'm not big fan of changing my main system too much. And as I'm the author of

Continue Reading »

Detailed log view - Git

Sometimes ago, I found a nice Git combination. Which's really handy, and I use it a lot to have a pretty log overview.

Continue Reading »

Update htpasswd if changed only - SaltStack

SaltStack has a lot of states and modules, an each one has different options, but some times the state/module doesn't support a certain function, so you have to use another

Continue Reading »

Resolve redirection internally from upstream - Nginx

For a reason or another, you maybe have a situation where you need to resolve redirection internally from upstream in Nginx. i.e when you request a

Continue Reading »

Key value array in shell scripts! - Bash

I really believe if you need key-value (aka associative array) in your script, then you need to move to a real language which most probably will be Python or so.

But for some reasons, you maybe need it in one of your bash scripts. And of course will not rewrite the script just for that part!

Continue Reading »

Emulate pbcopy and pbpaste in Linux with xsel - Bash

One of nice tools I did use before are "pbcopy" and "pbpaste", which are simply copy and paste any data to and from clipboard in command line! It's a Mac OS tool, and unfortunately there is no port for Linux!

However you still can get the same functionality on Linux using

Continue Reading »

Trigger click event on elements - Native JavaScript and JQuery

Maybe I'm a Sysops guy, and don't use JavaScript on a daily basis, but JS became a mandatory language and got an elastic eco-system, so as a power user you may need it from time to time to do some fancy things :D

Continue Reading »

Efficient and quick way to delete files! - Bash

On the sidelines of previous post "An efficient and quick way to search with find, grep, and xargs", I've just remembered one of servers has an issue with sessions garbage collection! So, after a while

Continue Reading »