08/08/2022

2 ways to route Ingress traffic across namespaces - Kubernetes

The tech industry is full of workarounds, and you probably rely on one or more. There is no problem with that per se, but it's important to review your workarounds from time to time because there could be a new standard/intuitive way to make it.

The Problem

A couple of years ago I had a use case where a single domain has 2 sub-paths each of them using its own service in different namespaces. Let's see this example:

example.com/app => service "backend" in namespace "app"
example.com/blog => service "wordpress" in namespace "blog"

The problem was that the Ingress object is namespaced which means that it interacts with services within the same namespace. Also, only one ingress object per host/domain is allowed.

So at that time I found a generic solution which looks like a workaround. Actually, by thinking about it now, it was not a bad workaround. It depends on how you manage your infrastructure, and you can think about it as a centralized vs decentralized approach.

The Solution

So here are the 2 ways to route Ingress traffic across namespaces in Kubernetes. The 1st is a generic way that will work with any Ingress controller. The 2nd relies on the Ingress controller capabilities NGINX Ingress Controller by NGINX, Inc. (NOT Ingress-NGINX Controller by Kubernetes).

Option One: Generic method - ExternalName Service

This method relies on native Kubernetes ExternalName Service which is simply a DNS CNAME! This method is centralized where it uses the normal Ingress object in addition to ExternalName Service within the same namespace as a bridge to the services in any other namespace.

The following is an example of that setup with a single Ingress resource and 2 ExternalName services (3 endpoints which are /, /coffee, and /tea).

Config for shop.example.com including the 2 sub-paths /coffee and /tea in addition to the root /.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress
  namespace: shop
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - shop.example.com
    secretName: shop-secret
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /coffee
        pathType: Prefix
        backend:
          service:
            name: coffee-svc-bridge
            port:
              number: 80
      - path: /tea
        pathType: Prefix
        backend:
          service:
            name: tea-svc-bridge
            port:
              number: 80

The coffee-svc-bridge service in the shop namespace is a CNAME for the coffee-svc service in coffee namespace:

apiVersion: v1
kind: Service
metadata:
  name: coffee-svc-bridge
  namespace: shop
spec:
  type: ExternalName
  externalName: coffee-svc.coffee

The tea-svc-bridge service in the shop namespace is a CNAME for the tea-svc service in tea namespace:

apiVersion: v1
kind: Service
metadata:
  name: tea-svc-bridge
  namespace: shop
spec:
  type: ExternalName
  externalName: tea-svc.tea

As you see, the Ingress config comes in 1 part and is normal. And use the ExternalName services as a bridge to access the services in the other namespaces.

Option Two: Controller-specific method - Mergeable Ingress Resources

The other option is using controller-specific capabilities to achieve that goal. There are dozens of Ingress controllers for Kubernetes like the Ingress-NGINX (by Kubernetes project), NGINX Ingress Controller (by NGINX, Inc.), Traefik, HAProxy, Istio, and many more.

Here I will cover only NGINX Ingress Controller by NGINX, Inc., but the idea is the same, using the controller-specific features.

If you took a look at the official Nginx docs you will find the Cross-namespace Configuration page suggests using Mergeable Ingress Resources.

That approach relies on a simple idea, there is a single Ingress resource that has all configurations related to the host/domain and that resource is called "master", and any number of the Ingress resources handles the paths under that host/domain and each of these resources is called "minion".

Each one of the master or minion can or can not contain some Ingress annotations based on their role. Here I will use here the examples from the official documentation.

Config for shop.example.com like TLS and host-level annotations.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress-master
  namespace: shop
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "master"
spec:
  tls:
  - hosts:
    - shop.example.com
    secretName: shop-secret
  rules:
  - host: shop.example.com

Config for shop.example.com/coffee which is in the coffee namespace and routes the traffic of the coffee-svc service.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress-coffee-minion
  namespace: coffee
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "minion"
spec:
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /coffee
        pathType: Prefix
        backend:
          service:
            name: coffee-svc
            port:
              number: 80

Config for shop.example.com/tea which is in the tea namespace and routes the traffic of the tea-svc service.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress-tea-minion
  namespace: tea
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "minion"
spec:
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /tea
        pathType: Prefix
        backend:
          service:
            name: tea-svc
            port:
              number: 80

As you see, the Ingress config is split into 2 parts, the host/domain config, and the paths config. Each one of them could be in a different namespace and handles the services in that namespace.

Conclusion

Maybe the first approach looks like a workaround, but for many workloads could be better and easier to follow and digest. But in general, it's good to have different ways to use what's fit better.

Enjoy :-)

Continue Reading »
Powered by Blogger.

Hello, my name is Ahmed AbouZaid, I'm a passionate Tech Lead DevOps Engineer. 👋

With 16+ years of open-source contributions, 12+ years of professional hands-on experience in DevOps, and an M.Sc. in Data Engineering from Edinburgh Napier University (UK), I enjoy facilitating the growth of both businesses and individuals.

I specialize in Cloud-Native and Kubernetes. I'm also a Free/Open source geek and book author. My favorite topics are DevOps transformation, automation, data, and metrics.

Contact Me

Name

Email *

Message *

Start Your DevOps Engineer Journey!

Start Your DevOps Engineer Journey!
Start your DevOps career for free the Agile way in 2024 with the Dynamic DevOps Roadmap ⭐

Latest Post

2023 Highlights

Image generated with Craiyon . Finally, 2023 is over! What a year! One more crazy year, but it was the culmination ...

Popular Posts

Blog Archive