The tech industry is full of workarounds, and you probably rely on one or more. There is no problem with that per se, but it's important to review your workarounds from time to time because there could be a new standard/intuitive way to make it.
The Problem
A couple of years ago I had a use case where a single domain has 2 sub-paths each of them using its own service in different namespaces. Let's see this example:
example.com/app => service "backend" in namespace "app" example.com/blog => service "wordpress" in namespace "blog"
The problem was that the Ingress object is namespaced which means that it interacts with services within the same namespace. Also, only one ingress object per host/domain is allowed.
So at that time I found a generic solution which looks like a workaround. Actually, by thinking about it now, it was not a bad workaround. It depends on how you manage your infrastructure, and you can think about it as a centralized vs decentralized approach.
The Solution
So here are the 2 ways to route Ingress traffic across namespaces in Kubernetes. The 1st is a generic way that will work with any Ingress controller. The 2nd relies on the Ingress controller capabilities NGINX Ingress Controller by NGINX, Inc. (NOT Ingress-NGINX Controller by Kubernetes).
Option One: Generic method - ExternalName Service
This method relies on native Kubernetes ExternalName Service which is simply a DNS CNAME! This method is centralized where it uses the normal Ingress object in addition to ExternalName Service within the same namespace as a bridge to the services in any other namespace.
The following is an example of that setup with a single Ingress resource and 2 ExternalName services (3 endpoints which are /, /coffee, and /tea).
Config for shop.example.com including the 2 sub-paths /coffee and /tea in addition to the root /.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: shop-ingress
namespace: shop
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- shop.example.com
secretName: shop-secret
rules:
- host: shop.example.com
http:
paths:
- path: /coffee
pathType: Prefix
backend:
service:
name: coffee-svc-bridge
port:
number: 80
- path: /tea
pathType: Prefix
backend:
service:
name: tea-svc-bridge
port:
number: 80
The coffee-svc-bridge service in the shop namespace is a CNAME for the coffee-svc service in coffee namespace:
apiVersion: v1 kind: Service metadata: name: coffee-svc-bridge namespace: shop spec: type: ExternalName externalName: coffee-svc.coffee
The tea-svc-bridge service in the shop namespace is a CNAME for the tea-svc service in tea namespace:
apiVersion: v1 kind: Service metadata: name: tea-svc-bridge namespace: shop spec: type: ExternalName externalName: tea-svc.tea
As you see, the Ingress config comes in 1 part and is normal. And use the ExternalName services as a bridge to access the services in the other namespaces.
Option Two: Controller-specific method - Mergeable Ingress Resources
The other option is using controller-specific capabilities to achieve that goal. There are dozens of Ingress controllers for Kubernetes like the Ingress-NGINX (by Kubernetes project), NGINX Ingress Controller (by NGINX, Inc.), Traefik, HAProxy, Istio, and many more.
Here I will cover only NGINX Ingress Controller by NGINX, Inc., but the idea is the same, using the controller-specific features.
If you took a look at the official Nginx docs you will find the Cross-namespace Configuration page suggests using Mergeable Ingress Resources.
That approach relies on a simple idea, there is a single Ingress resource that has all configurations related to the host/domain and that resource is called "master", and any number of the Ingress resources handles the paths under that host/domain and each of these resources is called "minion".
Each one of the master or minion can or can not contain some Ingress annotations based on their role. Here I will use here the examples from the official documentation.
Config for shop.example.com like TLS and host-level annotations.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: shop-ingress-master
namespace: shop
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.org/mergeable-ingress-type: "master"
spec:
tls:
- hosts:
- shop.example.com
secretName: shop-secret
rules:
- host: shop.example.com
Config for shop.example.com/coffee which is in the coffee namespace and routes the traffic of the coffee-svc service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: shop-ingress-coffee-minion
namespace: coffee
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.org/mergeable-ingress-type: "minion"
spec:
rules:
- host: shop.example.com
http:
paths:
- path: /coffee
pathType: Prefix
backend:
service:
name: coffee-svc
port:
number: 80
Config for shop.example.com/tea which is in the tea namespace and routes the traffic of the tea-svc service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: shop-ingress-tea-minion
namespace: tea
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.org/mergeable-ingress-type: "minion"
spec:
rules:
- host: shop.example.com
http:
paths:
- path: /tea
pathType: Prefix
backend:
service:
name: tea-svc
port:
number: 80
As you see, the Ingress config is split into 2 parts, the host/domain config, and the paths config. Each one of them could be in a different namespace and handles the services in that namespace.
Conclusion
Maybe the first approach looks like a workaround, but for many workloads could be better and easier to follow and digest. But in general, it's good to have different ways to use what's fit better.
Enjoy :-)
