A lot of people seem confused about how Ingress works in Kubernetes and questions come up almost daily in Slack. It’s not their fault, either — unfortunately the Kubernetes documentation is pretty weak in this area. Updating the documentation itself is probably a better use of time than writing this, I just prefer this …medium. I’ll try to do both, though!
What is Ingress?
It’s probably worth a quick introduction to clear things up. Traditionally, you would create a LoadBalancer service for each public system you want to expose. This can get rather expensive. Ingress gives you a way to route requests to services based on the request host or path, centralizing a number of services into a single entrypoint.
Ingress Resources
Ingress is split up into two main pieces. The first is an Ingress resource, which defines how you want requests routed to the backing services.
For example, a definition that defines an Ingress to handle requests for www.mysite.com and forums.mysite.com and routes them to the Kubernetes services named website and forums respectively would look like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
spec:
rules:
- host: www.mysite.com
http:
paths:
- backend:
serviceName: website
servicePort: 80
- host: forums.mysite.com
http:
paths:
- path:
backend:
serviceName: forums
servicePort: 80
The documentation on the Ingress resource itself really isn’t bad. You can find official examples here.
Here is where things seem to get confusing, though. Ingress on it’s own does not really do anything. You need something to listen to the Kubernetes API for Ingress resources and then handle requests that match them. This is where the second piece to the puzzle comes in — the Ingress Controller.
Ingress Controllers can technically be any system capable of reverse proxying, but the most common is Nginx. A full example Nginx Ingress Controller (and LoadBalancer service) is as follows. Please note that if you are not on a provider that supports LoadBalancer services (ie. bare-metal), you can create a NodePort Service instead and point to your nodes with an alternative solution that fills that role — a reverse proxy capable of routing requests to the exposed NodePort for the Ingress Controller on each of your nodes.
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
spec:
type: LoadBalancer
selector:
app: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: ingress-nginx
spec:
replicas: 1
template:
metadata:
labels:
app: ingress-nginx
spec:
terminationGracePeriodSeconds: 60
containers:
- image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
name: ingress-nginx
imagePullPolicy: Always
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend
This is essentially an Nginx image that monitors Ingress resources for requested routes to serve. One callout you may have noticed is that it specifies a --default-backend-service as a startup argument, passing in nginx-default-backend. This is wanting a service that can simply handle returning a 404 response for requests that the Ingress Controller is unable to match to an Ingress rule. Let’s create that as well with the specified name.
kind: Service
apiVersion: v1
metadata:
name: nginx-default-backend
spec:
ports:
- port: 80
targetPort: http
selector:
app: nginx-default-backend
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nginx-default-backend
spec:
replicas: 1
template:
metadata:
labels:
app: nginx-default-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
image: gcr.io/google_containers/defaultbackend:1.0
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
ports:
- name: http
containerPort: 8080
protocol: TCP
I said Ingress is made up of two main components and we introduced three new things. This is because the default backend isn’t really a piece of Ingress itself, but rather is something that the Nginx Ingress Controller requires. Other Ingress Controllers won’t necessarily have this component.
Wiring it Up
Assuming you’ve created the Ingress Controller above with the dependent default backend, your Ingress resources should be handled by the LoadBalancer created with the Ingress Controller service. Of course, you would need services named website and forums to route to in the above example.
As a quick test, you can deploy the following instead.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: demo-ingress
spec:
rules:
- host: mysite.com
http:
paths:
- backend:
serviceName: nginx
servicePort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
ports:
- port: 80
targetPort: 80
selector:
app: nginx
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: echoserver
image: nginx
ports:
- containerPort: 80
To test things out, you need to get your Ingress Controller entrypoint.
For LoadBalancer services that will be:
kubectl get service ingress-nginx -owide
For NodePort services, you can find the exposed port with:
kubectl describe service ingress-nginx
You should then be able to test the Ingress Controller entrypoint to get the a 404 response from the default backend:
LoadBalancer: curl [ELB_DNS]
NodePort: curl [NODE_IP]:[NODE_PORT]
If you match the Ingress rules, you will receive a default Nginx response:
LoadBalancer: curl -H 'Host:mysite.com' [ELB_DNS]
NodePort: curl -H 'Host:mysite.com' [NODE_IP]:[NODE_PORT]
Wrap Up
Hopefully this sheds a little light on the confusing Ingress resources. Once you have the Ingress Controller set up and respecting Ingress requests, I highly recommend going back to the Ingress documentation with a fresh understanding to learn different ways to route requests. There is also a rather complete example of Ingress in the Kops repository.
As an alternative to Nginx (and the default backend!) you can look at systems like Traefik, which has native support for Ingress. For a writeup on how to integrate this properly with Kubernetes, I highly recommend reading Patrick Easters article Using Traefik with TLS on Kubernetes. It’s an excellent writeup that follows Kubernetes best-practices.
Another option to solve this problem is to use a system like Kong. While Kong does not support Kubernetes Ingress, it solves the same problem — you deploy it as a LoadBalancer service and register mappings to services with it. Only instead of using Ingress, you use the Kong API.
References
Everything here can really be found in the official documentation for the most part — it’s just that it’s not organized properly yet. Here are some links to the pieces I used so you can reference the real docs rather than trust a random guy on Medium. 😉
- Official Ingress Documentation: While not perfect, it’s still the official documentation and I expect it will improve greatly with so much interest in Ingress
- Nginx Ingress Controller: I’ve essentially taken this one and only updated it to be a
Deploymentinstead of aReplicationController. - Default HTTP Backend: Again, I’ve essentially just updated this example to use a
Deploymentinstead of aReplicationController. - Full Example: This lives in the Kops repository and is actually based on my changes with some tweaks added by Justin, the primary maintainer.
copied from – > https://medium.com/@cashisclay/kubernetes-ingress-82aa960f658e