Just finished a weekend of being totally confused about k8s ingress. I'm glad to say that I finally managed (using my OCD powers and lots of snacks/coffee) to work out what's going on. Here's what I learned. (NOTE: I'm doing this on Minikube with no add-ons)
The confusion I experienced was made worse by several guides that skipped over the key detail that was confusing me, i.e the ingress-controller, because they either:
- Assumed I was running on GKE which handles the ingress-controller automatically
- Assumed I had enabled ingress in Minikube as an add-on which also handles the ingress-controller automatically
Now, call me old fashioned, but when I'm learning something I like to do everything myself so that I fully understand every piece.
Oddly enough, a couple of guides were plain wrong and exposed the services they were publishing via ingress as NodePort. This really confused me because the ingress controller is inside the cluster, along with the services it's routing to and therefore can contact the services "natively" on their ClusterIP. The point here (and this sounds obvious to me now but before the penny dropped it wasn't) is that the only service that needs to be exposed to the outside world when using ingress is the service that sits in front of the ingress controller itself.
The aim of this guide is to:
- Help me not forget what I learned (always good to write things down)
- Maybe help other out people that might find ingress confusing.
So here goes...
The diagram below set's out the target system:
Here's the yaml for each component, starting from the top down:
ingress-controller-service.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-controller
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: http
nodePort: 32000
selector:
app: ingress-controller
ingress-controller-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ingress-controller
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: ingress-controller
template:
metadata:
labels:
app: ingress-controller
spec:
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-backend
- --annotations-prefix=nginx.ingress.kubernetes.io
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
ingress-resource.yaml
An interesting point to note from the yaml below is that I've set the host to minikube. I originally left this blank which means that any hostname or IP specified in the URL would be acceptable. however when I did this the nginx controller kept trying to redirect me (via a 308) to https. I googled around a bit and found out that this is expected behavior for the nginx controller and that specifying a host stops this from happening. So I tried putting my minikube IP address in, kubectl complained about this stating that hostnames and not IP's had to be used in the ingress host field. This being the case I just placed an entry in my local /etc/hosts
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cluster-ingress-rules
spec:
rules:
- host: minikube
http:
paths:
- path: /hello-world
backend:
serviceName: hello-world
servicePort: 8080
hello-world-service.yaml
apiVersion: v1
kind: Service
metadata:
name: hello-world
labels:
run: hello-world
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
run: hello-world
hello-world-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello-world
labels:
run: hello-world
spec:
replicas: 1
selector:
matchLabels:
run: hello-world
template:
metadata:
creationTimestamp: null
labels:
run: hello-world
spec:
containers:
- image: gcr.io/google-samples/node-hello:1.0
name: hello-world
ports:
- containerPort: 8080
default-backend-service.yaml
apiVersion: v1
kind: Service
metadata:
name: default-backend
labels:
app: default-backend
spec:
ports:
- port: 80
targetPort: 8080
selector:
app: default-backend
default-backend-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: default-backend
labels:
app: default-backend
spec:
replicas: 1
selector:
matchLabels:
app: default-backend
template:
metadata:
labels:
app: default-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-backend
image: gcr.io/google_containers/defaultbackend:1.4
ports:
- containerPort: 8080
Applying this lot to my cluster results in the following:
----- Ingress ------------------------------------------------------------------
NAME HOSTS ADDRESS PORTS AGE
cluster-ingress-rules minikube 80 20m
----- Services -----------------------------------------------------------------
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-backend ClusterIP 10.100.69.250 <none> 80/TCP 20m
hello-world ClusterIP 10.110.199.3 <none> 8080/TCP 20m
ingress-controller NodePort 10.111.99.142 <none> 80:32000/TCP 20m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16d
----- Deployments --------------------------------------------------------------
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
default-backend 1 1 1 1 20m
hello-world 1 1 1 1 20m
ingress-controller 1 1 1 1 20m
----- Pods ---------------------------------------------------------------------
NAME READY STATUS RESTARTS AGE
default-backend-79f984dc75-2bbbf 1/1 Running 0 20m
hello-world-554998f545-wjj49 1/1 Running 0 20m
ingress-controller-5cddb96544-2k9w7 1/1 Running 0 20m
As can be seen from the above, the only service exposed to the outside world is the ingress-controller service
Running minikube service list produces the following result:
|-------------|----------------------|-----------------------------|
| NAMESPACE | NAME | URL |
|-------------|----------------------|-----------------------------|
| default | default-backend | No node port |
| default | hello-world | No node port |
| default | ingress-controller | http://192.168.99.100:32000 |
| default | kubernetes | No node port |
| kube-system | kube-dns | No node port |
| kube-system | kubernetes-dashboard | http://192.168.99.100:30000 |
| kube-system | tiller-deploy | No node port |
|-------------|----------------------|-----------------------------|
and putting http://minikube:32000/hello-world into a browser results in the message Hello Kubernetes!
And there you have it, hopefully I haven't missed anything.
