Kubernetes Ingress de-mysified

Just finished a weekend of being totally confused about k8s ingress. I'm glad to say that I finally managed (using my OCD powers and lots of snacks/coffee) to work out what's going on. Here's what I learned. (NOTE: I'm doing this on Minikube with no add-ons) 

The confusion I experienced was made worse by several guides that skipped over the key detail that was confusing me, i.e the ingress-controller, because they either:
- Assumed I was running on GKE which handles the ingress-controller automatically
- Assumed I had enabled ingress in Minikube as an add-on which also handles the ingress-controller automatically

Now, call me old fashioned, but when I'm learning something I like to do everything myself so that I fully understand every piece.

Oddly enough, a couple of guides were plain wrong and exposed the services they were publishing via ingress as NodePort. This really confused me because the ingress controller is inside the cluster, along with the services it's routing to and therefore can contact the services "natively" on their ClusterIP. The point here (and this sounds obvious to me now but before the penny dropped it wasn't) is that the only service that needs to be exposed to the outside world when using ingress is the service that sits in front of the ingress controller itself.

The aim of this guide is to:
- Help me not forget what I learned (always good to write things down) 
- Maybe help other out people that might find ingress confusing.

So here goes...

The diagram below set's out the target system:

Here's the yaml for each component, starting from the top down:

ingress-controller-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: ingress-controller
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: http
    nodePort: 32000
  selector:
    app: ingress-controller

ingress-controller-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: ingress-controller
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-controller
  template:
    metadata:
      labels:
        app: ingress-controller
    spec:
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
            - --annotations-prefix=nginx.ingress.kubernetes.io
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80

ingress-resource.yaml

An interesting point to note from the yaml below is that I've set the host to minikube. I originally left this blank which means that any hostname or IP specified in the URL would be acceptable. however when I did this the nginx controller kept trying to redirect me (via a 308) to https. I googled around a bit and found out that this is expected behavior for the nginx controller and that specifying a host stops this from happening. So I tried putting my minikube IP address in, kubectl complained about this stating that hostnames and not IP's had to be used in the ingress host field. This being the case I just placed an entry in my local /etc/hosts

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cluster-ingress-rules
spec:
  rules:
  - host: minikube
    http:
      paths:
      - path: /hello-world
        backend:
          serviceName: hello-world
          servicePort: 8080

hello-world-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: hello-world
  labels:
    run: hello-world
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    run: hello-world

hello-world-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-world
  labels:
    run: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      run: hello-world
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: hello-world
    spec:
      containers:
      - image: gcr.io/google-samples/node-hello:1.0
        name: hello-world
        ports:
        - containerPort: 8080

default-backend-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: default-backend
  labels:
    app: default-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-backend

default-backend-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-backend
  labels:
    app: default-backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-backend
  template:
    metadata:
      labels:
        app: default-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-backend
        image: gcr.io/google_containers/defaultbackend:1.4
        ports:
        - containerPort: 8080

Applying this lot to my cluster results in the following:

----- Ingress ------------------------------------------------------------------
NAME                    HOSTS      ADDRESS   PORTS     AGE
cluster-ingress-rules   minikube             80        20m

----- Services -----------------------------------------------------------------
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
default-backend      ClusterIP   10.100.69.250   <none>        80/TCP         20m
hello-world          ClusterIP   10.110.199.3    <none>        8080/TCP       20m
ingress-controller   NodePort    10.111.99.142   <none>        80:32000/TCP   20m
kubernetes           ClusterIP   10.96.0.1       <none>        443/TCP        16d

----- Deployments --------------------------------------------------------------
NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
default-backend      1         1         1            1           20m
hello-world          1         1         1            1           20m
ingress-controller   1         1         1            1           20m

----- Pods ---------------------------------------------------------------------
NAME                                  READY     STATUS    RESTARTS   AGE
default-backend-79f984dc75-2bbbf      1/1       Running   0          20m
hello-world-554998f545-wjj49          1/1       Running   0          20m
ingress-controller-5cddb96544-2k9w7   1/1       Running   0          20m

As can be seen from the above, the only service exposed to the outside world is the ingress-controller service

Running minikube service list produces the following result:

|-------------|----------------------|-----------------------------|
|  NAMESPACE  |         NAME         |             URL             |
|-------------|----------------------|-----------------------------|
| default     | default-backend      | No node port                |
| default     | hello-world          | No node port                |
| default     | ingress-controller   | http://192.168.99.100:32000 |
| default     | kubernetes           | No node port                |
| kube-system | kube-dns             | No node port                |
| kube-system | kubernetes-dashboard | http://192.168.99.100:30000 |
| kube-system | tiller-deploy        | No node port                |
|-------------|----------------------|-----------------------------|

and putting http://minikube:32000/hello-world into a browser results in the message Hello Kubernetes!

And there you have it, hopefully I haven't missed anything.