Traefik Ingress Controller On Premise with Cert-manager certificate renewal
Hello there !
Recently I had to deploy a Baremetal Kubernetes Cluster for a Client.
With baremetal Kubernetes comes some challenges , one of these is networking.
NOTE: You can skip to the configuration process if you already know what you want , an just need the explanations !
This Post was writter in September 2020 , using a 1.18 Kubernetes cluster and deploying Traefik v2.3 and Cert-Manager 1.16
In a Managed Kubernetes Cluster , the networking is integrated is somewhat transparent , you can expose the one of you service on the internet via a LoadBalancer service.
Networking is one of the “advantages” of a Managed Kubernetes Cluster, but we are not going to discuss the advantage of a Managed Kubernetes here, plenty of blog posts already do it on the web !
In a Baremetal Cluster, or Any non-managed Kubernetes Cluster, Exposing a service on the internet is not done that smoothly, since you loose the ability to create and use smoothly LoadBalancer you have to explore other option to expose your beautifull application to the internet.
In a Baremetal Cluster, the configuration will be done on the Ingress Controller(aka the logical component that will manage incoming network in your cluster), thing that I will do in this Blog post, by configuring Traefik.
The Ingress controller aka the logical component that will manage incoming network in your cluster.
Kubernetes , since 1.19 have augmented the capabilities of their basic ingress controller, I do recommend to consider using it, if you have basics needs.
I chose Traefik, for it’s performance, the integration with cert-managed, the easy installation using nodeport (and the cool web interface !).
If you want help, in the choice making of your ingress, I can recommend you the site kubedex , and his article on the subject.
Also , check the official doc that list the most advanced ingress controllers:
Please note that , the choices I made are based on my situation and need of my client , I might not suit you. And if you are in the same spot as me , and you have another/better solution, feel free to reach out to me!
First, we are going to see the option we have, and the one I chose. And to Finish we are going to configure the Traefik ingress.
1 .Difficulties of exposing Baremetal Kubernetes Cluster services to the internet
As said a the start of this Blog post, in a Baremetal Cluster you loose the ability to dynamically create a LoaBalancer, and thus, exposing service to the internet become a little challenge.
Yes, you could Create a LoadBalancer in front of your cluster (and I do recommend to do this, as it is a Best Practice, for a Lot of Reasons , but I am sure that you already tought of this !).
But, you still need to expose your cluster in a way or another, as the LoadBalancer need to point to something, an endpoint of your cluster.
Here is the official documentation on the subject of services.
In our case we can consider two options : HostPort and NodePort.
Since HostPort are a big security and resilience Loophole , you cannot use hostport in production context. (Not using hostport is a Kubernetes Best Practice ).
The best choice is to use NodePort.
But using nodePort in Kuberntes have one big flaw : you can only expose port from range 30000–32767.
Using Nodeport crate other challenges, as well in term of networking management (the port range is not really common, and not really practical, you will likely have to make a port translation at some point) and in security (as a port on your cluster is open). Good Article about NodePort flaws
But since your are on premise, you have no other choice but to use NodePort, and to enforce strong security policy on your network, host and front Load-Balancer.
We will configure Traefik to bind itself to the port 32080 (for http) and port 32443 (for https).
So , In the end my infrastructure looked like this :
The domain name of my client pointed to the LoadBalancer public IP.
The LoadBalancer made the port forwarding from port 80 to 32080 and to port 443 to 32443, all requests were distributed to the NodePort Endpoints.
The NodePort Service then forwarded the request to the Traefik Service , that, based on the rules I defined when I created a IngressRoute , will then forward the request to the wanted service.
2. Deployment
Please note that you are not committed to use cert-manager or to deploy a certificate to use Traefik , I just described the way I configured it.
2.1. Deployment of Cert-Manager
I will not cover the creation of a certificate with Cert-Manger , because setting up a challenge really depend on your configuration . I wrote another article on how to setup a DNS01 challenge if your domain is hosted on OVH. Here is the link .
Once Cert-Manager is configured , you can issue a Certificate like so:
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: sample_name # set the name of the certificate
labels:
app.kubernetes.io/name: # set the name of the certifictate
app.kubernetes.io/tier: backend
app.kubernetes.io/managed-by: Opsspec:
dnsNames:
- sample.domain.com # add here all the domain name you want to validate
issuerRef:
name: sample_issuer # reference here the issuer created in the configuration of the cert-manager
kind: Issuer # set here if you created an issuer or a CLusterIssuer
secretName: gotest-certificate #set here the name of the secret that will contain the certificate , it will be created by cert-manager
Now verify that the certificate is created by typing kubectl get certificates, if the certificate state is true everything is good. If you have any problems with the creation of the certificate, follow this link, that will guide you through the troubleshooting process.
2.3. Configuration of the Traefik helm charts
To deploy Traefik, we will use the official helm chart provided by the company behind traefik, containous.
We will configure the helm charts by customizing the value.yaml file , I will go through all the options that we need to set.
First clone the official github repo to get the value.yaml file.
additionnalArgument:”— providers.kubernetesingress.ingressClass=traefik-cert-manager”: to allow the use of cert-manager
additionalArgument:“ — log.level=DEBUG” : to enable debug logging, you can customize the level of logging
additionalArgument:“ — metrics.prometheus=true” : enable Prometheus metric endpoint
port:web:nodePort: 32080 : here we set the nodeport of the http port , that port will be exposed to the internet , you can use the one you wish , s long as it is on the authrized kubernetes port range
port:websecure:nodePort: 32443 : Here we set the nodePort for the https port , same as the http port , it will be exposed on the internet.
servicetype: NodePort : The type of service that we will use , we set the value as NodePort
As plain words are not usefull understanding yaml, you can find my github repo contain this value.yaml file here.
To put this in simple word , we added logging and enabled Prometheus Monitoring , we configured the traefik helm to create and use a service of type NodePort , and we setted the port pour the http port as 32080 and https port to use 32443.
You can modify other values to correspond to your needs add other arguments to additionalArguments for example.
2.4. Deployment of the traefik helm chart
#Add the containous traefik helm chartshelm repo add traefik https://containous.github.io/traefik-helm-chart#Updating our helm repo
helm repo update#Creation of the traefik namespace
kubectl create namespace traefik#Deployment of the helm chart , while provinding the value.yaml
helm install --namespace traefik trefik trefik/traefik --values .value.yaml
2.5. Creation of a Ingress route with a test Application
To test your newly installed traefik ingress you can deploy a test application.
You can apply this yaml, that will deploy a deployment of a go application I wrote that return the date and time when accessing the port 8080 and a service to expose the application.
# Source: geowallet-backend-mobile/templates/geowallet-backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: gotest
labels:
app.kubernetes.io/name: gotest
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: gotest
template:
metadata:
labels:
app.kubernetes.io/name: gotest
spec:
containers:
- name: gotest
image: "popopame/gotest:latest"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
---
# Source: geowallet-backend-mobile/templates/geowallet-backend-service.yaml
apiVersion: v1
kind: Service
metadata:
name: gotest-service
labels:
app.kubernetes.io/name: gotest-service
spec:
type: NodePort
selector:
app.kubernetes.io/name: gotest-service
ports:
- protocol: TCP
port: 8080
targetPort: 8080Then you can deploy an ingress route :
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: gotest #Name of the ingressRoute
namespace: gotest
spec:
entryPoints:
- web #traefik endpoint that will be used , since this is not a secured endpoint , we use the web endpoint
routes:
- match: Host(`gotest.example.io`) #rule for the routing
kind: Rule
services:
- name: gotest-service # Name of the service that will be the endpoint of the ingress route
port: 8080 # port of the endpoint serviceDon’t forget to modify the routes: -match: Host to the domain name pointing to your cluster Public IP.
I created a subdomain for this tutorial , but adapt to your needs !
2.6. Creation of an ingressRoute with ssl termination
Setting up ssl termination with traefik is pretty straight forward , you only need to provide to traefik the secret name containing the certificate.
If you want to set up automatic certificate renewal , you can deploy cert-manager. I wrote a Blog post explaining how to do so:
If you do not want to set up automatic renewal of the certificate , you can create a secret like this:
apiVersion: v1
kind: Secret
metadata:
name: my_certificate
type: Opaque
data:
tls.crt: <Write HERE YOUR TLS.CRT BASE64 ENCODED>
tls.key: <WRITE HERE YOUR TLS.KEY BASE64 ENCODED>Now , to deploy the ingressRoute :
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: gotest-secure #Name of the ingressRoute
namespace: gotest
spec:
entryPoints:
- websecure #traefik endpoint that will be used , since we are using a certificate , we use the websecure
routes:
- match: Host(`gotest.example.io`) #rule for the routing
kind: Rule
services:
- name: gotest-service # Name of the service that will be the endpoint of the ingress route
port: 8080 # port of the endpoint service
tls:
secretName: gotest-certificate # name of the secret containing the certificateAgain don’t forget to modify the host path
You have now a fully functional traefik ingress on your baremetal kubernetes cluster , Congratulations !
Now , the next steps would be to add middleware to your ingressroute if you need , explore in depth the possibility traefik , creating resources limitations for your pod and configuring prometheus Monitoring !
Godspeed !
Main Links Used:
