In the rapidly evolving world of cloud-native applications, Kubernetes has emerged as the leading container orchestration platform, offering a robust API and extensibility framework. One of the core components that allow Kubernetes to manage resources effectively is the Controller. In this article, we will dive into what custom controllers are, why they are essential, and how to build one from scratch.
Understanding Controllers in Kubernetes
A Kubernetes Controller continuously watches the state of your cluster, comparing the current state of its resources to the desired state defined by the user. If there’s a discrepancy, the controller takes action to rectify the situation. Controllers work behind the scenes to manage various aspects of your environment, from scaling applications to handling network traffic.
Custom Controllers: Definition and Use Cases
While Kubernetes API already includes several built-in controllers (like Deployments and StatefulSets), there are scenarios where custom controllers are necessary:
- CRDs (Custom Resource Definitions): If your application has unique requirements that don’t fit into existing Kubernetes constructs, you can create your own CRDs and controllers.
- Automation: Custom controllers can automate repetitive tasks specific to your workload, improving operational efficiency.
- Integration: You can integrate third-party systems by creating controllers that manage their lifecycle based on Kubernetes events.
Prerequisites
Knowledge
Before diving into custom controllers, you should have a solid understanding of:
- Kubernetes architecture and concepts like Pods, Deployments, Services, and Volumes.
- Go programming language, as most Kubernetes client libraries and examples are written in it.
- Familiarity with Kubernetes client-go library.
Environment Setup
- Kubernetes Cluster: You can set up a local cluster using Minikube or Kind.
- Go: Install Go on your local machine.
- kubectl: Ensure you have
kubectlconfigured to communicate with your Kubernetes cluster.
Creating a Custom Controller
Step 1: Define a Custom Resource Definition (CRD)
The first step in creating a custom controller is defining a CRD. A CRD allows you to extend Kubernetes with your own resource types.
Here’s an example of a CRD that defines a CronJob resource:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: cronjobs.example.com
spec:
group: example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
schedule:
type: string
image:
type: string
scope: Namespaced
names:
plural: cronjobs
singular: cronjob
kind: CronJob
shortNames:
- cjStep 2: Create the Controller
-
Initialize a Go Module:
Start a new Go module for your controller:
mkdir cronjob-controller
cd cronjob-controller
go mod init example.com/cronjob-controller -
Install Client Libraries:
Install necessary dependencies:
go get k8s.io/[email protected]
go get k8s.io/[email protected]
go get k8s.io/[email protected] -
Write the Controller Logic:
Create a new file,
main.go, and start writing your controller. Here’s a simplified example:package main
import (
"context"
"os"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/source"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/controller/controlleroptions"
"sigs.k8s.io/controller-runtime/pkg/client"
)
type CronJobReconciler struct {
client.Client
}
func (r *CronJobReconciler) Reconcile(req reconcile.Request) (reconcile.Result, error) {
ctx := context.Background()
cronJob := &corev1.Pod{}
if err := r.Get(ctx, req.NamespacedName, cronJob); err != nil {
return reconcile.Result{}, client.IgnoreNotFound(err)
}
// Implement your reconciliation logic here
return reconcile.Result{}, nil
}
func main() {
mgr, err := manager.New(config.GetConfigOrDie(), manager.Options{})
if err != nil {
os.Exit(1)
}
if err := controller.New("cronjob-controller", mgr, controlleroptions.Options{}).
Watch(&source.Kind{Type: &corev1.Pod{}}, &handler.EnqueueRequestForObject{}); err != nil {
os.Exit(1)
}
if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
os.Exit(1)
}
}
Step 3: Deploy the Controller
-
Dockerize Your Application:
Create a Dockerfile to build your controller image.
FROM golang:1.20 AS builder
WORKDIR /app
COPY . .
RUN go build -o cronjob-controller .
FROM alpine:latest
WORKDIR /root/
COPY --from=builder /app/cronjob-controller .
CMD ["./cronjob-controller"] -
Build and Push the Docker Image:
docker build -t your-docker-repo/cronjob-controller:v1 .
docker push your-docker-repo/cronjob-controller:v1 -
Create Kubernetes Deployment:
Deploy your controller to Kubernetes using a Deployment manifest.
apiVersion: apps/v1
kind: Deployment
metadata:
name: cronjob-controller
spec:
replicas: 1
selector:
matchLabels:
app: cronjob-controller
template:
metadata:
labels:
app: cronjob-controller
spec:
containers:
- name: cronjob-controller
image: your-docker-repo/cronjob-controller:v1
imagePullPolicy: Always -
Apply the CRD and Deployment:
kubectl apply -f cronjob-crd.yaml
kubectl apply -f cronjob-controller-deployment.yaml
Step 4: Test Your Custom Controller
Create a Custom Resource instance to see your controller in action:
apiVersion: example.com/v1
kind: CronJob
metadata:
name: my-cronjob
spec:
schedule: "*/1 * * * *"
image: "busybox"Deploy it using:
kubectl apply -f my-cronjob.yamlMonitor the logs of your controller to verify that it is reconciling the custom resource as expected.
Conclusion
Building a custom controller in Kubernetes provides immense flexibility and control over your workloads. With the power of CRDs and custom logic, you can automate complex scenarios and ensure your applications are managed effectively.
By following this guide, you should now be equipped to create, deploy, and test a custom controller tailored to your needs. Stay tuned for more insights and advanced topics on Kubernetes and cloud-native technologies on WafaTech Blogs!
