Kubernetes CronJobs

Introduction to CronJobs

In Kubernetes, CronJobs are native objects that manage time-based jobs. They are essentially the Kubernetes equivalent of a cron task in traditional systems. A CronJob will run jobs on a time-based schedule, and those jobs will create individual Pods for their tasks which are then managed by the CronJob until they complete.

Defining a CronJob

Here’s a basic structure for a CronJob in YAML:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: my-cronjob
spec:
  schedule: "*/1 * * * *"  # Every minute
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: my-container
            image: my-image
          restartPolicy: OnFailure
  • schedule: This is a string that defines when the job should run. It follows the standard cron format.
  • jobTemplate: The job that should be run at the specified schedule. It is essentially a pod specification with added fields relevant to jobs.

Monitoring and Troubleshooting CronJobs

  1. List CronJobs: Use the command kubectl get cronjobs to list existing CronJobs.

  2. View CronJob Details: To see more details, use kubectl describe cronjob <cronjob-name>.

  3. View Logs: Since CronJobs create Jobs, which in turn create Pods, to view the logs of a CronJob, you first need to identify the specific Job/Pod. This can be done with kubectl get jobs and then kubectl logs <pod-name> to fetch the logs.

  4. Common Issues:

    • Misconfigured schedule
    • Image or command errors inside the container
    • Insufficient resources or quotas
    • Job running longer than expected

Best Practices

  1. Idempotence: Ensure that the tasks being run are idempotent. If a CronJob fails and is retried, it shouldn’t create unintended side-effects.

  2. Concurrency: By default, CronJobs are allowed to run concurrently. Use the concurrencyPolicy field to adjust this if needed.

  3. Failure Handling: Utilize the restartPolicy to define what should happen if the job fails. Most of the time, OnFailure is a good choice.

  4. Cleanup Old Jobs: By default, all successful and failed job pods are kept, which can clutter the cluster. Adjust the .spec.successfulJobsHistoryLimit and .spec.failedJobsHistoryLimit fields to clean up old job pods.

Exercises for CronJobs, Troubleshooting, and Imperative Commands

Exercise 1: Creating a CronJob Imperatively

Objective: Create a CronJob that runs every 5 minutes using the busybox image and the command echo "Hello from CronJob!".

  1. Use the imperative command to create a CronJob.
  2. Validate that the CronJob has been created.
  3. Observe the Pods created by the CronJob over a 10-minute period.

Solution:

kubectl create cronjob hello-cron --image=busybox --schedule="*/5 * * * *" -- echo "Hello from CronJob!"
kubectl get cronjobs
# Wait and check for pods periodically
kubectl get pods

Exercise 2: Troubleshooting a Failing CronJob

Objective: Troubleshoot a CronJob that is not producing any Pods.

  1. Here’s a CronJob YAML for a task that’s supposed to run every minute:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: faulty-cron
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: my-container
            image: busybox
            command:
            - "/bin/nonexistent"
          restartPolicy: OnFailure
  1. Apply this YAML to your cluster.
  2. Identify the issue that’s preventing the CronJob from creating any Pods.
  3. Fix the issue and verify.

Solution:

# Apply the CronJob
kubectl apply -f faulty-cron.yaml

# Check the jobs
kubectl get jobs

# Describe the job to see what's wrong
kubectl describe job <job-name>

# From the describe command, you'll notice a command error. 
# The command doesn't exist, which causes the container to fail immediately.

# To fix, modify the YAML to have a valid command, e.g., ["echo", "Fixed!"]

Exercise 3: Using Imperative Commands for Troubleshooting

Objective: Delete all the Pods associated with a specific CronJob.

  1. Use the previous CronJob definition (hello-cron).
  2. Generate some Pods by waiting for 10 minutes.
  3. Use imperative commands to fetch all the Pods associated with hello-cron.
  4. Delete all these Pods using a single command.

Solution:

# Fetch all pods associated with the CronJob
pods=$(kubectl get pods --selector=job-name=hello-cron-<unique_id> -o=jsonpath='{.items[*].metadata.name}')

# Delete these pods
kubectl delete pods $pods