Search for a command to run...
I used to be a .NET developer. Nowaways, I am a DevOps solutions architect with a focus on Azure and Kubernetes.
I also love productivity topics, especially when it comes to doing more with less of my time. I'm also a daddy, so time is a limited resource for me.
No comments yet. Be the first to comment.
I'm going to share all my secrets with you
Using a pull-request workflow
This is the guide I wish I had
Connecting your GitHub Workflows to Azure AD using OpenID Connect
In this post, I’ll walk through the essentials of configuring an AKS cluster to use Argo CD with an NGINX Ingress Controller. Pre-Requisites You’ll need: An AKS cluster. Here’s a Gist I use to create a simple cluster. Helm 3 Azure CLI A Custom DNS/A...
Sometimes, we want to run a one-time job during our deployments. For example, running an Entity Framework migration. By creating a console application, packaged as a Docker image, we can run these one-time jobs as a Kubernetes Job upon a deployment.
I created a simple .NET Core console app that will run as a fake one-time job. In following posts, I will show how to run an Entity Framework migration instead.
The example source code can be found here.
# Build imaage
FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build-env
WORKDIR /app
COPY *.csproj ./
RUN dotnet restore
COPY . ./
RUN dotnet publish -c Release -o output
# Runtime image
FROM mcr.microsoft.com/dotnet/core/runtime:2.2
WORKDIR /app
COPY --from=build-env /app/output .
ENTRYPOINT ["dotnet", "HelloFakeJob.dll"]
This is important. We want to make sure that the container stops itself after the task inside the container is finished. Otherwise, you will have a never ending job in Kubernetes.
For example:
docker build -t hellokubernetesjob .
Then running and ensuring it is not running by looking at the STATUS column:
➜ docker run -it hellokubernetesjob
Hello World!
➜ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5e5d66390226 hellokubernetesjob "dotnet HelloFakeJob…" 2 minutes ago Exited (0) 2 minutes ago vigila
apiVersion: batch/v1
kind: Job
metadata:
name: hellokubernetesjob
spec:
template:
spec:
containers:
- name: hellokubernetesjob
image: myregistry.azurecr.io/samples/hellokubernetesjob
restartPolicy: OnFailure
backoffLimit: 4
After having pushed the docker image to a container registry, deploy the job
➜ kubectl apply -f hellofakejob.yml
job.batch/hellokubernetesjob created
Kubernetes will run the job and keep the job metadata and pod forever. The purpose of this is to see what jobs were run and have access to their logs.
The jobs will have to be manually deleted. I recommend that you create a runbook pipeline or script that runs periodically and deletes the old jobs.
➜ kubectl get job/hellokubernetesjob
NAME COMPLETIONS DURATION AGE
hellokubernetesjob 1/1 21s 36s
Ensure that your job completes. You can also retrieve the container logs:
➜ kubectl get pods --selector=job-name=hellokubernetesjob
NAME READY STATUS RESTARTS AGE
hellokubernetesjob-lbxsm 0/1 Completed 0 9m14s
➜ kubectl logs hellokubernetesjob-lbxsm
Hello World!
Try to verify the behavior of something terribly wrong happening within the container. I will modify my sample app to throw an exception.
using System;
namespace HelloFakeJob
{
class Program
{
static void Main(string[] args)
{
// Console.WriteLine("Hello World!");
throw new Exception("Uh ohhhhhhhh");
}
}
}
Note - Any subsequent job deployments require a different name. Otherwise, Kubernetes will not apply. Instead it will say:
job.batch/hellokubernetesjob unchanged
It’s best that you append an identifier to the job name. For example, a pipeline build number, a datestamp, etc.
➜ kubectl apply -f hellofakejob.yml
job.batch/hellokubernetesjob-failure created
➜ kubectl get pods --selector=job-name=hellokubernetesjob-failure
NAME READY STATUS RESTARTS AGE
hellokubernetesjob-failure-hckhz 0/1 CrashLoopBackOff 2 41s
As expected, the pod will continuously restart 4 times as per the backoffLimit setting.
One feature in helm that can be useful for issuing these one time jobs are deployment hooks.
Helm provides hooks in the helm chart lifecycle such as:
The deployment hooks allow us to run a job before a deployment using the pre-install hook. To declare a job to run before the installation of a helm chart, you would add an annotation:
annotations:
"helm.sh/hook": pre-install
You can also indicate the order of which hooks have to run using weights. The lower the number, the sooner it will run.
Lastly, you can clean up these one time jobs and remove the need to create a runbook pipeline or script that cleans up these jobs later. The self cleanup is possible using the hook deletion policies.
annotations:
"helm.sh/hook-delete-policy": hook-succeeded
apiVersion: batch/v1
kind: Job
metadata:
name: "{{.Release.Name}}"
labels:
app.kubernetes.io/managed-by: {{.Release.Service | quote }}
app.kubernetes.io/instance: {{.Release.Name | quote }}
app.kubernetes.io/version: {{ .Chart.AppVersion }}
helm.sh/chart: "{{.Chart.Name}}-{{.Chart.Version}}"
annotations:
"helm.sh/hook": pre-install
"helm.sh/hook-weight": "1"
"helm.sh/hook-delete-policy": hook-succeeded
spec:
template:
spec:
containers:
- name: hellokubernetesjob
image: myregistry.azurecr.io/samples/hellokubernetesjob
restartPolicy: OnFailure
backoffLimit: 4
kubectl apply before the main deployment and also have the job name be unique per deployment