Running self-hosted bitbucket runners on Kubernetes#
Runners allow you to run pipelines from your repositories. Bitbucket allows you to host these runners in your own infrastructure, the so-called self-hosted runners.
In this article, we’ll walk you through how to create a workspace-level self-hosted runner and run it on Kubernetes.
Creating a runner in bitbucket#
The first step is to create a new runner. Let’s create it at workspace level:
Navigate to your profile and click on the workspace you want.
Click on settings and select workspace runners.
Add runner.
Define the name for the runner and a label. The labels self.hosted and linux are defaults, we need to add a new label so we can identify in the pipeline which runner the build should be executed in, if there are other runners created.
Click on next. Output will be generated with the following docker command:
docker container run -it -v /tmp:/tmp -v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-e ACCOUNT_UUID={ACCOUNT_UUID} \
-e RUNNER_UUID={RUNNER_UUID} \
-e RUNTIME_PREREQUISITES_ENABLED=true \
-e OAUTH_CLIENT_ID=OAUTH_CLIENT_ID \
-e OAUTH_CLIENT_SECRET=OAUTH_CLIENT_SECRET \
-e WORKING_DIRECTORY=/tmp \
--name runner-87c1a9ae-c216-5512-a92d-12fb95f77f0b \ docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner:1
Copy this command, as we are going to use the values of the variables presented for the configuration of our self-hosted runner and click on finish.
Notice that the runner was created in the workspace, but its status is set to UNREGISTRED. After configuring the runner and running it in the Kubernetes environment, its status will change to ONLINE.
Kubernetes Configuration#
The next step is to build the kubernetes files to host runner.
First let’s handle the variables given by the command:
-e ACCOUNT_UUID={ACCOUNT_UUID} \
-e RUNNER_UUID={RUNNER_UUID} \
-e RUNTIME_PREREQUISITES_ENABLED=true \
-e OAUTH_CLIENT_ID=OAUTH_CLIENT_ID \
-e OAUTH_CLIENT_SECRET=OAUTH_CLIENT_SECRET \
-e WORKING_DIRECTORY=/tmp \
Copy the value of the variables ACCOUNT_UUID RUNNER_UUID* without the { }.
The variables OAUTH_CLIENT_ID and OAUTH_CLIENT_SECRET must be converted to base64
echo -n OAUTH_CLIENT_ID | base64
echo -n OAUTH_CLIENT_SECRET | base64
Once this is done, we move on to the stage of building the Kubernetes files:
secret.yaml
job.yaml
Add variable values:
secret.yaml#
apiVersion: v1
kind: Secret
metadata:
name: runner-01-oauth-credentials
labels:
accountUuid: {ACCOUNT_UUID}
runnerUuid: {RUNNER_UUID}
data:
oauthClientId: {OAUTH_CLIENT_ID}
oauthClientSecret: {OAUTH_CLIENT_SECRET}
job.yaml#
Important: In the job.yaml file the accountUuid and runnerUuid variables, in the ‘container variables’ section, must be added between “ ” and { }, in this way “{ACCOUNT_UUID}”
apiVersion: batch/v1
kind: Job
metadata:
name: runner-01
spec:
template:
metadata:
labels:
accountUuid: {ACCOUNT_UUID}
runnerUuid: {RUNNER_UUID}
spec:
containers:
- name: bitbucket-runner-01
image: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
env:
- name: ACCOUNT_UUID
value: “{${ACCOUNT_UUID}}”
- name: RUNNER_UUID
value: “{${RUNNER_UUID}}”
- name: RUNTIME_PREREQUISITES_ENABLED
value: true
- name: WORKING_DIRECTORY
value: "/tmp"
- name: OAUTH_CLIENT_ID
valueFrom:
secretKeyRef:
name: runner-01-oauth-credentials
key: oauthClientId
- name: OAUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: runner-01-oauth-credentials
key: oauthClientSecret
volumeMounts:
- name: tmp
mountPath: /tmp
- name: docker-containers
mountPath: /var/lib/docker/containers
readOnly: true
- name: var-run
mountPath: /var/run
- name: docker-in-docker
image: docker:20.10.7-dind
securityContext:
privileged: true
volumeMounts:
- name: tmp
mountPath: /tmp
- name: docker-containers
mountPath: /var/lib/docker/containers
- name: var-run
mountPath: /var/run
restartPolicy: OnFailure
volumes:
- name: tmp
- name: docker-containers
- name: var-run
backoffLimit: 6
completions: 1
parallelism: 1
Now let’s apply the configurations built above.
kubectl -n namespace apply -f secrets.yaml
kubectl -n namespace apply -f job.yaml
Testing the new runner#
pipelines:
branches:
'main':
- step:
runs-on: my.custom.label
script:
- echo "Hello world"
Your self-hosted runner is ready!
For more information about runners visit the bitbucket documentation runners.
Need help?#
Access our page Contact and chat with us.
Success!