Creating a linux arm64 github actions runner

github actions is great, but it does not offer any arm hardware to run your tests on. So, I bought a Traverse Ten64 with the hope of using it as a github actions runner for the Solang Solidity Compiler.

In order to run the runner, you run some dotnet code called the GitHub Actions Runner. This software connects to github, and listens to instructions of what jobs to run. There are a few attack vectors here: someone could create a pull request with some malicious code in it (there is some mitigation against this), or github itself might be coerced into sending malicious commands to your runner.

The other consideration is that you would like each time that your runner executes some code, it has a clean environment with no residue from the last run. So, I think the best way to go about this is to run the runner in a VM, which then shuts itself down after each run, and then is restored from a VM snapshot.

First you want to set up your VM.

1
2
3
4
5
6
7
8
qemu-img create /home/sean/runner-image/sean-ci-linux-arm64.qcow2 -f qcow2 200G

virt-install -n action-runner --memory 16384 --arch aarch64 --vcpus 8 \
     --disk /home/sean/runner-image/sean-ci-linux-arm64.qcow2,device=disk,bus=virtio \
     --os-type=linux \
     --nographic \
     --boot uefi \
     --location 'http://ports.ubuntu.com/dists/focal/main/installer-arm64/'

I would create a user called runner, with no password and no root password either, so that sudo apt-get install foo... works from github actions. You can remove a password with passwd -d.

Now in your VM, download the github runner as instructed by github in your project settings, and configure it and register it using ./config.sh. Do not install the service, we will be creating our own.

In the same directory as the runner, create a shell script called runshutdown.sh with the following contents:

1
2
3
#!/bin/bash
./run.sh --once
sudo shutdown -h now

Also create a service in /etc/systemd/system/actions-runner.service. Note this could probably be a user service.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[Unit]
Description=GitHub Actions Runner
After=network.target

[Service]
ExecStart=/home/runner/actions-runner/runshutdown.sh
User=runner
WorkingDirectory=/home/runner/actions-runner
KillMode=process
KillSignal=SIGTERM
TimeoutStopSec=5min

[Install]
WantedBy=multi-user.target

Enable the service with:

1
2
systemctl daemon-reload
systemctl enable actions-runner.service

Now your VM should be ready for snapshotting, so shut it down with shutdown -h now.

Once it is shutdown, create a snapshot using:

1
qemu-img snapshot -c restore-me sean-ci-linux-arm64.qcow

Now we’re going to create a script which restores the snapshot, starts the VM, and loops when the VM shuts down. I could not find a way of doing this in shell script, so this is done using the python libvirt api.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
#!/usr/bin/python3 -u

import libvirt
import sys
import os
import threading
import time

eventLoopThread = None

def virEventLoopNativeRun():
	while True:
		libvirt.virEventRunDefaultImpl()

def virEventLoopNativeStart():
	global eventLoopThread
	libvirt.virEventRegisterDefaultImpl()
	eventLoopThread = threading.Thread(target=virEventLoopNativeRun, name="libvirtEventLoop")
	eventLoopThread.setDaemon(True)
	eventLoopThread.start()


virEventLoopNativeStart()

try:
	conn = libvirt.open(None)
except libvirt.libVirtError:
	print('error: failed to connect to the hypervisor')
	sys.exit(2)

try:
	dom0 = conn.lookupByName('action-runner')
except libvirt.libvirtError:
	print('error: cannot find action-runner')
	sys.exit(2)

def start_if_necessary():
	if dom0.isActive():
		print('info: already running')
	else:
		print('info: restoring snapshot')
		os.system('qemu-img snapshot -a restore-me sean-ci-linux-arm64.qcow2')
		print('info: starting action-runner')
		if dom0.create() < 0:
			print('error: failed to start action-runner')

def lifecycle(conn, dom, event, detail, opaque):
	if dom.name() == 'action-runner':
		start_if_necessary()

conn.domainEventRegister(lifecycle, None)

start_if_necessary()

conn.setKeepAlive(5, 3)

while conn.isAlive() == 1:
	time.sleep(10)

conn.close()

Create a file ~/.config/systemd/user/action-runner-vm.service:

1
2
3
4
5
6
7
8
9
10
[Unit]
Description=Run VM for github actions

[Service]
ExecStart=/home/sean/runner-image/actions-vm-respawn.py
WorkingDirectory=/home/sean/runner-image
StandardOutput=journal

[Install]
WantedBy=default.target

Now start this service with:

1
2
3
systemctl daemon-reload
systemctl --user start action-runner-vm.service
systemctl --user enable action-runner-vm.service

And you should be in business. You can always connect to the console of the actions runner with virsh console actions-runner.