In my current organization we have physical CICD runners that have been installed with qemu, and the binaries have been registered in binfmt to automatically start qemu when an armv7/armhf binary is invoked.
echo -1 > /proc/sys/fs/binfmt_misc/qemu-arm
echo ':qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:OCF' > /proc/sys/fs/binfmt_misc/register
This is used to start armhf containers for building and running. So the build configuration is always native, it always picks the underlying architecture, that being either amd64 or armv7. So we do not have any cross-compilation configuration. Now we want to move this infrastructure to Gitlab SaaS preferably, or alternatively GCP. That mean handing out control of the system the jobs get executed on, to 3rd parties and so not having the low-level QEMU setup that allows us to fake a native armv7 arch.
Going on the cloud means we don't have system access, so we can't set the ephemeral system for QEMU, nor would it make sense since it's going to disappear anyway. Of course we can set up static VMs, much like the physical hw, and it would work, but it would increase the fixed costs, and it wouldn't scale.
So the question is how can we virtualize jobs while having these properties
- make them system agnostic, they can run anywhere on the cloud
- preserve our native build configuration, we don't want to change the build config anywhere (CICD config we can change)
- fake an armv7/armhf arch without root privileges when required (does using arm64 help here?)
