So I had a really simple workflow of using intermediate containers to build something from source, then a prod container that pulls build artifacts from the build container.

Yesterday I tried building an image and saw that buildx was the preferred engine since the legacy build is now deprecated, so after reading the docs I got the impression that I could mostly use the same workflow as before. After a few tries of this the buildx plugin would not build anything since would not fully read my dockerfile, or it would not even start the build since it was trying to remove a zfs snapshot from my graph directory that didn't actually exist.

Today I took another stab at it, and find that there's a separate daemon called buildkitd that doesn't seem to be available via the apt packages, so I grab the tarball from GitHub. Then I have to hunt in the repo to find systemd service files.

https://github.com/moby/buildkit

So I extract the tarball and then find that buildkitd won't start because it seems to be hardcoded to look in /usr/local/bin instead of PATH. I move them there, then my image will actually start building. The build stopped partway through because of an update to Maven, which is not a big deal, but then I find that I can't step into an intermediate container to inspect a broken build, since this isn't exposed anymore.

https://stackoverflow.com/a/70715535

I fix the build issue and the build finally succeeds, then I find that the image isn't loaded into my docker daemon, so now I also have to figure that out.

It's on me for not keeping up with the transition to buildx, but did anyone else go through this much pain? I'd like to think that there's a path of least resistance or documentation I missed somewhere, but I'm surprised that attempting to do something this simple took this much effort, when the docs for buildx make it seem like it's a drop in replacement. Had I not just invested time in locking down my docker containers with privileges and userns I probably would be looking at using podman instead.