Remote registry cache & cache mounts

[AaronDewes]

Question

Hi!

I've recently moved my CI to Earthly (running inside GitLab CI). I'm using the built-in Rust integration and --remote-cache=harbor.example.com/build_cache/cache:tag,image-manifest=true,oci-mediatypes=true. However, it seems like cache mounts are not being stored this way.

Is there another good way to keep cache (without moving to Earthly Cloud)? If not, is there any change remote-cache can be extended to also push cache mounts?

[negz]

This might be a blocker for adopting Earthly for me, too.

I'm a @crossplane maintainer. We're a CNCF open source project, and our maintainers have hated our make-library-in-a-git-submodule build system for many years. I'm a big fan of Earthly so far, but I can't see a way to get effective caching in CI.

We use GitHub Actions, and we get a decent speedup by using GitHub's cache action to cache $GOCACHE and $GOMODCACHE - i.e. the Go build and module caches - between CI runs.

I've configured Earthly to use cache mounts to ensure we get that speedup during local development (i.e. while BuildKit has a hot cache). That's great for local development - it means Earthly is as fast or sometimes a little faster than our Make builds.

Unfortunately given that we get a fresh BuildKit on every GitHub Actions run, those cache mounts will always be cold. I had hoped we could at least load them from an explicit cache image, but that doesn't appear to be the case. I've tried RUN --mount, CACHE, and CACHE --persist together with --remote-cache without success.

I understand using a remote runner would solve the problem, but this is pretty much off the table for us as an OSS project. A large part of the appeal of GitHub Actions is that it's free for open source, and we don't need to run any infrastructure to use it.

[negz]

Ah, I think what I'm looking for might be a duplicate of #1540.

[AaronDewes]

#1116 may be useful for you too. Earthly also has a --max-remote-cache option that caches everything, but this makes caches larger.

https://docs.earthly.dev/docs/caching/caching-via-registry#optimizing-explicit-cache-performance-advanced