Skip to content

ci: dedicated cache warming job - #652

Open
Firestar99 wants to merge 5 commits into
mainfrom
ci_cache_warning_job
Open

Firestar99 wants to merge 5 commits into
mainfrom
ci_cache_warning_job

Conversation

@Firestar99

@Firestar99 Firestar99 commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

After the cargo update in #647, our rust-cache contains not just the intermediary artifacts of that lockfile, but also intermediaries from the previous runs. This is growing the linux "base" cache from 600MiB to 1.7GiB, though other OS and the difftest cache are following a similar trend. Since we only got 10GiB of cache, that growth is problematic.

This PR:

  • creates a new "cache-warming" job that warms the build directory without restoring it first from the cache, so it doesn't grow continually as we update dependencies
    • cache warming takes 20-30min on Linux & Mac, 50min on Windows.
    • Is only ran when main updates, branches can use the most recent cache from main
    • separate difftest cache was removed for simplicity, we only have one cache for both workspaces
  • adds caching for the cargo-gpu-os and cargo-gpu-backwards-compat jobs
  • in compiletest skip some targets on windows to speed up CI
  • deduplicates some copy-paste in CI yaml

PR CI runtime is now ~13min end to end, including windows (ignore the cache-warming jobs in the list, these were temporarily enabled on this branch for testing. The most recent run is without any caches, since the cache-warming needs to run on main first.)

@Firestar99
Firestar99 force-pushed the ci_cache_warning_job branch 2 times, most recently from a323102 to 8577f52 Compare September 28, 2026 15:59
@Firestar99 Firestar99 changed the title ci: dedicated cache warning job ci: dedicated cache warming job Sep 28, 2026
@Firestar99
Firestar99 force-pushed the ci_cache_warning_job branch 9 times, most recently from e1a765b to 3d0934a Compare September 29, 2026 11:57
@Firestar99
Firestar99 force-pushed the ci_cache_warning_job branch 2 times, most recently from b1bc1d9 to 75b015c Compare September 29, 2026 15:14
@Firestar99
Firestar99 force-pushed the ci_cache_warning_job branch 3 times, most recently from 00c249d to f1c953d Compare September 29, 2026 21:50
@Firestar99
Firestar99 marked this pull request as ready for review September 29, 2026 21:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant