Michael Suchacz
f8e8f979a2
## Problem `go run` caches the final linked executable in `~/.cache/go-build`. Every helper invocation via `go run ./scripts/<tool>` stores a copy, and because the cache key includes build metadata, the same tool accumulates multiple cached executables over time. With 12+ helper binaries invoked during `make gen` and `make pre-commit`, this is a meaningful contributor to GOCACHE growth. ## Fix Replace `go run` with `go build -o _gen/bin/<tool>` for 12 repo-local helper packages (16 Makefile callsites). Each helper is an explicit Make file target with `$(wildcard *.go)` prerequisites, so `make -j` serializes builds correctly instead of racing on shared output paths. Helpers converted: `apitypings`, `auditdocgen`, `check-scopes`, `clidocgen`, `dbdump`, `examplegen`, `gensite`, `apikeyscopesgen`, `metricsdocgen`, `metricsdocgen-scanner`, `modeloptionsgen`, `typegen`. Left on `go run` (intentionally): `migrate-ci` and `migrate-test` (CI/test-only, not on common developer paths). `_gen/` is already in `.gitignore`. The `clean` target removes `_gen/bin`. ## GOCACHE growth (isolated cache, single `make gen`) | | Old (`go run`) | New (`go build -o`) | |--|----------------|---------------------| | Total cache size | 2.9 GB | 2.6 GB | | Cached executables | 11 | 4 | | Executable bytes | 401 MB | 25 MB | The 4 remaining executables come from tools outside this change (`dbgen` and `goimports` from `generate.sh`, plus two `main` binaries from deferred helpers). Helper binaries now live in `_gen/bin/` (581 MB, gitignored, cleaned by `make clean`). ## Build time benchmarks **Source changed** (content hash invalidated, forces recompile): | Helper | `go run` | `go build -o` + run | Overhead | |--------|---------|---------------------|----------| | typegen | 1.50s | 2.03s | +0.52s | | examplegen | 1.37s | 1.67s | +0.30s | | apikeyscopesgen | 1.21s | 1.71s | +0.50s | | modeloptionsgen | 1.23s | 1.64s | +0.41s | **Repeat invocation** (no source change, the common `make gen` / `make pre-commit` path): | Helper | `go run` (cache lookup) | Cached binary | Speedup | |--------|------------------------|---------------|---------| | typegen | 0.346s | 0.037s | 9.4x | | examplegen | 0.368s | 0.037s | 9.9x | | modeloptionsgen | 0.342s | 0.021s | 16.3x | | apikeyscopesgen | 0.298s | 0.030s | 9.9x | When source changes, `go build -o` is 0.3-0.5s slower per helper (it writes a local binary instead of caching in GOCACHE). On repeat runs (the common path), the pre-built binary is 10-16x faster because `go run` still does a staleness check while the binary just executes. > This PR was authored by Mux on behalf of Mike.
56 KiB
56 KiB