Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Incremental Builds

This page explains what go2nix actually puts in the Nix store, what gets reused on rebuilds, and how that differs from buildGoModule.

If you only want the API surface, see Builder API. If you want the step-by-step eval flow, see Default Mode.

The shape of a build

buildGoModule (nixpkgs)

┌──────────────────────────┐     ┌──────────────────────────┐
│ vendor FOD               │ ──▶ │ app derivation           │
│ (all modules, one hash)  │     │ (go build ./..., 1 drv)  │
└──────────────────────────┘     └──────────────────────────┘

Two derivations total. Any change to any .go file rebuilds the whole app derivation; any go.sum change re-downloads the entire vendor tree.

go2nix (default mode)

The derivations of one go2nix build: module fetches (gomod) feed third-party package compiles (gopkg), which feed the compiles of your own packages (golocal), which feed the application derivation that links and runs the tests; the standard library is one derivation read by every compile, and an importcfg bundle collects the third-party entries for the link. Editing internal/web rebuilds internal/web, cmd/app and the application only.

The same thing by layer:

  module FODs                        one fixed-output derivation (FOD)
       │                             per module@version the build uses
       ▼
  third-party package drvs (.a)      one per imported third-party package
       │                  │
       ▼                  ▼
  local package drvs     importcfg bundle     bundle = stdlib + third-party
  (.a; plus .x with      (one per app)        entries, no local packages
   contentAddressed)          │
       │                      │
       └──────────┬───────────┘
                  ▼
              app drv                compiles the main package(s), links,
                                     and runs the tests (doCheck)

  stdlib drv ──► an input of every compile and of the bundle
                 (Go toolchain + scope goEnv only)

.a = compiled package archive; .x = export-data interface, which only local packages get and only with contentAddressed = true (see Early cutoff below). The importcfg is a file that maps each import path to its compiled .a archive in the store — go tool compile and go tool link read it instead of searching GOPATH.

For a non-trivial application this is hundreds to thousands of derivations instead of two — but almost all of them are reusable across rebuilds.

What gets cached

LayerOne derivation perCache key (informally)Rebuilds when
stdlibGo toolchain and scope goEnvGo version + goEnv (GOOS/GOARCH when cross-compiling, CGO_ENABLED, GOFIPS140, …)Go is bumped or goEnv changes
module FODmodule path@version the build usesmodule path@version + NAR hashthat module is bumped
third-party packageimported packagemodule FOD + import deps + tags + gcflagsthe module or any of its transitive deps change
local packagelocal Go packagethe package’s directory (every file in it, _test.go and testdata/ included, nested packages excluded) + import depsany file in that directory or a dep changes
importcfg bundleappthe stdlib and the third-party package outputsa third-party package output changes
appappimportcfg bundle + local package archives + the filtered source (mainSrc: with doCheck, every local package’s sources, tests and testdata/)anything above changes

Third-party module FODs and third-party package derivations are shared between every application in the flake (and across flakes, via the binary cache). Bumping a single module re-fetches one FOD and recompiles only the packages that transitively import it.

Local package derivations use a builtins.path-filtered source: only the package’s own directory is hashed — its whole subtree, minus nested package directories and nested modules, and whatever srcFilter rejects — so editing pkg/a/a.go does not change the input hash of the pkg/b derivation unless b imports a. No go.mod comes along (the go directive is passed in separately). Everything in the directory counts, not just what gets compiled: embedded assets, but also _test.go files, testdata/ and a README next to the sources participate in the package’s cache key.

Rebuild propagation

When you edit a single local package, only the reverse-dependency cone of that package rebuilds:

  1. The edited package recompiles.
  2. Each package that imports it (directly or transitively) recompiles.
  3. The final derivation rebuilds: it compiles the main package, links, and runs the tests. The importcfg bundle does not: it only holds standard library and third-party entries, which a local edit does not touch.

Packages outside the cone keep their existing store paths and are not rebuilt.

The same reasoning for other kinds of change:

You changeWhat rebuilds
a file in a local package’s directory (source, test or testdata/)that package, the local packages that import it directly or transitively, and the app. With contentAddressed = true the dependents are skipped when the package’s export data came out the same, which is always the case for a test-only edit
an import between packages that already existthe importing package and its cone. Nothing to regenerate: the lockfile pins modules, not the graph
one module’s versionits FOD, the packages of that module, everything that imports them, the importcfg bundle and the app
a test-only dependencyits testPackages derivations, the test importcfg bundle and the app (relink, tests re-run)
ldflags, checkFlagsthe app only
tags, gcflags, pgoProfileevery package compile (they are part of each compile manifest), then the bundle and the app. The stdlib is not affected
the scope’s goEnv, or the Go toolchainthe stdlib, and everything after it

To get a feel for how big the cone is in your project, see Benchmarking.

Early cutoff with contentAddressed = true

By default, per-package derivations are input-addressed: if a package’s inputs change, every downstream derivation gets a new store path even if the compiled output happens to be byte-identical.

Setting contentAddressed = true opts into two coupled mechanisms:

  • Floating-CA outputs. Local-package derivations and the importcfg bundle become content-addressed, so a rebuild that produces a byte-identical .a resolves to the same store path and short-circuits downstream rebuilds. Third-party packages stay input-addressed: their source never changes, so CA would only add resolution overhead.
  • iface output split. Each local-package derivation gains a second iface output containing only the export data (the .x file produced by go tool compile -linkobj). Downstream compiles depend on iface instead of the full .a, so changes to private symbols that don’t alter the package’s exported API don’t cascade. This mirrors the .x model used by Bazel’s rules_go.
The same private edit to internal/web built twice. Input-addressed, the default: internal/web, cmd/app and the application all rebuild. With contentAddressed = true, internal/web has an archive output (.a), which changes, and an interface output (.x), which comes out byte-identical and keeps its store path; cmd/app compiles against the .x and is reused, and only the link, which needs the .a, runs again.

The two are coupled by design: CA without iface only short-circuits comment-only edits, and iface without CA can’t cut off anything because the input-addressed .x path still changes whenever src changes.

Requires the ca-derivations experimental feature in Nix. The final binary stays input-addressed.

Known limitation: adding the first package-level initializer to a previously init-free package still flips a bit in the .x file, so that particular edit cascades even though the API didn’t change. This is rare in practice.

The cost: eval time

go2nix trades build time for eval time. Every nix build evaluation:

  1. Calls builtins.resolveGoPackages (the Nix plugin), which runs go list -json -deps against your source tree — and a second go list -deps -test pass when doCheck is on, which is the default.
  2. Instantiates one derivation per package in the resulting graph.

For a large application (~3,500 packages) the warm-cache go list step takes on the order of a few hundred milliseconds, and instantiation adds a similar amount on top. This is the floor on every rebuild — even a no-change rebuild — and is the main reason go2nix is overkill for small single-binary projects.

The plugin call is impure (it reads GOMODCACHE), so the result is not cached by the Nix evaluator across invocations. See Nix Plugin for details.