Improving performance

Measure the slow operation, change one setting, and compare results on the same workload. Keep the defaults unless a change helps. Start with the cache and working-file model if these operations are new to you.

Slow operation	Start here
Git status in a large asset tree	Combine exclusions
Small updates to a huge lock	Shard the lock
Copying large cached files	Use reflinks
Syncing files you do not need	Limit the working set
Repeated downloads across clones	Share a cache
Ingesting stable large files	Try hybrid ingest
Checking many unchanged files	Trust recorded state

Combine file exclusions

By default, Gat writes an exact .git/info/exclude rule for each tracked path that has no newline. For a directory used only for Gat assets, replace thousands of exact rules with a broad pattern:

gat config git.ignore_patterns '/data/' '/models/'

Gat omits redundant exact rules for paths covered by these patterns. Use one argument per pattern; commas do not separate list elements. Negated patterns such as !keep.txt are unsupported.

These rules also hide files Gat does not track. Use them only for directories that should stay entirely out of ordinary Git tracking.

Keep temporary files out of Gat

.gatignore filters files during gat add. For example:

.gatignore

data/tmp/
**/*.partial
**/*.log

git.ignore_patterns controls Git exclusions after tracking. .gatignore controls which files Gat ingests.

Filenames containing newlines

Git’s ignore format cannot express an exact path containing LF. Gat records an escaped comment instead. The file remains visible to Git unless a broader pattern covers it. Review git status before staging.

Shard a very large `gat.lock`

A flat lock is simplest for most repositories. If small add, rm, or mv operations spend substantial time rewriting it, start with one shard level:

gat config lock.shard_levels 1

Paths are distributed across up to 256 files such as gat.lock/a4.tsv. Routine sparse updates can then rewrite only affected shards.

Choose the layout

Use 1 to start. 2 adds another level, such as gat.lock/a4/7f.tsv, at the cost of more directories. 0 returns to a flat lock.

Convert and review

The next lock-writing or reshaping command, such as gat sync, converts the full lock once. Inspect the resulting metadata with Git.

Commit the shared setting and lock

git add gat.yaml gat.lock
git commit -m "Shard Gat lock"

Keep the setting in project configuration. Conflicting local values can repeatedly convert the layout.

What sharding does and does not speed up

Sharding reduces lock rewrites. Explicit root-anchored directory exclusions such as /models/ also let Gat skip covered ranges while rebuilding Git exclusions. Other patterns may still require checking individual paths.Existing state databases pay a one-time indexing cost. Memory and output size still grow with uncovered paths. Measure the full command, not just lock writes.

Avoid unnecessary copying during materialization

Copy-on-write clones share disk blocks until a file changes; edits keep the cache intact. Try them with an independent-copy fallback:

gat config --local cache.materialization_strategy reflink copy

Strategies are attempted in order. A local setting suits machines with different filesystem capabilities.

Strategy	Behavior	Tradeoff
`copy` (default)	Independent bytes	Copies the entire file
`reflink`		Requires filesystem support
`hardlink`	Cache and working file share an	Same filesystem; writes can affect cached bytes
`symlink`	Working path points to the cache object	Exposes the cache path; writes can affect cached bytes

Use hardlinks or symlinks only when you accept shared bytes between the cache and working tree. Prefer reflink copy for files that tools may edit.

Changing the setting affects future materializations. To recreate already-correct files using the new strategy:

gat sync --rematerialize --dry-run
gat sync --rematerialize

This revisits every selected clean file and can do much more I/O than ordinary sync.

Reconcile less of a huge repository

For one operation:

gat sync --path models --exclude 'legacy/**'

For a saved working set, use the experimental selection command:

gat selection add working --local --include 'models/**' --include 'datasets/current/**'
gat selection default working --local

The default applies to sync (including hooks), pull, fetch, push, status, diff, and ls-files. Explicit path filters replace it completely; --path . selects all repository paths. See Path selection.

Excluded files are not reconciled. They may remain absent or contain a version from an earlier checkout until you sync them explicitly.

Reuse a cache across repositories

Choose a writable directory shared by the clones that reuse content:

gat config --local cache.location "$HOME/.cache/gat/objects"

gat config --local cache.location "$HOME\gat-cache\objects"

GAT_CACHE_DIR overrides the configured location. Identical bytes share one cached object, avoiding repeated storage and downloads.

Garbage collection does not discover other cache users. Supply every sharing repository with gat gc --repository LOCATION, repeating the option as needed. Only committed locks from additional repositories are protected; their uncommitted changes are not. See gat gc.

Choose a faster ingest strategy

happens when you add or update tracked content.

Safe (default)
Hybrid (experimental)

gat config --local cache.ingest_strategy safe

Gat copies the source, then hashes the completed copy. The content ID describes the exact published bytes even if the source changes during ingest.

gat config --local cache.ingest_strategy hybrid

Copying and hashing run concurrently. Gat falls back to safe ingest when source metadata changes, but this detection is a heuristic: a same-size rewrite within the filesystem’s timestamp resolution can escape detection.Use only for completed artifacts whose producer has stopped writing, and benchmark against safe on your storage.

safe is the only strategy that unconditionally establishes the object ID from the completed bytes being published. mmap is deprecated; do not adopt it for new tuning. See Feature lifecycle.

Use the trust-state fast path for sync

Normal sync checks file metadata and hashes content only when needed. If even those checks are expensive, you can trust recorded state for one run:

gat sync --trust-state

Or save the choice locally:

gat config --local sync.trust_state true

When desired and recorded materialized state agree, this mode skips file inspection. It can miss external edits or deletions. Use it only when that tradeoff is acceptable, such as .

To restore validation even if a lower layer enables trust-state:

gat config --local sync.trust_state false

Configuration reference

Defaults, constraints, and accepted values.

Sync reference

Preview, fetch, repair, and rematerialize files.

Getting Started

Concepts

Guides

References

Commands

Improving performance

Combine file exclusions

Shard a very large `gat.lock`

Avoid unnecessary copying during materialization

Reconcile less of a huge repository

Reuse a cache across repositories

Choose a faster ingest strategy

Use the trust-state fast path for sync

Configuration reference

Sync reference

​Combine file exclusions

​Shard a very large gat.lock

​Avoid unnecessary copying during materialization

​Reconcile less of a huge repository

​Reuse a cache across repositories

​Choose a faster ingest strategy

​Use the trust-state fast path for sync

Configuration reference

Sync reference

Combine file exclusions

Shard a very large `gat.lock`

Avoid unnecessary copying during materialization

Reconcile less of a huge repository

Reuse a cache across repositories

Choose a faster ingest strategy

Use the trust-state fast path for sync