Configuration
Every option of the services.gradient NixOS module, generated from nix/modules. Each option will map onto a flag and an environment variable of the same name.
upload.bytesBudget is --upload-bytes-budget and GRADIENT_UPLOAD_BYTES_BUDGET.
worker.build.maxConcurrent is --build-max-concurrent and GRADIENT_WORKER_BUILD_MAX_CONCURRENT.
- Module-only options (
packages, reverseProxy, postgres, ...) have no environment variable.
Declarative entities under services.gradient.state are in the state reference.
General
| Option |
Type |
Default |
Env |
Description |
baseDir |
path |
"/var/lib/gradient" |
GRADIENT_BASE_DIR |
Directory holding Gradient's state, NAR files and caches. |
domain |
string |
- |
- |
Domain under which Gradient is reachable. |
enable |
bool |
false |
- |
Whether to enable Gradient. |
environmentVariables |
attribute set of (null or bool or int or string or path) |
{ } |
- |
Extra environment variables of the Gradient server, taking precedence over variables from other options. null unsets a variable. |
listenAddr |
string |
"127.0.0.1" |
GRADIENT_LISTEN_ADDR |
IP address for the Gradient server to listen on. |
localWorker |
bool |
worker.enable |
- |
Whether to register the worker on this host automatically, with an ID from the hostname and a token and peers file from the first start. Members of the state-declared team server reach every new project, the local worker included. No UUID, token or web UI step is necessary. Disable it to register the local worker like a remote worker, with worker.id and worker.peersFile. |
port |
port |
3000 |
GRADIENT_PORT |
Port for the Gradient server to listen on. |
publicStats |
bool |
false |
GRADIENT_PUBLIC_STATS |
Expose GET /api/v1/workers and worker stats without authentication. Only superusers can access those endpoints with false (default). |
retentionDays |
unsigned int |
90 |
GRADIENT_RETENTION_DAYS |
Days to keep job assignment records, finished deliveries, worker connection history, webhook and task action deliveries, expired sessions and CLI logins. Finished admin tasks, the audit log, per-build resource samples and finished cluster jobs share the same limit. Removed resource samples no longer feed build predictions. Finished cluster jobs without remaining members go on the next hourly pass. Active cluster jobs and the newest finished admin task of each kind stay exempt. Open worker connections stay until the next connection of the same worker. Set it to 0 to keep every record forever. |
serveUrl |
string |
derived |
GRADIENT_SERVE_URL |
Public URL under which clients reach Gradient. Set this for a URL other than domain, for example behind a port mapping. |
storePath |
null or string |
null |
GRADIENT_STORE_PATH |
|
useQuic |
bool |
false |
GRADIENT_USE_QUIC |
Whether to enable advertising HTTP/3 (QUIC) to clients. |
useTls |
bool |
true |
GRADIENT_USE_TLS |
Whether to enable TLS. |
build
| Option |
Type |
Default |
Env |
Description |
build.defaultMaxSilentSecs |
unsigned int |
3600 |
GRADIENT_BUILD_DEFAULT_MAX_SILENT_SECS |
Timeout in seconds without build output for derivations without maxSilent. Set it to 0 to disable the timeout. |
build.defaultTimeoutSecs |
unsigned int |
14400 |
GRADIENT_BUILD_DEFAULT_TIMEOUT_SECS |
Build timeout in seconds for derivations without timeout. Set it to 0 to disable the timeout. |
build.inputsUnavailableMaxLoops |
positive int |
3 |
GRADIENT_BUILD_INPUTS_UNAVAILABLE_MAX_LOOPS |
Times a build can retry after missing inputs before failing instead of retrying again. |
build.maxAttempts |
positive int |
3 |
GRADIENT_BUILD_MAX_ATTEMPTS |
Build or eval job attempts before a transient failure is permanent. |
build.retryBackoffSecs |
unsigned int |
30 |
GRADIENT_BUILD_RETRY_BACKOFF_SECS |
Seconds before retrying a transient build failure, doubled for every previous attempt. |
build.substituteMissEscalationThreshold |
positive int |
2 |
GRADIENT_BUILD_SUBSTITUTE_MISS_ESCALATION_THRESHOLD |
Free re-queues of a derivation available in a cache within one evaluation, before building the derivation like any other. A re-queue does not count as a build attempt. This threshold is the only bound on that loop. |
cache
| Option |
Type |
Default |
Env |
Description |
cache.debugIndexIntervalSecs |
positive int |
300 |
GRADIENT_CACHE_DEBUG_INDEX_INTERVAL_SECS |
Seconds between passes of the DWARF build ID index backfill. Uploads enter the index immediately. Only paths cached before the index existed or interrupted by a restart need the pass. |
cache.maxStorageGb |
unsigned int |
0 |
GRADIENT_CACHE_MAX_STORAGE_GB |
Instance-wide limit on cached NAR storage in GB. New evaluations wait while every writable cache of a project has less than 10 MiB left. Set it to 0 to disable the limit. Per-cache limits still apply. |
cache.signSweepIntervalSecs |
positive int |
3600 |
GRADIENT_CACHE_SIGN_SWEEP_INTERVAL_SECS |
Seconds between NAR signature backfill passes. Uploads receive their signature immediately. Only subscription placeholders and unsigned leftovers need the pass. |
cache.upstreamQueryConcurrency |
positive int |
32 |
GRADIENT_CACHE_UPSTREAM_QUERY_CONCURRENCY |
Maximum simultaneous narinfo requests to upstream caches across the server. |
database
| Option |
Type |
Default |
Env |
Description |
database.cache.maxConnections |
positive int |
32 |
GRADIENT_DATABASE_CACHE_MAX_CONNECTIONS |
Maximum connections of the cache query pool. It is separate from the scheduler pool to keep a large evaluation's prefetch traffic from starving job assignment. |
database.cache.minConnections |
unsigned int |
2 |
GRADIENT_DATABASE_CACHE_MIN_CONNECTIONS |
Minimum connections kept open in the cache query pool. |
database.maxConnections |
positive int |
16 |
GRADIENT_DATABASE_MAX_CONNECTIONS |
Maximum connections of the scheduler and worker pool. Server processes each open up to the sum of database.maxConnections, database.cache.maxConnections and database.web.maxConnections. PostgreSQL's max_connections must accommodate that sum. |
database.minConnections |
unsigned int |
2 |
GRADIENT_DATABASE_MIN_CONNECTIONS |
Minimum connections kept open in the scheduler and worker pool. |
database.url |
string |
"postgresql://gradient@localhost/gradient?host=/run/postgresql" |
GRADIENT_DATABASE_URL |
PostgreSQL connection URL. |
database.urlFile |
path |
derived |
GRADIENT_DATABASE_URL_FILE |
File containing the PostgreSQL connection URL, taking precedence over database.url. |
database.web.maxConnections |
positive int |
8 |
GRADIENT_DATABASE_WEB_MAX_CONNECTIONS |
Maximum connections of the HTTP API pool. |
database.web.minConnections |
unsigned int |
1 |
GRADIENT_DATABASE_WEB_MIN_CONNECTIONS |
Minimum connections kept open in the HTTP API pool. |
email
| Option |
Type |
Default |
Env |
Description |
email.enable |
bool |
false |
GRADIENT_EMAIL_ENABLE |
Whether to enable sending email. |
email.from.address |
string |
- |
GRADIENT_EMAIL_FROM_ADDRESS |
Sender email address. |
email.from.name |
string |
"Gradient" |
GRADIENT_EMAIL_FROM_NAME |
Sender display name. |
email.requireVerification |
bool |
false |
GRADIENT_EMAIL_REQUIRE_VERIFICATION |
Whether to enable email verification for new accounts. |
email.smtp.host |
string |
- |
GRADIENT_EMAIL_SMTP_HOST |
SMTP server host name. |
email.smtp.passwordFile |
path |
- |
GRADIENT_EMAIL_SMTP_PASSWORD_FILE |
File containing the SMTP password. |
email.smtp.port |
port |
587 |
GRADIENT_EMAIL_SMTP_PORT |
SMTP server port. |
email.smtp.useTls |
bool |
true |
GRADIENT_EMAIL_SMTP_USE_TLS |
Whether to enable TLS for SMTP. |
email.smtp.username |
string |
- |
GRADIENT_EMAIL_SMTP_USERNAME |
SMTP user name. |
eval
| Option |
Type |
Default |
Env |
Description |
eval.cache.maxAgeDays |
unsigned int |
30 |
GRADIENT_EVAL_CACHE_MAX_AGE_DAYS |
Days after which eval cache blobs leave the cache, regardless of the size limit. |
eval.cache.maxTotalBytes |
unsigned int |
10737418240 |
GRADIENT_EVAL_CACHE_MAX_TOTAL_BYTES |
Total size in bytes of shared eval cache blobs. Older blobs leave the cache until the total is within this size. |
eval.cache.sweepIntervalSecs |
positive int |
3600 |
GRADIENT_EVAL_CACHE_SWEEP_INTERVAL_SECS |
Seconds between eval cache eviction passes. |
eval.maxKeep |
unsigned int |
30 |
GRADIENT_EVAL_MAX_KEEP |
Maximum number of evaluations kept per task. It is the cap for the per-task setting, and new tasks start at the lower of 30 and this value. Set it to 0 to disable the limit. |
frontend
| Option |
Type |
Default |
Env |
Description |
frontend.enable |
bool |
true |
- |
Whether to enable the Gradient web frontend. |
frontend.logoUrl |
null or string |
null |
GRADIENT_FRONTEND_LOGO_URL |
URL of an image to display instead of the Gradient logo on the sign-in pages, in its original colors. Set it to null to keep the Gradient logo. |
frontend.url |
string |
serveUrl |
GRADIENT_FRONTEND_URL |
Public URL of the Gradient frontend, used for links in CI status reports. |
gc
| Option |
Type |
Default |
Env |
Description |
gc.deepIntervalSecs |
unsigned int |
3600 |
GRADIENT_GC_DEEP_INTERVAL_SECS |
Seconds from the end of one background deep garbage collection round to the start of the next. Set it to 0 to run rounds only on request. |
gc.deepPaceMs |
unsigned int |
1000 |
GRADIENT_GC_DEEP_PACE_MS |
Milliseconds between two units of a storage migration or a background deep garbage collection round. Requested rounds run their units without a pause. |
gc.intervalSecs |
positive int |
3600 |
GRADIENT_GC_INTERVAL_SECS |
Seconds between garbage collection passes. |
gc.narTtlHours |
unsigned int |
336 |
GRADIENT_GC_NAR_TTL_HOURS |
Hours to keep a cached path outside the live closure of retained evaluations after its last fetch, or its upload if never fetched. These hours always come on top of gc.narUploadGraceHours. Set it to 0 to keep nothing beyond that grace. |
gc.narUploadGraceHours |
unsigned int |
24 |
GRADIENT_GC_NAR_UPLOAD_GRACE_HOURS |
Hours before the deletion of an unreferenced NAR object, covering the window between its upload and the commit of its database rows. |
gc.orphanDerivationHours |
unsigned int |
24 |
GRADIENT_GC_ORPHAN_DERIVATION_HOURS |
Hours before the deletion of a derivation outside the build closure of every retained evaluation. Quick re-evaluations can reuse the derivation during this grace. Set it to 0 to delete such derivations on the next run. |
gc.wedgedEvalHours |
unsigned int |
24 |
GRADIENT_GC_WEDGED_EVAL_HOURS |
Hours an evaluation can stay in one phase before counting as stuck. Stuck evaluations no longer block evaluation garbage collection. Set it to 0 to treat no evaluation as stuck. |
githubApp
| Option |
Type |
Default |
Env |
Description |
githubApp.enable |
bool |
false |
- |
Whether to enable the GitHub App integration for webhooks and CI status reports. |
githubApp.id |
positive int |
- |
GRADIENT_GITHUB_APP_ID |
GitHub App ID, shown on the App's settings page. |
githubApp.privateKeyFile |
path |
- |
GRADIENT_GITHUB_APP_PRIVATE_KEY_FILE |
File containing the GitHub App's RS256 private key in PEM format. |
githubApp.webhookSecretFile |
path |
- |
GRADIENT_GITHUB_APP_WEBHOOK_SECRET_FILE |
File containing the secret for verifying GitHub App webhook payloads. It must match the secret set on the App's webhook settings page. |
gradientCi
| Option |
Type |
Default |
Env |
Description |
gradientCi.enable |
bool |
true |
GRADIENT_GRADIENT_CI_ENABLE |
Whether to enable the Gradient.CI Servers offer on the workers pages. |
http
| Option |
Type |
Default |
Env |
Description |
http.localIps |
list of string |
[ "192.168.0.0/16" "172.16.0.0/12" "100.64.0.0/10" "10.0.0.0/8" "fc00::/7" ] |
GRADIENT_HTTP_LOCAL_IPS |
Clients in these CIDR ranges receive a cache's local_priority, if set and non-zero. |
http.maxRequestSize |
positive int |
2097152 |
GRADIENT_HTTP_MAX_REQUEST_SIZE |
Maximum HTTP request body size in bytes for most endpoints, to keep an unbounded body from exhausting server memory. Build request blob uploads use a fixed 20 MiB cap. |
http.maxSourceUploadSize |
positive int |
536870912 |
GRADIENT_HTTP_MAX_SOURCE_UPLOAD_SIZE |
Maximum size in bytes of a source upload to POST /build-requests/source (as sent by gradient build) and of a chunked manifest in total. The built-in reverse proxies raise their body size limit to fit it. |
http.trustedProxies |
list of string |
[ "127.0.0.1/8" "::1/128" ] |
GRADIENT_HTTP_TRUSTED_PROXIES |
CIDR ranges of peers allowed to set X-Forwarded-For. The defaults trust a reverse proxy on the same host. |
log
| Option |
Type |
Default |
Env |
Description |
log.chunkBytes |
positive int |
262144 |
GRADIENT_LOG_CHUNK_BYTES |
Target uncompressed size in bytes of a stored build log chunk. Chunks split on line boundaries, and a long line can exceed the target. |
log.level.cache |
null or one of trace debug info warn error |
null |
GRADIENT_LOG_LEVEL_CACHE |
Log level of the cache. Set it to null to use log.level.default. |
log.level.default |
one of trace debug info warn error |
"info" |
GRADIENT_LOG_LEVEL_DEFAULT |
Default log level. |
log.level.proto |
null or one of trace debug info warn error |
null |
GRADIENT_LOG_LEVEL_PROTO |
Log level of the protocol layer. Set it to null to use log.level.default. |
log.level.scheduler |
null or one of trace debug info warn error |
null |
GRADIENT_LOG_LEVEL_SCHEDULER |
Log level of the scheduler. Set it to null to use log.level.default. |
log.level.web |
null or one of trace debug info warn error |
null |
GRADIENT_LOG_LEVEL_WEB |
Log level of the web API. Set it to null to use log.level.default. |
log.traceDir |
null or path |
null |
GRADIENT_LOG_TRACE_DIR |
Directory receiving every closed stage span of the server as JSON lines, one file per process. Set it to null to disable span tracing. |
metrics
| Option |
Type |
Default |
Env |
Description |
metrics.cacheFlushIntervalSecs |
positive int |
10 |
GRADIENT_METRICS_CACHE_FLUSH_INTERVAL_SECS |
Seconds between flushes of cache traffic counters to the database. |
metrics.graphConsistencyIntervalSecs |
unsigned int |
300 |
GRADIENT_METRICS_GRAPH_CONSISTENCY_INTERVAL_SECS |
Seconds between build graph consistency checks. These checks also repair the NAR reference counter. Set it to 0 to disable both. |
metrics.instanceIntervalSecs |
positive int |
30 |
GRADIENT_METRICS_INSTANCE_INTERVAL_SECS |
Seconds between updates of the instance-wide metric window. |
metrics.labelTopn |
unsigned int |
20 |
GRADIENT_METRICS_LABEL_TOPN |
Maximum distinct label values per rollup dimension, by activity. |
metrics.otlp.endpoint |
null or string |
null |
GRADIENT_METRICS_OTLP_ENDPOINT |
OTLP collector endpoint to push metrics to. Set it to null to disable OTLP export. |
metrics.otlp.pushIntervalSecs |
positive int |
30 |
GRADIENT_METRICS_OTLP_PUSH_INTERVAL_SECS |
Seconds between OTLP metric pushes. |
metrics.retention.rawDays |
unsigned int |
14 |
GRADIENT_METRICS_RETENTION_RAW_DAYS |
Days to keep raw phase and worker samples and the per-minute cache and upstream traffic counters. Set it to 0 to keep them forever. |
metrics.retention.rollupDays |
unsigned int |
400 |
GRADIENT_METRICS_RETENTION_ROLLUP_DAYS |
Days to keep minute and hour rollups. Day and week rollups stay forever. Set it to 0 to keep every rollup forever. |
metrics.rollupIntervalSecs |
positive int |
60 |
GRADIENT_METRICS_ROLLUP_INTERVAL_SECS |
Seconds between metric rollup passes. |
metrics.tokenFile |
null or path |
null |
GRADIENT_METRICS_TOKEN_FILE |
File containing the bearer token required to scrape GET /metrics. Set it to null to disable the endpoint. |
metrics.workerSampleIntervalSecs |
positive int |
15 |
GRADIENT_METRICS_WORKER_SAMPLE_INTERVAL_SECS |
Seconds between worker metric samples. |
nar
| Option |
Type |
Default |
Env |
Description |
nar.chunkBytes |
int from 4096 to 4194304 |
524288 |
GRADIENT_NAR_CHUNK_BYTES |
Size in bytes of each outbound NarPush chunk, from 4 KiB to 4 MiB. Smaller chunks can help slow or high-latency links like Tailscale over DERP. |
nar.hotCacheBytes |
unsigned int |
536870912 |
GRADIENT_NAR_HOT_CACHE_BYTES |
Capacity in bytes of the in-memory NAR cache. Set it to 0 to disable the cache. |
nar.maxConcurrentDownloads |
positive int |
16 |
GRADIENT_NAR_MAX_CONCURRENT_DOWNLOADS |
NAR downloads from storage that can run at once across all connections, keeping the storage near its best total throughput. |
nar.maxConcurrentServes |
positive int |
8 |
GRADIENT_NAR_MAX_CONCURRENT_SERVES |
NAR serving tasks that can run at once per worker connection, bounding memory and storage fan-out for large batches. |
nar.maxUploadSize |
positive int |
536870912 |
GRADIENT_NAR_MAX_UPLOAD_SIZE |
Maximum size in bytes of a NAR uploaded to the cache upload endpoint. |
nar.partialTtlSecs |
unsigned int |
86400 |
GRADIENT_NAR_PARTIAL_TTL_SECS |
Maximum age in seconds of an unfinished upload staged under <baseDir>, counted from its last write. Older uploads disappear in the next deep GC. Set it to 0 to keep every unfinished upload. |
nar.sendChunkTimeoutSecs |
positive int |
30 |
GRADIENT_NAR_SEND_CHUNK_TIMEOUT_SECS |
Seconds an outbound NarPush chunk can wait for the WebSocket to drain before an abort of the transfer with NarAbort. |
nar.smallBytes |
unsigned int |
1048576 |
GRADIENT_NAR_SMALL_BYTES |
Size in bytes up to which NAR downloads come from the server itself instead of a presigned S3 URL. NARs up to this size also stay in the in-memory cache. Uploads do not depend on this value. |
nar.storageOpenTimeoutSecs |
positive int |
60 |
GRADIENT_NAR_STORAGE_OPEN_TIMEOUT_SECS |
Seconds to wait for a NAR object stream from storage (for example an S3 GET) before answering the worker with NarAbort. Workers retry after a NarAbort. |
nar.verifyDigest |
bool |
false |
GRADIENT_NAR_VERIFY_DIGEST |
Whether to download NARs committed through presigned S3 uploads and verify their hash, catching same-length corruption at the cost of a full object read. Size checks run either way. Passthrough and REST uploads always go through verification. |
oidc
| Option |
Type |
Default |
Env |
Description |
oidc.clientId |
string |
- |
GRADIENT_OIDC_CLIENT_ID |
OIDC client ID. |
oidc.clientSecretFile |
path |
- |
GRADIENT_OIDC_CLIENT_SECRET_FILE |
File containing the OIDC client secret. |
oidc.discoveryUrl |
string |
- |
GRADIENT_OIDC_DISCOVERY_URL |
OIDC discovery URL. |
oidc.enable |
bool |
false |
GRADIENT_OIDC_ENABLE |
Whether to enable OIDC. |
oidc.required |
bool |
false |
GRADIENT_OIDC_REQUIRED |
Whether to enable OIDC as the only login method. |
oidc.scopes |
list of string |
[ "openid" "email" "profile" ] |
GRADIENT_OIDC_SCOPES |
OIDC scopes to request. |
packages
| Option |
Type |
Default |
Env |
Description |
packages.frontend |
package |
derived |
- |
The gradient-frontend package to use. |
packages.server |
package |
derived |
- |
The gradient package to use. |
permissions
| Option |
Type |
Default |
Env |
Description |
permissions.createCache |
one of none superusers everyone |
"everyone" |
GRADIENT_PERMISSIONS_CREATE_CACHE |
Who can create caches through the API: none (only the declarative state), superusers, or everyone (any authenticated user). |
permissions.createProject |
one of none superusers everyone |
"everyone" |
GRADIENT_PERMISSIONS_CREATE_PROJECT |
Who can create projects through the API: none (only the declarative state), superusers, or everyone (any authenticated user). |
permissions.createTeam |
one of none superusers everyone |
"everyone" |
GRADIENT_PERMISSIONS_CREATE_TEAM |
Who can create teams through the API: none (only the declarative state), superusers, or everyone (any authenticated user). |
postgres
| Option |
Type |
Default |
Env |
Description |
postgres.effectiveCacheSize |
null or string |
null |
- |
effective_cache_size of the cluster set up by postgres.enable. Set it to three quarters of the host's RAM. It is a planner hint, not an allocation. Set it to null to keep the PostgreSQL default. |
postgres.enable |
bool |
false |
- |
Whether to enable a local PostgreSQL database for Gradient. |
postgres.maintenanceWorkMem |
null or string |
null |
- |
maintenance_work_mem of the cluster set up by postgres.enable, used for building indexes and by autovacuum. Each of autovacuum_max_workers can claim this much at once. Set it to null to keep the PostgreSQL default. |
postgres.sharedBuffers |
null or string |
"512MB" |
- |
shared_buffers of the cluster set up by postgres.enable. Size it to a quarter of the host's RAM. Gradient's working set is the build graph's indexes, and the stock 128 MB cannot keep them resident. This value is a single fixed allocation, unlike work_mem and maintenance_work_mem. The default is a quarter of the smallest supported host. Raise it on larger hosts. Set it to null to keep the PostgreSQL default. |
postgres.workMem |
null or string |
null |
- |
work_mem of the cluster set up by postgres.enable. It is the floor for every query. Graph walks raise their own ceiling for a statement, except the walks seeded by a root array, which keep the PostgreSQL default. Every sort or hash node in a query can claim this much. Use "32MB" on a host sized for the three server pools. It is too much for a small host. Set it to null to keep the PostgreSQL default. |
proto
| Option |
Type |
Default |
Env |
Description |
proto.anonymousCache.enable |
bool |
true |
GRADIENT_PROTO_ANONYMOUS_CACHE_ENABLE |
Whether unauthenticated clients can use GET /cache/{cache}/proto for public caches. Private caches always require an API key. |
proto.anonymousCache.maxConnectionsPerIp |
positive int |
32 |
GRADIENT_PROTO_ANONYMOUS_CACHE_MAX_CONNECTIONS_PER_IP |
Maximum simultaneous anonymous /cache/proto connections per client IP. |
proto.discoverable |
bool |
true |
GRADIENT_PROTO_DISCOVERABLE |
Whether to enable incoming worker and federation connections on /proto. |
proto.federate |
bool |
false |
GRADIENT_PROTO_FEDERATE |
Whether to enable federation with other Gradient servers over /proto. |
proto.maxConnections |
positive int |
256 |
GRADIENT_PROTO_MAX_CONNECTIONS |
Maximum simultaneous /proto WebSocket connections. |
proto.public |
bool |
false |
- |
Whether to enable exposing /proto through the reverse proxy for remote workers and federation. |
proto.workerHeartbeatTimeoutSecs |
unsigned int |
120 |
GRADIENT_PROTO_WORKER_HEARTBEAT_TIMEOUT_SECS |
Seconds a connected worker can stay silent before counting as dead, with its jobs re-queued. Workers heartbeat every 10 seconds, giving the default a tolerance of twelve missed beats. The timeout is the only detection for a worker lost without a clean TCP close (OOM kill, frozen host, network partition). Set it to 0 to disable the watchdog. |
pullRequests
| Option |
Type |
Default |
Env |
Description |
pullRequests.commitEmail |
null or string |
null |
GRADIENT_PULL_REQUESTS_COMMIT_EMAIL |
Git author and committer email for commits of the open_pr action. Set it to null to use the default of the Git host, as listed for pullRequests.commitName. |
pullRequests.commitName |
null or string |
null |
GRADIENT_PULL_REQUESTS_COMMIT_NAME |
Git author and committer name for commits of the open_pr action. Set it to null to use the default of the Git host. - GitHub: the App bot, with the commit marked as verified. - Gitea, Forgejo and GitLab: the token owner, with Gradient <gradient@users.noreply.HOST> as fallback. Grant the token the read:user or read_user scope for the owner lookup. |
registration
| Option |
Type |
Default |
Env |
Description |
registration.enable |
bool |
true |
GRADIENT_REGISTRATION_ENABLE |
Whether to enable self-service user registration. |
reverseProxy
| Option |
Type |
Default |
Env |
Description |
reverseProxy.caddy.enable |
bool |
false |
- |
Whether to enable a Caddy virtual host for Gradient. |
reverseProxy.caddy.extraConfig |
strings concatenated with "\n" |
"" |
- |
Additional lines appended to services.caddy.virtualHosts.<name>.extraConfig after the reverse proxy setup. |
reverseProxy.caddy.useACMEHost |
null or string |
null |
- |
Host of an existing ACME certificate for services.caddy.virtualHosts.<name>.useACMEHost, used instead of requesting a new certificate. Set it to null to leave certificate management to Caddy. |
reverseProxy.nginx.enable |
bool |
!reverseProxy.caddy.enable |
- |
Whether to enable an nginx virtual host for Gradient. |
reverseProxy.nginx.manageTls |
bool |
true |
- |
Whether to let nginx obtain and present the TLS certificate itself through the virtual host's enableACME and forceSSL settings. Disable it for upstream proxies (Traefik, Cloudflare, load balancers) that terminate TLS and forward plain HTTP to nginx. Keep useTls enabled in that case for https:// URLs and Secure session cookies. It has no effect with useTls disabled. |
s3
| Option |
Type |
Default |
Env |
Description |
s3.accessKeyId |
null or string |
null |
GRADIENT_S3_ACCESS_KEY_ID |
AWS access key ID. Set it to null to use instance credentials or the environment. |
s3.bucket |
string |
"" |
GRADIENT_S3_BUCKET |
Name of the S3 bucket. The bucket must not have versioning, object lock or replication enabled. Object overwrites happen in place, and old versions stay in the bucket. Versioned buckets thus keep an unreclaimable copy per upload. |
s3.enable |
bool |
false |
- |
Whether to enable storing NARs in S3. |
s3.endpoint |
null or string |
null |
GRADIENT_S3_ENDPOINT |
Endpoint of an S3-compatible service such as MinIO or Cloudflare R2. Set it to null to use AWS. |
s3.maxRetries |
unsigned int |
3 |
GRADIENT_S3_MAX_RETRIES |
Retries of a failed S3 request. |
s3.prefix |
string |
"" |
GRADIENT_S3_PREFIX |
Key prefix inside the bucket, such as gradient/. Leave it empty to store at the bucket root. |
s3.readTimeoutSecs |
unsigned int |
60 |
GRADIENT_S3_READ_TIMEOUT_SECS |
Seconds an S3 response can stall before failing the request. Received chunks reset the timer, and large NARs keep streaming as long as they make progress. |
s3.region |
string |
"us-east-1" |
GRADIENT_S3_REGION |
Region of the S3 bucket. |
s3.retryTimeoutSecs |
unsigned int |
250 |
GRADIENT_S3_RETRY_TIMEOUT_SECS |
Seconds after the first attempt past which no S3 retries start. Keep it above (maxRetries + 1) * readTimeoutSecs to still retry requests that fail on the read timeout. Keep it below 5 minutes as well, since retries reuse the original credentials. |
s3.secretAccessKeyFile |
null or path |
null |
GRADIENT_S3_SECRET_ACCESS_KEY_FILE |
File containing the AWS secret access key. Set it to null to use instance credentials. |
s3.virtualHostedStyle |
bool |
false |
GRADIENT_S3_VIRTUAL_HOSTED_STYLE |
Whether to address a custom s3.endpoint virtual-hosted style (https://<bucket>.<endpoint>/key) instead of path style (https://<endpoint>/<bucket>/key). MinIO, Garage and most self-hosted services need path style. It has no effect without a custom endpoint. |
scheduler
| Option |
Type |
Default |
Env |
Description |
scheduler.clusterPrepareTimeoutSecs |
positive int |
30 |
GRADIENT_SCHEDULER_CLUSTER_PREPARE_TIMEOUT_SECS |
Seconds for every member of a cluster job attempt to accept its assignment. Attempts not accepted by all members in time abort, and their cluster jobs return to the queue. |
scheduler.clusterReserveAfterSecs |
positive int |
600 |
GRADIENT_SCHEDULER_CLUSTER_RESERVE_AFTER_SECS |
Seconds cluster jobs that can start wait for enough simultaneously idle workers before reserving a placement. Reserved workers receive no new single jobs until the cluster job's start or the reservation's expiry. |
scheduler.clusterReserveTimeoutSecs |
positive int |
1800 |
GRADIENT_SCHEDULER_CLUSTER_RESERVE_TIMEOUT_SECS |
Seconds to hold a cluster job reservation before releasing it and planning the cluster job again. |
scheduler.scoringPolicy |
one of simple resource-aware |
"resource-aware" |
GRADIENT_SCHEDULER_SCORING_POLICY |
Policy ranking queued jobs for a requesting worker. Factors of simple: path availability, NAR size, dependency count, waiting time, builtins and fetch worker reservation. Factors of resource-aware: the same plus memory fit, worker saturation, CPU, disk and network affinity and preferLocalBuild. |
scim
| Option |
Type |
Default |
Env |
Description |
scim.enable |
bool |
false |
GRADIENT_SCIM_ENABLE |
Whether to enable SCIM provisioning. |
scim.hardDelete |
bool |
false |
GRADIENT_SCIM_HARD_DELETE |
Whether to enable deleting users on SCIM DELETE instead of disabling them. |
scim.tokenFile |
path |
- |
GRADIENT_SCIM_TOKEN_FILE |
File containing the SCIM bearer token. |
secrets
| Option |
Type |
Default |
Env |
Description |
secrets.cryptFile |
path |
- |
GRADIENT_SECRETS_CRYPT_FILE |
File containing the key used to encrypt secrets in the database. |
secrets.jwtFile |
path |
- |
GRADIENT_SECRETS_JWT_FILE |
File containing the secret used to sign JWTs. |
sentry
| Option |
Type |
Default |
Env |
Description |
sentry.dsn |
null or string |
null |
GRADIENT_SENTRY_DSN |
Sentry DSN to use with sentry.enable set. Set it to null to send reports to the upstream Wavelens instance at reports.wavelens.io. |
sentry.enable |
bool |
false |
GRADIENT_SENTRY_ENABLE |
Whether to enable error reporting to Sentry. |
ssh
| Option |
Type |
Default |
Env |
Description |
ssh.enable |
bool |
false |
GRADIENT_SSH_ENABLE |
Whether to enable the Nix daemon over SSH for ssh-ng:// substituters, nix copy and nixos-rebuild --build-host. |
ssh.hostKeyFile |
null or path |
null |
GRADIENT_SSH_HOST_KEY_FILE |
File containing the OpenSSH private host key. Set it to null to generate an ed25519 key in baseDir on first start. |
ssh.listenAddresses |
list of string |
[ "::" ] |
GRADIENT_SSH_LISTEN_ADDRESSES |
IP addresses for the SSH server to listen on. IPv6 addresses also accept IPv4 clients. The default :: thus means listening on every address of both families. |
ssh.openFirewall |
bool |
false |
- |
Whether to enable the SSH port in the firewall. |
ssh.port |
port |
2222 |
GRADIENT_SSH_PORT |
Port for the SSH server to listen on. |
upload
| Option |
Type |
Default |
Env |
Description |
upload.bytesBudget |
positive int |
8589934592 |
GRADIENT_UPLOAD_BYTES_BUDGET |
Total size in bytes of admitted uploads. Uploads that exceed the remaining budget wait. An upload larger than the whole budget can run alone once nothing else is in flight. |
upload.concurrency |
positive int |
8 |
GRADIENT_UPLOAD_CONCURRENCY |
Number of uploads over 1 MiB (NARs and eval cache blobs) admitted at once across all workers and REST clients. Smaller uploads have a window of 128 of their own. Uploads hold their permit until the object is in storage. Other uploads wait for a permit in the meantime. |
upload.leaseIdleSecs |
positive int |
300 |
GRADIENT_UPLOAD_LEASE_IDLE_SECS |
Seconds granted worker uploads can go without data before they lose their permit and their workers receive a request to retry. |
upload.restWaitSecs |
positive int |
30 |
GRADIENT_UPLOAD_REST_WAIT_SECS |
Seconds NAR uploads to the cache upload endpoint wait for a permit before getting a 503 answer with Retry-After. |
worker
| Option |
Type |
Default |
Env |
Description |
worker.acceptedServerTokensFile |
null or path |
null |
GRADIENT_WORKER_ACCEPTED_SERVER_TOKENS_FILE |
File of token hashes a server must present when it connects to this worker, one peer_id:hash per line. Lines starting with # are ignored. A hash is an argon2 PHC string or the lowercase SHA-256 hex digest of the token, for example from printf %s "$TOKEN" \| sha256sum. The peer ID * matches any peer. The file is only read with worker.discoverable. null accepts every server, and the worker logs a warning at start. |
worker.baseDir |
path |
"/var/lib/gradient-worker" |
GRADIENT_WORKER_BASE_DIR |
Directory holding the worker's state. |
worker.discoverable |
bool |
false |
GRADIENT_WORKER_DISCOVERABLE |
Whether to enable incoming server connections on /proto. |
worker.domain |
string |
"" |
- |
Domain of the worker's reverse proxy virtual host. It only applies with a reverse proxy enabled. |
worker.enable |
bool |
false |
- |
Whether to enable the Gradient worker. |
worker.endpoint |
null or string |
null |
GRADIENT_WORKER_ENDPOINT |
Address at which other members of a cluster job reach this worker, passed through verbatim in the cluster roster. null advertises none. |
worker.environmentVariables |
attribute set of (null or bool or int or string or path) |
{ } |
- |
Extra environment variables of the Gradient worker, taking precedence over variables from other options. null unsets a variable. |
worker.gcrootsDir |
string |
"/nix/var/nix/gcroots/gradient" |
GRADIENT_WORKER_GCROOTS_DIR |
Directory for the indirect GC roots pinning each running build's inputs and outputs against a concurrent nix-collect-garbage. An empty string disables pinning. |
worker.id |
null or string |
null |
GRADIENT_WORKER_ID |
Worker UUID. null generates one on first start and stores it in <worker.baseDir>/worker-id. Set it when the UUID must be known in advance, for example to register the worker in state.workers. |
worker.listenAddr |
string |
"127.0.0.1" |
GRADIENT_WORKER_LISTEN_ADDR |
IP address the worker listens on for incoming server connections. |
worker.peersFile |
null or path |
null |
GRADIENT_WORKER_PEERS_FILE |
File of peer tokens for challenge-response authentication with the server, one peer_id:token per line. Lines starting with # are ignored. <uuid>:<token> *:<token> The peer ID * matches any UUID in the server's challenge. Each token is a 48 byte random secret, for example from openssl rand -base64 48, registered through POST /api/v1/projects/{project}/workers. Pin a project UUID with state.projects.<name>.id to reference it here. null connects in open mode, where the server accepts the worker without a token. |
worker.port |
port |
3100 |
GRADIENT_WORKER_PORT |
Port the worker listens on for incoming server connections. |
worker.serverUrl |
null or string |
null |
GRADIENT_WORKER_SERVER_URL |
WebSocket URL of the Gradient server's /proto endpoint. null leaves the worker waiting for the server to connect. |
worker.useTls |
bool |
true |
- |
Whether to enable TLS. |
worker.zone |
null or string |
null |
GRADIENT_WORKER_ZONE |
Locality label advertised to the scheduler. A cluster job asking for one zone places every member on workers with the same label. null puts the worker in the zone of all unlabelled workers. |
worker.build
| Option |
Type |
Default |
Env |
Description |
worker.build.cgroup |
null or string |
if worker.build.metrics then "/sys/fs/cgroup/system.slice/nix-daemon.service" else null |
GRADIENT_WORKER_BUILD_CGROUP |
Cgroup directory of nix-daemon.service, the parent of the build cgroups. The worker is reading the peak memory of each running build from it and sending the values with its heartbeat. The scheduler is reserving the larger of the predicted and the reported peak. The build cgroups need worker.build.metrics and Gradient's Nix fork on the daemon. null reports nothing. |
worker.build.maxConcurrent |
positive int |
32 |
GRADIENT_WORKER_BUILD_MAX_CONCURRENT |
Maximum simultaneous builds. |
worker.build.maxCores |
null or (positive int) |
null |
GRADIENT_WORKER_BUILD_MAX_CORES |
CPU cores a single build can use, passed as --cores. null uses all cores. |
worker.build.metrics |
bool |
false |
- |
Whether to record per-build peak memory, CPU time, disk I/O and out-of-memory kills. Enabling it is turning on Nix's experimental cgroups feature and use-cgroups and delegating cgroup controllers to nix-daemon.service. The daemon is reporting the values in each build result. Peak memory, disk I/O and out-of-memory kills need Gradient's Nix fork on the daemon, and nix.package is defaulting to its package. Wall-clock time is always recorded. |
worker.build.oomScoreAdjust |
null or integer between -1000 and 1000 (both inclusive) |
500 |
- |
Out-of-memory score adjustment for nix-daemon.service, inherited by every build process. With a positive value the kernel is killing a build process before the worker on a host out of memory, failing that build alone. Evaluation subprocesses use 600 and go first. null leaves the daemon unchanged. |
worker.capabilities
| Option |
Type |
Default |
Env |
Description |
worker.capabilities.build |
bool |
true |
GRADIENT_WORKER_CAPABILITIES_BUILD |
Whether to enable Nix builds. |
worker.capabilities.eval |
bool |
true |
GRADIENT_WORKER_CAPABILITIES_EVAL |
Whether to enable Nix flake evaluations. |
worker.capabilities.federate |
bool |
false |
GRADIENT_WORKER_CAPABILITIES_FEDERATE |
Whether to enable forwarding work and NARs between workers and servers (requiring discoverable). |
worker.capabilities.fetch |
bool |
true |
GRADIENT_WORKER_CAPABILITIES_FETCH |
Whether to enable prefetching flake inputs and sources. |
worker.eval
| Option |
Type |
Default |
Env |
Description |
worker.eval.cache.dir |
null or string |
null |
GRADIENT_WORKER_EVAL_CACHE_DIR |
Eval cache directory, exported to evaluation subprocesses as NIX_CACHE_HOME. null uses <worker.baseDir>/eval-cache. |
worker.eval.cache.share |
bool |
true |
GRADIENT_WORKER_EVAL_CACHE_SHARE |
Whether to share eval cache blobs with other workers through the server. |
worker.eval.forkWorkers |
null or (positive int) |
null |
GRADIENT_WORKER_EVAL_FORK_WORKERS |
Evaluation subprocesses in the pool, equal to the evaluation concurrency. null sizes the pool to the host's core count, capped at 16. |
worker.eval.maxConcurrent |
positive int |
1 |
GRADIENT_WORKER_EVAL_MAX_CONCURRENT |
Maximum simultaneous evaluations. |
worker.eval.maxRss |
positive int |
2147483648 |
GRADIENT_WORKER_EVAL_MAX_RSS |
Resident memory in bytes for an idle evaluation subprocess. Subprocesses above the limit restart after their call and free the Nix heap. A call may grow past the limit, with worker.system.minFreeRamMb as the host's guard. Keep it above a typical evaluation's heap. |
worker.eval.metrics |
bool |
true |
GRADIENT_WORKER_EVAL_METRICS |
Whether to collect per-evaluation Nix statistics (thunks, heap, peak memory, hotspots, flake graph). Disabling it removes their overhead. |
worker.log
| Option |
Type |
Default |
Env |
Description |
worker.log.burstBytesPerMin |
unsigned int |
8388608 |
GRADIENT_WORKER_LOG_BURST_BYTES_PER_MIN |
Build log bytes forwarded per build within any minute. The worker stops forwarding a build's log past this limit. The build itself continues. |
worker.log.fetchFromStore |
bool |
true |
GRADIENT_WORKER_LOG_FETCH_FROM_STORE |
Whether to forward the stored build log of a derivation already built locally and therefore producing no new log. |
worker.log.level.build |
null or one of trace debug info warn error |
null |
GRADIENT_WORKER_LOG_LEVEL_BUILD |
Log level of the builder. null uses worker.log.level.default. |
worker.log.level.default |
one of trace debug info warn error |
"info" |
GRADIENT_WORKER_LOG_LEVEL_DEFAULT |
Default log level. |
worker.log.level.eval |
null or one of trace debug info warn error |
null |
GRADIENT_WORKER_LOG_LEVEL_EVAL |
Log level of the evaluator. null uses worker.log.level.default. |
worker.log.level.proto |
null or one of trace debug info warn error |
null |
GRADIENT_WORKER_LOG_LEVEL_PROTO |
Log level of the protocol layer. null uses worker.log.level.default. |
worker.log.sustainedBytesPerHour |
unsigned int |
67108864 |
GRADIENT_WORKER_LOG_SUSTAINED_BYTES_PER_HOUR |
Build log bytes forwarded per build within any hour. |
worker.log.traceDir |
null or path |
null |
GRADIENT_WORKER_LOG_TRACE_DIR |
Directory receiving every closed stage span of the worker and its eval subprocesses as JSON lines, one file per process. null disables span tracing. |
worker.nar
| Option |
Type |
Default |
Env |
Description |
worker.nar.chunkBytes |
int from 4096 to 4194304 |
524288 |
GRADIENT_WORKER_NAR_CHUNK_BYTES |
Size in bytes of each outbound NAR upload chunk, from 4 KiB to 4 MiB. Smaller chunks can help slow or high-latency links like Tailscale over DERP. |
worker.nar.maxConcurrentUploads |
positive int |
16 |
GRADIENT_WORKER_NAR_MAX_CONCURRENT_UPLOADS |
Upload requests over 1 MiB the worker keeps open at once, waiting for a server grant or transferring. One job holds at most half. Smaller uploads have a window of 128 of their own. The server's upload budget decides how many run. This limit bounds worker memory. |
worker.nar.partialTtlSecs |
unsigned int |
86400 |
GRADIENT_WORKER_NAR_PARTIAL_TTL_SECS |
Seconds after its last write before the deletion of an unfinished NAR download under <worker.baseDir>/nar-partial. 0 disables the cleanup. |
worker.nixDaemon
| Option |
Type |
Default |
Env |
Description |
worker.nixDaemon.maxConnections |
positive int |
worker.build.maxConcurrent * 9 + 16 |
GRADIENT_WORKER_NIX_DAEMON_MAX_CONNECTIONS |
Maximum connections to the local Nix daemon. Each build holds one for its whole run plus up to 8 for parallel NAR imports. The rest is headroom for path checks. |
worker.packages
| Option |
Type |
Default |
Env |
Description |
worker.packages.git |
package |
derived |
- |
Git package available to the worker for cloning repositories. |
worker.packages.gradient |
package |
derived |
- |
The gradient package to use. |
worker.packages.nix |
package |
derived |
- |
Nix package whose nix the worker runs to detect its system features. The default is Gradient's Nix fork, matching the worker's embedded evaluator. |
worker.packages.ssh |
package |
derived |
- |
OpenSSH package used as GIT_SSH_COMMAND to fetch private flake inputs. |
worker.reverseProxy
| Option |
Type |
Default |
Env |
Description |
worker.reverseProxy.caddy.enable |
bool |
false |
- |
Whether to enable a Caddy virtual host for the worker listener. |
worker.reverseProxy.caddy.extraConfig |
strings concatenated with "\n" |
"" |
- |
Additional lines appended to services.caddy.virtualHosts.<name>.extraConfig after the reverse proxy setup. |
worker.reverseProxy.caddy.useACMEHost |
null or string |
null |
- |
Host of an existing ACME certificate to use, passed to services.caddy.virtualHosts.<name>.useACMEHost. No certificate is requested for it. null leaves certificate management to Caddy. |
worker.reverseProxy.nginx.enable |
bool |
false |
- |
Whether to enable an nginx virtual host for the worker listener. |
worker.system
| Option |
Type |
Default |
Env |
Description |
worker.system.architectures |
list of string |
derived |
GRADIENT_WORKER_SYSTEM_ARCHITECTURES |
Nix system types this worker builds for. |
worker.system.cpuCoreScore |
null or (positive int) |
null |
GRADIENT_WORKER_SYSTEM_CPU_CORE_SCORE |
Single-core speed score advertised to the scheduler, higher is faster. null benchmarks the host at startup. |
worker.system.features |
list of string |
[ ] |
GRADIENT_WORKER_SYSTEM_FEATURES |
Nix system features this worker advertises. An empty list detects them at runtime from nix config show system-features, including CPU-derived gccarch-* levels. |
worker.system.minFreeRamMb |
unsigned int |
0 |
GRADIENT_WORKER_SYSTEM_MIN_FREE_RAM_MB |
Free memory in MiB below which the worker kills the one evaluation subprocess large enough to restore the margin. The worker reports that evaluation as failed instead of letting the host freeze. 0 uses 10% of total memory, clamped to 128 MiB to 1 GiB. worker.eval.maxRss still bounds steady-state memory. |
Build Failures
| Status |
Retried |
Cause |
FailedPermanent |
No |
The builder exited non-zero, or no connected worker had enough free memory |
FailedTransient |
Yes, build.maxAttempts times with a doubling build.retryBackoffSecs |
Out of memory, full disk, network or substitution failure, builder crash |
FailedTimeout |
No |
build.defaultTimeoutSecs or build.defaultMaxSilentSecs exceeded |
timeout, maxSilent and preferLocalBuild on a derivation override the server defaults. meta.* attributes never reach the .drv and have no effect here.
Local Worker
localWorker and worker.enable together register the worker of the same host without manual steps.
- A stable worker ID derived from the host name.
- A token generated on first start in
/var/lib/gradient-worker/local-token. A deletion of the file and a restart of both services will rotate the token.
- A worker of the state-declared team
server, set to grant every new project its workers. Projects created before the team existed get the worker once they grant the server team.
The worker will wait in reconnect backoff (at most 60 s) until a project with a cache subscription is available.
Postgres Sizing
postgres.enable will set host-independent defaults, each a mkDefault.
| Setting |
Default |
Reason |
random_page_cost |
1.1 |
SSDs. Keeping the planner on index-only scans of the graph tables |
max_connections |
200 |
Covering the three server pools (56 connections by default) plus autovacuum and psql |
max_locks_per_transaction |
1024 |
Graph writes are locking every shared build they touch. The server is warning below 256 |
The RAM-dependent settings are module options.
| Option |
Rule of thumb |
postgres.sharedBuffers |
A quarter of the RAM, e.g. "4GB" on 16 GB |
postgres.effectiveCacheSize |
Three quarters of the RAM, e.g. "12GB" on 16 GB |
postgres.workMem |
"32MB" for the default pool sizes |
postgres.maintenanceWorkMem |
Up to "1GB" with RAM to spare |
An external database (database.urlFile) must have the same seven values, set by hand.
Prometheus and OpenTelemetry
Setup, metric names and alert examples are in Monitor Gradient.
Hashing
Gradient will hash NARs and cache files with SHA-256. Nix clients need no experimental feature. blake3: hashes from older uploads are still resolving.