Instance Admin Guide

As the instance administrator, you make the analysis tools available and keep Pointy running. You provide the Git repository that stores the work, the machines that run it, access to the AI agent, and backups. Researchers can then create, inspect, and edit steps without managing those services themselves.

This guide covers hosting and operations. For everyday research work, start with AI Agent, Working with Steps, and Running Steps and Viewing Results. To add analysis tools, see Setting Up the User Repository.

Runtime architecture

A Pointy deployment has five moving parts:

1. Frontend: the browser UI where users manage folders, steps, runs, outputs, and share links.
2. Backend: a service that reads and writes the user repository, serves API endpoints, streams live step statuses, accepts uploads, runs ingest jobs, scratch wraps, and agent sessions, and starts or stops builds.
3. Nix evaluation: two long-lived nix repl sessions evaluate the user repository: a pure one for pinned git+file revisions and an impure one for write worktrees and raw expressions. Short-lived nix eval processes handle bulk certificate work, and a write-once certificate store under /home/backend/.local/state/pointy/certificates/ caches each step key's certificate and output paths.
4. User repository: a Git-backed Nix flake that stores templates, optional presets, step definitions, folder definitions, and source files. Folders nest: projects/<id>.nix holds tagged children, and the root folder Home is projects/0.nix.
5. Nix / Slurm runtime: builds are submitted as Slurm batch jobs that call nix build against a pinned commit of the user repository.

When a user saves a folder or step, the backend rewrites its .nix file, commits the change, and pushes it to the configured remote. Organize actions are queued by the UI and sent as one POST /projects/batch five seconds after the last action, producing a single commit that may touch several files. Each run uses a specific Git commit, so later edits do not change what that run means. Status updates appear in every project that links the step directly; ancestors see step counts and status totals from GET /project-rollup.

Setting up a host

Start with an x86_64-linux NixOS host, a hostname pointing at it, and a compatible user repository. Give the backend a deploy key that can both read and push the configured branch. Read-only Git access is not enough: saving a step is a repository write.

Add Pointy as an input to your NixOS flake and pass it to your host modules, for example through specialArgs = { inherit pointy; };. A host module can then contain:

{ pointy, pkgs, ... }:
{
  imports = [ pointy.nixosModules.pointy-host ];

  services.pointy-host = {
    hostname = "pointy.example.org";
    frontendPackage = pointy.packages.${pkgs.stdenv.hostPlatform.system}.frontend;
    basicAuthFile = "/run/secrets/pointy-htpasswd";
  };
  services.pointy-backend.agentEnvFile = "/run/secrets/pointy-agent-env";

  security.acme = {
    acceptTerms = true;
    defaults.email = "admin@example.org";
  };
  networking.firewall.allowedTCPPorts = [ 80 443 ];
}

Replace the hostname, email, and secret paths. The module creates the backend account and service, but does not provision the repository config or deploy key for you. Install config.toml at /home/backend/config.toml, give the account access to its configured key, and keep secrets out of Nix source files and the world-readable Nix store. Runtime secret files should be readable only by the accounts that need them. You can instead set systemd.services.backend.environment.POINTY_CONFIG_PATH to another runtime config path.

The example enables nginx basic authentication. With basicAuthFile = null, the virtual host has no application login gate. Use HTTPS and protect the backend port too: the backend itself does not enforce per-user authorization. A share link pins a revision; it is not an access token.

The supplied Slurm configuration is a single-node starting point. On a non-VM host its fallback advertises only one CPU and 1024 MiB of memory. services.slurm.nodeName is a list option, and the module sets it as a plain value rather than with mkDefault, so a plain override is merged as a second NodeName line; use lib.mkForce to replace the fallback, as the development VM does. Do not advertise more RAM than slurmd reports. The partition is fixed to pointy with Default=YES MaxTime=INFINITE State=UP. The backend submits builds through Slurm; there is no configuration switch to a local non-Slurm runner.

After provisioning the config and secrets, restart backend and check its journal, sinfo, and a small build before inviting users. A successful HTTP response alone does not establish that Git pushes, evaluation, and scheduling work.

Paths and ports

nixosModules.shared installs the backend, Slurm, sandbox, and common nginx settings without the host virtual host. nixosModules.pointy-host adds that virtual host; nixosModules.dev-vm adds the development VM; nixosModules.bootstrap supplies their packages and internal configuration. Use the public module options rather than overriding services.pointy.internal; those options are read-only.

The module does not raise nginx's client_max_body_size, so the default 10 MiB request-body limit applies to uploads through the supplied vhost. Raise services.nginx.clientMaxBodySize if users upload larger files. The /backend/job-ended route is reachable through the proxy; it only wakes build waiters after Slurm confirms the job has ended.

Backend configuration

The backend reads /home/backend/config.toml by default; set POINTY_CONFIG_PATH when the deployment keeps this instance-specific configuration elsewhere. A config file that does not parse crashes startup with Failed to parse config: …, and runtime requests that re-read it fail until it is fixed.

The current configuration format is:

[user-repo]
url = "git@example.com:org/user-repo.git"
keyfile = "/path/to/deploy-key"
branch = "main"

[slurm]
# Optional. partition defaults to ""; omit account and time-limit, or leave them empty, to use Slurm defaults.
partition = ""
account = ""
time-limit = ""
extra = []
# "scheduler" passes --mem and --cpus-per-task; "metadata-only" records the requirements in the job environment only.
enforcement = "scheduler"

[agent]
# The backend invokes runner-command through sbox-command from the session worktree.
sbox-command = "sbox"
sbox-args = ["--bind", "{home}", "{home}"]
runner-command = "pi"
runner-args = ["--model", "deepseek/deepseek-v4-flash"]
timeout-seconds = 1800
output-limit-bytes = 1048576
# Parsed for configuration compatibility; automatic age-based pruning is not currently implemented.
session-retention-days = 7
# Omit this field to use the built-in repository-context bootstrap prompt.
# Set it to an empty string to disable warm bootstrapping and the built-in agent rules.
bootstrap-prompt = "Read AGENTS.md and skill://pointy-router. Do not modify any files. Reply READY when you understand the keyword skill map for non-technical user requests."
title-prompt = "Name the chat that opens with the request in this message. Reply with the title only: five words at most, no quotes, no punctuation, no explanation."

These settings tell the backend:

• which Git remote to use for the user repository
• which SSH key to use for Git operations
• which branch contains the live Pointy state

The backend re-reads config.toml for every repository transaction, Slurm submission, and /user-repo-info request, so changes to the branch, keyfile, and [slurm] settings apply without a restart. A changed repository url applies only at startup: the backend moves /home/backend/user-repo.git to /home/backend/user-repo.git-replaced and clones the new remote fresh.

The optional [slurm] section controls how the backend submits build jobs:

• partition: passed as --partition when non-empty.
• account: passed as --account when non-empty.
• time-limit: passed as --time when non-empty.
• extra: appends raw sbatch flags after the computed flags; a later duplicate flag wins.
• enforcement: sets the Slurm resource-enforcement mode. "scheduler" (the default) passes --mem and --cpus-per-task to sbatch. "metadata-only" omits those two flags but still exports the requirements as POINTY_REQ_CPU, POINTY_REQ_RAM, POINTY_REQ_IOR, and POINTY_REQ_IOW in the job environment, like "scheduler" does. I/O requirements are never enforced, and any other value behaves as "scheduler".

There is no [nix-evaluator] table any more; a table with that name is ignored. Evaluation memory is not configurable: each nix repl session is released when it has been idle above a fixed 2 GiB RSS threshold, and the next request starts a fresh session. There is no warm-up and no sharded cache.

The [agent] section configures the embedded LLM agent. When this section is absent, the backend uses the defaults shown above. Runner secrets are provisioned through the NixOS module, not through config.toml; see NixOS module options. For the user-facing agent workflow, see the AI Agent guide.

Environment variables

NixOS module options

Import pointy.nixosModules.pointy-host to install the backend service, single-node Slurm runtime, nginx virtual host, and agent sandbox. The module provides these options:

The nginx configuration keeps /backend/ SSE responses unbuffered and allows hour-long reads. If you replace the supplied reverse proxy, preserve those settings for the status, ingest, cluster, and agent-turn streams.

Data ingest and scratch space

Uploads and scratch wraps run as background ingest jobs. For an upload, the backend stages the files under uploadDirectory (or the system temporary directory) and runs pointy-ingest DIR store-ref as the backend user. For a wrap, it runs the same program on the chosen scratch directory. The resulting NAR hash goes into the step's args.uploaded.hash, which is committed like any other record change. A browser tab that started or saw the job shows its progress and can run the step once the job succeeds.

The default pointy-ingest copies the directory into the store with nix store add --mode nar --hash-algo sha256, then prints the result from nix path-info --json. It prints no progress lines and deletes nothing. The backend deletes the staged upload after a success that reports references_source: false, and after any failure whatever references_source says.

A host can replace pointy-ingest through ingestPackage, for example with a program that indexes the bytes where they already are instead of copying them. The program prints JSON lines: optional {"progress":{"done":N,"total":N}} lines, then either {"ok":true,"store_path":…,"nar_hash":…,"nar_size":N,"references_source":B} or {"ok":false,"error":…}. Non-JSON lines are ignored, and the last ok line wins. Success needs ok: true and exit code 0; only nar_hash is used. With references_source = true, the store path keeps reading the source bytes, so the backend never deletes them. In that case uploadDirectory must be persistent, and a scratch directory that has been wrapped must not be changed or removed.

An upload request may contain at most 100 file fields; more than 100 fails request parsing with an internal server error. Any field name is accepted, names are reduced to their base name, and the backend enforces no file-size limit. The supplied nginx vhost, however, inherits nginx's default 10 MiB client_max_body_size, so larger request bodies fail at the proxy unless you raise it. An upload never writes srcFiles/: only args.uploaded.hash changes, and it is committed before the job reports success.

Anything inside the canonical scratchDirectory can be wrapped, except the root itself and plain files. Readability is checked when browsing; a wrap of an unreadable directory fails when the job runs. Agents see the scratch directory read-only. When storeUrl = "unix://SOCKET?root=ROOT", the backend serves store files from ROOT/nix/store, and agents get the socket and ROOT bound into their sandbox.

Ingested store paths get no GC root of their own; only a later successful run of the step roots its output. Until such a run happens, a garbage collection can remove the uploaded data.

What the user repository must contain

Keep the flake and templates separate from the step and folder files that Pointy writes. You can evolve the available tools without hand-editing every step. Source files can be edited through the UI as well as directly in the repository.

At minimum, the user repository needs:

• flake.nix
• flake.lock
• pointy.nix
• templates/
• main.pointy, the language entry program
• steps/, with at least one step file
• projects/, including projects/0.nix to create or organize folders
• optional presets/
• optional srcFiles/

flake.nix calls pointy-stdlib.lib.mkFlake { inherit inputs; }; pointy.nix supplies pkgs and the language source as { pkgs; source = ./main.pointy; }. The stdlib copies flake.nix, pointy.nix, templates/, packages/, and main.pointy into the store as global sources, so helper files reachable from templates must live under packages/; changing any global source changes every step key. The current frontend requires step-config version 4. A stdlib that publishes pointy.schemaVersion enables key-based certificate evaluation; older locks fall back to the slower per-project evaluation. Upgrade the notebook and its user-repository inputs together rather than assuming every stdlib branch emits the same format.

For the concrete flake setup, see Setting Up the User Repository.

Folder definitions must choose either a preset or a custom template list; the root's empty templates = [ ] counts as a list. If templates, presets, or step ids disappear from the user repository, Pointy keeps the folder loadable where possible and shows validation errors in the UI. A folder whose header fails to evaluate is served as a placeholder named Project <id>.

Repository synchronization

The backend commits and pushes on every user edit. It does not fetch periodically. A fetch happens only when one of these triggers fires:

• backend startup;
• every unpinned GET /step-config, which the UI sends on each app load;
• every unpinned POST /autocomplete;
• a read of a commit that is missing locally;
• a write that hits Concurrent modification detected…;
• agent session creation, refresh, preparation, or confirmation, which always force-fetches.

Until one of these runs, a direct push to the remote is invisible to unpinned reads. When histories diverge, the backend resolves conflicts automatically where it can.

Writes run in a temporary worktree under /tmp/pointy-worktree-* and take one exclusive lock; reads wait for writes, including while a push runs.

On fetch

When a fetch is rejected as non-fast-forward, the backend:

1. tries to push any unpublished local commits
2. retries the fetch if that push succeeds
3. falls back to a force-fetch if the push also fails

The force-fetch fallback makes the remote authoritative; it is not a guarantee that unpublished local work is preserved. Back up the local repository as well as the remote, and investigate repeated synchronization errors before changing refs by hand.

On push

When a push is rejected because the remote moved ahead, the backend runs git pull --rebase in the temporary worktree and pushes again. A rebase conflict aborts the write with Concurrent modification detected. The remote repository was updated by another process. Please retry your operation. The status code depends on the route: 409 for PATCH /step, /projects/batch, and /src-files; 400 for POST /step and POST /projects; 500 for PATCH /projects.

Live status updates

The frontend opens one app-global SSE connection at /backend/step-status-stream. It takes no project or commit query parameters. Its snapshot events identify a projectId, commit, and a partial map of step statuses. Snapshots are partial, so merge them per stepId. A step's projectId is a project that links the step directly: a step linked from several folders produces several snapshots, ancestors do not receive their descendants' steps, and unfiled steps never appear on the stream.

Status strings are not-started, running, success, failure, built-not-certified, and certification-failed. A step whose output is valid but whose certificate is not shows built-not-certified; when a failure log for the certificate exists it shows certification-failed with the log line as error. Slurm PENDING jobs count as running; there is no queued status. While a build of a step at a commit is tracked, its other statuses are dropped before broadcast and before the replay buffer.

To request a fresh snapshot, the client posts to /backend/project-status?project_id=<id>, optionally with &commit=<hash>. The response returns before evaluation finishes; the results arrive on the global stream as partial snapshots. New connections replay up to 256 recent snapshots, not a guaranteed full inventory of every project.

/backend/cluster-status-stream separately sends cluster availability, a detail string, and running step ids, which feed the status bar. The backend probes sinfo at startup and every 30 seconds. unavailable means the probe failed, degraded means some node or partition is not healthy and the detail names it, and available is neither. Both streams send 30-second heartbeats. The API guide gives payloads and reconnection details.

If you place Pointy behind a reverse proxy, make sure these endpoints are allowed to stay open for a long time and that response buffering is disabled.

Nix evaluation

User-repository expressions are evaluated by two long-lived nix repl sessions: a pure session for pinned git+file revisions and an impure session for write worktrees and raw expressions. Each session serves one request at a time from a FIFO queue and binds each distinct installable once, reusing it for later requests. When a session has been idle, the backend reads its memory; above 2 GiB RSS it closes the session and logs Warning: <PureRepl|ImpureRepl> nix repl released at <N> MiB RSS. The next request starts a fresh session, and a request whose session died is retried once. There is no warm-up and no configurable memory limit.

Bulk certificate work runs short-lived nix eval processes, with step keys evaluated in batches of 512. After every successful write the backend forks a certificate refresh at the new head; a refresh is also scheduled when a status request meets step keys that are not stored yet. Only one refresh runs at a time and the newest pending commit is queued; the journal logs Certificate refresh for <c> skipped|failed: ….

The certificate store at /home/backend/.local/state/pointy/certificates/ maps a step key to its certificate path and <key>.output to its output path. Entries are write-once and never removed; they are a cache that evaluation can warm again.

Agent runner configuration

The embedded agent runner is configured through the [agent] table in config.toml. Every field is optional; the backend falls back to the defaults shown in the configuration reference above.

Working copies and permitted files

Each chat has a separate Git working copy, or worktree. The backend commits the agent's edits there after each request; they reach the shared branch only when a user applies them.

Before a request, Pointy fetches the configured target branch. A failed fetch does not stop the turn; it is logged as Warning: could not fetch latest repo state: …. If the chat has no local commits, its working copy is cleaned and reset to the current branch. Otherwise Pointy attempts to merge the branch into it with the commit message Merge latest <branch> into agent session. A merge conflict is aborted and logged as a [system] line, which the chat does not display; the agent continues from its previous base. Conflicts are resolved later, during apply preparation.

Only changes to these paths can be committed and applied:

• projects/<numeric-id>.nix
• steps/<numeric-id>.nix
• srcFiles/<numeric-step-id>/<relative-path>

Other changed paths are reported in the log and excluded. In particular, the embedded agent cannot update flake.nix or templates; administrators make those changes with their repository tools. Applied changes are ordinary Git commits, available to other instances using that repository after their next fetch.

Applying a changeset uses a temporary worktree at the current target-branch commit. Pointy combines the agent's edits into one commit, evaluates the result, refuses the changeset if it introduces evaluation failures that are not already on the shared branch, and rejects it if it touches a reviewed step. It checks that neither side has changed during preparation, then pushes it. When preparation finds conflicts, the agent resolves them in that worktree and you prepare again. The API guide describes the preparation and confirmation requests.

Sandbox and credential boundaries

The agent process runs inside sbox (configurable via sbox-command and sbox-args), with the session worktree as its working directory. By default, the per-session {home} directory is bind-mounted into the sandbox; the user-repository Git directory and the optional warm-session file are mounted read-only. Chat turns do not receive the Pi configuration directory itself: Pointy copies models.json and settings.json and links the extensions into the per-session HOME. The config directory is bound read-only only for title generation and warm bootstrap, and the apply worktree is bound read-write while resolving a conflict. The sandbox also gets the Nix daemon compatibility socket when present and, with storeUrl, the store socket read-write and ROOT read-only. The backend's own HOME, SSH keys, and Git credentials are not passed to the runner.

The runner receives the backend's PATH, selected user/locale/XDG variables, and these provider keys when present: DEEPSEEK_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, GROQ_API_KEY, CEREBRAS_API_KEY, XAI_API_KEY, OPENROUTER_API_KEY, MISTRAL_API_KEY, GOOGLE_API_KEY, and GEMINI_API_KEY. When storeUrl is set, it also receives NIX_REMOTE. Each turn additionally receives HOME, POINTY_AGENT_WORKTREE, POINTY_AGENT_SESSION_ID, and POINTY_AGENT_OUTPUT_MARKER. Other backend environment variables are not passed through. Supply provider secrets through services.pointy-backend.agentEnvFile, not config.toml.

The module configures sbox with network = "isolated", shared shell history off, and known-host sharing on. Network isolation here is a sandbox setting, not a promise that the agent cannot contact a model provider. Check the sandbox's connectivity and any extra binds as part of your deployment policy.

The module grants the backend user a narrow polkit rule for managing systemd units whose names start with nix-build-, and declares the pointy-builds slice. That is separate from ordinary step scheduling through Slurm. Treat custom sandbox binds and privilege overrides as security-sensitive configuration.

Placeholder expansion

For normal agent turns, placeholders in both sbox-args and runner-args are expanded before the process starts:

• {worktree}: absolute path to the session worktree (the sandbox CWD)
• {home}: per-session HOME directory ({worktree}/../home), created before the runner starts
• {sessionRoot}: {worktree}/.., the session root directory
• {sessionId}: short opaque session identifier

Title generation expands the same placeholders. Warm bootstrap expands only sbox-args and does not use runner-args.

The backend manages RPC mode, session continuation, and forking itself. Do not put --mode, --session, --fork, -c/--continue, --no-session, or -p/--print in runner-args. There is no {prompt} substitution; that legacy token is stripped, and prompts go over the runner's RPC input.

Model selection and chat titles

Select the model explicitly in runner-args. The supplied Pi configuration uses deepseek-v4-flash, but a provider registry or resumed conversation can otherwise select a different model. --model takes precedence. Custom models.json and settings.json files belong under agent/ in your piConfigDir.

title-prompt controls automatic naming of unnamed chats. A blank value disables it. Title generation is a separate short runner request and does not overwrite a user-chosen name.

Warm bootstrap

When bootstrap-prompt is non-empty, the backend prepares a Pi session with the repository context already loaded, including the built-in agent rules that describe the permitted paths, the folder format, and the evaluation-refusal check. It reuses that preparation for new chats until the repository base commit or prompt changes. Later requests continue the chat's own conversation. Set bootstrap-prompt to an empty string to skip this preparation; this also removes those built-in rules.

Time, output, and session storage

• timeout-seconds (default 1800): active-time budget for a single agent turn, sampled every second and paused while a question waits for an answer. On expiry the backend logs Runner timed out; aborting the turn, sends pi an abort request, waits 10 seconds, then terminates the process.
• output-limit-bytes (default 1048576, 1 MiB): per-turn log size cap. Once the log reaches this limit, further output lines are discarded.
• session-retention-days (default 7): accepted by the current configuration schema but not yet used for automatic pruning. Users remove stored sessions with Delete permanently after archiving them.

The [agent] section is re-read when a turn starts, when a session is created, and when a changeset is applied, so edits apply to the next turn without a restart.

NixOS integration

Use services.pointy-backend.agentEnvFile for provider API keys and services.pointy-backend.piConfigDir for model and tool configuration; both options are described under NixOS module options.

The service copies piConfigDir to /home/backend/.pi before each start and links the question extension there. Only symlinked entries under agent/extensions/ reach chat sessions; ship a custom extension as a symlink for it to load in chats. Make durable configuration changes in the source directory, not in the generated copy. A missing optional environment file does not prevent service startup, so a running service does not prove the model key was loaded.

Build execution and GC roots

Each build runs as a Slurm batch job. The job name is pointy-nix-build- plus the sanitized build-target store path plus a 16-character FNV-1a hash, at most 128 characters in total. The job comment is a JSON restore record, {"kind":"step"|"extras","step":<id>,"commit":"<c>","outPath":"<target>"}, and every job exports POINTY_REQ_CPU, POINTY_REQ_RAM, POINTY_REQ_IOR, and POINTY_REQ_IOW. Job stdout and stderr go to /dev/null.

The supplied module configures a single-node runtime. A multi-node deployment needs matching scheduler configuration and access to the required repository and store paths; the module does not arrange a distributed Nix store for you. The module installs a Slurm job-completion hook that POSTs the job name to http://127.0.0.1:8081/job-ended for pointy-nix-build-* jobs; the backend only wakes waiters after squeue confirms the job has ended. A 30-second reconciler covers job ends missed while the backend was down, and the backend re-attaches running jobs from their JSON comments at startup.

Running a step expands its complete transitive step-reference graph. Pointy submits one Slurm job for each unbuilt step in dependency order, or reuses the step's valid build target and in-flight jobs with the same name. Dependent jobs are connected with --dependency=afterok: over every enqueued dependency and --kill-on-invalid-dep=yes; a dependent is not submitted when one of its prerequisites cannot be evaluated or scheduled, and the stream reports dependency step(s) [<ids>] could not be scheduled.

On a revision whose stdlib publishes pointy.schemaVersion, the build target is the step's certificate (#pointy.steps.<id>.certificate) rather than its output. A step whose output is already valid but whose certificate is not is reported as built-not-certified and still gets a certificate build; a certificate whose log shows a failure is reported as certification-failed.

When a watched run ends successfully, the backend registers GC roots with nix-store --add-root /home/backend/.local/state/pointy/gc-roots/<store-basename> --realise <path>:

• a run that finds a valid certificate roots the certificate;
• an enqueued or re-attached job that ends with a valid certificate roots the certificate and the step output;
• extras outputs are rooted.

Roots are created only on run paths. A build that finished while nothing watched it, a status probe, a review, and an ingested upload create no root. The backend never removes roots. A GC root is a reference, not a backup: removing one permits collection and can make old share links or reviewed outputs unavailable.

Logs and troubleshooting

On the host:

systemctl status backend nginx slurmctld slurmd munged
journalctl -u backend -f
journalctl -u slurmctld -u slurmd -f
sinfo
squeue
scontrol show job <job-id>

Use the step's log popover or nix log <output-or-derivation-path> for Nix build output. On certificate revisions the logged target is the certificate derivation. Slurm submissions send their own stdout/stderr to /dev/null; do not expect slurm-<job-id>.out to contain the build log. Agent turn logs are at /home/backend/agent-sessions/<session-id>/turns/<turn-id>.log.

The backend loads the config, ensures or clones the repository, resets stale agent sessions, and fetches before it listens. After it listens, it restores Slurm jobs and running statuses, runs the first cluster probe, and starts the 30-second cluster poller and job reconciler. There is no project warm-up and no dedicated readiness endpoint that certifies all of these operations. Monitor service health, Git access, disk/store capacity, and Slurm separately.

Backups and recovery

Git preserves definitions, not all the data. A useful backup covers:

• The remote repository and /home/backend/user-repo.git, including unpublished commits and agent branches (agent/<session-id>), plus /home/backend/user-repo.git-replaced if a repository URL change created it.
• /home/backend/agent-sessions/, with its worktrees, conversation files, per-session homes, and turn logs. Each session directory holds session.json, turns/<turn-id>.json and .log, worktree/, home/, and a transient apply-worktree/; warm-template/ is shared. Capture it consistently with the repository it points into.
• The required Nix store closures, especially uploaded input objects and results that cannot simply be fetched again. Store hashes in a step file are not copies of the bytes, and an ingested path has no GC root until a successful run roots it. If your ingest program references its source, also back up uploadDirectory and every wrapped scratch directory, because the store paths read those bytes.
• GC-root state under /home/backend/.local/state/pointy/gc-roots/, restored together with its store targets and Nix registrations.
• Runtime configuration, deploy keys, and provider secrets: /home/backend/config.toml, /home/backend/agent-env (a symlink when agentEnvFile is set), /home/backend/.ssh/known_hosts, and your NixOS configuration, stored with appropriate access protection.

The diff reports under /home/backend/.local/state/pointy/diff-reports/, the certificate store under /home/backend/.local/state/pointy/certificates/, and /home/backend/.pi can be regenerated; they are not substitutes for the inputs above. Preserve Nix logs too if you need historical failure diagnostics.

For a consistent recovery point, stop new edits and agent turns, allow or stop scheduled builds, then snapshot the repository, sessions, and store using your normal system backup tooling. A filesystem copy of /nix/store alone is not a complete Nix store backup: preserve its database and registrations, or export/import the required closures through Nix.

Test restoration on a separate instance with the same paths and ownership. Check an old shared result, a reviewed step, an uploaded input, and a saved chat before considering the backup usable. Do not let a recovery test push changes to the production branch.

Upgrading

Back up first and let active agent turns finish. Pin a matching notebook, stdlib, and language revision, then evaluate the user repository and build a representative step before deploying the new frontend/backend together. Keep the old configuration and input lock files so you can reproduce the previous state.

The current platform expects folders: projects with tagged children and a root projects/0.nix. Repositories written in the former steps format can still be read, but every create and organize write refuses them with projects/<id>.nix uses the legacy steps format. Run pointy-migrate-project-tree to convert it. Run nix shell .#backend -c pointy-migrate-project-tree <user-repo-worktree> on a checkout of the branch, commit and push the result, and move the stdlib to a revision that reads children (353a8d1 or later) in the same change. The tool refuses to run when projects/ is missing or when projects/0.nix already exists, and it neither commits nor pushes. Remove any [nix-evaluator] table from config.toml; it is ignored.

On restart, the backend recovers running Slurm jobs from their recorded metadata and re-attaches only kind: step jobs that squeue still lists; extras jobs are not re-attached. Agent sessions marked running are reset, and unfinished turns get runner exited while backend was offline. Even so, avoid upgrading while people are editing or agents are working. A NixOS rollback restores software, not repository commits, deleted chats, or store objects removed by garbage collection.

Development VM

From the Pointy application checkout, copy backend/example-config.toml to backend/dev-config.toml and fill in the repository URL, branch, and the deploy-key path inside the guest. Run the launcher from the repository root; it requires that file and otherwise fails with error: backend/dev-config.toml is missing. Copy backend/example-config.toml to backend/dev-config.toml and fill in your settings. It shares the backend directory at /shared/dev-config; the guest installs the TOML as /home/backend/config.toml with mode 0600 and copies an optional backend/agent-env in for model credentials. Keep both files out of commits.

nix run .#dev-vm

The launcher runs nixos-shell --flake .#dev-vm. The VM has 4 CPUs, 8 GiB RAM, and a 20 GiB virtual disk. It forwards host localhost:8080 to guest nginx and localhost:2222 to guest SSH. The guest nginx serves only /backend/ and /api/; it has no UI, so port 8080 is not the app. It is a backend development environment, not the production static-frontend host.

For the browser UI, run the frontend dev server from frontend/:

nix develop .#frontend -c npm run dev-vm

nix develop .#frontend resolves to the frontend package's development shell and provides npm, Node, Elm, and webpack, with NODE_PATH pointing at the Nix-built dependencies, so no separate npm ci is needed. Open http://localhost:3000. Webpack proxies /backend through the VM's nginx. The backend shell is nix develop .#backend.

Press Ctrl+a, then x, to stop the VM console. Restart the VM to pick up backend changes. The VM disk retains state between launches; deleting nixos.qcow2 is a destructive reset, not a normal restart. If a host Nix daemon is exposed on port 5000, the guest's daemon proxy uses it; otherwise it uses the guest daemon. Do not expose that daemon port on an untrusted network. The guest shares the host's ~/.cache/nix at /var/cache/nix and the host's /nix/var/log.

Related admin docs

• Setting Up the User Repository
• Type Reference
• CLI Reference
• AI Agent
• Using the Backend API