Instance Admin Guide

Setting Up the User Repository

Repository structure, minimal flake setup, project presets, template authoring, and package integration.

The user repository is the single source of truth for everything Pointy runs: step templates, projects, steps, presets, and source files. You wire it up through pointy-stdlib.lib.mkFlake, which turns ordinary Nix files into the flake outputs both backend and frontend consume. The result is a repository you can edit with your usual tools, like VS Code, an LLM agent, or plain git. Pointy picks up the changes without keeping a second internal copy. See Architecture & Configuration for the runtime surrounding the repository.

Minimal repository structure

Create these items at the root of the repository:

  • flake.nix
  • flake.lock
  • templates/
  • steps/
  • projects/
  • srcFiles/
  • optionally presets/, if you prefer to keep preset definitions in separate files

Administrators own templates/, optional presets/, and srcFiles/. The backend owns steps/ and projects/, which it writes in response to UI operations. The srcFiles/ directory must exist even when empty because it is part of the flake interface, not an optional convention.

Minimal flake.nix

At the repository root, create a flake.nix like this:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs";
    pointy-stdlib = {
      url = "github:421anon/pointy-stdlib";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs =
    inputs@{ pointy-stdlib, ... }:
    pointy-stdlib.lib.mkFlake { inherit inputs; } {
      pointy = {
        stepDefs = pointy-stdlib.lib.loadDir ./steps;
        templates = pointy-stdlib.lib.loadDir ./templates;
        projects = pointy-stdlib.lib.loadDir ./projects;
        presets = { }; # or pointy-stdlib.lib.loadDir ./presets; omit entirely when not using presets
        srcFiles = ./srcFiles;
      };
    };
}

This is enough to expose the flake outputs that Pointy needs, including:

  • .#pointy.stepConfig
  • .#pointy.presets
  • .#pointy.stepDefs
  • .#pointy.projects
  • .#pointy.srcFiles
  • .#pointy.dependencies
  • per-system .#pointy.steps.<id>, .#pointy.projectOutPaths, and .#pointy.autocomplete

See the CLI Reference for concrete commands against those outputs.

Project presets

Presets are named bundles of templates that let administrators present a concise menu such as "RNA-seq", "Variant calling", or "Custom QC" instead of requiring every user to reconstruct an approved template set for each project. They constrain the available step types, not the steps themselves, so the workflow remains composed of the same explicit definitions whether its templates came from a preset or a custom selection.

A preset definition has this shape:

{
  displayName = "RNA-seq";
  description = "Standard read-processing templates."; # optional
  sortKey = 10; # optional
  templates = [ "dataSource" "fastqc" "script" ];
}

Then expose the presets through the pointy.presets flake input, for example with pointy-stdlib.lib.loadDir ./presets. Pointy validates preset definitions when evaluating .#pointy.presets; a preset that names an unknown template fails evaluation so the admin can fix it before users see it.

When a repository does not use presets, the pointy.presets input may be omitted from the flake entirely; it defaults to { }. Projects in such repositories still select their templates through a custom templates list.

Each project must define exactly one of:

  • preset = "<preset-name>"; to follow a preset bundle
  • templates = [ "templateA" "templateB" ]; for a custom template list

Changing a project's active preset or custom template list does not delete assigned steps. Steps whose template is no longer active are surfaced in the web UI as orphaned steps until a user re-enables the template or unassigns the step.

Making extra packages available to templates

Templates receive pkgs. The supported way to add custom packages is to extend pkgs in perSystem.

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs";
    pointy-stdlib = {
      url = "github:421anon/pointy-stdlib";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    myTool.url = "github:example/my-tool";
  };

  outputs =
    inputs@{ pointy-stdlib, ... }:
    pointy-stdlib.lib.mkFlake { inherit inputs; } {
      pointy = {
        stepDefs = pointy-stdlib.lib.loadDir ./steps;
        templates = pointy-stdlib.lib.loadDir ./templates;
        projects = pointy-stdlib.lib.loadDir ./projects;
        srcFiles = ./srcFiles;
      };

      perSystem =
        { system, inputs', ... }:
        {
          _module.args.pkgs =
            inputs.nixpkgs.legacyPackages.${system}.extend
              (_: prev: {
                myOrg.myTool = inputs'.myTool.packages.default;
                myOrg.helper = prev.callPackage ./packages/helper { };
              });
        };
    };
}

Templates can then reference pkgs.myOrg.myTool and pkgs.myOrg.helper. A packages/ directory is just a repository convention; Pointy does not load it automatically.

Writing step templates

A template file does two things. The top-level pointy.type.* declaration tells Pointy what kind of step this is: file upload, derivation, or download. The module = ... block declares the step's options and how to build it. Keeping the form declaration and the build logic in the same file means the browser form and the computation stay in sync without maintaining separate schemas.

Example: file upload template

This pattern matches the sample dataSource template:

{
  sortKey = 2;
  displayName = "Data Source";
  description = "Sequencing reads as FASTQ files.";

  pointy.type.fileUpload = {
    allowedExtensions = [ ".fastq.gz" ".fastq" ".fq.gz" ".fq" ];
  };

  module =
    { dream2nix, config, lib, ... }:
    let
      cfg = config.pointy.dataSource;
    in
    {
      imports = [ dream2nix.modules.dream2nix.mkDerivation ];

      config = {
        version = "1";
        name = "dataSource";
        mkDerivation = {
          dontUnpack = true;
          installPhase = "ln -s ${cfg.uploaded} $out";
          dontFixup = true;
          passthru.meta.pointy = {
            type = "dataSource";
            inherit (cfg) id;
          };
        };
      };

      options.pointy.dataSource = with config._pointy.lib.types; {
        id = lib.mkOption {
          type = pointy.string { description = ""; };
          visible = false;
        };

        uploaded = lib.mkOption { type = lib.types.package; };
      };
    };
}

Notes:

  • allowedExtensions controls the frontend file picker.
  • uploaded is filled in by Pointy automatically; template authors do not create a form field for it.
  • cfg.uploaded is a store path pointing at the uploaded payload directory.

Example: derivation template

This pattern matches the sample fastqc template:

{
  sortKey = 5;
  displayName = "FastQC";
  description = "Per-base quality report with FastQC.";

  pointy.type.derivation = { };

  module =
    { dream2nix, config, lib, pkgs, ... }:
    let
      cfg = config.pointy.fastqc;
    in
    {
      imports = [ dream2nix.modules.dream2nix.mkDerivation ];

      config = {
        version = "1";
        name = "fastqc";
        mkDerivation = {
          dontUnpack = true;
          buildInputs = [ pkgs.fastqc ];
          installPhase = ''
            mkdir $out
            ln -s ${cfg.dataSource}/* .
            fastqc \
              --threads 32 \
              --outdir $out \
              *.fastq* \
          '' + cfg.extraArgs;
          passthru.meta.pointy = {
            type = "fastqc";
            inherit (cfg) id;
          };
        };
      };

      options.pointy.fastqc = with config._pointy.lib.types; {
        id = lib.mkOption {
          type = pointy.string { description = ""; };
          visible = false;
        };

        dataSource = lib.mkOption {
          type = pointy.step {
            allowedTypes = [ "dataSource" "merge" ];
            displayName = "Reads";
            description = "FASTQ source to QC.";
          };
        };

        extraArgs = lib.mkOption {
          type = pointy.string {
            displayName = "Extra FastQC args";
            description = "Extra flags appended to the fastqc command.";
            display.command = "fastqc";
          };
          default = "";
        };
      };
    };
}

Notes:

  • pointy.step options resolve to upstream step outputs at build time.
  • pointy.string with display.command renders a command-style argument field in the UI.
  • displayName becomes the visible form label; description becomes the hint shown under the field.
  • passthru.meta.pointy should include both the template type and the step id.

For the available option types, see the Type Reference.

Example: download template

This pattern shows a minimal download template:

{
  sortKey = 3;
  displayName = "Reference Data";
  description = "Fetch a remote reference file into the Nix store.";

  pointy.type.download = { };

  module =
    { dream2nix, config, lib, pkgs, ... }:
    let
      cfg = config.pointy.refData;
    in
    {
      imports = [ dream2nix.modules.dream2nix.mkDerivation ];

      config = {
        version = "1";
        name = "refData";
        mkDerivation = {
          dontUnpack = true;
          installPhase = "ln -s ${cfg.downloaded} $out";
          dontFixup = true;
          passthru.meta.pointy = {
            type = "refData";
            inherit (cfg) id;
          };
        };
      };

      options.pointy.refData = with config._pointy.lib.types; {
        id = lib.mkOption {
          type = pointy.string { description = ""; };
          visible = false;
        };

        downloaded = lib.mkOption { type = lib.types.package; };
      };
    };
}

Notes:

  • pointy.type.download = { }; declares the step kind; there are no extra attributes beyond the empty set.
  • url is managed entirely by the frontend and backend: the user enters it in the web form, and the backend stores the prefetched hash alongside it. You do not define url as a module option.
  • downloaded is filled in automatically from the stored args.downloaded via pkgs.fetchurl. The template declares it as lib.types.package and the stdlib resolver produces the store path.
  • The installPhase can symlink, copy, or otherwise use cfg.downloaded like any other store path.

For the complete type declaration and lifecycle, see the Type Reference.

Resource requirements

Every template declares a default resource policy through requirements:

requirements = args: {
  cpu = 1;
  ram = "1G";
  ior = "0";
  iow = "0";
};

The function receives the step's resolved arguments so it can vary resources by input size or upstream step choice. If a template omits requirements, it falls back to the defaults shown above.

Individual steps can override these defaults by supplying their own requirements attribute in the step definition file (steps/<id>.nix):

{
  type = "myTemplate";
  name = "Large alignment";
  args = { … };
  requirements = {
    cpu = 16;
    ram = "64G";
    ior = "10M";
    iow = "10M";
  };
}

A step-level requirements always takes precedence; when absent (the default), the template-level function applies. See the CLI Reference for inspecting resolved requirements from the command line.

Optional step notices

Templates can publish non-blocking, field-scoped notices through passthru.meta.pointy.notices:

passthru.meta.pointy.notices = [
  {
    field = "nixDeps";
    severity = "info";
    message = "This step uses a package with special license terms.";
  }
];

Pointy fetches these notices when a user opens the step parameters form. Notices are informational only; they do not prevent saving, evaluating, or building the step.

The hidden id option

Every template should define an id option like this:

id = lib.mkOption {
  type = pointy.string { description = ""; };
  visible = false;
};

Pointy injects the numeric step identifier into this option, and templates commonly forward it into passthru.meta.pointy.id. Besides preserving the identity in exported step metadata, this makes the id available to builds and helper scripts without asking users to enter a value Pointy already knows.

Injecting srcFiles into a build

If a derivation type should receive repository-backed source files at build time, enable withSrcFiles:

pointy.type.derivation = {
  withSrcFiles = true;
};

With this flag enabled, Pointy symlinks every top-level entry from srcFiles/<step-id>/ into the build working directory and exposes a Source Files section in the web UI for that step type. The files visible beside a step and those available to its build come from the same repository-backed directory. The sample script template demonstrates the pattern; see Execution and Data Management for the user-facing experience.

Injecting typed column metadata via extras

If a derivation type emits CSV or TSV output files, you can publish typed column schemas so that Pointy's grid viewer treats numeric columns as numbers for filtering and sorting instead of strings. Schemas are produced by a sibling derivation referenced as passthru.meta.pointy.extras.

passthru.meta.pointy = {
  type = "myStep";
  inherit (cfg) id;

  extras = pkgs.runCommand "myStep-extras" { } ''
    mkdir -p $out
    cat > $out/meta.json <<'JSON'
    {
      "results.csv": {
        "columns": [
          { "type": "int",    "nullable": false },
          { "type": "float",  "nullable": true  },
          { "type": "string", "nullable": false }
        ]
      }
    }
    JSON
  '';
};

The extras derivation must produce an output tree mirroring the step's output directory layout. For each directory that contains CSV/TSV files, write a meta.json at $out/<dir>/meta.json keyed by file name. Pointy fetches each meta.json lazily, only for directories the user actually opens.

Each per-directory meta.json has this shape:

{
  "<file-name>": {
    "columns": [
      { "type": "int" | "float" | "string", "nullable": true | false }
    ]
  }
}

Notes:

  • Columns are matched positionally against the file's header row. Extra trailing columns in the file (beyond the schema) default to nullable strings.
  • Unrecognized values for type fall back to string.
  • The extras derivation is built alongside the main step, and is also built lazily the first time the UI requests metadata for a step whose extras have not been realised. Failures are non-fatal: the file still renders, just without typed columns. Errors are visible in the backend log.
  • Each per-directory meta.json is capped at 10 MiB.

Optional: resource requirements for the extras build

By default the extras derivation builds with conservative slurm resources (cpu = 1, ram = "1G", ior = "0", iow = "0"). For larger metadata jobs, attach a passthru.requirements to the extras derivation:

extras = pkgs.runCommand "myStep-extras" {
  passthru.requirements = {
    cpu = 2;
    ram = "4G";
    ior = "0";
    iow = "0";
  };
} ''
  ...
'';

Pointy reads meta.pointy.extras.requirements when scheduling the extras build. The shape matches a regular step's requirements.

After you commit template changes

Once template changes are committed and pushed to the user repository, the backend can evaluate them immediately. Frontend users will see the new or updated step types the next time the UI reloads its step configuration, such as on a page refresh.

For command-line inspection of the generated outputs, see the CLI Reference.

Ready to author templates? The Type Reference lists every option type pointy-stdlib exposes. For workflow-building from the user's side, head to Building Workflows (Steps).

Built withSourcey
GitHub