sandbox-cli
setup

From a cold machine to a sandbox that ran

Pick where sandboxes will run. Every path installs the binaries, starts sandboxd, and ends with sandbox-cli doctor, because installing is the easy half: what a sandboxd can actually deliver depends on the machine and how it was started.

Before you start

Storage first: it is the requirement that fails late, when an image or a snapshot fills a disk halfway through a run.

Storage: 20 GiB free to try it

The base image, installed2.5 GiB
A small image, e.g. python:3.13-slimabout 120 MiB
Each sandboxwhat it writes, up to its disk size (10 GiB by default); the disk is sparse
Each snapshotits memory plus its written disk: 1 GiB for an idle 1 GiB sandbox
Building from a checkouta few GiB of Go caches in your home, and about 0.8 GiB for Studio's UI

A server others depend on needs a drive of its own for sandboxes, so a full one never fills /. Measured on a real host; What the machine needs has the rest.

Where each path keeps it

Linux server (root)
/var/lib/sandboxd. Give it a drive of its own, or bind-mount a directory of a larger filesystem there, before installing: the server path's first steps do it.
Linux, quick try
~/.local/share/sandboxd, on the filesystem that holds your home.
Mac
Images and sandboxes' disks in the container runtime's own store, on the startup disk; ~/.local/share/sandboxd holds only records and the audit log.
Client only
Nothing but the binary and its config: sandboxes live on the server.

And the machine: KVM on Linux, macOS 26 on Apple silicon, Go to build

$df -h /var/lib ~ # free space where the state will go: 20 GiB to try it
$ls -l /dev/kvm # Linux: must exist (a cloud VM needs nested virtualisation)
$go version # 1.25 or later, to build until the first microVM release

Each path's first step checks the rest: the tools, the versions checked, and whether the drive is mounted.

macOS

On a Mac

Each sandbox is a lightweight VM of the native container runtime, with a kernel of its own. The API is the same as on a Linux server or in the cloud, and the local endpoint is a unix socket only you can open.

Before you start: the macOS backend was written and tested on Linux, against a fake runtime that runs the real guest agent. Its first runs on a real Mac (macOS 26.1) boot a sandbox in under a second, but the full check has not run yet. Expect rough edges; Local on a Mac lists the open points.

  1. 1

    Start the container runtime

    $container system start

    The native container runtime runs each sandbox as a lightweight VM with a kernel of its own. It needs macOS 26 or later on Apple silicon; an Intel Mac can only be a client.

  2. 2

    Build and install

    $git clone https://github.com/Amitgb14/sandbox-cli && cd sandbox-cli
    $make studio build # Studio's UI (Node 20+), then bin/sandbox-cli, bin/sandboxd, bin/sandbox-guestd
    $install -d ~/.local/bin
    $install -m 0755 bin/sandbox-cli bin/sandboxd bin/sandbox-guestd ~/.local/bin/
    $export PATH="$HOME/.local/bin:$PATH" # this shell; put the same line in ~/.zshrc or ~/.bashrc
    $command -v sandboxd sandbox-cli # both must print a path under ~/.local/bin

    No published release has sandboxd yet (0.0.1 is the container design's, client only), so this builds from a checkout: Go 1.25+, and Node 20+ for Studio's UI. sandbox-cli, sandboxd and the guest agent go side by side into ~/.local/bin. The guest agent is the Linux arm64 build: it runs inside the sandbox, mounted read-only from beside sandboxd, so any image works and the agent always matches the server. ~/.local/bin must be on your PATH, or the next steps answer sandboxd: command not found: put export PATH="$HOME/.local/bin:$PATH" in ~/.zshrc.

  3. 3

    Start sandboxd as a launch agent

    $curl -fsSL https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/packaging/launchd/dev.sandbox.sandboxd.plist \
    | sed "s#/usr/local/bin/sandboxd#$HOME/.local/bin/sandboxd#" \
    > ~/Library/LaunchAgents/dev.sandbox.sandboxd.plist
    $launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/dev.sandbox.sandboxd.plist

    The launch agent in the repository runs /usr/local/bin/sandboxd; the sed points it at the copy in ~/.local/bin, so nothing needs root. It starts at login, restarts if it exits, logs to /tmp/sandboxd.log, and listens on a unix socket only you can open, which is the CLI's context named local. If you have added other contexts, sandbox-cli context use local switches back to it.

  4. 4

    Ask what this sandboxd can deliver

    $sandbox-cli doctor

    It prints the context, the backend, the API version, the network policy's default and ceiling, what it can do (egress allowlist, suspend, snapshots, volumes, audit) and its limits. A request for anything missing from that list is refused, never served weaker, so this is where you find out, not halfway through an agent's run.

  5. 5

    Run something

    $sandbox-cli run -- uname -a
    $sandbox-cli agent claude

    The first run builds the image's root disk, which takes a while; later ones start fast. A sandbox starts in /sandbox/home, its own home directory, and nothing on your machine is mounted in: ask the agent to clone what it needs. The agent's login is saved when the run ends, so you log in once.

  6. 6

    Open Studio

    $sandbox-cli studio
    # Studio: http://127.0.0.1:7080/#token=…

    The browser view of the same sandboxes: launch a command or an agent, use its terminal, watch its output, files and events. sandbox-cli serves it on a loopback port for whichever sandboxd your context points at, and holds that sandboxd's token itself; open the address it prints, token and all. More about Studio.

What is different from Linux

macOS (local)Linux (self-hosted, cloud)
Sandboxa VM of the container runtimea Firecracker microVM
Egressnone, or open if your policy allows itnone or an allowlist, enforced on the host
Network policy change on a running sandboxnoyes

To stop sandboxd: launchctl bootout gui/$(id -u)/dev.sandbox.sandboxd.

linux, quick try

On Linux, without root

The fastest way to a real microVM: sandboxd in a terminal, as you. Every sandbox is a Firecracker microVM with its own kernel.

Before you start: without root there are no tap devices, so sandboxes get no network at all and a request for an allowlist is refused, never served open. Boot, run, files, snapshots and volumes all work. Run these steps as yourself, not in a root shell (sudo su): they install into your home. For networking, use the Linux server path.

  1. 1

    Check the machine: KVM, the tools, Go

    $uname -m # x86_64 (checked) or aarch64 (built, not yet run)
    $ls -l /dev/kvm # must exist; a cloud VM needs nested virtualisation
    $uname -r # host kernel; checked: 6.12
    # mkfs.ext4 (e2fsprogs 1.43+; checked 1.47.1), ip (iproute2; checked 6.17.0),
    # nft (nftables; checked 1.1.5), and git, make, curl, file to build
    $sudo dnf install -y e2fsprogs iproute nftables git make curl file # Fedora, RHEL and rebuilds
    $sudo apt-get install -y e2fsprogs iproute2 nftables git make curl file # Debian, Ubuntu (not yet checked)
    $go version # 1.25 or later builds the binaries
    $df -h /var/lib ~ # 20 GiB free where the state goes: /var/lib/sandboxd as root, ~/.local/share as you
    $sudo usermod -aG kvm $USER # then log out and back in: your user must open /dev/kvm

    Checked on x86_64, an EL10 distribution with host kernel 6.12, xfs and firewalld; arm64 and other distributions are built for but not yet run. Disk: the base image takes 2.5 GiB installed, each sandbox what it writes (up to 10 GiB by default), each snapshot its memory plus its disk, and the build a few GiB of caches in your home; 20 GiB free is enough to try it. If / is the full one, put the state on a larger filesystem; see What the machine needs. The host kernel only needs KVM, tun and nftables. A distribution's Go is often older than 1.25; go.dev/dl has the current one. Every version checked is in Versions checked. Without root, ip and nft go unused.

  2. 2

    Build and install

    $git clone https://github.com/Amitgb14/sandbox-cli && cd sandbox-cli
    $make studio build # Studio's UI (Node 20+), then bin/sandbox-cli, bin/sandboxd, bin/sandbox-guestd
    $install -d ~/.local/bin
    $install -m 0755 bin/sandbox-cli bin/sandboxd bin/sandbox-guestd ~/.local/bin/
    $export PATH="$HOME/.local/bin:$PATH" # this shell; put the same line in ~/.zshrc or ~/.bashrc
    $command -v sandboxd sandbox-cli # both must print a path under ~/.local/bin

    No published release has sandboxd yet (0.0.1 is the container design's, client only), so this builds from a checkout: Go 1.25+, and Node 20+ for Studio's UI. sandbox-cli, sandboxd and the guest agent go side by side into ~/.local/bin; sandboxd is installed, not started. Run this as yourself, not root. If command -v prints nothing, ~/.local/bin is not on your PATH: the export line puts it there, and the same line in your shell's startup file keeps it there.

  3. 3

    Get Firecracker 1.17.0

    $ARCH=$(uname -m)
    $release_url=https://github.com/firecracker-microvm/firecracker/releases
    $latest=v1.17.0 # the version checked on a real host
    $curl -fsSL $release_url/download/$latest/firecracker-$latest-$ARCH.tgz | tar -xz
    $install -d ~/.local/bin # as yourself, not root: this path runs sandboxd as you
    $install -m 0755 release-$latest-$ARCH/firecracker-$latest-$ARCH ~/.local/bin/firecracker
    $install -m 0755 release-$latest-$ARCH/jailer-$latest-$ARCH ~/.local/bin/jailer
    $~/.local/bin/firecracker --version # Firecracker v1.17.0

    Firecracker and its jailer come from the project's own releases. This takes 1.17.0, the version sandboxd has been checked with, for your architecture, and puts both beside sandbox-cli. It is pinned: a newer one may work, but has not been run. Run this path as yourself, not in a root shell: as root, ~ is /root, and a sandboxd for others belongs under systemd (the Linux server path below).

  4. 4

    Get the guest kernel, 6.1.155

    $ARCH=$(uname -m) # on its own line: zsh escapes ( ) pasted inside a URL
    $mkdir -p ~/.local/share/sandboxd
    $curl -fsSL -o ~/.local/share/sandboxd/vmlinux \
    https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/$ARCH/vmlinux-6.1.155
    $file ~/.local/share/sandboxd/vmlinux # must say ELF 64-bit; anything else is an error page

    Each sandbox boots this kernel, whatever the host runs. It must be built with CONFIG_IP_PNP, CONFIG_VIRTIO_VSOCKETS, CONFIG_OVERLAY_FS, CONFIG_EXT4_FS, CONFIG_VIRTIO_BLK and CONFIG_VIRTIO_NET; Firecracker's CI kernel 6.1.155 has them all, for x86_64 (checked) and arm64. It is pinned: the newest Firecracker release does not always have CI kernels published yet. Keep it anywhere you like and name it with --kernel.

  5. 5

    Start sandboxd in a terminal

    $sandboxd --backend firecracker \
    --kernel ~/.local/share/sandboxd/vmlinux \
    --firecracker ~/.local/bin/firecracker

    It listens on $XDG_RUNTIME_DIR/sandboxd.sock, the CLI's context named local, and its last line names its version and what it will serve. Leave it running and use a second terminal for the rest. If you have added other contexts, switch back with sandbox-cli context use local; otherwise the CLI asks the one you added, and says sandboxd did not answer.

  6. 6

    Or serve it on an IP address

    $IP=10.0.0.17 # this machine's address, as clients dial it
    $curl -fsSLO https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/packaging/fleet/make-certs.sh
    $sh make-certs.sh -o certs $IP
    $sh -c 'umask 077; head -c 32 /dev/urandom | base64 > ~/sandboxd.token'
    $sandboxd --backend firecracker \
    --kernel ~/.local/share/sandboxd/vmlinux --firecracker ~/.local/bin/firecracker \
    --listen $IP:7443 --token-file ~/sandboxd.token \
    --tls-cert certs/node-$IP.pem --tls-key certs/node-$IP-key.pem --allowed-host $IP
    # on a client, with the token and certs/ca.pem copied across
    $sandbox-cli context add box https://10.0.0.17:7443 --token-file sandboxd.token --ca ca.pem
    $sandbox-cli context use box

    Instead of the socket, for other machines. An address other machines can reach needs a token, TLS with a certificate naming that IP, and --allowed-host; sandboxd refuses to start without them and says which is missing. On 127.0.0.1 a token is enough. Loopback, the firewall and what each flag guards are in On an IP address.

  7. 7

    Ask what this sandboxd can deliver

    $sandbox-cli doctor

    It prints the context, the backend, the API version, the network policy's default and ceiling, what it can do (egress allowlist, suspend, snapshots, volumes, audit) and its limits. A request for anything missing from that list is refused, never served weaker, so this is where you find out, not halfway through an agent's run.

  8. 8

    Run something

    $sandbox-cli run -- uname -a
    $sandbox-cli agent claude

    The first run builds the image's root disk, which takes a while; later ones start fast. A sandbox starts in /sandbox/home, its own home directory, and nothing on your machine is mounted in: ask the agent to clone what it needs. The agent's login is saved when the run ends, so you log in once.

  9. 9

    Open Studio

    $sandbox-cli studio
    # Studio: http://127.0.0.1:7080/#token=…

    The browser view of the same sandboxes: launch a command or an agent, use its terminal, watch its output, files and events. sandbox-cli serves it on a loopback port for whichever sandboxd your context points at, and holds that sandboxd's token itself; open the address it prints, token and all. More about Studio.

linux server

On a Linux server

One machine serves the sandbox API to your team or your laptop, with egress enforced on the host, where the guest cannot reach it. It runs as root under systemd; the VMs do not: the jailer gives each one an unprivileged uid and a chroot.

  1. 1

    Check the machine: KVM, the tools, Go

    $uname -m # x86_64 (checked) or aarch64 (built, not yet run)
    $ls -l /dev/kvm # must exist; a cloud VM needs nested virtualisation
    $uname -r # host kernel; checked: 6.12
    # mkfs.ext4 (e2fsprogs 1.43+; checked 1.47.1), ip (iproute2; checked 6.17.0),
    # nft (nftables; checked 1.1.5), and git, make, curl, file to build
    $sudo dnf install -y e2fsprogs iproute nftables git make curl file # Fedora, RHEL and rebuilds
    $sudo apt-get install -y e2fsprogs iproute2 nftables git make curl file # Debian, Ubuntu (not yet checked)
    $go version # 1.25 or later builds the binaries
    $df -h /var/lib ~ # 20 GiB free where the state goes: /var/lib/sandboxd as root, ~/.local/share as you

    Checked on x86_64, an EL10 distribution with host kernel 6.12, xfs and firewalld; arm64 and other distributions are built for but not yet run. Disk: the base image takes 2.5 GiB installed, each sandbox what it writes (up to 10 GiB by default), each snapshot its memory plus its disk, and the build a few GiB of caches in your home; 20 GiB free is enough to try it. If / is the full one, put the state on a larger filesystem; see What the machine needs. The host kernel only needs KVM, tun and nftables. A distribution's Go is often older than 1.25; go.dev/dl has the current one. Every version checked is in Versions checked.

  2. 2

    Give sandboxes a drive of their own

    $lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS # the spare one has no FSTYPE and no MOUNTPOINTS
    $DISK=/dev/nvme1n1 # yours, from the list above
    $lsblk -f $DISK # must show no filesystem and no mount point
    $sudo wipefs -n $DISK # must print nothing: no signature on it
    $sudo mkfs.xfs $DISK # ERASES IT
    $sudo install -d -m 0700 /var/lib/sandboxd
    $echo "UUID=$(sudo blkid -s UUID -o value $DISK) /var/lib/sandboxd xfs defaults,noatime 0 2" \
    | sudo tee -a /etc/fstab
    $sudo systemctl daemon-reload && sudo mount /var/lib/sandboxd
    $sudo chmod 0700 /var/lib/sandboxd
    $command -v restorecon >/dev/null && sudo restorecon -R /var/lib/sandboxd # SELinux hosts only
    $findmnt /var/lib/sandboxd && df -h /var/lib/sandboxd

    Images, every sandbox's disk, snapshots and volumes live under /var/lib/sandboxd; on a drive of their own, a sandbox that fills its disk fills that and not /. Mount it before the next steps, which put the kernel there. Check the drive is the empty one before mkfs, which erases it. By UUID, since device names can swap between boots, and without nofail: the unit has RequiresMountsFor=/var/lib/sandboxd, so sandboxd does not start on the empty directory underneath. Keep the whole directory on it: its parts are hard-linked into each jail. On a machine just for trying, with 20 GiB free on /, skip this step and the next.

  3. 3

    Or, with no spare drive: a directory of a larger filesystem

    $sudo install -d -m 0700 /home/sandboxd /var/lib/sandboxd
    $echo "/home/sandboxd /var/lib/sandboxd none bind 0 0" | sudo tee -a /etc/fstab
    $sudo systemctl daemon-reload && sudo mount /var/lib/sandboxd
    $findmnt /var/lib/sandboxd && df -h /var/lib/sandboxd

    Instead of the step above, when / is small and another filesystem, /home say, has the room: a bind mount puts a directory of it at /var/lib/sandboxd. It is one filesystem, as the jail needs, and the unit waits for it as for a drive.

  4. 4

    Build sandboxd, and install it as root with Firecracker 1.17.0

    $git clone https://github.com/Amitgb14/sandbox-cli && cd sandbox-cli
    $make studio build # Studio's UI (Node 20+), then bin/sandbox-cli, bin/sandboxd, bin/sandbox-guestd
    $sudo install -m 0755 bin/sandboxd bin/sandbox-guestd bin/sandbox-cli /usr/local/bin/
    $ARCH=$(uname -m)
    $release_url=https://github.com/firecracker-microvm/firecracker/releases
    $latest=v1.17.0 # the version checked on a real host
    $curl -fsSL $release_url/download/$latest/firecracker-$latest-$ARCH.tgz | tar -xz
    $sudo install -m 0755 release-$latest-$ARCH/firecracker-$latest-$ARCH /usr/local/bin/firecracker
    $sudo install -m 0755 release-$latest-$ARCH/jailer-$latest-$ARCH /usr/local/bin/jailer
    $firecracker --version # Firecracker v1.17.0

    No published release has sandboxd yet (0.0.1 is the container design's, client only), so this builds from a checkout: Go 1.25+, and Node 20+ for Studio's UI. sandboxd and the guest agent land side by side in /usr/local/bin, where the systemd unit runs them from; the guest agent is put into every image's root disk, so it must sit beside sandboxd. Firecracker and its jailer come from Firecracker's own releases: 1.17.0, the version checked; see Versions checked.

  5. 5

    Make its directories, the guest kernel (6.1.155) and the token

    $sudo install -d -m 0700 /etc/sandboxd /etc/sandboxd/tls /var/lib/sandboxd
    $ARCH=$(uname -m)
    $sudo curl -fsSL -o /var/lib/sandboxd/vmlinux \
    https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/$ARCH/vmlinux-6.1.155
    $file /var/lib/sandboxd/vmlinux # must say ELF 64-bit
    $sudo sh -c 'umask 077; head -c 32 /dev/urandom | base64 > /etc/sandboxd/token'

    The guest kernel is Firecracker's CI kernel 6.1.155, with every option sandboxd needs (as in the quick try above). Image disks are hard-linked into each sandbox's jail, so /var/lib/sandboxd must be one filesystem; on xfs or btrfs snapshot and fork copies are reflinks. The token is what clients present: at least 16 characters, readable by root only. sandboxd refuses a token file other users can read.

  6. 6

    Add TLS and a policy

    $sudo install -m 0600 cert.pem key.pem /etc/sandboxd/tls/
    $curl -fsSL https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/packaging/systemd/policy.example.yaml \
    | sudo tee /etc/sandboxd/policy.yaml >/dev/null

    The certificate is your CA's, for the name clients will use. The policy sets the default image, the resource defaults and limits, the network default and ceiling, and pools. A request can only ask for less, and an unknown key is an error, so a misspelt celing: none cannot leave the server more open than you think.

  7. 7

    Run it as a service

    $curl -fsSL https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/packaging/systemd/sandboxd.service \
    | sudo tee /etc/systemd/system/sandboxd.service >/dev/null
    $sudo sed -i 's/sandbox.example.internal/box.example.internal/' /etc/systemd/system/sandboxd.service
    $sudo systemctl daemon-reload && sudo systemctl enable --now sandboxd
    $journalctl -u sandboxd -f

    Set --allowed-host to the name clients use (the sed line). As root, sandboxd enforces the egress allowlist on the host and runs every VM under the jailer with a uid of its own. It serves 0.0.0.0:7443, and refuses a network address without both a token and TLS. Add --keep-sandboxes to ExecStart and a restart or an upgrade leaves running sandboxes, and their processes, running.

  8. 8

    Ask what this sandboxd can deliver

    $sandbox-cli doctor

    It prints the context, the backend, the API version, the network policy's default and ceiling, what it can do (egress allowlist, suspend, snapshots, volumes, audit) and its limits. A request for anything missing from that list is refused, never served weaker, so this is where you find out, not halfway through an agent's run.

  9. 9

    Install images ahead of their first sandbox

    $sandbox-cli image pull ghcr.io/amitgb14/sandbox-desktop:edge # waits; --no-wait returns at once
    $sandbox-cli image ls # state, size, sandboxes using each

    Otherwise an image is pulled and built into a root disk when a sandbox first asks for it, and that sandbox waits a minute or more. image rm frees one nothing uses. The policy's images: list limits installs as it limits runs; Studio's Images screen does the same.

Before others depend on it, go through the production checklist: room kept back with --capacity-disk-mb, an alert on the disk's real free space, the audit log rotated, volumes backed up. Where it keeps things, upgrading without stopping sandboxes, pools, volumes, the audit log, how the allowlist is enforced and living with a host firewall are in Self-hosting on Linux. To check the whole API against your server, run the conformance suite from a checkout:

$SANDBOX_CONFORMANCE_ENDPOINT=https://box.example.internal:7443 \
SANDBOX_CONFORMANCE_TOKEN=$(cat box.token) \
go test ./internal/api/conformance -run TestEndpoint -v
client

A client pointed at a server

Your laptop runs sandbox-cli and Studio; the sandboxes run on the server. Nothing about the commands changes, only the context.

  1. 1

    Build and install the client

    $git clone https://github.com/Amitgb14/sandbox-cli && cd sandbox-cli
    $make studio build # Studio's UI (Node 20+), then bin/sandbox-cli, bin/sandboxd, bin/sandbox-guestd
    $install -d ~/.local/bin && install -m 0755 bin/sandbox-cli ~/.local/bin/
    $export PATH="$HOME/.local/bin:$PATH" # this shell; put the same line in ~/.zshrc or ~/.bashrc
    $command -v sandbox-cli # must print a path under ~/.local/bin

    For a laptop that should not run VMs, an Intel Mac, or any machine talking to a server; on Windows, go build -o sandbox-cli.exe ./cmd/sandbox-cli. No published release has sandboxd yet (0.0.1 is the container design's, client only), so this builds from a checkout: Go 1.25+, and Node 20+ for Studio's UI. The published 0.0.1 client is the container design's and cannot talk to a sandboxd.

  2. 2

    Add the server as a context

    $sandbox-cli context add box https://box.example.internal:7443 \
    --token-file box.token --ca box-ca.pem
    $sandbox-cli context use box

    Copy the server's token into box.token, and its CA certificate if your machine does not already trust it. The token is read from the file and never appears in an argv. Contexts are how one client talks to your Mac, your server and the cloud: the commands stay the same.

  3. 3

    Ask what this sandboxd can deliver

    $sandbox-cli doctor

    It prints the context, the backend, the API version, the network policy's default and ceiling, what it can do (egress allowlist, suspend, snapshots, volumes, audit) and its limits. A request for anything missing from that list is refused, never served weaker, so this is where you find out, not halfway through an agent's run.

  4. 4

    Run something

    $sandbox-cli run -- uname -a
    $sandbox-cli agent claude

    The first run builds the image's root disk, which takes a while; later ones start fast. A sandbox starts in /sandbox/home, its own home directory, and nothing on your machine is mounted in: ask the agent to clone what it needs. The agent's login is saved when the run ends, so you log in once.

  5. 5

    Open Studio

    $sandbox-cli studio
    # Studio: http://127.0.0.1:7080/#token=…

    The browser view of the same sandboxes: launch a command or an agent, use its terminal, watch its output, files and events. sandbox-cli serves it on a loopback port for whichever sandboxd your context points at, and holds that sandboxd's token itself; open the address it prints, token and all. More about Studio.

fleet

Many servers behind a gateway

sandbox-gateway serves the same API in front of any number of sandboxd nodes. Users reach only the gateway, each with an API key of their own; the gateway picks a node for every sandbox and routes every later call to it.

  1. 1

    Make the certificates

    $curl -fsSLO https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/packaging/fleet/make-certs.sh
    $sh make-certs.sh -o fleet-certs \
    -g gateway.example.internal 10.0.0.17 10.0.0.18

    A private CA, the gateway's client certificate, and a server certificate per node for the address the gateway dials (-g adds one for the gateway's own API). Nodes accept a connection only with the gateway's certificate, and still check their token on every request. Keep ca-key.pem offline.

  2. 2

    Start each node on the private network

    $sudo sandboxd --backend firecracker ... \
    --listen 10.0.0.17:7443 --allowed-host 10.0.0.17 \
    --token-file /etc/sandboxd/token \
    --tls-cert /etc/sandboxd/tls/node-10.0.0.17.pem \
    --tls-key /etc/sandboxd/tls/node-10.0.0.17-key.pem \
    --client-ca /etc/sandboxd/tls/ca.pem --node-id n17

    Each node is a Linux server as above, listening only on an address the gateway shares with it. --client-ca turns on mutual TLS; --node-id must match the name the gateway knows it by, and every sandbox id the node makes carries it; --allowed-host is the host the gateway dials.

  3. 3

    Install the gateway and make the first admin key

    $make build # in the checkout; then
    $sudo install -m 0755 bin/sandbox-gateway bin/sandbox-cli /usr/local/bin/
    $sudo useradd --system --home-dir /var/lib/sandbox-gateway --shell /usr/sbin/nologin sandbox-gateway
    $sudo install -d -o sandbox-gateway -g sandbox-gateway -m 0700 /etc/sandbox-gateway /var/lib/sandbox-gateway
    # certificates, node tokens and nodes.yaml into /etc/sandbox-gateway, owned by it, 0600
    $sudo -u sandbox-gateway sandbox-gateway --state /var/lib/sandbox-gateway/state.json \
    keys create --user ops --scope admin
    $curl -fsSL https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/packaging/systemd/sandbox-gateway.service \
    | sudo tee /etc/systemd/system/sandbox-gateway.service >/dev/null
    $sudo systemctl enable --now sandbox-gateway

    sandbox-gateway needs no root and no KVM: the unit runs it as a user of its own with no capabilities. Nodes go in /etc/sandbox-gateway/nodes.yaml (an example is in packaging/fleet/) or are added with sandbox-gateway nodes add. The key's secret is printed once and stored only as a hash.

  4. 4

    Give users keys

    $sudo -u sandbox-gateway sandbox-gateway --state /var/lib/sandbox-gateway/state.json keys create \
    --user alice --tenant team-a --scope sandbox:read --scope sandbox:create \
    --scope sandbox:delete --scope sandbox:ssh # with the gateway stopped, or POST /v1/admin/keys
    # on alice's machine
    $sandbox-cli context add fleet https://gateway.example.internal:8443 \
    --token-file fleet.key --ca ca.pem
    $sandbox-cli context use fleet && sandbox-cli whoami
    $sandbox-cli ssh demo

    Each user gets an API key with the scopes they need, and sees only the sandboxes they made. Users never hold a node's token. sandbox-cli ssh registers their public key and pins the gateway's host key, after which plain ssh demo@gateway -p 2222 works too.

  5. 5

    Use it: SSH, secrets, jobs and services

    $sandbox-cli run --keep --name demo -- uname -a
    $sandbox-cli ssh demo # a shell; exit leaves demo running
    $scp -P 2222 file [email protected]: # sftp, rsync and ssh -L work too
    $sandbox-cli ssh-access demo --ttl 10m # a one-off ssh line, no key registered
    $printf %s "$TOKEN" | sandbox-cli secret set GITHUB_TOKEN # sealed; never shown again
    $sandbox-cli job run -f job.yaml --wait # a fresh sandbox per run, output kept a day
    $sandbox-cli service deploy -f service.yaml # replicas kept healthy, rolled out one at a time

    One SSH port reaches every sandbox, with no tunnel or port per sandbox: the user name is the sandbox. Logging in needs a key with sandbox:ssh. A secret is sealed on the gateway and reaches only the runs that name it. A job runs each attempt in a fresh sandbox and keeps its output and the files it names; a service keeps its replicas healthy, rolls a new spec out one at a time, and with serve --router-domain answers at <service>--<org>.DOMAIN.

  6. 6

    Organizations

    $sandbox-cli org create acme # needs a key with org:create; you own it
    $sandbox-cli org use acme # this context now acts in acme
    $sandbox-cli org members add bob # owners only; --role owner to share ownership
    $sandbox-cli --org team-a ls # one command in another organization
    $sandbox-cli org ls # the organizations you may act in

    An organization is isolated: its sandboxes, secrets, jobs, services and quota are its own, and a name or an id from another one answers as if it did not exist. A key acts in its own organization, and in those its user created or was added to; nothing else, whatever a request asks for. Removing a member ends what they had open there at once. Studio has the same switcher at the top of its sidebar.

  7. 7

    Revoke a key: access ends now

    $curl -sS --cacert ca.pem -H "Authorization: Bearer $(cat ops.key)" \
    -X DELETE https://gateway.example.internal:8443/v1/admin/keys/KEY_ID
    $sandbox-cli gateway audit # ssh.revoked, api.revoked, job.revoked

    Revoking ends what the key already has open — SSH sessions, followed logs, attach sessions and tunnels — and cancels its user's running jobs when no other key of theirs is active, rather than waiting for them to finish. Each is recorded in the audit log, by key id and never by secret.

On one machine the gateway can sit beside sandboxd, reaching it on a loopback port with its token. Node and gateway flags, the admin API, scopes, quotas, the security model and what is not done yet are in the gateway docs, with SSH, organizations, jobs and secrets, services and operations on pages of their own. To check a fleet end to end — one KVM machine and a laptop, every step with what a pass looks like — follow the fleet walkthrough.

troubleshooting

When a step fails

sandboxd refuses rather than degrades: a control it cannot deliver stops the run with a reason. These are the reasons a new setup usually meets.

firecracker backend: /dev/kvm: open /dev/kvm: permission denied

Your user cannot open /dev/kvm. Add it to the kvm group and log in again. If /dev/kvm does not exist, the machine has no KVM: enable virtualisation in the firmware, or nested virtualisation on a cloud VM.

firecracker backend: mkfs.ext4 is required (e2fsprogs)

Install e2fsprogs (apt install e2fsprogs, dnf install e2fsprogs).

A run asks for an allowlist and is refused: network mode "allowlist" is above this server's ceiling (none)

sandboxd is running without root, so it has no tap devices and no network to give. That is the quick-try path working as designed. Run it as root (the Linux server path) for an enforced allowlist, or ask for --network none.

sandboxd will not start: refusing to serve it without --token-file

A TCP address, loopback included, is reachable by other users. Serve on the default unix socket, or pass --token-file (and TLS for anything beyond loopback).

Every sandbox fails to start: linking … into the jail (the jail must be on the same filesystem as the image cache)

Part of /var/lib/sandboxd (jail/, rootfs/ or volumes/) is on a mount of its own. Its parts are hard-linked into each jail, so they must share one filesystem: mount the disk at the whole state directory instead. Only the audit log can go elsewhere, with --audit-log.

sandboxd does not start after a reboot: Dependency failed for sandboxd

The disk for /var/lib/sandboxd did not mount, and the unit's RequiresMountsFor keeps sandboxd from filling / instead. Check findmnt /var/lib/sandboxd and the UUID in /etc/fstab against blkid.

sandbox-cli cannot connect to sandboxd

Check sandboxd is running and the context points at it (sandbox-cli context ls). On a Mac: launchctl print gui/$(id -u)/dev.sandbox.sandboxd and /tmp/sandboxd.log, and container system start if the runtime is not up. On a server: journalctl -u sandboxd.

On a Mac, a run asks for an allowlist and is refused

The macOS backend does not enforce an allowlist yet, so it does not claim one, and the default is none. To let sandboxes reach the network at all, set ceiling: open in a policy file passed with --policy, knowing that is what it means.

uninstall

Taking it off again

  1. 1

    Remove the binaries

    $curl -fsSL https://raw.githubusercontent.com/Amitgb14/sandbox-cli/main/install.sh | sh -s -- --uninstall

    Deletes sandbox-cli, sandboxd and the guest agent from ~/.local/bin (it checks /usr/local/bin too, and reports a file it may not remove rather than stopping), then reports what else is on disk without touching it. A running sandboxd keeps running until you stop its launch agent or unit.

  2. 2

    Then decide about logins and volumes

    $sh install.sh --uninstall --purge

    --purge also deletes ~/.config/sandbox — your config and every saved agent login — and sandboxd's state directory, which holds image disks, your volumes and the audit log. It is a separate flag because signing you out of every agent and deleting your volumes is not something an uninstaller should do on its own.