# Plan: Add Employee Helper CLI and Home Manager Integration For Safe VPS Provisioning

## Summary

Build a packaged helper CLI that is exported from the existing root flake and installed through the existing Home Manager module. The helper suite will wrap the current manual provisioning flow into validated, guided commands that:

- inspect a live VPS over SSH
- scaffold `hosts/<name>/` files in this repo
- generate or update `disko.nix` safely from probed facts
- create the OpenBao policy/AppRole/bootstrap files for Tailscale enrollment
- generate and optionally execute the `nixos-anywhere` install command
- run post-install checks
- generate the corresponding `colmena` host stanza instructions

The helper CLI will be the supported employee interface. Raw manual commands remain possible, but the helper commands become the documented and safest path.

Chosen defaults:
- helper surface: packaged CLI installed by Home Manager
- mutation mode: interactive confirm for risky operations
- scope: full VPS workflow
- repo writes: yes, write directly into this repo after validation
- OpenBao auth: use the employee’s existing `bao` login/session
- host fact collection: SSH probe live host
- topology: implement as root-flake exports rather than a nested subflake, because the user marked either fine and this is lower-friction

## Goals

### User-facing goal

Employees should be able to provision a new VPS with a small number of safe commands and without manually editing Nix files or remembering the OpenBao/Tailscale bootstrap sequence.

### Success criteria

A new employee with:
- a checkout of this repo
- a working local `bao` login
- SSH access to a target VPS

can run helper commands that:
1. probe the target host and derive safe defaults
2. create or update the host files in `hosts/<name>/`
3. create the OpenBao AppRole bootstrap material for that host
4. print or run the exact `nixos-anywhere` command
5. verify first boot and tailnet enrollment
6. avoid footguns through validation, confirmations, and idempotency checks

## Architecture

## Repo additions

Add a helper package and Home Manager module to the existing root flake.

New files/directories:
- `pkgs/helpers/`
- `pkgs/helpers/cli.py` or equivalent main entrypoint
- `pkgs/helpers/templates/`
- `modules/helpers/home.nix`
- optional `pkgs/helpers/lib/` for command submodules
- optional `pkgs/helpers/README.md` if internal tool docs grow beyond the top-level README

Root flake exports to add:
- `packages.<system>.nodeiwest-helper`
- `apps.<system>.nodeiwest-helper`
- `homeManagerModules.helpers`

Existing `modules/home.nix` should import or include the helper Home Manager module so employees automatically get the command suite.

## Packaging choice

Use a packaged Python CLI with stdlib only unless a strong reason appears otherwise.

Why:
- safer structured argument parsing than shell aliases
- simpler file templating and repo mutation than Bash
- easier SSH probing output parsing and validation
- no need to depend on a persistent external runtime beyond `python3`
- clean packaging through `pkgs.writeShellApplication` or `python3Packages.buildPythonApplication`

The CLI should still shell out to:
- `ssh`
- `bao`
- `nix`
- `git`
- optionally `colmena`

These tools are already aligned with the repo’s workflow.

## Home Manager integration

### Existing behavior

Current `modules/home.nix` only installs:
- `openbao`
- `colmena`

### New behavior

Employees should also get:
- `nodeiwest` helper CLI command
- any runtime dependencies not guaranteed by the base environment, if needed

Recommended command name:
- `nodeiwest`

Reason:
- short
- organization-scoped
- extensible subcommands
- does not collide with upstream tools

## CLI command surface

The CLI should be subcommand-based and decision-complete from day one.

### 1. `nodeiwest host probe`

Purpose:
- SSH into the target host
- collect disk and boot facts
- output normalized machine facts

Inputs:
- `--ip <ip>`
- optional `--user <user>` default `root`

Behavior:
- run the exact discovery commands already documented in the README:
  - `lsblk -o NAME,SIZE,TYPE,MODEL,FSTYPE,PTTYPE,MOUNTPOINTS`
  - boot mode probe via `/sys/firmware/efi`
  - root mount source via `findmnt -no SOURCE /`
  - swap via `cat /proc/swaps`
- parse into structured output
- determine:
  - primary disk candidate
  - root partition
  - boot mode
  - whether current disk naming is `sda` / `vda` / `nvme`
  - whether current system appears UEFI or BIOS

Output:
- human-readable summary
- optional `--json` machine-readable output

Failure conditions:
- no SSH connectivity
- multiple ambiguous disk candidates
- missing required commands on remote host

### 2. `nodeiwest host init`

Purpose:
- create or update `hosts/<name>/configuration.nix`
- create or update `hosts/<name>/disko.nix`
- create placeholder `hardware-configuration.nix` if missing
- print exactly what changed
- ask for confirmation before writing

Inputs:
- `--name <host>`
- `--ip <ip>`
- optional `--user <ssh-user>` default `root`
- optional overrides:
  - `--disk /dev/sda`
  - `--boot-mode uefi|bios`
  - `--swap-size 4GiB`
  - `--timezone UTC`
  - `--tailscale-openbao on|off` default `on`
- optional `--write` or `--apply`
  - without it: dry-run plan only
  - with it: write after interactive confirmation

Behavior:
- call `host probe` implicitly unless overrides are provided
- derive safe defaults from live facts
- validate host name format
- validate that target files do not already contain contradictory data
- create backup copies before overwriting existing tracked files
- update `configuration.nix` with:
  - hostName
  - boot loader config matching boot mode
  - Tailscale AppRole bootstrap enabled by default
- update `disko.nix` with:
  - correct disk device
  - GPT
  - correct UEFI/BIOS boot partition shape
  - ext4 root
  - swap partition using chosen/default size
- create placeholder `hardware-configuration.nix` if absent
- detect if host is missing from `flake.nix` and report exact block to add

Safety checks:
- if boot mode is BIOS but host config would still emit EFI loader, abort
- if probed disk differs from existing `disko.nix`, require explicit confirmation
- if repo has unrelated dirty changes in the exact target host files, warn and stop unless `--force`

### 3. `nodeiwest openbao init-host`

Purpose:
- create the OpenBao policy and AppRole for a host
- generate bootstrap files locally for install-time injection

Inputs:
- `--name <host>`
- optional `--namespace it`
- optional `--secret-path tailscale`
- optional `--field auth_key`
- optional `--auth-path auth/approle`
- optional `--policy-name tailscale-<host>`
- optional `--role-name tailscale-<host>`
- optional `--out ./bootstrap`
- optional `--kv-mount-path <actual-policy-path>`
- optional `--cidr <cidr>` repeatable if later needed
- optional `--apply`
  - without it: show policy/AppRole plan and output paths
  - with it: execute after interactive confirmation

Behavior:
- verify `bao` is available
- verify employee is already authenticated:
  - e.g. `bao token lookup` or equivalent harmless auth check
- generate policy content from inputs
- write policy via `bao policy write`
- write AppRole via `bao write auth/approle/role/...`
- fetch `role_id`
- generate `secret_id`
- create local bootstrap directory:
  - `bootstrap/var/lib/nodeiwest/openbao-approle-role-id`
  - `bootstrap/var/lib/nodeiwest/openbao-approle-secret-id`
- chmod both `0400`
- print exact next-step command to install with `nixos-anywhere`

Defaults:
- namespace `it`
- auth path `auth/approle`
- secret path `tailscale`
- field `auth_key`
- role name `tailscale-<host>`
- policy name `tailscale-<host>`

Important validation:
- if `bao kv get` equivalent path probe fails for the chosen namespace/path, abort with a clear explanation that the KV mount path/policy path likely needs adjustment
- if OpenBao is unreachable or auth is missing, fail fast

### 4. `nodeiwest install plan`

Purpose:
- assemble and print the exact `nixos-anywhere` command
- validate all required local files exist before install

Inputs:
- `--name <host>`
- optional `--ip <ip>` default from flake/host inventory if available
- optional `--bootstrap-dir ./bootstrap`
- optional `--copy-host-keys on|off` default `on`
- optional `--generate-hardware-config on|off` default `on`

Behavior:
- validate:
  - `hosts/<name>/configuration.nix` exists
  - `hosts/<name>/disko.nix` exists
  - bootstrap role_id/secret_id files exist
  - target host is present in `flake.nix` or warn with exact missing stanza
- print the exact install command
- print preflight checklist:
  - provider snapshot taken
  - app/data backup taken
  - public SSH reachable
  - host keys may change

No mutation beyond optional temp checks.

### 5. `nodeiwest install run`

Purpose:
- run the validated `nixos-anywhere` command

Inputs:
- same as `install plan`
- requires explicit `--apply`

Behavior:
- run the same validation as `install plan`
- print command and require interactive confirmation
- execute `nix run github:nix-community/nixos-anywhere -- ...`
- stream logs
- on success, print post-install verification commands

Safety:
- refuse to run without confirmation unless `--yes`
- refuse to run if bootstrap files are missing
- refuse to run if target IP is not reachable over SSH

### 6. `nodeiwest verify host`

Purpose:
- verify first boot and Tailscale/OpenBao bootstrap

Inputs:
- `--name <host>`
- `--ip <ip>`
- optional `--user root`

Behavior:
- SSH to host and run:
  - `systemctl status vault-agent-tailscale`
  - `systemctl status nodeiwest-tailscale-authkey-ready`
  - `systemctl status tailscaled-autoconnect`
  - `tailscale status`
- summarize health
- print actionable failures grouped by likely cause:
  - missing AppRole files
  - OpenBao auth failed
  - wrong secret path / field
  - Tailscale autoconnect blocked

### 7. `nodeiwest colmena plan`

Purpose:
- ensure the host is ready for ongoing deploys
- print or verify the Colmena target block

Inputs:
- `--name <host>`
- optional `--ip <ip>`

Behavior:
- check that `flake.nix` has `colmena.<host>.deployment.targetHost`
- if missing, print exact snippet to add
- print the post-install deploy command:
  - `nix run .#colmena -- apply --on <host>`

## Templates and file generation

Use explicit templates stored under `pkgs/helpers/templates/`.

Templates to maintain:
- `configuration.nix.j2` equivalent
- `disko-uefi-ext4.nix`
- `disko-bios-ext4.nix` if BIOS support is intended
- `hardware-configuration.placeholder.nix`
- OpenBao policy template

Rendering rules:
- do not overwrite files blindly
- preserve existing CA key list and obvious host-specific overrides when possible
- if parsing existing files is too risky, switch to a guarded “abort and instruct user” policy rather than clever rewrites

Chosen default:
- for v1, only support the current ext4+swap single-disk pattern cleanly
- if the existing host directory contains custom structure outside the supported template shape, abort with a clear message instead of trying to merge arbitrarily

## Public interfaces to add

## Flake outputs

Add:
- `packages.<system>.nodeiwest-helper`
- `apps.<system>.nodeiwest-helper`
- `homeManagerModules.helpers`

The root Home Manager module should include or re-export the helper package, so employees get it automatically.

## Home Manager module contract

`modules/helpers/home.nix` should:
- install `packages.${pkgs.system}.nodeiwest-helper`
- optionally ensure helper runtime dependencies are present if not embedded by packaging

## CLI contract

Stable executable:
- `nodeiwest`

Stable subcommands for v1:
- `nodeiwest host probe`
- `nodeiwest host init`
- `nodeiwest openbao init-host`
- `nodeiwest install plan`
- `nodeiwest install run`
- `nodeiwest verify host`
- `nodeiwest colmena plan`

## Error-handling and safety policy

Global rules:
- every mutating command supports dry-run first
- every mutating command requires confirmation unless `--yes`
- every repo write command creates a timestamped backup copy of the target file
- every external command failure is surfaced with:
  - the exact subcommand
  - stdout/stderr summary
  - the next likely fix

Abort conditions:
- current repo is not the expected flake root
- `bao` is not authenticated for OpenBao actions
- target SSH host is unreachable
- disk facts are ambiguous
- target host files already exist in a shape the helper cannot safely reason about

## Out of scope for v1

Not included in the first helper bundle:
- multi-disk or ZFS disko generation
- non-AppRole Tailscale bootstrap flows
- OpenBao response wrapping automation
- automatic `flake.nix` AST rewriting beyond a tightly controlled supported block shape
- full Colmena block auto-editing if the flake layout drifts from the current structure
- vps2/OpenBao server provisioning

For unsupported cases, commands should fail with a precise message and the manual fallback step.

## Implementation details

## Packaging

Recommended implementation:
- Python CLI in `pkgs/helpers/`
- build with `python3` from nixpkgs
- keep dependencies stdlib-only if possible

Why not shell aliases:
- too weak for idempotent file writes and validation
- poorer error reporting
- weaker SSH probe parsing

## File writing strategy

For repo-tracked file mutation:
- read current file if it exists
- compare against generated output
- if unchanged, report no-op
- if changed, write a `.bak.<timestamp>` sibling first, then overwrite target after confirmation

Chosen default:
- direct repo writes are allowed because the user explicitly asked for helpers that make the workflow hard to get wrong
- commands remain conservative and stop on unsupported customizations

## OpenBao auth assumptions

Helpers assume:
- user already has a valid `bao` login/session locally
- `BAO_ADDR` is already set by Home Manager or can be inferred
- namespace defaults to `it`

If auth is missing:
- fail fast
- print the exact `bao` command that must work before retrying

## SSH probing assumptions

Helpers use:
- `ssh root@<ip>` by default
- override via `--user`

The CLI should probe live host facts by default and allow manual overrides for exceptional cases.

## README integration

After the helper CLI exists, the top-level `README.md` should be revised to:
- keep the underlying manual process documented
- make the helper commands the recommended flow
- reduce the manual sequence to a fallback / advanced section

## Test cases and scenarios

### Static/evaluation tests

1. Flake exposes:
- `packages.<system>.nodeiwest-helper`
- `apps.<system>.nodeiwest-helper`
- `homeManagerModules.helpers`

2. Home Manager module installs the helper package without breaking current `modules/home.nix`.

3. Generated commands reflect current repo paths:
- `hosts/<name>/configuration.nix`
- `hosts/<name>/disko.nix`
- `hosts/<name>/hardware-configuration.nix`

### CLI behavior tests

1. `host probe` on a UEFI `/dev/sda` VPS returns:
- boot mode `UEFI`
- disk `/dev/sda`
- root partition `/dev/sda2`

2. `host init` dry-run for a new host:
- prints file plan
- does not write files

3. `host init --apply`:
- creates host files
- uses probed disk and boot mode
- creates backups when overwriting

4. `openbao init-host` with valid local `bao` auth:
- writes policy and AppRole
- creates bootstrap files with mode `0400`

5. `openbao init-host` without local `bao` auth:
- fails before attempting writes

6. `install plan`:
- refuses to proceed if bootstrap files are missing
- prints exact `nixos-anywhere` command otherwise

7. `install run`:
- requires confirmation
- executes the printed command

8. `verify host` after successful install:
- reports agent healthy
- reports tailscaled autoconnect healthy
- reports Tailscale joined

### Failure scenarios

1. Multiple candidate disks on target host:
- helper aborts and requires explicit `--disk`

2. BIOS host with UEFI template:
- helper aborts before writing

3. Existing custom `disko.nix` not matching supported template:
- helper aborts with “manual intervention required”

4. OpenBao secret path exists but field `auth_key` is missing:
- helper fails during validation with a precise message

5. SSH probe works but `nixos-anywhere` later loses connectivity:
- helper reports the exact command that failed and reminds user to recover via provider console/public SSH

## Assumptions and defaults

Chosen assumptions:
- keep one root flake, not a nested helper flake
- helper package is installed through Home Manager
- command name is `nodeiwest`
- direct repo writes are acceptable with backups and confirmation
- employees already authenticate to OpenBao manually before using helper OpenBao commands
- live SSH probing is preferred over manual fact entry
- first release only supports the current single-disk ext4+swap provisioning shape cleanly
- helper commands should be conservative and stop rather than guess in unsupported cases

If these assumptions remain acceptable, the implementation can proceed without further design decisions.