# Clean-Room Onboarding Self-Assessment — RUNBOOK (autonomous, on `agentops@claudin`)

> Goal: re-run the 5-test onboarding battery on a host **outside** the current environment to obtain a **new precision
> rate that removes the specific T1 filesystem-discovery leak** of run 1 (residual bias mitigated, not zero — see below),
> with **no external intervention** during the run.
> The ONLY bridge between environments: the **same Claude Code account** + the **published FUV** over HTTPS.
> Nothing is hand-copied; no env vars, no secrets cross over; **no codebase clone is required** (see the default path below).

## Onboard WITHOUT cloning the codebase (DEFAULT)

Run 1's bias was the **control** arm reading the harness off local disk (cwd was `/data/ssothrut`). The fix that
actually matters is **not a different host** — it is a **text-only control** (no filesystem access at all). With that,
the leak is gone from ANY host, and onboarding needs **no local clone of `ssothrut` or `mywai`**. Everything comes from
what is already published:

- **ring-0 / T2 answer key / T3 leg** → the **live FUV already served by the same VPS that hosts the site**:
  `https://ssot.com.br/ns/core#` (per-term `…/ns/core/term/*.jsonld`).
- **the runnable protocol** → 3 small files fetched from GitHub (already on `main`) — a scorer, a rubric, an arm
  script; NOT a codebase. **The repo is PRIVATE, so `raw.githubusercontent.com` returns 404** — fetch via the
  authenticated `gh api` (the account's GitHub auth IS the bridge):
  ```bash
  p=repos/aiob3/ssothrut/contents/labs/clean-room-onboarding/protocol
  for f in score.py rubric.json onboarding-battery.wf.js; do
    gh api -H "Accept: application/vnd.github.raw" "$p/$f?ref=main" > "$f"
  done
  ```
  (verified on `agentops@claudin`, 28/06/2026 — the earlier `curl $RAW/...` form 404s on the private repo.)
- **terrain (`mywai`)** → injected as text by the arm script (its `TERRAIN` constant) — not cloned.

Then, from any host on the same Claude Code account:
1. **Run the 8 arms** — `Workflow({ scriptPath: "<abs>/onboarding-battery.wf.js" })` → save the returned JSON verbatim
   to `answers.json`. The **control arm is text-only** (no repo, no filesystem), so the run-1 leak cannot recur.
   Fallback: spawn the 8 prompt pairs as `Task` subagents if the `Workflow` tool is absent.
2. **Score** (no clone; the live FUV is both the T2 answer key and the T3 leg):
   ```bash
   python3 score.py --rubric rubric.json --answers answers.json --out results.json
   ```
   Here T3 = live reachability + Content-Type. Full local↔live **byte-parity** additionally needs a local projection
   (`--ns-local`), provided only by the optional clone path below — or fetch the source from GitHub raw and compare.
3. Read `results.json` → the de-biased `precisionRatePP`.

**Eligible enhancement (operator-gated, S8):** publish this same bundle (kit + RUNBOOK + `score.py` + `rubric.json`) on
the **VPS that already serves `ssot.com.br/ns/*`**, e.g. under `https://ssot.com.br/onboarding/`, so step-0 `curl`s from
the same host as the published site — fully self-hosted, no GitHub dependency. Deploy via the normal S8 gate.

The host-isolation + clone layout below is **OPTIONAL** — use it only for local byte-parity (`--ns-local`) or to inspect
the full repo. It is **NOT** required to onboard or to measure.

## (Optional) Why run 1 was biased — and the heavier host-isolation path

Run 1 executed with cwd = `/data/ssothrut` (the harness's real home). The **control** arm — meant to have NO SSOTHRUT
material — could and did *discover* `/data/ssothrut/AGENTS.md` on disk, inflating its T1 orientation score to 60% and
making the measured T1 delta (+40pp) a **lower bound**. On `claudin`, `/data/ssothrut` does not exist; the harness is
present only as a clone in an **unrelated path the control is never told about**, and the run executes from a **neutral
workspace** containing only the target repo. This **eliminates the specific run-1 leak** (cwd was the canonical harness,
trivially read). It does **not** make the host perfectly hermetic: a control subagent that *deliberately* searches
outside its cwd (e.g. `find ~ -name AGENTS.md` against `~/cr-kit`) could still reach the clone — the control prompt
instructs against this, but one host cannot hard-prevent it. So the clean-room **mitigates** the bias to a small
voluntary-exploration residual; it does not claim zero channel.

## Prerequisites (claudin)
- Logged into the **same Claude Code account** — the single bridge. That account also carries the GitHub auth
  (`gh auth status` shows `aiob3`), which is how the repo is cloned; **no separate credential or hand-copied file
  crosses environments**, and the `Workflow` tool used in step 4 is a Claude Code harness capability, not an in-repo dep.
- `python3` (3.12, stdlib only) on PATH.
- Network egress to `https://ssot.com.br` (Tailscale-gated per project) — used only for the T3 leg.

## Layout (this is the isolation contract — do not collapse the two dirs)
```
~/cr-kit/    <- ssothrut clone + scorer + answers.json + results.json   (NEVER a parent of cr-run)
~/cr-run/    <- NEUTRAL run workspace; holds ONLY the target repo (mywai)
```

## Steps (run top-to-bottom; autonomous)
```bash
# 0. Clean slate
rm -rf ~/cr-kit ~/cr-run && mkdir -p ~/cr-kit ~/cr-run

# 1. Bring the onboarding standard + scorer via the GitHub bridge (into cr-kit, OUTSIDE the run workspace).
#    Until PR #1 is merged to main, the lab lives on its branch — clone that ref explicitly.
#    After merge, the plain `gh repo clone aiob3/ssothrut ~/cr-kit/ssothrut` (default branch) suffices.
gh repo clone aiob3/ssothrut ~/cr-kit/ssothrut -- --branch feat/clean-room-onboarding

# 2. Bring the TARGET repo (terrain) into the NEUTRAL run workspace
gh repo clone aiob3/mywai ~/cr-run/mywai
```
3. **Open Claude Code with cwd = `~/cr-run`** (NOT inside any ssothrut clone). Confirm `~/cr-run` has no SSOTHRUT files:
   `ls -R ~/cr-run | grep -i ssot` must return nothing.
4. **Run the measurement battery** (the 8 arms; kit reaches treatment as injected text only). **Use ABSOLUTE paths in
   tool arguments** — tilde (`~`) is NOT expanded inside tool-arg strings the way it is in the shell; resolve `$HOME`
   first (e.g. `echo $HOME` → `/home/agentops`) and substitute the real absolute path:
   `Workflow({ scriptPath: "/home/agentops/cr-kit/ssothrut/labs/clean-room-onboarding/protocol/onboarding-battery.wf.js" })`
   Save the returned JSON **verbatim** to `/home/agentops/cr-kit/answers.json` (NOT into `cr-run` — keep scoring data
   away from the arms' workspace).
   **Fallback if this Claude Code instance lacks the `Workflow` tool:** spawn the 8 arms as parallel `Task`/Agent
   subagents using the prompt pairs embedded in `onboarding-battery.wf.js` (each test = treatment preamble vs control
   preamble + the test question + its JSON schema), then assemble their structured outputs into the same
   `answers.json` shape `{t1:{treatment,control}, t2:{...}, t4:{...}, t5:{...}}`.
5. **Score deterministically** (LLM off the truth path; fetches the live FUV for the T3 leg from this host).
   The methodology's T3 PASS requires ALL of: `validate_ssot.py` ok (structural leg), live HTTP 200 +
   deterministic Content-Type, **and** local↔live byte-parity. Run both legs:
```bash
# 5a. structural leg (methodology T3 clause a)
PYTHONDONTWRITEBYTECODE=1 python3 ~/cr-kit/ssothrut/tools/validate_ssot.py --root ~/cr-kit/ssothrut | grep '"ok"'
# 5b. score + reachability/Content-Type + live<->local byte-parity (clause d) against the cloned projection
cd ~/cr-kit/ssothrut/labs/clean-room-onboarding/protocol
python3 score.py --rubric rubric.json --answers ~/cr-kit/answers.json \
    --ns-local ~/cr-kit/ssothrut/public/ns/core/term --out ~/cr-kit/results.json
```
   `results.json` carries `valid:true` only when the T3 leg (with `--ns-local`) returns **PASS** — a SKIPPED or
   parity-FAIL leg yields `valid:false` and the precision rate is not certifiable.
6. **Write the report** `~/cr-kit/ssothrut/docs/CLEAN-ROOM-RESULTS-<yyyyMMdd-HHmmss>.md` (status `proposed`), comparing
   the new `precisionRatePP` to **run 1's +46.7pp**, and explicitly checking the bias was removed:
   - the **T1 control** answer must NOT cite any `/data/ssothrut` path and should score **lower** than run 1's 60%;
   - if so, the **T1 delta rises** and the precision rate is the de-biased figure.
   Then `git add`/`commit` the report on a branch and open a PR (do not push to `main` directly).

## Autonomy & gates
- The **run is measurement, not promotion** — it needs no operator gating and proceeds end-to-end unattended.
- **Promotion stays operator-gated (I8):** do NOT promote the battery, coin terms, alter `/ns/*`, or touch
  DNS/VPS/Docker. Results land as a `proposed` report + PR for the ROOT operator to review.
- **Secret redaction** applies to any output (`KEY_NAME` values → `[REDACTED]`).

## Interpreting the new precision rate
`precisionRatePP` = mean delta of the clean tests {T1, T2, T4}, T5 reported out-of-band, conditioned on `t3.result`.
Comparability is earned: `score.py` reproduces run-1's JS scorer **exactly** on run-1 data (T1=40, T4=33.3, T5=0
verified offline); `rubric.json`'s axes are a **verbatim port** of the run-1 scorer, so T2 scoring is structurally
identical even though run-1's full T2 text was not retained.
- A **higher** rate than +46.7pp ⇒ the filesystem leak was suppressing the measured effectiveness (expected, driven
  mainly by T1 de-biasing). **Attribution caveat:** this kit's control preamble is also *stricter* than run-1's
  ("you must NOT read files outside this repository"), so any gain reflects **both** the host fix **and** the tighter
  control — do not attribute it to the host alone.
- T2 should remain the dominant, stable contributor (the published FUV supplies idiosyncratic contrasts a bare model
  lacks); a large T2 swing would instead signal a rubric or answer-key drift — investigate before trusting the number.
- `valid:false` (T3 ≠ PASS) ⇒ the published-FUV leg is broken from claudin; the behavioral deltas are not certifiable
  until the namespace resolves — fix publication first.