fix: detect and clear stale ANTHROPIC_BASE_URL from crashed wrap sessions (headroomlabs-ai#1768) (headroomlabs-ai#1837)

## Description

`headroom wrap claude` writes `env.ANTHROPIC_BASE_URL` (or the
foundry/vertex variant) into a project's `.claude/settings.local.json`
so daemon-spawned Claude Code workers route through the local Headroom
proxy. Removal only happened in the wrap process's `finally:` block. An
unclean exit — `SIGKILL`, OOM, reboot, or terminal/tmux close (`SIGHUP`,
which was not caught; only `SIGINT`/`SIGTERM` were) — skipped that
cleanup, so the entry persisted indefinitely. Every subsequent bare
`claude` in that project then routed to the dead port and hung
indefinitely retrying it.

Closes headroomlabs-ai#1768

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `_write_claude_wrap_base_url` now optionally stamps a sidecar marker
(`.claude/.headroom_wrap_marker.json`) recording the writer's
pid/identity, the port, and the true prior value — kept out of
`settings.local.json` itself so Headroom bookkeeping never shows up as a
stray key in a file Claude Code's own config loader parses.
- A shared `_identity_mismatch` helper (factored out of the existing
`_marker_pid_reused` proxy-client-refcounting logic) lets a marker be
judged stale: missing/invalid pid, dead pid, or a live pid whose
identity doesn't match the recorded one (PID reuse after a crash).
- `claude()` now checks for — and self-heals — a stale marker
immediately before writing a fresh entry, restoring the recorded prior
value instead of trusting a leftover from a dead session.
- `claude()` now also registers a `SIGHUP` handler (guarded via
`hasattr`, since Windows has none) alongside the existing `SIGTERM`
handler, so terminal-close triggers the same cleanup/restore path.
- `headroom unwrap claude` now reads the marker's recorded prior value
before restoring, instead of unconditionally deleting the key — so a
user's own pre-existing `ANTHROPIC_BASE_URL` (set before ever running
`wrap`) isn't blindly wiped.
- `headroom doctor` gained a new check (`check_wrap_marker_staleness`)
that flags a stale project-local marker and points at `headroom unwrap
claude` to clean it up — separate from the existing global-settings
`check_claude_routing` check.
- (Unrelated, pre-existing on `main`) reformatted
`headroom/proxy/handlers/openai.py`,
`tests/test_openai_codex_ws_lifecycle.py`, `tests/test_output_shaper.py`
— whitespace/indentation only, no logic change — since they were already
failing `ruff format --check .` on `main` before this branch touched
anything, and the repo-wide lint gate blocks on it.

Out of scope: `wrap --worktree` — no such flag or multi-worktree
`.claude` handling exists anywhere in `wrap.py` today; not adding new
surface for an aspirational scenario the issue mentions but that isn't
implemented.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_cli/test_wrap_claude_base_url.py tests/test_cli/test_unwrap_claude.py tests/test_cli/test_wrap_stale_marker.py -q
42 passed

$ pytest tests/test_cli -q
512 passed, 1 failed (test_wrap_codex_prepare_only_registers_serena_when_uvx_exists —
confirmed to fail identically on a clean checkout of main with no changes applied;
test-order flake, unrelated to this PR)

$ ruff check .
All checks passed!

$ ruff format --check .
1047 files already formatted

$ mypy headroom/cli/wrap.py headroom/cli/doctor.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: local checkout, Python 3.13, Windows.
- Exact command / steps: wrote a base_url entry + marker via
`_write_claude_wrap_base_url(..., port=8787)`, then overwrote the
marker's recorded pid with a value guaranteed not to be a live process
(simulating the crash from the issue's own repro: `headroom wrap claude
-- -p ok & ; kill -9 <wrap-pid>`). Ran
`headroom.cli.doctor.check_wrap_marker_staleness()` against that path,
then called `_check_and_clear_stale_wrap_marker()` (the same check
`claude()` now runs before writing a fresh entry).
- Observed result: `doctor`'s check correctly reports `WARN` naming the
dead pid/port and pointing at `headroom unwrap claude`. The stale-check
call then self-heals: in the "nothing existed before wrap" case the
leaked entry is removed; in a second run seeded with a real pre-existing
`ANTHROPIC_BASE_URL` (set before `wrap` ever ran), that original value
is recovered instead of being deleted. In both cases the marker file is
cleared afterward.
- Not tested: actual OS-level signal delivery (`kill -HUP` against a
real running `headroom wrap claude` subprocess) — the SIGHUP
registration is exercised via a source-inspection test instead of a live
signal, since spawning/killing the real CLI subprocess isn't practical
in this environment; verified E2E via CI's `wrap-native` jobs
(Ubuntu/macOS) which passed.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A — CLI/backend fix, no UI surface.

## Additional Notes

- Documentation checklist item left unchecked: no user-facing docs
currently describe wrap's settings.local.json write/cleanup behavior in
enough detail to need updating; happy to add a troubleshooting note if
maintainers want one.
- `wrap --worktree` handling is out of scope (see Changes Made) —
flagging in case maintainers want it tracked as a separate follow-up
issue.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Author: Parideboy

CHANGELOG.md

@@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## Unreleased
 
 ### Fixed
+- `headroom wrap claude` no longer leaves a dead `ANTHROPIC_BASE_URL` in a
+  project's `.claude/settings.local.json` after an unclean exit (`SIGKILL`,
+  OOM, reboot, or terminal/tmux close via `SIGHUP`, which was not caught).
+  `_write_claude_wrap_base_url`/`_restore_claude_wrap_base_url` only removed
+  or restored the entry from the wrap process's own `finally` block, so a
+  crash skipped it and every later bare `claude` invocation in that project
+  inherited the stale proxy URL and hung indefinitely retrying a dead port.
+  A wrap session now stamps a sidecar marker (pid, port, prior value); the
+  next `wrap`, `unwrap`, or `headroom doctor` run detects a marker whose pid
+  is dead or reused and restores the recorded prior value automatically.
+  `claude()` also now catches `SIGHUP` alongside the existing `SIGTERM`
+  handler ([#1768](https://github.com/headroomlabs-ai/headroom/issues/1768)).
 - Non-finite values (`NaN`, `Infinity`) in `proxy_savings.json` or in upstream
   cost/token metadata no longer crash the proxy or corrupt the savings
   dashboard. `SavingsTracker`'s numeric coercion caught only `TypeError` and

headroom/cli/doctor.py

@@ -33,6 +33,7 @@
 )
 
 from .main import get_version, main
+from .wrap import _read_wrap_marker, _wrap_marker_is_stale
 
 PASS = "pass"
 WARN = "warn"
@@ -188,6 +189,34 @@ def check_claude_remote_control_gate(
     return None
 
 
+def check_wrap_marker_staleness(settings_path: Path) -> CheckResult:
+    """Flag a project-local ANTHROPIC_BASE_URL left by a crashed wrap session.
+
+    A crashed ``headroom wrap claude`` (SIGKILL, OOM, reboot) can leave
+    ``.claude/settings.local.json`` pointing at a dead proxy port, hanging
+    every subsequent bare ``claude`` invocation in the project (issue #1768).
+    This checks the project-local settings file — separate from the global
+    ``~/.claude/settings.json`` :func:`check_claude_routing` inspects.
+    """
+    name = "wrap_marker"
+    marker = _read_wrap_marker(settings_path)
+    if marker is None:
+        return CheckResult(name=name, status=SKIP, summary="no wrap marker found")
+    if not _wrap_marker_is_stale(marker):
+        return CheckResult(
+            name=name, status=PASS, summary=f"live wrap session (pid {marker.get('pid')})"
+        )
+    return CheckResult(
+        name=name,
+        status=WARN,
+        summary=(
+            f"stale ANTHROPIC_BASE_URL from crashed wrap session "
+            f"(pid {marker.get('pid')}, port {marker.get('port')}) — "
+            "run `headroom unwrap claude` to clean it up"
+        ),
+    )
+
+
 def check_codex_routing(config_path: Path, port: int) -> CheckResult:
     """Is Codex configured to route through the proxy?
 
@@ -419,6 +448,7 @@ def doctor(port: int, emit_json: bool) -> None:
         check_proxy_liveness(livez, base_url),
         check_version_drift(livez, installed),
         check_claude_routing(claude_settings_path(), port),
+        check_wrap_marker_staleness(Path.cwd() / ".claude" / "settings.local.json"),
         check_codex_routing(codex_config_path(), port),
         check_shell_env(os.environ, port),
         check_savings(stats, savings_path()),

headroom/cli/wrap.py

@@ -846,12 +846,94 @@ def _claude_wrap_base_url_env_key(*, foundry_mode: bool = False, vertex_mode: bo
     return "ANTHROPIC_BASE_URL"
 
 
+def _wrap_marker_path(settings_path: Path) -> Path:
+    """Sidecar marker path for a given settings.local.json path.
+
+    Kept out of settings.local.json itself so Headroom's own bookkeeping never
+    shows up as a stray key inside a file Claude Code's config loader parses.
+    """
+    return settings_path.parent / ".headroom_wrap_marker.json"
+
+
+def _write_wrap_marker(settings_path: Path, *, port: int, key: str, previous: str | None) -> None:
+    """Best-effort record of which (pid, port, key) wrote the base_url entry.
+
+    Lets a later wrap/doctor/unwrap invocation tell a stale leftover (writer
+    process is dead or its PID was recycled) from a still-live wrap session,
+    and recover the true prior value (issue #1768) instead of guessing.
+    """
+    try:
+        ident = _proc_identity(os.getpid())
+        payload = {
+            "pid": os.getpid(),
+            "start_src": ident[0] if ident else None,
+            "start_time": ident[1] if ident else None,
+            "port": port,
+            "key": key,
+            "previous": previous,
+        }
+        _write_text(_wrap_marker_path(settings_path), json.dumps(payload))
+    except OSError:
+        pass
+
+
+def _read_wrap_marker(settings_path: Path) -> dict[str, Any] | None:
+    marker = _wrap_marker_path(settings_path)
+    try:
+        rec = json.loads(_read_text(marker))
+    except (OSError, ValueError):
+        return None
+    return rec if isinstance(rec, dict) else None
+
+
+def _wrap_marker_is_stale(marker: dict[str, Any]) -> bool:
+    """True if ``marker`` describes a writer that is provably gone.
+
+    Missing/invalid pid, a dead pid, or a live pid whose recorded identity no
+    longer matches (PID reuse) all count as stale — the entry it describes was
+    left behind by a wrap session that no longer exists.
+    """
+    pid = marker.get("pid")
+    if not isinstance(pid, int):
+        return True
+    if not _pid_alive(pid):
+        return True
+    return _identity_mismatch(marker.get("start_src"), marker.get("start_time"), pid)
+
+
+def _clear_wrap_marker(settings_path: Path, *, key: str) -> None:
+    marker = _read_wrap_marker(settings_path)
+    if marker is not None and marker.get("key") == key:
+        _wrap_marker_path(settings_path).unlink(missing_ok=True)
+
+
+def _check_and_clear_stale_wrap_marker(settings_path: Path, *, key: str) -> str | None:
+    """If a stale wrap marker for ``key`` exists, restore its recorded prior
+    value and clear the marker. Returns the restored value, or None if there
+    was nothing stale to clean up.
+
+    Called before writing a fresh base_url entry so a crashed wrap session's
+    leftover doesn't get treated as this session's own state to restore later.
+    """
+    marker = _read_wrap_marker(settings_path)
+    if marker is None or marker.get("key") != key or not _wrap_marker_is_stale(marker):
+        return None
+    previous = marker.get("previous")
+    click.echo(
+        f"headroom: clearing stale {key} left by crashed wrap session (pid {marker.get('pid')})",
+        err=True,
+    )
+    _restore_claude_wrap_base_url(previous, settings_path=settings_path, _key_override=key)
+    return previous
+
+
 def _write_claude_wrap_base_url(
     proxy_url: str,
     *,
     foundry_mode: bool = False,
     vertex_mode: bool = False,
     settings_path: Path | None = None,
+    port: int | None = None,
 ) -> str | None:
     """Persist proxy URL into project-local settings env key for daemon child inheritance.
 
@@ -863,6 +945,10 @@ def _write_claude_wrap_base_url(
     initial launch — routes through the Headroom proxy without touching the
     global user settings file or affecting sessions in other projects. Returns
     the previous value so the caller can restore it on exit (issue #951).
+
+    When ``port`` is given, also stamps a sidecar marker recording this
+    process's identity and the previous value, so a later crash can be
+    detected and self-healed (issue #1768).
     """
     path = settings_path or (Path.cwd() / ".claude" / "settings.local.json")
     payload: dict[str, Any] = {}
@@ -880,6 +966,8 @@ def _write_claude_wrap_base_url(
     payload["env"] = env_map
     path.parent.mkdir(parents=True, exist_ok=True)
     _write_text(path, json.dumps(payload, indent=2) + "\n")
+    if port is not None:
+        _write_wrap_marker(path, port=port, key=key, previous=previous)
     return previous
 
 
@@ -889,16 +977,22 @@ def _restore_claude_wrap_base_url(
     foundry_mode: bool = False,
     vertex_mode: bool = False,
     settings_path: Path | None = None,
+    _key_override: str | None = None,
 ) -> None:
     """Restore (or remove) the env key written by _write_claude_wrap_base_url.
 
     Called in both the wrap-session finally block and unwrap_claude so the
     project-local settings entry is never left pointing at a dead proxy.  When
     ``previous`` is None the key is removed; when it has a value it is
-    restored — preserving any URL the project already had set.
+    restored — preserving any URL the project already had set. Also clears
+    this key's sidecar wrap marker, if any (issue #1768).
     """
     path = settings_path or (Path.cwd() / ".claude" / "settings.local.json")
+    key = _key_override or _claude_wrap_base_url_env_key(
+        foundry_mode=foundry_mode, vertex_mode=vertex_mode
+    )
     if not path.exists():
+        _clear_wrap_marker(path, key=key)
         return
     try:
         payload = json.loads(_read_text(path))
@@ -909,9 +1003,9 @@ def _restore_claude_wrap_base_url(
     env_map = payload.get("env")
     if not isinstance(env_map, dict):
         return
-    key = _claude_wrap_base_url_env_key(foundry_mode=foundry_mode, vertex_mode=vertex_mode)
     if previous is None:
         if key not in env_map:
+            _clear_wrap_marker(path, key=key)
             return
         del env_map[key]
         if env_map:
@@ -925,6 +1019,7 @@ def _restore_claude_wrap_base_url(
         _write_text(path, json.dumps(payload, indent=2) + "\n")
     else:
         path.unlink(missing_ok=True)
+    _clear_wrap_marker(path, key=key)
 
 
 def _setup_headroom_mcp(
@@ -3072,26 +3167,33 @@ def _pid_alive(pid: int) -> bool:
     return pid_alive(pid)
 
 
+def _identity_mismatch(src: Any, recorded: Any, pid: int) -> bool:
+    """True only if ``pid``'s current identity *provably* differs from the
+    recorded ``(src, recorded)`` identity (i.e. the PID was recycled).
+
+    Conservative by design: any uncertainty (unknown/legacy identity, unknown
+    start time, mismatched source) returns ``False`` — never claim a mismatch
+    without proof, since the caller uses this to decide whether to trust or
+    discard state tied to a live PID.
+    """
+    if not isinstance(src, str) or not isinstance(recorded, int | float):
+        return False  # legacy / identity-less record — can't tell
+    ident = _proc_identity(pid)
+    if ident is None or ident[0] != src:
+        return False  # can't compare like-for-like — don't claim mismatch
+    # Start times are stable per process; >1s apart means a different process.
+    return abs(ident[1] - float(recorded)) > 1.0
+
+
 def _marker_pid_reused(marker: Path, pid: int) -> bool:
     """True only if the live ``pid`` is *provably* a different process than the
     one that wrote ``marker`` (i.e. the PID was recycled after a crash).
-
-    Conservative by design: any uncertainty (legacy marker, unknown start time,
-    mismatched source) returns ``False`` so a real client is never pruned.
     """
     try:
         rec = json.loads(_read_text(marker))
     except (OSError, ValueError):
         return False
-    src = rec.get("start_src")
-    recorded = rec.get("start_time")
-    if not isinstance(src, str) or not isinstance(recorded, int | float):
-        return False  # legacy / identity-less marker — can't tell
-    ident = _proc_identity(pid)
-    if ident is None or ident[0] != src:
-        return False  # can't compare like-for-like — don't prune
-    # Start times are stable per process; >1s apart means a different process.
-    return abs(ident[1] - float(recorded)) > 1.0
+    return _identity_mismatch(rec.get("start_src"), rec.get("start_time"), pid)
 
 
 def _live_proxy_clients(port: int, *, exclude_self: bool = True) -> list[int]:
@@ -3600,6 +3702,10 @@ def claude(
     _register_proxy_client(port)
     signal.signal(signal.SIGINT, _ignore_child_sigint)
     signal.signal(signal.SIGTERM, cleanup)
+    if hasattr(signal, "SIGHUP"):
+        # Terminal close / tmux kill-session sends SIGHUP, not SIGTERM — without
+        # this, the finally block's base_url restore never runs (issue #1768).
+        signal.signal(signal.SIGHUP, cleanup)
 
     # Memory sync BEFORE proxy startup — sync headroom DB ↔ Claude's files
     if memory:
@@ -3764,6 +3870,13 @@ def claude(
         # daemon's environment) also route through Headroom.
         _settings_vertex[0] = bool(use_vertex)
         _settings_foundry[0] = bool(foundry_upstream) and not _settings_vertex[0]
+        _wrap_settings_path = Path.cwd() / ".claude" / "settings.local.json"
+        _check_and_clear_stale_wrap_marker(
+            _wrap_settings_path,
+            key=_claude_wrap_base_url_env_key(
+                foundry_mode=_settings_foundry[0], vertex_mode=_settings_vertex[0]
+            ),
+        )
         _saved_base_url[0] = _write_claude_wrap_base_url(
             (
                 _foundry_proxy_url(proxy_url)
@@ -3774,6 +3887,8 @@ def claude(
             ),
             foundry_mode=_settings_foundry[0],
             vertex_mode=_settings_vertex[0],
+            settings_path=_wrap_settings_path,
+            port=port,
         )
 
         # Per-project savings attribution: tag every request with the launch
@@ -3817,6 +3932,7 @@ def claude(
             _saved_base_url[0],
             foundry_mode=_settings_foundry[0],
             vertex_mode=_settings_vertex[0],
+            settings_path=_wrap_settings_path,
         )
         cleanup()
 
@@ -3884,9 +4000,19 @@ def unwrap_claude(
     else:
         click.echo("  Kept rtk Claude hooks (--keep-rtk).")
 
-    _restore_claude_wrap_base_url(None)
-    _restore_claude_wrap_base_url(None, foundry_mode=True)
-    _restore_claude_wrap_base_url(None, vertex_mode=True)
+    _unwrap_settings_path = Path.cwd() / ".claude" / "settings.local.json"
+    for _foundry, _vertex in ((False, False), (True, False), (False, True)):
+        _key = _claude_wrap_base_url_env_key(foundry_mode=_foundry, vertex_mode=_vertex)
+        _marker = _read_wrap_marker(_unwrap_settings_path)
+        _prior = (
+            _marker.get("previous") if _marker is not None and _marker.get("key") == _key else None
+        )
+        _restore_claude_wrap_base_url(
+            _prior,
+            foundry_mode=_foundry,
+            vertex_mode=_vertex,
+            settings_path=_unwrap_settings_path,
+        )
 
     click.echo()
     click.echo("✓ Claude is no longer durably wrapped by Headroom.")

tests/test_cli/test_unwrap_claude.py

@@ -223,10 +223,26 @@ def restore_base_url(previous: str | None, **kwargs: object) -> None:
         )
 
     assert result.exit_code == 0, result.output
+    settings_path = Path.cwd() / ".claude" / "settings.local.json"
     assert restore_calls == [
-        {"previous": None},
-        {"previous": None, "foundry_mode": True},
-        {"previous": None, "vertex_mode": True},
+        {
+            "previous": None,
+            "foundry_mode": False,
+            "vertex_mode": False,
+            "settings_path": settings_path,
+        },
+        {
+            "previous": None,
+            "foundry_mode": True,
+            "vertex_mode": False,
+            "settings_path": settings_path,
+        },
+        {
+            "previous": None,
+            "foundry_mode": False,
+            "vertex_mode": True,
+            "settings_path": settings_path,
+        },
     ]