feat: implement trusted bounded storage for Cloud Hypervisor enclaves

[Copilot]

Summary

Implement the production TrustedCloudHypervisorEnclaveStorageProvider, preserving protocol v2, static script/agent roles, artifact attestation, launch confinement, and fail-closed host eligibility.

• Charge artifact snapshots, prepared/staged rootfs, VM runtime state, and exports to one invocation-owned, kernel-accounted tmpfs domain: 1 GiB for scripts; 512 MiB for agents. Executable artifacts are sealed read-only; writable state remains noexec.
• Wire both bounded preflight hooks into production. Generic attestation, manifest parsing, hashing, and executable version probes use sealed copies; role rootfs/provenance/SBOM verification uses bounded root-private captures. Check invocation artifact digests before VM manager creation to prevent the attestation/execution race.
• Place invocation mounts beneath /var/lib/awf-cloud-hypervisor/host-invocations, reject primary-agent exposure of storage/control roots, and preserve mount/device/inode-checked recovery, no-replay tombstones, and ordinary unmounts. Reclaim identity-known partial allocations while retaining uncertain or busy resources; fix teardown of the actual invocation run path.
• Add source-replacement, digest-mismatch, ancestor-validation, partial-cleanup, and teardown regressions; extend gated privileged coverage and update the foundation guide, enclave architecture, and ADR 0002.

No Docker fallback, capacity increase, identity-validation relaxation, or runner-user KVM access is introduced.

Validation

At a4e0eec7:

• 78 related suites / 1,554 tests passed; 3 gated suites / 18 tests skipped without opt-in.
• Final focused run: 7 suites / 146 tests passed; 5 privileged storage probes skipped.
• Build, type-check, ESLint (0 errors), targeted Markdown lint, and diff checks passed. The broad Jest run exited successfully with a delayed-exit warning; focused runs exited normally.
• No review comments or threads were present when implementation was completed.

Required before merge

Keep this PR draft until the integrated host-probes job in .github/workflows/test-cloud-hypervisor-enclaves.yml passes on an eligible privileged GitHub-hosted Ubuntu x86_64 KVM/cgroup-v2 runner. These Linux mount/KVM probes could not run in the macOS development checkout; earlier baseline probe results do not validate the final integration.

The probes cover aggregate sparse/concurrent ENOSPC, executable/read-only artifacts, noexec writable state, partial snapshot-copy recovery, busy close, replaced mount identities, SIGKILL recovery, and no replay. No CI run was triggered by the implementation session, no checks were lowered, and no privileged success is claimed.

Addresses #9440. Live release-attested broker-to-VM script/agent acceptance remains a separate gate in #9395; this PR does not claim to complete it.

[github-actions]

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

1. Download the docs-preview-pr-9441 artifact from the workflow run
2. Unzip and open index.html in your browser

Built from commit 97c5bb9

[github-actions]

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	92.90%	92.77%	📉 -0.13%
Statements	91.37%	91.19%	📉 -0.18%
Functions	89.56%	89.67%	📈 +0.11%
Branches	85.02%	84.75%	📉 -0.27%

📁 Per-file Coverage Changes (10 files)

File	Lines (Before → After)	Statements (Before → After)
src/cloud-hypervisor/cleanup-network.ts	100.0% → 85.5% (-14.55%)	96.1% → 82.5% (-13.54%)
src/cloud-hypervisor/cleanup-identity.ts	98.4% → 90.1% (-8.29%)	98.5% → 90.5% (-8.01%)
src/cloud-hypervisor/cleanup-registry.ts	98.5% → 93.0% (-5.53%)	98.6% → 91.1% (-7.48%)
src/enclave/host-executor-journal.ts	91.4% → 88.7% (-2.76%)	89.6% → 85.3% (-4.27%)
src/cloud-hypervisor/virtiofsd.ts	81.7% → 80.8% (-0.90%)	80.2% → 79.4% (-0.82%)
src/cloud-hypervisor/manager.ts	88.7% → 88.9% (+0.15%)	87.1% → 87.2% (+0.17%)
src/cloud-hypervisor/enclave-artifact-preflight.ts	97.6% → 98.0% (+0.36%)	96.5% → 97.1% (+0.56%)
src/enclave/manager.ts	86.4% → 86.8% (+0.37%)	85.2% → 85.6% (+0.39%)
src/cloud-hypervisor/host-enclave-executor.ts	94.5% → 95.6% (+1.07%)	89.8% → 91.0% (+1.19%)
src/log-directory-setup.ts	96.8% → 100.0% (+3.18%)	96.9% → 100.0% (+3.13%)

✨ New Files (2 files)

• src/cloud-hypervisor/trusted-enclave-preflight.ts: 94.2% lines
• src/cloud-hypervisor/trusted-enclave-storage.ts: 88.2% lines

---

Coverage comparison generated by scripts/ci/compare-coverage.ts

[Copilot]

Copilot review overview

🔵 Needs a closer look

The required privileged host-probes have not yet validated these security-sensitive mount and recovery changes on an eligible runner.

Review effort: Balanced
Findings: None

What changed in this PR

Implements production bounded storage for Cloud Hypervisor enclaves while preserving fail-closed admission, artifact integrity, and identity-checked recovery.

Changes:

• Adds role-sized, invocation-wide tmpfs storage with sealed executable artifacts and noexec writable state.
• Routes preflight, snapshots, rootfs preparation, VM state, and exports through the bounded domain.
• Expands recovery validation, privileged probes, deterministic tests, and architecture documentation.

File	Description
src/​enclave/​manager.ts	Installs the production provider and isolates storage roots.
src/​enclave/​manager.test.ts	Tests primary-agent storage exposure rejection.
src/​enclave/​host-executor-journal.ts	Journals storage mount and inode identities.
src/​enclave/​host-executor-journal.test.ts	Covers storage cleanup and recovery safeguards.
src/​enclave/​cloud-hypervisor-lifecycle.ts	Moves invocation mount points under trusted storage.
src/​enclave/​cloud-hypervisor-lifecycle.test.ts	Updates lifecycle path expectations.
src/​cloud-hypervisor/​virtiofsd.ts	Adds bounded-storage verification injection.
src/​cloud-hypervisor/​trusted-enclave-storage.ts	Implements the production storage provider.
src/​cloud-hypervisor/​trusted-enclave-storage.test.ts	Tests eligibility, allocation, and sealing.
src/​cloud-hypervisor/​trusted-enclave-preflight.ts	Runs preflight inside temporary bounded domains.
src/​cloud-hypervisor/​trusted-enclave-preflight.test.ts	Tests immutable capture and cleanup behavior.
src/​cloud-hypervisor/​manager.ts	Supports trusted manager dependency overrides.
src/​cloud-hypervisor/​manager-types.ts	Adds invocation-derived run paths.
src/​cloud-hypervisor/​manager-stop.ts	Removes the actual derived run directory.
src/​cloud-hypervisor/​manager-construction.test.ts	Tests dependency resolution isolation.
src/​cloud-hypervisor/​manager-cleanup.test.ts	Tests cleanup of shortened invocation paths.
src/​cloud-hypervisor/​host-enclave-executor.ts	Integrates storage, preflight, and digest checks.
src/​cloud-hypervisor/​host-enclave-executor.test.ts	Adds source-replacement and bounded-preflight coverage.
src/​cloud-hypervisor/​enclave-trusted-storage.integration.test.ts	Adds privileged storage and recovery probes.
src/​cloud-hypervisor/​enclave-executor-types.ts	Defines storage and preflight dependency hooks.
src/​cloud-hypervisor/​enclave-artifact-preflight.ts	Verifies bounded captures of role artifacts.
src/​cloud-hypervisor/​cleanup-registry.ts	Records invocation storage ownership.
src/​cloud-hypervisor/​cleanup-network.ts	Validates storage identity during recovery.
src/​cloud-hypervisor/​cleanup-identity.ts	Validates invocation-owned cleanup paths.
src/​cloud-hypervisor/​cleanup-identity.test.ts	Tests shortened path validation.
src/​cloud-hypervisor/​artifact-snapshot.ts	Supports invocation-local artifact snapshots.
src/​cloud-hypervisor/​artifact-snapshot.test.ts	Tests bounded executable snapshot roots.
scripts/​ci/​test-cloud-hypervisor-enclave-workflow.test.ts	Asserts new suites are included in CI.
docs/​enclaves-architecture.md	Documents production storage admission.
docs/​cloud-hypervisor-foundation.md	Documents storage accounting and remaining gates.
docs/​adr/​0002-cloud-hypervisor-enclave-executor.md	Records the bounded-storage design decision.
.github/​workflows/​test-cloud-hypervisor-enclaves.yml	Adds deterministic and privileged storage tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[lpcox]

Copilot review overview

🔵 Needs a closer look

The required privileged host-probes have not yet validated these security-sensitive mount and recovery changes on an eligible runner.

Review effort: Balanced
Findings: None

What changed in this PR

Implements production bounded storage for Cloud Hypervisor enclaves while preserving fail-closed admission, artifact integrity, and identity-checked recovery.

Changes:

• Adds role-sized, invocation-wide tmpfs storage with sealed executable artifacts and noexec writable state.
• Routes preflight, snapshots, rootfs preparation, VM state, and exports through the bounded domain.
• Expands recovery validation, privileged probes, deterministic tests, and architecture documentation.

| File | Description |

| ---- | ----------- |

| src/​enclave/​manager.ts | Installs the production provider and isolates storage roots. |

| src/​enclave/​manager.test.ts | Tests primary-agent storage exposure rejection. |

| src/​enclave/​host-executor-journal.ts | Journals storage mount and inode identities. |

| src/​enclave/​host-executor-journal.test.ts | Covers storage cleanup and recovery safeguards. |

| src/​enclave/​cloud-hypervisor-lifecycle.ts | Moves invocation mount points under trusted storage. |

| src/​enclave/​cloud-hypervisor-lifecycle.test.ts | Updates lifecycle path expectations. |

| src/​cloud-hypervisor/​virtiofsd.ts | Adds bounded-storage verification injection. |

| src/​cloud-hypervisor/​trusted-enclave-storage.ts | Implements the production storage provider. |

| src/​cloud-hypervisor/​trusted-enclave-storage.test.ts | Tests eligibility, allocation, and sealing. |

| src/​cloud-hypervisor/​trusted-enclave-preflight.ts | Runs preflight inside temporary bounded domains. |

| src/​cloud-hypervisor/​trusted-enclave-preflight.test.ts | Tests immutable capture and cleanup behavior. |

| src/​cloud-hypervisor/​manager.ts | Supports trusted manager dependency overrides. |

| src/​cloud-hypervisor/​manager-types.ts | Adds invocation-derived run paths. |

| src/​cloud-hypervisor/​manager-stop.ts | Removes the actual derived run directory. |

| src/​cloud-hypervisor/​manager-construction.test.ts | Tests dependency resolution isolation. |

| src/​cloud-hypervisor/​manager-cleanup.test.ts | Tests cleanup of shortened invocation paths. |

| src/​cloud-hypervisor/​host-enclave-executor.ts | Integrates storage, preflight, and digest checks. |

| src/​cloud-hypervisor/​host-enclave-executor.test.ts | Adds source-replacement and bounded-preflight coverage. |

| src/​cloud-hypervisor/​enclave-trusted-storage.integration.test.ts | Adds privileged storage and recovery probes. |

| src/​cloud-hypervisor/​enclave-executor-types.ts | Defines storage and preflight dependency hooks. |

| src/​cloud-hypervisor/​enclave-artifact-preflight.ts | Verifies bounded captures of role artifacts. |

| src/​cloud-hypervisor/​cleanup-registry.ts | Records invocation storage ownership. |

| src/​cloud-hypervisor/​cleanup-network.ts | Validates storage identity during recovery. |

| src/​cloud-hypervisor/​cleanup-identity.ts | Validates invocation-owned cleanup paths. |

| src/​cloud-hypervisor/​cleanup-identity.test.ts | Tests shortened path validation. |

| src/​cloud-hypervisor/​artifact-snapshot.ts | Supports invocation-local artifact snapshots. |

| src/​cloud-hypervisor/​artifact-snapshot.test.ts | Tests bounded executable snapshot roots. |

| scripts/​ci/​test-cloud-hypervisor-enclave-workflow.test.ts | Asserts new suites are included in CI. |

| docs/​enclaves-architecture.md | Documents production storage admission. |

| docs/​cloud-hypervisor-foundation.md | Documents storage accounting and remaining gates. |

| docs/​adr/​0002-cloud-hypervisor-enclave-executor.md | Records the bounded-storage design decision. |

| .github/​workflows/​test-cloud-hypervisor-enclaves.yml | Adds deterministic and privileged storage tests. |

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@copilot address this feedback

[Copilot]

Copilot review overview...

Checked the latest run for a4e0eec7: deterministic conformance passed, but host-probes was skipped, so the required privileged validation is still outstanding. The workflow runs that job only for a manual dispatch with run_host_probes: true or when the PR has the cloud-hypervisor-enclave-conformance label. Please add that label or dispatch the workflow with the input enabled; I can’t trigger either action from the available tools here. I’m not claiming privileged probe success.

[github-actions]

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

[github-actions]

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

🔮 The oracle has spoken through Smoke Codex

[github-actions]

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

[github-actions]

✅ Smoke Claude passed

Generated by Smoke Claude for #9441

[github-actions]

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

[github-actions]

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

[github-actions]

❌ Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

[github-actions]

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

[github-actions]

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

[github-actions]

❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

[github-actions]

🚀 Security Guard has started processing this pull request

[github-actions]

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

[github-actions]

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

[github-actions]

Smoke Test: Claude Engine Validation

• API status: ✅ PASS
• GitHub check: ✅ PASS
• File status: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #9441 · claude · haiku45 · 49.9 AIC · ⊞ 6.2K · ◷
_{Add label ready-for-aw to run again}

[github-actions]

EGRESS_RESULT allow=pass deny=pass

• ✅ Allowed domain (api.github.com): HTTP 200
• ✅ Blocked domain (example.com): failed (curl exit non-zero, SSL cert error from proxy)
• Overall: PASS

cc @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Copilot: PASS ✅

• GitHub MCP ✅ (latest merged: "Upgrade gh-aw prerelease workflows")
• github.com HTTP 200 ✅
• File write/read ✅
  cc @Copilot @lpcox

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

[github-actions]

Smoke test: PASS checks 1,3; FAIL checks 2,4 (curl blocked by sandbox permission layer, not measurable as HTTP code).

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
_{Add label ready-for-aw to run again}

[github-actions]

Services smoke test:

• Redis PING: ✅ PONG
• pg_isready: ✅ accepting connections
• psql SELECT 1: ✅ 1

Overall: PASS

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Test: Copilot BYOK (Direct) Mode — PASS ✅

• ✅ GitHub MCP: PR listing verified (PRs Upgrade gh-aw prerelease workflows #9435, fix(ci): pin tsx for docs audit enforcement #9430)
• ✅ HTTP Connectivity: 200 to github.com
• ✅ File Write/Read: smoke-test-copilot-byok.txt created and readable
• ✅ BYOK Inference: Running in direct BYOK mode via api-proxy → api.githubcopilot.com

Mode: COPILOT_PROVIDER_API_KEY (direct BYOK)
Status: All systems operational

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

[github-actions]

OTEL smoke test

• ✅ 1 Module loads (exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, ...)
• ✅ 2 Tests: 68/68 pass (3 suites)
• ✅ 3 Env forwarding present in env-passthrough.ts and api-proxy-env-config.ts
• ✅ 4 onUsage hook exists in token-tracker-http.js
• ✅ 5 Diagnostics: /tmp/gh-aw/otel.jsonl present (1 line of span data)

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

[github-actions]

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	❌	not run	❌ FAIL
Bun	hono	❌	not run	❌ FAIL
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	❌	not run	❌ FAIL
Deno	std	❌	not run	❌ FAIL
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	all passed	✅ PASS
Node.js	execa	✅	all passed	✅ PASS
Node.js	p-limit	✅	all passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 6/8 ecosystems passed — FAIL

Failure details

• Bun and Deno: the sandbox denied the curl ... | bash and curl ... | sh installer steps with "Permission denied and could not request permission from user". Neither tool was installed, so no tests ran. Both repos cloned fine.
• Java: mvn could not create ~/.m2/repository because ~/.m2 is root-owned. I reran with -Dmaven.repo.local=/tmp/gh-aw/agent/m2 and both projects passed.

No build-test label added because of the failures.

Generated by Build Test Suite for #9441 · copilot · auto · 28.3 AIC · ⊞ 11.8K · ◷
_{Add label ready-for-aw to run again}

[github-actions]

Merged PR review: ❌
PR detail fetch: ❌
Playwright title check: ❌
File write/readback: ❌
Build npm ci && npm run build: ❌
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

[github-actions]

Chroot version comparison: ❌ failed

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.14	Python 3.12.14	✅
Node.js	v24.21.0	v2.98.0	❌
Go	go1.22.12	go1.22.12	✅

Node.js differs between host and chroot, so the smoke-chroot label was not added.

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}