You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Cloud Hypervisor enclaves (enclaves[].runtime: cloud-hypervisor) are wired end to end (#9397, #9398) and have deterministic conformance coverage (#9399), but they still fail closed in production. prepareEnclaves() calls assertCloudHypervisorEnclavePrerequisites(config, deps.cloudHypervisorStorageProvider) (src/enclave/manager.ts), and production never supplies a TrustedCloudHypervisorEnclaveStorageProvider (src/enclave/cloud-hypervisor-lifecycle.ts). Every script, agent, or combined configuration is rejected before seeds, runtime probes, the listener, or a VM are created.
This issue covers the next step: implement that provider so that all writable state of a single invocation is charged to one invocation-owned, bounded allocation domain. Recovery must remain identity-checked. This unblocks live broker-to-VM acceptance in #9395.
Aggregate per-invocation capacity: script 1 GiB, agent 512 MiB (CLOUD_HYPERVISOR_ENCLAVE_RESOURCE_PROFILES). The limit applies across all writable exports and storage, including sparse files and concurrent writers, artifact/rootfs snapshots, and runtime state.
Enforcement is in place before any host or artifact resource is created and remains until close() succeeds.
A free-space or statfs capacity check, sparse-file sizing, or an invocation/guest tmpfs alone is insufficient.
assertAvailable(config) fails closed when the host cannot provide this. There is no fallback to Docker, no relaxed limits, and no grant of runner-user KVM access.
Current gaps (see docs/cloud-hypervisor-foundation.md, "Enclave conformance evidence and remaining live gate")
Gap
Today
Artifact snapshots
createArtifactSnapshot() allocates under /var/lib/awf-cloud-hypervisor/trusted-artifacts/run-* and needs an executable mount. The invocation tmpfs is noexec, so copying the binaries there cannot meet launch confinement as-is.
VM run state
createCloudHypervisorRunPaths() uses /run/awf-cloud-hypervisor, independently of the invocation mount.
Rootfs preparation
startCloudHypervisor() prepares <workDir>/cloud-hypervisor-rootfs/<vmRunId> and stages a writable rootfs copy in the manager run directory. None of this is charged to invocation capacity.
Recovery ownership
HostExecutorResourceJournal.captureSnapshot() and isTrustedArtifactSnapshotDirectory() accept fixed trusted roots only. Redirecting paths without durable mount/inode ownership would break identity-checked recovery.
Scope
Define one invocation-owned allocation domain, for example a per-invocation bounded mount with separate executable read-only artifact staging and writable noexec state. Its aggregate usage must be enforced by a mechanism the kernel accounts for, not by advisory checks.
Route artifact snapshots, rootfs preparation and staging, manager run paths, and writable exports through it via backendDependencies and managerDependencies.
Preserve executable-artifact attestation and verification and launch confinement (network namespace, privilege drop, Landlock/seccomp).
Extend the resource journal so recovery identifies invocation-owned mounts and inodes durably. Do not relax path validation or add a global bind mount.
Wire the provider into production prepareEnclaves() only on eligible hosts (GitHub-hosted Ubuntu x86_64 with KVM and writable cgroup v2). Everywhere else, keep the current fail-closed error.
Acceptance criteria
Unit tests: the provider rejects unavailable hosts, and every writable path for both roles is inside the invocation domain. close() releases everything, and a failed close() keeps enforcement in place.
Gated privileged probes in .github/workflows/test-cloud-hypervisor-enclaves.yml show that aggregate ENOSPC at the role limit covers snapshots, rootfs, run state, and exports, including sparse and concurrent writes.
Crash/recovery test: recovery cleans only identity-matched invocation resources and never replays work.
Without the provider or eligibility, behavior is unchanged: the existing admission error, raised before any resource is created.
Docs are updated: docs/cloud-hypervisor-foundation.md gap table, docs/enclaves-architecture.md, and ADR 0002 if the design changes.
Summary
Cloud Hypervisor enclaves (
enclaves[].runtime: cloud-hypervisor) are wired end to end (#9397, #9398) and have deterministic conformance coverage (#9399), but they still fail closed in production.prepareEnclaves()callsassertCloudHypervisorEnclavePrerequisites(config, deps.cloudHypervisorStorageProvider)(src/enclave/manager.ts), and production never supplies aTrustedCloudHypervisorEnclaveStorageProvider(src/enclave/cloud-hypervisor-lifecycle.ts). Every script, agent, or combined configuration is rejected before seeds, runtime probes, the listener, or a VM are created.This issue covers the next step: implement that provider so that all writable state of a single invocation is charged to one invocation-owned, bounded allocation domain. Recovery must remain identity-checked. This unblocks live broker-to-VM acceptance in #9395.
Contract (from
TrustedCloudHypervisorEnclaveStorageProvider)CLOUD_HYPERVISOR_ENCLAVE_RESOURCE_PROFILES). The limit applies across all writable exports and storage, including sparse files and concurrent writers, artifact/rootfs snapshots, and runtime state.close()succeeds.statfscapacity check, sparse-file sizing, or an invocation/guest tmpfs alone is insufficient.assertAvailable(config)fails closed when the host cannot provide this. There is no fallback to Docker, no relaxed limits, and no grant of runner-user KVM access.Current gaps (see
docs/cloud-hypervisor-foundation.md, "Enclave conformance evidence and remaining live gate")createArtifactSnapshot()allocates under/var/lib/awf-cloud-hypervisor/trusted-artifacts/run-*and needs an executable mount. The invocation tmpfs isnoexec, so copying the binaries there cannot meet launch confinement as-is.createCloudHypervisorRunPaths()uses/run/awf-cloud-hypervisor, independently of the invocation mount.startCloudHypervisor()prepares<workDir>/cloud-hypervisor-rootfs/<vmRunId>and stages a writable rootfs copy in the manager run directory. None of this is charged to invocation capacity.HostExecutorResourceJournal.captureSnapshot()andisTrustedArtifactSnapshotDirectory()accept fixed trusted roots only. Redirecting paths without durable mount/inode ownership would break identity-checked recovery.Scope
noexecstate. Its aggregate usage must be enforced by a mechanism the kernel accounts for, not by advisory checks.backendDependenciesandmanagerDependencies.prepareEnclaves()only on eligible hosts (GitHub-hosted Ubuntu x86_64 with KVM and writable cgroup v2). Everywhere else, keep the current fail-closed error.Acceptance criteria
close()releases everything, and a failedclose()keeps enforcement in place..github/workflows/test-cloud-hypervisor-enclaves.ymlshow that aggregate ENOSPC at the role limit covers snapshots, rootfs, run state, and exports, including sparse and concurrent writes.docs/cloud-hypervisor-foundation.mdgap table,docs/enclaves-architecture.md, and ADR 0002 if the design changes.Related
#9394 (bounded storage contract), #9395 (live KVM acceptance, blocked on this), #9396, #9397, #9398, #9399