Skip to content

Split compose generator tests by runtime and topology - #9079

Merged
lpcox merged 6 commits into
mainfrom
copilot/refactor-split-compose-generator-tests
Sep 27, 2026
Merged

lpcox merged 6 commits into
mainfrom
copilot/refactor-split-compose-generator-tests

Conversation

Copilot AI commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

The monolithic generateDockerCompose suite mixed baseline behavior with security-sensitive runtime and topology variants, making fixtures and ownership difficult to isolate.

  • Baseline
    • Retain default compose-generation coverage in compose-generator.test.ts.
  • Topology
    • Move network-isolation coverage into a dedicated suite.
    • Group host-gateway and ARC-DinD sysroot staging coverage.
  • Runtimes
    • Isolate gVisor and microVM behavior.
  • Filesystem policy
    • Separate runtime-specific filesystem.allowWrite gating coverage.
  • Test isolation
    • Preserve per-file Jest mocks, temporary configuration lifecycle, and all existing test cases.

Copilot AI changed the title [WIP] Refactor: Split src/compose-generator.test.ts by runtime/topology variant Split compose generator tests by runtime and topology Sep 27, 2026
Copilot AI requested a review from lpcox September 27, 2026 16:12
@lpcox
lpcox marked this pull request as ready for review September 27, 2026 16:15
Copilot AI balanced review requested due to automatic review settings September 27, 2026 16:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The change cleanly reorganizes existing tests while preserving their fixtures, mocks, assertions, and lifecycle handling.

Review effort: Balanced
Findings: None

What changed in this PR

Splits the monolithic compose-generator test suite into focused runtime, topology, and filesystem-policy suites without changing production code.

Changes:

  • Retains baseline compose tests in the original suite.
  • Separates network isolation, runtime, ARC-DinD, and filesystem-gating coverage.
  • Preserves per-file mocks and temporary-directory cleanup.
File Description
src/​compose-generator.test.ts Retains baseline compose-generation tests.
src/​compose-generator-runtimes.test.ts Covers gVisor and microVM behavior.
src/​compose-generator-network-isolation.test.ts Covers isolated-network topology.
src/​compose-generator-filesystem-gating.test.ts Covers runtime-specific write-policy gating.
src/​compose-generator-arc-dind.test.ts Covers host-gateway and ARC-DinD staging.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 92.66% 92.67% 📈 +0.01%
Statements 91.14% 91.15% 📈 +0.01%
Functions 89.17% 89.17% ➡️ +0.00%
Branches 84.27% 84.28% 📈 +0.01%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • clients2.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "clients2.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Claude passed

Generated by Smoke Claude for #9079

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #9079

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation is verifying network-isolation egress on this pull request...

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Result
API status ✅ PASS
gh check ✅ PASS
File status ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #9079 · claude · haiku45 · 55.7 AIC · ⊞ 4.6K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot Engine — @lpcox

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Cloud Hypervisor + Copilot

  1. list_pull_requests (github/gh-aw-firewall): PASS — MCP call succeeded (returned PR Split compose generator tests by runtime and topology #9079).
  2. curl https://github.com: FAIL — bash tool denied network execution ("Permission denied and could not request permission from user"); could not obtain HTTP code.
  3. Write/read unique file in /tmp/gh-aw/agent/: PASS — wrote and read back unique-line-36337303881.
  4. curl (example.com/redacted) (expected blocked): FAIL — same bash network denial as check 2; could not confirm 000/403.

Overall: 2/4 PASS. Checks 2 & 4 could not run because the sandbox's bash tool blocked outbound curl invocations entirely (independent of AWF firewall behavior).

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK ✅

  • ✅ GitHub MCP connectivity (2 recent merged PRs fetched)
  • ✅ HTTP 200 to github.com
  • ✅ File write/read verified
  • ✅ BYOK inference working (this response proves it)

Mode: Direct BYOK via COPILOT_PROVIDER_API_KEY → api-proxy sidecar → api.githubcopilot.com

Result: PASS

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: GHA Services Connectivity

  • Redis PING: ✅ (PONG)
  • PostgreSQL pg_isready: ✅ (accepting connections)
  • PostgreSQL SELECT 1: ✅ (1)

Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@lpcox
lpcox deployed to aoai-model September 27, 2026 17:36 — with GitHub Actions Active
@lpcox
lpcox deployed to aoai-model September 27, 2026 17:36 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅ YES
Node.js v24.21.0 v22.23.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: Not all tests passed — Node.js version mismatch between host and chroot environments. The smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

📡 OTel Tracing Smoke Test Results

Scenario Result
1. Module Loading ✅ otel.js loaded; isEnabled() → true; exports include startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, plus internal helpers
2. Test Suite ✅ 68/68 tests passed across 3 suites (otel.test.js + split modules) — spans, gen_ai.usage.* attributes, budget attributes, parent trace propagation, OTLP export, FileSpanExporter fallback
3. Env Var Forwarding ✅ GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID present in env-passthrough.ts; GH_AW_OTLP_ENDPOINTS, OTEL_EXPORTER_OTLP_ENDPOINT, and both trace-context vars present in api-proxy-env-config.ts
4. Token Tracker Integration ✅ onUsage callback present in token-tracker-http.js (OTEL hook point)
5. OTEL Diagnostics ✅ No local otel.jsonl under sandbox logs (expected — OTLP endpoint was configured via secrets, so spans export directly to the collector instead of the local FileSpanExporter fallback); token-usage.jsonl shows 12 real token-usage records confirming the proxy pipeline is active

Overall: all scenarios passed, no regressions detected.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ✅ 1/1 passed ✅ PASS
Bun hono ✅ 1/1 passed ✅ PASS
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ 1/1 passed ✅ PASS
Go env ✅ 1/1 passed ✅ PASS
Go uuid ✅ 1/1 passed ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ passed ✅ PASS
Node.js execa ✅ passed ✅ PASS
Node.js p-limit ✅ passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — PASS

Notes (environment, not firewall-related):

  • Bun and Deno binaries were not pre-installed in this environment; the official install scripts (bun.sh/install, deno.land/install.sh) were blocked by sandbox policy, so both were installed instead via npm install -g --allow-scripts=<pkg> <pkg>, which succeeded.
  • The default Maven local repository (~/.m2/repository) was owned by root and not writable by the running user, so Java builds used -Dmaven.repo.local=/tmp/gh-aw/agent/m2repo as a workaround. All other Maven proxy configuration (~/.m2/settings.xml) worked as expected.

All 18 test projects across all 8 ecosystems built/installed and passed their test suites successfully with no network egress issues observed.

Generated by Build Test Suite for #9079 · copilot · auto · 38.9 AIC · ⊞ 11.8K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Split model-resolver.test.js into three focused test files ✅
fix(api-proxy): skip Responses custom-tool translation on Chat Completions routes ✅
Playwright title check ✅
File write/readback ✅
Build (npm ci && npm run build) ✅
Overall status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • clients2.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "clients2.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 332b40a into main Sep 27, 2026
130 of 137 checks passed
@lpcox
lpcox deleted the copilot/refactor-split-compose-generator-tests branch September 27, 2026 23:05

This branch was successfully deployed

1 active deployment
aoai-model — 79d661d8 Deployed Sep 27, 2026 by lpcox via conclusion #1787
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Refactoring] Split src/compose-generator.test.ts by runtime/topology variant

3 participants