test(api-proxy): add end-to-end coverage for routed Copilot efforts none and max - #9325
Conversation
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
none and max
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The focused test-only changes correctly cover the stated routing behavior and failure cases without altering production logic.
Review effort: Balanced
Findings: None
What changed in this PR
Adds end-to-end regression coverage ensuring routed Copilot efforts none and max remain supported across AWF’s API-proxy routing boundary.
Changes:
- Verifies candidate selection, persistence, reflection, and request telemetry.
- Tests omitted effort versus explicit
none. - Confirms unsupported efforts fail closed.
| File | Description |
|---|---|
containers/api-proxy/routing-efforts.test.js |
Adds comprehensive routed-effort regression tests. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
✅ Copilot review passed with no inline comments. @copilot Add the |
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 92.84% | 92.83% | 📉 -0.01% |
| Statements | 91.34% | 91.34% | ➡️ +0.00% |
| Functions | 89.37% | 89.37% | ➡️ +0.00% |
| Branches | 84.55% | 84.58% | 📈 +0.03% |
📁 Per-file Coverage Changes (9 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/cloud-hypervisor/manager-start.ts |
88.4% → 86.3% (-2.07%) | 87.8% → 85.8% (-1.99%) |
src/nvx/one-shot-adapter.ts |
83.5% → 82.9% (-0.60%) | 80.3% → 79.8% (-0.56%) |
src/services/enclave-mcp-service.ts |
97.9% → 97.8% (-0.07%) | 96.0% → 95.9% (-0.13%) |
src/enclave/github-gateway.ts |
82.6% → 82.6% (+0.00%) | 79.7% → 79.8% (+0.09%) |
src/cloud-hypervisor/preflight.ts |
95.9% → 95.9% (+0.02%) | 96.0% → 96.0% (+0.03%) |
src/microvm/network-plan.ts |
93.3% → 93.6% (+0.24%) | 92.7% → 93.0% (+0.25%) |
src/microvm/network-reservation.ts |
52.8% → 53.3% (+0.44%) | 51.1% → 51.5% (+0.42%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
src/cloud-hypervisor/cleanup-handle.ts |
88.3% → 93.6% (+5.32%) | 85.0% → 90.0% (+5.00%) |
✨ New Files (1 files)
src/cloud-hypervisor/enclave-network.ts: 81.0% lines
Coverage comparison generated by scripts/ci/compare-coverage.ts
…ries Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Documentation PreviewDocumentation has been built for this PR. To view locally:
Built from commit bb46e46 |
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...
|
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅
|
|
🚀 Security Guard has started processing this pull request |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
🔌 Smoke Services — All services reachable! ✅
|
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
✅ Smoke Claude passed
|
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
✅ Build Test Suite completed successfully!
|
|
❌ Smoke Gemini reports failed. Facets need polishing...
|
|
Smoke Cloud Hypervisor reports failed. Cloud Hypervisor + Copilot failed.
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅
|
|
🌑 The shadows whisper... Smoke Codex failed. The oracle requires further meditation...
|
Smoke Test: Claude Engine Validation
Overall result: PASS
|
|
EGRESS_RESULT allow=pass deny=pass
cc
|
Smoke Test: Copilot BYOK (Direct) Mode – PASS ✅Test Results:
Status: All tests passed. Direct BYOK mode operational.
|
Chroot Version Comparison
Result: not all tests passed (Node.js mismatch), so the
|
|
Smoke Copilot: PASS ✅
|
|
OTEL smoke test
|
🏗️ Build Test Suite Results
Overall: 5/8 ecosystems passed — FAIL Failures:
|
|
Smoke services: ✅ Redis PONG · ✅ pg_isready accepting connections · ✅ SELECT 1 = 1 → PASS
|
gh-aw's Copilot harness rejected router-selected
noneandmaxefforts (github/gh-aw#64831). AWF already supports both values. This PR adds coverage across the routed request boundary so a later change can't drop or alter them without a test failing. It is test-only: the current implementation passes as-is.What's added
containers/api-proxy/routing-efforts.test.jsruns the realcreateRoutingControllerinsidecreateProductionRoutingSession. Only the router, catalogue snapshot and executor are faked. Each case is followed through the whole chain:catalogue → candidate pool → router pick →
selection.json→getReflectState()→observeRequesttelemetrynone/max(viadescribe.each)none…max) is offered too.selection.json, the selection log record and/reflect(endpoint: /responses).as_selectedonly when model, endpoint and effort all match. Each of these is recorded asdeviatedwith the specific mismatches listed:/chat/completions, including when the effort is sent in the wrong shape for that endpointnoneeffortkey;/reflectshowseffort: nulland/chat/completions.reasoning_effort: 'none'against an omitted selection is a deviation, and so is an effortless request against anoneselection.routing_contract_error,failure.json, noselection.json, and a failed/reflectstate:ultranullnonewhere it was omittedultra,NONE) are filtered out. If none remain, the run fails withno_route.Note for reviewers
Routing selection is advisory: a request that doesn't match is not rejected. Here "accepted only on matching requests" means the request is recorded as
as_selectedrather thandeviated, which is what these tests assert. Actually rejecting mismatched requests would be a behaviour change and is out of scope.The tests fail if
maxornoneis removed fromEFFORTSinrouting-candidates.jsorrouting-contract.js. They also fail ifnoneis mapped to the omitted-effort endpoint inrouting-observation.js.noneandmax#9323