Skip to content

fix: diagnose public debug preflight and hand off startup audit - #10009

Merged
lpcox merged 2 commits into
mainfrom
lpcox-debug-smoke-preflight-failures
Oct 15, 2026
Merged

lpcox merged 2 commits into
mainfrom
lpcox-debug-smoke-preflight-failures

Conversation

@lpcox

@lpcox lpcox commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

Observed evidence and certainty

Investigates smoke run 38678864556, dispatched on main 1baba937445c020ead894d60acab8a2fe5c9b0fb using AWF v0.28.58 after #9995 and #10008.

  • The run failed at host-preflight, check startup/enclave-preparation, with Public enclave debug repository visibility could not be verified. AWF_STARTUP_EXIT reports containersStarted:false, agentStarted:false; the harness records not_started.
  • Compared the release-tagged visibility gate and main-action implementation with this checkout: both match. The gate uses host Node fetch, an unauthenticated GitHub repository metadata request, redirects rejected, a 5-second deadline and 64 KiB ceiling. This exact error branch establishes non-200 HTTP status or missing body, not which status or why. There is no evidence establishing rate limiting, authentication, TLS, DNS or timeout as the cause.
  • Cleanup refused to delete uncertain private recovery state. The compiler-owned redactor then reported stage=read, code=EACCES and removed an artifact source. Its log does not name the unreadable path. The downloaded agent artifact retains sandbox/firewall/logs/awf-startup-error.json but no firewall audit tree. The public-debug artifact contains only No public-fixture debug logs were produced for this run.
  • A concrete local ownership defect exists at /tmp/gh-aw/sandbox/firewall/audit/awf-resolved-config.json: sudo AWF creates it root-owned at 0600; generic cleanup normally repairs permissions, but this fail-closed cleanup path exits before that repair. This PR reproduces the missing handoff and fixes it; it does not assert this was the run's only unreadable file or prove which source the redactor removed.
  • Canonical diagnosis registry: no matching finding. No guest started, so this run does not reproduce or resolve the earlier host-executor launch failure. Missing enclave.jsonl is not invocation evidence.

Changes

  • Add bounded visibility failure reasons and HTTP status/allowlisted transport codes. Raw exceptions, response bodies/headers and repository names are excluded. This is diagnostic instrumentation, not a guessed metadata transport workaround.
  • Immediately hand only the resolved-config artifact and its audit directory to the sudo runner, using existing host identity helpers and directory/file descriptors. Keep the artifact 0600, exclusive file creation, regular/single-link validation, and reject symlink audit destinations. Publishing failures now warn instead of being visible only at debug level.
  • Cover early visibility rejection followed by failed cleanup, retained private state, runner redaction read/rewrite consumption, safe error reporting, exact size/deadline bounds, and existing debug-admission/lifecycle behavior. Document interpretation and release requirements.

Security and scope

No credentials added to metadata requests; no redirects, retries, egress expansion, public-only gate relaxation or attestation changes. Private recovery/control state and host-only raw debug artifacts are not recursively chowned/chmodded or exposed to guests. Payload-bearing API audit permissions remain unchanged. Cleanup failure remains a failure; no fabricated successful invocation or empty enclave audit.

No workflows/generated locks, release pins, invocation limits or retry settings changed. Compiler-owned secret-redaction behavior is left fail-closed; identifying any additional unreadable source/removal scope requires more discriminating upstream redactor evidence, not a generated-lock patch or masked failure.

Validation

  • Reproduced regression failures locally before production edits.
  • 9 focused Jest suites, 195 tests passed, covering visibility, main action/startup diagnostics, lifecycle, artifact preservation and host-only public debug behavior.
  • npm run build and npm run type-check passed.
  • Changed-file ESLint passed with 0 errors (66 security-rule warnings); commit hook repository ESLint/build also passed.
  • npx markdownlint docs/cloud-hypervisor-foundation.md and git diff --check passed.
  • No live smoke, dry-run, workflow dispatch or rerun performed. Existing run logs/artifacts were read-only evidence; request/storage/runtime behavior in tests is mocked.

Remaining unknowns and deployment

The original metadata HTTP status/cause, the exact unreadable redaction source(s), and the earlier guest host-executor launch failure remain unknown. A future separately authorized run can distinguish metadata failures using these bounded diagnostics. The smoke remains pinned to v0.28.58: an AWF release containing this fix and a separate release-pin follow-up are required before it benefits. This PR does not claim end-to-end KVM validation or authorize a new run or merge.

Keep public fixture verification fail-closed with bounded status and transport classes.
Hand the resolved config artifact to the sudo runner before cleanup can stop
to preserve private recovery state.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 04a0bb67-0f8b-4901-9f55-bf6b57c3d39b
Copilot AI max review requested due to automatic review settings October 15, 2026 04:11
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-10009 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 33297ce

Comment thread src/commands/main-action-audit-handoff.test.ts Fixed
Comment thread src/commands/main-action-audit-handoff.test.ts Fixed
Comment thread src/commands/main-action-audit-handoff.test.ts Fixed
Keep permission assertions and redaction reads and writes on the same opened file.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 04a0bb67-0f8b-4901-9f55-bf6b57c3d39b
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 93.31% 93.33% 📈 +0.02%
Statements 91.90% 91.92% 📈 +0.02%
Functions 90.67% 90.67% ➡️ +0.00%
Branches 85.72% 85.76% 📈 +0.04%
📁 Per-file Coverage Changes (5 files)
File Lines (Before → After) Statements (Before → After)
src/enclave/host-executor-server.ts 90.3% → 90.3% (+0.00%) 88.8% → 89.1% (+0.31%)
src/enclave/public-debug.ts 100.0% → 100.0% (+0.00%) 100.0% → 98.4% (-1.64%)
src/commands/main-action.ts 92.8% → 93.1% (+0.29%) 92.5% → 92.3% (-0.24%)
src/nvx/one-shot-adapter.ts 82.9% → 83.4% (+0.56%) 80.2% → 80.7% (+0.52%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Privileged audit publication has unresolved ownership and path-safety risks requiring human security review.

4 open findings
What changed in this PR

Adds bounded visibility diagnostics and early resolved-config audit handoff to help diagnose AWF enclave startup failures.

Changes:

  • Reports sanitized visibility failure reasons, HTTP status, and allowlisted transport codes.
  • Adds audit ownership handoff, destination checks, and publication warnings.
  • Expands regression coverage and documents release requirements.
File Description
src/​enclave/​public-debug.ts Classifies and sanitizes visibility failures.
src/​enclave/​public-debug.test.ts Tests diagnostics, disclosure limits, and request bounds.
src/​enclave/​cloud-hypervisor-lifecycle.test.ts Updates the visibility rejection assertion.
src/​commands/​main-action.ts Adds early audit handoff and validation.
src/​commands/​main-action.test.ts Checks publication failure warnings.
src/​commands/​main-action-audit-handoff.test.ts Covers failed-startup handoff and unsafe destinations.
docs/​cloud-hypervisor-foundation.md Documents diagnostics, handoff behavior, and deployment requirements.

🧠 Review effort: Max


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

fs.constants.O_RDONLY | fs.constants.O_DIRECTORY | fs.constants.O_NOFOLLOW);
try {
if (!fs.fstatSync(directoryFd).isDirectory()) throw new Error('Unsafe audit directory');
fs.fchownSync(directoryFd, owner.uid, owner.gid);
Comment on lines +131 to +132
const owner = process.getuid?.() === 0 && process.env.SUDO_UID
? { uid: Number(getSafeHostUid()), gid: Number(getSafeHostGid()) } : undefined;
Comment on lines +136 to +137
const directoryFd = fs.openSync(configArtifactDir,
fs.constants.O_RDONLY | fs.constants.O_DIRECTORY | fs.constants.O_NOFOLLOW);
if (!stat.isFile() || stat.nlink !== 1) throw new Error('Unsafe resolved config artifact');
fs.writeFileSync(fd, JSON.stringify(redactedConfig, null, 2) + '\n');
fs.fsyncSync(fd);
if (owner) fs.fchownSync(fd, owner.uid, owner.gid);
@lpcox
lpcox merged commit acd348d into main Oct 15, 2026
39 checks passed
@lpcox
lpcox deleted the lpcox-debug-smoke-preflight-failures branch October 15, 2026 04:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants