Skip to content

Clarify SPDD safeguards and spec-to-test mappings - #75198

Merged
pelikhan merged 2 commits into
mainfrom
copilot/spdd-daily-spec-work-plan-2026-11-05
Nov 5, 2026
Merged

pelikhan merged 2 commits into
mainfrom
copilot/spdd-daily-spec-work-plan-2026-11-05

Conversation

Copilot AI commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

The SPDD review identified gaps in OTLP safeguard ordering, evaluator traceability, replace-label error ownership, Appendix G test references, and retention of older audit details.

  • OTLP: Consolidate fail-closed safeguards, specify five-step shutdown ordering, and clarify that CLI outcome evaluation has no OTLP exporter.
  • Outcome evaluation: Name the entity and evaluator in all 30 safe-output sections; add API-failure conformance coverage.
  • Replace-label: Map each error category to an owning stage and document a fixture with a completeness test.
  • Security and audit records: Add test references for Appendix G categories and summarize mapping audits older than 30 days.

Copilot AI linked an issue Nov 5, 2026 that may be closed by this pull request
1 of 10 tasks
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI changed the title [WIP] Update daily spec work plan for 2026-11-05 Clarify SPDD safeguards and spec-to-test mappings Nov 5, 2026
Copilot AI requested a review from gh-aw-bot November 5, 2026 16:37
@pelikhan
pelikhan marked this pull request as ready for review November 5, 2026 17:56
Copilot AI adaptive review requested due to automatic review settings November 5, 2026 17:56
@pelikhan
pelikhan merged commit bfbbe70 into main Nov 5, 2026
39 checks passed
@pelikhan
pelikhan deleted the copilot/spdd-daily-spec-work-plan-2026-11-05 branch November 5, 2026 17:56
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #75198

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: gpt56 max · ~1 AIC · ⌖ ~1 AIC · ⊞ 6.3K · ◷

🧊 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗ïļ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗ïļ

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗ïļ Â· routed: opus55 medium · ~12 AIC · ⌖ ~2 AIC · ⊞ 7.7K · ◷

🏗ïļ ADR gate enforced by Design Decision Gate 🏗ïļ

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ðŸŸĄ Changes recommended

Default telemetry endpoints with missing headers remain exportable under warn or error, allowing unauthenticated transmission before the separate credential check fails.

2 open issues
What changed in this PR

This PR clarifies gh-aw’s safeguards and improves specification-to-test traceability following the SPDD review in #75197. Changes focus on documentation, conformance fixtures, and tests rather than runtime behavior.

Changes:

  • Consolidate OTLP safeguards and finalization ordering, and clarify the CLI outcome evaluator’s export boundary.
  • Identify entities and evaluators for all 30 safe-output types and add API-failure conformance coverage.
  • Map replace-label errors and security checklist items to coverage, and summarize historical audits.
File Description
specs/​security-architecture-spec.md Expands Appendix G test references and coverage qualifications.
specs/​safe-output-outcome-evaluation.md Names entities and evaluators; clarifies API and OTLP failure boundaries.
specs/​replace-label-spec.md Assigns error ownership and specifies blocklist-first validation.
specs/​replace-label-compliance/​rl-004-error-categories.yaml Adds error-category, stage, and result mappings.
specs/​replace-label-compliance/​README.md Documents the new fixture and completeness check.
specs/​otel-observability-spec.md Consolidates safeguards and defines five-step finalization ordering.
specs/​compiler-threat-detection-changelog.md Replaces older detailed audits with a historical summary.
pkg/​workflow/​replace_label_formal_test.go Checks error-fixture completeness and expected mappings.
pkg/​cli/​testdata/​outcome_conformance.json Adds a close-issue API-unavailability case.
pkg/​cli/​outcome_eval_formal_test.go Renames and clarifies the existing API-failure test.

Review effort: Balanced (auto)


1. Drop such an endpoint from every runtime span emission path (job-setup, conclusion, outcome, and MCP gateway spans) before any network call is attempted, not only in the job that performs credential validation; and
2. Additionally fail at least one job in the workflow (for example, via a runtime guard step) with a diagnostic that identifies the missing headers secret, so a misconfigured default is visible rather than silently degrading to no telemetry.
A default fallback endpoint that resolves to a non-empty URL without its configured headers MUST fail the default-credentials check with a diagnostic identifying the missing headers secret. When the effective `if-missing` policy is `ignore`, every runtime exporter MUST also omit the endpoint before any network call, including exporters in jobs that do not execute that check. The complete fail-closed behavior and ordering are specified in §16.6.
Comment on lines +1931 to 1932
Path check revalidated on 2026-11-05: every named test below resolves in
`pkg/workflow/security_architecture_sg_formal_test.go`.
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[spdd] Daily spec work plan - 2026-11-05

4 participants