Repository navigation
Clarify SPDD safeguards and spec-to-test mappings - #75198
Conversation
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
â PR Code Quality Reviewer completed the code quality review.
|
|
â Ponytail Reviewer completed successfully!
|
|
â Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
â Design Decision Gate ðïļ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate ðïļOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
ð§ Matt Pocock Skills Reviewer has completed the skills-based review. â
|
There was a problem hiding this comment.
ðĄ Changes recommended
Default telemetry endpoints with missing headers remain exportable under warn or error, allowing unauthenticated transmission before the separate credential check fails.
2 open issues
What changed in this PR
This PR clarifies gh-awâs safeguards and improves specification-to-test traceability following the SPDD review in #75197. Changes focus on documentation, conformance fixtures, and tests rather than runtime behavior.
Changes:
- Consolidate OTLP safeguards and finalization ordering, and clarify the CLI outcome evaluatorâs export boundary.
- Identify entities and evaluators for all 30 safe-output types and add API-failure conformance coverage.
- Map replace-label errors and security checklist items to coverage, and summarize historical audits.
| File | Description |
|---|---|
specs/âsecurity-architecture-spec.md |
Expands Appendix G test references and coverage qualifications. |
specs/âsafe-output-outcome-evaluation.md |
Names entities and evaluators; clarifies API and OTLP failure boundaries. |
specs/âreplace-label-spec.md |
Assigns error ownership and specifies blocklist-first validation. |
specs/âreplace-label-compliance/ârl-004-error-categories.yaml |
Adds error-category, stage, and result mappings. |
specs/âreplace-label-compliance/âREADME.md |
Documents the new fixture and completeness check. |
specs/âotel-observability-spec.md |
Consolidates safeguards and defines five-step finalization ordering. |
specs/âcompiler-threat-detection-changelog.md |
Replaces older detailed audits with a historical summary. |
pkg/âworkflow/âreplace_label_formal_test.go |
Checks error-fixture completeness and expected mappings. |
pkg/âcli/âtestdata/âoutcome_conformance.json |
Adds a close-issue API-unavailability case. |
pkg/âcli/âoutcome_eval_formal_test.go |
Renames and clarifies the existing API-failure test. |
Review effort: Balanced (auto)
|
|
||
| 1. Drop such an endpoint from every runtime span emission path (job-setup, conclusion, outcome, and MCP gateway spans) before any network call is attempted, not only in the job that performs credential validation; and | ||
| 2. Additionally fail at least one job in the workflow (for example, via a runtime guard step) with a diagnostic that identifies the missing headers secret, so a misconfigured default is visible rather than silently degrading to no telemetry. | ||
| A default fallback endpoint that resolves to a non-empty URL without its configured headers MUST fail the default-credentials check with a diagnostic identifying the missing headers secret. When the effective `if-missing` policy is `ignore`, every runtime exporter MUST also omit the endpoint before any network call, including exporters in jobs that do not execute that check. The complete fail-closed behavior and ordering are specified in §16.6. |
| Path check revalidated on 2026-11-05: every named test below resolves in | ||
| `pkg/workflow/security_architecture_sg_formal_test.go`. |
|
ð This pull request is included in a new release. Release: |


The SPDD review identified gaps in OTLP safeguard ordering, evaluator traceability, replace-label error ownership, Appendix G test references, and retention of older audit details.