Repository navigation
Honor Retry-After in Copilot harness HTTP 429 retries - #76807
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
â PR Code Quality Reviewer completed the code quality review. PR Code Quality ReviewerOutcome: The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
â PR Specification Reviewer completed successfully! Commit pushed:
|
|
â Design Decision Gate ðïļ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate ðïļOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
ð§ Matt Pocock Skills Reviewer has completed the skills-based review. â
|
|
â Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
â Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
Comment MemoryPeek at saved memory (pr-code-quality-reviewer)Note This comment is managed by comment memory.Expand the saved memory block to view or edit the persistent context for this thread.
|
There was a problem hiding this comment.
ð Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~5 AIC · â ~1 AIC · â 1.6K
Comment /review to run again
| if (NATIVE_RATE_LIMIT_ERROR_PATTERN.test(JSON.stringify(event))) nativeTerminalRateLimitError = true; | ||
| } | ||
| }, | ||
| onStderrLine: line => { |
There was a problem hiding this comment.
observeRetryAfter accepts every stderr line for the entire attempt, unlike stdout where non-terminal event types are filtered. This means unrelated diagnostic text can combine a 429 mention with a Retry-After header and override the normal failure classification, causing an unnecessary long sleep or rate_limit_wait_exceeded stop even when the terminal failure is not rate limiting. Please restrict stderr evidence to a recognized terminal error/report (or correlate the header with the same error) before applying it.
There was a problem hiding this comment.
Fixed in 61477796: stderr Retry-After evidence is now accepted only alongside a recognized response status, and assistant/tool events remain excluded. Header evidence is retained independently of the output tail.
There was a problem hiding this comment.
Found one blocking correctness issue: stderr Retry-After evidence is not scoped to terminal rate-limit errors, so unrelated diagnostic output can trigger an incorrect wait or stop. Inline comment added. Static review only; tests were not run per review instructions.
ð Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~5 AIC · â ~1 AIC · â 1.6K
Comment /review to run again
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Reviewed head d4fb266 against the applicable formal contract. The implementation and tests intentionally add bounded Copilot terminal HTTP 429 Retry-After handling, while preserving invalid/missing/expired-header behavior and excluding assistant/tool events from retry evidence. This behavior is covered by the synchronized Unified Agent Session Specification v2.9.0, Section 6.4, T-UAS-090: numeric-seconds and HTTP-date parsing, max-delay/soft-timeout bounds, and allowlisted rate_limit_wait_exceeded classification. The Copilot SDK Driver Specification v1.0.2 is out of scope. Specification commit e0a6f87 was queued for publication to the PR branch. No blocking contract violations found.
Generated by PR Specification Reviewer for #76807 · copilot · gpt50 · ~5 AIC · â ~2 AIC · â 1.6K
Comment /review-specs to run again
There was a problem hiding this comment.
ðĄ Changes recommended
Retry-After handling has correctness gaps: early retry branches bypass delay limits, valid HTTP-date formats are rejected, and rate-limit detection can misclassify responses.
3 open issues
What changed in this PR
This PR adds bounded Retry-After handling to the Copilot harness, addressing retries that otherwise occur inside an upstream HTTP 429 rate-limit window.
Changes:
- Parse retry delays from terminal error events and stderr, excluding assistant/tool events.
- Apply delay and timeout limits, recording
rate_limit_wait_exceededwhen the wait cannot fit. - Add parser and integration tests, extend telemetry types, and document configuration.
| File | Description |
|---|---|
docs/âsrc/âcontent/âdocs/âreference/âengines.md |
Documents retry timing and configured bounds. |
docs/âpublic/âschemas/âunified-session.schema.json |
Adds the new failure class to the schema. |
actions/âsetup/âjs/âtypes/âunified_session.d.ts |
Extends the failure-class type. |
actions/âsetup/âjs/âharness_retry_runner.cjs |
Documents retry-delay metadata on attempt results. |
actions/âsetup/âjs/âharness_retry_record.test.cjs |
Covers normalization of the new failure class. |
actions/âsetup/âjs/âharness_retry_record.cjs |
Recognizes the new failure class. |
actions/âsetup/âjs/âcopilot_retry_after.test.cjs |
Adds parser and harness integration coverage. |
actions/âsetup/âjs/âcopilot_retry_after.cjs |
Implements Retry-After parsing. |
actions/âsetup/âjs/âcopilot_harness.cjs |
Collects retry evidence and applies bounded waits. |
Review effort: Balanced (auto)
| if (result.retryAfterMs != null) { | ||
| // Node timers overflow above 2^31-1 ms and would otherwise retry immediately. | ||
| const exceedsDelayCap = result.retryAfterMs > Math.min(maxDelayMs, 2147483647); | ||
| const exceedsTimeout = softTimeoutGuard && Date.now() + result.retryAfterMs >= softTimeoutGuard.softDeadlineMs; |
There was a problem hiding this comment.
Fixed in 61477796: every retry decision now passes through the same Retry-After delay and budget checks; the existing fresh-versus-continue choice is preserved.
| */ | ||
| function parseCopilotRetryAfterMs(output, now = Date.now()) { | ||
| const text = output.replace(/\\[rn]/g, "\n").replace(/\\"/g, '"'); | ||
| if (!/\b429\b/.test(text)) return null; |
There was a problem hiding this comment.
Fixed in 61477796: Retry-After is associated with its explicit response status or recognized Copilot error signature. CAPIError: Too Many Requests is supported, while a 503 with a numeric Retry-After value is not treated as a 429.
| } else if (/^[A-Za-z]{3}, /.test(value)) { | ||
| ms = Date.parse(value) - now; |
There was a problem hiding this comment.
Fixed in 61477796: the parser now supports IMF-fixdate, RFC 850, and asctime forms, with UTC interpretation and the RFC two-digit-year rule; regression tests cover both obsolete formats.
There was a problem hiding this comment.
Skills-Based Review ð§
Applied /diagnosing-bugs to the Retry-After branch in copilot_harness.cjs. The design is sound: the parser ignores assistant and tool events, the max-delay-ms cap and soft-timeout guard prevent unbounded waits, and exhaustion stops like the existing paths.
ð Key Themes & Highlights
Key Themes
- Quota ordering: the Retry-After branch runs before the quota stop, so a
429 quota exceededwithretry-afteris retried. Confirmed with node. - Output truncation: the 64 KB tail buffer can drop an early
Retry-Afterheader.
Positive Highlights
- â Timer overflow above 2^31-1 ms is handled, with tests.
- â Assistant text cannot trigger a wait, since only error events and stderr are observed.
- â
Exhaustion returns
stop, consistent with the existing paths.
Limitations
- The vitest suites could not run here:
npm cifailed on a registry TLS certificate error, so the test results are unverified.
@copilot please address the review comments above.
ð§ Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~7 AIC · â ~2 AIC · â 6.6K
Comment /matt to run again
| log(`attempt ${attempt + 1}: startup interruption detected (driver-handoff Turns=0) but retry budget exhausted â no attempts remain`); | ||
| } | ||
|
|
||
| if (result.retryAfterMs != null) { |
There was a problem hiding this comment.
[/diagnosing-bugs] This Retry-After branch runs before the isQuotaExceeded stop (line 2550). A hard quota error such as CAPIError: 429 429 quota exceeded with a retry-after header parses to a delay, and isCAPIQuotaExceededError also matches it, so it is classified as rate_limit_error and retried instead of stopped. I confirmed this with node: the parser returns 2000 and the quota matcher returns true for that input.
ðĄ Suggested fix or test
If a quota error should stop, gate the branch on !isQuotaExceeded. If retrying is intended, add a regression test that pins the quota-plus-Retry-After case so the ordering is explicit.
@copilot please address this.
There was a problem hiding this comment.
Fixed in 61477796: explicit hard quota-exceeded errors remain terminal even with Retry-After, while retryable CAPIError: Too Many Requests responses honor the header.
| } catch { | ||
| // Ignore JSON parsing failures: native CLI errors and header dumps are plain text. | ||
| } | ||
| retryAfterOutput = (retryAfterOutput + "\n" + line).slice(-64 * 1024); |
There was a problem hiding this comment.
[/diagnosing-bugs] The 64 KB buffer keeps only the tail of the output (slice(-64 * 1024)). A Retry-After header printed early in a long run is dropped, and the harness silently falls back to exponential backoff.
ðĄ Suggested change
Keep header-bearing lines separately, or retain the first N KB along with the tail. Add a test that puts the header before a more-than-64 KB prefix so the truncation is covered.
@copilot please address this.
There was a problem hiding this comment.
Fixed in 61477796: Retry-After evidence is retained separately from the output tail, so an early header is not lost after long stderr output. Added a >64 KB regression test.
|
ðģ A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "50a880d0a0f8f8fa6460d0d02db24c9d5348434e",
"kind": "requested",
"schema_version": "1",
"state": "0331cbd9b8b4ba0dfd13630e4b14dc5c00611b506d2b0db80bc81be6ac6d089f",
"work": [
"49be6728adf5406903178a22e1d8e990267ed84176ca555f2587afda3d483a59",
"867e2f680ea9b4e83bba85be20ab1d3b4351ca6a4e2642c8a266c5be76c16be2",
"facb9d97420d5c78f5ceda7e09fb82b05dd669a3254219d0a33bd1f49caa8baf"
],
"workflow": "pr-sous-chef"
}
|
âĶer-http-429 Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Completed in |
| } | ||
| }, | ||
| onStderrLine: line => { | ||
| observeStderrRetryAfter(line); |
|
Category: bug | Risk: medium | Score: 83/100 (impact 41 + urgency 27 + quality 15). Recommended action: fast_track. Batch: pr-batch:copilot-runtime. Reason: fixes a real runtime failure in Copilot 429 handling and is already green on CI.
|
|
@copilot generalize to other agentic engines or any engine using GitHub inference |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Implemented in |
|
@copilot only enable this check if the provider of the model is GitHub |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Implemented in |
| } | ||
| }, | ||
| onStderrLine: line => { | ||
| if (honorRetryAfter) observeStderrRetryAfter(line); |
|
@copilot run pr-finisher skill |
|
ð This pull request is included in a new release. Release: |

The Copilot harness retried HTTP 429 failures on fixed backoff, exhausting attempts inside the upstream rate-limit window.
Retry-Afterseconds or HTTP dates from terminal error events and stderr. Wait the requested duration within the configured retry budget.rate_limit_wait_exceededwhen the wait exceedsmax-delay-msor the remaining timeout budget, preserving shutdown time and structured telemetry.A 245-second window requires sufficient remaining execution time and:
Retry-Afteron HTTP 429 instead of retrying inside the window #76767