Repository navigation
Let a model expression choose the routing level at run time - #77588
Conversation
When the main agent's model is a single expression on a workflow eligible for default routing (Copilot, Claude or Codex), it may resolve to router, router/economy, router/balanced or router/robust. The activation job resolves the expression once and publishes the value, routing decision and mode; later jobs read that output, which also fixes models chosen in pre-activation being empty in the agent job. Expressions that activation cannot resolve as later jobs would keep the existing compilation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
ðĄ Changes recommended
The routing validation checks the trimmed selector, allowing leading or trailing line breaks to bypass rejection and incorrectly enable routing.
1 open issue
What changed in this PR
This PR lets eligible Copilot, Claude, and Codex workflows choose a routing level through a model expression. Activation resolves the expression once and shares the decision with downstream jobs, including models selected by pre-activation steps.
Changes:
- Add runtime routing selection, validation, and activation outputs.
- Apply the shared decision to engine models, threat detection, metadata, and firewall configuration.
- Add regression tests and document supported expressions and router-value restrictions.
| File | Description |
|---|---|
pkg/âworkflow/âthreat_detection_helpers.go |
Use detection defaults when dynamic routing applies. |
pkg/âworkflow/âpi_engine_config_compilation_test.go |
Update Copilot expression expectations. |
pkg/âworkflow/âmodel_routing.go |
Track dynamic selectors and expose routing mode. |
pkg/âworkflow/âmodel_routing_test.go |
Remove the obsolete non-routed expression expectation. |
pkg/âworkflow/âmodel_routing_dynamic.go |
Implement eligibility checks and dynamic routing generation. |
pkg/âworkflow/âmodel_routing_dynamic_test.go |
Cover runtime decisions and downstream behavior. |
pkg/âworkflow/âmodel_routing_default.go |
Share runtime guards with dynamic routing. |
pkg/âworkflow/âcopilot_engine_execution.go |
Guard Copilotâs model during routed runs. |
pkg/âworkflow/âcompiler_yaml_step_lifecycle.go |
Record activationâs resolved model and routing metadata. |
pkg/âworkflow/âcompiler_orchestrator_workflow.go |
Apply dynamic routing during compilation. |
pkg/âworkflow/âcompiler_activation_context.go |
Publish selector and routing-mode outputs. |
pkg/âworkflow/âcodex_engine.go |
Support routed and plain runtime model branches. |
pkg/âworkflow/âclaude_engine.go |
Guard Claude model variables for dynamic routing. |
pkg/âworkflow/âawf_command_builder.go |
Validate and inject the runtime firewall routing mode. |
docs/âsrc/âcontent/âdocs/âspecs/âmodel-alias-specification.md |
Specify expression-based routing in version 1.4.0. |
docs/âsrc/âcontent/âdocs/âreference/âmodel-routing.md |
Document runtime selection and eligibility. |
docs/âsrc/âcontent/âdocs/âreference/âcost-management.md |
Clarify restrictions on model override variables. |
actions/âsetup/âsh/âresolve_model_routing.sh |
Resolve and validate dynamic model selectors. |
actions/âsetup/âjs/âgenerate_aw_info.cjs |
Update comments and errors for expression-based routing. |
Review effort: Balanced
Leading or trailing line breaks were trimmed as whitespace before the check, so a value such as router/robust followed by a newline still routed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
â Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
ð§ Matt Pocock Skills Reviewer is reviewing this pull request using Matt Pocock's engineering skills... |
|
â PR Code Quality Reviewer completed the code quality review.
|
|
â Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
â PR Specification Reviewer completed successfully!
|
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Reviewed the current diff and existing review threads. I found no additional actionable correctness, security, performance, or maintainability issues on changed lines. Review was static; tests were not run per review instructions.
ð Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~4 AIC · â ~2 AIC · â 1.5K
Comment /review to run again
Comment MemoryPeek at saved memory (review-history)History |
ðïļ Design Decision Gate â ADR RequiredThis PR makes significant changes to core business logic (674 added lines under ð Draft ADR committed:
ð What to do next
Once an ADR is linked in the PR body, this gate will re-run and verify that the implementation matches the decision. ð Evidence used for the draft
|
There was a problem hiding this comment.
Reviewed PR head f68e079 against Model Alias Format Specification v1.4.0 §9.4 and the model-routing reference. The implementation and tests support the intended activation-time routing behavior; no blocking conformance regression found. Synchronized docs/specs/model-alias-specification.md in commit dc16477 to make provider/eligibility limits, non-routed fallback, activation failure cases, activation-output propagation, threat-detection isolation, and model-variable restrictions normative. Publication of that specification-only commit is queued on the same-repository PR branch. Coverage was static; tests were not run per review policy.
Generated by PR Specification Reviewer for #77588 · copilot · gpt50 · ~8 AIC · â ~2 AIC · â 1.6K
Comment /review-specs to run again
Comment MemoryPeek at saved memory (spec-review-history)History |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
@pelikhan this is ready for review at e314d2e. CI is green (50 checks passed, none failing), the branch is clean against Validation
The Matt Pocock Skills Reviewer run failed with Could you merge if it looks good? |
| set +o histexpand | ||
| set -euo pipefail | ||
|
|
||
| # Dynamic routing: the main agent's model is an expression, resolved here once and |
There was a problem hiding this comment.
@rabeckett consider moving to javascript in the next iteration
|
ð This pull request is included in a new release. Release: |
code-scanning-fixer (Copilot), static-analysis-report (Claude) and schema-consistency-checker (Codex) now choose their routing level from their model expression instead of an engine.model-routing block, so the run-time routing path from #77588 runs on each engine. Scheduled runs keep their current level; a manual dispatch defaults to it and can pick another. code-scanning-fixer resolves the level in its pre-activation job. TestRecompileDefaultModelRouting_EligibleLocksOnly now tells the dynamic decision step apart from the implicit routing default. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This lets a workflow choose its routing level when the run starts, through the
model:field it already has. Whenthe main agent's model is a single expression, such as
model: ${{ inputs.level || 'router/balanced' }}, and theworkflow would otherwise be eligible for default routing on Copilot, Claude or Codex, the value may now be
router,router/economy,router/balancedorrouter/robust. Any other value runs as before.The activation job resolves the expression once, in a new branch of
resolve_model_routing.sh, and publishes thevalue, whether routing applies, and the mode. The compiler points the main model at that output, the same way
experiment variants already reach later jobs, so the agent, threat detection and aw_info all see one decision. This
also fixes a model chosen by a pre-activation step being empty in the agent job, which today compiles with
needs-consistencywarnings. The agent job writes the routed or plain AWF config at run time, as default routingalready does, with the mode filled in from activation. A routed run never passes the router value to the engine, and
threat detection falls back to its own default model.
Only expressions that activation resolves exactly as later jobs would are eligible: ones that read
inputs,vars,run-wide
githubproperties, experiments or pre-activation outputs. An expression that reads a step's or anotherjob's outputs, a job-local property such as
github.job, or callshashFiles()keeps compiling exactly as it doestoday. Malformed router values such as
router/fast,Router/Economyorrouter/economy?effort=high, and valuescontaining line breaks, stop the run in activation.
Router values in the
GH_AW_MODEL_AGENT_<ENGINE>andGH_AW_DEFAULT_MODEL_<ENGINE>variables, and pi, are left forfollow-ups, as the issue describes.
The model-routing reference gains a short "Choosing the level at run time" section, the router value restrictions
and the model alias specification (now 1.4.0) describe the new rule, and the cost-management note on model variables
is narrowed to the variables.
Tests in
model_routing_dynamic_test.gorun the generated activation decision step and evaluate the agent,detection, aw_info and AWF config branches for router, concrete, empty, whitespace and malformed values. They also
cover eligibility on each engine, experiment variants including an experiment named
model, the pre-activation casewithout
needs-consistencywarnings, and the expression contexts that activation can and cannot resolve. Twoexisting tests that pinned the old non-routed compilation of a whole-expression Copilot model were updated.
Recompiling the committed workflows produces no lock-file changes from this branch.
Fixes #77587