Skip to content

[BUG] apm audit --ci ref-consistency always fails for marketplace object-form dependencies #3220

Closed

Description

Describe the bug

A dependency declared in the marketplace object form ({name, marketplace[, version]}) installs correctly, but apm audit --ci then fails the ref-consistency check with:

ref-consistency details:
  - _marketplace/<marketplace>/<name>: not found in lockfile

_check_ref_consistency (src/apm_cli/policy/ci_checks.py) looks up dep_ref.get_unique_key() in the lockfile. For marketplace object-form entries, DependencyReference is built with the placeholder repo_url=f"_marketplace/{marketplace}/{name}" (src/apm_cli/models/dependency/reference.py). The lockfile, however, records the resolved git coordinates (repo_url: owner/repo, virtual_path: ...), so the lookup can never match. Because ref-consistency fails, the remaining audit checks are not run ("1 of 3 check(s) failed").

The same packages pass every check in two other cases:

  • declared as git coordinates (owner/repo/sub/path#^1.0.0);
  • installed with apm install name@marketplace, which writes a git:/path:/ref: object into apm.yml.

Only the marketplace object form is affected, including when it has a version range.

To Reproduce

mkdir repro && cd repro
apm marketplace add github/awesome-copilot
cat > apm.yml <<'EOF'
name: repro
version: 0.0.0
target: copilot
dependencies:
  apm:
    - name: accessibility-kanban
      marketplace: awesome-copilot
EOF
apm install          # succeeds; lockfile has repo_url: github/awesome-copilot, virtual_path: plugins/accessibility-kanban
apm audit --ci       # exit 1

Output (excerpt):

│ [+] │ lockfile-exists │ Lockfile present                                │
│     │ ref-consistency │ 1 ref mismatch -- run 'apm install --update' to │
│     │                 │ update lockfile                                 │
  ref-consistency details:
    - _marketplace/awesome-copilot/accessibility-kanban: not found in lockfile

That plugin also trips an unrelated drift error about the Native Agent Plugin IR. We reproduced the same ref-consistency failure on a private marketplace with git-subdir entries (tag_pattern: "{name}-v{version}"). There it is the only failing check: object-form deps with and without version: "^1.0.0" fail, while git-coordinate deps on the same packages pass all 10 checks. apm install --update does not help.

Expected behavior

ref-consistency maps a marketplace object-form dep to its resolved lockfile entry, as install does, and passes when they agree. With the current behaviour, a project that declares marketplace deps the documented way cannot use apm audit --ci as a required CI check.

Environment

  • OS: Linux (x86_64)
  • APM Version: 0.33.0 (18c4c43), release binary. _check_ref_consistency is unchanged on main as of 2026-10-13.

Additional context

Workaround: declare the dependency as git coordinates, or install it with apm install name@marketplace so that apm.yml holds the resolved git:/ref: form.

Activity

  1. github-actions commented on Oct 17, 2026

    @github-actions

    Thanks for the detailed diagnosis. Pointing at _check_ref_consistency and the placeholder repo for marketplace object-form dependencies makes this easy to follow.

    Proposed scope (for maintainer approval): make the ref-consistency check resolve marketplace object-form entries to the same lockfile key that install writes, with a regression test. Reviewers should be familiar with the policy/CI checks and marketplace resolution. Exclusions: changing the lockfile key format.

    This is a recommendation only. A maintainer must approve scope and a review contact before implementation starts.

    Generated by Triage Panel · copilot · auto · 45.4 AIC · ⌖ 0.775 AIC · ⊞ 12K · ◷

  2. danielmeppiel commented on Oct 20, 2026

    @danielmeppiel
    Collaborator

    Decision: approve
    Area: project
    Scope: For #3220, reproduce and repair marketplace object-form install/audit consistency by associating each supported marketplace declaration with the correct resolved lockfile dependency through existing manifest, marketplace-resolution and lock-provenance owners. Cover reference consistency and directly coupled orphan detection, honor existing requested version/range/ref semantics, and preserve non-mutating audit. Audit verifies locked evidence rather than re-resolving today's catalog. Include directly related diagnostics, regression coverage, documentation and packaged usage guidance.
    Done when: Installed-CLI APMLifecycle scenarios prove registration through an existing catalog format, selected installation, clean audit, reinstall and repeat audit for object-form declarations without and with supported version selections, equivalent raw-coordinate controls, mixed declarations and supported package layouts. Genuine version/ref mismatches, missing or ambiguous provenance, actual orphans, missing content and drift still fail appropriately. Audit preserves manifest, lockfile and deployment bytes and succeeds for an otherwise valid installation when its catalog is unavailable. Reuse canonical owners; when existing metadata cannot prove a unique match, report the missing evidence instead of guessing or silently passing.
    Out of scope: New dependency syntax, lockfile schema or key formats; catalog-following update semantics tracked in #2903; catalog authentication or additional transports; generic provenance or trust-identity redesign; treating a machine-local alias as durable trust identity; blanket marketplace/plugin audit exemptions; live catalog fetching or dependency updates during audit; automatic repair; assignment, release commitments, approval or merge of PR #3221, automation dispatch or restarting blocked delivery.
    Review contact: Daniel Meppiel (@danielmeppiel)

  3. danielmeppiel commented on Oct 20, 2026

    @danielmeppiel
    Collaborator

    Accepted within the bounded scope recorded above. A supported marketplace declaration should work consistently across installation and audit; users should not have to replace it with raw Git coordinates or turn off CI auditing.

    For example, a project can ask for security-review from marketplace acme. Installation resolves that selection to a real package source and records it in the lockfile. Audit must verify that same locked package, rather than look for _marketplace/acme/security-review as though the placeholder were the resolved repository key. The directly related orphan check must agree too, so fixing the first lookup does not merely expose a second false failure.

    This is not an exemption from audit. Requested version constraints, genuine source/ref mismatches, actual orphans and content drift must remain enforceable. Audit stays read-only and must not fetch the live catalog or change dependencies to produce a pass. Missing or ambiguous locked evidence needs a clear failure, not a guessed match. Durable catalog-following updates remain #2903's separate design work.

    Thank you for the concrete report, and thank you Abhinav Maurya (@Fire162) for the existing contribution in #3221. Please keep that PR as the contribution to coordinate; this scope record is not technical approval of its current implementation. Daniel Meppiel (@danielmeppiel) is the review contact. The issue remains open until the bounded outcome and installed-CLI lifecycle evidence are complete, and it stays linked to enterprise marketplace epic #3297. No assignment, release promise, merge or delivery restart is implied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    status/acceptedHuman scope approval; verify the issue's approval record and review contact before work.triage/recommendedAutomated advice completed; not human scope approval.

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions