Skip to content

feat: Decode -> Prefill cached kv transfer - #340

Merged
ptarasiewiczNV merged 6 commits into
mainfrom
ptarasiewicz/D2P-kv-transfer
Mar 26, 2025
Merged

ptarasiewiczNV merged 6 commits into
mainfrom
ptarasiewicz/D2P-kv-transfer

Conversation

@ptarasiewiczNV

Copy link
Copy Markdown
Contributor

Overview:

Allow PrefillWorker to fetch precomputed blocks from VllmWorker KV cache.

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@ptarasiewiczNV
ptarasiewiczNV requested a review from a team March 21, 2025 17:24
@ptarasiewiczNV ptarasiewiczNV changed the title feature: Decode -> Prefill cached kv transfer feat: Decode -> Prefill cached kv transfer Mar 21, 2025
@ptarasiewiczNV
ptarasiewiczNV merged commit 09e8a0c into main Mar 26, 2025
@ptarasiewiczNV
ptarasiewiczNV deleted the ptarasiewicz/D2P-kv-transfer branch March 26, 2025 19:41
kylehh pushed a commit to kylehh/dynamo that referenced this pull request Apr 11, 2025

This branch was previously deployed

1 inactive deployment
GITLAB — fbc272dd Deployed Mar 26, 2025 by copy-pr-bot[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants