Skip to content

[Feature] Add A New Approach of Dataset Integration based on ChatML Template with Eval Examples - #2277

Merged
Myhs-phz merged 10 commits into
open-compass:mainfrom
Myhs-phz:newcustom
Oct 14, 2025
Merged

Myhs-phz merged 10 commits into
open-compass:mainfrom
Myhs-phz:newcustom

Conversation

@Myhs-phz

Copy link
Copy Markdown
Collaborator

WIP

@Myhs-phz Myhs-phz changed the title [Feature] Add chat_obj_custom dataset config [Feature] Add ChatMLDataset and Chat_Obj_Custom Dataset Config Sep 25, 2025
@Myhs-phz Myhs-phz removed the WIP label Sep 29, 2025
Comment thread opencompass/datasets/chatml/chatml.py Resolved
Comment thread opencompass/utils/run.py Resolved
@tonysy

tonysy commented Sep 30, 2025

Copy link
Copy Markdown
Collaborator

Also provide a doc on how to use the chatmldataset(via config or CLI), and how to add your own chatmldatset.
We hope such a major feature will ease the support of new benchmark or the user can evaluate their dataset without modfiy the source code(only use the opencompass).

@Myhs-phz Myhs-phz changed the title [Feature] Add ChatMLDataset and Chat_Obj_Custom Dataset Config [Feature] Add A New Approach of Dataset Integration based on ChatML Template with Eval Examples Sep 30, 2025
@Myhs-phz
Myhs-phz requested a review from tonysy October 10, 2025 06:03

@zhulinJulia24 zhulinJulia24 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tonysy tonysy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Myhs-phz
Myhs-phz merged commit 8ee07ac into open-compass:main Oct 14, 2025
2 checks passed
iamkaia pushed a commit to iamkaia/opencompass that referenced this pull request Feb 4, 2026
â€Ķemplate with Eval Examples (open-compass#2277)

* test

* add feature

* fix

* add chatml dataset

* fix

* fix

* fix

* add datasets, example and doc

* fix

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants