Skip to content

Fix early stopping in the Python package - #4638

Merged
chyunsu3 merged 4 commits into
dmlc:masterfrom
chyunsu3:fix_early_stopping_python
Jul 7, 2019
Merged

chyunsu3 merged 4 commits into
dmlc:masterfrom
chyunsu3:fix_early_stopping_python

Conversation

@chyunsu3

@chyunsu3 chyunsu3 commented Jul 4, 2019 •

Copy link
Copy Markdown
Collaborator

Closes #4630
Closes #4421

@chyunsu3

chyunsu3 commented Jul 4, 2019 •

Copy link
Copy Markdown
Collaborator Author

In the future, I may want to overhaul how the evaluation metrics are passed from C++ to Python/R/Java. Currently, the C++ function XGBoosterEvalOneIter() is returning a string consisting of metric names and values all combined. This means that Python/R/Java bindings have to parse the returned string. It may be better to have the C++ function return a structured object instead (something like a key-value store).

@chyunsu3

chyunsu3 commented Jul 4, 2019

Copy link
Copy Markdown
Collaborator Author

@sriramch @rongou

@chyunsu3
chyunsu3 merged commit 1aaf4a6 into dmlc:master Jul 7, 2019
@chyunsu3
chyunsu3 deleted the fix_early_stopping_python branch July 7, 2019 08:01
thesuperzapper pushed a commit to thesuperzapper/xgboost that referenced this pull request Jul 8, 2019
* Fix dmlc#4630, dmlc#4421: Preserve correct ordering between metrics, and always use last metric for early stopping

* Clarify semantics of early stopping in presence of multiple valid sets and metrics

* Add a test

* Fix lint
@sriramch

sriramch commented Jul 8, 2019

Copy link
Copy Markdown
Contributor

Currently, the C++ function XGBoosterEvalOneIter() is returning a string consisting of metric names and values all combined. This means that Python/R/Java bindings have to parse the returned string. It may be better to have the C++ function return a structured object instead (something like a key-value store).

@hcho3 are there plans to use protobuf or similar serialized data formats for things such as this? this may also have a nice side effect of supporting compatibility when the return data evolves, as opposed to a binary structs.

@trivialfis

Copy link
Copy Markdown
Member

@sriramch There is a RFC of using JSON for everything. But currently I'm not sure about its priority.

sperlingxx pushed a commit to alipay/ant-xgboost that referenced this pull request Jul 17, 2019
* Fix dmlc#4630, dmlc#4421: Preserve correct ordering between metrics, and always use last metric for early stopping

* Clarify semantics of early stopping in presence of multiple valid sets and metrics

* Add a test

* Fix lint
sperlingxx added a commit to alipay/ant-xgboost that referenced this pull request Jul 17, 2019
@lock lock Bot locked as resolved and limited conversation to collaborators Oct 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

4 participants