You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[EPIC] Allow failed worker retry in distributed training #4753
Fault recovery in native distributed XGB training (not xgb-spark) have been broken for more than a year. Community have seen various of issue like this dmlc/rabit#63
This looks interesting indeed. One question: "What is native distributed XGB training?" How can I use it? Is it the one which involves a python package + dask?
This looks interesting indeed. One question: "What is native distributed XGB training?" How can I use it? Is it the one which involves a python package + dask?
If you try xgboost with more than one machine. You are already using rabit, we added feature and track changes in progress to xgb layer
@chenqin, I understand that xgboost uses rabit as AllReduce implementation. I still don't really know what is a native distributed XGB training?
Does xgboost-spark run native distributed XGB training? If yes then why do we add "native" here? What other distributed XGB training do we have?
I am new to the project and I am still discovering different ways to use xgboost so this is an honest question.
@chenqin, I understand that xgboost uses rabit as AllReduce implementation. I still don't really know what is a native distributed XGB training?
Does xgboost-spark run native distributed XGB training? If yes then why do we add "native" here? What other distributed XGB training do we have?
I am new to the project and I am still discovering different ways to use xgboost so this is an honest question.
In some use cases where user don't have complex feature generating needs, user can launch xgboost native worker (c++) without data processing
framework. https://github.com/kubeflow/xgboost-operator
changed the title [-]Allow failed worker retry in distributed training[/-][+][EPIC] Allow failed worker retry in distributed training[/+]on Sep 19, 2019
Activity
trivialfis commentedon Aug 12, 2019
Looks interesting. ;-)
trams commentedon Aug 20, 2019
This looks interesting indeed. One question: "What is native distributed XGB training?" How can I use it? Is it the one which involves a python package + dask?
chenqin commentedon Aug 23, 2019
If you try xgboost with more than one machine. You are already using rabit, we added feature and track changes in progress to xgb layer
https://www.slideshare.net/ChenQin1/scaling-xgboost
trams commentedon Aug 23, 2019
@chenqin, I understand that xgboost uses rabit as AllReduce implementation. I still don't really know what is a native distributed XGB training?
Does xgboost-spark run native distributed XGB training? If yes then why do we add "native" here? What other distributed XGB training do we have?
I am new to the project and I am still discovering different ways to use xgboost so this is an honest question.
chenqin commentedon Aug 23, 2019
In some use cases where user don't have complex feature generating needs, user can launch xgboost native worker (c++) without data processing
framework. https://github.com/kubeflow/xgboost-operator
workaround booster save/load inconsistency, leave fix to item 3 in dmâĶ
chenqin commentedon Aug 29, 2019
from description in this thread, yes, dask-xgboost leverage rabit as well.
#2032
[-]Allow failed worker retry in distributed training[/-][+][EPIC] Allow failed worker retry in distributed training[/+]chyunsu3 commentedon Sep 19, 2019
@chenqin It's epic :)
nateagr commentedon Dec 11, 2019
Hi there! Any update on this epic ? Thanks.
chenqin commentedon Dec 14, 2019
Yes, it's going (holiday session slow), still miss the final piece of patch XGBoost-spark after we can get current open pr landed.
trivialfis commentedon Sep 27, 2020
I don't think this is possible in short term. Let's stick with fail all strategy for now. We @hcho3 are thinking about redesigning RABIT from scratch.