DanilZ <danil....@me.com> added the comment:
I have managed to solve the problem by inserting in the beginning of my program: import multiprocessing multiprocessing.set_start_method('forkserver') as this is explained here: https://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux <https://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux> It works, but the shell looses some level of interactivity as the results intermediate results don't get printed as the program is executed. > On 2 Nov 2020, at 19:03, Ken Jin <rep...@bugs.python.org> wrote: > > > Ken Jin <kenjin4...@gmail.com> added the comment: > > Hmm apologies I'm stumped then. The only things I managed to surmise from > xgboost's and scikit-learn's GitHub issues is that this is a recurring issue > specifically when using GridSearchCV : > > Threads with discussions on workarounds: > https://github.com/scikit-learn/scikit-learn/issues/6627 > https://github.com/scikit-learn/scikit-learn/issues/5115 > > Issues reported: > https://github.com/dmlc/xgboost/issues/2163 > https://github.com/scikit-learn/scikit-learn/issues/10533 > https://github.com/scikit-learn/scikit-learn/issues/10538 (this looks quite > similar to your issue) > > Some quick workarounds I saw were: > 1. Remove n_jobs argument from GridSearchCV > 2. Use parallel_backend from sklearn.externals.joblib rather than > concurrent.futures so that the pools from both libraries don't have weird > interactions. > > I recommend opening an issue on scikit-learn/XGBoost's GitHub. This seems > like a common problem that they face. > > ---------- > > _______________________________________ > Python tracker <rep...@bugs.python.org> > <https://bugs.python.org/issue42245> > _______________________________________ ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue42245> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com