DanilZ <[email protected]> added the comment:
I have managed to solve the problem by inserting in the beginning of my program:
import multiprocessing
multiprocessing.set_start_method('forkserver')
as this is explained here:
https://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux
<https://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux>
It works, but the shell looses some level of interactivity as the results
intermediate results don't get printed as the program is executed.
> On 2 Nov 2020, at 19:03, Ken Jin <[email protected]> wrote:
>
>
> Ken Jin <[email protected]> added the comment:
>
> Hmm apologies I'm stumped then. The only things I managed to surmise from
> xgboost's and scikit-learn's GitHub issues is that this is a recurring issue
> specifically when using GridSearchCV :
>
> Threads with discussions on workarounds:
> https://github.com/scikit-learn/scikit-learn/issues/6627
> https://github.com/scikit-learn/scikit-learn/issues/5115
>
> Issues reported:
> https://github.com/dmlc/xgboost/issues/2163
> https://github.com/scikit-learn/scikit-learn/issues/10533
> https://github.com/scikit-learn/scikit-learn/issues/10538 (this looks quite
> similar to your issue)
>
> Some quick workarounds I saw were:
> 1. Remove n_jobs argument from GridSearchCV
> 2. Use parallel_backend from sklearn.externals.joblib rather than
> concurrent.futures so that the pools from both libraries don't have weird
> interactions.
>
> I recommend opening an issue on scikit-learn/XGBoost's GitHub. This seems
> like a common problem that they face.
>
> ----------
>
> _______________________________________
> Python tracker <[email protected]>
> <https://bugs.python.org/issue42245>
> _______________________________________
----------
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue42245>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
