Abhishek Reddy Chamakura created SPARK-24623:
------------------------------------------------
Summary: Hadoop - Spark Cluster - Python XGBoost - Not working in
distributed mode
Key: SPARK-24623
URL: https://issues.apache.org/jira/browse/SPARK-24623
Project: Spark
Issue Type: Bug
Components: Deploy
Affects Versions: 2.1.1
Environment: Hadoop - Hortonworks Cluster
Total Nodes - 18
Worker Nodes - 13
Reporter: Abhishek Reddy Chamakura
Hi
We recently installed python on the Hadoop cluster with lot of data science
python modules including xgboost , spicy , scikit learn , pandas
Using pyspark the data scientists are able to test there scoring models in the
distributed mode on the Hadoop cluster. But with python - xgboost the pyspark
job is not getting distributed and it is trying to run only on one instance.
we are trying to achieve the distributed mode when using python xgboost via
pyspark.
It would be a great help if you can direct me on how to achieve this.
Thanks,
Abhishek
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
