[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213783#comment-15213783
]
Qi Dai commented on SPARK-14136:
In case any other people ran into this issue: I figured out the issue.
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213781#comment-15213781
]
Qi Dai commented on SPARK-13289:
I figured out the issue. It's caused by the SPARK_CONF set in my .bashrc
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213778#comment-15213778
]
Qi Dai commented on SPARK-13289:
I tested this commit on the "One Billion Words Language Modeling"
[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210874#comment-15210874
]
Qi Dai commented on SPARK-14136:
Got you. Thanks a lot!
> Spark 2.0 can't start with yarn mode with
[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210870#comment-15210870
]
Qi Dai commented on SPARK-14136:
Do you know what's the required version of HDP to get spark2.0 working?
[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210850#comment-15210850
]
Qi Dai commented on SPARK-14136:
Does Spark2.0 depend on this class? I never had this issue with 1.x
[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210844#comment-15210844
]
Qi Dai commented on SPARK-14136:
The parameters are: ./bin/spark-shell --master yarn --num-executors 50
[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210841#comment-15210841
]
Qi Dai commented on SPARK-14136:
The stack for pyspark is:
Traceback (most recent call last):
File
[
https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210840#comment-15210840
]
Qi Dai commented on SPARK-14136:
It might related with https://issues.apache.org/jira/browse/AMBARI-11186
Qi Dai created SPARK-14136:
--
Summary: Spark 2.0 can't start with yarn mode with
ClassNotFoundException: org.apache.spark.deploy.yarn.history.YarnHistoryService
Key: SPARK-14136
URL:
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208665#comment-15208665
]
Qi Dai commented on SPARK-13289:
sc also failed to start
> Word2Vec generate infinite distances when
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204983#comment-15204983
]
Qi Dai commented on SPARK-13289:
I'm trying to test it with the current master branch and nightly build
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176296#comment-15176296
]
Qi Dai commented on SPARK-13289:
I tried "build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157401#comment-15157401
]
Qi Dai commented on SPARK-13289:
Do you have a runable distribution of the latest master somewhere? I
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148937#comment-15148937
]
Qi Dai commented on SPARK-13289:
Yes, just download the data from the url. Unzip it and change the path
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148920#comment-15148920
]
Qi Dai commented on SPARK-13289:
I'm not familiar with the algorithm and implementation. Maybe we need to
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148872#comment-15148872
]
Qi Dai edited comment on SPARK-13289 at 2/16/16 4:35 PM:
-
Hi Sean, Are you able
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148872#comment-15148872
]
Qi Dai commented on SPARK-13289:
Hi Sean, Are you able to reproduce the issue? Do you need any other
[
https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144685#comment-15144685
]
Qi Dai commented on SPARK-13289:
I'm using the "One Billion Words Language Modeling" dataset available
Qi Dai created SPARK-13289:
--
Summary: Word2Vec generate infinite distances when numIterations>5
Key: SPARK-13289
URL: https://issues.apache.org/jira/browse/SPARK-13289
Project: Spark
Issue Type:
20 matches
Mail list logo
