[jira] [Updated] (SPARK-2595) The driver run garbage collection, when the executor throws OutOfMemoryError exception

[ https://issues.apache.org/jira/browse/SPARK-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2595: --- Description: [SPARK-1103|https://issues.apache.org/jira/browse/SPARK-1103] implementation GC-based

[jira] [Comment Edited] (SPARK-2595) The driver run garbage collection, when the executor throws OutOfMemoryError exception

[ https://issues.apache.org/jira/browse/SPARK-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067653#comment-14067653 ] Guoqiang Li edited comment on SPARK-2595 at 7/19/14 7:45 PM: -

[jira] [Commented] (SPARK-2595) The driver run garbage collection, when the executor throws OutOfMemoryError exception

[ https://issues.apache.org/jira/browse/SPARK-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067653#comment-14067653 ] Guoqiang Li commented on SPARK-2595: Sorry I removed it. The driver run garbage

[jira] [Commented] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066176#comment-14066176 ] Guoqiang Li commented on SPARK-2491: [~sarutak] I submitted a patch.

[jira] [Updated] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2491: --- Component/s: YARN When an OOM is thrown,the executor does not stop properly.

[jira] [Updated] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2491: --- Summary: When an OOM is thrown,the executor does not stop properly. (was: When an OOM is thrown,the

[jira] [Updated] (SPARK-2520) the executor is thrown java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2520: --- Description: This issue occurs with a very small probability. I can not reproduce it. The executor

[jira] [Created] (SPARK-2544) Improve ALS algorithm resource usage

Guoqiang Li created SPARK-2544: -- Summary: Improve ALS algorithm resource usage Key: SPARK-2544 URL: https://issues.apache.org/jira/browse/SPARK-2544 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in start-history-server.sh

[ https://issues.apache.org/jira/browse/SPARK-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2481: --- Summary: The environment variables SPARK_HISTORY_OPTS is covered in start-history-server.sh (was:

[jira] [Updated] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in start-history-server.sh

[ https://issues.apache.org/jira/browse/SPARK-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2481: --- Description: In If we have the following code in the conf/spark-env.sh {{export

[jira] [Updated] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in start-history-server.sh

[ https://issues.apache.org/jira/browse/SPARK-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2481: --- Description: In If we have the following code in the conf/spark-env.sh {{export

[jira] [Updated] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in start-history-server.sh

[ https://issues.apache.org/jira/browse/SPARK-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2481: --- Description: If we have the following code in the conf/spark-env.sh {{export

[jira] [Created] (SPARK-2491) When an exception is thrown,the executor is not properly stopped,

Guoqiang Li created SPARK-2491: -- Summary: When an exception is thrown,the executor is not properly stopped, Key: SPARK-2491 URL: https://issues.apache.org/jira/browse/SPARK-2491 Project: Spark

[jira] [Commented] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in start-history-server.sh

[ https://issues.apache.org/jira/browse/SPARK-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063036#comment-14063036 ] Guoqiang Li commented on SPARK-2481: [~tsudukim] There is only one need to write that

[jira] [Updated] (SPARK-2491) When an OOM is thrown,the executor is not properly stopped.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2491: --- Summary: When an OOM is thrown,the executor is not properly stopped. (was: When an exception is

[jira] [Created] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

Guoqiang Li created SPARK-2480: -- Summary: Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag Key: SPARK-2480 URL: https://issues.apache.org/jira/browse/SPARK-2480 Project: Spark

[jira] [Created] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in spark-env.sh

Guoqiang Li created SPARK-2481: -- Summary: The environment variables SPARK_HISTORY_OPTS is covered in spark-env.sh Key: SPARK-2481 URL: https://issues.apache.org/jira/browse/SPARK-2481 Project: Spark

[jira] [Created] (SPARK-2482) Resolve sbt warnings during build

Guoqiang Li created SPARK-2482: -- Summary: Resolve sbt warnings during build Key: SPARK-2482 URL: https://issues.apache.org/jira/browse/SPARK-2482 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2484) By default does not run hive compatibility tests

Guoqiang Li created SPARK-2484: -- Summary: By default does not run hive compatibility tests Key: SPARK-2484 URL: https://issues.apache.org/jira/browse/SPARK-2484 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061707#comment-14061707 ] Guoqiang Li commented on SPARK-2480: I'm sorry, my description is not correct . I mean

[jira] [Comment Edited] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061707#comment-14061707 ] Guoqiang Li edited comment on SPARK-2480 at 7/15/14 5:50 AM: -

[jira] [Reopened] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li reopened SPARK-2480: Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

[jira] [Updated] (SPARK-2480) Resolve sbt warnings NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2480: --- Summary: Resolve sbt warnings NOTE: SPARK_YARN is deprecated, please use -Pyarn flag (was: Remove

[jira] [Commented] (SPARK-2098) All Spark processes should support spark-defaults.conf, config file

[ https://issues.apache.org/jira/browse/SPARK-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059700#comment-14059700 ] Guoqiang Li commented on SPARK-2098: PR: https://github.com/apache/spark/pull/1256

[jira] [Comment Edited] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058298#comment-14058298 ] Guoqiang Li edited comment on SPARK-2398 at 7/11/14 3:16 AM: -

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058298#comment-14058298 ] Guoqiang Li commented on SPARK-2398: Q1. {{-Xmx}} is only the heap space, {{native

[jira] [Comment Edited] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058298#comment-14058298 ] Guoqiang Li edited comment on SPARK-2398 at 7/11/14 3:17 AM: -

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054434#comment-14054434 ] Guoqiang Li commented on SPARK-2398: Seems to be related to

[jira] [Issue Comment Deleted] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Comment: was deleted (was: [~pwendell] I'm not familiar with python, where relevant code?) The

[jira] [Updated] (SPARK-1681) Handle hive support correctly in ./make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1681: --- Summary: Handle hive support correctly in ./make-distribution.sh (was: Handle hive support

[jira] [Commented] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050005#comment-14050005 ] Guoqiang Li commented on SPARK-1989: In this case should also triggers the driver

[jira] [Commented] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049033#comment-14049033 ] Guoqiang Li commented on SPARK-2167: [~tgraves] I have some work related to the

[jira] [Commented] (SPARK-2085) Apply user-specific regularization instead of uniform regularization in Alternating Least Squares (ALS)

[ https://issues.apache.org/jira/browse/SPARK-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045554#comment-14045554 ] Guoqiang Li commented on SPARK-2085: [~mengxr] This PR should be merged into version

[jira] [Comment Edited] (SPARK-2085) Apply user-specific regularization instead of uniform regularization in Alternating Least Squares (ALS)

[ https://issues.apache.org/jira/browse/SPARK-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045554#comment-14045554 ] Guoqiang Li edited comment on SPARK-2085 at 6/27/14 5:40 AM: -

[jira] [Commented] (SPARK-2248) spark.default.parallelism does not apply in local mode

[ https://issues.apache.org/jira/browse/SPARK-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041838#comment-14041838 ] Guoqiang Li commented on SPARK-2248: PR: https://github.com/apache/spark/pull/1194

[jira] [Closed] (SPARK-1698) Improve spark integration

[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1698. -- Resolution: Won't Fix Improve spark integration - Key:

[jira] [Updated] (SPARK-2229) SizeBasedRollingPolicy throw an java.lang.IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2229: --- Summary: SizeBasedRollingPolicy throw an java.lang.IllegalArgumentException (was:

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038964#comment-14038964 ] Guoqiang Li commented on SPARK-2224: {{mvn -Phadoop-2.3 -Pyarn

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035467#comment-14035467 ] Guoqiang Li commented on SPARK-1291: PR: https://github.com/apache/spark/pull/1112

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035522#comment-14035522 ] Guoqiang Li commented on SPARK-1291: [~rahulsinghal.iitd] Thank you, looking forward

[jira] [Commented] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035795#comment-14035795 ] Guoqiang Li commented on SPARK-2156: The problem here:

[jira] [Updated] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2156: --- Affects Version/s: 0.9.1 When the size of serialized results for one partition is slightly smaller

[jira] [Commented] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036838#comment-14036838 ] Guoqiang Li commented on SPARK-2156: [~pwend...@gmail.com] When the size of

[jira] [Issue Comment Deleted] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2156: --- Comment: was deleted (was: [~pwend...@gmail.com]) When the size of serialized results for one

[jira] [Commented] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036844#comment-14036844 ] Guoqiang Li commented on SPARK-2156: [~pwendell] This seems to is not the same as

[jira] [Resolved] (SPARK-1471) Worker not recognize Driver state at standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li resolved SPARK-1471. Resolution: Fixed Worker not recognize Driver state at standalone mode

[jira] [Updated] (SPARK-1471) Worker not recognize Driver state at standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1471: --- Fix Version/s: 1.0.0 Worker not recognize Driver state at standalone mode

[jira] [Updated] (SPARK-1471) Worker not recognize Driver state at standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1471: --- Fix Version/s: (was: 1.0.0) 1.0.1 Worker not recognize Driver state at

[jira] [Updated] (SPARK-1471) Worker not recognize Driver state at standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1471: --- Fix Version/s: 1.1.0 Worker not recognize Driver state at standalone mode

[jira] [Closed] (SPARK-1699) Python relative should be independence from the core, becomes subprojects

[ https://issues.apache.org/jira/browse/SPARK-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1699. -- Resolution: Unresolved Python relative should be independence from the core, becomes subprojects

[jira] [Updated] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1719: --- Fix Version/s: 1.1.0 spark.executor.extraLibraryPath isn't applied on yarn

[jira] [Created] (SPARK-2033) Automatically cleanup checkpoint

Guoqiang Li created SPARK-2033: -- Summary: Automatically cleanup checkpoint Key: SPARK-2033 URL: https://issues.apache.org/jira/browse/SPARK-2033 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-2033) Automatically cleanup checkpoint

[ https://issues.apache.org/jira/browse/SPARK-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018621#comment-14018621 ] Guoqiang Li commented on SPARK-2033: The PR: https://github.com/apache/spark/pull/855

[jira] [Created] (SPARK-2051) In yarn.ClientBase spark.yarn.dist.* do not work

Guoqiang Li created SPARK-2051: -- Summary: In yarn.ClientBase spark.yarn.dist.* do not work Key: SPARK-2051 URL: https://issues.apache.org/jira/browse/SPARK-2051 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2051) In yarn.ClientBase spark.yarn.dist.* do not work

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2051: --- Description: Spark configuration {{conf/spark-defaults.conf}}: {quote} spark.yarn.dist.archives

[jira] [Commented] (SPARK-2051) In yarn.ClientBase spark.yarn.dist.* do not work

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019560#comment-14019560 ] Guoqiang Li commented on SPARK-2051: The PR: https://github.com/apache/spark/pull/969

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node

[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016535#comment-14016535 ] Guoqiang Li commented on SPARK-2000: How to reproduce the issue? cannot connect to

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016600#comment-14016600 ] Guoqiang Li commented on SPARK-2000: I tested, have not found the problem. {{cat

[jira] [Updated] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2000: --- Fix Version/s: 1.1.0 1.0.1 cannot connect to cluster in Standalone mode when run

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016609#comment-14016609 ] Guoqiang Li commented on SPARK-2000: Ok, I have to reproduce the issue. cannot

[jira] [Commented] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14017322#comment-14017322 ] Guoqiang Li commented on SPARK-1930: [~pwendell] I'm not familiar with python, where

[jira] [Commented] (SPARK-1978) In some cases, spark-yarn does not automatically restart the failed container

[ https://issues.apache.org/jira/browse/SPARK-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014563#comment-14014563 ] Guoqiang Li commented on SPARK-1978: The PR: https://github.com/apache/spark/pull/921

[jira] [Updated] (SPARK-1951) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1951: --- Description: {{HADOOP_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --archives

[jira] [Created] (SPARK-1950) spark on yarn can't start

Guoqiang Li created SPARK-1950: -- Summary: spark on yarn can't start Key: SPARK-1950 URL: https://issues.apache.org/jira/browse/SPARK-1950 Project: Spark Issue Type: Bug Components:

[jira] [Closed] (SPARK-1929) DAGScheduler suspended by local task OOM

[ https://issues.apache.org/jira/browse/SPARK-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1929. -- Resolution: Duplicate DAGScheduler suspended by local task OOM

[jira] [Updated] (SPARK-1930) Container is running beyond physical memory limits

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Fix Version/s: 1.0.1 Container is running beyond physical memory limits

[jira] [Updated] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Summary: The Container is running beyond physical memory limits, so as to be killed. (was:

[jira] [Commented] (SPARK-1928) DAGScheduler suspended by local task OOM

[ https://issues.apache.org/jira/browse/SPARK-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008850#comment-14008850 ] Guoqiang Li commented on SPARK-1928: How to reproduce the issue? DAGScheduler

[jira] [Updated] (SPARK-1930) When the yarn containers occupies 8G memory ,the containers were killed

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Description: When the containers occupies 8G memory ,the containers were killed yarn node manager

[jira] [Updated] (SPARK-1930) Container memory beyond limit, were killed

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Summary: Container memory beyond limit, were killed (was: When the yarn containers occupies 8G

[jira] [Commented] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001450#comment-14001450 ] Guoqiang Li commented on SPARK-1875: ||hadoop or hive ||commons-lang|| |hive 0.12

[jira] [Comment Edited] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001450#comment-14001450 ] Guoqiang Li edited comment on SPARK-1875 at 5/19/14 7:26 AM: -

[jira] [Comment Edited] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001379#comment-14001379 ] Guoqiang Li edited comment on SPARK-1875 at 5/19/14 4:23 AM: -

[jira] [Commented] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001379#comment-14001379 ] Guoqiang Li commented on SPARK-1875: [~ pwendell], [~ matei] Do you have time to

[jira] [Created] (SPARK-1842) update scala-logging-slf4j to version 2.1.2

Guoqiang Li created SPARK-1842: -- Summary: update scala-logging-slf4j to version 2.1.2 Key: SPARK-1842 URL: https://issues.apache.org/jira/browse/SPARK-1842 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-1841) update scalatest to version 2.1.5

[ https://issues.apache.org/jira/browse/SPARK-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1841: --- Description: scalatest 1.9.* not support Scala 2.11 (was: scalatest 1.9.* not Scala 2.11) update

[jira] [Created] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

Guoqiang Li created SPARK-1760: -- Summary: mvn -Dsuites=* test throw an ClassNotFoundException Key: SPARK-1760 URL: https://issues.apache.org/jira/browse/SPARK-1760 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991556#comment-13991556 ] Guoqiang Li edited comment on SPARK-1712 at 5/14/14 6:03 AM: -

[jira] [Created] (SPARK-1756) Add missing description to spark-env.sh.template

Guoqiang Li created SPARK-1756: -- Summary: Add missing description to spark-env.sh.template Key: SPARK-1756 URL: https://issues.apache.org/jira/browse/SPARK-1756 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1644) The org.datanucleus:* should not be packaged into spark-assembly-*.jar

[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994131#comment-13994131 ] Guoqiang Li commented on SPARK-1644: Why {{assignee to me}} button can not find?

[jira] [Closed] (SPARK-1711) We should not use two build tools at the same time, should remove one of them

[ https://issues.apache.org/jira/browse/SPARK-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1711. -- Resolution: Duplicate We should not use two build tools at the same time, should remove one of them

[jira] [Commented] (SPARK-1756) Add missing description to spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992535#comment-13992535 ] Guoqiang Li commented on SPARK-1756: [PR 646|

[jira] [Commented] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992873#comment-13992873 ] Guoqiang Li commented on SPARK-1760: Hi, [~srowen] Is there a perfect solution? The [

[jira] [Commented] (SPARK-1779) Warning when spark.storage.memoryFraction is not between 0 and 1

[ https://issues.apache.org/jira/browse/SPARK-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993615#comment-13993615 ] Guoqiang Li commented on SPARK-1779: I think here should throw an exception. Warning

[jira] [Updated] (SPARK-571) Forbid return statements when cleaning closures

[ https://issues.apache.org/jira/browse/SPARK-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-571: -- Description: By mistake, I wrote some code like this: {code} object Foo { def main() { val sc = new

[jira] [Updated] (SPARK-1470) Use the scala-logging wrapper instead of the directly sfl4j api

[ https://issues.apache.org/jira/browse/SPARK-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1470: --- Fix Version/s: 1.1.0 Use the scala-logging wrapper instead of the directly sfl4j api

[jira] [Comment Edited] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992873#comment-13992873 ] Guoqiang Li edited comment on SPARK-1760 at 5/8/14 4:05 PM:

[jira] [Commented] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993505#comment-13993505 ] Guoqiang Li commented on SPARK-1760: Yes, [PR 712

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990527#comment-13990527 ] Guoqiang Li edited comment on SPARK-1712 at 5/7/14 5:10 AM:

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990527#comment-13990527 ] Guoqiang Li edited comment on SPARK-1712 at 5/7/14 5:10 AM:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991556#comment-13991556 ] Guoqiang Li commented on SPARK-1712: [The PR

[jira] [Updated] (SPARK-1734) spark-submit throws an exception: Exception in thread main java.lang.ClassNotFoundException: org.apache.spark.broadcast.TorrentBroadcastFactory

[ https://issues.apache.org/jira/browse/SPARK-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1734: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-1652 spark-submit throws an exception:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990373#comment-13990373 ] Guoqiang Li commented on SPARK-1712: May be caused by the imprope

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990390#comment-13990390 ] Guoqiang Li commented on SPARK-1712: Hi [~pwendell] What are your thoughts on this

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1712: --- Priority: Blocker (was: Major) ParallelCollectionRDD operations hanging forever without any error

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990388#comment-13990388 ] Guoqiang Li edited comment on SPARK-1712 at 5/6/14 7:46 AM:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990420#comment-13990420 ] Guoqiang Li commented on SPARK-1712: This is just a part of it, the whole file

[jira] [Commented] (SPARK-1734) spark-submit throws an exception: Exception in thread main java.lang.ClassNotFoundException: org.apache.spark.broadcast.TorrentBroadcastFactory

[ https://issues.apache.org/jira/browse/SPARK-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990468#comment-13990468 ] Guoqiang Li commented on SPARK-1734: There are spaces behind the

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990527#comment-13990527 ] Guoqiang Li commented on SPARK-1712: {{TaskDescription}} instance is too big causesthe

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1712: --- Description: conf/spark-defaults.conf {code} spark.akka.frameSize 5

<    1   2   3   4   5   >