[jira] [Commented] (SPARK-1659) improvements spark-submit usage

2014-04-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983197#comment-13983197 ] Guoqiang Li commented on SPARK-1659: ./bin/spark-submit

[jira] [Assigned] (SPARK-1644) hql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) throw an exception

2014-04-29 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li reassigned SPARK-1644: -- Assignee: Guoqiang Li hql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) throw an

[jira] [Assigned] (SPARK-1629) Spark should inline use of commons-lang `SystemUtils.IS_OS_WINDOWS`

2014-04-29 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li reassigned SPARK-1629: -- Assignee: Guoqiang Li Spark should inline use of commons-lang `SystemUtils.IS_OS_WINDOWS`

[jira] [Commented] (SPARK-1644) hql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) throw an exception

2014-04-29 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984114#comment-13984114 ] Guoqiang Li commented on SPARK-1644: {code} # When Hive support is needed, Datanucleus

[jira] [Updated] (SPARK-1644) The org.datanucleus:* should not be packaged into spark-assembly-*.jar

2014-04-30 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1644: --- Summary: The org.datanucleus:* should not be packaged into spark-assembly-*.jar (was: hql(CREATE

[jira] [Commented] (SPARK-1629) Spark should inline use of commons-lang `SystemUtils.IS_OS_WINDOWS`

2014-04-30 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985352#comment-13985352 ] Guoqiang Li commented on SPARK-1629: [The PR

[jira] [Created] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when built for hadoop 2.3.0 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1693: -- Summary: Most of the tests throw a java.lang.SecurityException when built for hadoop 2.3.0 2.4.0 Key: SPARK-1693 URL: https://issues.apache.org/jira/browse/SPARK-1693

[jira] [Updated] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1693: --- Attachment: log.txt Most of the tests throw a java.lang.SecurityException when spark built for

[jira] [Updated] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1693: --- Summary: Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 ,

[jira] [Assigned] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li reassigned SPARK-1693: -- Assignee: Guoqiang Li Most of the tests throw a java.lang.SecurityException when spark built

[jira] [Commented] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986492#comment-13986492 ] Guoqiang Li commented on SPARK-1693: Hi, Sean Owen The simplest solution is to remove

[jira] [Commented] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986504#comment-13986504 ] Guoqiang Li commented on SPARK-1693: The dependency path is as follows

[jira] [Comment Edited] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986504#comment-13986504 ] Guoqiang Li edited comment on SPARK-1693 at 5/1/14 11:20 AM: -

[jira] [Commented] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986518#comment-13986518 ] Guoqiang Li commented on SPARK-1693: Servlet 3.0, 2.5 together on the classpath,Is

[jira] [Commented] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986607#comment-13986607 ] Guoqiang Li commented on SPARK-1693: Hi Sean Owen You have time to review [the

[jira] [Commented] (SPARK-1693) Most of the tests throw a java.lang.SecurityException when spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986636#comment-13986636 ] Guoqiang Li commented on SPARK-1693: I didn't find servlet-api 3.0.0 stable version

[jira] [Updated] (SPARK-1693) Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when Spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1693: --- Summary: Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when

[jira] [Commented] (SPARK-1693) Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when Spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986656#comment-13986656 ] Guoqiang Li commented on SPARK-1693: Thanks Sean Owen . Code has been submitted to [PR

[jira] [Created] (SPARK-1695) java8-tests compiler error: package com.google.common.collections does not exist

2014-05-01 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1695: -- Summary: java8-tests compiler error: package com.google.common.collections does not exist Key: SPARK-1695 URL: https://issues.apache.org/jira/browse/SPARK-1695 Project:

[jira] [Commented] (SPARK-1693) Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when Spark built for hadoop 2.3.0 , 2.4.0

2014-05-01 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987323#comment-13987323 ] Guoqiang Li commented on SPARK-1693: Now, the main problem is to ensure that hadoop

[jira] [Created] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1698: -- Summary: Improve spark integration Key: SPARK-1698 URL: https://issues.apache.org/jira/browse/SPARK-1698 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987682#comment-13987682 ] Guoqiang Li commented on SPARK-1698: [~srowen] About

[jira] [Issue Comment Deleted] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1698: --- Comment: was deleted (was: [The PR | https://github.com/apache/spark/pull/598]) Improve spark

[jira] [Commented] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987699#comment-13987699 ] Guoqiang Li commented on SPARK-1698: [~srowen] In [The PR

[jira] [Commented] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987708#comment-13987708 ] Guoqiang Li commented on SPARK-1698: [~srowen] In [The PR

[jira] [Updated] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1698: --- Description: Use the shade plugin to create a big JAR with all the dependencies can cause a few

[jira] [Comment Edited] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987699#comment-13987699 ] Guoqiang Li edited comment on SPARK-1698 at 5/2/14 2:32 PM:

[jira] [Comment Edited] (SPARK-1698) Improve spark integration

2014-05-02 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987699#comment-13987699 ] Guoqiang Li edited comment on SPARK-1698 at 5/2/14 2:36 PM:

[jira] [Commented] (SPARK-1699) Python relative should be independence from the core, becomes subprojects

2014-05-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988596#comment-13988596 ] Guoqiang Li commented on SPARK-1699: [The PR 624

[jira] [Commented] (SPARK-1693) Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when Spark built for hadoop 2.3.0 , 2.4.0

2014-05-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988618#comment-13988618 ] Guoqiang Li commented on SPARK-1693: [~srowen] You're right,preferably only one

[jira] [Updated] (SPARK-1698) Improve spark integration

2014-05-04 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1698: --- Fix Version/s: (was: 1.0.0) 1.1.0 Improve spark integration

[jira] [Updated] (SPARK-1644) The org.datanucleus:* should not be packaged into spark-assembly-*.jar

2014-05-04 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1644: --- Fix Version/s: 1.1.0 The org.datanucleus:* should not be packaged into spark-assembly-*.jar

[jira] [Created] (SPARK-1711) We should not use two build tools at the same time, sbt or maven should be removed.

2014-05-04 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1711: -- Summary: We should not use two build tools at the same time, sbt or maven should be removed. Key: SPARK-1711 URL: https://issues.apache.org/jira/browse/SPARK-1711

[jira] [Updated] (SPARK-1711) We should not use two build tools at the same time, should remove one of them

2014-05-04 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1711: --- Summary: We should not use two build tools at the same time, should remove one of them (was: We

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989696#comment-13989696 ] Guoqiang Li commented on SPARK-1712: Unable to reproduce the bug.Can you upload the

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990200#comment-13990200 ] Guoqiang Li commented on SPARK-1712: Add conf/log4j.properties file {code} # Set

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990200#comment-13990200 ] Guoqiang Li edited comment on SPARK-1712 at 5/6/14 2:04 AM:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990206#comment-13990206 ] Guoqiang Li commented on SPARK-1712: What is the value of {{spark.akka.frameSize}}?

[jira] [Updated] (SPARK-1734) spark-submit throws an exception: Exception in thread main java.lang.ClassNotFoundException: org.apache.spark.broadcast.TorrentBroadcastFactory

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1734: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-1652 spark-submit throws an exception:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990373#comment-13990373 ] Guoqiang Li commented on SPARK-1712: May be caused by the imprope

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990390#comment-13990390 ] Guoqiang Li commented on SPARK-1712: Hi [~pwendell] What are your thoughts on this

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1712: --- Priority: Blocker (was: Major) ParallelCollectionRDD operations hanging forever without any error

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990388#comment-13990388 ] Guoqiang Li edited comment on SPARK-1712 at 5/6/14 7:46 AM:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990420#comment-13990420 ] Guoqiang Li commented on SPARK-1712: This is just a part of it, the whole file

[jira] [Commented] (SPARK-1734) spark-submit throws an exception: Exception in thread main java.lang.ClassNotFoundException: org.apache.spark.broadcast.TorrentBroadcastFactory

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990468#comment-13990468 ] Guoqiang Li commented on SPARK-1734: There are spaces behind the

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990527#comment-13990527 ] Guoqiang Li commented on SPARK-1712: {{TaskDescription}} instance is too big causesthe

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1712: --- Description: conf/spark-defaults.conf {code} spark.akka.frameSize 5

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990769#comment-13990769 ] Guoqiang Li commented on SPARK-1712: Thank you, temporarily need not, I roughly to

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-07 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990527#comment-13990527 ] Guoqiang Li edited comment on SPARK-1712 at 5/7/14 5:10 AM:

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-07 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990527#comment-13990527 ] Guoqiang Li edited comment on SPARK-1712 at 5/7/14 5:10 AM:

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-07 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991556#comment-13991556 ] Guoqiang Li commented on SPARK-1712: [The PR

[jira] [Commented] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

2014-05-10 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993505#comment-13993505 ] Guoqiang Li commented on SPARK-1760: Yes, [PR 712

[jira] [Updated] (SPARK-571) Forbid return statements when cleaning closures

2014-05-11 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-571: -- Description: By mistake, I wrote some code like this: {code} object Foo { def main() { val sc = new

[jira] [Updated] (SPARK-1470) Use the scala-logging wrapper instead of the directly sfl4j api

2014-05-11 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1470: --- Fix Version/s: 1.1.0 Use the scala-logging wrapper instead of the directly sfl4j api

[jira] [Comment Edited] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

2014-05-11 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992873#comment-13992873 ] Guoqiang Li edited comment on SPARK-1760 at 5/8/14 4:05 PM:

[jira] [Commented] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

2014-05-12 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992873#comment-13992873 ] Guoqiang Li commented on SPARK-1760: Hi, [~srowen] Is there a perfect solution? The [

[jira] [Commented] (SPARK-1779) Warning when spark.storage.memoryFraction is not between 0 and 1

2014-05-12 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993615#comment-13993615 ] Guoqiang Li commented on SPARK-1779: I think here should throw an exception. Warning

[jira] [Commented] (SPARK-1756) Add missing description to spark-env.sh.template

2014-05-13 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992535#comment-13992535 ] Guoqiang Li commented on SPARK-1756: [PR 646|

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991556#comment-13991556 ] Guoqiang Li edited comment on SPARK-1712 at 5/14/14 6:03 AM: -

[jira] [Created] (SPARK-1756) Add missing description to spark-env.sh.template

2014-05-14 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1756: -- Summary: Add missing description to spark-env.sh.template Key: SPARK-1756 URL: https://issues.apache.org/jira/browse/SPARK-1756 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1644) The org.datanucleus:* should not be packaged into spark-assembly-*.jar

2014-05-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994131#comment-13994131 ] Guoqiang Li commented on SPARK-1644: Why {{assignee to me}} button can not find?

[jira] [Closed] (SPARK-1711) We should not use two build tools at the same time, should remove one of them

2014-05-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1711. -- Resolution: Duplicate We should not use two build tools at the same time, should remove one of them

[jira] [Created] (SPARK-1842) update scala-logging-slf4j to version 2.1.2

2014-05-15 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1842: -- Summary: update scala-logging-slf4j to version 2.1.2 Key: SPARK-1842 URL: https://issues.apache.org/jira/browse/SPARK-1842 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-1841) update scalatest to version 2.1.5

2014-05-15 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1841: --- Description: scalatest 1.9.* not support Scala 2.11 (was: scalatest 1.9.* not Scala 2.11) update

[jira] [Created] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

2014-05-15 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1760: -- Summary: mvn -Dsuites=* test throw an ClassNotFoundException Key: SPARK-1760 URL: https://issues.apache.org/jira/browse/SPARK-1760 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

2014-05-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001379#comment-14001379 ] Guoqiang Li edited comment on SPARK-1875 at 5/19/14 4:23 AM: -

[jira] [Commented] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

2014-05-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001379#comment-14001379 ] Guoqiang Li commented on SPARK-1875: [~ pwendell], [~ matei] Do you have time to

[jira] [Commented] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

2014-05-19 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001450#comment-14001450 ] Guoqiang Li commented on SPARK-1875: ||hadoop or hive ||commons-lang|| |hive 0.12

[jira] [Comment Edited] (SPARK-1875) NoClassDefFoundError: StringUtils when building against Hadoop 1

2014-05-19 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001450#comment-14001450 ] Guoqiang Li edited comment on SPARK-1875 at 5/19/14 7:26 AM: -

[jira] [Commented] (SPARK-1928) DAGScheduler suspended by local task OOM

2014-05-26 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008850#comment-14008850 ] Guoqiang Li commented on SPARK-1928: How to reproduce the issue? DAGScheduler

[jira] [Updated] (SPARK-1930) When the yarn containers occupies 8G memory ,the containers were killed

2014-05-26 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Description: When the containers occupies 8G memory ,the containers were killed yarn node manager

[jira] [Updated] (SPARK-1930) Container memory beyond limit, were killed

2014-05-26 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Summary: Container memory beyond limit, were killed (was: When the yarn containers occupies 8G

[jira] [Closed] (SPARK-1929) DAGScheduler suspended by local task OOM

2014-05-27 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1929. -- Resolution: Duplicate DAGScheduler suspended by local task OOM

[jira] [Updated] (SPARK-1930) Container is running beyond physical memory limits

2014-05-27 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Fix Version/s: 1.0.1 Container is running beyond physical memory limits

[jira] [Updated] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

2014-05-27 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1930: --- Summary: The Container is running beyond physical memory limits, so as to be killed. (was:

[jira] [Updated] (SPARK-1951) spark on yarn can't start

2014-05-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1951: --- Description: {{HADOOP_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --archives

[jira] [Created] (SPARK-1950) spark on yarn can't start

2014-05-28 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-1950: -- Summary: spark on yarn can't start Key: SPARK-1950 URL: https://issues.apache.org/jira/browse/SPARK-1950 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-1978) In some cases, spark-yarn does not automatically restart the failed container

2014-05-31 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014563#comment-14014563 ] Guoqiang Li commented on SPARK-1978: The PR: https://github.com/apache/spark/pull/921

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016535#comment-14016535 ] Guoqiang Li commented on SPARK-2000: How to reproduce the issue? cannot connect to

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016600#comment-14016600 ] Guoqiang Li commented on SPARK-2000: I tested, have not found the problem. {{cat

[jira] [Updated] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2000: --- Fix Version/s: 1.1.0 1.0.1 cannot connect to cluster in Standalone mode when run

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016609#comment-14016609 ] Guoqiang Li commented on SPARK-2000: Ok, I have to reproduce the issue. cannot

[jira] [Commented] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

2014-06-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14017322#comment-14017322 ] Guoqiang Li commented on SPARK-1930: [~pwendell] I'm not familiar with python, where

[jira] [Created] (SPARK-2033) Automatically cleanup checkpoint

2014-06-05 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2033: -- Summary: Automatically cleanup checkpoint Key: SPARK-2033 URL: https://issues.apache.org/jira/browse/SPARK-2033 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-2033) Automatically cleanup checkpoint

2014-06-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018621#comment-14018621 ] Guoqiang Li commented on SPARK-2033: The PR: https://github.com/apache/spark/pull/855

[jira] [Created] (SPARK-2051) In yarn.ClientBase spark.yarn.dist.* do not work

2014-06-05 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2051: -- Summary: In yarn.ClientBase spark.yarn.dist.* do not work Key: SPARK-2051 URL: https://issues.apache.org/jira/browse/SPARK-2051 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2051) In yarn.ClientBase spark.yarn.dist.* do not work

2014-06-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2051: --- Description: Spark configuration {{conf/spark-defaults.conf}}: {quote} spark.yarn.dist.archives

[jira] [Commented] (SPARK-2051) In yarn.ClientBase spark.yarn.dist.* do not work

2014-06-05 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019560#comment-14019560 ] Guoqiang Li commented on SPARK-2051: The PR: https://github.com/apache/spark/pull/969

[jira] [Updated] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

2014-06-09 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1719: --- Fix Version/s: 1.1.0 spark.executor.extraLibraryPath isn't applied on yarn

[jira] [Closed] (SPARK-1699) Python relative should be independence from the core, becomes subprojects

2014-06-11 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1699. -- Resolution: Unresolved Python relative should be independence from the core, becomes subprojects

[jira] [Resolved] (SPARK-1471) Worker not recognize Driver state at standalone mode

2014-06-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li resolved SPARK-1471. Resolution: Fixed Worker not recognize Driver state at standalone mode

[jira] [Updated] (SPARK-1471) Worker not recognize Driver state at standalone mode

2014-06-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1471: --- Fix Version/s: 1.0.0 Worker not recognize Driver state at standalone mode

[jira] [Updated] (SPARK-1471) Worker not recognize Driver state at standalone mode

2014-06-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1471: --- Fix Version/s: (was: 1.0.0) 1.0.1 Worker not recognize Driver state at

[jira] [Updated] (SPARK-1471) Worker not recognize Driver state at standalone mode

2014-06-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1471: --- Fix Version/s: 1.1.0 Worker not recognize Driver state at standalone mode

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

2014-06-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035467#comment-14035467 ] Guoqiang Li commented on SPARK-1291: PR: https://github.com/apache/spark/pull/1112

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

2014-06-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035522#comment-14035522 ] Guoqiang Li commented on SPARK-1291: [~rahulsinghal.iitd] Thank you, looking forward

[jira] [Commented] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

2014-06-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035795#comment-14035795 ] Guoqiang Li commented on SPARK-2156: The problem here:

[jira] [Updated] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

2014-06-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2156: --- Affects Version/s: 0.9.1 When the size of serialized results for one partition is slightly smaller

[jira] [Commented] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

2014-06-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036838#comment-14036838 ] Guoqiang Li commented on SPARK-2156: [~pwend...@gmail.com] When the size of

[jira] [Issue Comment Deleted] (SPARK-2156) When the size of serialized results for one partition is slightly smaller than 10MB (the default akka.frameSize), the execution blocks

2014-06-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2156: --- Comment: was deleted (was: [~pwend...@gmail.com]) When the size of serialized results for one

  1   2   3   4   5   >