[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263516#comment-14263516 ] Cheng Lian commented on SPARK-1529: --- Hi [~srowen], first of all we are not trying to put

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-03 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263458#comment-14263458 ] Chip Senkbeil edited comment on SPARK-4923 at 1/3/15 6:01 PM: --

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-03 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263458#comment-14263458 ] Chip Senkbeil edited comment on SPARK-4923 at 1/3/15 5:35 PM: --

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-03 Thread Alexander Bezzubov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263539#comment-14263539 ] Alexander Bezzubov commented on SPARK-4923: --- Just FYI, same for us here

[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263578#comment-14263578 ] Sean Owen commented on SPARK-1529: -- Hm, how do these APIs preclude the direct use of

[jira] [Commented] (SPARK-5059) list of user's objects in Spark REPL

2015-01-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263483#comment-14263483 ] Sean Owen commented on SPARK-5059: -- It makes some sense although the Spark shell is

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-03 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263458#comment-14263458 ] Chip Senkbeil edited comment on SPARK-4923 at 1/3/15 6:09 PM: --

[jira] [Commented] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

2015-01-03 Thread Elmer Garduno (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263617#comment-14263617 ] Elmer Garduno commented on SPARK-5052: -- The problem seem to be fixed when using

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-03 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263458#comment-14263458 ] Chip Senkbeil edited comment on SPARK-4923 at 1/3/15 6:02 PM: --

[jira] [Resolved] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

2015-01-03 Thread Elmer Garduno (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elmer Garduno resolved SPARK-5052. -- Resolution: Not a Problem com.google.common.base.Optional binary has a wrong method signatures

[jira] [Resolved] (SPARK-609) Add instructions for enabling Akka debug logging

2015-01-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-609. -- Resolution: Incomplete I'm going to close out this issue for now, since I think it's no longer an issue

[jira] [Commented] (SPARK-5059) list of user's objects in Spark REPL

2015-01-03 Thread Tomas Hudik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263620#comment-14263620 ] Tomas Hudik commented on SPARK-5059: thanks Sean. Actually, after some googling I

[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-03 Thread Kannan Rajah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263603#comment-14263603 ] Kannan Rajah commented on SPARK-1529: - It cannot preclude the use of java.io

[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-03 Thread Kannan Rajah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263600#comment-14263600 ] Kannan Rajah commented on SPARK-1529: - [~lian cheng] Can you upload this prototype

[jira] [Closed] (SPARK-5059) list of user's objects in Spark REPL

2015-01-03 Thread Tomas Hudik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomas Hudik closed SPARK-5059. -- Resolution: Not a Problem If you press _TAB_ button In Spark Shell (Scala REPL) you'll get a list of

[jira] [Resolved] (SPARK-988) Write PySpark profiling guide

2015-01-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-988. -- Resolution: Fixed We added distributed Python profiling support in 1.2 (see SPARK-3478). Write PySpark

[jira] [Resolved] (SPARK-2084) Mention SPARK_JAR in env var section on configuration page

2015-01-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2084. --- Resolution: Won't Fix I'm resolving this as Won't Fix since the SPARK_JAR environment variable was

[jira] [Comment Edited] (SPARK-5023) In Web UI job history, the total job duration is incorrect (much smaller than the sum of its stages)

2015-01-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263664#comment-14263664 ] Josh Rosen edited comment on SPARK-5023 at 1/3/15 10:57 PM:

[jira] [Commented] (SPARK-5023) In Web UI job history, the total job duration is incorrect (much smaller than the sum of its stages)

2015-01-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263664#comment-14263664 ] Josh Rosen commented on SPARK-5023: --- This sounds superficially similar to SPARK-4836;

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-03 Thread Peng Cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263681#comment-14263681 ] Peng Cheng commented on SPARK-4923: --- You are right, in fact 'Dev's API' simply means

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263694#comment-14263694 ] Saisai Shao commented on SPARK-5001: Hi [~hanhg], I don't think it is a problem of

[jira] [Reopened] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

2015-01-03 Thread Elmer Garduno (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elmer Garduno reopened SPARK-5052: -- False alarm, still getting the same errors when running on the standalone cluster.

[jira] [Created] (SPARK-5064) GraphX rmatGraph hangs

2015-01-03 Thread Michael Malak (JIRA)
Michael Malak created SPARK-5064: Summary: GraphX rmatGraph hangs Key: SPARK-5064 URL: https://issues.apache.org/jira/browse/SPARK-5064 Project: Spark Issue Type: Bug Components:

[jira] [Issue Comment Deleted] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread hanhonggen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hanhonggen updated SPARK-5001: -- Comment: was deleted (was: I think it's impossible to make sure that all jobs generated in spark

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread hanhonggen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263701#comment-14263701 ] hanhonggen commented on SPARK-5001: --- I think it's impossible to make sure that all jobs

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread hanhonggen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263702#comment-14263702 ] hanhonggen commented on SPARK-5001: --- I think it's impossible to make sure that all jobs

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread hanhonggen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263699#comment-14263699 ] hanhonggen commented on SPARK-5001: --- I think it's impossible to make sure that all jobs

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread hanhonggen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263700#comment-14263700 ] hanhonggen commented on SPARK-5001: --- I think it's impossible to make sure that all jobs

[jira] [Issue Comment Deleted] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread hanhonggen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hanhonggen updated SPARK-5001: -- Comment: was deleted (was: I think it's impossible to make sure that all jobs generated in spark

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

2015-01-03 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263704#comment-14263704 ] Saisai Shao commented on SPARK-5001: Yeah, It's Spark Streaming design model's

[jira] [Created] (SPARK-5065) BroadCast can still work after sc had been stopped.

2015-01-03 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-5065: --- Summary: BroadCast can still work after sc had been stopped. Key: SPARK-5065 URL: https://issues.apache.org/jira/browse/SPARK-5065 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5065) BroadCast can still work after sc had been stopped.

2015-01-03 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-5065: Description: Code as follow: {code:borderStyle=solid} val sc1 = new SparkContext val sc2 = new

[jira] [Commented] (SPARK-5065) BroadCast can still work after sc had been stopped.

2015-01-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263729#comment-14263729 ] Apache Spark commented on SPARK-5065: - User 'SaintBacchus' has created a pull request

[jira] [Created] (SPARK-5066) Can not get all value that has the same key when reading key ordered from different Streaming.

2015-01-03 Thread DoingDone9 (JIRA)
DoingDone9 created SPARK-5066: - Summary: Can not get all value that has the same key when reading key ordered from different Streaming. Key: SPARK-5066 URL: https://issues.apache.org/jira/browse/SPARK-5066

[jira] [Updated] (SPARK-5066) Can not get all key when reading key ordered from different Streaming.

2015-01-03 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-5066: -- Summary: Can not get all key when reading key ordered from different Streaming. (was: Can not get

[jira] [Updated] (SPARK-5066) Can not get all key that has same hashcode when reading key ordered from different Streaming.

2015-01-03 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-5066: -- Summary: Can not get all key that has same hashcode when reading key ordered from different

[jira] [Updated] (SPARK-5066) Can not get all key that has same hashcode when reading key ordered from different Streaming.

2015-01-03 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-5066: -- Description: when spill is open, data ordered by hashCode will be spilled to disk. We need get all key

[jira] [Updated] (SPARK-5066) Can not get all key that has same hashcode when reading key ordered from different Streaming.

2015-01-03 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-5066: -- Affects Version/s: 1.2.0 Can not get all key that has same hashcode when reading key ordered from

[jira] [Created] (SPARK-5067) testTaskInfo doesn't compare SparkListenerApplicationStart.appId

2015-01-03 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-5067: --- Summary: testTaskInfo doesn't compare SparkListenerApplicationStart.appId Key: SPARK-5067 URL: https://issues.apache.org/jira/browse/SPARK-5067 Project: Spark

[jira] [Resolved] (SPARK-5058) Typos and broken URL

2015-01-03 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-5058. -- Resolution: Fixed Fix Version/s: 1.3.0 Typos and broken URL

[jira] [Updated] (SPARK-5067) testTaskInfo doesn't compare SparkListenerApplicationStart.appId

2015-01-03 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5067: Component/s: Spark Core testTaskInfo doesn't compare SparkListenerApplicationStart.appId

[jira] [Commented] (SPARK-5067) testTaskInfo doesn't compare SparkListenerApplicationStart.appId

2015-01-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263748#comment-14263748 ] Apache Spark commented on SPARK-5067: - User 'zsxwing' has created a pull request for

[jira] [Commented] (SPARK-3306) Addition of external resource dependency in executors

2015-01-03 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263761#comment-14263761 ] koert kuipers commented on SPARK-3306: -- i am also interested in resources that can be

[jira] [Created] (SPARK-5068) When the path not found in the hdfs,we can't get the result

2015-01-03 Thread jeanlyn (JIRA)
jeanlyn created SPARK-5068: -- Summary: When the path not found in the hdfs,we can't get the result Key: SPARK-5068 URL: https://issues.apache.org/jira/browse/SPARK-5068 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5068) When the path not found in the hdfs,we can't get the result

2015-01-03 Thread jeanlyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jeanlyn updated SPARK-5068: --- Description: when the partion path was found in the metastore but not found in the hdfs,it will casue some

[jira] [Updated] (SPARK-5068) When the path not found in the hdfs,we can't get the result

2015-01-03 Thread jeanlyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jeanlyn updated SPARK-5068: --- Description: when the partion path was found in the metastore but not found in the hdfs,it will casue some