[jira] [Updated] (SPARK-3469) All TaskCompletionListeners should be called even if some of them fail

2014-09-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3469: --- Summary: All TaskCompletionListeners should be called even if some of them fail (was: Make sure

[jira] [Updated] (SPARK-3469) All TaskCompletionListeners should be called even if some of them fail

2014-09-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3469: --- Component/s: Spark Core All TaskCompletionListeners should be called even if some of them fail

[jira] [Resolved] (SPARK-3282) It should support multiple receivers at one socketInputDStream

2014-09-10 Thread shenhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenhong resolved SPARK-3282. - Resolution: Won't Fix It should support multiple receivers at one socketInputDStream

[jira] [Created] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-10 Thread Shay Rojansky (JIRA)
Shay Rojansky created SPARK-3470: Summary: Have JavaSparkContext implement Closeable/AutoCloseable Key: SPARK-3470 URL: https://issues.apache.org/jira/browse/SPARK-3470 Project: Spark Issue

[jira] [Resolved] (SPARK-3395) {SQL] DSL uses incorrect attribute ids after a distinct()

2014-09-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3395. - Resolution: Fixed Fix Version/s: 1.2.0 {SQL] DSL uses incorrect attribute ids

[jira] [Commented] (SPARK-3469) All TaskCompletionListeners should be called even if some of them fail

2014-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128149#comment-14128149 ] Apache Spark commented on SPARK-3469: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-3471) Automatic resource manager for SparkContext in Scala?

2014-09-10 Thread Shay Rojansky (JIRA)
Shay Rojansky created SPARK-3471: Summary: Automatic resource manager for SparkContext in Scala? Key: SPARK-3471 URL: https://issues.apache.org/jira/browse/SPARK-3471 Project: Spark Issue

[jira] [Closed] (SPARK-2972) APPLICATION_COMPLETE not created in Python unless context explicitly stopped

2014-09-10 Thread Shay Rojansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Rojansky closed SPARK-2972. Resolution: Won't Fix APPLICATION_COMPLETE not created in Python unless context explicitly stopped

[jira] [Created] (SPARK-3472) Option to take top n elements (unsorted)

2014-09-10 Thread Kanwaljit Singh (JIRA)
Kanwaljit Singh created SPARK-3472: -- Summary: Option to take top n elements (unsorted) Key: SPARK-3472 URL: https://issues.apache.org/jira/browse/SPARK-3472 Project: Spark Issue Type: New

[jira] [Closed] (SPARK-3472) Option to take top n elements (unsorted)

2014-09-10 Thread Kanwaljit Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kanwaljit Singh closed SPARK-3472. -- Resolution: Invalid Option to take top n elements (unsorted)

[jira] [Resolved] (SPARK-3364) Zip equal-length but unequally-partition

2014-09-10 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li resolved SPARK-3364. Resolution: Fixed Zip equal-length but unequally-partition

[jira] [Resolved] (SPARK-3345) Do correct parameters for ShuffleFileGroup

2014-09-10 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li resolved SPARK-3345. Resolution: Fixed Fix Version/s: (was: 1.1.1) Do correct parameters for

[jira] [Resolved] (SPARK-3326) can't access a static variable after init in mapper

2014-09-10 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li resolved SPARK-3326. Resolution: Not a Problem can't access a static variable after init in mapper

[jira] [Created] (SPARK-3473) Expose task status when converting TaskInfo into JSON representation

2014-09-10 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3473: - Summary: Expose task status when converting TaskInfo into JSON representation Key: SPARK-3473 URL: https://issues.apache.org/jira/browse/SPARK-3473 Project: Spark

[jira] [Closed] (SPARK-3473) Expose task status when converting TaskInfo into JSON representation

2014-09-10 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta closed SPARK-3473. - Resolution: Won't Fix I can know task status from failed field and finishTime so I close this.

[jira] [Commented] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128232#comment-14128232 ] Sean Owen commented on SPARK-3470: -- If you implement {{AutoCloseable}}, then Spark will

[jira] [Commented] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-10 Thread Shay Rojansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128285#comment-14128285 ] Shay Rojansky commented on SPARK-3470: -- Good point about AutoCloseable. Yes, the idea

[jira] [Created] (SPARK-3474) Rename the env variable SPARK_MASTER_IP to SPARK_MASTER_HOST

2014-09-10 Thread Chunjun Xiao (JIRA)
Chunjun Xiao created SPARK-3474: --- Summary: Rename the env variable SPARK_MASTER_IP to SPARK_MASTER_HOST Key: SPARK-3474 URL: https://issues.apache.org/jira/browse/SPARK-3474 Project: Spark

[jira] [Commented] (SPARK-3474) Rename the env variable SPARK_MASTER_IP to SPARK_MASTER_HOST

2014-09-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128300#comment-14128300 ] Sean Owen commented on SPARK-3474: -- (You can deprecate but still support old variable

[jira] [Commented] (SPARK-3407) Add Date type support

2014-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128421#comment-14128421 ] Apache Spark commented on SPARK-3407: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

2014-09-10 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128451#comment-14128451 ] Cody Koeninger commented on SPARK-3462: --- Created a PR for feedback.

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

2014-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128478#comment-14128478 ] Apache Spark commented on SPARK-3462: - User 'koeninger' has created a pull request for

[jira] [Commented] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128552#comment-14128552 ] Apache Spark commented on SPARK-3470: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

2014-09-10 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128682#comment-14128682 ] Cody Koeninger commented on SPARK-3462: --- Tested this on a cluster against unions of

[jira] [Resolved] (SPARK-3286) Cannot view ApplicationMaster UI when Yarn’s url scheme is https

2014-09-10 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3286. -- Resolution: Fixed Fix Version/s: 1.2.0 Cannot view ApplicationMaster UI when Yarn’s url

[jira] [Created] (SPARK-3475) dev/merge_spark_pr.py fails on mac

2014-09-10 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3475: Summary: dev/merge_spark_pr.py fails on mac Key: SPARK-3475 URL: https://issues.apache.org/jira/browse/SPARK-3475 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-10 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128751#comment-14128751 ] Matthew Farrellee commented on SPARK-3470: -- while you can implement Closeable in

[jira] [Commented] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128764#comment-14128764 ] Sean Owen commented on SPARK-3470: -- Spark retains compatibility with Java 6 on purpose

[jira] [Commented] (SPARK-1484) MLlib should warn if you are using an iterative algorithm on non-cached data

2014-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128800#comment-14128800 ] Apache Spark commented on SPARK-1484: - User 'staple' has created a pull request for

[jira] [Commented] (SPARK-3478) Profile Python tasks stage by stage in worker

2014-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129503#comment-14129503 ] Apache Spark commented on SPARK-3478: - User 'davies' has created a pull request for

[jira] [Updated] (SPARK-3480) Throws out Not a valid command 'yarn-alpha/scalastyle' in dev/scalastyle for sbt build tool during 'Running Scala style checks'

2014-09-10 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-3480: --- Description: Symptom: Run ./dev/run-tests and dump outputs as following: SBT_MAVEN_PROFILES_ARGS=-Pyarn

[jira] [Resolved] (SPARK-3447) Kryo NPE when serializing JListWrapper

2014-09-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3447. - Resolution: Fixed Fix Version/s: 1.2.0 Kryo NPE when serializing JListWrapper