[jira] [Created] (SPARK-12710) Create local CoGroup operator

2016-01-07 Thread Mao, Wei (JIRA)
Mao, Wei created SPARK-12710: Summary: Create local CoGroup operator Key: SPARK-12710 URL: https://issues.apache.org/jira/browse/SPARK-12710 Project: Spark Issue Type: Sub-task Componen

[jira] [Created] (SPARK-12709) Create local Except operator

2016-01-07 Thread Mao, Wei (JIRA)
Mao, Wei created SPARK-12709: Summary: Create local Except operator Key: SPARK-12709 URL: https://issues.apache.org/jira/browse/SPARK-12709 Project: Spark Issue Type: Sub-task Component

[jira] [Created] (SPARK-12708) Sorting task error in Stages Page when yarn mode

2016-01-07 Thread Koyo Yoshida (JIRA)
Koyo Yoshida created SPARK-12708: Summary: Sorting task error in Stages Page when yarn mode Key: SPARK-12708 URL: https://issues.apache.org/jira/browse/SPARK-12708 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-12687) Support from clause surrounded by `()`

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12687: Assignee: Apache Spark > Support from clause surrounded by `()` >

[jira] [Assigned] (SPARK-12687) Support from clause surrounded by `()`

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12687: Assignee: (was: Apache Spark) > Support from clause surrounded by `()` > -

[jira] [Commented] (SPARK-12687) Support from clause surrounded by `()`

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088836#comment-15088836 ] Apache Spark commented on SPARK-12687: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088822#comment-15088822 ] Xiao Li commented on SPARK-12705: - Could you explain how to reproduce it? In sqlquerysu

[jira] [Commented] (SPARK-5569) Checkpoints cannot reference classes defined outside of Spark's assembly

2016-01-07 Thread David Winters (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088794#comment-15088794 ] David Winters commented on SPARK-5569: -- I have encountered this issue also. I have a

[jira] [Resolved] (SPARK-5487) Dockerfile to build spark's custom akka.

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-5487. --- Resolution: Won't Fix Resolving as "Won't Fix" for now, since this is no longer needed now that SPARK

[jira] [Commented] (SPARK-4628) Put external projects and examples behind a build flag

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088784#comment-15088784 ] Apache Spark commented on SPARK-4628: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-4628) Put external projects and examples behind a build flag

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-4628: - Assignee: Josh Rosen > Put external projects and examples behind a build flag > -

[jira] [Assigned] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12707: Assignee: Apache Spark > Remove submit python/R scripts through pyspark/sparkR > -

[jira] [Assigned] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12707: Assignee: (was: Apache Spark) > Remove submit python/R scripts through pyspark/sparkR

[jira] [Commented] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088757#comment-15088757 ] Apache Spark commented on SPARK-12707: -- User 'zjffdu' has created a pull request for

[jira] [Created] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-12707: -- Summary: Remove submit python/R scripts through pyspark/sparkR Key: SPARK-12707 URL: https://issues.apache.org/jira/browse/SPARK-12707 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8540) KMeans-based outlier detection

2016-01-07 Thread Rakesh Chalasani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088746#comment-15088746 ] Rakesh Chalasani commented on SPARK-8540: - I see that this hasn't moved forward, s

[jira] [Updated] (SPARK-12693) OffsetOutOfRangeException caused by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rado Buransky updated SPARK-12693: -- Summary: OffsetOutOfRangeException caused by retention (was: OffsetOutOfRangeException cause b

[jira] [Updated] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12706: --- Assignee: (was: Davies Liu) > support grouping/grouping_id function together group set >

[jira] [Commented] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088734#comment-15088734 ] Rado Buransky commented on SPARK-12693: --- Ok. Let's forget about short time periods.

[jira] [Updated] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12706: --- Description: https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation,+Cube,+Grouping+an

[jira] [Updated] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12706: --- Summary: support grouping/grouping_id function together group set (was: support grouping function to

[jira] [Commented] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088725#comment-15088725 ] Cody Koeninger commented on SPARK-12693: What is your actual use case for changin

[jira] [Resolved] (SPARK-12317) Support configurate value for AUTO_BROADCASTJOIN_THRESHOLD and SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE with unit(e.g. kb/mb/gb) in SQLConf

2016-01-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12317. - Resolution: Fixed Assignee: kevin yu Fix Version/s: 2.0.0 > Support configurate v

[jira] [Assigned] (SPARK-12706) support grouping function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12706: -- Assignee: Davies Liu > support grouping function together group set >

[jira] [Created] (SPARK-12706) support grouping function together group set

2016-01-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12706: -- Summary: support grouping function together group set Key: SPARK-12706 URL: https://issues.apache.org/jira/browse/SPARK-12706 Project: Spark Issue Type: New Feat

[jira] [Created] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12705: -- Summary: Sorting column can't be resolved if it's not in projection Key: SPARK-12705 URL: https://issues.apache.org/jira/browse/SPARK-12705 Project: Spark Issue

[jira] [Comment Edited] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088704#comment-15088704 ] Rado Buransky edited comment on SPARK-12693 at 1/8/16 4:50 AM:

[jira] [Commented] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088704#comment-15088704 ] Rado Buransky commented on SPARK-12693: --- I wasn't actually right. This issue is not

[jira] [Resolved] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-07 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-12591. --- Resolution: Fixed Fix Version/s: 2.0.0 > NullPointerException using checkpointed mapWi

[jira] [Updated] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-07 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-12591: -- Assignee: Shixiong Zhu > NullPointerException using checkpointed mapWithState with KryoSerializ

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088650#comment-15088650 ] Wenchen Fan commented on SPARK-12704: - yea, sub-optimal is more proper, and I'm ok to

[jira] [Updated] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12704: Description: The implementation of {{HashPartitioning.compatibleWith}} has been sub-optimal for a

[jira] [Updated] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12704: Description: The implementation of {{HashPartitioning.compatibleWith}} has been wrong for a while.

[jira] [Updated] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-12704: - Issue Type: Improvement (was: Bug) > we may repartition a relation even it's not needed

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088639#comment-15088639 ] Yin Huai commented on SPARK-12704: -- [~cloud_fan] Thank you for bringing it up. Right now

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088638#comment-15088638 ] Michael Armbrust commented on SPARK-12704: -- I think this explanation might be cl

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088635#comment-15088635 ] Wenchen Fan commented on SPARK-12704: - cc [~joshrosen] [~nongli] [~marmbrus] [~yhuai]

[jira] [Created] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-12704: --- Summary: we may repartition a relation even it's not needed Key: SPARK-12704 URL: https://issues.apache.org/jira/browse/SPARK-12704 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088633#comment-15088633 ] Apache Spark commented on SPARK-12591: -- User 'zsxwing' has created a pull request fo

[jira] [Resolved] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2016-01-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-4066. --- Resolution: Later Haven't heard of much feedback so far. Resolving for now. > Make whether maven builds fail

[jira] [Resolved] (SPARK-4438) Add HistoryServer RESTful API

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4438. --- Resolution: Fixed Fix Version/s: 1.4.0 [~jonathak], thanks for pointing that out. I'm going to

[jira] [Commented] (SPARK-4438) Add HistoryServer RESTful API

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088563#comment-15088563 ] Josh Rosen commented on SPARK-4438: --- Also, note that the canonical documentation for thi

[jira] [Commented] (SPARK-4991) Worker should reconnect to Master when Master actor restart

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088560#comment-15088560 ] Josh Rosen commented on SPARK-4991: --- Is this still relevant after we remove the Akka RPC

[jira] [Resolved] (SPARK-12507) Expose closeFileAfterWrite and allowBatching configurations for Streaming

2016-01-07 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-12507. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Expose closeFileAfte

[jira] [Assigned] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12639: Assignee: Apache Spark > Improve Explain for DataSources with Handled Predicate Pushdowns

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Russell Alexander Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088558#comment-15088558 ] Russell Alexander Spitzer commented on SPARK-12639: --- https://github.com

[jira] [Resolved] (SPARK-2690) Make unidoc part of our test process

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2690. --- Resolution: Duplicate > Make unidoc part of our test process > >

[jira] [Assigned] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12639: Assignee: (was: Apache Spark) > Improve Explain for DataSources with Handled Predicate

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088557#comment-15088557 ] Apache Spark commented on SPARK-12639: -- User 'RussellSpitzer' has created a pull req

[jira] [Resolved] (SPARK-2690) Make unidoc part of our test process

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2690. --- Resolution: Fixed This is a duplicate of SPARK-7019 > Make unidoc part of our test process >

[jira] [Reopened] (SPARK-2690) Make unidoc part of our test process

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-2690: --- > Make unidoc part of our test process > > > Key: SPA

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088555#comment-15088555 ] Josh Rosen commented on SPARK-4066: --- [~ted_yu], if this issue is still relevant can you

[jira] [Closed] (SPARK-11798) Datanucleus jars is missing under lib_managed/jars

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen closed SPARK-11798. -- Resolution: Cannot Reproduce Resolving as "Cannot Reproduce" for now. Please re-open if this problem is

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Russell Alexander Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088537#comment-15088537 ] Russell Alexander Spitzer commented on SPARK-12639: --- This is a regressi

[jira] [Updated] (SPARK-12703) Spark KMeans Documentation Python Api

2016-01-07 Thread Anton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton updated SPARK-12703: -- Description: In the documentation of Spark's Kmeans - python api: http://spark.apache.org/docs/latest/mllib-clu

[jira] [Created] (SPARK-12703) Spark KMeans Documentation Python Api

2016-01-07 Thread Anton (JIRA)
Anton created SPARK-12703: - Summary: Spark KMeans Documentation Python Api Key: SPARK-12703 URL: https://issues.apache.org/jira/browse/SPARK-12703 Project: Spark Issue Type: Documentation C

[jira] [Created] (SPARK-12702) Populate statistics for DataFrame when reading CSV

2016-01-07 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-12702: -- Summary: Populate statistics for DataFrame when reading CSV Key: SPARK-12702 URL: https://issues.apache.org/jira/browse/SPARK-12702 Project: Spark Issue

[jira] [Updated] (SPARK-5162) Python yarn-cluster mode

2016-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-5162: Description: 2Running pyspark in yarn is currently limited to ‘yarn-client’ mode. It would be great

[jira] [Updated] (SPARK-5162) Python yarn-cluster mode

2016-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-5162: Description: Running pyspark in yarn is currently limited to ‘yarn-client’ mode. It would be great

[jira] [Commented] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088466#comment-15088466 ] Apache Spark commented on SPARK-12701: -- User 'BryanCutler' has created a pull reques

[jira] [Assigned] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12701: Assignee: (was: Apache Spark) > Logging FileAppender should use join to ensure thread

[jira] [Assigned] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12701: Assignee: Apache Spark > Logging FileAppender should use join to ensure thread is finished

[jira] [Commented] (SPARK-11157) Allow Spark to be built without assemblies

2016-01-07 Thread Kent Murra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088443#comment-15088443 ] Kent Murra commented on SPARK-11157: Having a folder of jars as an option would be gr

[jira] [Comment Edited] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088429#comment-15088429 ] Bryan Cutler edited comment on SPARK-12701 at 1/8/16 12:07 AM:

[jira] [Updated] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-12701: - Issue Type: Improvement (was: Bug) > Logging FileAppender should use join to ensure thread is fi

[jira] [Commented] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088429#comment-15088429 ] Bryan Cutler commented on SPARK-12701: -- I can submit a PR for this. > Logging FileA

[jira] [Created] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-12701: Summary: Logging FileAppender should use join to ensure thread is finished Key: SPARK-12701 URL: https://issues.apache.org/jira/browse/SPARK-12701 Project: Spark

[jira] [Commented] (SPARK-11157) Allow Spark to be built without assemblies

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088411#comment-15088411 ] Josh Rosen commented on SPARK-11157: For my own reference / ease-of-searchability, he

[jira] [Updated] (SPARK-11157) Allow Spark to be built without assemblies

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-11157: --- Target Version/s: 2.0.0 > Allow Spark to be built without assemblies > --

[jira] [Comment Edited] (SPARK-4257) Spark master can only be accessed by hostname

2016-01-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088405#comment-15088405 ] Jakob Odersky edited comment on SPARK-4257 at 1/7/16 11:46 PM: -

[jira] [Commented] (SPARK-4257) Spark master can only be accessed by hostname

2016-01-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088405#comment-15088405 ] Jakob Odersky commented on SPARK-4257: -- The way I interpret the documentation {{-h HO

[jira] [Commented] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088364#comment-15088364 ] Apache Spark commented on SPARK-12700: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12700: Assignee: (was: Apache Spark) > SortMergeJoin and BroadcastHashJoin should support con

[jira] [Assigned] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12700: Assignee: Apache Spark > SortMergeJoin and BroadcastHashJoin should support condition > --

[jira] [Commented] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-07 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088354#comment-15088354 ] Bo Meng commented on SPARK-12691: - I believe this is not a bug. "unionAll" is equal to "U

[jira] [Created] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12700: -- Summary: SortMergeJoin and BroadcastHashJoin should support condition Key: SPARK-12700 URL: https://issues.apache.org/jira/browse/SPARK-12700 Project: Spark Iss

[jira] [Commented] (SPARK-12686) Support group-by push down into data sources

2016-01-07 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088343#comment-15088343 ] Yan commented on SPARK-12686: - Spark-12449 seems to be a super set of this Jira. > Support g

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-01-07 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088334#comment-15088334 ] Yan commented on SPARK-12449: - Stephan, thanks for your explanations and questions. My answer

[jira] [Commented] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088322#comment-15088322 ] Apache Spark commented on SPARK-12699: -- User 'felixcheung' has created a pull reques

[jira] [Assigned] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12699: Assignee: (was: Apache Spark) > R driver process should start in a clean state > -

[jira] [Assigned] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12699: Assignee: Apache Spark > R driver process should start in a clean state >

[jira] [Created] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-12699: Summary: R driver process should start in a clean state Key: SPARK-12699 URL: https://issues.apache.org/jira/browse/SPARK-12699 Project: Spark Issue Type: Bu

[jira] [Assigned] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12654: Assignee: Thomas Graves (was: Apache Spark) > sc.wholeTextFiles with spark.hadoop.cloneCo

[jira] [Commented] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088290#comment-15088290 ] Apache Spark commented on SPARK-12654: -- User 'tgravescs' has created a pull request

[jira] [Assigned] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12654: Assignee: Apache Spark (was: Thomas Graves) > sc.wholeTextFiles with spark.hadoop.cloneCo

[jira] [Resolved] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

2016-01-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-12675. --- Resolution: Not A Problem Glad to hear it works well now! I'll close this issue then

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2016-01-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12580: - Assignee: Kazuaki Ishizaki (was: Apache Spark) > Remove string concatenations from usage and extended in

[jira] [Resolved] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2016-01-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-12580. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10524 [https://github.com/

[jira] [Updated] (SPARK-12696) Dataset serialization error

2016-01-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-12696: - Target Version/s: 1.6.1 (was: 1.6.1, 2.0.0) > Dataset serialization error >

[jira] [Updated] (SPARK-9716) BinaryClassificationEvaluator should accept Double prediction column

2016-01-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9716: - Assignee: Benjamin Fradet Target Version/s: 2.0.0 > BinaryClassificationEvalua

[jira] [Assigned] (SPARK-12696) Dataset serialization error

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12696: Assignee: Apache Spark (was: Michael Armbrust) > Dataset serialization error > --

[jira] [Commented] (SPARK-12696) Dataset serialization error

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088190#comment-15088190 ] Apache Spark commented on SPARK-12696: -- User 'marmbrus' has created a pull request f

[jira] [Assigned] (SPARK-12696) Dataset serialization error

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12696: Assignee: Michael Armbrust (was: Apache Spark) > Dataset serialization error > --

[jira] [Updated] (SPARK-12598) Bug in setMinPartitions function of StreamFileInputFormat

2016-01-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12598: -- Assignee: Darek Blasiak > Bug in setMinPartitions function of StreamFileInputFormat > -

[jira] [Resolved] (SPARK-12598) Bug in setMinPartitions function of StreamFileInputFormat

2016-01-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12598. --- Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by pull request

[jira] [Assigned] (SPARK-12576) Enable expression parsing (used in DataFrames)

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12576: Assignee: (was: Apache Spark) > Enable expression parsing (used in DataFrames) > -

[jira] [Assigned] (SPARK-12576) Enable expression parsing (used in DataFrames)

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12576: Assignee: Apache Spark > Enable expression parsing (used in DataFrames) >

[jira] [Commented] (SPARK-12576) Enable expression parsing (used in DataFrames)

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088055#comment-15088055 ] Apache Spark commented on SPARK-12576: -- User 'hvanhovell' has created a pull request

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Evan Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088017#comment-15088017 ] Evan Chan commented on SPARK-12639: --- +1 > Improve Explain for DataSources with Handled

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-01-07 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15087983#comment-15087983 ] Yan commented on SPARK-12449: - Stephan, By "partial op" I mean, for instance, partial map-s

  1   2   >