[jira] [Commented] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088322#comment-15088322 ] Apache Spark commented on SPARK-12699: -- User 'felixcheung' has created a pull request for this

[jira] [Assigned] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12699: Assignee: (was: Apache Spark) > R driver process should start in a clean state >

[jira] [Assigned] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12699: Assignee: Apache Spark > R driver process should start in a clean state >

[jira] [Commented] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-07 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088354#comment-15088354 ] Bo Meng commented on SPARK-12691: - I believe this is not a bug. "unionAll" is equal to "UNION ALL" in SQL

[jira] [Commented] (SPARK-11157) Allow Spark to be built without assemblies

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088411#comment-15088411 ] Josh Rosen commented on SPARK-11157: For my own reference / ease-of-searchability, here's a backlink

[jira] [Updated] (SPARK-11157) Allow Spark to be built without assemblies

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-11157: --- Target Version/s: 2.0.0 > Allow Spark to be built without assemblies >

[jira] [Commented] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088429#comment-15088429 ] Bryan Cutler commented on SPARK-12701: -- I can submit a PR for this. > Logging FileAppender should

[jira] [Created] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-12701: Summary: Logging FileAppender should use join to ensure thread is finished Key: SPARK-12701 URL: https://issues.apache.org/jira/browse/SPARK-12701 Project: Spark

[jira] [Commented] (SPARK-11157) Allow Spark to be built without assemblies

2016-01-07 Thread Kent Murra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088443#comment-15088443 ] Kent Murra commented on SPARK-11157: Having a folder of jars as an option would be great. In our

[jira] [Updated] (SPARK-5162) Python yarn-cluster mode

2016-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-5162: Description: 2Running pyspark in yarn is currently limited to ‘yarn-client’ mode. It would be

[jira] [Created] (SPARK-12702) Populate statistics for DataFrame when reading CSV

2016-01-07 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-12702: -- Summary: Populate statistics for DataFrame when reading CSV Key: SPARK-12702 URL: https://issues.apache.org/jira/browse/SPARK-12702 Project: Spark Issue

[jira] [Updated] (SPARK-12703) Spark KMeans Documentation Python Api

2016-01-07 Thread Anton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton updated SPARK-12703: -- Description: In the documentation of Spark's Kmeans - python api:

[jira] [Commented] (SPARK-12686) Support group-by push down into data sources

2016-01-07 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088343#comment-15088343 ] Yan commented on SPARK-12686: - Spark-12449 seems to be a super set of this Jira. > Support group-by push

[jira] [Commented] (SPARK-4257) Spark master can only be accessed by hostname

2016-01-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088405#comment-15088405 ] Jakob Odersky commented on SPARK-4257: -- The way I interpret the documentation {{-h HOST, --host HOST

[jira] [Comment Edited] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088429#comment-15088429 ] Bryan Cutler edited comment on SPARK-12701 at 1/8/16 12:07 AM: --- I can

[jira] [Created] (SPARK-12703) Spark KMeans Documentation Python Api

2016-01-07 Thread Anton (JIRA)
Anton created SPARK-12703: - Summary: Spark KMeans Documentation Python Api Key: SPARK-12703 URL: https://issues.apache.org/jira/browse/SPARK-12703 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12700: -- Summary: SortMergeJoin and BroadcastHashJoin should support condition Key: SPARK-12700 URL: https://issues.apache.org/jira/browse/SPARK-12700 Project: Spark

[jira] [Commented] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088364#comment-15088364 ] Apache Spark commented on SPARK-12700: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12700: Assignee: (was: Apache Spark) > SortMergeJoin and BroadcastHashJoin should support

[jira] [Assigned] (SPARK-12700) SortMergeJoin and BroadcastHashJoin should support condition

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12700: Assignee: Apache Spark > SortMergeJoin and BroadcastHashJoin should support condition >

[jira] [Created] (SPARK-12699) R driver process should start in a clean state

2016-01-07 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-12699: Summary: R driver process should start in a clean state Key: SPARK-12699 URL: https://issues.apache.org/jira/browse/SPARK-12699 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-4257) Spark master can only be accessed by hostname

2016-01-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088405#comment-15088405 ] Jakob Odersky edited comment on SPARK-4257 at 1/7/16 11:46 PM: --- The way I

[jira] [Updated] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-12701: - Issue Type: Improvement (was: Bug) > Logging FileAppender should use join to ensure thread is

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-01-07 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088334#comment-15088334 ] Yan commented on SPARK-12449: - Stephan, thanks for your explanations and questions. My answers are as

[jira] [Assigned] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12701: Assignee: (was: Apache Spark) > Logging FileAppender should use join to ensure thread

[jira] [Assigned] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12701: Assignee: Apache Spark > Logging FileAppender should use join to ensure thread is

[jira] [Commented] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088466#comment-15088466 ] Apache Spark commented on SPARK-12701: -- User 'BryanCutler' has created a pull request for this

[jira] [Updated] (SPARK-5162) Python yarn-cluster mode

2016-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-5162: Description: Running pyspark in yarn is currently limited to ‘yarn-client’ mode. It would be great

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Russell Alexander Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088537#comment-15088537 ] Russell Alexander Spitzer commented on SPARK-12639: --- This is a regression of

[jira] [Commented] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

2016-01-07 Thread Alexandru Rosianu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087142#comment-15087142 ] Alexandru Rosianu commented on SPARK-12675: --- The input dataset had 16368 partitions. I

[jira] [Commented] (SPARK-12430) Temporary folders do not get deleted after Task completes causing problems with disk space.

2016-01-07 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087136#comment-15087136 ] Jean-Baptiste Onofré commented on SPARK-12430: -- Yes, I agree Fede. I'm testing couple of

[jira] [Commented] (SPARK-12692) Scala style: check no white space before comma and colon

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087153#comment-15087153 ] Apache Spark commented on SPARK-12692: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-01-07 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-12316: - Description: When application end, AM will clean the staging dir. But if the driver trigger to

[jira] [Commented] (SPARK-12622) spark-submit fails on executors when jar has a space in it

2016-01-07 Thread Ajesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087157#comment-15087157 ] Ajesh Kumar commented on SPARK-12622: - In that case, whether we need to fix this under spark or

[jira] [Commented] (SPARK-12622) spark-submit fails on executors when jar has a space in it

2016-01-07 Thread Adrian Bridgett (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087159#comment-15087159 ] Adrian Bridgett commented on SPARK-12622: - Hi Ajesh - I think it's in the mesos-specific Spark

[jira] [Commented] (SPARK-12006) GaussianMixture.train crashes if an initial model is not None

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087219#comment-15087219 ] Apache Spark commented on SPARK-12006: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-12604) Java count(AprroxDistinct)ByKey methods return Scala Long not Java

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087139#comment-15087139 ] Apache Spark commented on SPARK-12604: -- User 'srowen' has created a pull request for this issue:

[jira] [Updated] (SPARK-12662) Add a local sort operator to DataFrame used by randomSplit

2016-01-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12662: Target Version/s: 1.5.2, 1.6.1, 2.0.0 (was: 2.0.0) > Add a local sort operator to DataFrame used

[jira] [Created] (SPARK-12692) Scala style: check no white space before comma and colon

2016-01-07 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-12692: -- Summary: Scala style: check no white space before comma and colon Key: SPARK-12692 URL: https://issues.apache.org/jira/browse/SPARK-12692 Project: Spark

[jira] [Assigned] (SPARK-12523) Support long-running of the Spark On HBase and hive meta store.

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12523: Assignee: (was: Apache Spark) > Support long-running of the Spark On HBase and hive

[jira] [Updated] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rado Buransky updated SPARK-12693: -- Attachment: kafka-log.txt > OffsetOutOfRangeException cause by retention >

[jira] [Updated] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rado Buransky updated SPARK-12693: -- Description: I am running Kafka server locally with extremely low retention of 3 seconds and

[jira] [Commented] (SPARK-6476) Spark fileserver not started on same IP as using spark.driver.host

2016-01-07 Thread Kyle Sutton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087530#comment-15087530 ] Kyle Sutton commented on SPARK-6476: The specific problem seems to be with the translation of an added

[jira] [Created] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
Rado Buransky created SPARK-12693: - Summary: OffsetOutOfRangeException cause by retention Key: SPARK-12693 URL: https://issues.apache.org/jira/browse/SPARK-12693 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

2016-01-07 Thread mustafa elbehery (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087301#comment-15087301 ] mustafa elbehery commented on SPARK-5226: - I have tried to use Aliaksei's implementation on 500MB

[jira] [Updated] (SPARK-12295) Manage the memory used by window function

2016-01-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12295: -- Assignee: Davies Liu > Manage the memory used by window function >

[jira] [Assigned] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12639: Assignee: Apache Spark > Improve Explain for DataSources with Handled Predicate Pushdowns

[jira] [Resolved] (SPARK-12507) Expose closeFileAfterWrite and allowBatching configurations for Streaming

2016-01-07 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-12507. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Expose

[jira] [Commented] (SPARK-4991) Worker should reconnect to Master when Master actor restart

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088560#comment-15088560 ] Josh Rosen commented on SPARK-4991: --- Is this still relevant after we remove the Akka RPC / use of Akka

[jira] [Commented] (SPARK-4438) Add HistoryServer RESTful API

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088563#comment-15088563 ] Josh Rosen commented on SPARK-4438: --- Also, note that the canonical documentation for this API is in the

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088635#comment-15088635 ] Wenchen Fan commented on SPARK-12704: - cc [~joshrosen] [~nongli] [~marmbrus] [~yhuai] [~rxin] > we

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088650#comment-15088650 ] Wenchen Fan commented on SPARK-12704: - yea, sub-optimal is more proper, and I'm ok to re-visit it

[jira] [Resolved] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-07 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-12591. --- Resolution: Fixed Fix Version/s: 2.0.0 > NullPointerException using checkpointed

[jira] [Updated] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-07 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-12591: -- Assignee: Shixiong Zhu > NullPointerException using checkpointed mapWithState with

[jira] [Commented] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088704#comment-15088704 ] Rado Buransky commented on SPARK-12693: --- I wasn't actually right. This issue is not related to

[jira] [Updated] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-12704: - Issue Type: Improvement (was: Bug) > we may repartition a relation even it's not needed

[jira] [Commented] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088633#comment-15088633 ] Apache Spark commented on SPARK-12591: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12704: Description: The implementation of {{HashPartitioning.compatibleWith}} has been wrong for a

[jira] [Updated] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12704: Description: The implementation of {{HashPartitioning.compatibleWith}} has been sub-optimal for a

[jira] [Comment Edited] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088704#comment-15088704 ] Rado Buransky edited comment on SPARK-12693 at 1/8/16 4:50 AM: --- I wasn't

[jira] [Created] (SPARK-12694) The detailed rest API documentation for each field is missing

2016-01-07 Thread Zhuo Liu (JIRA)
Zhuo Liu created SPARK-12694: Summary: The detailed rest API documentation for each field is missing Key: SPARK-12694 URL: https://issues.apache.org/jira/browse/SPARK-12694 Project: Spark Issue

[jira] [Commented] (SPARK-12536) Fix the Explain Outputs of Empty LocalRelation and LocalTableScan

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087653#comment-15087653 ] Apache Spark commented on SPARK-12536: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Resolved] (SPARK-4438) Add HistoryServer RESTful API

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4438. --- Resolution: Fixed Fix Version/s: 1.4.0 [~jonathak], thanks for pointing that out. I'm going to

[jira] [Closed] (SPARK-11798) Datanucleus jars is missing under lib_managed/jars

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen closed SPARK-11798. -- Resolution: Cannot Reproduce Resolving as "Cannot Reproduce" for now. Please re-open if this problem

[jira] [Resolved] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2016-01-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-4066. --- Resolution: Later Haven't heard of much feedback so far. Resolving for now. > Make whether maven builds

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088555#comment-15088555 ] Josh Rosen commented on SPARK-4066: --- [~ted_yu], if this issue is still relevant can you submit this

[jira] [Reopened] (SPARK-2690) Make unidoc part of our test process

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-2690: --- > Make unidoc part of our test process > > > Key:

[jira] [Resolved] (SPARK-2690) Make unidoc part of our test process

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2690. --- Resolution: Duplicate > Make unidoc part of our test process > >

[jira] [Assigned] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12639: Assignee: (was: Apache Spark) > Improve Explain for DataSources with Handled

[jira] [Resolved] (SPARK-2690) Make unidoc part of our test process

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2690. --- Resolution: Fixed This is a duplicate of SPARK-7019 > Make unidoc part of our test process >

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Russell Alexander Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088558#comment-15088558 ] Russell Alexander Spitzer commented on SPARK-12639: ---

[jira] [Commented] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088557#comment-15088557 ] Apache Spark commented on SPARK-12639: -- User 'RussellSpitzer' has created a pull request for this

[jira] [Created] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-12704: --- Summary: we may repartition a relation even it's not needed Key: SPARK-12704 URL: https://issues.apache.org/jira/browse/SPARK-12704 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088639#comment-15088639 ] Yin Huai commented on SPARK-12704: -- [~cloud_fan] Thank you for bringing it up. Right now,

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2016-01-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088638#comment-15088638 ] Michael Armbrust commented on SPARK-12704: -- I think this explanation might be clearer with with

[jira] [Commented] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088725#comment-15088725 ] Cody Koeninger commented on SPARK-12693: What is your actual use case for changing the log

[jira] [Resolved] (SPARK-5487) Dockerfile to build spark's custom akka.

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-5487. --- Resolution: Won't Fix Resolving as "Won't Fix" for now, since this is no longer needed now that

[jira] [Commented] (SPARK-5569) Checkpoints cannot reference classes defined outside of Spark's assembly

2016-01-07 Thread David Winters (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088794#comment-15088794 ] David Winters commented on SPARK-5569: -- I have encountered this issue also. I have a Spark streaming

[jira] [Commented] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088822#comment-15088822 ] Xiao Li commented on SPARK-12705: - Could you explain how to reproduce it? In sqlquerysuit of

[jira] [Created] (SPARK-12708) Sorting task error in Stages Page when yarn mode

2016-01-07 Thread Koyo Yoshida (JIRA)
Koyo Yoshida created SPARK-12708: Summary: Sorting task error in Stages Page when yarn mode Key: SPARK-12708 URL: https://issues.apache.org/jira/browse/SPARK-12708 Project: Spark Issue Type:

[jira] [Created] (SPARK-12706) support grouping function together group set

2016-01-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12706: -- Summary: support grouping function together group set Key: SPARK-12706 URL: https://issues.apache.org/jira/browse/SPARK-12706 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-12317) Support configurate value for AUTO_BROADCASTJOIN_THRESHOLD and SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE with unit(e.g. kb/mb/gb) in SQLConf

2016-01-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12317. - Resolution: Fixed Assignee: kevin yu Fix Version/s: 2.0.0 > Support configurate

[jira] [Updated] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12706: --- Summary: support grouping/grouping_id function together group set (was: support grouping function

[jira] [Commented] (SPARK-12693) OffsetOutOfRangeException cause by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088734#comment-15088734 ] Rado Buransky commented on SPARK-12693: --- Ok. Let's forget about short time periods. Now I believe

[jira] [Updated] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12706: --- Description:

[jira] [Updated] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12706: --- Assignee: (was: Davies Liu) > support grouping/grouping_id function together group set >

[jira] [Updated] (SPARK-12693) OffsetOutOfRangeException caused by retention

2016-01-07 Thread Rado Buransky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rado Buransky updated SPARK-12693: -- Summary: OffsetOutOfRangeException caused by retention (was: OffsetOutOfRangeException cause

[jira] [Commented] (SPARK-8540) KMeans-based outlier detection

2016-01-07 Thread Rakesh Chalasani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088746#comment-15088746 ] Rakesh Chalasani commented on SPARK-8540: - I see that this hasn't moved forward, so trying to

[jira] [Created] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-12707: -- Summary: Remove submit python/R scripts through pyspark/sparkR Key: SPARK-12707 URL: https://issues.apache.org/jira/browse/SPARK-12707 Project: Spark Issue

[jira] [Assigned] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12707: Assignee: (was: Apache Spark) > Remove submit python/R scripts through pyspark/sparkR

[jira] [Commented] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088757#comment-15088757 ] Apache Spark commented on SPARK-12707: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12707) Remove submit python/R scripts through pyspark/sparkR

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12707: Assignee: Apache Spark > Remove submit python/R scripts through pyspark/sparkR >

[jira] [Commented] (SPARK-12687) Support from clause surrounded by `()`

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088836#comment-15088836 ] Apache Spark commented on SPARK-12687: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12687) Support from clause surrounded by `()`

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12687: Assignee: Apache Spark > Support from clause surrounded by `()` >

[jira] [Assigned] (SPARK-12687) Support from clause surrounded by `()`

2016-01-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12687: Assignee: (was: Apache Spark) > Support from clause surrounded by `()` >

[jira] [Created] (SPARK-12710) Create local CoGroup operator

2016-01-07 Thread Mao, Wei (JIRA)
Mao, Wei created SPARK-12710: Summary: Create local CoGroup operator Key: SPARK-12710 URL: https://issues.apache.org/jira/browse/SPARK-12710 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12709) Create local Except operator

2016-01-07 Thread Mao, Wei (JIRA)
Mao, Wei created SPARK-12709: Summary: Create local Except operator Key: SPARK-12709 URL: https://issues.apache.org/jira/browse/SPARK-12709 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12705: -- Summary: Sorting column can't be resolved if it's not in projection Key: SPARK-12705 URL: https://issues.apache.org/jira/browse/SPARK-12705 Project: Spark Issue

[jira] [Assigned] (SPARK-12706) support grouping function together group set

2016-01-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12706: -- Assignee: Davies Liu > support grouping function together group set >

[jira] [Assigned] (SPARK-4628) Put external projects and examples behind a build flag

2016-01-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-4628: - Assignee: Josh Rosen > Put external projects and examples behind a build flag >

  1   2   >