[jira] [Assigned] (SPARK-26317) Upgrade SBT to 0.13.18

[ https://issues.apache.org/jira/browse/SPARK-26317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26317: Assignee: (was: Apache Spark) > Upgrade SBT to 0.13.18 > -- > >

[jira] [Commented] (SPARK-26317) Upgrade SBT to 0.13.18

[ https://issues.apache.org/jira/browse/SPARK-26317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714394#comment-16714394 ] Apache Spark commented on SPARK-26317: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-26317) Upgrade SBT to 0.13.18

[ https://issues.apache.org/jira/browse/SPARK-26317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26317: Assignee: Apache Spark > Upgrade SBT to 0.13.18 > -- > >

[jira] [Created] (SPARK-26317) Upgrade SBT to 0.13.18

Dongjoon Hyun created SPARK-26317: - Summary: Upgrade SBT to 0.13.18 Key: SPARK-26317 URL: https://issues.apache.org/jira/browse/SPARK-26317 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

[ https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26316: Assignee: (was: Apache Spark) > Because of the perf degradation in TPC-DS, we

[jira] [Commented] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

[ https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714367#comment-16714367 ] Apache Spark commented on SPARK-26316: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

[ https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26316: Assignee: Apache Spark > Because of the perf degradation in TPC-DS, we currently partial

[jira] [Resolved] (SPARK-26307) Fix CTAS when INSERT a partitioned table using Hive serde

[ https://issues.apache.org/jira/browse/SPARK-26307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26307. - Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 2.3.3

[jira] [Resolved] (SPARK-26287) Don't need to create an empty spill file when memory has no records

[ https://issues.apache.org/jira/browse/SPARK-26287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26287. --- Resolution: Fixed Assignee: wangjiaochun Fix Version/s: 3.0.0 This is

[jira] [Updated] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

[ https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-26316: --- Description: The code of  

[jira] [Created] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

Ke Jia created SPARK-26316: -- Summary: Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join, Key: SPARK-26316 URL:

[jira] [Created] (SPARK-26315) auto cast threshold from Integer to Float in approxSimilarityJoin of BucketedRandomProjectionLSHModel

Song Ci created SPARK-26315: --- Summary: auto cast threshold from Integer to Float in approxSimilarityJoin of BucketedRandomProjectionLSHModel Key: SPARK-26315 URL: https://issues.apache.org/jira/browse/SPARK-26315

[jira] [Commented] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714245#comment-16714245 ] ASF GitHub Bot commented on SPARK-11248: GitHub user VicoWu opened a pull request:

[jira] [Commented] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714220#comment-16714220 ] Liang-Chi Hsieh commented on SPARK-26306: - I have not noticed the evidence that this test is

[jira] [Closed] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26314. - > support Confluent encoded Avro in Spark Structured Streaming >

[jira] [Resolved] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26314. --- Resolution: Invalid I'll close this as 'Invalid' for now. > support Confluent encoded Avro

[jira] [Commented] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714133#comment-16714133 ] Dongjoon Hyun commented on SPARK-26314: --- One more thing: Apache Spark supports Apache Kafka, not a

[jira] [Commented] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714131#comment-16714131 ] Dongjoon Hyun commented on SPARK-26314: --- Hi, [~davidahern]. Apache Spark supports Apache Avro. I

[jira] [Commented] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714127#comment-16714127 ] Dongjoon Hyun commented on SPARK-21708: --- Could you make a PR if you are using `sbt 1.x` with

[jira] [Commented] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714109#comment-16714109 ] Apache Spark commented on SPARK-25401: -- User 'davidvrba' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25401: Assignee: (was: Apache Spark) > Reorder the required ordering to match the table's

[jira] [Commented] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714108#comment-16714108 ] Apache Spark commented on SPARK-25401: -- User 'davidvrba' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25401: Assignee: Apache Spark > Reorder the required ordering to match the table's output

[jira] [Commented] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714101#comment-16714101 ] Darcy Shen commented on SPARK-21708: Please make the Priority higher. sbt 1.x is better than sbt

[jira] [Resolved] (SPARK-26283) When zstd compression enabled, Inprogress application in the history server appUI showing finished job as running

[ https://issues.apache.org/jira/browse/SPARK-26283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26283. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23241

[jira] [Assigned] (SPARK-26283) When zstd compression enabled, Inprogress application in the history server appUI showing finished job as running

[ https://issues.apache.org/jira/browse/SPARK-26283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26283: - Assignee: shahid > When zstd compression enabled, Inprogress application in the history server

[jira] [Commented] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714002#comment-16714002 ] David Vrba commented on SPARK-25401: Ok. I implemented the above idea and it is working, so i will

[jira] [Commented] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713986#comment-16713986 ] David Ahern commented on SPARK-26314: -

[jira] [Commented] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713985#comment-16713985 ] David Ahern commented on SPARK-26314: -

[jira] [Created] (SPARK-26314) support Confluent encoded Avro in Spark Structured Streaming

David Ahern created SPARK-26314: --- Summary: support Confluent encoded Avro in Spark Structured Streaming Key: SPARK-26314 URL: https://issues.apache.org/jira/browse/SPARK-26314 Project: Spark

[jira] [Comment Edited] (SPARK-25992) Accumulators giving KeyError in pyspark

[ https://issues.apache.org/jira/browse/SPARK-25992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713965#comment-16713965 ] Abdeali Kothari edited comment on SPARK-25992 at 12/9/18 1:05 PM: -- Here

[jira] [Commented] (SPARK-25992) Accumulators giving KeyError in pyspark

[ https://issues.apache.org/jira/browse/SPARK-25992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713965#comment-16713965 ] Abdeali Kothari commented on SPARK-25992: - Here is a reproducible example in pyspark where using

[jira] [Comment Edited] (SPARK-25992) Accumulators giving KeyError in pyspark

[ https://issues.apache.org/jira/browse/SPARK-25992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713965#comment-16713965 ] Abdeali Kothari edited comment on SPARK-25992 at 12/9/18 1:02 PM: -- Here

[jira] [Commented] (SPARK-26305) Breakthrough the memory limitation of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-26305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713917#comment-16713917 ] Liang-Chi Hsieh commented on SPARK-26305: - I'm working on SPARK-25549, which provides an API to

[jira] [Commented] (SPARK-26313) move read related methods from Table to read related mix-in traits

[ https://issues.apache.org/jira/browse/SPARK-26313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713899#comment-16713899 ] Apache Spark commented on SPARK-26313: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26313) move read related methods from Table to read related mix-in traits

[ https://issues.apache.org/jira/browse/SPARK-26313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26313: Assignee: Wenchen Fan (was: Apache Spark) > move read related methods from Table to

[jira] [Assigned] (SPARK-26313) move read related methods from Table to read related mix-in traits

[ https://issues.apache.org/jira/browse/SPARK-26313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26313: Assignee: Apache Spark (was: Wenchen Fan) > move read related methods from Table to

[jira] [Created] (SPARK-26313) move read related methods from Table to read related mix-in traits

Wenchen Fan created SPARK-26313: --- Summary: move read related methods from Table to read related mix-in traits Key: SPARK-26313 URL: https://issues.apache.org/jira/browse/SPARK-26313 Project: Spark