[jira] [Resolved] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32570. -- Resolution: Invalid > Thriftserver LDAP failed > > >

[jira] [Updated] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32570: - Component/s: (was: Security) SQL > Thriftserver LDAP failed >

[jira] [Commented] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173547#comment-17173547 ] Takeshi Yamamuro commented on SPARK-32570: -- Please [use 

[jira] [Assigned] (SPARK-32568) Upgrade Kafka to 2.6.0

[ https://issues.apache.org/jira/browse/SPARK-32568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32568: Assignee: Dongjoon Hyun > Upgrade Kafka to 2.6.0 > -- > >

[jira] [Resolved] (SPARK-32568) Upgrade Kafka to 2.6.0

[ https://issues.apache.org/jira/browse/SPARK-32568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32568. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29386

[jira] [Created] (SPARK-32571) yarnClient.killApplication(appId) is never called

A Tester created SPARK-32571: Summary: yarnClient.killApplication(appId) is never called Key: SPARK-32571 URL: https://issues.apache.org/jira/browse/SPARK-32571 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-32569) Gaussian can not handle data close to MaxDouble

Tobias Haar created SPARK-32569: --- Summary: Gaussian can not handle data close to MaxDouble Key: SPARK-32569 URL: https://issues.apache.org/jira/browse/SPARK-32569 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32568) Upgrade Kafka to 2.6.0

[ https://issues.apache.org/jira/browse/SPARK-32568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173407#comment-17173407 ] Apache Spark commented on SPARK-32568: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32568) Upgrade Kafka to 2.6.0

[ https://issues.apache.org/jira/browse/SPARK-32568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32568: Assignee: Apache Spark > Upgrade Kafka to 2.6.0 > -- > >

[jira] [Created] (SPARK-32568) Upgrade Kafka to 2.6.0

Dongjoon Hyun created SPARK-32568: - Summary: Upgrade Kafka to 2.6.0 Key: SPARK-32568 URL: https://issues.apache.org/jira/browse/SPARK-32568 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32567) Code-gen for full outer shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-32567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32567: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Code-gen for full outer shuffled

[jira] [Created] (SPARK-32567) Code-gen for full outer shuffled hash join

Cheng Su created SPARK-32567: Summary: Code-gen for full outer shuffled hash join Key: SPARK-32567 URL: https://issues.apache.org/jira/browse/SPARK-32567 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32191) Migration Guide

[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173262#comment-17173262 ] Apache Spark commented on SPARK-32191: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32191) Migration Guide

[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32191: Assignee: (was: Apache Spark) > Migration Guide > --- > >

[jira] [Commented] (SPARK-32191) Migration Guide

[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173263#comment-17173263 ] Apache Spark commented on SPARK-32191: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32191) Migration Guide

[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32191: Assignee: Apache Spark > Migration Guide > --- > > Key:

[jira] [Updated] (SPARK-31703) Changes made by SPARK-26985 break reading parquet files correctly in BigEndian architectures (AIX + LinuxPPC64)

[ https://issues.apache.org/jira/browse/SPARK-31703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31703: -- Labels: BigEndian correctness (was: BigEndian) > Changes made by SPARK-26985 break reading

[jira] [Commented] (SPARK-32187) User Guide - Shipping Python Package

[ https://issues.apache.org/jira/browse/SPARK-32187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173247#comment-17173247 ] Hyukjin Kwon commented on SPARK-32187: -- Thank you so much [~fhoering]. > User Guide - Shipping

[jira] [Commented] (SPARK-32187) User Guide - Shipping Python Package

[ https://issues.apache.org/jira/browse/SPARK-32187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173237#comment-17173237 ] Fabian Höring commented on SPARK-32187: --- OK. I'll have a look into that next week. > User Guide -

[jira] [Comment Edited] (SPARK-32565) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173226#comment-17173226 ] Dongjoon Hyun edited comment on SPARK-32565 at 8/7/20, 3:47 PM: IIUC,

[jira] [Commented] (SPARK-32565) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173226#comment-17173226 ] Dongjoon Hyun commented on SPARK-32565: --- IIUC, this is for Parquet 1.12. Apache Spark is not using

[jira] [Updated] (SPARK-32565) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32565: -- Description: Provide an interface for Parquet column encryption policy engine clients.

[jira] [Assigned] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32564: - Assignee: Takeshi Yamamuro > Inject data statistics to simulate plan generation on

[jira] [Resolved] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32564. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29384

[jira] [Commented] (SPARK-32562) Pyspark drop duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-32562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173199#comment-17173199 ] Rohit Mishra commented on SPARK-32562: -- [~amotex], Please avoid adding fix version. You can always

[jira] [Updated] (SPARK-32562) Pyspark drop duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-32562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32562: - Fix Version/s: (was: 3.0.0) > Pyspark drop duplicate columns >

[jira] [Resolved] (SPARK-25557) ORC predicate pushdown for nested fields

[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25557. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28761

[jira] [Assigned] (SPARK-25557) ORC predicate pushdown for nested fields

[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25557: - Assignee: L. C. Hsieh > ORC predicate pushdown for nested fields >

[jira] [Commented] (SPARK-32565) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173166#comment-17173166 ] Takeshi Yamamuro commented on SPARK-32565: -- Could you describe more to make other developers

[jira] [Updated] (SPARK-32565) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated SPARK-32565: - Summary: Parquet encryption policy interface (was: Encryption policy interface) >

[jira] [Updated] (SPARK-32565) Encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated SPARK-32565: - Description: Provide an interface for Parquet column encryption policy engine clients.

[jira] [Commented] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173131#comment-17173131 ] Takeshi Yamamuro commented on SPARK-32563: -- I tried it in master/branch-3.0/branch-2.4 and they

[jira] [Commented] (SPARK-32566) kafka consumer cache capacity is unclear

[ https://issues.apache.org/jira/browse/SPARK-32566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173119#comment-17173119 ] Jungtaek Lim commented on SPARK-32566: -- The doc you've referred is for DStream. The doc for

[jira] [Commented] (SPARK-32565) Encryption policy interface

[ https://issues.apache.org/jira/browse/SPARK-32565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173108#comment-17173108 ] Takeshi Yamamuro commented on SPARK-32565: -- Please fill the description, first. > Encryption

[jira] [Commented] (SPARK-32566) kafka consumer cache capacity is unclear

[ https://issues.apache.org/jira/browse/SPARK-32566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173107#comment-17173107 ] Takeshi Yamamuro commented on SPARK-32566: -- cc: [~kabhwan] > kafka consumer cache capacity is

[jira] [Updated] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated SPARK-32563: Description: spark-sql doesn't support insert into mixed static & dynamic partition, for example:

[jira] [Updated] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated SPARK-32563: Description: spark-sql doesn't support insert into mixed static & dynamic partition, for example:

[jira] [Updated] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated SPARK-32563: Attachment: SPARK-32563.log > spark-sql doesn't support insert into mixed static & dynamic partition >

[jira] [Updated] (SPARK-32566) kafka consumer cache capacity is unclear

[ https://issues.apache.org/jira/browse/SPARK-32566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] srpn updated SPARK-32566: - Description: The [docs|https://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html] mention

[jira] [Created] (SPARK-32566) kafka consumer cache capacity is unclear

srpn created SPARK-32566: Summary: kafka consumer cache capacity is unclear Key: SPARK-32566 URL: https://issues.apache.org/jira/browse/SPARK-32566 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-32565) Encryption policy interface

Gidon Gershinsky created SPARK-32565: Summary: Encryption policy interface Key: SPARK-32565 URL: https://issues.apache.org/jira/browse/SPARK-32565 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173041#comment-17173041 ] Apache Spark commented on SPARK-32564: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173040#comment-17173040 ] Apache Spark commented on SPARK-32564: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32564: Assignee: Apache Spark > Inject data statistics to simulate plan generation on actual

[jira] [Assigned] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32564: Assignee: (was: Apache Spark) > Inject data statistics to simulate plan generation

[jira] [Created] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

Takeshi Yamamuro created SPARK-32564: Summary: Inject data statistics to simulate plan generation on actual TPCDS data Key: SPARK-32564 URL: https://issues.apache.org/jira/browse/SPARK-32564

[jira] [Resolved] (SPARK-32556) Fix release script to uri encode the user provided passwords.

[ https://issues.apache.org/jira/browse/SPARK-32556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma resolved SPARK-32556. - Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved

[jira] [Assigned] (SPARK-32556) Fix release script to uri encode the user provided passwords.

[ https://issues.apache.org/jira/browse/SPARK-32556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-32556: --- Assignee: Prashant Sharma > Fix release script to uri encode the user provided

[jira] [Commented] (SPARK-32180) Getting Started - Installation

[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172985#comment-17172985 ] Hyukjin Kwon commented on SPARK-32180: -- Nope, I haven't started it yet. I currently am basically

[jira] [Created] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

yx91490 created SPARK-32563: --- Summary: spark-sql doesn't support insert into mixed static & dynamic partition Key: SPARK-32563 URL: https://issues.apache.org/jira/browse/SPARK-32563 Project: Spark

[jira] [Resolved] (SPARK-32528) The analyze method should make sure the plan is analyzed

[ https://issues.apache.org/jira/browse/SPARK-32528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32528. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29349

[jira] [Comment Edited] (SPARK-32526) Let sql/catalyst module tests pass for Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172091#comment-17172091 ] Yang Jie edited comment on SPARK-32526 at 8/7/20, 7:20 AM: --- Detailed error

[jira] [Comment Edited] (SPARK-32536) deleted not existing hdfs locations when use spark sql to execute "insert overwrite" statement to dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172889#comment-17172889 ] yx91490 edited comment on SPARK-32536 at 8/7/20, 6:49 AM: -- the method 

[jira] [Commented] (SPARK-32180) Getting Started - Installation

[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172938#comment-17172938 ] Rohit Mishra commented on SPARK-32180: -- [~hyukjin.kwon], I will start contributing to this module,