[jira] [Commented] (SPARK-28613) Spark SQL action collect just judge size of compressed RDD's size, not accurate enough

[ https://issues.apache.org/jira/browse/SPARK-28613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902704#comment-16902704 ] Shivu Sondur commented on SPARK-28613: -- I will check > Spark SQL action collect just judge size of

[jira] [Resolved] (SPARK-28634) Failed to start SparkSession with Keytab file

[ https://issues.apache.org/jira/browse/SPARK-28634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-28634. - Resolution: Not A Problem > Failed to start SparkSession with Keytab file >

[jira] [Commented] (SPARK-28634) Failed to start SparkSession with Keytab file

[ https://issues.apache.org/jira/browse/SPARK-28634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902702#comment-16902702 ] Yuming Wang commented on SPARK-28634: - Thank you [~vanzin] It works. > Failed to start SparkSession

[jira] [Updated] (SPARK-28330) ANSI SQL: Top-level in

[ https://issues.apache.org/jira/browse/SPARK-28330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28330: Description: h2. {{LIMIT}} and {{OFFSET}} LIMIT and OFFSET allow you to retrieve just a portion

[jira] [Updated] (SPARK-28330) ANSI SQL: Top-level in

[ https://issues.apache.org/jira/browse/SPARK-28330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28330: Summary: ANSI SQL: Top-level in (was: Enhance query limit) > ANSI SQL: Top-level in >

[jira] [Resolved] (SPARK-28454) Validate LongType in _make_type_verifier

[ https://issues.apache.org/jira/browse/SPARK-28454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28454. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25117

[jira] [Assigned] (SPARK-28454) Validate LongType in _make_type_verifier

[ https://issues.apache.org/jira/browse/SPARK-28454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28454: Assignee: AY > Validate LongType in _make_type_verifier >

[jira] [Issue Comment Deleted] (SPARK-28474) Lower JDBC client cannot read binary type

[ https://issues.apache.org/jira/browse/SPARK-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28474: Comment: was deleted (was: I'm working on.) > Lower JDBC client cannot read binary type >

[jira] [Assigned] (SPARK-28649) Git Ignore does not ignore python/.eggs

[ https://issues.apache.org/jira/browse/SPARK-28649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28649: - Assignee: Rob Vesse > Git Ignore does not ignore python/.eggs >

[jira] [Resolved] (SPARK-28649) Git Ignore does not ignore python/.eggs

[ https://issues.apache.org/jira/browse/SPARK-28649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28649. --- Resolution: Fixed Fix Version/s: 2.4.4 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-28617) Fix misplacement when comment is at the end of the query

[ https://issues.apache.org/jira/browse/SPARK-28617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28617: - Assignee: Yuming Wang > Fix misplacement when comment is at the end of the query >

[jira] [Resolved] (SPARK-28617) Fix misplacement when comment is at the end of the query

[ https://issues.apache.org/jira/browse/SPARK-28617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28617. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25357

[jira] [Resolved] (SPARK-28331) Catalogs.load always throws CatalogNotFoundException on loading built-in catalogs

[ https://issues.apache.org/jira/browse/SPARK-28331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-28331. - Resolution: Fixed Fix Version/s: 3.0.0 Resolved with 

[jira] [Assigned] (SPARK-28331) Catalogs.load always throws CatalogNotFoundException on loading built-in catalogs

[ https://issues.apache.org/jira/browse/SPARK-28331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz reassigned SPARK-28331: --- Assignee: Gengliang Wang > Catalogs.load always throws CatalogNotFoundException on loading

[jira] [Resolved] (SPARK-28470) Honor spark.sql.decimalOperations.nullOnOverflow in Cast

[ https://issues.apache.org/jira/browse/SPARK-28470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28470. -- Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 3.0.0 Resolved

[jira] [Created] (SPARK-28652) spark.kubernetes.pyspark.pythonVersion is never passed to executors

nanav yorbiz created SPARK-28652: Summary: spark.kubernetes.pyspark.pythonVersion is never passed to executors Key: SPARK-28652 URL: https://issues.apache.org/jira/browse/SPARK-28652 Project: Spark

[jira] [Assigned] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-28651: Assignee: Shixiong Zhu > Streaming file source doesn't change the schema to nullable

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Labels: release-notes (was: ) > Streaming file source doesn't change the schema to nullable

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Description: Right now, batch DataFrame always changes the schema to nullable automatically

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Description: Right now, batch DataFrame always changes the schema to nullable automatically

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Reporter: Tomasz (was: Shixiong Zhu) > Streaming file source doesn't change the schema to

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Description: Right now, batch DataFrame always changes the schema to nullable automatically

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Description: Right now, batch DataFrame always changes the schema to nullable automatically

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Description: Right now, batch DataFrame always changes the schema to nullable automatically

[jira] [Created] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

Shixiong Zhu created SPARK-28651: Summary: Streaming file source doesn't change the schema to nullable automatically Key: SPARK-28651 URL: https://issues.apache.org/jira/browse/SPARK-28651 Project:

[jira] [Updated] (SPARK-27492) GPU scheduling - High level user documentation

[ https://issues.apache.org/jira/browse/SPARK-27492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27492: -- Description: For the SPIP - Accelerator-aware task scheduling for Spark, 

[jira] [Comment Edited] (SPARK-26152) Synchronize Worker Cleanup with Worker Shutdown

[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902477#comment-16902477 ] Shixiong Zhu edited comment on SPARK-26152 at 8/7/19 9:07 PM: --

[jira] [Commented] (SPARK-26152) Synchronize Worker Cleanup with Worker Shutdown

[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902477#comment-16902477 ] Shixiong Zhu commented on SPARK-26152: -- [~ajithshetty] does your PR fix the flaky test? If I read

[jira] [Created] (SPARK-28650) Fix the guarantee of ForeachWriter

Shixiong Zhu created SPARK-28650: Summary: Fix the guarantee of ForeachWriter Key: SPARK-28650 URL: https://issues.apache.org/jira/browse/SPARK-28650 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-28583) Subqueries should not call `onUpdatePlan` in Adaptive Query Execution

[ https://issues.apache.org/jira/browse/SPARK-28583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-28583. --- Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 3.0.0 >

[jira] [Created] (SPARK-28649) Git Ignore does not ignore python/.eggs

Rob Vesse created SPARK-28649: - Summary: Git Ignore does not ignore python/.eggs Key: SPARK-28649 URL: https://issues.apache.org/jira/browse/SPARK-28649 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-28648) Adds support to `groups` unit type in window clauses

[ https://issues.apache.org/jira/browse/SPARK-28648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-28648: - Summary: Adds support to `groups` unit type in window clauses (was: Adds support to `groups`

[jira] [Created] (SPARK-28648) Adds support to `groups` in window clauses

Dylan Guedes created SPARK-28648: Summary: Adds support to `groups` in window clauses Key: SPARK-28648 URL: https://issues.apache.org/jira/browse/SPARK-28648 Project: Spark Issue Type:

[jira] [Created] (SPARK-28647) Remove additional-metrics.js

Kousuke Saruta created SPARK-28647: -- Summary: Remove additional-metrics.js Key: SPARK-28647 URL: https://issues.apache.org/jira/browse/SPARK-28647 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-28124) Faster S3 file source with SQS

[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902382#comment-16902382 ] Abhishek Dixit edited comment on SPARK-28124 at 8/7/19 6:16 PM: As

[jira] [Comment Edited] (SPARK-28124) Faster S3 file source with SQS

[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902382#comment-16902382 ] Abhishek Dixit edited comment on SPARK-28124 at 8/7/19 6:15 PM: As

[jira] [Resolved] (SPARK-28124) Faster S3 file source with SQS

[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit resolved SPARK-28124. Resolution: Feedback Received As suggested by [~gsomogyi], [~ste...@apache.org],

[jira] [Created] (SPARK-28646) Allow usage of `count` only for parameterless aggregate function

Dylan Guedes created SPARK-28646: Summary: Allow usage of `count` only for parameterless aggregate function Key: SPARK-28646 URL: https://issues.apache.org/jira/browse/SPARK-28646 Project: Spark

[jira] [Updated] (SPARK-28645) Throw an error on window redefinition

[ https://issues.apache.org/jira/browse/SPARK-28645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-28645: - Summary: Throw an error on window redefinition (was: Block redefinition of window) > Throw an

[jira] [Created] (SPARK-28645) Block redefinition of window

Dylan Guedes created SPARK-28645: Summary: Block redefinition of window Key: SPARK-28645 URL: https://issues.apache.org/jira/browse/SPARK-28645 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-28634) Failed to start SparkSession with Keytab file

[ https://issues.apache.org/jira/browse/SPARK-28634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902238#comment-16902238 ] Marcelo Vanzin commented on SPARK-28634: Ah. If you use {{--principal}} and {{--keytab}} this

[jira] [Comment Edited] (SPARK-28634) Failed to start SparkSession with Keytab file

[ https://issues.apache.org/jira/browse/SPARK-28634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902238#comment-16902238 ] Marcelo Vanzin edited comment on SPARK-28634 at 8/7/19 4:48 PM: Ah. If

[jira] [Updated] (SPARK-28644) Port HIVE-10646: ColumnValue does not handle NULL_TYPE

[ https://issues.apache.org/jira/browse/SPARK-28644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28644: Description: Port HIVE-10646 to fix Hive 0.12's beeline can not handle NULL_TYPE: {code:sql}

[jira] [Updated] (SPARK-28644) Port HIVE-10646: ColumnValue does not handle NULL_TYPE

[ https://issues.apache.org/jira/browse/SPARK-28644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28644: Description: Port HIVE-10646 to fix Hive 0.12's JDBC client can not handle NULL_TYPE: {code:sql}

[jira] [Created] (SPARK-28644) Port HIVE-10646: ColumnValue does not handle NULL_TYPE

Yuming Wang created SPARK-28644: --- Summary: Port HIVE-10646: ColumnValue does not handle NULL_TYPE Key: SPARK-28644 URL: https://issues.apache.org/jira/browse/SPARK-28644 Project: Spark Issue

[jira] [Resolved] (SPARK-28472) Add a test for testing different protocol versions

[ https://issues.apache.org/jira/browse/SPARK-28472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28472. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > Add a test for testing

[jira] [Created] (SPARK-28643) Use Java 8 time API in Dataset.show

Maxim Gekk created SPARK-28643: -- Summary: Use Java 8 time API in Dataset.show Key: SPARK-28643 URL: https://issues.apache.org/jira/browse/SPARK-28643 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely no

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely no

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902076#comment-16902076 ] Thomas Graves commented on SPARK-27495: --- Thanks for the comments. # I was trying to leave the API

[jira] [Resolved] (SPARK-26667) Add `Scanning Input Table` to Performance Tuning Guide

[ https://issues.apache.org/jira/browse/SPARK-26667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26667. --- Resolution: Won't Fix > Add `Scanning Input Table` to Performance Tuning Guide >

[jira] [Resolved] (SPARK-28595) explain should not trigger partition listing

[ https://issues.apache.org/jira/browse/SPARK-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28595. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25328

[jira] [Updated] (SPARK-28642) Hide credentials in show create table

[ https://issues.apache.org/jira/browse/SPARK-28642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28642: Description: {code:sql} spark-sql> show create table mysql_federated_sample; CREATE TABLE

[jira] [Created] (SPARK-28642) Hide credentials in show create table

Yuming Wang created SPARK-28642: --- Summary: Hide credentials in show create table Key: SPARK-28642 URL: https://issues.apache.org/jira/browse/SPARK-28642 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

MariaCarrie created SPARK-28641: --- Summary: MicroBatchExecution committed offsets greater than available offsets Key: SPARK-28641 URL: https://issues.apache.org/jira/browse/SPARK-28641 Project: Spark

[jira] [Commented] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901908#comment-16901908 ] angerszhu commented on SPARK-11248: --- I have make a patch for this problem.    

[jira] [Comment Edited] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901847#comment-16901847 ] angerszhu edited comment on SPARK-21918 at 8/7/19 7:58 AM: --- .    

[jira] [Comment Edited] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901847#comment-16901847 ] angerszhu edited comment on SPARK-21918 at 8/7/19 7:55 AM: --- .    

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901846#comment-16901846 ] angerszhu commented on SPARK-5159: -- I have make a patch for this problem.    

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901847#comment-16901847 ] angerszhu commented on SPARK-21918: --- I have make a patch for this problem.    

[jira] [Resolved] (SPARK-27916) SparkThreatServer memory leak when 'spark.sql.hive.thriftServer.singleSession' enabled

[ https://issues.apache.org/jira/browse/SPARK-27916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-27916. --- Resolution: Invalid > SparkThreatServer memory leak when >

[jira] [Updated] (SPARK-28169) Spark can’t push down partition predicate for OR expression

[ https://issues.apache.org/jira/browse/SPARK-28169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28169: -- Summary: Spark can’t push down partition predicate for OR expression (was: Spark can’t push down

[jira] [Resolved] (SPARK-28238) DESCRIBE TABLE for Data Source V2 tables

[ https://issues.apache.org/jira/browse/SPARK-28238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28238. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25040

[jira] [Assigned] (SPARK-28238) DESCRIBE TABLE for Data Source V2 tables

[ https://issues.apache.org/jira/browse/SPARK-28238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28238: --- Assignee: Matt Cheah > DESCRIBE TABLE for Data Source V2 tables >