[jira] [Created] (SPARK-37607) Optimized antlr4 syntax file

2021-12-09 Thread melin (Jira)
melin created SPARK-37607: - Summary: Optimized antlr4 syntax file Key: SPARK-37607 URL: https://issues.apache.org/jira/browse/SPARK-37607 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-37603) org.apache.spark.sql.catalyst.expressions.ListQuery; no valid constructor

2021-12-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456920#comment-17456920 ] Yuming Wang commented on SPARK-37603: - How to reproduce this issue. >

[jira] [Commented] (SPARK-37606) ALTER NAMESPACE ... SET PROPERTIES v2 command should use existing properties in addition to new properties

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456900#comment-17456900 ] Apache Spark commented on SPARK-37606: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37606) ALTER NAMESPACE ... SET PROPERTIES v2 command should use existing properties in addition to new properties

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37606: Assignee: Apache Spark > ALTER NAMESPACE ... SET PROPERTIES v2 command should use

[jira] [Assigned] (SPARK-37606) ALTER NAMESPACE ... SET PROPERTIES v2 command should use existing properties in addition to new properties

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37606: Assignee: (was: Apache Spark) > ALTER NAMESPACE ... SET PROPERTIES v2 command should

[jira] [Assigned] (SPARK-36588) Use v2 commands by default

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36588: Assignee: Apache Spark (was: Terry Kim) > Use v2 commands by default >

[jira] [Assigned] (SPARK-36588) Use v2 commands by default

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36588: Assignee: Terry Kim (was: Apache Spark) > Use v2 commands by default >

[jira] [Commented] (SPARK-36588) Use v2 commands by default

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456897#comment-17456897 ] Apache Spark commented on SPARK-36588: -- User 'imback82' has created a pull request for this issue:

[jira] [Created] (SPARK-37606) ALTER NAMESPACE ... SET PROPERTIES v2 command should use existing properties in addition to new properties

2021-12-09 Thread Terry Kim (Jira)
Terry Kim created SPARK-37606: - Summary: ALTER NAMESPACE ... SET PROPERTIES v2 command should use existing properties in addition to new properties Key: SPARK-37606 URL:

[jira] [Assigned] (SPARK-37604) The parameter emptyValueInRead in CSVOptions is not designed as supposed to be

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37604: Assignee: (was: Apache Spark) > The parameter emptyValueInRead in CSVOptions is not

[jira] [Commented] (SPARK-37604) The parameter emptyValueInRead in CSVOptions is not designed as supposed to be

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456891#comment-17456891 ] Apache Spark commented on SPARK-37604: -- User 'wayneguow' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37604) The parameter emptyValueInRead in CSVOptions is not designed as supposed to be

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37604: Assignee: Apache Spark > The parameter emptyValueInRead in CSVOptions is not designed as

[jira] [Commented] (SPARK-37605) Support the configuration of the initial number of scan partitions when executing a take on a query

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456884#comment-17456884 ] Apache Spark commented on SPARK-37605: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37605) Support the configuration of the initial number of scan partitions when executing a take on a query

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37605: Assignee: Apache Spark > Support the configuration of the initial number of scan

[jira] [Commented] (SPARK-37605) Support the configuration of the initial number of scan partitions when executing a take on a query

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456883#comment-17456883 ] Apache Spark commented on SPARK-37605: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37605) Support the configuration of the initial number of scan partitions when executing a take on a query

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37605: Assignee: (was: Apache Spark) > Support the configuration of the initial number of

[jira] [Created] (SPARK-37605) Support the configuration of the initial number of scan partitions when executing a take on a query

2021-12-09 Thread dzcxzl (Jira)
dzcxzl created SPARK-37605: -- Summary: Support the configuration of the initial number of scan partitions when executing a take on a query Key: SPARK-37605 URL: https://issues.apache.org/jira/browse/SPARK-37605

[jira] [Updated] (SPARK-37604) The parameter emptyValueInRead in CSVOptions is not designed as supposed to be

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guo Wei updated SPARK-37604: Description: For null values, the parameter nullValue can be set when reading or writing  in CSVOptions:

[jira] [Updated] (SPARK-37604) The parameter emptyValueInRead in CSVOptions is not designed as supposed to be

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guo Wei updated SPARK-37604: Summary: The parameter emptyValueInRead in CSVOptions is not designed as supposed to be (was: The

[jira] [Updated] (SPARK-37604) The parameter emptyValueInRead is CSVOptions is not designed as supposed to be

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guo Wei updated SPARK-37604: Description: For null values, the parameter nullValue can be set when reading or writing  in CSVOptions:

[jira] [Updated] (SPARK-37604) The parameter emptyValueInRead is CSVOptions is not designed as supposed to be

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guo Wei updated SPARK-37604: Description: For null values, the parameter nullValue can be set when reading or writing  in CSVOptions:

[jira] [Created] (SPARK-37604) The parameter emptyValueInRead is CSVOptions is not designed as supposed to be

2021-12-09 Thread Guo Wei (Jira)
Guo Wei created SPARK-37604: --- Summary: The parameter emptyValueInRead is CSVOptions is not designed as supposed to be Key: SPARK-37604 URL: https://issues.apache.org/jira/browse/SPARK-37604 Project: Spark

[jira] [Updated] (SPARK-37604) The parameter emptyValueInRead is CSVOptions is not designed as supposed to be

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guo Wei updated SPARK-37604: Affects Version/s: 3.2.0 > The parameter emptyValueInRead is CSVOptions is not designed as supposed to be

[jira] [Resolved] (SPARK-37573) IsolatedClient fallbackVersion should be build in version, not always 2.7.4

2021-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-37573. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34830

[jira] [Assigned] (SPARK-37573) IsolatedClient fallbackVersion should be build in version, not always 2.7.4

2021-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-37573: Assignee: angerszhu > IsolatedClient fallbackVersion should be build in version, not always

[jira] [Updated] (SPARK-37567) reuse Exchange failed

2021-12-09 Thread junbiao chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junbiao chen updated SPARK-37567: - Labels: bugfix performance pull-request-available (was: performance) > reuse Exchange failed

[jira] [Created] (SPARK-37603) org.apache.spark.sql.catalyst.expressions.ListQuery; no valid constructor

2021-12-09 Thread huangweiguo (Jira)
huangweiguo created SPARK-37603: --- Summary: org.apache.spark.sql.catalyst.expressions.ListQuery; no valid constructor Key: SPARK-37603 URL: https://issues.apache.org/jira/browse/SPARK-37603 Project:

[jira] [Updated] (SPARK-37567) reuse Exchange failed

2021-12-09 Thread junbiao chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junbiao chen updated SPARK-37567: - Description: PR available:https://github.com/apache/spark/pull/34858 use case:query2 in

[jira] [Updated] (SPARK-37567) reuse Exchange failed

2021-12-09 Thread junbiao chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junbiao chen updated SPARK-37567: - Description:   use case:query2 in TPC-DS.There are three exchange subquery will scan the same

[jira] [Updated] (SPARK-37567) reuse Exchange failed

2021-12-09 Thread junbiao chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junbiao chen updated SPARK-37567: - Description: PR :https://github.com/chenjunbiao001/spark/pull/1 use case:query2 in

[jira] [Resolved] (SPARK-37597) Deduplicate the right side of left-semi join and left-anti join

2021-12-09 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-37597. -- Resolution: Duplicate > Deduplicate the right side of left-semi join and left-anti join >

[jira] [Commented] (SPARK-26589) proper `median` method for spark dataframe

2021-12-09 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456860#comment-17456860 ] Nicholas Chammas commented on SPARK-26589: -- That makes sense to me. I've been struggling with

[jira] [Commented] (SPARK-36850) Migrate CreateTableStatement to v2 command framework

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456857#comment-17456857 ] Apache Spark commented on SPARK-36850: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-36850) Migrate CreateTableStatement to v2 command framework

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456856#comment-17456856 ] Apache Spark commented on SPARK-36850: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-37602) Add config property to set default Spark listeners

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456833#comment-17456833 ] Apache Spark commented on SPARK-37602: -- User 'shardulm94' has created a pull request for this

[jira] [Assigned] (SPARK-37602) Add config property to set default Spark listeners

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37602: Assignee: Apache Spark > Add config property to set default Spark listeners >

[jira] [Assigned] (SPARK-37602) Add config property to set default Spark listeners

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37602: Assignee: (was: Apache Spark) > Add config property to set default Spark listeners >

[jira] [Commented] (SPARK-37602) Add config property to set default Spark listeners

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456835#comment-17456835 ] Apache Spark commented on SPARK-37602: -- User 'shardulm94' has created a pull request for this

[jira] [Updated] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

2021-12-09 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37601: -- Description: {code:python} def foo(df: DataFrame, p: int) -> DataFrame ... # current:

[jira] [Updated] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

2021-12-09 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37601: -- Description: {code:python} def foo(df: DataFrame, p: int) -> DataFrame ... # current:

[jira] [Created] (SPARK-37602) Add config property to set default Spark listeners

2021-12-09 Thread Shardul Mahadik (Jira)
Shardul Mahadik created SPARK-37602: --- Summary: Add config property to set default Spark listeners Key: SPARK-37602 URL: https://issues.apache.org/jira/browse/SPARK-37602 Project: Spark

[jira] [Updated] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

2021-12-09 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37601: -- Description: {code:python} def foo(df: DataFrame, p: int) -> DataFrame ... # current:

[jira] [Updated] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

2021-12-09 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37601: -- Description: {code:python} def foo(df: DataFrame, p: int) -> DataFrame ... # current:

[jira] [Updated] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

2021-12-09 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37601: -- Description: {noformat} def foo(df: DataFrame, p: int) -> DataFrame ... # current: from

[jira] [Updated] (SPARK-19256) Hive bucketing write support

2021-12-09 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-19256: - Affects Version/s: 3.3.0 > Hive bucketing write support > > >

[jira] [Created] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

2021-12-09 Thread Rafal Wojdyla (Jira)
Rafal Wojdyla created SPARK-37601: - Summary: Could/should sql.DataFrame.transform accept function parameters? Key: SPARK-37601 URL: https://issues.apache.org/jira/browse/SPARK-37601 Project: Spark

[jira] [Assigned] (SPARK-37600) Upgrade to Hadoop 3.3.2

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37600: Assignee: (was: Apache Spark) > Upgrade to Hadoop 3.3.2 > --- >

[jira] [Commented] (SPARK-37600) Upgrade to Hadoop 3.3.2

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456694#comment-17456694 ] Apache Spark commented on SPARK-37600: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37600) Upgrade to Hadoop 3.3.2

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37600: Assignee: Apache Spark > Upgrade to Hadoop 3.3.2 > --- > >

[jira] (SPARK-37575) Null values are saved as quoted empty Strings "" rather than nothing

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37575 ] Guo Wei deleted comment on SPARK-37575: - was (Author: wayne guo): I also found that if emptyValueInRead set to "\"\"", reading csv data as show below: {noformat} name,brand,comment

[jira] [Updated] (SPARK-37575) Null values are saved as quoted empty Strings "" rather than nothing

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guo Wei updated SPARK-37575: Summary: Null values are saved as quoted empty Strings "" rather than nothing (was: Empty strings and

[jira] [Created] (SPARK-37600) Upgrade to Hadoop 3.3.2

2021-12-09 Thread Chao Sun (Jira)
Chao Sun created SPARK-37600: Summary: Upgrade to Hadoop 3.3.2 Key: SPARK-37600 URL: https://issues.apache.org/jira/browse/SPARK-37600 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-34332) Unify v1 and v2 ALTER NAMESPACE .. SET LOCATION tests

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456676#comment-17456676 ] Apache Spark commented on SPARK-34332: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-34332) Unify v1 and v2 ALTER NAMESPACE .. SET LOCATION tests

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456674#comment-17456674 ] Apache Spark commented on SPARK-34332: -- User 'imback82' has created a pull request for this issue:

[jira] [Created] (SPARK-37599) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-12-09 Thread Terry Kim (Jira)
Terry Kim created SPARK-37599: - Summary: Unify v1 and v2 ALTER TABLE .. SET LOCATION tests Key: SPARK-37599 URL: https://issues.apache.org/jira/browse/SPARK-37599 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37599) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-12-09 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-37599: -- Description: Extract ALTER TABLE .. SET LOCATION tests to the common place to run them for V1 and v2

[jira] [Updated] (SPARK-34332) Unify v1 and v2 ALTER NAMESPACE .. SET LOCATION tests

2021-12-09 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-34332: -- Summary: Unify v1 and v2 ALTER NAMESPACE .. SET LOCATION tests (was: Unify v1 and v2 ALTER TABLE ..

[jira] [Updated] (SPARK-34332) Unify v1 and v2 ALTER NAMESPACE .. SET LOCATION tests

2021-12-09 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-34332: -- Description: Extract ALTER NAMESPACE .. SET LOCATION tests to the common place to run them for V1 and

[jira] [Assigned] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37575: Assignee: (was: Apache Spark) > Empty strings and null values are both saved as

[jira] [Assigned] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37575: Assignee: Apache Spark > Empty strings and null values are both saved as quoted empty

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456662#comment-17456662 ] Apache Spark commented on SPARK-37575: -- User 'wayneguow' has created a pull request for this issue:

[jira] [Commented] (SPARK-37598) Pyspark's newAPIHadoopRDD() method fails with ShortWritables

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456623#comment-17456623 ] Apache Spark commented on SPARK-37598: -- User 'masseyke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37598) Pyspark's newAPIHadoopRDD() method fails with ShortWritables

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37598: Assignee: Apache Spark > Pyspark's newAPIHadoopRDD() method fails with ShortWritables >

[jira] [Commented] (SPARK-37598) Pyspark's newAPIHadoopRDD() method fails with ShortWritables

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456622#comment-17456622 ] Apache Spark commented on SPARK-37598: -- User 'masseyke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37598) Pyspark's newAPIHadoopRDD() method fails with ShortWritables

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37598: Assignee: (was: Apache Spark) > Pyspark's newAPIHadoopRDD() method fails with

[jira] [Created] (SPARK-37598) Pyspark's newAPIHadoopRDD() method fails with ShortWritables

2021-12-09 Thread Keith Massey (Jira)
Keith Massey created SPARK-37598: Summary: Pyspark's newAPIHadoopRDD() method fails with ShortWritables Key: SPARK-37598 URL: https://issues.apache.org/jira/browse/SPARK-37598 Project: Spark

[jira] [Closed] (SPARK-37589) Improve the performance of PlanParserSuite

2021-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37589. - > Improve the performance of PlanParserSuite > -- > >

[jira] [Resolved] (SPARK-37589) Improve the performance of PlanParserSuite

2021-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37589. --- Assignee: (was: Apache Spark) Resolution: Won't Do This is closed according to

[jira] [Commented] (SPARK-37578) DSV2 is not updating Output Metrics

2021-12-09 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456589#comment-17456589 ] L. C. Hsieh commented on SPARK-37578: - Thanks [~cloud_fan]. I will take a look. > DSV2 is not

[jira] [Assigned] (SPARK-37591) Support the GCM mode by aes_encrypt()/aes_decrypt()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37591: Assignee: Apache Spark (was: Max Gekk) > Support the GCM mode by

[jira] [Commented] (SPARK-37591) Support the GCM mode by aes_encrypt()/aes_decrypt()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456587#comment-17456587 ] Apache Spark commented on SPARK-37591: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37591) Support the GCM mode by aes_encrypt()/aes_decrypt()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37591: Assignee: Max Gekk (was: Apache Spark) > Support the GCM mode by

[jira] [Resolved] (SPARK-37451) Performance improvement regressed String to Decimal cast

2021-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37451. --- Fix Version/s: 3.1.3 3.2.1 Assignee: Yuming Wang

[jira] [Comment Edited] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

2021-12-09 Thread Guo Wei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456167#comment-17456167 ] Guo Wei edited comment on SPARK-37575 at 12/9/21, 4:37 PM: --- I also found that

[jira] [Commented] (SPARK-28955) Support for LocalDateTime semantics

2021-12-09 Thread Bill Schneider (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456539#comment-17456539 ] Bill Schneider commented on SPARK-28955: I see a bunch of related tickets with fix version 3.3.0

[jira] [Resolved] (SPARK-37584) New SQL function: map_contains_key

2021-12-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-37584. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34836

[jira] [Updated] (SPARK-37588) lot of strings get accumulated in the heap dump of spark thrift server

2021-12-09 Thread ramakrishna chilaka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramakrishna chilaka updated SPARK-37588: Description: I am starting spark thrift server using the following options ```

[jira] [Updated] (SPARK-37588) lot of strings get accumulated in the heap dump of spark thrift server

2021-12-09 Thread ramakrishna chilaka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramakrishna chilaka updated SPARK-37588: Description: I am starting spark thrift server using the following options ```

[jira] [Updated] (SPARK-37588) lot of strings get accumulated in the heap dump of spark thrift server

2021-12-09 Thread ramakrishna chilaka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramakrishna chilaka updated SPARK-37588: Attachment: screenshot-1.png > lot of strings get accumulated in the heap dump of

[jira] [Commented] (SPARK-37451) Performance improvement regressed String to Decimal cast

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456431#comment-17456431 ] Apache Spark commented on SPARK-37451: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-37597) Deduplicate the right side of left-semi join and left-anti join

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456423#comment-17456423 ] Apache Spark commented on SPARK-37597: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-37597) Deduplicate the right side of left-semi join and left-anti join

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37597: Assignee: Apache Spark > Deduplicate the right side of left-semi join and left-anti join

[jira] [Assigned] (SPARK-37597) Deduplicate the right side of left-semi join and left-anti join

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37597: Assignee: Apache Spark > Deduplicate the right side of left-semi join and left-anti join

[jira] [Assigned] (SPARK-37597) Deduplicate the right side of left-semi join and left-anti join

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37597: Assignee: (was: Apache Spark) > Deduplicate the right side of left-semi join and

[jira] [Created] (SPARK-37597) Deduplicate the right side of left-semi join and left-anti join

2021-12-09 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-37597: Summary: Deduplicate the right side of left-semi join and left-anti join Key: SPARK-37597 URL: https://issues.apache.org/jira/browse/SPARK-37597 Project: Spark

[jira] [Assigned] (SPARK-37596) Add the support for struct type column in the DropDuplicate in spark

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37596: Assignee: (was: Apache Spark) > Add the support for struct type column in the

[jira] [Commented] (SPARK-37596) Add the support for struct type column in the DropDuplicate in spark

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456351#comment-17456351 ] Apache Spark commented on SPARK-37596: -- User 'SaurabhChawla100' has created a pull request for this

[jira] [Assigned] (SPARK-37596) Add the support for struct type column in the DropDuplicate in spark

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37596: Assignee: Apache Spark > Add the support for struct type column in the DropDuplicate in

[jira] [Created] (SPARK-37596) Add the support for struct type column in the DropDuplicate in spark

2021-12-09 Thread Saurabh Chawla (Jira)
Saurabh Chawla created SPARK-37596: -- Summary: Add the support for struct type column in the DropDuplicate in spark Key: SPARK-37596 URL: https://issues.apache.org/jira/browse/SPARK-37596 Project:

[jira] [Assigned] (SPARK-37583) Support the binary type by startswith() and endswith()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37583: Assignee: (was: Apache Spark) > Support the binary type by startswith() and

[jira] [Updated] (SPARK-37581) sql hang at planning stage

2021-12-09 Thread ocean (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ocean updated SPARK-37581: -- Description: when exec a sql, this sql hang at planning stage. when disable DPP, sql can finish very

[jira] [Updated] (SPARK-37581) sql hang at planning stage

2021-12-09 Thread ocean (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ocean updated SPARK-37581: -- Description: when exec a sql, this sql hang at planning stage. when disable DPP, sql can finish very

[jira] [Commented] (SPARK-37583) Support the binary type by startswith() and endswith()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456344#comment-17456344 ] Apache Spark commented on SPARK-37583: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-37583) Support the binary type by startswith() and endswith()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456342#comment-17456342 ] Apache Spark commented on SPARK-37583: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-37583) Support the binary type by startswith() and endswith()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456343#comment-17456343 ] Apache Spark commented on SPARK-37583: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37583) Support the binary type by startswith() and endswith()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37583: Assignee: Apache Spark > Support the binary type by startswith() and endswith() >

[jira] [Commented] (SPARK-37582) Support the binary type by contains()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456340#comment-17456340 ] Apache Spark commented on SPARK-37582: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37582) Support the binary type by contains()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37582: Assignee: Apache Spark > Support the binary type by contains() >

[jira] [Assigned] (SPARK-37582) Support the binary type by contains()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37582: Assignee: (was: Apache Spark) > Support the binary type by contains() >

[jira] [Commented] (SPARK-37582) Support the binary type by contains()

2021-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456339#comment-17456339 ] Apache Spark commented on SPARK-37582: -- User 'AngersZh' has created a pull request for this

  1   2   >