[jira] [Assigned] (SPARK-34383) Optimize WAL commit phase on SS

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34383: Assignee: (was: Apache Spark) > Optimize WAL commit phase on SS >

[jira] [Assigned] (SPARK-34383) Optimize WAL commit phase on SS

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34383: Assignee: Apache Spark > Optimize WAL commit phase on SS >

[jira] [Commented] (SPARK-34383) Optimize WAL commit phase on SS

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280075#comment-17280075 ] Apache Spark commented on SPARK-34383: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Created] (SPARK-34383) Optimize WAL commit phase on SS

2021-02-05 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-34383: Summary: Optimize WAL commit phase on SS Key: SPARK-34383 URL: https://issues.apache.org/jira/browse/SPARK-34383 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-33410) Resolve SQL query reference a column by an alias

2021-02-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33410: Description: This pr add support resolve SQL query reference a column by an alias, for example:

[jira] [Commented] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

2021-02-05 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280051#comment-17280051 ] Xiao Li commented on SPARK-34382: - reopened. This is a nice SQL feature we can support. This is also

[jira] [Created] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

2021-02-05 Thread Xiao Li (Jira)
Xiao Li created SPARK-34382: --- Summary: ANSI SQL: LATERAL derived table(T491) Key: SPARK-34382 URL: https://issues.apache.org/jira/browse/SPARK-34382 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-27877) ANSI SQL: LATERAL derived table(T491)

2021-02-05 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27877: Issue Type: Technical task (was: Sub-task) > ANSI SQL: LATERAL derived table(T491) >

[jira] [Updated] (SPARK-27877) ANSI SQL: LATERAL derived table(T491)

2021-02-05 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27877: Issue Type: Sub-task (was: Technical task) > ANSI SQL: LATERAL derived table(T491) >

[jira] [Updated] (SPARK-34346) io.file.buffer.size set by spark.buffer.size will override by hive-site.xml may cause perf regression

2021-02-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34346: -- Affects Version/s: 3.0.2 > io.file.buffer.size set by spark.buffer.size will override by

[jira] [Resolved] (SPARK-34346) io.file.buffer.size set by spark.buffer.size will override by hive-site.xml may cause perf regression

2021-02-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34346. --- Fix Version/s: 3.1.1 3.0.2 Assignee: Kent Yao Resolution:

[jira] [Commented] (SPARK-18591) Replace hash-based aggregates with sort-based ones if inputs already sorted

2021-02-05 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279995#comment-17279995 ] Cheng Su commented on SPARK-18591: -- Just came across this Jira. We added the same feature 2 years ago

[jira] [Commented] (SPARK-34380) Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279982#comment-17279982 ] Apache Spark commented on SPARK-34380: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-34380) Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279981#comment-17279981 ] Apache Spark commented on SPARK-34380: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34380) Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34380: Assignee: Apache Spark > Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES >

[jira] [Assigned] (SPARK-34380) Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34380: Assignee: (was: Apache Spark) > Support ifExists for ALTER TABLE ... UNSET

[jira] [Updated] (SPARK-34381) c

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Summary: c (was: Encoding is not working if multiLine option is true. Spark 2.4.0) > c > - >

[jira] [Updated] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Attachment: (was: hive.PNG) > Encoding is not working if multiLine option is true. Spark

[jira] [Updated] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Attachment: (was: spark.png) > Encoding is not working if multiLine option is true. Spark

[jira] [Resolved] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes resolved SPARK-34381. -- Resolution: Fixed > Encoding is not working if multiLine option is true. Spark 2.4.0 >

[jira] [Updated] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Attachment: (was: csv.PNG) > Encoding is not working if multiLine option is true. Spark

[jira] [Updated] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Description: (was: I have a csv file with multiline and encoding ISO=8859-1, but if I

[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2021-02-05 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279947#comment-17279947 ] Cheng Su commented on SPARK-33207: -- [~yumwang] - just an update, after

[jira] [Updated] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2021-02-05 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-33207: - Attachment: Screen Shot 2021-02-05 at 11.44.12 AM.png > Reduce the number of tasks launched after

[jira] [Updated] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Description: I have a csv file with multiline and encoding ISO=8859-1, but if I enable

[jira] [Updated] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Fortes updated SPARK-34381: - Attachment: spark.png csv.PNG hive.PNG > Encoding is not

[jira] [Created] (SPARK-34381) Encoding is not working if multiLine option is true. Spark 2.4.0

2021-02-05 Thread Bruno Fortes (Jira)
Bruno Fortes created SPARK-34381: Summary: Encoding is not working if multiLine option is true. Spark 2.4.0 Key: SPARK-34381 URL: https://issues.apache.org/jira/browse/SPARK-34381 Project: Spark

[jira] [Comment Edited] (SPARK-33501) Encoding is not working if multiLine option is true.

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279928#comment-17279928 ] Bruno Fortes edited comment on SPARK-33501 at 2/5/21, 7:19 PM: --- Hey, did

[jira] [Commented] (SPARK-33501) Encoding is not working if multiLine option is true.

2021-02-05 Thread Bruno Fortes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279928#comment-17279928 ] Bruno Fortes commented on SPARK-33501: -- Hey, did you guys find any solution for this issue? I'm

[jira] [Assigned] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2021-02-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-26836: - Assignee: Attila Zsolt Piros > Columns get switched in Spark SQL using Avro backed

[jira] [Resolved] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2021-02-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26836. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31133

[jira] [Commented] (SPARK-34363) Allow users to configure a maximum amount of remote shuffle block storage

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279899#comment-17279899 ] Apache Spark commented on SPARK-34363: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-34363) Allow users to configure a maximum amount of remote shuffle block storage

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279898#comment-17279898 ] Apache Spark commented on SPARK-34363: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34363) Allow users to configure a maximum amount of remote shuffle block storage

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34363: Assignee: (was: Apache Spark) > Allow users to configure a maximum amount of remote

[jira] [Assigned] (SPARK-34363) Allow users to configure a maximum amount of remote shuffle block storage

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34363: Assignee: Apache Spark > Allow users to configure a maximum amount of remote shuffle

[jira] [Commented] (SPARK-34346) io.file.buffer.size set by spark.buffer.size will override by hive-site.xml may cause perf regression

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279896#comment-17279896 ] Apache Spark commented on SPARK-34346: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-34380) Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES

2021-02-05 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279893#comment-17279893 ] Terry Kim commented on SPARK-34380: --- Looking > Support ifExists for ALTER TABLE ... UNSET

[jira] [Commented] (SPARK-34346) io.file.buffer.size set by spark.buffer.size will override by hive-site.xml may cause perf regression

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279894#comment-17279894 ] Apache Spark commented on SPARK-34346: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Created] (SPARK-34380) Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES

2021-02-05 Thread Terry Kim (Jira)
Terry Kim created SPARK-34380: - Summary: Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES Key: SPARK-34380 URL: https://issues.apache.org/jira/browse/SPARK-34380 Project: Spark Issue

[jira] [Commented] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279892#comment-17279892 ] Apache Spark commented on SPARK-34379: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34379: Assignee: Kousuke Saruta (was: Apache Spark) > Map JDBC RowID to StringType rather than

[jira] [Assigned] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34379: Assignee: Apache Spark (was: Kousuke Saruta) > Map JDBC RowID to StringType rather than

[jira] [Commented] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279891#comment-17279891 ] Apache Spark commented on SPARK-34379: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34379: --- Description: In the current implementation, JDBC RowID type is mapped to LongType except

[jira] [Created] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34379: -- Summary: Map JDBC RowID to StringType rather than LongType Key: SPARK-34379 URL: https://issues.apache.org/jira/browse/SPARK-34379 Project: Spark Issue

[jira] [Updated] (SPARK-34379) Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34379: --- Issue Type: Bug (was: Improvement) > Map JDBC RowID to StringType rather than LongType >

[jira] [Assigned] (SPARK-34365) Support configurable Avro schema field matching for positional or by-name

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34365: Assignee: (was: Apache Spark) > Support configurable Avro schema field matching for

[jira] [Commented] (SPARK-34365) Support configurable Avro schema field matching for positional or by-name

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279862#comment-17279862 ] Apache Spark commented on SPARK-34365: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34365) Support configurable Avro schema field matching for positional or by-name

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34365: Assignee: Apache Spark > Support configurable Avro schema field matching for positional

[jira] [Commented] (SPARK-34134) LDAP authentication of spark thrift server support user id mapping

2021-02-05 Thread Timothy Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279847#comment-17279847 ] Timothy Zhang commented on SPARK-34134: --- Almost applications I used can support it, such as

[jira] [Commented] (SPARK-34378) Support extra optional Avro fields in AvroSerializer

2021-02-05 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279839#comment-17279839 ] Erik Krogen commented on SPARK-34378: - Internally we build this feature on top of SPARK-34365, so I

[jira] [Created] (SPARK-34378) Support extra optional Avro fields in AvroSerializer

2021-02-05 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-34378: --- Summary: Support extra optional Avro fields in AvroSerializer Key: SPARK-34378 URL: https://issues.apache.org/jira/browse/SPARK-34378 Project: Spark Issue

[jira] [Commented] (SPARK-34298) SaveMode.Overwrite not usable when using s3a root paths

2021-02-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279798#comment-17279798 ] Steve Loughran commented on SPARK-34298: well, I'm sure a PR with tests will get reviewed... >

[jira] [Commented] (SPARK-34377) Support parquet datasource options to control datetime rebasing in read

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279731#comment-17279731 ] Apache Spark commented on SPARK-34377: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34377) Support parquet datasource options to control datetime rebasing in read

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34377: Assignee: (was: Apache Spark) > Support parquet datasource options to control

[jira] [Assigned] (SPARK-34377) Support parquet datasource options to control datetime rebasing in read

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34377: Assignee: Apache Spark > Support parquet datasource options to control datetime rebasing

[jira] [Created] (SPARK-34377) Support parquet datasource options to control datetime rebasing in read

2021-02-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34377: -- Summary: Support parquet datasource options to control datetime rebasing in read Key: SPARK-34377 URL: https://issues.apache.org/jira/browse/SPARK-34377 Project: Spark

[jira] [Resolved] (SPARK-34350) replace withTimeZone defined in OracleIntegrationSuite with DateTimeTestUtils.withDefaultTimeZone

2021-02-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-34350. -- Fix Version/s: 3.2.0 Resolution: Fixed Resolved by 

[jira] [Resolved] (SPARK-32985) Decouple bucket filter pruning and bucket table scan

2021-02-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32985. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31413

[jira] [Assigned] (SPARK-32985) Decouple bucket filter pruning and bucket table scan

2021-02-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32985: --- Assignee: Cheng Su > Decouple bucket filter pruning and bucket table scan >

[jira] [Updated] (SPARK-34359) add a legacy config to restore the output schema of SHOW DATABASES

2021-02-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-34359: Fix Version/s: (was: 3.2.0) 3.1.1 3.0.2 > add a legacy

[jira] [Resolved] (SPARK-32698) Do not fall back to default parallelism if the minimum number of coalesced partitions is not set in AQE

2021-02-05 Thread Manu Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved SPARK-32698. Resolution: Won't Do > Do not fall back to default parallelism if the minimum number of coalesced

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279623#comment-17279623 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:44 AM: Caffeine is

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:45 AM: [~dongjoon] I

[jira] [Commented] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279623#comment-17279623 ] Yang Jie commented on SPARK-34309: -- Caffeine is different from guava cache in maximum size mechanism：

[jira] [Updated] (SPARK-34376) Support regexp as a function

2021-02-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34376: - Description: We have equality in SqlBase.g4 for RLIKE: 'RLIKE' | 'REGEXP'; We seemed to miss adding

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:20 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:19 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:13 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:16 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:20 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:19 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:15 AM: [~dongjoon] I

[jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:08 AM: [~dongjoon] I

[jira] [Commented] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279548#comment-17279548 ] Yang Jie commented on SPARK-34309: -- [~dongjoon] I use the code in

[jira] [Assigned] (SPARK-34375) Replaces `Mockito.initMocks` with `Mockito.openMocks`

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34375: Assignee: Apache Spark > Replaces `Mockito.initMocks` with `Mockito.openMocks` >

[jira] [Assigned] (SPARK-34375) Replaces `Mockito.initMocks` with `Mockito.openMocks`

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34375: Assignee: (was: Apache Spark) > Replaces `Mockito.initMocks` with

[jira] [Commented] (SPARK-34375) Replaces `Mockito.initMocks` with `Mockito.openMocks`

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279491#comment-17279491 ] Apache Spark commented on SPARK-34375: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-34309) Use Caffeine instead of Guava Cache

2021-02-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-34309: - Attachment: screenshot-1.png > Use Caffeine instead of Guava Cache >

[jira] [Commented] (SPARK-34359) add a legacy config to restore the output schema of SHOW DATABASES

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279447#comment-17279447 ] Apache Spark commented on SPARK-34359: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34376) Support regexp as a function

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34376: Assignee: (was: Apache Spark) > Support regexp as a function >

[jira] [Assigned] (SPARK-34376) Support regexp as a function

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34376: Assignee: Apache Spark > Support regexp as a function > > >

[jira] [Commented] (SPARK-34376) Support regexp as a function

2021-02-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279495#comment-17279495 ] Apache Spark commented on SPARK-34376: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Updated] (SPARK-34376) Support regexp as a function

2021-02-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34376: - Summary: Support regexp as a function (was: Support regexp as function) > Support regexp as a function

[jira] [Created] (SPARK-34376) Support regexp as function

2021-02-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-34376: Summary: Support regexp as function Key: SPARK-34376 URL: https://issues.apache.org/jira/browse/SPARK-34376 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-34375) Replaces `Mockito.initMocks` with `Mockito.openMocks`

2021-02-05 Thread Yang Jie (Jira)
Yang Jie created SPARK-34375: Summary: Replaces `Mockito.initMocks` with `Mockito.openMocks` Key: SPARK-34375 URL: https://issues.apache.org/jira/browse/SPARK-34375 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34376) Support regexp as function

2021-02-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34376: - Issue Type: New Feature (was: Bug) > Support regexp as function > -- > >

[jira] [Commented] (SPARK-34134) LDAP authentication of spark thrift server support user id mapping

2021-02-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279472#comment-17279472 ] angerszhu commented on SPARK-34134: --- Seems not a common usage. > LDAP authentication of spark thrift

[jira] [Resolved] (SPARK-34331) Speed up DS v2 metadata col resolution

2021-02-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34331. - Fix Version/s: 3.1.1 Resolution: Fixed Issue resolved by pull request 31440