[jira] [Updated] (SPARK-34727) Difference in results of casting float to timestamp

2021-03-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34727: - Fix Version/s: 3.1.2 > Difference in results of casting float to timestamp >

[jira] [Assigned] (SPARK-34755) Support the utils for transform number format

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34755: Assignee: (was: Apache Spark) > Support the utils for transform number format >

[jira] [Commented] (SPARK-34755) Support the utils for transform number format

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302198#comment-17302198 ] Apache Spark commented on SPARK-34755: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34755) Support the utils for transform number format

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34755: Assignee: Apache Spark > Support the utils for transform number format >

[jira] [Created] (SPARK-34755) Support the utils for transform number format

2021-03-15 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-34755: -- Summary: Support the utils for transform number format Key: SPARK-34755 URL: https://issues.apache.org/jira/browse/SPARK-34755 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-34754) sparksql 'add jar' not support hdfs ha mode in k8s

2021-03-15 Thread lithiumlee-_- (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lithiumlee-_- updated SPARK-34754: -- Description: Submit app to K8S,  the driver already running  but meet exception 

[jira] [Updated] (SPARK-34754) sparksql 'add jar' not support hdfs ha mode in k8s

2021-03-15 Thread lithiumlee-_- (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lithiumlee-_- updated SPARK-34754: -- Description:  The driver already running , but meet exception 

[jira] [Created] (SPARK-34754) sparksql 'add jar' not support hdfs ha mode in k8s

2021-03-15 Thread lithiumlee-_- (Jira)
lithiumlee-_- created SPARK-34754: - Summary: sparksql 'add jar' not support hdfs ha mode in k8s Key: SPARK-34754 URL: https://issues.apache.org/jira/browse/SPARK-34754 Project: Spark

[jira] [Assigned] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34752: Assignee: Erik Krogen > Upgrade Jetty to 9.4.37 to fix CVE-2020-27223 >

[jira] [Resolved] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34752. -- Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by pull

[jira] [Reopened] (SPARK-21449) Hive client's SessionState was not closed properly in HiveExternalCatalog

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-21449: - > Hive client's SessionState was not closed properly in HiveExternalCatalog >

[jira] [Updated] (SPARK-21449) Hive client's SessionState was not closed properly in HiveExternalCatalog

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21449: Labels: (was: bulk-closed) > Hive client's SessionState was not closed properly in

[jira] [Resolved] (SPARK-21449) Hive client's SessionState was not closed properly in HiveExternalCatalog

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-21449. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31833

[jira] [Updated] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23745: Labels: (was: bulk-closed) > Remove the directories of the “hive.downloaded.resources.dir” when

[jira] [Resolved] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-23745. - Fix Version/s: 3.2.0 Assignee: Kent Yao Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302171#comment-17302171 ] Hyukjin Kwon commented on SPARK-34694: -- Oh, okay. it more proposes to handle column references.

[jira] [Reopened] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-23745: - > Remove the directories of the “hive.downloaded.resources.dir” when > HiveThriftServer2 stopped >

[jira] [Assigned] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23745: Assignee: (was: Apache Spark) > Remove the directories of the

[jira] [Assigned] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23745: Assignee: Apache Spark > Remove the directories of the “hive.downloaded.resources.dir”

[jira] [Assigned] (SPARK-21449) Hive client's SessionState was not closed properly in HiveExternalCatalog

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-21449: --- Assignee: Kent Yao > Hive client's SessionState was not closed properly in

[jira] [Updated] (SPARK-34753) Deadlock in executor RPC shutdown hook

2021-03-15 Thread Dylan Patterson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Patterson updated SPARK-34753: Attachment: sb-dylanw-spark-0ec26858-b72ed278375bf3a9-exec-38.log > Deadlock in executor

[jira] [Commented] (SPARK-34753) Deadlock in executor RPC shutdown hook

2021-03-15 Thread Dylan Patterson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302090#comment-17302090 ] Dylan Patterson commented on SPARK-34753: - Aside from fixing the underlying issue it might be

[jira] [Created] (SPARK-34753) Deadlock in executor RPC shutdown hook

2021-03-15 Thread Dylan Patterson (Jira)
Dylan Patterson created SPARK-34753: --- Summary: Deadlock in executor RPC shutdown hook Key: SPARK-34753 URL: https://issues.apache.org/jira/browse/SPARK-34753 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-34751) Parquet with invalid chars on column name reads double as null when a clean schema is applied

2021-03-15 Thread Nivas Umapathy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nivas Umapathy updated SPARK-34751: --- Description: I have a parquet file that has data with invalid column names on it.

[jira] [Commented] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302043#comment-17302043 ] Apache Spark commented on SPARK-34729: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302028#comment-17302028 ] Apache Spark commented on SPARK-34752: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34752: Assignee: (was: Apache Spark) > Upgrade Jetty to 9.4.37 to fix CVE-2020-27223 >

[jira] [Assigned] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34752: Assignee: Apache Spark > Upgrade Jetty to 9.4.37 to fix CVE-2020-27223 >

[jira] [Updated] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-34752: Summary: Upgrade Jetty to 9.4.37 to fix CVE-2020-27223 (was: Upgrade Jetty to 9.3.37 to fix

[jira] [Updated] (SPARK-34752) Upgrade Jetty to 9.4.37 to fix CVE-2020-27223

2021-03-15 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-34752: Description: Another day, another Jetty CVE :)  Our internal build tools are complaining about

[jira] [Updated] (SPARK-34752) Upgrade Jetty to 9.3.37 to fix CVE-2020-27223

2021-03-15 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-34752: Description: Another day, another Jetty CVE :)  Our internal build tools are complaining about

[jira] [Updated] (SPARK-34752) Upgrade Jetty to 9.3.37 to fix CVE-2020-27223

2021-03-15 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-34752: Description: Another day, another Jetty CVE :)  Our internal build tools are complaining about

[jira] [Created] (SPARK-34752) Upgrade Jetty to 9.3.37 to fix CVE-2020-27223

2021-03-15 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-34752: --- Summary: Upgrade Jetty to 9.3.37 to fix CVE-2020-27223 Key: SPARK-34752 URL: https://issues.apache.org/jira/browse/SPARK-34752 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-15 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp reassigned SPARK-34738: --- Assignee: Shane Knapp > Upgrade Minikube and kubernetes cluster version on Jenkins >

[jira] [Commented] (SPARK-34646) TreeNode bind issue for duplicate column name.

2021-03-15 Thread loc nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301924#comment-17301924 ] loc nguyen commented on SPARK-34646: I am little confused with your response.  What are information

[jira] [Updated] (SPARK-34751) Parquet with invalid chars on column name reads double as null when a clean schema is applied

2021-03-15 Thread Nivas Umapathy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nivas Umapathy updated SPARK-34751: --- Description: I have a parquet file that has data with invalid column names on it.

[jira] [Updated] (SPARK-34751) Parquet with invalid chars on column name reads double as null when a clean schema is applied

2021-03-15 Thread Nivas Umapathy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nivas Umapathy updated SPARK-34751: --- Attachment: invalid_columns_double.parquet > Parquet with invalid chars on column name

[jira] [Created] (SPARK-34751) Parquet with invalid chars on column name reads double as null when a clean schema is applied

2021-03-15 Thread Nivas Umapathy (Jira)
Nivas Umapathy created SPARK-34751: -- Summary: Parquet with invalid chars on column name reads double as null when a clean schema is applied Key: SPARK-34751 URL: https://issues.apache.org/jira/browse/SPARK-34751

[jira] [Created] (SPARK-34750) Parquet with invalid chars on column name reads double as null when a clean schema is applied

2021-03-15 Thread Nivas Umapathy (Jira)
Nivas Umapathy created SPARK-34750: -- Summary: Parquet with invalid chars on column name reads double as null when a clean schema is applied Key: SPARK-34750 URL: https://issues.apache.org/jira/browse/SPARK-34750

[jira] [Assigned] (SPARK-34749) Simplify CreateNamedStruct

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34749: Assignee: Apache Spark > Simplify CreateNamedStruct > -- > >

[jira] [Assigned] (SPARK-34749) Simplify CreateNamedStruct

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34749: Assignee: (was: Apache Spark) > Simplify CreateNamedStruct >

[jira] [Commented] (SPARK-34749) Simplify CreateNamedStruct

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301827#comment-17301827 ] Apache Spark commented on SPARK-34749: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-34749) Simplify CreateNamedStruct

2021-03-15 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-34749: --- Summary: Simplify CreateNamedStruct Key: SPARK-34749 URL: https://issues.apache.org/jira/browse/SPARK-34749 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-34731) ConcurrentModificationException in EventLoggingListener when redacting properties

2021-03-15 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-34731: -- Affects Version/s: 3.1.1 > ConcurrentModificationException in EventLoggingListener when

[jira] [Assigned] (SPARK-34748) Create a rule of the analysis logic for streaming write

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34748: Assignee: (was: Apache Spark) > Create a rule of the analysis logic for streaming

[jira] [Commented] (SPARK-34748) Create a rule of the analysis logic for streaming write

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301792#comment-17301792 ] Apache Spark commented on SPARK-34748: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-34748) Create a rule of the analysis logic for streaming write

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34748: Assignee: Apache Spark > Create a rule of the analysis logic for streaming write >

[jira] [Commented] (SPARK-34747) Add virtual operators to the built-in function document.

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301791#comment-17301791 ] Apache Spark commented on SPARK-34747: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34747) Add virtual operators to the built-in function document.

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34747: Assignee: Kousuke Saruta (was: Apache Spark) > Add virtual operators to the built-in

[jira] [Assigned] (SPARK-34747) Add virtual operators to the built-in function document.

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34747: Assignee: Apache Spark (was: Kousuke Saruta) > Add virtual operators to the built-in

[jira] [Commented] (SPARK-34747) Add virtual operators to the built-in function document.

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301789#comment-17301789 ] Apache Spark commented on SPARK-34747: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-34748) Create a rule of the analysis logic for streaming write

2021-03-15 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-34748: --- Summary: Create a rule of the analysis logic for streaming write Key: SPARK-34748 URL: https://issues.apache.org/jira/browse/SPARK-34748 Project: Spark Issue

[jira] [Created] (SPARK-34747) Add virtual operators to the built-in function document.

2021-03-15 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34747: -- Summary: Add virtual operators to the built-in function document. Key: SPARK-34747 URL: https://issues.apache.org/jira/browse/SPARK-34747 Project: Spark

[jira] [Commented] (SPARK-34746) Spark dependencies require scala 2.12.12

2021-03-15 Thread Peter Kaiser (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301733#comment-17301733 ] Peter Kaiser commented on SPARK-34746: -- works when enforcing Scala 2.12.10 in the gradle build

[jira] [Created] (SPARK-34746) Spark dependencies require scala 2.12.12

2021-03-15 Thread Peter Kaiser (Jira)
Peter Kaiser created SPARK-34746: Summary: Spark dependencies require scala 2.12.12 Key: SPARK-34746 URL: https://issues.apache.org/jira/browse/SPARK-34746 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou edited comment on SPARK-34694 at 3/15/21, 3:26 PM: Hi Hyukjin, I

[jira] [Comment Edited] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou edited comment on SPARK-34694 at 3/15/21, 3:24 PM: Hi Hyukjin, I

[jira] [Comment Edited] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou edited comment on SPARK-34694 at 3/15/21, 3:23 PM: Hi Hyukjin, I

[jira] [Comment Edited] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou edited comment on SPARK-34694 at 3/15/21, 3:23 PM: Hi Hyukjin, I

[jira] [Comment Edited] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou edited comment on SPARK-34694 at 3/15/21, 3:23 PM: Hi Hyukjin, I

[jira] [Comment Edited] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou edited comment on SPARK-34694 at 3/15/21, 3:22 PM: Hi Hyukjin, I

[jira] [Commented] (SPARK-34694) Improve Spark SQL Source Filter to allow pushdown of filters span multiple columns

2021-03-15 Thread Chen Zou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301701#comment-17301701 ] Chen Zou commented on SPARK-34694: -- Hi Hyukjin, I think the design you described would work. But the

[jira] [Commented] (SPARK-34745) Unify overflow exception error message of integral types

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301693#comment-17301693 ] Apache Spark commented on SPARK-34745: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-34745) Unify overflow exception error message of integral types

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34745: Assignee: Apache Spark (was: Gengliang Wang) > Unify overflow exception error message

[jira] [Assigned] (SPARK-34745) Unify overflow exception error message of integral types

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34745: Assignee: Gengliang Wang (was: Apache Spark) > Unify overflow exception error message

[jira] [Updated] (SPARK-34745) Unify overflow exception error message of integral types

2021-03-15 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34745: --- Description: Current, the overflow exception error messages of integral types are

[jira] [Commented] (SPARK-34744) Improve error message for casting cause overflow error

2021-03-15 Thread Xingchao, Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301687#comment-17301687 ] Xingchao, Zhang commented on SPARK-34744: - I will try to fix it > Improve error message for

[jira] [Created] (SPARK-34745) Unify overflow exception error message of integral types

2021-03-15 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34745: -- Summary: Unify overflow exception error message of integral types Key: SPARK-34745 URL: https://issues.apache.org/jira/browse/SPARK-34745 Project: Spark

[jira] [Updated] (SPARK-34744) Improve error message for casting cause overflow error

2021-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34744: Description: For example: {code:sql} set spark.sql.ansi.enabled=true; select tinyint(128) *

[jira] [Created] (SPARK-34744) Improve error message for casting cause overflow error

2021-03-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34744: --- Summary: Improve error message for casting cause overflow error Key: SPARK-34744 URL: https://issues.apache.org/jira/browse/SPARK-34744 Project: Spark Issue

[jira] [Commented] (SPARK-34087) a memory leak occurs when we clone the spark session

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301665#comment-17301665 ] Apache Spark commented on SPARK-34087: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Resolved] (SPARK-34739) Add an year-month interval to a timestamp

2021-03-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-34739. -- Resolution: Done > Add an year-month interval to a timestamp >

[jira] [Resolved] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34743. --- Fix Version/s: 3.0.3 3.1.2 2.4.8

[jira] [Assigned] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34743: - Assignee: Dongjoon Hyun > ExpressionEncoderSuite should use deepEquals when we expect

[jira] [Resolved] (SPARK-34639) always remove unnecessary Alias in Analyzer.resolveExpression

2021-03-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34639. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31758

[jira] [Assigned] (SPARK-34639) always remove unnecessary Alias in Analyzer.resolveExpression

2021-03-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34639: --- Assignee: Wenchen Fan > always remove unnecessary Alias in Analyzer.resolveExpression >

[jira] [Assigned] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34743: Assignee: (was: Apache Spark) > ExpressionEncoderSuite should use deepEquals when we

[jira] [Commented] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301488#comment-17301488 ] Apache Spark commented on SPARK-34743: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34743: Assignee: Apache Spark > ExpressionEncoderSuite should use deepEquals when we expect

[jira] [Commented] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301487#comment-17301487 ] Apache Spark commented on SPARK-34743: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-34743) ExpressionEncoderSuite should use deepEquals when we expect `array of array`

2021-03-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34743: - Summary: ExpressionEncoderSuite should use deepEquals when we expect `array of array` Key: SPARK-34743 URL: https://issues.apache.org/jira/browse/SPARK-34743

[jira] [Commented] (SPARK-34742) ANSI mode: Abs throws exception if input is out of range

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301476#comment-17301476 ] Apache Spark commented on SPARK-34742: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-34742) ANSI mode: Abs throws exception if input is out of range

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34742: Assignee: Gengliang Wang (was: Apache Spark) > ANSI mode: Abs throws exception if input

[jira] [Commented] (SPARK-34742) ANSI mode: Abs throws exception if input is out of range

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301477#comment-17301477 ] Apache Spark commented on SPARK-34742: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-34742) ANSI mode: Abs throws exception if input is out of range

2021-03-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34742: Assignee: Apache Spark (was: Gengliang Wang) > ANSI mode: Abs throws exception if input

[jira] [Created] (SPARK-34742) ANSI mode: Abs throws exception if input is out of range

2021-03-15 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34742: -- Summary: ANSI mode: Abs throws exception if input is out of range Key: SPARK-34742 URL: https://issues.apache.org/jira/browse/SPARK-34742 Project: Spark

[jira] [Resolved] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34729. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31821

[jira] [Assigned] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34729: - Assignee: Cheng Su > Faster execution for broadcast nested loop join (left semi/anti

[jira] [Resolved] (SPARK-34722) Clean up deprecated API usage related to JUnit

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34722. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31815

[jira] [Assigned] (SPARK-34722) Clean up deprecated API usage related to JUnit

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34722: - Assignee: Yang Jie > Clean up deprecated API usage related to JUnit >

[jira] [Resolved] (SPARK-34725) Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats

2021-03-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34725. --- Resolution: Later The PR is closed. > Do not show Statistics(sizeInBytes=8.0 EiB) if we