[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK11

2021-01-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261186#comment-17261186 ] Yuming Wang commented on SPARK-24417: - Hadoop 2.7 users should back port HADOOP-12760 if running

[jira] [Created] (SPARK-34031) Union operator missing rowCount when enable CBO

2021-01-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34031: --- Summary: Union operator missing rowCount when enable CBO Key: SPARK-34031 URL: https://issues.apache.org/jira/browse/SPARK-34031 Project: Spark Issue Type:

[jira] [Created] (SPARK-33979) Filter predicate reorder

2021-01-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33979: --- Summary: Filter predicate reorder Key: SPARK-33979 URL: https://issues.apache.org/jira/browse/SPARK-33979 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-33979) Filter predicate reorder

2021-01-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258011#comment-17258011 ] Yuming Wang commented on SPARK-33979: - I'm working on. > Filter predicate reorder >

[jira] [Commented] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

2021-01-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257989#comment-17257989 ] Yuming Wang commented on SPARK-33977: - Example: 

[jira] [Created] (SPARK-33970) Check isNull and isNotNull in tests

2021-01-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33970: --- Summary: Check isNull and isNotNull in tests Key: SPARK-33970 URL: https://issues.apache.org/jira/browse/SPARK-33970 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-33960) LimitPushDown support Sort

2021-01-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33960. - Resolution: Not A Problem > LimitPushDown support Sort > -- > >

[jira] [Created] (SPARK-33960) LimitPushDown support Sort

2021-01-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33960: --- Summary: LimitPushDown support Sort Key: SPARK-33960 URL: https://issues.apache.org/jira/browse/SPARK-33960 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33958) spark sql DoubleType(0 * (-1)) return "-0.0"

2021-01-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257347#comment-17257347 ] Yuming Wang commented on SPARK-33958: - PostgreSQL also returns -0: {noformat} postgres=# create

[jira] [Updated] (SPARK-33959) Improve the statistics estimation of the Tail

2021-01-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33959: Description: {code:scala} spark.sql("set spark.sql.cbo.enabled=true")

[jira] [Created] (SPARK-33959) Improve the statistics estimation of the Tail

2021-01-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33959: --- Summary: Improve the statistics estimation of the Tail Key: SPARK-33959 URL: https://issues.apache.org/jira/browse/SPARK-33959 Project: Spark Issue Type:

[jira] [Created] (SPARK-33956) Add rowCount for Range operator

2021-01-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33956: --- Summary: Add rowCount for Range operator Key: SPARK-33956 URL: https://issues.apache.org/jira/browse/SPARK-33956 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-33954) Some operator missing rowCount when enable CBO

2021-01-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33954: --- Summary: Some operator missing rowCount when enable CBO Key: SPARK-33954 URL: https://issues.apache.org/jira/browse/SPARK-33954 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-33704) Support latest version of initialize() in HiveGenericUDTF

2020-12-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33704. - Resolution: Duplicate > Support latest version of initialize() in HiveGenericUDTF >

[jira] [Commented] (SPARK-33736) Handle MERGE in ReplaceNullWithFalseInPredicate

2020-12-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256533#comment-17256533 ] Yuming Wang commented on SPARK-33736: - Any update? > Handle MERGE in

[jira] [Created] (SPARK-33938) Optimize Like Any/All by LikeSimplification

2020-12-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33938: --- Summary: Optimize Like Any/All by LikeSimplification Key: SPARK-33938 URL: https://issues.apache.org/jira/browse/SPARK-33938 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33910. - Fix Version/s: 3.2.0 Resolution: Fixed > Simplify/Optimize conditional expressions >

[jira] [Assigned] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33910: --- Assignee: Yuming Wang > Simplify/Optimize conditional expressions >

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: 1. Push down the foldable expressions through CaseWhen/If 2. Simplify conditional in

[jira] [Updated] (SPARK-33884) Simplify CaseWhenclauses with (true and false) and (false and true)

2020-12-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33884: Description: Simplify CaseWhenclauses with (true and false) and (false and true):

[jira] [Updated] (SPARK-33884) Simplify CaseWhenclauses with (true and false) and (false and true)

2020-12-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33884: Description: This pr simplify {{CaseWhen}} when only one branch and one clause is null and

[jira] [Updated] (SPARK-33884) Simplify CaseWhenclauses with (true and false) and (false and true)

2020-12-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33884: Summary: Simplify CaseWhenclauses with (true and false) and (false and true) (was: Simplify

[jira] [Resolved] (SPARK-27367) Faster RoaringBitmap Serialization with v0.8.0

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-27367. - Resolution: Duplicate Issue fixed by SPARK-32437. > Faster RoaringBitmap Serialization with

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: 1. Push down the foldable expressions through CaseWhen/If 2. Simplify conditional in

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: 1. Push down the foldable expressions through CaseWhen/If 2. Simplify conditional in

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: 1. Push down the foldable expressions through CaseWhen/If 2. Simplify conditional in

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: 1. Push down the foldable expressions through CaseWhen/If 2. Simplify conditional in

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: 1. Push down the foldable expressions through CaseWhen/If 2. Simplify conditional in

[jira] [Updated] (SPARK-33848) Push the UnaryExpression into (if / case) branches

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33848: Summary: Push the UnaryExpression into (if / case) branches (was: Push the cast into (if / case)

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: Simplify/Optimize conditional expressions. We can improve these cases: 1. Reduce

[jira] [Updated] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Summary: Simplify/Optimize conditional expressions (was: Simplify conditional) >

[jira] [Updated] (SPARK-33910) Simplify conditional

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: Simplify CaseWhen/If conditionals. We can improve these cases: 1. Reduce read

[jira] [Updated] (SPARK-33910) Simplify conditional

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33910: Description: Simplify CaseWhen/If with EqualTo if all values are Literal and always false, this

[jira] [Updated] (SPARK-33884) Simplify conditional if all branches are foldable boolean type

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33884: Summary: Simplify conditional if all branches are foldable boolean type (was: Simplify CaseWhen

[jira] [Updated] (SPARK-33798) Add new rule to push down the foldable expressions through CaseWhen/If

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33798: Parent: SPARK-33910 Issue Type: Sub-task (was: Improvement) > Add new rule to push down

[jira] [Updated] (SPARK-33845) Improve SimplifyConditionals

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33845: Parent: SPARK-33910 Issue Type: Sub-task (was: Improvement) > Improve

[jira] [Updated] (SPARK-33847) Replace None of elseValue inside CaseWhen if all branches are FalseLiteral

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33847: Parent: SPARK-33910 Issue Type: Sub-task (was: Improvement) > Replace None of elseValue

[jira] [Updated] (SPARK-33848) Push the cast into (if / case) branches

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33848: Parent: SPARK-33910 Issue Type: Sub-task (was: Improvement) > Push the cast into (if /

[jira] [Updated] (SPARK-33861) Simplify conditional in predicate

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33861: Parent: SPARK-33910 Issue Type: Sub-task (was: Improvement) > Simplify conditional in

[jira] [Updated] (SPARK-33884) Simplify CaseWhen when one clause is null and another is boolean

2020-12-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33884: Parent: SPARK-33910 Issue Type: Sub-task (was: Improvement) > Simplify CaseWhen when one

[jira] [Created] (SPARK-33910) Simplify conditional

2020-12-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33910: --- Summary: Simplify conditional Key: SPARK-33910 URL: https://issues.apache.org/jira/browse/SPARK-33910 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-33884) Simplify CaseWhen when one clause is null and another is boolean

2020-12-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33884: Description: This pr simplify {{CaseWhen}} when only one branch and one clause is null and

[jira] [Created] (SPARK-33884) Simplify CaseWhen when one clause is null and another is boolean

2020-12-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33884: --- Summary: Simplify CaseWhen when one clause is null and another is boolean Key: SPARK-33884 URL: https://issues.apache.org/jira/browse/SPARK-33884 Project: Spark

[jira] [Created] (SPARK-33861) Simplify conditional in predicate

2020-12-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33861: --- Summary: Simplify conditional in predicate Key: SPARK-33861 URL: https://issues.apache.org/jira/browse/SPARK-33861 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33855) Add spark job maximum created files limit configuration

2020-12-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252408#comment-17252408 ] Yuming Wang commented on SPARK-33855: - May be you can repartition by dynamic partition columns

[jira] [Commented] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252299#comment-17252299 ] Yuming Wang commented on SPARK-25548: - Could you check the master branch. This issue should be

[jira] [Updated] (SPARK-33798) Add new rule to push down the foldable expressions through CaseWhen/If

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33798: Summary: Add new rule to push down the foldable expressions through CaseWhen/If (was: Simplify

[jira] [Comment Edited] (SPARK-33798) Simplify EqualTo(CaseWhen/If, Literal) always false

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252186#comment-17252186 ] Yuming Wang edited comment on SPARK-33798 at 12/19/20, 2:01 PM:

[jira] [Commented] (SPARK-33798) Simplify EqualTo(CaseWhen/If, Literal) always false

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252186#comment-17252186 ] Yuming Wang commented on SPARK-33798: - {noformat} === Metrics of Analyzer/Optimizer Rules === Total

[jira] [Commented] (SPARK-33798) Simplify EqualTo(CaseWhen/If, Literal) always false

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252185#comment-17252185 ] Yuming Wang commented on SPARK-33798: - {noformat} === Metrics of Analyzer/Optimizer Rules === Total

[jira] [Updated] (SPARK-33848) Push the cast into (if / case) branches

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33848: Description: Push the cast into (if / case) branches. The use case is: {code:sql} create table

[jira] [Created] (SPARK-33848) Push the cast into (if / case) branches

2020-12-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33848: --- Summary: Push the cast into (if / case) branches Key: SPARK-33848 URL: https://issues.apache.org/jira/browse/SPARK-33848 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33847) Replace None of elseValue inside CaseWhen if all branches are FalseLiteral

2020-12-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33847: Description: {code:scala} spark.sql("create table t1 using parquet as select id from range(10)")

[jira] [Created] (SPARK-33847) Replace None of elseValue inside CaseWhen if all branches are FalseLiteral

2020-12-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33847: --- Summary: Replace None of elseValue inside CaseWhen if all branches are FalseLiteral Key: SPARK-33847 URL: https://issues.apache.org/jira/browse/SPARK-33847 Project:

[jira] [Created] (SPARK-33845) Improve SimplifyConditionals

2020-12-18 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33845: --- Summary: Improve SimplifyConditionals Key: SPARK-33845 URL: https://issues.apache.org/jira/browse/SPARK-33845 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30186) support Dynamic Partition Pruning in Adaptive Execution

2020-12-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30186: Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > support Dynamic Partition

[jira] [Assigned] (SPARK-27620) Update jetty to 9.4.18.v20190429

2020-12-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-27620: --- Assignee: Yuming Wang (was: yuming.wang) > Update jetty to 9.4.18.v20190429 >

[jira] [Updated] (SPARK-33798) Simplify EqualTo(CaseWhen/If, Literal) always false

2020-12-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33798: Description: Simplify CaseWhen/If with EqualTo if all values are Literal and always false, this

[jira] [Created] (SPARK-33798) Simplify EqualTo(CaseWhen/If, Literal) always false

2020-12-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33798: --- Summary: Simplify EqualTo(CaseWhen/If, Literal) always false Key: SPARK-33798 URL: https://issues.apache.org/jira/browse/SPARK-33798 Project: Spark Issue

[jira] [Updated] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33696: Description: Hive 2.3.8 changes: HIVE-19662: Upgrade Avro to 1.8.2 HIVE-24324: Remove

[jira] [Commented] (SPARK-33791) grouping__id() result does not consistent with hive's version < 2.3

2020-12-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249680#comment-17249680 ] Yuming Wang commented on SPARK-33791: -

[jira] [Commented] (SPARK-33766) Upgrade Jackson to 2.11.4

2020-12-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249553#comment-17249553 ] Yuming Wang commented on SPARK-33766: - I converted this ticket and SPARK-33696 to sub-task of

[jira] [Updated] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33696: Parent: SPARK-26345 Issue Type: Sub-task (was: Improvement) > Upgrade built-in Hive to

[jira] [Updated] (SPARK-33766) Upgrade Jackson to 2.11.4

2020-12-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33766: Parent: SPARK-26345 Issue Type: Sub-task (was: Improvement) > Upgrade Jackson to 2.11.4

[jira] [Resolved] (SPARK-33709) Refactor FileTable

2020-12-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33709. - Resolution: Won't Fix > Refactor FileTable > -- > > Key:

[jira] [Assigned] (SPARK-33673) Do not push down partition filters to ParquetScan for DataSourceV2

2020-12-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33673: --- Assignee: Yang Jie > Do not push down partition filters to ParquetScan for DataSourceV2 >

[jira] [Resolved] (SPARK-33673) Do not push down partition filters to ParquetScan for DataSourceV2

2020-12-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33673. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30652

[jira] [Created] (SPARK-33771) Fix Invalid value for HourOfAmPm when testing on JDK 14

2020-12-13 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33771: --- Summary: Fix Invalid value for HourOfAmPm when testing on JDK 14 Key: SPARK-33771 URL: https://issues.apache.org/jira/browse/SPARK-33771 Project: Spark Issue

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248526#comment-17248526 ] Yuming Wang commented on SPARK-26345: - [~jamestaylor] Please see

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248374#comment-17248374 ] Yuming Wang commented on SPARK-26345: - Benchmark and benchmark result: {code:scala} /* * Licensed

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248323#comment-17248323 ] Yuming Wang commented on SPARK-26345: - We have a pr test compatibility against Parquet 1.11.1, Avro

[jira] [Updated] (SPARK-33766) Upgrade Jackson to 2.11.4

2020-12-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33766: Description: Upgrade Jackson to 2.11.4 to make it easy to upgrade Avro 1.10.1. More details:

[jira] [Created] (SPARK-33766) Upgrade Jackson to 2.11.4

2020-12-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33766: --- Summary: Upgrade Jackson to 2.11.4 Key: SPARK-33766 URL: https://issues.apache.org/jira/browse/SPARK-33766 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-33707) Support multiple types of function partition pruning on hive metastore

2020-12-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33707: Affects Version/s: (was: 2.4.3) (was: 2.3.4)

[jira] [Updated] (SPARK-33707) Support multiple types of function partition pruning on hive metastore

2020-12-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33707: Parent: SPARK-33537 Issue Type: Sub-task (was: Improvement) > Support multiple types of

[jira] [Resolved] (SPARK-33584) Partition predicate pushdown into Hive metastore support cast string type to date type

2020-12-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33584. - Resolution: Won't Fix > Partition predicate pushdown into Hive metastore support cast string

[jira] [Created] (SPARK-33709) Refactor FileTable

2020-12-08 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33709: --- Summary: Refactor FileTable Key: SPARK-33709 URL: https://issues.apache.org/jira/browse/SPARK-33709 Project: Spark Issue Type: Sub-task Components:

[jira] [Updated] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33696: Description: Hive 2.3.8 changes: HIVE-19662: Upgrade Avro to 1.8.2 HIVE-24324: Remove deprecated

[jira] [Created] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33696: --- Summary: Upgrade built-in Hive to 2.3.8 Key: SPARK-33696 URL: https://issues.apache.org/jira/browse/SPARK-33696 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-33687) ANALYZE command support analyze all tables in a specific database

2020-12-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33687: Description: PostgreSQL support this feature: https://www.postgresql.org/docs/12/sql-analyze.html

[jira] [Updated] (SPARK-33687) ANALYZE command support analyze all tables in a specific database

2020-12-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33687: Summary: ANALYZE command support analyze all tables in a specific database (was: ANALYZE

[jira] [Updated] (SPARK-33687) ANALYZE command support analyze every tables in a specific database

2020-12-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33687: Parent: SPARK-30374 Issue Type: Sub-task (was: Improvement) > ANALYZE command support

[jira] [Created] (SPARK-33687) ANALYZE command support analyze every tables in a specific database

2020-12-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33687: --- Summary: ANALYZE command support analyze every tables in a specific database Key: SPARK-33687 URL: https://issues.apache.org/jira/browse/SPARK-33687 Project: Spark

[jira] [Resolved] (SPARK-33674) Show Slowpoke notifications in SBT tests

2020-12-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33674. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30621

[jira] [Commented] (SPARK-33673) Do not push down partition filters to ParquetScan for DataSourceV2

2020-12-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244624#comment-17244624 ] Yuming Wang commented on SPARK-33673: - Yes, you can verify it by: {code:sh} mvn -Dtest=none

[jira] [Commented] (SPARK-33673) Do not push down partition filters to ParquetScan for DataSourceV2

2020-12-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244612#comment-17244612 ] Yuming Wang commented on SPARK-33673: - [~cloud_fan] [~rdblue] [~viirya] > Do not push down

[jira] [Created] (SPARK-33673) Do not push down partition filters to ParquetScan for DataSourceV2

2020-12-05 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33673: --- Summary: Do not push down partition filters to ParquetScan for DataSourceV2 Key: SPARK-33673 URL: https://issues.apache.org/jira/browse/SPARK-33673 Project: Spark

[jira] [Created] (SPARK-33630) Support save SHOW command result to table

2020-12-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33630: --- Summary: Support save SHOW command result to table Key: SPARK-33630 URL: https://issues.apache.org/jira/browse/SPARK-33630 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33628) Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

2020-12-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242196#comment-17242196 ] Yuming Wang commented on SPARK-33628: - Thank you for reporting this issue. > Use the

[jira] [Created] (SPARK-33617) Avoid generating small files for INSERT INTO TABLE from VALUES

2020-12-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33617: --- Summary: Avoid generating small files for INSERT INTO TABLE from VALUES Key: SPARK-33617 URL: https://issues.apache.org/jira/browse/SPARK-33617 Project: Spark

[jira] [Updated] (SPARK-33589) Close opened session if the initialization fails

2020-11-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33589: Summary: Close opened session if the initialization fails (was: Add try catch when opening

[jira] [Created] (SPARK-33589) Add try catch when opening session

2020-11-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33589: --- Summary: Add try catch when opening session Key: SPARK-33589 URL: https://issues.apache.org/jira/browse/SPARK-33589 Project: Spark Issue Type: Bug

[jira] [Issue Comment Deleted] (SPARK-33584) Partition predicate pushdown into Hive metastore support cast string type to date type

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33584: Comment: was deleted (was: This change should after SPARK-33581, I have prepared the pr:

[jira] [Issue Comment Deleted] (SPARK-33582) Partition predicate pushdown into Hive metastore support not-equals

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33582: Comment: was deleted (was: This change should after SPARK-33581, I have prepared the pr:

[jira] [Commented] (SPARK-33581) Refactor HivePartitionFilteringSuite

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240171#comment-17240171 ] Yuming Wang commented on SPARK-33581: - Issue resolved by pull request 30525

[jira] [Resolved] (SPARK-33581) Refactor HivePartitionFilteringSuite

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33581. - Fix Version/s: 3.1.0 Resolution: Fixed User 'wangyum' has created a pull request for

[jira] [Issue Comment Deleted] (SPARK-33581) Refactor HivePartitionFilteringSuite

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33581: Comment: was deleted (was: User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33581) Refactor HivePartitionFilteringSuite

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33581: --- Assignee: Yuming Wang > Refactor HivePartitionFilteringSuite >

[jira] [Commented] (SPARK-33584) Partition predicate pushdown into Hive metastore support cast string type to date type

2020-11-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17239935#comment-17239935 ] Yuming Wang commented on SPARK-33584: - This change should after SPARK-33581, I have prepared the pr:

[jira] [Created] (SPARK-33584) Partition predicate pushdown into Hive metastore support cast string type to date type

2020-11-28 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33584: --- Summary: Partition predicate pushdown into Hive metastore support cast string type to date type Key: SPARK-33584 URL: https://issues.apache.org/jira/browse/SPARK-33584

<    8   9   10   11   12   13   14   15   16   17   >