[jira] [Resolved] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-05-06 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you resolved SPARK-34226. - Resolution: Won't Fix > Reduce RepartitionOperation num partitions to its child max row >

[jira] [Updated] (SPARK-35282) Support AQE side shuffled hash join formula

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35282: Description: Use AQE runtime statistics to decide if we can use shuffled hash join instead of

[jira] [Updated] (SPARK-34980) Support coalesce partition through union

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34980: Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Support coalesce partition

[jira] [Updated] (SPARK-35214) OptimizeSkewedJoin support ShuffledHashJoinExec

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35214: Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > OptimizeSkewedJoin support

[jira] [Updated] (SPARK-35282) Support AQE side shuffled hash join formula

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35282: Parent: SPARK-33828 Issue Type: Sub-task (was: New Feature) > Support AQE side shuffled

[jira] [Updated] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35239: Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Coalesce shuffle

[jira] [Updated] (SPARK-34899) Use origin plan if can not coalesce shuffle partition

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34899: Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Use origin plan if can not

[jira] [Commented] (SPARK-35264) Support AQE side broadcastJoin threshold

2021-05-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339958#comment-17339958 ] ulysses you commented on SPARK-35264: - [~dongjoon] thank you for moving this to AQE umbrella, I

[jira] [Created] (SPARK-35282) Support AQE side shuffled hash join formula

2021-04-30 Thread ulysses you (Jira)
ulysses you created SPARK-35282: --- Summary: Support AQE side shuffled hash join formula Key: SPARK-35282 URL: https://issues.apache.org/jira/browse/SPARK-35282 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-35264) Support AQE side broadcastJoin threshold

2021-04-29 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35264: Issue Type: New Feature (was: Improvement) > Support AQE side broadcastJoin threshold >

[jira] [Updated] (SPARK-35264) Support AQE side broadcastJoin threshold

2021-04-28 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35264: Description: The main idea here is that make join config isolation between normal planner and

[jira] [Updated] (SPARK-35264) Support AQE side broadcastJoin threshold

2021-04-28 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35264: Description: The main idea here is that make join config isolation between normal planner and

[jira] [Updated] (SPARK-35264) Support AQE side broadcastJoin threshold

2021-04-28 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35264: Description: The main idea here is that make join config isolation between normal planner and

[jira] [Created] (SPARK-35264) Support AQE side broadcastJoin threshold

2021-04-28 Thread ulysses you (Jira)
ulysses you created SPARK-35264: --- Summary: Support AQE side broadcastJoin threshold Key: SPARK-35264 URL: https://issues.apache.org/jira/browse/SPARK-35264 Project: Spark Issue Type:

[jira] [Created] (SPARK-35239) Coalesce shuffle partition should handle empty input RDD

2021-04-27 Thread ulysses you (Jira)
ulysses you created SPARK-35239: --- Summary: Coalesce shuffle partition should handle empty input RDD Key: SPARK-35239 URL: https://issues.apache.org/jira/browse/SPARK-35239 Project: Spark

[jira] [Updated] (SPARK-35221) Add join hint build side check

2021-04-25 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35221: Description: Currently we support specify the join implementation with hint, but Spark did not 

[jira] [Created] (SPARK-35221) Add join hint build side check

2021-04-25 Thread ulysses you (Jira)
ulysses you created SPARK-35221: --- Summary: Add join hint build side check Key: SPARK-35221 URL: https://issues.apache.org/jira/browse/SPARK-35221 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35214) OptimizeSkewedJoin support ShuffledHashJoinExec

2021-04-24 Thread ulysses you (Jira)
ulysses you created SPARK-35214: --- Summary: OptimizeSkewedJoin support ShuffledHashJoinExec Key: SPARK-35214 URL: https://issues.apache.org/jira/browse/SPARK-35214 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35137) Revise outputpartitioning number in some SparkPlan

2021-04-19 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35137: Description: Revise the outputPartitioning number in some `SparkPlan` * UnionExec *

[jira] [Updated] (SPARK-35137) Revise outputpartitioning number in some SparkPlan

2021-04-19 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35137: Description: Revise the outputPartitioning number in some `SparkPlan` * UnionExec *

[jira] [Created] (SPARK-35137) Revise outputpartitioning number in some SparkPlan

2021-04-19 Thread ulysses you (Jira)
ulysses you created SPARK-35137: --- Summary: Revise outputpartitioning number in some SparkPlan Key: SPARK-35137 URL: https://issues.apache.org/jira/browse/SPARK-35137 Project: Spark Issue Type:

[jira] [Created] (SPARK-35083) Support remote scheduler pool file

2021-04-14 Thread ulysses you (Jira)
ulysses you created SPARK-35083: --- Summary: Support remote scheduler pool file Key: SPARK-35083 URL: https://issues.apache.org/jira/browse/SPARK-35083 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35041) Revise the overflow in UTF8String

2021-04-12 Thread ulysses you (Jira)
ulysses you created SPARK-35041: --- Summary: Revise the overflow in UTF8String Key: SPARK-35041 URL: https://issues.apache.org/jira/browse/SPARK-35041 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35005) Improve error msg if UTF8String concatWs length overflow

2021-04-09 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35005: Summary: Improve error msg if UTF8String concatWs length overflow (was: Improve error msg if

[jira] [Created] (SPARK-35005) Improve error msg if UTF8String concatWs over int length

2021-04-09 Thread ulysses you (Jira)
ulysses you created SPARK-35005: --- Summary: Improve error msg if UTF8String concatWs over int length Key: SPARK-35005 URL: https://issues.apache.org/jira/browse/SPARK-35005 Project: Spark Issue

[jira] [Resolved] (SPARK-34991) Mark GlobalLimit outputPartitioning as SinglePartition

2021-04-08 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you resolved SPARK-34991. - Resolution: Won't Fix > Mark GlobalLimit outputPartitioning as SinglePartition >

[jira] [Updated] (SPARK-34991) Mark GlobalLimit outputPartitioning as SinglePartition

2021-04-08 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34991: Description: Accroding to the semantics of `GlobalLimitExec`, the outputPartitioning should

[jira] [Created] (SPARK-34991) Mark GlobalLimit outputPartitioning as SinglePartition

2021-04-08 Thread ulysses you (Jira)
ulysses you created SPARK-34991: --- Summary: Mark GlobalLimit outputPartitioning as SinglePartition Key: SPARK-34991 URL: https://issues.apache.org/jira/browse/SPARK-34991 Project: Spark Issue

[jira] [Created] (SPARK-34980) Support coalesce partition through union

2021-04-07 Thread ulysses you (Jira)
ulysses you created SPARK-34980: --- Summary: Support coalesce partition through union Key: SPARK-34980 URL: https://issues.apache.org/jira/browse/SPARK-34980 Project: Spark Issue Type:

[jira] [Created] (SPARK-34919) Change partitioning to SinglePartition if partition number is 1

2021-03-31 Thread ulysses you (Jira)
ulysses you created SPARK-34919: --- Summary: Change partitioning to SinglePartition if partition number is 1 Key: SPARK-34919 URL: https://issues.apache.org/jira/browse/SPARK-34919 Project: Spark

[jira] [Created] (SPARK-34914) Local scheduler backend support update token

2021-03-30 Thread ulysses you (Jira)
ulysses you created SPARK-34914: --- Summary: Local scheduler backend support update token Key: SPARK-34914 URL: https://issues.apache.org/jira/browse/SPARK-34914 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34899) Use origin plan if can not coalesce shuffle partition

2021-03-29 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34899: Description: The `CoalesceShufflePartitions` can not coalesce such case if the total shuffle

[jira] [Created] (SPARK-34899) Use origin plan if can not coalesce shuffle partition

2021-03-29 Thread ulysses you (Jira)
ulysses you created SPARK-34899: --- Summary: Use origin plan if can not coalesce shuffle partition Key: SPARK-34899 URL: https://issues.apache.org/jira/browse/SPARK-34899 Project: Spark Issue

[jira] [Created] (SPARK-34852) Close Hive session state should use withHiveState

2021-03-24 Thread ulysses you (Jira)
ulysses you created SPARK-34852: --- Summary: Close Hive session state should use withHiveState Key: SPARK-34852 URL: https://issues.apache.org/jira/browse/SPARK-34852 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

2021-03-17 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: With custom `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars`.

[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

2021-03-17 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: With custom `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars`.

[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

2021-03-17 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set

[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

2021-03-17 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set

[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

2021-03-17 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set

[jira] [Created] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

2021-03-17 Thread ulysses you (Jira)
ulysses you created SPARK-34772: --- Summary: RebaseDateTime loadRebaseRecords should use Spark classloader instead of context Key: SPARK-34772 URL: https://issues.apache.org/jira/browse/SPARK-34772

[jira] [Updated] (SPARK-34766) Do not capture maven config for views

2021-03-16 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34766: Description: Due to the bad network, we always use the thirdparty maven repo to run test. e.g.,

[jira] [Updated] (SPARK-34766) Do not capture maven config for views

2021-03-16 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34766: Environment: (was: Due to the bad network, we always use the thirdparty maven repo to run

[jira] [Created] (SPARK-34766) Do not capture maven config for views

2021-03-16 Thread ulysses you (Jira)
ulysses you created SPARK-34766: --- Summary: Do not capture maven config for views Key: SPARK-34766 URL: https://issues.apache.org/jira/browse/SPARK-34766 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-34735) Add modified configs for SQL execution in UI

2021-03-13 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34735: Attachment: sql-ui.jpg > Add modified configs for SQL execution in UI >

[jira] [Created] (SPARK-34735) Add modified configs for SQL execution in UI

2021-03-13 Thread ulysses you (Jira)
ulysses you created SPARK-34735: --- Summary: Add modified configs for SQL execution in UI Key: SPARK-34735 URL: https://issues.apache.org/jira/browse/SPARK-34735 Project: Spark Issue Type: New

[jira] [Created] (SPARK-34725) Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats

2021-03-12 Thread ulysses you (Jira)
ulysses you created SPARK-34725: --- Summary: Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats Key: SPARK-34725 URL: https://issues.apache.org/jira/browse/SPARK-34725 Project:

[jira] [Resolved] (SPARK-34690) Support Not(Int) and Not(InSet) propagate null

2021-03-10 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you resolved SPARK-34690. - Resolution: Duplicate > Support Not(Int) and Not(InSet) propagate null >

[jira] [Resolved] (SPARK-34691) Support Not(Int) and Not(InSet) propagate null

2021-03-10 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you resolved SPARK-34691. - Resolution: Duplicate > Support Not(Int) and Not(InSet) propagate null >

[jira] [Created] (SPARK-34692) Support Not(Int) and Not(InSet) propagate null

2021-03-10 Thread ulysses you (Jira)
ulysses you created SPARK-34692: --- Summary: Support Not(Int) and Not(InSet) propagate null Key: SPARK-34692 URL: https://issues.apache.org/jira/browse/SPARK-34692 Project: Spark Issue Type:

[jira] [Created] (SPARK-34691) Support Not(Int) and Not(InSet) propagate null

2021-03-10 Thread ulysses you (Jira)
ulysses you created SPARK-34691: --- Summary: Support Not(Int) and Not(InSet) propagate null Key: SPARK-34691 URL: https://issues.apache.org/jira/browse/SPARK-34691 Project: Spark Issue Type:

[jira] [Created] (SPARK-34690) Support Not(Int) and Not(InSet) propagate null

2021-03-10 Thread ulysses you (Jira)
ulysses you created SPARK-34690: --- Summary: Support Not(Int) and Not(InSet) propagate null Key: SPARK-34690 URL: https://issues.apache.org/jira/browse/SPARK-34690 Project: Spark Issue Type:

[jira] [Created] (SPARK-34613) Fix view does not capture disable hint config

2021-03-03 Thread ulysses you (Jira)
ulysses you created SPARK-34613: --- Summary: Fix view does not capture disable hint config Key: SPARK-34613 URL: https://issues.apache.org/jira/browse/SPARK-34613 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-34550) Skip InSet null value during push filter to Hive metastore

2021-02-25 Thread ulysses you (Jira)
ulysses you created SPARK-34550: --- Summary: Skip InSet null value during push filter to Hive metastore Key: SPARK-34550 URL: https://issues.apache.org/jira/browse/SPARK-34550 Project: Spark

[jira] [Updated] (SPARK-34538) Hive Metastore support filter by not-in

2021-02-25 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34538: Description: `NOT IN` is a useful condition to prune partition, it would be better to support it.

[jira] [Updated] (SPARK-34538) Hive Metastore support filter by not-in

2021-02-25 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34538: Description: `NOT IN` is a useful condition to prune partition, it would be better to support it.

[jira] [Updated] (SPARK-34538) Hive Metastore support filter by not-in

2021-02-25 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34538: Description: `NOT IN` is a useful condition to prune partition, it would be better to push it to

[jira] [Updated] (SPARK-34538) Hive Metastore support filter by not-in

2021-02-25 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34538: Parent: SPARK-33537 Issue Type: Sub-task (was: Improvement) > Hive Metastore support

[jira] [Created] (SPARK-34538) Hive Metastore support filter by not-in

2021-02-25 Thread ulysses you (Jira)
ulysses you created SPARK-34538: --- Summary: Hive Metastore support filter by not-in Key: SPARK-34538 URL: https://issues.apache.org/jira/browse/SPARK-34538 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34515) Fix NPE if InSet contains null value during getPartitionsByFilter

2021-02-23 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34515: Description: Spark will convert InSet to `>= and <=` if it's values size over

[jira] [Created] (SPARK-34515) Fix NPE if InSet contains null value during getPartitionsByFilter

2021-02-23 Thread ulysses you (Jira)
ulysses you created SPARK-34515: --- Summary: Fix NPE if InSet contains null value during getPartitionsByFilter Key: SPARK-34515 URL: https://issues.apache.org/jira/browse/SPARK-34515 Project: Spark

[jira] [Updated] (SPARK-34400) Check spark.job.interruptOnCancel value is illegal before submit job

2021-02-08 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34400: Description: Add value check before setProperty if key is `spark.job.interruptOnCancel`.

[jira] [Created] (SPARK-34400) Check spark.job.interruptOnCancel value is illegal before submit job

2021-02-08 Thread ulysses you (Jira)
ulysses you created SPARK-34400: --- Summary: Check spark.job.interruptOnCancel value is illegal before submit job Key: SPARK-34400 URL: https://issues.apache.org/jira/browse/SPARK-34400 Project: Spark

[jira] [Updated] (SPARK-34396) Add a new build-in function delegate

2021-02-07 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34396: Description: Delegate is just like a big box that can include some other expressions. It will

[jira] [Updated] (SPARK-34396) Add a new build-in function delegate

2021-02-07 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34396: Description: Delegate is just like a big box that can include some other expressions. It will

[jira] [Created] (SPARK-34396) Add a new build-in function delegate

2021-02-07 Thread ulysses you (Jira)
ulysses you created SPARK-34396: --- Summary: Add a new build-in function delegate Key: SPARK-34396 URL: https://issues.apache.org/jira/browse/SPARK-34396 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-34355) Add log and time cost for commit job

2021-02-04 Thread ulysses you (Jira)
ulysses you created SPARK-34355: --- Summary: Add log and time cost for commit job Key: SPARK-34355 URL: https://issues.apache.org/jira/browse/SPARK-34355 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-34272) Pretty SQL should check NonSQLExpression

2021-01-27 Thread ulysses you (Jira)
ulysses you created SPARK-34272: --- Summary: Pretty SQL should check NonSQLExpression Key: SPARK-34272 URL: https://issues.apache.org/jira/browse/SPARK-34272 Project: Spark Issue Type:

[jira] [Created] (SPARK-34261) Avoid side effect if create exists temporary function

2021-01-27 Thread ulysses you (Jira)
ulysses you created SPARK-34261: --- Summary: Avoid side effect if create exists temporary function Key: SPARK-34261 URL: https://issues.apache.org/jira/browse/SPARK-34261 Project: Spark Issue

[jira] [Created] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-01-25 Thread ulysses you (Jira)
ulysses you created SPARK-34226: --- Summary: Reduce RepartitionOperation num partitions to its child max row Key: SPARK-34226 URL: https://issues.apache.org/jira/browse/SPARK-34226 Project: Spark

[jira] [Updated] (SPARK-34166) Fix flaky test in DecommissionWorkerSuite

2021-01-19 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34166: Description: The test `decommission workers ensure that shuffle output is regenerated even with

[jira] [Created] (SPARK-34166) Fix flaky test in DecommissionWorkerSuite

2021-01-19 Thread ulysses you (Jira)
ulysses you created SPARK-34166: --- Summary: Fix flaky test in DecommissionWorkerSuite Key: SPARK-34166 URL: https://issues.apache.org/jira/browse/SPARK-34166 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34150) Strip Null literal.sql in resolve alias

2021-01-18 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34150: Description: We will convert `Literal(null)` to target data type during analysis. Then the

[jira] [Created] (SPARK-34150) Strip Null literal.sql in resolve alias

2021-01-17 Thread ulysses you (Jira)
ulysses you created SPARK-34150: --- Summary: Strip Null literal.sql in resolve alias Key: SPARK-34150 URL: https://issues.apache.org/jira/browse/SPARK-34150 Project: Spark Issue Type:

[jira] [Created] (SPARK-34072) Fix empty array failed in functions.lit()

2021-01-11 Thread ulysses you (Jira)
ulysses you created SPARK-34072: --- Summary: Fix empty array failed in functions.lit() Key: SPARK-34072 URL: https://issues.apache.org/jira/browse/SPARK-34072 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-34069) Kill barrier tasks should respect SPARK_JOB_INTERRUPT_ON_CANCEL

2021-01-11 Thread ulysses you (Jira)
ulysses you created SPARK-34069: --- Summary: Kill barrier tasks should respect SPARK_JOB_INTERRUPT_ON_CANCEL Key: SPARK-34069 URL: https://issues.apache.org/jira/browse/SPARK-34069 Project: Spark

[jira] [Created] (SPARK-34030) Fold RepartitionExpression num partition should at Optimizer

2021-01-06 Thread ulysses you (Jira)
ulysses you created SPARK-34030: --- Summary: Fold RepartitionExpression num partition should at Optimizer Key: SPARK-34030 URL: https://issues.apache.org/jira/browse/SPARK-34030 Project: Spark

[jira] [Resolved] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2021-01-05 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you resolved SPARK-33949. - Resolution: Won't Fix > Make approx_count_distinct result consistent whether Optimize rule

[jira] [Updated] (SPARK-33989) Strip auto-generated cast when using Cast.sql

2021-01-04 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33989: Summary: Strip auto-generated cast when using Cast.sql (was: Strip auto-generated cast when

[jira] [Created] (SPARK-33989) Strip auto-generated cast when resolving UnresolvedAlias

2021-01-04 Thread ulysses you (Jira)
ulysses you created SPARK-33989: --- Summary: Strip auto-generated cast when resolving UnresolvedAlias Key: SPARK-33989 URL: https://issues.apache.org/jira/browse/SPARK-33989 Project: Spark Issue

[jira] [Updated] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2020-12-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33949: Description: This code will fail because folabe value not fold, we should keep {code:java} val

[jira] [Updated] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2020-12-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33949: Description: This code will fail because folabe value not fold, we should keep result consistent

[jira] [Created] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2020-12-31 Thread ulysses you (Jira)
ulysses you created SPARK-33949: --- Summary: Make approx_count_distinct result consistent whether Optimize rule exists or not Key: SPARK-33949 URL: https://issues.apache.org/jira/browse/SPARK-33949

[jira] [Updated] (SPARK-33939) Make Column.named use UnresolvedAlias to assign name

2020-12-30 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33939: Summary: Make Column.named use UnresolvedAlias to assign name (was: Do not display the seed of

[jira] [Updated] (SPARK-33939) Do not display the seed of uuid/shuffle if user not specify

2020-12-30 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33939: Description: It's more reasonable to treat user specify expr as unresolved. (was: Keep the

[jira] [Created] (SPARK-33939) Do not display the seed of uuid/shuffle if user not specify

2020-12-29 Thread ulysses you (Jira)
ulysses you created SPARK-33939: --- Summary: Do not display the seed of uuid/shuffle if user not specify Key: SPARK-33939 URL: https://issues.apache.org/jira/browse/SPARK-33939 Project: Spark

[jira] [Updated] (SPARK-33909) Check rand functions seed is legal at analyer side

2020-12-24 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33909: Description: It's better to check seed expression is legal at analyzer side instead of execution.

[jira] [Updated] (SPARK-33909) Check rand functions seed is legal at analyer side

2020-12-24 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33909: Summary: Check rand functions seed is legal at analyer side (was: Check rand functions seed data

[jira] [Created] (SPARK-33909) Check rand functions seed data type at analyer side

2020-12-24 Thread ulysses you (Jira)
ulysses you created SPARK-33909: --- Summary: Check rand functions seed data type at analyer side Key: SPARK-33909 URL: https://issues.apache.org/jira/browse/SPARK-33909 Project: Spark Issue

[jira] [Updated] (SPARK-33860) Make CatalystTypeConverters.convertToCatalyst match special Array value

2020-12-20 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33860: Summary: Make CatalystTypeConverters.convertToCatalyst match special Array value (was: Change

[jira] [Updated] (SPARK-33860) Change CatalystTypeConverters.convertToCatalyst match special Array value

2020-12-20 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33860: Summary: Change CatalystTypeConverters.convertToCatalyst match special Array value (was: Change

[jira] [Created] (SPARK-33860) Change CatalystTypeConverters.convertToCatalyst match Any to _

2020-12-20 Thread ulysses you (Jira)
ulysses you created SPARK-33860: --- Summary: Change CatalystTypeConverters.convertToCatalyst match Any to _ Key: SPARK-33860 URL: https://issues.apache.org/jira/browse/SPARK-33860 Project: Spark

[jira] [Updated] (SPARK-33857) Unify random functions and make Uuid Shuffle support seed in SQL

2020-12-20 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33857: Description: Unify the seed of random functions 1. Add a hold place expression `DefaultSeed` as

[jira] [Updated] (SPARK-33857) Unify random functions and make Uuid Shuffle support seed in SQL

2020-12-20 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33857: Summary: Unify random functions and make Uuid Shuffle support seed in SQL (was: Unify randoms

[jira] [Updated] (SPARK-33857) Unify randoms and make Uuid and Shuffle support seed in SQL

2020-12-20 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33857: Summary: Unify randoms and make Uuid and Shuffle support seed in SQL (was: Make Uuid and Shuffle

[jira] [Created] (SPARK-33857) Make Uuid and Shuffle support special seed in SQL

2020-12-19 Thread ulysses you (Jira)
ulysses you created SPARK-33857: --- Summary: Make Uuid and Shuffle support special seed in SQL Key: SPARK-33857 URL: https://issues.apache.org/jira/browse/SPARK-33857 Project: Spark Issue Type:

[jira] [Created] (SPARK-33840) Add spark.sql.files.minPartitionNum to performence tuning doc

2020-12-18 Thread ulysses you (Jira)
ulysses you created SPARK-33840: --- Summary: Add spark.sql.files.minPartitionNum to performence tuning doc Key: SPARK-33840 URL: https://issues.apache.org/jira/browse/SPARK-33840 Project: Spark

[jira] [Created] (SPARK-33733) PullOutNondeterministic should check and collect deterministic field

2020-12-09 Thread ulysses you (Jira)
ulysses you created SPARK-33733: --- Summary: PullOutNondeterministic should check and collect deterministic field Key: SPARK-33733 URL: https://issues.apache.org/jira/browse/SPARK-33733 Project: Spark

[jira] [Created] (SPARK-33526) Add config to control if cancel invoke interrupt task on thriftserver

2020-11-23 Thread ulysses you (Jira)
ulysses you created SPARK-33526: --- Summary: Add config to control if cancel invoke interrupt task on thriftserver Key: SPARK-33526 URL: https://issues.apache.org/jira/browse/SPARK-33526 Project: Spark

[jira] [Updated] (SPARK-33497) Override maxRows in some LogicalPlan

2020-11-19 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33497: Summary: Override maxRows in some LogicalPlan (was: Override max row in some LogicalPlan) >

[jira] [Created] (SPARK-33497) Override max row in some LogicalPlan

2020-11-19 Thread ulysses you (Jira)
ulysses you created SPARK-33497: --- Summary: Override max row in some LogicalPlan Key: SPARK-33497 URL: https://issues.apache.org/jira/browse/SPARK-33497 Project: Spark Issue Type: Improvement

  1   2   3   >