[jira] [Created] (SPARK-47991) Arrange the test cases for window frames and window functions.

2024-04-25 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-47991: -- Summary: Arrange the test cases for window frames and window functions. Key: SPARK-47991 URL: https://issues.apache.org/jira/browse/SPARK-47991 Project: Spark

[jira] [Created] (SPARK-47795) Supplement the doc of job schedule for K8S

2024-04-10 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-47795: -- Summary: Supplement the doc of job schedule for K8S Key: SPARK-47795 URL: https://issues.apache.org/jira/browse/SPARK-47795 Project: Spark Issue Type:

[jira] [Created] (SPARK-47391) Remove the test case workaround for JDK 8

2024-03-14 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-47391: -- Summary: Remove the test case workaround for JDK 8 Key: SPARK-47391 URL: https://issues.apache.org/jira/browse/SPARK-47391 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-46929) Use ThreadUtils.shutdown to close thread pools

2024-01-30 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46929: --- Component/s: Connect Spark Core SS (was: SQL)

[jira] [Created] (SPARK-46929) Use ThreadUtils.shutdown to close thread pools

2024-01-30 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46929: -- Summary: Use ThreadUtils.shutdown to close thread pools Key: SPARK-46929 URL: https://issues.apache.org/jira/browse/SPARK-46929 Project: Spark Issue Type:

[jira] [Created] (SPARK-46895) Replace Timer with single thread scheduled executor

2024-01-28 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46895: -- Summary: Replace Timer with single thread scheduled executor Key: SPARK-46895 URL: https://issues.apache.org/jira/browse/SPARK-46895 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46882) Remove unnecessary AtomicInteger

2024-01-26 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46882: --- Summary: Remove unnecessary AtomicInteger (was: Remove unnessary AtomicInteger) > Remove

[jira] [Created] (SPARK-46882) Remove unnessary AtomicInteger

2024-01-26 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46882: -- Summary: Remove unnessary AtomicInteger Key: SPARK-46882 URL: https://issues.apache.org/jira/browse/SPARK-46882 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-46760) Make the document of spark.sql.adaptive.coalescePartitions.parallelismFirst clearer

2024-01-18 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46760: -- Summary: Make the document of spark.sql.adaptive.coalescePartitions.parallelismFirst clearer Key: SPARK-46760 URL: https://issues.apache.org/jira/browse/SPARK-46760

[jira] [Created] (SPARK-46611) Remove ThreadLocal by replace SimpleDateFormat with DateTimeFormatter

2024-01-06 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46611: -- Summary: Remove ThreadLocal by replace SimpleDateFormat with DateTimeFormatter Key: SPARK-46611 URL: https://issues.apache.org/jira/browse/SPARK-46611 Project: Spark

[jira] [Resolved] (SPARK-46494) Remove the parse rule of First, Last and Any_value

2023-12-24 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46494. Resolution: Won't Fix > Remove the parse rule of First, Last and Any_value >

[jira] [Updated] (SPARK-46494) Remove the parse rule of First, Last and Any_value

2023-12-23 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46494: --- Description: Spark have separate parse rule for First, Last and Any_value. In fact, the parse rule

[jira] [Updated] (SPARK-46494) Remove the parse rule of First, Last and Any_value

2023-12-23 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46494: --- Description: Spark have separate parse rule for Merge the parse rule of PercentileCont and

[jira] [Updated] (SPARK-46494) Remove the parse rule of First, Last and Any_value

2023-12-23 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46494: --- Description: Spark have separate parse rule for > Remove the parse rule of First, Last and

[jira] [Created] (SPARK-46494) Remove the parse rule of First, Last and Any_value

2023-12-23 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46494: -- Summary: Remove the parse rule of First, Last and Any_value Key: SPARK-46494 URL: https://issues.apache.org/jira/browse/SPARK-46494 Project: Spark Issue Type:

[jira] [Created] (SPARK-46491) Eliminate the aggregation if the group keys is the subset of the partition keys

2023-12-23 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46491: -- Summary: Eliminate the aggregation if the group keys is the subset of the partition keys Key: SPARK-46491 URL: https://issues.apache.org/jira/browse/SPARK-46491 Project:

[jira] [Assigned] (SPARK-46207) Support MergeInto in DataFrameWriterV2

2023-12-20 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-46207: -- Assignee: Huaxin Gao > Support MergeInto in DataFrameWriterV2 >

[jira] [Resolved] (SPARK-46207) Support MergeInto in DataFrameWriterV2

2023-12-20 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46207. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44119

[jira] [Updated] (SPARK-46443) Decimal precision and scale should decided by JDBC dialect.

2023-12-18 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46443: --- Summary: Decimal precision and scale should decided by JDBC dialect. (was: Ensure Decimal

[jira] [Created] (SPARK-46442) DS V2 supports push down PERCENTILE_CONT and PERCENTILE_DISC

2023-12-17 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46442: -- Summary: DS V2 supports push down PERCENTILE_CONT and PERCENTILE_DISC Key: SPARK-46442 URL: https://issues.apache.org/jira/browse/SPARK-46442 Project: Spark

[jira] [Resolved] (SPARK-46406) Assign a name to the error class _LEGACY_ERROR_TEMP_1023

2023-12-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46406. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44355

[jira] [Resolved] (SPARK-45795) DS V2 supports push down Mode

2023-12-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45795. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43661

[jira] [Assigned] (SPARK-46403) Decode parquet binary with getBytesUnsafe method

2023-12-15 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-46403: -- Assignee: Wan Kun > Decode parquet binary with getBytesUnsafe method >

[jira] [Resolved] (SPARK-46403) Decode parquet binary with getBytesUnsafe method

2023-12-15 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46403. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44351

[jira] [Created] (SPARK-46406) Assign a name to the error class _LEGACY_ERROR_TEMP_1023

2023-12-14 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46406: -- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_1023 Key: SPARK-46406 URL: https://issues.apache.org/jira/browse/SPARK-46406 Project: Spark Issue

[jira] [Resolved] (SPARK-45796) Support MODE() WITHIN GROUP (ORDER BY col)

2023-12-14 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45796. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44184

[jira] [Created] (SPARK-46363) Improve the java text block with java15 feature

2023-12-11 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46363: -- Summary: Improve the java text block with java15 feature Key: SPARK-46363 URL: https://issues.apache.org/jira/browse/SPARK-46363 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-12-11 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45649. Resolution: Fixed > Unify the prepare framework for `OffsetWindowFunctionFrame` >

[jira] [Created] (SPARK-46270) Use java14 instanceof expressions to replace the java8 instanceof statement

2023-12-05 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46270: -- Summary: Use java14 instanceof expressions to replace the java8 instanceof statement Key: SPARK-46270 URL: https://issues.apache.org/jira/browse/SPARK-46270 Project:

[jira] [Resolved] (SPARK-46009) Merge the parse rule of PercentileCont and PercentileDisc into functionCall

2023-12-04 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46009. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43910

[jira] [Updated] (SPARK-46101) Replace (string|array).size with (string|array).length in all the modules

2023-11-27 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46101: --- Summary: Replace (string|array).size with (string|array).length in all the modules (was: Replace

[jira] [Updated] (SPARK-46100) Replace (string|array).size with (string|array).length in all the modules

2023-11-27 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46100: --- Summary: Replace (string|array).size with (string|array).length in all the modules (was: Replace

[jira] [Updated] (SPARK-46100) Replace (string|array).size with (string|array).length in module core

2023-11-27 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46100: --- Summary: Replace (string|array).size with (string|array).length in module core (was: Replace

[jira] [Created] (SPARK-46101) Fix these issue in module sql

2023-11-25 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46101: -- Summary: Fix these issue in module sql Key: SPARK-46101 URL: https://issues.apache.org/jira/browse/SPARK-46101 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-46100) Fix these issue in module core

2023-11-25 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46100: -- Summary: Fix these issue in module core Key: SPARK-46100 URL: https://issues.apache.org/jira/browse/SPARK-46100 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-46098) Reduce stack depth by replace (string|array).size with (string|array).length

2023-11-25 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46098: --- Description: There are a lot of (string|array).size called. In fact, the size calls the underlying

[jira] [Updated] (SPARK-46098) Reduce stack depth by replace (string|array).size with (string|array).length

2023-11-25 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46098: --- Description: There are a lot of (string|array).size called. In fact, the size calls the underlying

[jira] [Updated] (SPARK-46098) Reduce stack depth by replace (string|array).size with (string|array).length

2023-11-25 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46098: --- Description: There are a lot of # Replace .size with .length on arrays and strings # Replace .size

[jira] [Commented] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-11-21 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788329#comment-17788329 ] Jiaan Geng commented on SPARK-45649: [~cloud_fan]I see. I will investigate this bug. > Unify the

[jira] [Updated] (SPARK-46029) Escape the single quote, _ and % for DS V2 pushdown

2023-11-21 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46029: --- Summary: Escape the single quote, _ and % for DS V2 pushdown (was: Escape the ', _ and % for DS V2

[jira] [Updated] (SPARK-46009) Merge the parse rule of PercentileCont and PercentileDisc into functionCall

2023-11-20 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46009: --- Description: Spark SQL parser have a special rule to parse

[jira] [Updated] (SPARK-46009) Merge the parse rule of PercentileCont and PercentileDisc into functionCall

2023-11-20 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46009: --- Description: Spark SQL parse have a special rule to parse

[jira] [Updated] (SPARK-46009) Merge the parse rule of PercentileCont and PercentileDisc into functionCall

2023-11-20 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46009: --- Description: Spark SQL parse have a special rule to parse

[jira] [Created] (SPARK-46009) Merge the parse rule of PercentileCont and PercentileDisc into functionCall

2023-11-20 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46009: -- Summary: Merge the parse rule of PercentileCont and PercentileDisc into functionCall Key: SPARK-46009 URL: https://issues.apache.org/jira/browse/SPARK-46009 Project:

[jira] [Created] (SPARK-45933) Runtime filter should infers more application side.

2023-11-15 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45933: -- Summary: Runtime filter should infers more application side. Key: SPARK-45933 URL: https://issues.apache.org/jira/browse/SPARK-45933 Project: Spark Issue Type:

[jira] [Created] (SPARK-45904) Mode function supports sort direction

2023-11-13 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45904: -- Summary: Mode function supports sort direction Key: SPARK-45904 URL: https://issues.apache.org/jira/browse/SPARK-45904 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-45840) Fix these issue in module sql/hive, sql/hive-thriftserver

2023-11-08 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45840: --- Summary: Fix these issue in module sql/hive, sql/hive-thriftserver (was: Fix these issue in module

[jira] [Created] (SPARK-45840) Fix these issue in module sql/hive

2023-11-08 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45840: -- Summary: Fix these issue in module sql/hive Key: SPARK-45840 URL: https://issues.apache.org/jira/browse/SPARK-45840 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45839) Fix these issue in module sql/api

2023-11-08 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45839: -- Summary: Fix these issue in module sql/api Key: SPARK-45839 URL: https://issues.apache.org/jira/browse/SPARK-45839 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45838) Fix these issue in module sql/core

2023-11-08 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45838: -- Summary: Fix these issue in module sql/core Key: SPARK-45838 URL: https://issues.apache.org/jira/browse/SPARK-45838 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45825) Fix these issue in module sql/catalyst

2023-11-08 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45825: --- Summary: Fix these issue in module sql/catalyst (was: Fix these issue in package sql/catalyst) >

[jira] [Resolved] (SPARK-45816) Return null when overflowing during casting from timestamp to integers

2023-11-08 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45816. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43694

[jira] [Assigned] (SPARK-45816) Return null when overflowing during casting from timestamp to integers

2023-11-08 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-45816: -- Assignee: L. C. Hsieh > Return null when overflowing during casting from timestamp to

[jira] [Resolved] (SPARK-45606) Release restrictions on multi-layer runtime filter

2023-11-08 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45606. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43449

[jira] [Created] (SPARK-45832) Fix 'Super method + is deprecated.'

2023-11-07 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45832: -- Summary: Fix 'Super method + is deprecated.' Key: SPARK-45832 URL: https://issues.apache.org/jira/browse/SPARK-45832 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45825) Fix these issue in package sql/catalyst

2023-11-07 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45825: -- Summary: Fix these issue in package sql/catalyst Key: SPARK-45825 URL: https://issues.apache.org/jira/browse/SPARK-45825 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Description: # Replace .size with .length on arrays and strings # The enclosing block is redundant

[jira] [Updated] (SPARK-45823) Fix some scala compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Summary: Fix some scala compile warnings (was: Fix some compile warnings) > Fix some scala compile

[jira] [Updated] (SPARK-45823) Fix some compile warnings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Summary: Fix some compile warnings (was: Fix Replace .size with .length on arrays and strings) >

[jira] [Updated] (SPARK-45823) Fix Replace .size with .length on arrays and strings

2023-11-07 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45823: --- Summary: Fix Replace .size with .length on arrays and strings (was: Fix The enclosing block is

[jira] [Created] (SPARK-45823) Fix The enclosing block is redundant

2023-11-07 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45823: -- Summary: Fix The enclosing block is redundant Key: SPARK-45823 URL: https://issues.apache.org/jira/browse/SPARK-45823 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-45793) Improve the built-in compression codecs

2023-11-06 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45793. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43659

[jira] [Resolved] (SPARK-45758) Introduce a mapper for hadoop compression codecs

2023-11-06 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45758. Resolution: Resolved > Introduce a mapper for hadoop compression codecs >

[jira] [Commented] (SPARK-45758) Introduce a mapper for hadoop compression codecs

2023-11-06 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783169#comment-17783169 ] Jiaan Geng commented on SPARK-45758: Resolved by https://github.com/apache/spark/pull/43620 >

[jira] [Updated] (SPARK-45796) Support MODE() WITHIN GROUP (ORDER BY col)

2023-11-05 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45796: --- Description: Many mainstream databases supports the syntax show below. { MODE() WITHIN GROUP (ORDER

[jira] [Created] (SPARK-45796) Support MODE() WITHIN GROUP (ORDER BY col)

2023-11-05 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45796: -- Summary: Support MODE() WITHIN GROUP (ORDER BY col) Key: SPARK-45796 URL: https://issues.apache.org/jira/browse/SPARK-45796 Project: Spark Issue Type: New

[jira] [Created] (SPARK-45795) DS V2 supports push down Mode

2023-11-05 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45795: -- Summary: DS V2 supports push down Mode Key: SPARK-45795 URL: https://issues.apache.org/jira/browse/SPARK-45795 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45793) Improve the built-in compression codecs

2023-11-04 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45793: --- Description: Currently, Spark supported many built-in compression codecs used for I/O and storage.

[jira] [Updated] (SPARK-45758) Introduce a mapper for hadoop compression codecs

2023-11-01 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45758: --- Description: Currently, Spark supported partial Hadoop compression codecs, but the Hadoop

[jira] [Created] (SPARK-45758) Introduce a mapper for hadoop compression codecs

2023-11-01 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45758: -- Summary: Introduce a mapper for hadoop compression codecs Key: SPARK-45758 URL: https://issues.apache.org/jira/browse/SPARK-45758 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-45755) Push down limit through Dataset.isEmpty()

2023-11-01 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-45755: -- Assignee: Yuming Wang > Push down limit through Dataset.isEmpty() >

[jira] [Resolved] (SPARK-45755) Push down limit through Dataset.isEmpty()

2023-11-01 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45755. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43617

[jira] [Resolved] (SPARK-45711) Introduce a mapper for avro compression codecs

2023-10-28 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45711. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43562

[jira] [Updated] (SPARK-45711) Introduce a mapper for avro compression codecs

2023-10-27 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45711: --- Description: Currently, Spark supported all the avro compression codecs, but the avro supported

[jira] [Updated] (SPARK-45711) Introduce a mapper for avro compression codecs

2023-10-27 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45711: --- Description: Currently, Spark supported all the avro compression codecs, but the avro supported

[jira] [Updated] (SPARK-45711) Introduce a mapper for avro compression codecs

2023-10-27 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45711: --- Description: Currently, Spark supported all the orc compression codecs, but the orc supported

[jira] [Resolved] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-26 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45481. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43308

[jira] [Updated] (SPARK-45664) Introduce a mapper for orc compression codecs

2023-10-25 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45664: --- Description: Currently, Spark supported all the orc compression codecs, but the orc supported

[jira] [Updated] (SPARK-45664) Introduce a mapper for orc compression codecs

2023-10-25 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45664: --- Description: Currently, Spark supported all the orc compression codecs, but the orc supported

[jira] [Updated] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-25 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45481: --- Description: Currently, Spark supported all the parquet compression codecs, but the parquet

[jira] [Updated] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-10-24 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45649: --- Summary: Unify the prepare framework for `OffsetWindowFunctionFrame` (was: Unified the prepare

[jira] [Created] (SPARK-45649) Unified the prepare framework for `OffsetWindowFunctionFrame`

2023-10-24 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45649: -- Summary: Unified the prepare framework for `OffsetWindowFunctionFrame` Key: SPARK-45649 URL: https://issues.apache.org/jira/browse/SPARK-45649 Project: Spark

[jira] [Resolved] (SPARK-45543) InferWindowGroupLimit causes bug if the other window functions haven't the same window frame as the rank-like functions

2023-10-19 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45543. Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-45606) Release restrictions on multi-layer runtime filter

2023-10-19 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45606: -- Summary: Release restrictions on multi-layer runtime filter Key: SPARK-45606 URL: https://issues.apache.org/jira/browse/SPARK-45606 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45543) InferWindowGroupLimit causes bug if the other window functions haven't the same window frame as the rank-like functions

2023-10-18 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: InferWindowGroupLimit causes bug if the other window functions haven't the same window

[jira] [Updated] (SPARK-45543) InferWindowGroupLimit causes bug if the window frame is different between rank-like functions and others

2023-10-18 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: InferWindowGroupLimit causes bug if the window frame is different between rank-like

[jira] [Updated] (SPARK-45543) InferWindowGroupLimit causes bug if the other window functions haven't the same window frame as the rank-like functions

2023-10-18 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: InferWindowGroupLimit causes bug if the other window functions haven't the same window

[jira] [Resolved] (SPARK-45484) Fix the bug that uses incorrect parquet compression codec lz4raw

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45484. Fix Version/s: 3.5.1 Resolution: Fixed Issue resolved by pull request 43330

[jira] [Updated] (SPARK-45543) WindowGroupLimit causes bug if the other window functions haven't the same window frame as the rank-like functions

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: WindowGroupLimit causes bug if the other window functions haven't the same window frame as

[jira] [Commented] (SPARK-45543) WindowGroupLimit causes bug if the other window functions haven't the same window frame as the rank-like functions

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775974#comment-17775974 ] Jiaan Geng commented on SPARK-45543: [~ronserruya] Thank you. > WindowGroupLimit causes bug if the

[jira] [Updated] (SPARK-45543) WindowGroupLimit causes bug if window expressions exists non-rank function

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: WindowGroupLimit causes bug if window expressions exists non-rank function (was:

[jira] [Updated] (SPARK-45543) WindowGroupLimit causes bug if window expressions exists non-rank window function

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: WindowGroupLimit causes bug if window expressions exists non-rank window function (was:

[jira] [Updated] (SPARK-45543) Cannot insert WindowGroupLimit if window expressions exists non-rank window function

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: Cannot insert WindowGroupLimit if window expressions exists non-rank window function

[jira] [Updated] (SPARK-45543) WindowGroupLimit Causes incorrect results when window expressions exists non-rank function

2023-10-16 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45543: --- Summary: WindowGroupLimit Causes incorrect results when window expressions exists non-rank function

  1   2   >