[jira] [Created] (SPARK-48619) Fix GROUP BY MapType feature
Stevo Mitric created SPARK-48619: Summary: Fix GROUP BY MapType feature Key: SPARK-48619 URL: https://issues.apache.org/jira/browse/SPARK-48619 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 3.4.3 Reporter: Stevo Mitric There are issues in the current implementation, which causes broken plans to be produced: * The InsertMapSortInGroupingExpressions transformation only replaces the grouping expressions with mapsort, not the aggregate expressions. This causes broken plans like select c from t group by mapsort(c), where c now becomes a non-aggregating expression. * Parent operators referencing the transformed columns are not updated, which means that we know have broken references. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-47634) Legacy support for map normalization
Stevo Mitric created SPARK-47634: Summary: Legacy support for map normalization Key: SPARK-47634 URL: https://issues.apache.org/jira/browse/SPARK-47634 Project: Spark Issue Type: Task Components: SQL Affects Versions: 4.0.0 Reporter: Stevo Mitric Add legacy support for creating a map without normalizing keys before inserting in `ArrayBasedMapBuilder`. Key normalization change can be found in this PR: https://issues.apache.org/jira/browse/SPARK-47563 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-47563) Map normalization upon creation
Stevo Mitric created SPARK-47563: Summary: Map normalization upon creation Key: SPARK-47563 URL: https://issues.apache.org/jira/browse/SPARK-47563 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 4.0.0 Reporter: Stevo Mitric Add handling of map normalization upon creation in ArrayBasedMapBuilder. Currently a map with keys 0.0 and -0.0 will behave as if they are separate values. This will cause issues when doing GROUP BY on map types. Refer to this conversion [https://github.com/apache/spark/pull/45549#discussion_r1537803505] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-47349) new test suite for UTF8String
[
https://issues.apache.org/jira/browse/SPARK-47349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stevo Mitric updated SPARK-47349:
-
Description:
{{CollationSuite}} is a bit over cluttered and it should be used for for E2E
testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
unit string-level tests for {{startsWith}} and {{endsWith}} functions.
Changes originally proposed in [this
PR](https://github.com/apache/spark/pull/45421#discussion_r1519451854)
was:{{CollationSuite}} is a bit over cluttered and it should be used for for
E2E testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
unit string-level tests for {{startsWith}} and {{endsWith}} functions.
> new test suite for UTF8String
> -
>
> Key: SPARK-47349
> URL: https://issues.apache.org/jira/browse/SPARK-47349
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 4.0.0
>Reporter: Uroš Bojanić
>Priority: Major
> Labels: pull-request-available
>
> {{CollationSuite}} is a bit over cluttered and it should be used for for E2E
> testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
> unit string-level tests for {{startsWith}} and {{endsWith}} functions.
>
> Changes originally proposed in [this
> PR](https://github.com/apache/spark/pull/45421#discussion_r1519451854)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-47349) new test suite for UTF8String
[
https://issues.apache.org/jira/browse/SPARK-47349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stevo Mitric updated SPARK-47349:
-
Description:
{{CollationSuite}} is a bit over cluttered and it should be used for for E2E
testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
unit string-level tests for {{startsWith}} and {{endsWith}} functions.
Changes originally proposed in [this
PR|https://github.com/apache/spark/pull/45421#discussion_r1519451854].
was:
{{CollationSuite}} is a bit over cluttered and it should be used for for E2E
testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
unit string-level tests for {{startsWith}} and {{endsWith}} functions.
Changes originally proposed in [this
PR](https://github.com/apache/spark/pull/45421#discussion_r1519451854)
> new test suite for UTF8String
> -
>
> Key: SPARK-47349
> URL: https://issues.apache.org/jira/browse/SPARK-47349
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 4.0.0
>Reporter: Uroš Bojanić
>Priority: Major
> Labels: pull-request-available
>
> {{CollationSuite}} is a bit over cluttered and it should be used for for E2E
> testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
> unit string-level tests for {{startsWith}} and {{endsWith}} functions.
>
> Changes originally proposed in [this
> PR|https://github.com/apache/spark/pull/45421#discussion_r1519451854].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-47349) new test suite for UTF8String
[
https://issues.apache.org/jira/browse/SPARK-47349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stevo Mitric updated SPARK-47349:
-
Description: {{CollationSuite}} is a bit over cluttered and it should be
used for for E2E testing. Create {{UTF8StringSuiteWithCollation}} suite which
should contain unit string-level tests for {{startsWith}} and {{endsWith}}
functions.
> new test suite for UTF8String
> -
>
> Key: SPARK-47349
> URL: https://issues.apache.org/jira/browse/SPARK-47349
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 4.0.0
>Reporter: Uroš Bojanić
>Priority: Major
>
> {{CollationSuite}} is a bit over cluttered and it should be used for for E2E
> testing. Create {{UTF8StringSuiteWithCollation}} suite which should contain
> unit string-level tests for {{startsWith}} and {{endsWith}} functions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
