[ 
https://issues.apache.org/jira/browse/SPARK-61527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-61527:
-----------------------------------
    Labels: pull-request-available sql  (was: sql)

> approx_top_k counts positive and negative zero as separate items
> ----------------------------------------------------------------
>
>                 Key: SPARK-61527
>                 URL: https://issues.apache.org/jira/browse/SPARK-61527
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 4.3.0
>         Environment: {code}
> Spark: 4.3.0 
> Commit: 98bc0212c94705434471627068f41e374ccab143
> Java: OpenJDK 17.0.17
> OS: macOS 15.7.4
> Master: local[2]
> {code}
>            Reporter: Yibo Dong
>            Priority: Major
>              Labels: pull-request-available, sql
>
> {{approx_top_k}} treats positive zero ({{0.0}}) and negative zero ({{-0.0}}) 
> as separate items, although Spark SQL considers them equal.
> For two input rows containing {{0.0}} and {{-0.0}}, {{approx_top_k(v, 2, 
> 10)}} returns two items with count 1 each, while {{GROUP BY}} returns one 
> group with count 2.
> h2. Steps to Reproduce
> Save the following SQL as {{tmp.sql}}:
> {code}
> WITH input AS (
> SELECT /*+ REPARTITION(3) */ v
> FROM VALUES (0D), (-0D) AS t(v)
> )
> SELECT approx_top_k(v, 2, 10) AS frequent_items
> FROM input;
> SELECT v, count(*) AS frequency
> FROM VALUES (0D), (-0D) AS t(v)
> GROUP BY v;
> {code}
> Execute:
> {code}
> spark-sql --master 'local[2]' 
> --conf spark.ui.enabled=false -f tmp.sql
> {code}
> h2. Expected Result
> The first query should return a single zero-valued item with count 2:
> {code}
> [{"item":0.0,"count":2}]
> {code}
> The sign of the zero-valued item is not important.
> Spark SQL treats positive and negative zero as equal in grouping, as 
> documented in the [SQL migration 
> guide|https://spark.apache.org/docs/4.3.0/sql-migration-guide.html].
> With only two input rows and a tracking capacity of ten items, the 
> discrepancy cannot be attributed to insufficient sketch capacity.
> h2. Actual Result
> {code}
> [{"item":0.0,"count":1},{"item":-0.0,"count":1}]
> {code}
> The two zero representations are counted separately. The independent {{GROUP 
> BY}} query correctly returns one zero-valued group with frequency 2.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to