Github user adoron commented on the issue:
https://github.com/apache/spark/pull/23239
@cloud-fan what about UnsafeRow::setDouble/Float? It doesn't go through the
same flow. Is it not used?
---
-
To unsubscribe, e
Github user adoron commented on a diff in the pull request:
https://github.com/apache/spark/pull/23141#discussion_r236189376
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -17,14 +17,16 @@ displayTitle: Spark SQL Upgrading Guide
- Since Spark 3.0, the `from_json
Github user adoron commented on a diff in the pull request:
https://github.com/apache/spark/pull/23043#discussion_r235685712
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -723,4 +723,18 @@ class DataFrameAggregateSuite extends
Github user adoron commented on a diff in the pull request:
https://github.com/apache/spark/pull/23043#discussion_r235683861
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -723,4 +723,18 @@ class DataFrameAggregateSuite extends
Github user adoron commented on a diff in the pull request:
https://github.com/apache/spark/pull/23043#discussion_r234942649
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -723,4 +723,32 @@ class DataFrameAggregateSuite extends
Github user adoron commented on the issue:
https://github.com/apache/spark/pull/23043
@kiszk is there a use case where the preliminary RDD isn't created with
UnsafeRows? If not then the data will already be corrected on reading.
Anyway, looking at all different
Github user adoron commented on a diff in the pull request:
https://github.com/apache/spark/pull/23043#discussion_r234676540
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -723,4 +723,32 @@ class DataFrameAggregateSuite extends
Github user adoron commented on a diff in the pull request:
https://github.com/apache/spark/pull/23043#discussion_r234674948
--- Diff:
common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java ---
@@ -157,4 +159,15 @@ public void heapMemoryReuse
Github user adoron commented on the issue:
https://github.com/apache/spark/pull/23043
@cloud-fan changing writeDouble/writeFloat in UnsafeWriter indeed do the
trick!
I'll fix the PR. I was thinking about making the change in
`Platform::putDouble` so all accesses get affected
Github user adoron commented on the issue:
https://github.com/apache/spark/pull/23043
@cloud-fan that's what I thought as well at first, but the flow doesn't go
through that code -
running `Seq(0.0d, 0.0d, -0.0d).toDF("i").groupBy("i").count().collect()`
and
Github user adoron commented on the issue:
https://github.com/apache/spark/pull/23043
@srowen @gatorsmile @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user adoron opened a pull request:
https://github.com/apache/spark/pull/23043
[SPARK-26021][SQL] replace minus zero with zero in UnsafeProjection
GROUP BY treats -0.0 and 0.0 as different values which is unlike hive's
behavior.
In addition current behavior with codegen
12 matches
Mail list logo
