[
https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-16807.
----------------------------------
Resolution: Incomplete
> Optimize some ABS() statements
> ------------------------------
>
> Key: SPARK-16807
> URL: https://issues.apache.org/jira/browse/SPARK-16807
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Sylvain Zimmer
> Priority: Minor
> Labels: bulk-closed
>
> I'm not a Catalyst expert, but I think some use cases for the ABS() function
> could generate simpler code.
> This is the code generated when doing something like {{ABS(x - y) > 0}} or
> {{ABS(x - y) = 0}} in Spark SQL:
> {code}
> /* 267 */ float filter_value6 = -1.0f;
> /* 268 */ filter_value6 = agg_value27 - agg_value32;
> /* 269 */ float filter_value5 = -1.0f;
> /* 270 */ filter_value5 = (float)(java.lang.Math.abs(filter_value6));
> /* 271 */
> /* 272 */ boolean filter_value4 = false;
> /* 273 */ filter_value4 =
> org.apache.spark.util.Utils.nanSafeCompareFloats(filter_value5, 0.0f) > 0;
> /* 274 */ if (!filter_value4) continue;
> {code}
> Maybe it could all be simplified to something like this?
> {code}
> filter_value4 = (agg_value27 != agg_value32)
> {code}
> (Of course you could write {{x != y}} directly in the SQL query, but the
> {{0}} in my example could be a configurable threshold, not something you can
> hardcode)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
