[ https://issues.apache.org/jira/browse/SPARK-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073098#comment-15073098 ]
Tristan commented on SPARK-12491: --------------------------------- Confirmed: Second 'show' of the problematic query works fine. {code} scala> import com.pipeline.spark._ import com.pipeline.spark._ scala> sqlContext.udf.register("gm", new GeometricMean) res0: org.apache.spark.sql.expressions.UserDefinedAggregateFunction = com.pipeline.spark.GeometricMean@1b7a2459 scala> import org.apache.spark.sql.functions._ import org.apache.spark.sql.functions._ scala> val ids = sqlContext.range(1, 20) ids: org.apache.spark.sql.DataFrame = [id: bigint] scala> ids.registerTempTable("ids") scala> val df = sqlContext.sql("select id, id % 3 as group_id from ids") df: org.apache.spark.sql.DataFrame = [id: bigint, group_id: bigint] scala> df.registerTempTable("simple") scala> sqlContext.sql("select group_id, gm(id) from simple group by group_id").show() +--------+---+ |group_id|_c1| +--------+---+ | 0|0.0| | 1|0.0| | 2|0.0| +--------+---+ scala> sqlContext.sql("select group_id, gm(id) from simple group by group_id").show() +--------+-----------------+ |group_id| _c1| +--------+-----------------+ | 0|8.981385496571725| | 1|7.301716979342118| | 2|7.706253151292568| +--------+-----------------+ scala> sqlContext.sql("select group_id, gm(id) as geomean from simple group by group_id").show() +--------+-----------------+ |group_id| geomean| +--------+-----------------+ | 0|8.981385496571725| | 1|7.301716979342118| | 2|7.706253151292568| +--------+-----------------+ {code} > UDAF result differs in SQL if alias is used > ------------------------------------------- > > Key: SPARK-12491 > URL: https://issues.apache.org/jira/browse/SPARK-12491 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.5.2 > Reporter: Tristan > Attachments: UDAF_GM.zip > > > Using the GeometricMean UDAF example > (https://databricks.com/blog/2015/09/16/spark-1-5-dataframe-api-highlights-datetimestring-handling-time-intervals-and-udafs.html), > I found the following discrepancy in results: > {code} > scala> sqlContext.sql("select group_id, gm(id) from simple group by > group_id").show() > +--------+---+ > |group_id|_c1| > +--------+---+ > | 0|0.0| > | 1|0.0| > | 2|0.0| > +--------+---+ > scala> sqlContext.sql("select group_id, gm(id) as GeometricMean from simple > group by group_id").show() > +--------+-----------------+ > |group_id| GeometricMean| > +--------+-----------------+ > | 0|8.981385496571725| > | 1|7.301716979342118| > | 2|7.706253151292568| > +--------+-----------------+ > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org