Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r160918997 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583| +---+-------------------+ + Notes on grouping column: --- End diff -- From a SQL background, I think we should add grouping keys to the input of UDF. Sometimes users do need to read the grouping keys when aggregating, and we should give users a way to do it. BTW this is also consistent with Dataset, see `KeyValueGroupedDataset.mapGroups`.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org