huaxingao commented on a change in pull request #28087:
[SPARK-31319][SQL][DOCS] Document UDFs/UDAFs in SQL Reference
URL: https://github.com/apache/spark/pull/28087#discussion_r401974890
##########
File path: docs/sql-ref-functions-udf-aggregate.md
##########
@@ -21,17 +21,79 @@ license: |
### Description
-User-Defined Aggregate Functions (UDAFs) are user-programmable routines that
act on multiple rows at once and return a single aggregated value as a result.
This documentation contains examples that demonstrate how to define and
register UDAFs in Scala and invoke them in Spark SQL.
+User-Defined Aggregate Functions (UDAFs) are user-programmable routines that
act on multiple rows at once and return a single aggregated value as a result.
This documentation lists the classes that are required for creating and
registering UDAFs. It also contains examples that demonstrate how to define and
register UDAFs in Scala and invoke them in Spark SQL.
+
+### org.apache.spark.sql.expressions.Aggregator[-IN, BUF, OUT]
+
+A base class for user-defined aggregations, which can be used in Dataset
operations to take all of the elements of a group and reduce them to a single
value.
+- IN The input type for the aggregation.
+
+- BUF The type of the intermediate value of the reduction.
+
+- OUT The type of the final output result.
+
+<dl>
+ <dt><code><em>bufferEncoder: Encoder[BUF]</em></code></dt>
+ <dd>
+ Register a deterministic Java UDF(0-22) instance as user-defined function
(UDF).
+ </dd>
+</dl>
+
+
+<dl>
+ <dt><code><em>finish(reduction: BUF): OUT</em></code></dt>
+ <dd>
+ Transform the output of the reduction.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>merge(b1: BUF, b2: BUF): BUF</em></code></dt>
+ <dd>
+ Merge two intermediate values.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>outputEncoder: Encoder[OUT]</em></code></dt>
+ <dd>
+ Specifies the Encoder for the final output value type.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>reduce(b: BUF, a: IN): BUF</em></code></dt>
+ <dd>
+ Combine two values to produce a new value. For performance, the function
may modify b and return it instead of constructing new object for b.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>zero: BUF</em></code></dt>
+ <dd>
+ A zero value for this aggregation.
+ </dd>
+</dl>
+
+### org.apache.spark.sql.UDFRegistration
+
+Functions for registering user-defined functions. Use `SparkSession.udf` to
access this: `spark.udf`
+
+<dl>
+ <dt><code><em>register(name: String, udf: UserDefinedFunction):
UserDefinedFunction</em></code></dt>
+ <dd>
+ Registers a user-defined function (UDF).
+ </dd>
+</dl>
### Examples
{% highlight sql %}
// Define and register a UDAF to calculate the sum of product of two columns
// Scala
-import org.apache.spark.sql.Encoder
Review comment:
I am confused with this import. My test works OK without it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
