Gengliang Wang created SPARK-53885:
--------------------------------------
Summary: Introduce frequency estimation functions
Key: SPARK-53885
URL: https://issues.apache.org/jira/browse/SPARK-53885
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 4.1.0
Reporter: Gengliang Wang
Assignee: Yuchuan Huang
Introduce the following frequency estimation functions:
* *APPROX_TOP_K(expr[, k[, maxItemsTracked]])* – Returns an approximate list
of the top _k_ most frequent values in the input expression using a
probabilistic algorithm.
* *APPROX_TOP_K_ACCUMULATE(expr[, maxItemsTracked])* – Creates a state object
that accumulates frequency statistics for use in later estimation or
combination.
* *APPROX_TOP_K_ESTIMATE(state [, k] ])* – Extracts and returns the
approximate top _k_ values and their estimated frequencies from an accumulated
state.
* *APPROX_TOP_K_COMBINE(expr[, maxItemsTracked])* – Merges intermediate
APPROX_TOP_K state objects, allowing distributed or parallel computation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
