Fucun Chu has uploaded this change for review. (
http://gerrit.cloudera.org:8080/17048
Change subject: IMPALA-10467: Implement ds_theta_union() function
......................................................................
IMPALA-10467: Implement ds_theta_union() function
This function receives a set of serialized Apache DataSketches Theta
sketches produced by ds_theta_sketch() and merges them into a single
sketch.
An example usage is to create a sketch for each partition of a table,
write these sketches to a separate table and based on which partition
the user is interested of the relevant sketches can be union-ed
together to get an estimate. E.g.:
SELECT
ds_theta_estimate(ds_theta_union(sketch_col))
FROM sketch_tbl
WHERE partition_col=1 OR partition_col=5;
Testing:
- Apart from the automated tests I added to this patch I also
tested ds_theta_union() on a bigger dataset to check that
serialization, deserialization and merging steps work well. I
took TPCH25.linelitem, created a number of sketches with grouping
by l_shipdate and called ds_theta_union() on those sketches
Change-Id: I91baf58c76eb43748acd5245047edac8c66761b2
---
M be/src/exprs/aggregate-functions-ir.cc
M be/src/exprs/aggregate-functions.h
M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java
M testdata/data/README
A testdata/data/theta_sketches_from_impala.parquet
M testdata/workloads/functional-query/queries/QueryTest/datasketches-theta.test
M tests/query_test/test_datasketches.py
7 files changed, 162 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17048/1
--
To view, visit http://gerrit.cloudera.org:8080/17048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I91baf58c76eb43748acd5245047edac8c66761b2
Gerrit-Change-Number: 17048
Gerrit-PatchSet: 1
Gerrit-Owner: Fucun Chu <chufu...@hotmail.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
