techdocsmith commented on code in PR #17658:
URL: https://github.com/apache/druid/pull/17658#discussion_r1940186800
##########
docs/querying/sql-functions.md:
##########
@@ -1282,22 +1282,75 @@ Returns the following:
## BLOOM_FILTER
-Computes a Bloom filter from values produced by the specified expression.
+Computes a [Bloom filter](../development/extensions-core/bloom-filter.md) from
values provided in an expression.
-* **Syntax**: `BLOOM_FILTER(expr, <NUMERIC>)`
+
+* **Syntax:** `BLOOM_FILTER(expr, numEntries)`
+ `numEntries` specifies the maximum number of distinct values before the
false positive rate increases.
* **Function type:** Aggregation
+<details><summary>Example</summary>
+
+The following example returns a Base64-encoded Bloom filter string for entries
in `agent_category`:
+
+```sql
+SELECT
+ agent_category,
+ BLOOM_FILTER(agent_category, 10) as bloom
+FROM "kttm"
+ GROUP BY agent_category
+```
+
+Returns the following:
+
+| `agent_keys` | `bloom` |
Review Comment:
Suggest changing the example as suggested in line 1294, but note that the
SELECT is on `agent_category` and the results column is `agent_keys`.
##########
docs/querying/sql-functions.md:
##########
@@ -1282,22 +1282,75 @@ Returns the following:
## BLOOM_FILTER
-Computes a Bloom filter from values produced by the specified expression.
+Computes a [Bloom filter](../development/extensions-core/bloom-filter.md) from
values provided in an expression.
-* **Syntax**: `BLOOM_FILTER(expr, <NUMERIC>)`
+
+* **Syntax:** `BLOOM_FILTER(expr, numEntries)`
+ `numEntries` specifies the maximum number of distinct values before the
false positive rate increases.
* **Function type:** Aggregation
+<details><summary>Example</summary>
+
+The following example returns a Base64-encoded Bloom filter string for entries
in `agent_category`:
+
+```sql
+SELECT
+ agent_category,
+ BLOOM_FILTER(agent_category, 10) as bloom
+FROM "kttm"
+ GROUP BY agent_category
+```
+
+Returns the following:
+
+| `agent_keys` | `bloom` |
+| -- | -- |
+| `empty` |
`"BAAAAAgAAAAAABAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEABAAAAAA"`
|
+| `Game console` |
`"BAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAQAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgBAAAAAAAAAAAAAAAA"`
|
+| `Personal computer` |
`"BAAAAAgAAAAAAEAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAA"`
|
+| `Smart TV` |
`"BAAAAAgAAAAAAAAAAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAA"`
|
+| `Smartphone` |
`"BAAAAAgAAACAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAA"`
|
+| `Tablet` |
`"BAAAAAgAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAIA"`
|
+
+</details>
+
[Learn more](sql-aggregations.md)
## BLOOM_FILTER_TEST
-Returns true if the expression is contained in a Base64-serialized Bloom
filter.
+Returns true if an expression is contained in a Base64-encoded [Bloom
filter](../development/extensions-core/bloom-filter.md) string.
-* **Syntax**: `BLOOM_FILTER_TEST(expr, <STRING>)`
+* **Syntax:** `BLOOM_FILTER_TEST(expr, <STRING>)`
* **Function type:** Scalar, other
+<details><summary>Example</summary>
+
+The following example returns `true` for the Bloom filter string associated
with `agent_filter` entry `Game console`:
Review Comment:
This description doesn't describe what the example does. I suggest
simplyfying as follows:
The following example checks for the existence of the agent category `Rotary
phone` in the specified bloom filter.
```sql
SELECT BLOOM_FILTER_TEST('Rotary phone',
'BAAAAAgAAACAAFAAQAAAgAIAAkIAAAAAQAgAAAAAAAAAAgAAAAIIAAAAAgAAAAAAAAACAAIAAAgBAQAAAAAEABAAAgIA'
```
Returns the following:
|`bloom_test`|
| --- |
|`false`|
The following example checks for the existence of the agent category
`Personal computer` in the specified bloom filter:
```sql
SELECT BLOOM_FILTER_TEST('Personal computer',
'BAAAAAgAAACAAFAAQAAAgAIAAkIAAAAAQAgAAAAAAAAAAgAAAAIIAAAAAgAAAAAAAAACAAIAAAgBAQAAAAAEABAAAgIA')
AS "bloom_test"
```
Returns the following:
|`bloom_test`|
| --- |
|`true`|
##########
docs/querying/sql-functions.md:
##########
@@ -1756,39 +1809,66 @@ Returns the following:
## DECODE_BASE64_COMPLEX
-Decodes a Base64-encoded string into a complex data type, where `dataType` is
the complex data type and `expr` is the Base64-encoded string to decode.
+Decodes a Base64-encoded expression into a complex data type.
+
+You can use the function to ingest data when a column contains an encoded data
sketch such as Theta or HLL.
-* **Syntax**: `DECODE_BASE64_COMPLEX(dataType, expr)`
+The function supports `hyperUnique` and `serializablePairLongString` data
types by default.
+You can enable support for the following complex data types by [loading their
extensions](../configuration/extensions.md):
Review Comment:
```suggestion
To enable support for a complex data type load the [corresponding
extension](../configuration/extensions.md):
```
##########
docs/querying/sql-functions.md:
##########
@@ -1282,22 +1282,75 @@ Returns the following:
## BLOOM_FILTER
-Computes a Bloom filter from values produced by the specified expression.
+Computes a [Bloom filter](../development/extensions-core/bloom-filter.md) from
values provided in an expression.
-* **Syntax**: `BLOOM_FILTER(expr, <NUMERIC>)`
+
+* **Syntax:** `BLOOM_FILTER(expr, numEntries)`
+ `numEntries` specifies the maximum number of distinct values before the
false positive rate increases.
* **Function type:** Aggregation
+<details><summary>Example</summary>
+
+The following example returns a Base64-encoded Bloom filter string for entries
in `agent_category`:
Review Comment:
I don't understand this example. It doesn't make sense to have a bloom
filter per agent category. Checking the [doc], it looks like it is more common
to check for the entries for a time frame.
The following example returns a Base64-encoded Bloof filter string for
entries in `agent_category` for events before 12 pm on a specific date:
```sql
SELECT BLOOM_FILTER(agent_category, 10) as agent_bloom
FROM "kttm"
WHERE __time BETWEEN '2019-08-25T00:00:00.686Z' AND
'2019-08-25T12:00:00.686Z'
```
Returns the following :
|`agent_bloom`|
| --- |
|`"BAAAAAgAAACAAFAAQAAAgAIAAkIAAAAAQAgAAAAAAAAAAgAAAAIIAAAAAgAAAAAAAAACAAIAAAgBAQAAAAAEABAAAgIA"`|
##########
docs/querying/sql-functions.md:
##########
@@ -1756,39 +1809,66 @@ Returns the following:
## DECODE_BASE64_COMPLEX
-Decodes a Base64-encoded string into a complex data type, where `dataType` is
the complex data type and `expr` is the Base64-encoded string to decode.
+Decodes a Base64-encoded expression into a complex data type.
+
+You can use the function to ingest data when a column contains an encoded data
sketch such as Theta or HLL.
-* **Syntax**: `DECODE_BASE64_COMPLEX(dataType, expr)`
+The function supports `hyperUnique` and `serializablePairLongString` data
types by default.
+You can enable support for the following complex data types by [loading their
extensions](../configuration/extensions.md):
+
+- `druid-bloom-filter`: `bloom`
+- `druid-datasketches`: `arrayOfDoublesSketch`, `HLLSketch`,
`KllDoublesSketch`, `KllFloatsSketch`, `quantilesDoublesSketch`, `thetaSketch`
+- `druid-histogram`: `approximateHistogram`, `fixedBucketsHistogram`
+- `druid-stats`: `variance`
+- `druid-compressed-bigdecimal`: `compressedBigDecimal`
+- `druid-momentsketch`: `momentSketch`
+- `druid-tdigestsketch`: `tDigestSketch`
+
+* **Syntax:** `DECODE_BASE64_COMPLEX(dataType, expr)`
* **Function type:** Scalar, other
-[Learn more](sql-scalar.md#other-scalar-functions)
+<details><summary>Example</summary>
Review Comment:
These examples are a little different than the rest of the reference because
they refer to an imaginary value `theta_input`. My suggestion is to remove the
second example and add the following results for the first:
```sql
DECODE_BASE64_COMPLEX('thetaSketch', "theta_input") AS "theta_decoded"
```
Returns the following:
|`theta_decoded`|
|---|
|`AgMDAAAazJNBAAAAAACAP...`|
##########
docs/querying/sql-functions.md:
##########
@@ -1756,39 +1809,66 @@ Returns the following:
## DECODE_BASE64_COMPLEX
-Decodes a Base64-encoded string into a complex data type, where `dataType` is
the complex data type and `expr` is the Base64-encoded string to decode.
+Decodes a Base64-encoded expression into a complex data type.
+
+You can use the function to ingest data when a column contains an encoded data
sketch such as Theta or HLL.
-* **Syntax**: `DECODE_BASE64_COMPLEX(dataType, expr)`
+The function supports `hyperUnique` and `serializablePairLongString` data
types by default.
+You can enable support for the following complex data types by [loading their
extensions](../configuration/extensions.md):
+
+- `druid-bloom-filter`: `bloom`
+- `druid-datasketches`: `arrayOfDoublesSketch`, `HLLSketch`,
`KllDoublesSketch`, `KllFloatsSketch`, `quantilesDoublesSketch`, `thetaSketch`
+- `druid-histogram`: `approximateHistogram`, `fixedBucketsHistogram`
+- `druid-stats`: `variance`
+- `druid-compressed-bigdecimal`: `compressedBigDecimal`
+- `druid-momentsketch`: `momentSketch`
+- `druid-tdigestsketch`: `tDigestSketch`
+
+* **Syntax:** `DECODE_BASE64_COMPLEX(dataType, expr)`
* **Function type:** Scalar, other
-[Learn more](sql-scalar.md#other-scalar-functions)
+<details><summary>Example</summary>
+
+The following example decodes a Theta sketch from a Base64-encoded sketch
contained in `theta_input`:
+
+```sql
+DECODE_BASE64_COMPLEX('thetaSketch', "theta_input")
+```
+The following example counts the distinct values in an encoded Theta sketch
column using
[`APPROX_COUNT_DISTINCT_DS_THETA`](#approx_count_distinct_ds_theta):
+
+```sql
+APPROX_COUNT_DISTINCT_DS_THETA(DECODE_BASE64_COMPLEX('thetaSketch',
"theta_input"))
+```
+
+</details>
+
+[Learn more](./sql-scalar.md#other-scalar-functions)
## DECODE_BASE64_UTF8
-Decodes a Base64-encoded string into a UTF-8 encoded string.
+Decodes a Base64-encoded expression into a UTF-8 encoded string.
* **Syntax:** `DECODE_BASE64_UTF8(expr)`
* **Function type:** Scalar, string
<details><summary>Example</summary>
-The following example converts the base64 encoded string
`SGVsbG8gV29ybGQhCg==` into an UTF-8 encoded string.
+The following example decodes the Base64-encoded representation of "Hello,
World!":
```sql
SELECT
- 'SGVsbG8gV29ybGQhCg==' AS "base64_encoding",
- DECODE_BASE64_UTF8('SGVsbG8gV29ybGQhCg==') AS "convert_to_UTF8_encoding"
+ DECODE_BASE64_UTF8('SGVsbG8sIFdvcmxkIQ==') as decoded
```
Returns the following:
-| `base64_encoding` | `convert_to_UTF8_encoding` |
-| -- | -- |
-| `SGVsbG8gV29ybGQhCg==` | `Hello World!` |
+| `agent_keys` |
Review Comment:
```suggestion
| `decoded` |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
