vtlim commented on code in PR #17658: URL: https://github.com/apache/druid/pull/17658#discussion_r1929430492
########## docs/querying/sql-functions.md: ########## @@ -1282,21 +1282,74 @@ Returns the following: ## BLOOM_FILTER -Computes a Bloom filter from values produced by the specified expression. +Computes a [bloom filter](../development/extensions-core/bloom-filter.md) from values provided in an expression. -* **Syntax**: `BLOOM_FILTER(expr, <NUMERIC>)` +`numEntries` specifies the maximum number of distinct values before the false positive rate increases. + +* **Syntax:** `BLOOM_FILTER(expr, numEntries)` * **Function type:** Aggregation +<details><summary>Example</summary> + +The following example returns a base64-encoded bloom filter string for entries in `agent_category`: + +```sql +SELECT + agent_category, + BLOOM_FILTER(agent_category, 10) as bloom +FROM "kttm" + GROUP BY agent_category +``` + +Returns the following: + +| `agent_keys` | `bloom` | +| -- | -- | +| _`empty`_ | `"BAAAAAgAAAAAABAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEABAAAAAA"` | Review Comment: We might want to consider another way of formatting this (remove italics?) or see if it can be added to the spelling file. This line and the other instance of _`empty`_ are raising spelling errors. ########## docs/querying/sql-functions.md: ########## @@ -1282,21 +1282,74 @@ Returns the following: ## BLOOM_FILTER -Computes a Bloom filter from values produced by the specified expression. +Computes a [bloom filter](../development/extensions-core/bloom-filter.md) from values provided in an expression. -* **Syntax**: `BLOOM_FILTER(expr, <NUMERIC>)` +`numEntries` specifies the maximum number of distinct values before the false positive rate increases. + +* **Syntax:** `BLOOM_FILTER(expr, numEntries)` * **Function type:** Aggregation +<details><summary>Example</summary> + +The following example returns a base64-encoded bloom filter string for entries in `agent_category`: + +```sql +SELECT + agent_category, + BLOOM_FILTER(agent_category, 10) as bloom +FROM "kttm" + GROUP BY agent_category +``` + +Returns the following: + +| `agent_keys` | `bloom` | +| -- | -- | +| _`empty`_ | `"BAAAAAgAAAAAABAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEABAAAAAA"` | +| `Game console` | `"BAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAQAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgBAAAAAAAAAAAAAAAA"` | +| `Personal computer` | `"BAAAAAgAAAAAAEAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAA"` | +| `Smart TV` | `"BAAAAAgAAAAAAAAAAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAA"` | +| `Smartphone` | `"BAAAAAgAAACAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAA"` | +| `Tablet` | `"BAAAAAgAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAIA"` | + +</details> + [Learn more](sql-aggregations.md) ## BLOOM_FILTER_TEST -Returns true if the expression is contained in a Base64-serialized Bloom filter. +Returns true if an expression is contained in a base64-encoded [bloom filter](../development/extensions-core/bloom-filter.md) string. -* **Syntax**: `BLOOM_FILTER_TEST(expr, <STRING>)` -* **Function type:** Scalar, other +* **Syntax:** `BLOOM_FILTER_TEST(expr, <STRING>)` +* **Function type:** Aggregation Review Comment: Was this intentionally changed to aggregation? It's in the scalar function section. The SQL query example does look like it could be aggregating though. https://druid.apache.org/docs/latest/querying/sql-scalar/ https://druid.apache.org/docs/latest/querying/sql-aggregations/ ########## docs/querying/sql-functions.md: ########## @@ -1282,21 +1282,74 @@ Returns the following: ## BLOOM_FILTER -Computes a Bloom filter from values produced by the specified expression. +Computes a [bloom filter](../development/extensions-core/bloom-filter.md) from values provided in an expression. -* **Syntax**: `BLOOM_FILTER(expr, <NUMERIC>)` +`numEntries` specifies the maximum number of distinct values before the false positive rate increases. + +* **Syntax:** `BLOOM_FILTER(expr, numEntries)` * **Function type:** Aggregation +<details><summary>Example</summary> + +The following example returns a base64-encoded bloom filter string for entries in `agent_category`: Review Comment: Should Base64 be capitalized? ########## docs/querying/sql-functions.md: ########## @@ -1756,39 +1809,66 @@ Returns the following: ## DECODE_BASE64_COMPLEX -Decodes a Base64-encoded string into a complex data type, where `dataType` is the complex data type and `expr` is the Base64-encoded string to decode. +Decodes a base64-encoded expression into a complex data type. -* **Syntax**: `DECODE_BASE64_COMPLEX(dataType, expr)` -* **Function type:** Scalar, other +You can use the function to ingest data when a column contains an encoded data sketch such as Theta or HLL. -[Learn more](sql-scalar.md#other-scalar-functions) +The function supports `hyperUnique` and `serializablePairLongString` data types by default. +You can enable support for the following complex data types by [loading their extensions](../configuration/extensions.md): + +- `druid-bloom-filter`: `bloom` +- `druid-datasketches`: `arrayOfDoublesSketch`, `HLLSketch`, `KllDoublesSketch`, `KllFloatsSketch`, `quantilesDoublesSketch`, `thetaSketch` +- `druid-histogram`: `approximateHistogram`, `fixedBucketsHistogram` +- `druid-stats`: `variance` +- `druid-compressed-bigdecimal`: `compressedBigDecimal` +- `druid-momentsketch`: `momentSketch` +- `druid-tdigestsketch`: `tDigestSketch` + +* **Syntax:** `DECODE_BASE64_COMPLEX(dataType, expr)` +* **Function type:** Scalar + +<details><summary>Example</summary> + +The following example decodes a Theta sketch from a base64-encoded sketch contained in `theta_input`: + +```sql +DECODE_BASE64_COMPLEX('thetaSketch', "theta_input") +``` +The following example counts the distinct values in an encoded Theta sketch column using [`APPROX_COUNT_DISTINCT_DS_THETA`](#approx_count_distinct_ds_theta): + +```sql +APPROX_COUNT_DISTINCT_DS_THETA(DECODE_BASE64_COMPLEX('thetaSketch', "theta_input")) +``` + +</details> + +[Learn more](./sql-scalar.md#other-scalar-functions) ## DECODE_BASE64_UTF8 -Decodes a Base64-encoded string into a UTF-8 encoded string. +Decodes a base64-encoded expression into a UTF-8 encoded string. * **Syntax:** `DECODE_BASE64_UTF8(expr)` -* **Function type:** Scalar, string +* **Function type:** Scalar Review Comment: ```suggestion * **Function type:** Scalar, string ``` Stays consistent with other entries -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
