[
https://issues.apache.org/jira/browse/SPARK-58788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod KC updated SPARK-58788:
-----------------------------
Description:
Add support for the ANSI SQL:2016 array function TRIM_ARRAY(array, n).
TRIM_ARRAY(array, n) returns the input array with the last `n` elements
removed. Per the SQL standard, it raises an error when `n` is negative or
greater than the cardinality of the array.
Examples:
{code:java}
SELECT trim_array(array(1, 2, 3, 4, 5), 2); -- [1, 2, 3]
SELECT trim_array(array('a', 'b', 'c'), 0); -- [a, b, c]
SELECT trim_array(array(1, 2, 3), 3); -- []
SELECT trim_array(array(1, 2, 3), -1); -- error
SELECT trim_array(array(1, 2, 3), 4); -- error (n > cardinality)
{code}
Motivation:
The function is part of the SQL standard and is supported by PostgreSQL, IBM
DB2, Trino / Presto, so its absence is a friction point when migrating those
workloads to Spark. Today users hand-roll `slice(a, 1, size(a) - n)`, which has
an off-by-one hazard and, unlike the standard, silently clamps instead of
failing when `n` exceeds the array length.
was:
Add support for the ANSI SQL:2016 array function TRIM_ARRAY(array, n).
TRIM_ARRAY(array, n) returns the input array with the last `n` elements
removed. Per the SQL standard, it raises an error when `n` is negative or
greater than the cardinality of the array.
Examples:
{code:java}
SELECT trim_array(array(1, 2, 3, 4, 5), 2); -- [1, 2, 3]
SELECT trim_array(array('a', 'b', 'c'), 0); -- [a, b, c]
SELECT trim_array(array(1, 2, 3), 3); -- []
SELECT trim_array(array(1, 2, 3), -1); -- error
SELECT trim_array(array(1, 2, 3), 4); -- error (n > cardinality)
{code}
Motivation:
The function is part of the SQL standard and is supported by Oracle, IBM DB2,
and Snowflake, so its absence is a friction point when migrating those
workloads to Spark. Today users hand-roll `slice(a, 1, size(a) - n)`, which has
an off-by-one hazard and, unlike the standard, silently clamps instead of
failing when `n` exceeds the array length.
> Add the TRIM_ARRAY array function
> ---------------------------------
>
> Key: SPARK-58788
> URL: https://issues.apache.org/jira/browse/SPARK-58788
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Vinod KC
> Priority: Minor
>
> Add support for the ANSI SQL:2016 array function TRIM_ARRAY(array, n).
> TRIM_ARRAY(array, n) returns the input array with the last `n` elements
> removed. Per the SQL standard, it raises an error when `n` is negative or
> greater than the cardinality of the array.
> Examples:
> {code:java}
> SELECT trim_array(array(1, 2, 3, 4, 5), 2); -- [1, 2, 3]
> SELECT trim_array(array('a', 'b', 'c'), 0); -- [a, b, c]
> SELECT trim_array(array(1, 2, 3), 3); -- []
> SELECT trim_array(array(1, 2, 3), -1); -- error
> SELECT trim_array(array(1, 2, 3), 4); -- error (n > cardinality)
> {code}
> Motivation:
> The function is part of the SQL standard and is supported by PostgreSQL, IBM
> DB2, Trino / Presto, so its absence is a friction point when migrating those
> workloads to Spark. Today users hand-roll `slice(a, 1, size(a) - n)`, which
> has an off-by-one hazard and, unlike the standard, silently clamps instead of
> failing when `n` exceeds the array length.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
