Vinod KC created SPARK-58788:
--------------------------------
Summary: Add the TRIM_ARRAY array function
Key: SPARK-58788
URL: https://issues.apache.org/jira/browse/SPARK-58788
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.4.0
Reporter: Vinod KC
Add support for the ANSI SQL:2016 array function TRIM_ARRAY(array, n).
TRIM_ARRAY(array, n) returns the input array with the last `n` elements
removed. Per the SQL standard, it raises an error when `n` is negative or
greater than the cardinality of the array.
Examples:
{code:java}
SELECT trim_array(array(1, 2, 3, 4, 5), 2); -- [1, 2, 3]
SELECT trim_array(array('a', 'b', 'c'), 0); -- [a, b, c]
SELECT trim_array(array(1, 2, 3), 3); -- []
SELECT trim_array(array(1, 2, 3), -1); -- error
SELECT trim_array(array(1, 2, 3), 4); -- error (n > cardinality)
{code}
Motivation:
The function is part of the SQL standard and is supported by Oracle, IBM
DB2, and Snowflake, so its absence is a friction point when migrating those
workloads to Spark. Today users hand-roll `slice(a, 1, size(a) - n)`, which
has an off-by-one hazard and, unlike the standard, silently clamps instead
of failing when `n` exceeds the array length.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
