[jira] [Resolved] (SPARK-29119) DEFAULT option is not supported in Spark
[ https://issues.apache.org/jira/browse/SPARK-29119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29119. -- Resolution: Duplicate > DEFAULT option is not supported in Spark > > > Key: SPARK-29119 > URL: https://issues.apache.org/jira/browse/SPARK-29119 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Major > > PostgreSQL supports *default* option as below > **CREATE TABLE update_test ( > a INT *DEFAULT* 10, > b INT > ); > INSERT INTO update_test VALUES (5, 10); > INSERT INTO update_test(b) VALUES (15); > SELECT * FROM update_test; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29713) Support Interval Unit Abbreviations in Interval Literals
[ https://issues.apache.org/jira/browse/SPARK-29713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004671#comment-17004671 ] Takeshi Yamamuro commented on SPARK-29713: -- I'll close this because the corresponding pr has been closed. If necessary, please reopen this. Thanks, again. > Support Interval Unit Abbreviations in Interval Literals > > > Key: SPARK-29713 > URL: https://issues.apache.org/jira/browse/SPARK-29713 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Kent Yao >Priority: Major > > "year" | "years" | "y" | "yr" | "yrs" => YEAR > "month" | "months" | "mon" | "mons" => MONTH > "week" | "weeks" | "w" => WEEK > "day" | "days" | "d" => DAY > "hour" | "hours" | "h" | "hr" | "hrs" => HOUR > "minute" | "minutes" | "m" | "min" | "mins" => MINUTE > "second" | "seconds" | "s" | "sec" | "secs" => SECOND > "millisecond" | "milliseconds" | "ms" | "msec" | "msecs" | "mseconds" => > MILLISECOND > "microsecond" | "microseconds" | "us" | "usec" | "usecs" | "useconds" => > MICROSECOND -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-29713) Support Interval Unit Abbreviations in Interval Literals
[ https://issues.apache.org/jira/browse/SPARK-29713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29713. -- Resolution: Won't Fix > Support Interval Unit Abbreviations in Interval Literals > > > Key: SPARK-29713 > URL: https://issues.apache.org/jira/browse/SPARK-29713 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Kent Yao >Priority: Major > > "year" | "years" | "y" | "yr" | "yrs" => YEAR > "month" | "months" | "mon" | "mons" => MONTH > "week" | "weeks" | "w" => WEEK > "day" | "days" | "d" => DAY > "hour" | "hours" | "h" | "hr" | "hrs" => HOUR > "minute" | "minutes" | "m" | "min" | "mins" => MINUTE > "second" | "seconds" | "s" | "sec" | "secs" => SECOND > "millisecond" | "milliseconds" | "ms" | "msec" | "msecs" | "mseconds" => > MILLISECOND > "microsecond" | "microseconds" | "us" | "usec" | "usecs" | "useconds" => > MICROSECOND -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-27930) List all built-in UDFs have different names
[
https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-27930.
--
Resolution: Duplicate
> List all built-in UDFs have different names
> ---
>
> Key: SPARK-27930
> URL: https://issues.apache.org/jira/browse/SPARK-27930
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> This ticket list all built-in UDFs have different names:
> ||PostgreSQL||Spark SQL||Note||
> |random|rand| |
> |format|format_string|Spark's {{format_string}} is based on the
> implementation of {{java.util.Formatter}}.
> Which makes some formats of PostgreSQL can not supported, such as:
> {{format_string('>>%-s<<', 'Hello')}}|
> |to_hex|hex| |
> |strpos|locate/position| |
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27930) List all built-in UDFs have different names
[
https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004669#comment-17004669
]
Takeshi Yamamuro commented on SPARK-27930:
--
This ticket is not obvious and it seems this ticket is duplicated by some jira
tickets, e.g., https://issues.apache.org/jira/browse/SPARK-28086. So, I'll
close for now.
> List all built-in UDFs have different names
> ---
>
> Key: SPARK-27930
> URL: https://issues.apache.org/jira/browse/SPARK-27930
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> This ticket list all built-in UDFs have different names:
> ||PostgreSQL||Spark SQL||Note||
> |random|rand| |
> |format|format_string|Spark's {{format_string}} is based on the
> implementation of {{java.util.Formatter}}.
> Which makes some formats of PostgreSQL can not supported, such as:
> {{format_string('>>%-s<<', 'Hello')}}|
> |to_hex|hex| |
> |strpos|locate/position| |
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names
[
https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27930:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> List all built-in UDFs have different names
> ---
>
> Key: SPARK-27930
> URL: https://issues.apache.org/jira/browse/SPARK-27930
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> This ticket list all built-in UDFs have different names:
> ||PostgreSQL||Spark SQL||Note||
> |random|rand| |
> |format|format_string|Spark's {{format_string}} is based on the
> implementation of {{java.util.Formatter}}.
> Which makes some formats of PostgreSQL can not supported, such as:
> {{format_string('>>%-s<<', 'Hello')}}|
> |to_hex|hex| |
> |strpos|locate/position| |
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27764) Feature Parity between PostgreSQL and Spark
[ https://issues.apache.org/jira/browse/SPARK-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27764: - Description: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent behaviours and ANS/SQL standard topics, you need to check the two umbrella below; - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) - SPARK-30375 Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours) was: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent issues and ANS/SQL standard issues, you need to check the two umbrella below; - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) - SPARK-30375 Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours) > Feature Parity between PostgreSQL and Spark > --- > > Key: SPARK-27764 > URL: https://issues.apache.org/jira/browse/SPARK-27764 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.0.0 >Reporter: Xiao Li >Priority: Major > > PostgreSQL is one of the most advanced open source databases. This umbrella > Jira is trying to track the missing features and bugs. > UPDATE: This umbrella tickets basically intend to include bug reports and > general issues for the feature parity. For implementation-dependent > behaviours and ANS/SQL standard topics, you need to check the two umbrella > below; > - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) > - SPARK-30375 Feature Parity between PostgreSQL and Spark > (implementation-dependent behaviours) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27764) Feature Parity between PostgreSQL and Spark
[ https://issues.apache.org/jira/browse/SPARK-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27764: - Description: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent issues and ANS/SQL standard issues, you need to check the two umbrella below; - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) - SPARK-30375 Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours) was: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent issues and ANS/SQL standard issues, you need to check the two umbrella below; - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) > Feature Parity between PostgreSQL and Spark > --- > > Key: SPARK-27764 > URL: https://issues.apache.org/jira/browse/SPARK-27764 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.0.0 >Reporter: Xiao Li >Priority: Major > > PostgreSQL is one of the most advanced open source databases. This umbrella > Jira is trying to track the missing features and bugs. > UPDATE: This umbrella tickets basically intend to include bug reports and > general issues for the feature parity. For implementation-dependent issues > and ANS/SQL standard issues, you need to check the two umbrella below; > - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) > - SPARK-30375 Feature Parity between PostgreSQL and Spark > (implementation-dependent behaviours) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27764) Feature Parity between PostgreSQL and Spark
[ https://issues.apache.org/jira/browse/SPARK-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27764: - Description: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent issues and ANS/SQL standard issues, you need to check the two umbrella below; - was: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. > Feature Parity between PostgreSQL and Spark > --- > > Key: SPARK-27764 > URL: https://issues.apache.org/jira/browse/SPARK-27764 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.0.0 >Reporter: Xiao Li >Priority: Major > > PostgreSQL is one of the most advanced open source databases. This umbrella > Jira is trying to track the missing features and bugs. > UPDATE: This umbrella tickets basically intend to include bug reports and > general issues for the feature parity. For implementation-dependent issues > and ANS/SQL standard issues, you need to check the two umbrella below; > - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27764) Feature Parity between PostgreSQL and Spark
[ https://issues.apache.org/jira/browse/SPARK-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27764: - Description: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent issues and ANS/SQL standard issues, you need to check the two umbrella below; - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) was: PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. UPDATE: This umbrella tickets basically intend to include bug reports and general issues for the feature parity. For implementation-dependent issues and ANS/SQL standard issues, you need to check the two umbrella below; - > Feature Parity between PostgreSQL and Spark > --- > > Key: SPARK-27764 > URL: https://issues.apache.org/jira/browse/SPARK-27764 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.0.0 >Reporter: Xiao Li >Priority: Major > > PostgreSQL is one of the most advanced open source databases. This umbrella > Jira is trying to track the missing features and bugs. > UPDATE: This umbrella tickets basically intend to include bug reports and > general issues for the feature parity. For implementation-dependent issues > and ANS/SQL standard issues, you need to check the two umbrella below; > - SPARK-30374 Feature Parity between PostgreSQL and Spark (ANSI/SQL) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28377) Fully support correlation names in the FROM clause
[
https://issues.apache.org/jira/browse/SPARK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28377:
-
Parent Issue: SPARK-30374 (was: SPARK-27764)
> Fully support correlation names in the FROM clause
> --
>
> Key: SPARK-28377
> URL: https://issues.apache.org/jira/browse/SPARK-28377
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Specifying a list of column names is not fully support. Example:
> {code:sql}
> create or replace temporary view J1_TBL as select * from
> (values (1, 4, 'one'), (2, 3, 'two'))
> as v(i, j, t);
> create or replace temporary view J2_TBL as select * from
> (values (1, -1), (2, 2))
> as v(i, k);
> SELECT '' AS xxx, t1.a, t2.e
> FROM J1_TBL t1 (a, b, c), J2_TBL t2 (d, e)
> WHERE t1.a = t2.d;
> {code}
> PostgreSQL:
> {noformat}
> postgres=# SELECT '' AS xxx, t1.a, t2.e
> postgres-# FROM J1_TBL t1 (a, b, c), J2_TBL t2 (d, e)
> postgres-# WHERE t1.a = t2.d;
> xxx | a | e
> -+---+
> | 1 | -1
> | 2 | 2
> (2 rows)
> {noformat}
> Spark SQL:
> {noformat}
> spark-sql> SELECT '' AS xxx, t1.a, t2.e
> > FROM J1_TBL t1 (a, b, c), J2_TBL t2 (d, e)
> > WHERE t1.a = t2.d;
> Error in query: cannot resolve '`t1.a`' given input columns: [a, b, c, d, e];
> line 3 pos 8;
> 'Project [ AS xxx#21, 't1.a, 't2.e]
> +- 'Filter ('t1.a = 't2.d)
>+- Join Inner
> :- Project [i#14 AS a#22, j#15 AS b#23, t#16 AS c#24]
> : +- SubqueryAlias `t1`
> : +- SubqueryAlias `j1_tbl`
> :+- Project [i#14, j#15, t#16]
> : +- Project [col1#11 AS i#14, col2#12 AS j#15, col3#13 AS
> t#16]
> : +- SubqueryAlias `v`
> : +- LocalRelation [col1#11, col2#12, col3#13]
> +- Project [i#19 AS d#25, k#20 AS e#26]
> +- SubqueryAlias `t2`
> +- SubqueryAlias `j2_tbl`
>+- Project [i#19, k#20]
> +- Project [col1#17 AS i#19, col2#18 AS k#20]
> +- SubqueryAlias `v`
> +- LocalRelation [col1#17, col2#18]
> {noformat}
>
> *Feature ID*: E051-08
> [https://www.postgresql.org/docs/11/sql-expressions.html]
> [https://www.ibm.com/support/knowledgecenter/en/SSEPEK_10.0.0/sqlref/src/tpc/db2z_correlationnames.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29737) Concat for array in Spark SQL is not the one in PostgreSQL but array_cat
[
https://issues.apache.org/jira/browse/SPARK-29737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29737:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Concat for array in Spark SQL is not the one in PostgreSQL but array_cat
>
>
> Key: SPARK-29737
> URL: https://issues.apache.org/jira/browse/SPARK-29737
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Kent Yao
>Priority: Major
>
> {code:sql}
> postgres=# select array_cat(array[1,2], array[2]);
> array_cat
> ---
> {1,2,2}
> (1 row)
> postgres=# select concat(array[1,2], array[2]);
> concat
> --
> {1,2}{2}
> (1 row)
> {code}
> {code:sql}
> // Some comments here
> spark-sql> select concat(array(1,2), array(2));
> [1,2,2]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28448) Implement ILIKE operator
[
https://issues.apache.org/jira/browse/SPARK-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28448:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Implement ILIKE operator
>
>
> Key: SPARK-28448
> URL: https://issues.apache.org/jira/browse/SPARK-28448
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> The key word {{ILIKE}} can be used instead of {{LIKE}} to make the match
> case-insensitive according to the active locale. This is not in the SQL
> standard but is a PostgreSQL extension.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27879) Implement bitwise integer aggregates(BIT_AND and BIT_OR)
[
https://issues.apache.org/jira/browse/SPARK-27879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27879:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Implement bitwise integer aggregates(BIT_AND and BIT_OR)
>
>
> Key: SPARK-27879
> URL: https://issues.apache.org/jira/browse/SPARK-27879
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Kent Yao
>Priority: Major
> Fix For: 3.0.0
>
>
> {code:java}
> bit_and(expression) -- The bitwise AND of all non-null input values, or null
> if none
> {code}
> {code:java}
> bit_or(expression) -- The bitwise OR of all non-null input values, or null if
> none
> {code}
> More details:
> [https://www.postgresql.org/docs/9.3/functions-aggregate.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29849) Spark trunc() func does not support for number group as PostgreSQL
[ https://issues.apache.org/jira/browse/SPARK-29849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29849: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Spark trunc() func does not support for number group as PostgreSQL > -- > > Key: SPARK-29849 > URL: https://issues.apache.org/jira/browse/SPARK-29849 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > > PostgreSQL trunc() function accepts number group as below > SELECT trunc(1234567891.1234567891,4); > output > |1|1234567891,1234| > Spark does not accept > jdbc:hive2://10.18.19.208:23040/default> SELECT > trunc(1234567891.1234567891D,4); > Error: org.apache.spark.sql.AnalysisException: cannot resolve > 'trunc(1.2345678911234567E9D, CAST(4 AS STRING))' due to data type mismatch: > argument 1 requires date type, however, '1.2345678911234567E9D' is of double > type.; line 1 pos 7; > 'Project [unresolvedalias(trunc(1.2345678911234567E9, cast(4 as string)), > None)] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28007) Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift
[
https://issues.apache.org/jira/browse/SPARK-28007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28007:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in
> Postgres/Redshift
> --
>
> Key: SPARK-28007
> URL: https://issues.apache.org/jira/browse/SPARK-28007
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Josh Rosen
>Priority: Major
>
> The expression {{expr1 ^ expr2}} has different meanings in Spark and Postgres:
> * [In Postgres|https://www.postgresql.org/docs/11/functions-math.html] and
> [Redshift|https://docs.aws.amazon.com/redshift/latest/dg/r_OPERATOR_SYMBOLS.html]
> , this returns {{expr1}} raised to the exponent {{expr2}} (additionally, the
> Postgres docs explicitly state that this operation is left-associative).
> * [In Spark|https://spark.apache.org/docs/2.4.3/api/sql/index.html#_14] and
> [Hive|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-ArithmeticOperators],
> this returns the bitwise exclusive OR of {{expr1}} and {{expr2}}.
> I'm reporting this under the Postgres compatibility umbrella. If we have SQL
> dialect support (e.g. a Postgres compatibility dialect), maybe this behavior
> could be flagged there? Alternatively, maybe we could document this in a
> compatibility / porting guide?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28147) Support RETURNING cause
[ https://issues.apache.org/jira/browse/SPARK-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004665#comment-17004665 ] Takeshi Yamamuro commented on SPARK-28147: -- I'll close for now because I think this feature is not useful for Spark. > Support RETURNING cause > --- > > Key: SPARK-28147 > URL: https://issues.apache.org/jira/browse/SPARK-28147 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Peter Toth >Priority: Major > > PostgreSQL supports RETURNING cause on INSERT/UPDATE/DELETE statements to > return date from the modified rows. > [https://www.postgresql.org/docs/9.5/dml-returning.html] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-28147) Support RETURNING cause
[ https://issues.apache.org/jira/browse/SPARK-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28147. -- Resolution: Won't Fix > Support RETURNING cause > --- > > Key: SPARK-28147 > URL: https://issues.apache.org/jira/browse/SPARK-28147 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Peter Toth >Priority: Major > > PostgreSQL supports RETURNING cause on INSERT/UPDATE/DELETE statements to > return date from the modified rows. > [https://www.postgresql.org/docs/9.5/dml-returning.html] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28147) Support RETURNING cause
[ https://issues.apache.org/jira/browse/SPARK-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28147: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Support RETURNING cause > --- > > Key: SPARK-28147 > URL: https://issues.apache.org/jira/browse/SPARK-28147 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Peter Toth >Priority: Major > > PostgreSQL supports RETURNING cause on INSERT/UPDATE/DELETE statements to > return date from the modified rows. > [https://www.postgresql.org/docs/9.5/dml-returning.html] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28298) Fully support char and varchar types
[
https://issues.apache.org/jira/browse/SPARK-28298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28298:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Fully support char and varchar types
>
>
> Key: SPARK-28298
> URL: https://issues.apache.org/jira/browse/SPARK-28298
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Zhu, Lipeng
>Priority: Major
>
> Execute below SQL in Spark, the result is "abcdef". But the result of other
> DBMS is "abc"(I think this is more sensible).
> {code:sql}
> select cast("abcdef" as char(3));
> {code}
> And then I checked the source code, seems char/varchar only be used in DDL
> parse.
> {code:java}
> /**
> * Hive char type. Similar to other HiveStringType's, these datatypes should
> only used for
> * parsing, and should NOT be used anywhere else. Any instance of these data
> types should be
> * replaced by a [[StringType]] before analysis.
> */
> case class CharType(length: Int) extends HiveStringType {
> override def simpleString: String = s"char($length)"
> }
> /**
> * Hive varchar type. Similar to other HiveStringType's, these datatypes
> should only used for
> * parsing, and should NOT be used anywhere else. Any instance of these data
> types should be
> * replaced by a [[StringType]] before analysis.
> */
> case class VarcharType(length: Int) extends HiveStringType {
> override def simpleString: String = s"varchar($length)"
> }
> {code}
> Is this behavior expected?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28061) Support for converting float/double to binary format
[
https://issues.apache.org/jira/browse/SPARK-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004664#comment-17004664
]
Takeshi Yamamuro commented on SPARK-28061:
--
I'll close this because these functions are internal in postgresql and I think
we don't need to support this in spark.
> Support for converting float/double to binary format
>
>
> Key: SPARK-28061
> URL: https://issues.apache.org/jira/browse/SPARK-28061
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Examples:
> {code:sql}
> SELECT float4send('5e-20'::float4);
> SELECT float4send('67e14'::float4);
> SELECT float4send('985e15'::float4);
> SELECT float4send('55895e-16'::float4);
> SELECT float4send('7038531e-32'::float4);
> SELECT float4send('702990899e-20'::float4);
> {code}
> float4send:
> https://github.com/postgres/postgres/blob/16d489b0fe058e527619f5e9d92fd7ca3c6c2994/src/backend/utils/adt/float.c#L314-L326
> float8send:
> https://github.com/postgres/postgres/blob/16d489b0fe058e527619f5e9d92fd7ca3c6c2994/src/backend/utils/adt/float.c#L566-L578
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-28061) Support for converting float/double to binary format
[
https://issues.apache.org/jira/browse/SPARK-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-28061.
--
Resolution: Won't Fix
> Support for converting float/double to binary format
>
>
> Key: SPARK-28061
> URL: https://issues.apache.org/jira/browse/SPARK-28061
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Examples:
> {code:sql}
> SELECT float4send('5e-20'::float4);
> SELECT float4send('67e14'::float4);
> SELECT float4send('985e15'::float4);
> SELECT float4send('55895e-16'::float4);
> SELECT float4send('7038531e-32'::float4);
> SELECT float4send('702990899e-20'::float4);
> {code}
> float4send:
> https://github.com/postgres/postgres/blob/16d489b0fe058e527619f5e9d92fd7ca3c6c2994/src/backend/utils/adt/float.c#L314-L326
> float8send:
> https://github.com/postgres/postgres/blob/16d489b0fe058e527619f5e9d92fd7ca3c6c2994/src/backend/utils/adt/float.c#L566-L578
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28061) Support for converting float/double to binary format
[
https://issues.apache.org/jira/browse/SPARK-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28061:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support for converting float/double to binary format
>
>
> Key: SPARK-28061
> URL: https://issues.apache.org/jira/browse/SPARK-28061
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Examples:
> {code:sql}
> SELECT float4send('5e-20'::float4);
> SELECT float4send('67e14'::float4);
> SELECT float4send('985e15'::float4);
> SELECT float4send('55895e-16'::float4);
> SELECT float4send('7038531e-32'::float4);
> SELECT float4send('702990899e-20'::float4);
> {code}
> float4send:
> https://github.com/postgres/postgres/blob/16d489b0fe058e527619f5e9d92fd7ca3c6c2994/src/backend/utils/adt/float.c#L314-L326
> float8send:
> https://github.com/postgres/postgres/blob/16d489b0fe058e527619f5e9d92fd7ca3c6c2994/src/backend/utils/adt/float.c#L566-L578
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27880) Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY)
[
https://issues.apache.org/jira/browse/SPARK-27880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27880:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY)
> -
>
> Key: SPARK-27880
> URL: https://issues.apache.org/jira/browse/SPARK-27880
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Kent Yao
>Priority: Major
> Fix For: 3.0.0
>
>
> {code:sql}
> bool_and/booland_statefunc(expression) -- true if all input values are true,
> otherwise false
> {code}
> {code:sql}
> bool_or/boolor_statefunc(expression) -- true if at least one input value is
> true, otherwise false
> {code}
> {code:sql}
> every(expression) -- equivalent to bool_and
> {code}
> More details:
> [https://www.postgresql.org/docs/9.3/functions-aggregate.html]
>
> Presto and Vertica also support this feature:
> https://prestodb.github.io/docs/current/functions/aggregate.html
> https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Aggregate/AggregateFunctions.htm?tocpath=SQL%20Reference%20Manual%7CSQL%20Functions%7CAggregate%20Functions%7C_0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-29718) Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE
[ https://issues.apache.org/jira/browse/SPARK-29718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29718. -- Resolution: Won't Fix > Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE > --- > > Key: SPARK-29718 > URL: https://issues.apache.org/jira/browse/SPARK-29718 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > 5.10. Table Partitioning: > https://www.postgresql.org/docs/current/ddl-partitioning.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29718) Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE
[ https://issues.apache.org/jira/browse/SPARK-29718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004661#comment-17004661 ] Takeshi Yamamuro commented on SPARK-29718: -- I'll close this for now because I think this is useless for Spark. > Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE > --- > > Key: SPARK-29718 > URL: https://issues.apache.org/jira/browse/SPARK-29718 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > 5.10. Table Partitioning: > https://www.postgresql.org/docs/current/ddl-partitioning.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28690) Date/Time Functions: date_part for timestamps
[
https://issues.apache.org/jira/browse/SPARK-28690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28690:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Date/Time Functions: date_part for timestamps
> -
>
> Key: SPARK-28690
> URL: https://issues.apache.org/jira/browse/SPARK-28690
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Maxim Gekk
>Assignee: Maxim Gekk
>Priority: Major
> Fix For: 3.0.0
>
>
> ||Function||Return Type||Description||Example||Result||
> |{{date_part(}}{{text}}{{, }}{{timestamp}}{{)}}|{{double precision}}|Get
> subfield (equivalent to {{extract}}); see [Section
> 9.9.1|https://www.postgresql.org/docs/11/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT]|{{date_part('hour',
> timestamp '2001-02-16 20:38:40')}}|{{20}}|
> We can replace it with {{extract(field from timestamp)}}.
> https://www.postgresql.org/docs/11/functions-datetime.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28663) Aggregate Functions for Statistics
[
https://issues.apache.org/jira/browse/SPARK-28663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28663:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Aggregate Functions for Statistics
> --
>
> Key: SPARK-28663
> URL: https://issues.apache.org/jira/browse/SPARK-28663
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Argument Type||Return Type||Partial Mode||Description||
> |{{corr(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|correlation coefficient|
> |{{covar_pop(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|population covariance|
> |{{covar_samp(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|sample covariance|
> |{{regr_avgx(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|average of the independent variable
> ({{sum(_{{X_}})/_{{N}}_}})|
> |{{regr_avgy(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|average of the dependent variable ({{sum(_{{Y_}})/_{{N}}_}})|
> |{{regr_count(_Y_}}, _{{X}}_)|{{double precision}}|{{bigint}}|Yes|number of
> input rows in which both expressions are nonnull|
> |{{regr_intercept(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|y-intercept of the least-squares-fit linear equation
> determined by the (_{{X}}_, _{{Y}}_) pairs|
> |{{regr_r2(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|square of the correlation coefficient|
> |{{regr_slope(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|slope of the least-squares-fit linear equation determined by
> the (_{{X}}_, _{{Y}}_) pairs|
> |{{regr_sxx(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|{{sum(_{{X_}}^2) - sum(_{{X}}_)^2/_{{N}}_}} (“sum of squares”
> of the independent variable)|
> |{{regr_sxy(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|{{sum(_{{X_}}*_{{Y}}_) - sum(_{{X}}_) *
> sum(_{{Y}}_)/_{{N}}_}} (“sum of products”of independent times dependent
> variable)|
> |{{regr_syy(_Y_}}, _{{X}}_)|{{double precision}}|{{double
> precision}}|Yes|{{sum(_{{Y_}}^2) - sum(_{{Y}}_)^2/_{{N}}_}} (“sum of squares”
> of the dependent variable)|
> [https://www.postgresql.org/docs/11/functions-aggregate.html#FUNCTIONS-AGGREGATE-STATISTICS-TABLE]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29514) String function: string_to_array
[
https://issues.apache.org/jira/browse/SPARK-29514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29514:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String function: string_to_array
>
>
> Key: SPARK-29514
> URL: https://issues.apache.org/jira/browse/SPARK-29514
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Kent Yao
>Priority: Major
>
> |string_to_array}}(}}{{text}}{{, }}{{text}}{{ [,
> {{text}}])}}|{{text[]}}|splits string into array elements using supplied
> delimiter and optional null string|{{string_to_array('xx~^~yy~^~zz', '~^~',
> 'yy')}}|{{{xx,NULL,zz}}}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28669) System Information Functions
[
https://issues.apache.org/jira/browse/SPARK-28669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28669:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> System Information Functions
>
>
> Key: SPARK-28669
> URL: https://issues.apache.org/jira/browse/SPARK-28669
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Name||Return Type||Description||
> |{{current_catalog}}|{{name}}|name of current database (called “catalog” in
> the SQL standard)|
> |{{current_database()}}|{{name}}|name of current database|
> |{{current_query()}}|{{text}}|text of the currently executing query, as
> submitted by the client (might contain more than one statement)|
> |{{current_role}}|{{name}}|equivalent to {{current_user}}|
> |{{current_schema}}{{[()]}}|{{name}}|name of current schema|
> |{{current_schemas(}}{{boolean}}{{)}}|{{name[]}}|names of schemas in search
> path, optionally including implicit schemas|
> |{{current_user}}|{{name}}|user name of current execution context|
> |{{inet_client_addr()}}|{{inet}}|address of the remote connection|
> |{{inet_client_port()}}|{{int}}|port of the remote connection|
> |{{inet_server_addr()}}|{{inet}}|address of the local connection|
> |{{inet_server_port()}}|{{int}}|port of the local connection|
> |{{pg_backend_pid()}}|{{int}}|Process ID of the server process attached to
> the current session|
> |{{pg_blocking_pids(}}{{int}}{{)}}|{{int[]}}|Process ID(s) that are blocking
> specified server process ID from acquiring a lock|
> |{{pg_conf_load_time()}}|{{timestamp with time zone}}|configuration load time|
> |{{pg_current_logfile([{{text}}])}}|{{text}}|Primary log file name, or log in
> the requested format, currently in use by the logging collector|
> |{{pg_my_temp_schema()}}|{{oid}}|OID of session's temporary schema, or 0 if
> none|
> |{{pg_is_other_temp_schema(}}{{oid}}{{)}}|{{boolean}}|is schema another
> session's temporary schema?|
> |{{pg_listening_channels()}}|{{setof text}}|channel names that the session is
> currently listening on|
> |{{pg_notification_queue_usage()}}|{{double}}|fraction of the asynchronous
> notification queue currently occupied (0-1)|
> |{{pg_postmaster_start_time()}}|{{timestamp with time zone}}|server start
> time|
> |{{pg_safe_snapshot_blocking_pids(}}{{int}}{{)}}|{{int[]}}|Process ID(s) that
> are blocking specified server process ID from acquiring a safe snapshot|
> |{{pg_trigger_depth()}}|{{int}}|current nesting level of PostgreSQL triggers
> (0 if not called, directly or indirectly, from inside a trigger)|
> |{{session_user}}|{{name}}|session user name|
> |{{user}}|{{name}}|equivalent to {{current_user}}|
> Example:
> {code:sql}
> postgres=# SELECT pg_collation_for(description) FROM pg_description LIMIT 1;
> pg_collation_for
> --
> "default"
> (1 row)
> {code}
> https://www.postgresql.org/docs/10/functions-info.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28661) Hypothetical-Set Aggregate Functions
[
https://issues.apache.org/jira/browse/SPARK-28661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28661:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Hypothetical-Set Aggregate Functions
>
>
> Key: SPARK-28661
> URL: https://issues.apache.org/jira/browse/SPARK-28661
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Direct Argument Type(s)||Aggregated Argument Type(s)||Return
> Type||Partial Mode||Description||
> |{{rank(_args_}}) WITHIN GROUP (ORDER BY {{sorted_args}})|{{VARIADIC}}
> {{"any"}}|{{VARIADIC}} {{"any"}}|{{bigint}}|No|rank of the hypothetical row,
> with gaps for duplicate rows|
> |{{dense_rank(_args_}}) WITHIN GROUP (ORDER BY {{sorted_args}})|{{VARIADIC}}
> {{"any"}}|{{VARIADIC}} {{"any"}}|{{bigint}}|No|rank of the hypothetical row,
> without gaps|
> |{{percent_rank(_args_}}) WITHIN GROUP (ORDER BY
> {{sorted_args}})|{{VARIADIC}} {{"any"}}|{{VARIADIC}} {{"any"}}|{{double
> precision}}|No|relative rank of the hypothetical row, ranging from 0 to 1|
> |{{cume_dist(_args_}}) WITHIN GROUP (ORDER BY {{sorted_args}})|{{VARIADIC}}
> {{"any"}}|{{VARIADIC}} {{"any"}}|{{double precision}}|No|relative rank of the
> hypothetical row, ranging from 1/_{{N}}_ to 1|
> [https://www.postgresql.org/docs/11/functions-aggregate.html#FUNCTIONS-HYPOTHETICAL-TABLE]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-29892) Add built-in Array Functions: array_cat
[
https://issues.apache.org/jira/browse/SPARK-29892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-29892.
--
Resolution: Duplicate
> Add built-in Array Functions: array_cat
> ---
>
> Key: SPARK-29892
> URL: https://issues.apache.org/jira/browse/SPARK-29892
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_cat}}{{(}}{{anyarray}}{{,
> }}{{anyarray}}{{)}}|{{anyarray}}|concatenate two
> arrays|{{array_cat(ARRAY[1,2,3], ARRAY[4,5])}}|{\{ {1,2,3,4,5}}}|
> Other DBs:
> [https://phoenix.apache.org/language/functions.html#array_cat]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29892) Add built-in Array Functions: array_cat
[
https://issues.apache.org/jira/browse/SPARK-29892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29892:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_cat
> ---
>
> Key: SPARK-29892
> URL: https://issues.apache.org/jira/browse/SPARK-29892
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_cat}}{{(}}{{anyarray}}{{,
> }}{{anyarray}}{{)}}|{{anyarray}}|concatenate two
> arrays|{{array_cat(ARRAY[1,2,3], ARRAY[4,5])}}|{\{ {1,2,3,4,5}}}|
> Other DBs:
> [https://phoenix.apache.org/language/functions.html#array_cat]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28516) Data Type Formatting Functions: `to_char`
[
https://issues.apache.org/jira/browse/SPARK-28516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28516:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Data Type Formatting Functions: `to_char`
> -
>
> Key: SPARK-28516
> URL: https://issues.apache.org/jira/browse/SPARK-28516
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Dylan Guedes
>Priority: Major
>
> Currently, Spark does not have support for `to_char`. PgSQL, however,
> [does|[https://www.postgresql.org/docs/12/functions-formatting.html]]:
> Query example:
> {code:sql}
> SELECT to_char(SUM(n) OVER (ORDER BY i ROWS BETWEEN CURRENT ROW AND 1
> FOLLOWING),'9D9')
> {code}
> ||Function||Return Type||Description||Example||
> |{{to_char(}}{{timestamp}}{{, }}{{text}}{{)}}|{{text}}|convert time stamp to
> string|{{to_char(current_timestamp, 'HH12:MI:SS')}}|
> |{{to_char(}}{{interval}}{{, }}{{text}}{{)}}|{{text}}|convert interval to
> string|{{to_char(interval '15h 2m 12s', 'HH24:MI:SS')}}|
> |{{to_char(}}{{int}}{{, }}{{text}}{{)}}|{{text}}|convert integer to
> string|{{to_char(125, '999')}}|
> |{{to_char}}{{(}}{{double precision}}{{, }}{{text}}{{)}}|{{text}}|convert
> real/double precision to string|{{to_char(125.8::real, '999D9')}}|
> |{{to_char(}}{{numeric}}{{, }}{{text}}{{)}}|{{text}}|convert numeric to
> string|{{to_char(-125.8, '999D99S')}}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28459) Date/Time Functions: make_timestamp
[
https://issues.apache.org/jira/browse/SPARK-28459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28459:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Date/Time Functions: make_timestamp
> ---
>
> Key: SPARK-28459
> URL: https://issues.apache.org/jira/browse/SPARK-28459
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Maxim Gekk
>Priority: Major
> Fix For: 3.0.0
>
>
> ||Function||Return Type||Description||Example||Result||
> |{{make_timestamp(_year_ }}{{int}}{{, _month_ }}{{int}}{{, _day_ }}{{int}}{{,
> _hour_ }}{{int}}{{, _min_ }}{{int}}{{, _sec_}}{{double
> precision}}{{)}}|{{timestamp}}|Create timestamp from year, month, day, hour,
> minute and seconds fields|{{make_timestamp(2013, 7, 15, 8, 15,
> 23.5)}}|{{2013-07-15 08:15:23.5}}|
> https://www.postgresql.org/docs/11/functions-datetime.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28432) Add `make_date` function
[
https://issues.apache.org/jira/browse/SPARK-28432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28432:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add `make_date` function
>
>
> Key: SPARK-28432
> URL: https://issues.apache.org/jira/browse/SPARK-28432
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Maxim Gekk
>Priority: Major
> Fix For: 3.0.0
>
>
> ||Function||Return Type||Description||Example||Result||
> |{{make_date(_year_ }}{{int}}{{, _month_ }}{{int}}{{, _day_
> }}{{int}}{{)}}|{{date}}|Create date from year, month and day
> fields|{{make_date(2013, 7, 15)}}|{{2013-07-15}}|
> https://www.postgresql.org/docs/11/functions-datetime.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28427) Support more Postgres JSON functions
[
https://issues.apache.org/jira/browse/SPARK-28427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28427:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support more Postgres JSON functions
>
>
> Key: SPARK-28427
> URL: https://issues.apache.org/jira/browse/SPARK-28427
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Josh Rosen
>Priority: Major
>
> Postgres features a number of JSON functions that are missing in Spark:
> https://www.postgresql.org/docs/9.3/functions-json.html
> Redshift's JSON functions
> (https://docs.aws.amazon.com/redshift/latest/dg/json-functions.html) have
> partial overlap with the Postgres list.
> Some of these functions can be expressed in terms of compositions of existing
> Spark functions. For example, I think that {{json_array_length}} can be
> expressed with {{cardinality}} and {{from_json}}, but there's a caveat
> related to legacy Hive compatibility (see the demo notebook at
> https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/5796212617691211/45530874214710/4901752417050771/latest.html
> for more details).
> I'm filing this ticket so that we can triage the list of Postgres JSON
> features and decide which ones make sense to support in Spark. After we've
> done that, we can create individual tickets for specific functions and
> features.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28420) Date/Time Functions: date_part for intervals
[
https://issues.apache.org/jira/browse/SPARK-28420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28420:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Date/Time Functions: date_part for intervals
>
>
> Key: SPARK-28420
> URL: https://issues.apache.org/jira/browse/SPARK-28420
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Maxim Gekk
>Priority: Major
> Fix For: 3.0.0
>
>
> ||Function||Return Type||Description||Example||Result||
> |{{date_part(}}{{text}}{{, }}{{interval}}{{)}}|{{double precision}}|Get
> subfield (equivalent to {{extract}}); see [Section
> 9.9.1|https://www.postgresql.org/docs/11/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT]|{{date_part('month',
> interval '2 years 3 months')}}|{{3}}|
> We can replace it with {{extract(field from timestamp)}}.
> https://www.postgresql.org/docs/11/functions-datetime.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28429) SQL Datetime util function being casted to double instead of timestamp
[
https://issues.apache.org/jira/browse/SPARK-28429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28429:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> SQL Datetime util function being casted to double instead of timestamp
> --
>
> Key: SPARK-28429
> URL: https://issues.apache.org/jira/browse/SPARK-28429
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Dylan Guedes
>Priority: Major
>
> In the code below, 'now()+'100 days' are casted to double and then an error
> is thrown:
> {code:sql}
> CREATE TEMP VIEW v_window AS
> SELECT i, min(i) over (order by i range between '1 day' preceding and '10
> days' following) as min_i
> FROM range(now(), now()+'100 days', '1 hour') i;
> {code}
> Error:
> {code:sql}
> cannot resolve '(current_timestamp() + CAST('100 days' AS DOUBLE))' due to
> data type mismatch: differing types in '(current_timestamp() + CAST('100
> days' AS DOUBLE))' (timestamp and double).;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28401) Add built-in Array Functions: element to array concatenation
[
https://issues.apache.org/jira/browse/SPARK-28401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28401:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: element to array concatenation
>
>
> Key: SPARK-28401
> URL: https://issues.apache.org/jira/browse/SPARK-28401
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Major
>
> ||Function||Description||Example||Result||
> |{{\|\|}}|element-to-array concatenation|{{3 \|\|
> ARRAY[4,5,6]}}|{{{3,4,5,6}}}|
> |{{\|\|}}|array-to-element concatenation|{{ARRAY[4,5,6] \|\|
> 7}}|{{{4,5,6,7}}}|
> [https://www.postgresql.org/docs/current/functions-array.html]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28400) Add built-in Array Functions: array_upper
[
https://issues.apache.org/jira/browse/SPARK-28400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28400:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_upper
> -
>
> Key: SPARK-28400
> URL: https://issues.apache.org/jira/browse/SPARK-28400
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{array_upper}}{{(}}{{anyarray}}{{, }}{{int}}{{)}}|int|returns upper bound
> of the requested array dimension|array_upper(ARRAY[1,8,3,7], 1)|4|
> [https://www.postgresql.org/docs/current/functions-array.html]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28324) The LOG function using 10 as the base, but Spark using E
[
https://issues.apache.org/jira/browse/SPARK-28324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004653#comment-17004653
]
Takeshi Yamamuro commented on SPARK-28324:
--
I'll close this based on the discussion above. Thanks, all.
> The LOG function using 10 as the base, but Spark using E
>
>
> Key: SPARK-28324
> URL: https://issues.apache.org/jira/browse/SPARK-28324
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Spark SQL:
> {code:sql}
> spark-sql> select log(10);
> 2.302585092994046
> {code}
> PostgreSQL:
> {code:sql}
> postgres=# select log(10);
> log
> -
>1
> (1 row)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-28324) The LOG function using 10 as the base, but Spark using E
[
https://issues.apache.org/jira/browse/SPARK-28324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-28324.
--
Resolution: Won't Fix
> The LOG function using 10 as the base, but Spark using E
>
>
> Key: SPARK-28324
> URL: https://issues.apache.org/jira/browse/SPARK-28324
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Spark SQL:
> {code:sql}
> spark-sql> select log(10);
> 2.302585092994046
> {code}
> PostgreSQL:
> {code:sql}
> postgres=# select log(10);
> log
> -
>1
> (1 row)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28324) The LOG function using 10 as the base, but Spark using E
[
https://issues.apache.org/jira/browse/SPARK-28324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28324:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> The LOG function using 10 as the base, but Spark using E
>
>
> Key: SPARK-28324
> URL: https://issues.apache.org/jira/browse/SPARK-28324
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> Spark SQL:
> {code:sql}
> spark-sql> select log(10);
> 2.302585092994046
> {code}
> PostgreSQL:
> {code:sql}
> postgres=# select log(10);
> log
> -
>1
> (1 row)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28317) Built-in Mathematical Functions: SCALE
[
https://issues.apache.org/jira/browse/SPARK-28317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28317:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Built-in Mathematical Functions: SCALE
> --
>
> Key: SPARK-28317
> URL: https://issues.apache.org/jira/browse/SPARK-28317
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{scale(}}{{numeric}}{{)}}|{{integer}}|scale of the argument (the number of
> decimal digits in the fractional part)|{{scale(8.41)}}|{{2}}|
> https://www.postgresql.org/docs/11/functions-math.html#FUNCTIONS-MATH-FUNC-TABLE
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28123) String Functions: Add support btrim
[
https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28123:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: Add support btrim
> ---
>
> Key: SPARK-28123
> URL: https://issues.apache.org/jira/browse/SPARK-28123
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{btrim(_{{string}}_}}{{bytea}}{{,
> _{{bytes}}_}}{{bytea}}{{)}}|{{bytea}}|Remove the longest string containing
> only bytes appearing in _{{bytes}}_from the start and end of
> _{{string}}_|{{btrim('\000trim\001'::bytea, '\000\001'::bytea)}}|{{trim}}|
> More details: https://www.postgresql.org/docs/11/functions-binarystring.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28134) Trigonometric Functions
[
https://issues.apache.org/jira/browse/SPARK-28134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28134:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Trigonometric Functions
> ---
>
> Key: SPARK-28134
> URL: https://issues.apache.org/jira/browse/SPARK-28134
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function (radians)||Function (degrees)||Description||
> |{{acos(_x_}})|{{acosd(_x_}})|inverse cosine|
> |{{asin(_x_}})|{{asind(_x_}})|inverse sine|
> |{{atan(_x_}})|{{atand(_x_}})|inverse tangent|
> |{{atan2(_y_}}, _{{x}}_)|{{atan2d(_y_}}, _{{x}}_)|inverse tangent of
> {{_y_}}/_{{x}}_|
> |{{cos(_x_}})|{{cosd(_x_}})|cosine|
> |{{cot(_x_}})|{{cotd(_x_}})|cotangent|
> |{{sin(_x_}})|{{sind(_x_}})|sine|
> |{{tan(_x_}})|{{tand(_x_}})|tangent|
>
> [https://www.postgresql.org/docs/12/functions-math.html#FUNCTIONS-MATH-TRIG-TABLE]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28121) String Functions: decode/encode can not accept 'escape' and 'hex' as charset
[
https://issues.apache.org/jira/browse/SPARK-28121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28121:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: decode/encode can not accept 'escape' and 'hex' as charset
>
>
> Key: SPARK-28121
> URL: https://issues.apache.org/jira/browse/SPARK-28121
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> {noformat}
> postgres=# select decode('1234567890','escape');
> decode
>
> \x31323334353637383930
> (1 row)
> {noformat}
> {noformat}
> spark-sql> select decode('1234567890','escape');
> 19/06/20 01:57:33 ERROR SparkSQLDriver: Failed in [select
> decode('1234567890','escape')]
> java.io.UnsupportedEncodingException: escape
> at java.lang.StringCoding.decode(StringCoding.java:190)
> at java.lang.String.(String.java:426)
> at java.lang.String.(String.java:491)
> ...
> spark-sql> select decode('ff','hex');
> 19/08/16 21:44:55 ERROR SparkSQLDriver: Failed in [select decode('ff','hex')]
> java.io.UnsupportedEncodingException: hex
> at java.lang.StringCoding.decode(StringCoding.java:190)
> at java.lang.String.(String.java:426)
> at java.lang.String.(String.java:491)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28075) String Functions: Enhance TRIM function
[
https://issues.apache.org/jira/browse/SPARK-28075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28075:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: Enhance TRIM function
> ---
>
> Key: SPARK-28075
> URL: https://issues.apache.org/jira/browse/SPARK-28075
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Fix For: 3.0.0
>
>
> Add support {{TRIM(BOTH/LEADING/TRAILING FROM str)}} format.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27952) String Functions: regexp_replace is not compatible
[
https://issues.apache.org/jira/browse/SPARK-27952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27952:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: regexp_replace is not compatible
> --
>
> Key: SPARK-27952
> URL: https://issues.apache.org/jira/browse/SPARK-27952
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Zhu, Lipeng
>Priority: Major
>
> PostgreSQL's {{regexp_replace}} has the syntax: {{regexp_replace(source,
> pattern, replacement [, flags ])}}. This is not compatible with our syntax
> {{regexp_replace(str, regexp, rep)}}. The _*flags*_ parameter is an optional
> text string containing zero or more single-letter flags that change the
> function's behavior. Flag {{i}} specifies case-insensitive matching, while
> flag g specifies replacement of each matching substring rather than only the
> first one. Some examples:
> {noformat}
> regexp_replace('foobarbaz', 'b..', 'X')
>fooXbaz
> regexp_replace('foobarbaz', 'b..', 'X', 'g')
>fooXX
> regexp_replace('foobarbaz', 'b(..)', 'X\1Y', 'g')
>fooXarYXazY
> {noformat}
> More details:
> https://www.postgresql.org/docs/11/functions-matching.html#FUNCTIONS-POSIX-REGEXP
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27974) Add built-in Aggregate Function: array_agg
[
https://issues.apache.org/jira/browse/SPARK-27974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27974:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Aggregate Function: array_agg
> --
>
> Key: SPARK-27974
> URL: https://issues.apache.org/jira/browse/SPARK-27974
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Argument Type(s)||Return Type||Partial Mode||Description||
> |{{array_agg(_{{expression}}_)}}|any non-array type|array of the argument
> type|No|input values, including nulls, concatenated into an array|
> |{{array_agg(_{{expression}}_)}}|any array type|same as argument data
> type|No|input arrays concatenated into array of one higher dimension (inputs
> must all have same dimensionality, and cannot be empty or NULL)|
> https://www.postgresql.org/docs/11/functions-aggregate.html
> Other DBs:
> https://docs.teradata.com/reader/kmuOwjp1zEYg98JsB8fu_A/Xffi_w_BZeSoaXg6Fd76Qw
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28076) String Functions: SUBSTRING support regular expression
[
https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28076:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: SUBSTRING support regular expression
> --
>
> Key: SPARK-28076
> URL: https://issues.apache.org/jira/browse/SPARK-28076
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{substring(_string_}} from _{{pattern}}_)|{{text}}|Extract substring
> matching POSIX regular expression. See [Section
> 9.7|https://www.postgresql.org/docs/11/functions-matching.html] for more
> information on pattern matching.|{{substring('Thomas' from '...$')}}|{{mas}}|
> |{{substring(_string_}} from _{{pattern}}_ for _{{escape}}_)|{{text}}|Extract
> substring matching SQL regular expression. See [Section
> 9.7|https://www.postgresql.org/docs/11/functions-matching.html] for more
> information on pattern matching.|{{substring('Thomas' from '%#"o_a#"_' for
> '#')}}|{{oma}}|
> For example:
> {code:sql}
> -- T581 regular expression substring (with SQL's bizarre regexp syntax)
> SELECT SUBSTRING('abcdefg' FROM 'a#"(b_d)#"%' FOR '#') AS "bcd";
> {code}
> https://www.postgresql.org/docs/11/functions-string.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28402) Array indexing is 1-based
[
https://issues.apache.org/jira/browse/SPARK-28402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004651#comment-17004651
]
Takeshi Yamamuro commented on SPARK-28402:
--
I'll close for now because I think this is useless for Spark.
> Array indexing is 1-based
> -
>
> Key: SPARK-28402
> URL: https://issues.apache.org/jira/browse/SPARK-28402
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Major
>
> Array indexing is 1-based in PostgreSQL:
> [https://www.postgresql.org/docs/12/arrays.html]
>
> {quote}The array subscript numbers are written within square brackets. By
> default PostgreSQL uses a one-based numbering convention for arrays, that is,
> an array of _{{n}}_ elements starts with {{array[1]}} and ends with
> {{array[_{{n}}_]}}.{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27980) Ordered-Set Aggregate Functions
[
https://issues.apache.org/jira/browse/SPARK-27980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27980:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Ordered-Set Aggregate Functions
> ---
>
> Key: SPARK-27980
> URL: https://issues.apache.org/jira/browse/SPARK-27980
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Direct Argument Type(s)||Aggregated Argument Type(s)||Return
> Type||Partial Mode||Description||
> |{{mode() WITHIN GROUP (ORDER BY sort_expression)}}| |any sortable type|same
> as sort expression|No|returns the most frequent input value (arbitrarily
> choosing the first one if there are multiple equally-frequent results)|
> |{{percentile_cont(_fraction_}}) WITHIN GROUP (ORDER BY
> {{sort_expression}})|{{double precision}}|{{double precision}} or
> {{interval}}|same as sort expression|No|continuous percentile: returns a
> value corresponding to the specified fraction in the ordering, interpolating
> between adjacent input items if needed|
> |{{percentile_cont(_fractions_}}) WITHIN GROUP (ORDER BY
> {{sort_expression}})|{{double precision[]}}|{{double precision}} or
> {{interval}}|array of sort expression's type|No|multiple continuous
> percentile: returns an array of results matching the shape of the
> _{{fractions}}_ parameter, with each non-null element replaced by the value
> corresponding to that percentile|
> |{{percentile_disc(_fraction_}}) WITHIN GROUP (ORDER BY
> {{sort_expression}})|{{double precision}}|any sortable type|same as sort
> expression|No|discrete percentile: returns the first input value whose
> position in the ordering equals or exceeds the specified fraction|
> |{{percentile_disc(_fractions_}}) WITHIN GROUP (ORDER BY
> {{sort_expression}})|{{double precision[]}}|any sortable type|array of sort
> expression's type|No|multiple discrete percentile: returns an array of
> results matching the shape of the _{{fractions}}_ parameter, with each
> non-null element replaced by the input value corresponding to that percentile|
> [https://www.postgresql.org/docs/11/functions-aggregate.html#FUNCTIONS-ORDEREDSET-TABLE]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28402) Array indexing is 1-based
[
https://issues.apache.org/jira/browse/SPARK-28402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28402:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Array indexing is 1-based
> -
>
> Key: SPARK-28402
> URL: https://issues.apache.org/jira/browse/SPARK-28402
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Major
>
> Array indexing is 1-based in PostgreSQL:
> [https://www.postgresql.org/docs/12/arrays.html]
>
> {quote}The array subscript numbers are written within square brackets. By
> default PostgreSQL uses a one-based numbering convention for arrays, that is,
> an array of _{{n}}_ elements starts with {{array[1]}} and ends with
> {{array[_{{n}}_]}}.{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-28146) Support IS OF () predicate
[
https://issues.apache.org/jira/browse/SPARK-28146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-28146.
--
Resolution: Won't Fix
> Support IS OF () predicate
>
>
> Key: SPARK-28146
> URL: https://issues.apache.org/jira/browse/SPARK-28146
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Minor
>
> PostgreSQL supports IS OF () predicate, for example the following query
> is valid:
> {noformat}
> select 1 is of (int), true is of (bool)
> true true
> {noformat}
> I can't find PostgreSQL documentation about it, but here is how it works in
> Oracle:
>
> [https://docs.oracle.com/cd/B28359_01/server.111/b28286/conditions014.htm#SQLRF52157]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28146) Support IS OF () predicate
[
https://issues.apache.org/jira/browse/SPARK-28146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004650#comment-17004650
]
Takeshi Yamamuro commented on SPARK-28146:
--
I'll close for now because I think this is useless for Spark. Feel free to
reopen if you're interested in this.
> Support IS OF () predicate
>
>
> Key: SPARK-28146
> URL: https://issues.apache.org/jira/browse/SPARK-28146
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Minor
>
> PostgreSQL supports IS OF () predicate, for example the following query
> is valid:
> {noformat}
> select 1 is of (int), true is of (bool)
> true true
> {noformat}
> I can't find PostgreSQL documentation about it, but here is how it works in
> Oracle:
>
> [https://docs.oracle.com/cd/B28359_01/server.111/b28286/conditions014.htm#SQLRF52157]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28146) Support IS OF () predicate
[
https://issues.apache.org/jira/browse/SPARK-28146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28146:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support IS OF () predicate
>
>
> Key: SPARK-28146
> URL: https://issues.apache.org/jira/browse/SPARK-28146
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Minor
>
> PostgreSQL supports IS OF () predicate, for example the following query
> is valid:
> {noformat}
> select 1 is of (int), true is of (bool)
> true true
> {noformat}
> I can't find PostgreSQL documentation about it, but here is how it works in
> Oracle:
>
> [https://docs.oracle.com/cd/B28359_01/server.111/b28286/conditions014.htm#SQLRF52157]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28322) DIV support decimal type
[
https://issues.apache.org/jira/browse/SPARK-28322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28322:
-
Parent Issue: SPARK-30374 (was: SPARK-27764)
> DIV support decimal type
>
>
> Key: SPARK-28322
> URL: https://issues.apache.org/jira/browse/SPARK-28322
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Marco Gaido
>Priority: Major
> Fix For: 3.0.0
>
>
> Spark SQL:
> {code:sql}
> spark-sql> SELECT DIV(CAST(10 AS DECIMAL), CAST(3 AS DECIMAL));
> Error in query: cannot resolve '(CAST(10 AS DECIMAL(10,0)) div CAST(3 AS
> DECIMAL(10,0)))' due to data type mismatch: '(CAST(10 AS DECIMAL(10,0)) div
> CAST(3 AS DECIMAL(10,0)))' requires integral type, not decimal(10,0); line 1
> pos 7;
> 'Project [unresolvedalias((cast(10 as decimal(10,0)) div cast(3 as
> decimal(10,0))), None)]
> +- OneRowRelation
> {code}
> PostgreSQL:
> {code:sql}
> postgres=# SELECT DIV(CAST(10 AS DECIMAL), CAST(3 AS DECIMAL));
> div
> -
>3
> (1 row)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28078) String Functions: Add support other 4 REGEXP functions
[
https://issues.apache.org/jira/browse/SPARK-28078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28078:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: Add support other 4 REGEXP functions
> --
>
> Key: SPARK-28078
> URL: https://issues.apache.org/jira/browse/SPARK-28078
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> {{regexp_match}}, {{regexp_matches}}, {{regexp_split_to_array}} and
> {{regexp_split_to_table}}
> [https://www.postgresql.org/docs/11/functions-string.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-29715) Support SELECT statements in VALUES of INSERT INTO
[
https://issues.apache.org/jira/browse/SPARK-29715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-29715.
--
Resolution: Won't Fix
> Support SELECT statements in VALUES of INSERT INTO
> --
>
> Key: SPARK-29715
> URL: https://issues.apache.org/jira/browse/SPARK-29715
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Takeshi Yamamuro
>Priority: Major
>
> In PgSQL, we can use SELECT statements in VALUES of INSERT INTO;
> {code}
> postgres=# create table t (c0 int, c1 int);
> CREATE TABLE
> postgres=# insert into t values (3, (select 1));
> INSERT 0 1
> postgres=# select * from t;
> c0 | c1
> +
> 3 | 1
> (1 row)
> {code}
> {code}
> scala> sql("""create table t (c0 int, c1 int) using parquet""")
> scala> sql("""insert into t values (3, (select 1))""")
> org.apache.spark.sql.AnalysisException: unresolved operator 'Project
> [unresolvedalias(1, None)];;
> 'InsertIntoStatement 'UnresolvedRelation [t], false, false
> +- 'UnresolvedInlineTable [col1, col2], [List(3, scalar-subquery#0 [])]
> +- 'Project [unresolvedalias(1, None)]
> +- OneRowRelation
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:47)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis$(CheckAnalysis.scala:46)
> at
> org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:122)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$36(CheckAnalysis.scala:540)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$36$adapted(CheckAnalysis.scala:538)
> at
> org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:154)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29715) Support SELECT statements in VALUES of INSERT INTO
[
https://issues.apache.org/jira/browse/SPARK-29715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004649#comment-17004649
]
Takeshi Yamamuro commented on SPARK-29715:
--
I'll close this because this is pg-specific and useless for Spark.
> Support SELECT statements in VALUES of INSERT INTO
> --
>
> Key: SPARK-29715
> URL: https://issues.apache.org/jira/browse/SPARK-29715
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Takeshi Yamamuro
>Priority: Major
>
> In PgSQL, we can use SELECT statements in VALUES of INSERT INTO;
> {code}
> postgres=# create table t (c0 int, c1 int);
> CREATE TABLE
> postgres=# insert into t values (3, (select 1));
> INSERT 0 1
> postgres=# select * from t;
> c0 | c1
> +
> 3 | 1
> (1 row)
> {code}
> {code}
> scala> sql("""create table t (c0 int, c1 int) using parquet""")
> scala> sql("""insert into t values (3, (select 1))""")
> org.apache.spark.sql.AnalysisException: unresolved operator 'Project
> [unresolvedalias(1, None)];;
> 'InsertIntoStatement 'UnresolvedRelation [t], false, false
> +- 'UnresolvedInlineTable [col1, col2], [List(3, scalar-subquery#0 [])]
> +- 'Project [unresolvedalias(1, None)]
> +- OneRowRelation
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:47)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis$(CheckAnalysis.scala:46)
> at
> org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:122)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$36(CheckAnalysis.scala:540)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$36$adapted(CheckAnalysis.scala:538)
> at
> org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:154)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29715) Support SELECT statements in VALUES of INSERT INTO
[
https://issues.apache.org/jira/browse/SPARK-29715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29715:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support SELECT statements in VALUES of INSERT INTO
> --
>
> Key: SPARK-29715
> URL: https://issues.apache.org/jira/browse/SPARK-29715
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Takeshi Yamamuro
>Priority: Major
>
> In PgSQL, we can use SELECT statements in VALUES of INSERT INTO;
> {code}
> postgres=# create table t (c0 int, c1 int);
> CREATE TABLE
> postgres=# insert into t values (3, (select 1));
> INSERT 0 1
> postgres=# select * from t;
> c0 | c1
> +
> 3 | 1
> (1 row)
> {code}
> {code}
> scala> sql("""create table t (c0 int, c1 int) using parquet""")
> scala> sql("""insert into t values (3, (select 1))""")
> org.apache.spark.sql.AnalysisException: unresolved operator 'Project
> [unresolvedalias(1, None)];;
> 'InsertIntoStatement 'UnresolvedRelation [t], false, false
> +- 'UnresolvedInlineTable [col1, col2], [List(3, scalar-subquery#0 [])]
> +- 'Project [unresolvedalias(1, None)]
> +- OneRowRelation
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:47)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis$(CheckAnalysis.scala:46)
> at
> org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:122)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$36(CheckAnalysis.scala:540)
> at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$36$adapted(CheckAnalysis.scala:538)
> at
> org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:154)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-19799) Support WITH clause in subqueries
[ https://issues.apache.org/jira/browse/SPARK-19799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-19799: - Parent Issue: SPARK-30374 (was: SPARK-27764) > Support WITH clause in subqueries > - > > Key: SPARK-19799 > URL: https://issues.apache.org/jira/browse/SPARK-19799 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 2.2.0 >Reporter: Giambattista >Assignee: Peter Toth >Priority: Major > Fix For: 3.0.0 > > > Because of Spark-17590 it should be relatively easy to support WITH clause in > subqueries besides nested CTE definitions. > Here an example of a query that does not run on spark: > create table test (seqno int, k string, v int) using parquet; > insert into TABLE test values (1,'a', 99),(2, 'b', 88),(3, 'a', 77),(4, 'b', > 66),(5, 'c', 55),(6, 'a', 44),(7, 'b', 33); > SELECT percentile(b, 0.5) FROM (WITH mavg AS (SELECT k, AVG(v) OVER > (PARTITION BY k ORDER BY seqno ROWS BETWEEN 3 PRECEDING AND CURRENT ROW) as b > FROM test ORDER BY seqno) SELECT k, MAX(b) as b FROM mavg GROUP BY k); -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-28500) adds support for `filter` clause
[
https://issues.apache.org/jira/browse/SPARK-28500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-28500.
--
Resolution: Duplicate
> adds support for `filter` clause
>
>
> Key: SPARK-28500
> URL: https://issues.apache.org/jira/browse/SPARK-28500
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Dylan Guedes
>Priority: Major
>
> Definition: "The {{filter}} clause extends aggregate functions ({{sum}},
> {{avg}}, {{count}}, …) by an additional {{where}} clause. The result of the
> aggregate is built from only the rows that satisfy the additional {{where}}
> clause too." [source|[https://modern-sql.com/feature/filter]]
> Also, PgSQL currently support `filter` while Spark doesn't.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28500) adds support for `filter` clause
[
https://issues.apache.org/jira/browse/SPARK-28500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28500:
-
Parent Issue: SPARK-30374 (was: SPARK-27764)
> adds support for `filter` clause
>
>
> Key: SPARK-28500
> URL: https://issues.apache.org/jira/browse/SPARK-28500
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Dylan Guedes
>Priority: Major
>
> Definition: "The {{filter}} clause extends aggregate functions ({{sum}},
> {{avg}}, {{count}}, …) by an additional {{where}} clause. The result of the
> aggregate is built from only the rows that satisfy the additional {{where}}
> clause too." [source|[https://modern-sql.com/feature/filter]]
> Also, PgSQL currently support `filter` while Spark doesn't.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner
[
https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27768:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Infinity, -Infinity, NaN should be recognized in a case insensitive manner
> --
>
> Key: SPARK-27768
> URL: https://issues.apache.org/jira/browse/SPARK-27768
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Xiao Li
>Assignee: Dilip Biswal
>Priority: Major
> Fix For: 3.0.0
>
>
> When the inputs contain the constant 'infinity', Spark SQL does not generate
> the expected results.
> {code:java}
> SELECT avg(CAST(x AS DOUBLE)), var_pop(CAST(x AS DOUBLE))
> FROM (VALUES ('1'), (CAST('infinity' AS DOUBLE))) v(x);
> SELECT avg(CAST(x AS DOUBLE)), var_pop(CAST(x AS DOUBLE))
> FROM (VALUES ('infinity'), ('1')) v(x);
> SELECT avg(CAST(x AS DOUBLE)), var_pop(CAST(x AS DOUBLE))
> FROM (VALUES ('infinity'), ('infinity')) v(x);
> SELECT avg(CAST(x AS DOUBLE)), var_pop(CAST(x AS DOUBLE))
> FROM (VALUES ('-infinity'), ('infinity')) v(x);{code}
> The root cause: Spark SQL does not recognize the special constants in a case
> insensitive way. In PostgreSQL, they are recognized in a case insensitive
> way.
> Link: https://www.postgresql.org/docs/9.3/datatype-numeric.html
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28088) String Functions: Enhance LPAD/RPAD function
[
https://issues.apache.org/jira/browse/SPARK-28088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28088:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: Enhance LPAD/RPAD function
>
>
> Key: SPARK-28088
> URL: https://issues.apache.org/jira/browse/SPARK-28088
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Fix For: 3.0.0
>
>
> Enhance LPAD/RPAD function to make {{pad}} parameter optional.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28296) Improved VALUES support
[
https://issues.apache.org/jira/browse/SPARK-28296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28296:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Improved VALUES support
> ---
>
> Key: SPARK-28296
> URL: https://issues.apache.org/jira/browse/SPARK-28296
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Peter Toth
>Priority: Major
>
> These are valid queries in PostgreSQL, but they don't work in Spark SQL:
> {noformat}
> values ((select 1));
> values ((select c from test1));
> select (values(c)) from test10;
> with cte(foo) as ( values(42) ) values((select foo from cte));
> {noformat}
> where test1 and test10:
> {noformat}
> CREATE TABLE test1 (c INTEGER);
> INSERT INTO test1 VALUES(1);
> CREATE TABLE test10 (c INTEGER);
> INSERT INTO test10 SELECT generate_sequence(1, 10);
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-29716) Support User-defined Types
[ https://issues.apache.org/jira/browse/SPARK-29716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29716. -- Resolution: Won't Fix > Support User-defined Types > -- > > Key: SPARK-29716 > URL: https://issues.apache.org/jira/browse/SPARK-29716 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > [https://www.postgresql.org/docs/9.5/xtypes.html] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29716) Support User-defined Types
[ https://issues.apache.org/jira/browse/SPARK-29716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29716: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Support User-defined Types > -- > > Key: SPARK-29716 > URL: https://issues.apache.org/jira/browse/SPARK-29716 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > [https://www.postgresql.org/docs/9.5/xtypes.html] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29775) Support truncate multiple tables
[ https://issues.apache.org/jira/browse/SPARK-29775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29775: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Support truncate multiple tables > > > Key: SPARK-29775 > URL: https://issues.apache.org/jira/browse/SPARK-29775 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 2.4.4 >Reporter: jobit mathew >Priority: Minor > > Spark sql Support truncate single table like > TRUNCATE table t1; > But postgresql support truncating multiple tables like > TRUNCATE bigtable, fattable; > So spark also can support truncating multiple tables > [https://www.postgresql.org/docs/12/sql-truncate.html] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28017) Enhance DATE_TRUNC
[ https://issues.apache.org/jira/browse/SPARK-28017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28017: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Enhance DATE_TRUNC > -- > > Key: SPARK-28017 > URL: https://issues.apache.org/jira/browse/SPARK-28017 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Yuming Wang >Assignee: Maxim Gekk >Priority: Major > Fix For: 3.0.0 > > > For DATE_TRUNC, we need support: microseconds, milliseconds, decade, century, > millennium. > https://www.postgresql.org/docs/11/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29632) Support ALTER TABLE [relname] SET SCHEMA [dbname]
[
https://issues.apache.org/jira/browse/SPARK-29632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29632:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support ALTER TABLE [relname] SET SCHEMA [dbname]
> -
>
> Key: SPARK-29632
> URL: https://issues.apache.org/jira/browse/SPARK-29632
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Takeshi Yamamuro
>Priority: Major
>
> {code}
> CREATE SCHEMA temp_view_test;
> CREATE TABLE tx1 (x1 int, x2 int, x3 string) using parquet;
> ALTER TABLE tx1 SET SCHEMA temp_view_test;
> {code}
> {code}
> ALTER TABLE [ IF EXISTS ] name
> SET SCHEMA new_schema
> {code}
> https://www.postgresql.org/docs/current/sql-altertable.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28333) NULLS FIRST for DESC and NULLS LAST for ASC
[
https://issues.apache.org/jira/browse/SPARK-28333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28333:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> NULLS FIRST for DESC and NULLS LAST for ASC
> ---
>
> Key: SPARK-28333
> URL: https://issues.apache.org/jira/browse/SPARK-28333
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:sql}
> spark-sql> create or replace temporary view t1 as select * from (values(1),
> (2), (null), (3), (null)) as v (val);
> spark-sql> select * from t1 order by val asc;
> NULL
> NULL
> 1
> 2
> 3
> spark-sql> select * from t1 order by val desc;
> 3
> 2
> 1
> NULL
> NULL
> {code}
> {code:sql}
> postgres=# create or replace temporary view t1 as select * from (values(1),
> (2), (null), (3), (null)) as v (val);
> CREATE VIEW
> postgres=# select * from t1 order by val asc;
> val
> -
>1
>2
>3
> (5 rows)
> postgres=# select * from t1 order by val desc;
> val
> -
>3
>2
>1
> (5 rows)
> {code}
> https://www.postgresql.org/docs/11/queries-order.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29393) Add make_interval() function
[
https://issues.apache.org/jira/browse/SPARK-29393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29393:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add make_interval() function
>
>
> Key: SPARK-29393
> URL: https://issues.apache.org/jira/browse/SPARK-29393
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Maxim Gekk
>Assignee: Maxim Gekk
>Priority: Major
> Fix For: 3.0.0
>
>
> PostgreSQL allows to make an interval by using the make_interval() function:
> |{{make_interval(_{{years}}_ }}{{int}}{{ DEFAULT 0, _{{months}}_ }}{{int}}{{
> DEFAULT 0, _{{weeks}}_ }}{{int}}{{ DEFAULT 0, _{{days}}_ }}{{int}}{{ DEFAULT
> 0, _{{hours}}_ }}{{int}}{{ DEFAULT 0, _{{mins}}_ }}{{int}}{{ DEFAULT 0,
> _{{secs}}_ }}{{double precision}}{{ DEFAULT 0.0)}}|{{interval}}|Create
> interval from years, months, weeks, days, hours, minutes and seconds
> fields|{{make_interval(days => 10)}}|{{10 days}}|
> See https://www.postgresql.org/docs/12/functions-datetime.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28329) SELECT INTO syntax
[
https://issues.apache.org/jira/browse/SPARK-28329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28329:
-
Parent Issue: SPARK-30374 (was: SPARK-27764)
> SELECT INTO syntax
> --
>
> Key: SPARK-28329
> URL: https://issues.apache.org/jira/browse/SPARK-28329
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> h2. Synopsis
> {noformat}
> [ WITH [ RECURSIVE ] with_query [, ...] ]
> SELECT [ ALL | DISTINCT [ ON ( expression [, ...] ) ] ]
> * | expression [ [ AS ] output_name ] [, ...]
> INTO [ TEMPORARY | TEMP | UNLOGGED ] [ TABLE ] new_table
> [ FROM from_item [, ...] ]
> [ WHERE condition ]
> [ GROUP BY expression [, ...] ]
> [ HAVING condition [, ...] ]
> [ WINDOW window_name AS ( window_definition ) [, ...] ]
> [ { UNION | INTERSECT | EXCEPT } [ ALL | DISTINCT ] select ]
> [ ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST |
> LAST } ] [, ...] ]
> [ LIMIT { count | ALL } ]
> [ OFFSET start [ ROW | ROWS ] ]
> [ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLY ]
> [ FOR { UPDATE | SHARE } [ OF table_name [, ...] ] [ NOWAIT ] [...] ]
> {noformat}
> h2. Description
> {{SELECT INTO}} creates a new table and fills it with data computed by a
> query. The data is not returned to the client, as it is with a normal
> {{SELECT}}. The new table's columns have the names and data types associated
> with the output columns of the {{SELECT}}.
>
> {{CREATE TABLE AS}} offers a superset of the functionality offered by
> {{SELECT INTO}}.
> [https://www.postgresql.org/docs/11/sql-selectinto.html]
> [https://www.postgresql.org/docs/11/sql-createtableas.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28382) Array Functions: unnest
[
https://issues.apache.org/jira/browse/SPARK-28382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28382:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Array Functions: unnest
> ---
>
> Key: SPARK-28382
> URL: https://issues.apache.org/jira/browse/SPARK-28382
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{unnest}}({{anyarray}})|set of anyelement|expand an array to a set of
> rows|unnest(ARRAY[1,2])|1
> 2
> (2 rows)|
>
> https://www.postgresql.org/docs/11/functions-array.html
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28122) Binary String Functions: SHA functions
[
https://issues.apache.org/jira/browse/SPARK-28122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28122:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Binary String Functions: SHA functions
> ---
>
> Key: SPARK-28122
> URL: https://issues.apache.org/jira/browse/SPARK-28122
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{sha224(}}{{bytea}}{{)}}|{{bytea}}|SHA-224
> hash|{{sha224('abc')}}|{{\x23097d223405d8228642a477bda255b32aadbce4bda0b3f7e36c9da7}}|
> |{{sha256(}}{{bytea}}{{)}}|{{bytea}}|SHA-256
> hash|{{sha256('abc')}}|{{\xba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad}}|
> |{{sha384(}}{{bytea}}{{)}}|{{bytea}}|SHA-384
> hash|{{sha384('abc')}}|{{\xcb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7}}|
> |{{sha512(}}{{bytea}}{{)}}|{{bytea}}|SHA-512
> hash|{{sha512('abc')}}|{{\xddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a964b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f}}|
> More details: https://www.postgresql.org/docs/11/functions-binarystring.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-27978) Add built-in Aggregate Functions: string_agg
[
https://issues.apache.org/jira/browse/SPARK-27978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-27978:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Aggregate Functions: string_agg
>
>
> Key: SPARK-27978
> URL: https://issues.apache.org/jira/browse/SPARK-27978
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Argument Type(s)||Return Type||Partial Mode||Description||
> |string_agg(_{{expression}}_,_{{delimiter}}_)|({{text}}, {{text}}) or
> ({{bytea}}, {{bytea}})|same as argument types|No|input values concatenated
> into a string, separated by delimiter|
> https://www.postgresql.org/docs/current/functions-aggregate.html
> We can workaround it by concat_ws(_{{delimiter}}_,
> collect_list(_{{expression}}_)) currently.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28087) String Functions: Add support split_part
[
https://issues.apache.org/jira/browse/SPARK-28087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28087:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> String Functions: Add support split_part
>
>
> Key: SPARK-28087
> URL: https://issues.apache.org/jira/browse/SPARK-28087
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{split_part(_string_ }}{{text}}{{,_delimiter_ }}{{text}}{{, _field_
> }}{{int}}{{)}}|{{text}}|Split _string_ on _delimiter_ and return the given
> field (counting from one)|split_part('abc\~@\~def\~@\~ghi', '\~@\~',
> 2)|{{def}}|
> [https://www.postgresql.org/docs/11/functions-string.html]
> [http://prestodb.github.io/docs/current/functions/string.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29941) Support dec as shorthand for decimal
[
https://issues.apache.org/jira/browse/SPARK-29941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29941:
-
Parent Issue: SPARK-30374 (was: SPARK-27764)
> Support dec as shorthand for decimal
>
>
> Key: SPARK-29941
> URL: https://issues.apache.org/jira/browse/SPARK-29941
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Kent Yao
>Assignee: Kent Yao
>Priority: Major
> Fix For: 3.0.0
>
>
> {code:java}
> CHAR is equivalent to CHARACTER. DEC is equivalent to DECIMAL. INT is
> equivalent to INTEGER. VARCHAR is equivalent to CHARACTER VARYING. ...
> {code}
> According to SQL standard, we decide to add dec for decimal as short hand.
> also checked with Postgres,
> ```sql
> postgres=# select dec '1.0';
> numeric
> -
> 1.0
> (1 row)
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29926) interval `1. second` should be invalid as PostgreSQL
[
https://issues.apache.org/jira/browse/SPARK-29926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29926:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> interval `1. second` should be invalid as PostgreSQL
>
>
> Key: SPARK-29926
> URL: https://issues.apache.org/jira/browse/SPARK-29926
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Kent Yao
>Assignee: Kent Yao
>Priority: Minor
> Fix For: 3.0.0
>
>
> Spark
> {code:sql}
> -- !query 134
> select interval '1. second'
> -- !query 134 schema
> struct<1 seconds:interval>
> -- !query 134 output
> 1 seconds
> -- !query 135
> select cast('1. second' as interval)
> -- !query 135 schema
> struct
> -- !query 135 output
> 1 seconds
> {code}
> PostgreSQL
> {code:sql}
> ostgres=# select interval '1. seconds';
> ERROR: invalid input syntax for type interval: "1. seconds"
> LINE 1: select interval '1. seconds';
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30147) Trim the string when cast string type to booleans
[
https://issues.apache.org/jira/browse/SPARK-30147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-30147:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Trim the string when cast string type to booleans
> -
>
> Key: SPARK-30147
> URL: https://issues.apache.org/jira/browse/SPARK-30147
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Kent Yao
>Assignee: Kent Yao
>Priority: Major
> Fix For: 3.0.0
>
>
> Now, we trim the string when casting string value to those `canCast` types
> values, e.g. int, double, decimal, interval, date, timestamps, except for
> boolean.
> This behavior makes type cast and coercion inconsistency in Spark.
> Not fit ANSI SQL standard either.
> {code:java}
> If TD is boolean, then
> Case:
> a) If SD is character string, then SV is replaced by
> TRIM ( BOTH ' ' FROM VE )
> Case:
> i) If the rules for literal in Subclause 5.3, “literal”, can be applied
> to SV to determine a valid
> value of the data type TD, then let TV be that value.
>ii) Otherwise, an exception condition is raised: data exception — invalid
> character value for cast.
> b) If SD is boolean, then TV is SV
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29717) Support [CREATE|DROP] RULE - define a new plan rewrite rule
[ https://issues.apache.org/jira/browse/SPARK-29717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004643#comment-17004643 ] Takeshi Yamamuro commented on SPARK-29717: -- I'll close this because I think this feature is pg-specific and useless for Spark. > Support [CREATE|DROP] RULE - define a new plan rewrite rule > --- > > Key: SPARK-29717 > URL: https://issues.apache.org/jira/browse/SPARK-29717 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > https://www.postgresql.org/docs/current/sql-createrule.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-29717) Support [CREATE|DROP] RULE - define a new plan rewrite rule
[ https://issues.apache.org/jira/browse/SPARK-29717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29717. -- Resolution: Won't Fix > Support [CREATE|DROP] RULE - define a new plan rewrite rule > --- > > Key: SPARK-29717 > URL: https://issues.apache.org/jira/browse/SPARK-29717 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > https://www.postgresql.org/docs/current/sql-createrule.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29717) Support [CREATE|DROP] RULE - define a new plan rewrite rule
[ https://issues.apache.org/jira/browse/SPARK-29717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29717: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Support [CREATE|DROP] RULE - define a new plan rewrite rule > --- > > Key: SPARK-29717 > URL: https://issues.apache.org/jira/browse/SPARK-29717 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > https://www.postgresql.org/docs/current/sql-createrule.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30182) Support nested aggregates
[
https://issues.apache.org/jira/browse/SPARK-30182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-30182:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support nested aggregates
> -
>
> Key: SPARK-30182
> URL: https://issues.apache.org/jira/browse/SPARK-30182
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> Spark SQL cannot supports a SQL with nested aggregate as below:
> {code:java}
> SELECT sum(salary), row_number() OVER (ORDER BY depname), sum(
> sum(salary) FILTER (WHERE enroll_date > '2007-01-01')
> ) FILTER (WHERE depname <> 'sales') OVER (ORDER BY depname DESC) AS
> "filtered_sum",
> depname
> FROM empsalary GROUP BY depname;{code}
> And Spark will throw exception as follows:
> {code:java}
> org.apache.spark.sql.AnalysisException
> It is not allowed to use an aggregate function in the argument of another
> aggregate function. Please use the inner aggregate function in a
> sub-query.{code}
> But PostgreSQL supports this syntax.
> {code:java}
> SELECT sum(salary), row_number() OVER (ORDER BY depname), sum(
> sum(salary) FILTER (WHERE enroll_date > '2007-01-01')
> ) FILTER (WHERE depname <> 'sales') OVER (ORDER BY depname DESC) AS
> "filtered_sum",
> depname
> FROM empsalary GROUP BY depname;
> sum | row_number | filtered_sum | depname
> ---++--+---
> 25100 | 1 | 22600 | develop
> 7400 | 2 | 3500 | personnel
> 14600 | 3 | | sales
> (3 rows){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29718) Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE
[ https://issues.apache.org/jira/browse/SPARK-29718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29718: - Parent Issue: SPARK-30375 (was: SPARK-27764) > Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE > --- > > Key: SPARK-29718 > URL: https://issues.apache.org/jira/browse/SPARK-29718 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > 5.10. Table Partitioning: > https://www.postgresql.org/docs/current/ddl-partitioning.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29982) Add built-in Array Functions: array_append
[
https://issues.apache.org/jira/browse/SPARK-29982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29982:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_append
> --
>
> Key: SPARK-29982
> URL: https://issues.apache.org/jira/browse/SPARK-29982
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_append}}{{(}}{{anyarray}}{{,
> }}{{anyelement}}{{)}}|{{anyarray}}|append an element to the end of an
> array|{{array_append(ARRAY[1,2], 3)}}|{\{ {1,2,3}}}|
> Other DBs:
> [https://www.postgresql.org/docs/11/functions-array.html]
> [https://phoenix.apache.org/language/functions.html#array_append]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29984) Add built-in Array Functions: array_ndims
[
https://issues.apache.org/jira/browse/SPARK-29984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29984:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_ndims
> -
>
> Key: SPARK-29984
> URL: https://issues.apache.org/jira/browse/SPARK-29984
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_ndims}}{{(}}{{anyarray}}{{)}}|{{int}}|returns the number of
> dimensions of the array|{{array_ndims(ARRAY[[1,2,3], [4,5,6]])}}|{{2}}|
> [https://www.postgresql.org/docs/11/functions-array.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28037) Add built-in String Functions: quote_literal
[
https://issues.apache.org/jira/browse/SPARK-28037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28037:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in String Functions: quote_literal
>
>
> Key: SPARK-28037
> URL: https://issues.apache.org/jira/browse/SPARK-28037
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||Result||
> |{{quote_literal(_{{string}}_ }}{{text}}{{)}}|{{text}}|Return the given
> string suitably quoted to be used as a string literal in an SQL statement
> string. Embedded single-quotes and backslashes are properly doubled. Note
> that {{quote_literal}} returns null on null input; if the argument might be
> null, {{quote_nullable}} is often more suitable. See also [Example
> 43.1|https://www.postgresql.org/docs/11/plpgsql-statements.html#PLPGSQL-QUOTE-LITERAL-EXAMPLE].|{{quote_literal(E'O\'Reilly')}}|{{'O''Reilly'}}|
> |{{quote_literal(_{{value}}_ }}{{anyelement}}{{)}}|{{text}}|Coerce the given
> value to text and then quote it as a literal. Embedded single-quotes and
> backslashes are properly doubled.|{{quote_literal(42.5)}}|{{'42.5'}}|
> https://www.postgresql.org/docs/11/functions-string.html
> https://docs.aws.amazon.com/redshift/latest/dg/r_QUOTE_LITERAL.html
> https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/String/QUOTE_LITERAL.htm?tocpath=SQL%20Reference%20Manual%7CSQL%20Functions%7CString%20Functions%7C_38
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28086) Adds `random()` sql function
[
https://issues.apache.org/jira/browse/SPARK-28086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28086:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Adds `random()` sql function
>
>
> Key: SPARK-28086
> URL: https://issues.apache.org/jira/browse/SPARK-28086
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Dylan Guedes
>Assignee: Dongjoon Hyun
>Priority: Major
> Fix For: 3.0.0
>
>
> Currently, Spark does not have a `random()` function. Postgres, however, does.
> For instance, this one is not valid:
> {code:sql}
> SELECT rank() OVER (ORDER BY rank() OVER (ORDER BY random()))
> {code}
> Because of the `random()` call. On the other hand, [Postgres has
> it.|https://www.postgresql.org/docs/8.2/functions-math.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-29891) Add built-in Array Functions: array_length
[
https://issues.apache.org/jira/browse/SPARK-29891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-29891:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_length
> --
>
> Key: SPARK-29891
> URL: https://issues.apache.org/jira/browse/SPARK-29891
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_length}}{{(}}{{anyarray}}{{, }}{{int}}{{)}}|{{int}}|returns the
> length of the requested array dimension|{{array_length(array[1,2,3],
> 1)}}|{{3}}|
> | | | | | |
> Other DBs:
> [https://phoenix.apache.org/language/functions.html#array_length]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28137) Data Type Formatting Functions: `to_number`
[
https://issues.apache.org/jira/browse/SPARK-28137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28137:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Data Type Formatting Functions: `to_number`
> ---
>
> Key: SPARK-28137
> URL: https://issues.apache.org/jira/browse/SPARK-28137
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> ||Function||Return Type||Description||Example||
> |{{to_number(}}{{text}}{{, }}{{text}}{{)}}|{{numeric}}|convert string to
> numeric|{{to_number('12,454.8-', '99G999D9S')}}|
> https://www.postgresql.org/docs/12/functions-formatting.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30375) Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours)
[ https://issues.apache.org/jira/browse/SPARK-30375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-30375: - Description: Among the feature parity between Spark/PostgreSQL (SPARK-27764), this umbrella holds issues for the behaviours that some parts of DBMS-like systems (e.g., PostgreSQL, Oracle, MySQL, ...) follow. (was: Among the feature parity between Spark/PostgreSQL (SPARK-27764), this umbrella holds issues for the features that some parts of DBMS-like systems (e.g., PostgreSQL, Oracle, MySQL, ...) support.) > Feature Parity between PostgreSQL and Spark (implementation-dependent > behaviours) > - > > Key: SPARK-30375 > URL: https://issues.apache.org/jira/browse/SPARK-30375 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > Among the feature parity between Spark/PostgreSQL (SPARK-27764), this > umbrella holds issues for the behaviours that some parts of DBMS-like systems > (e.g., PostgreSQL, Oracle, MySQL, ...) follow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30375) Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours)
[ https://issues.apache.org/jira/browse/SPARK-30375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-30375: - Summary: Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours) (was: Feature Parity between PostgreSQL and Spark (implementation-dependent features)) > Feature Parity between PostgreSQL and Spark (implementation-dependent > behaviours) > - > > Key: SPARK-30375 > URL: https://issues.apache.org/jira/browse/SPARK-30375 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.0.0 >Reporter: Takeshi Yamamuro >Priority: Major > > Among the feature parity between Spark/PostgreSQL (SPARK-27764), this > umbrella holds issues for the features that some parts of DBMS-like systems > (e.g., PostgreSQL, Oracle, MySQL, ...) support. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-28036) Support negative length at LEFT/RIGHT SQL functions
[
https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-28036:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Support negative length at LEFT/RIGHT SQL functions
> ---
>
> Key: SPARK-28036
> URL: https://issues.apache.org/jira/browse/SPARK-28036
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> PostgreSQL:
> {code:sql}
> postgres=# select left('ahoj', -2), right('ahoj', -2);
> left | right
> --+---
> ah | oj
> (1 row)
> {code}
> Spark SQL:
> {code:sql}
> spark-sql> select left('ahoj', -2), right('ahoj', -2);
> spark-sql>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30042) Add built-in Array Functions: array_dims
[
https://issues.apache.org/jira/browse/SPARK-30042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-30042:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_dims
>
>
> Key: SPARK-30042
> URL: https://issues.apache.org/jira/browse/SPARK-30042
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_dims}}{{(}}{{anyarray}}{{)}}|{{text}}|returns a text representation
> of array's dimensions|{{array_dims(ARRAY[[1,2,3], [4,5,6]])}}|{{[1:2][1:3]}}|
> [https://www.postgresql.org/docs/11/functions-array.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30043) Add built-in Array Functions: array_fill
[
https://issues.apache.org/jira/browse/SPARK-30043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-30043:
-
Parent Issue: SPARK-30375 (was: SPARK-27764)
> Add built-in Array Functions: array_fill
>
>
> Key: SPARK-30043
> URL: https://issues.apache.org/jira/browse/SPARK-30043
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: jiaan.geng
>Priority: Major
>
> |{{array_fill}}{{(}}{{anyelement}}{{, }}{{int[]}}{{ [,
> {{int[]}}])}}|{{anyarray}}|returns an array initialized with supplied value
> and dimensions, optionally with lower bounds other than 1|{{array_fill(7,
> ARRAY[3], ARRAY[2])}}|{{[2:4]=\{7,7,7}}}|
> [https://www.postgresql.org/docs/11/functions-array.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
