[jira] [Commented] (SPARK-21674) Support double/single-quoted strings for alias names in SQL
[
https://issues.apache.org/jira/browse/SPARK-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134681#comment-16134681
]
Chin Han Yu commented on SPARK-21674:
-
Found that back quote is working fine for alias names.
Do we also need to support single/double quotes?
{code:java}
scala> spark.sql("""SELECT 1 as `NAME` from people""").show()
++
|NAME|
++
| 1|
| 1|
| 1|
++
{code}
> Support double/single-quoted strings for alias names in SQL
> ---
>
> Key: SPARK-21674
> URL: https://issues.apache.org/jira/browse/SPARK-21674
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.2.0
>Reporter: Takeshi Yamamuro
>Priority: Trivial
> Labels: starter
>
> In the master, users can't use single/double-quoted strings for alias names
> in SQL;
> {code}
> scala> sql("""SELECT 1 AS "NAME" """)
> org.apache.spark.sql.catalyst.parser.ParseException:
> extraneous input '"NAME"' expecting {, ',', 'FROM', 'WHERE', 'GROUP',
> 'ORDER', 'HAVING', 'LIMIT', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS',
> 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE'}(line 1, pos 12)
> == SQL ==
> SELECT 1 AS "NAME"
> ^^^
> at
> org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:217)
> at
> org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:114)
> ...
> {code}
> PostgreSQL and MySQL support this;
> {code}
> ~:$psql -d postgres
> psql (9.5beta1)
> Type "help" for help.
> postgres=# select 1 AS "NAME";
> NAME
> --
> 1
> (1 row)
> mysql> select 1 AS "NAME";
> +--+
> | NAME |
> +--+
> |1 |
> +--+
> 1 row in set (0.00 sec)
> {code}
> Hive does not;
> {code}
> hive> select 1 AS "NAME";
> NoViableAltException(352@[87:7: ( ( ( KW_AS )? identifier ) | ( KW_AS LPAREN
> identifier ( COMMA identifier )* RPAREN ) )?])
> at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
> at org.antlr.runtime.DFA.predict(DFA.java:116)
> {code}
> Trivial though, it might some help for PostgreSQL/MySQL users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21674) Support double/single-quoted strings for alias names in SQL
[
https://issues.apache.org/jira/browse/SPARK-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134669#comment-16134669
]
Chin Han Yu edited comment on SPARK-21674 at 8/21/17 3:37 AM:
--
Hi, I would like to work on this ticket.
Since it's common to use double quotes or single quotes to escape reserved word
as a column name or alias name to PostgreSQL/MySQL users, this improvement may
be helpful.
was (Author: byakuinss):
Hi, I would like to work on this ticket.
Since it's common to use double quotes or single quotes to escape reserved word
as a column name to PostgreSQL/MySQL users, this improvement may be helpful.
> Support double/single-quoted strings for alias names in SQL
> ---
>
> Key: SPARK-21674
> URL: https://issues.apache.org/jira/browse/SPARK-21674
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.2.0
>Reporter: Takeshi Yamamuro
>Priority: Trivial
> Labels: starter
>
> In the master, users can't use single/double-quoted strings for alias names
> in SQL;
> {code}
> scala> sql("""SELECT 1 AS "NAME" """)
> org.apache.spark.sql.catalyst.parser.ParseException:
> extraneous input '"NAME"' expecting {, ',', 'FROM', 'WHERE', 'GROUP',
> 'ORDER', 'HAVING', 'LIMIT', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS',
> 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE'}(line 1, pos 12)
> == SQL ==
> SELECT 1 AS "NAME"
> ^^^
> at
> org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:217)
> at
> org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:114)
> ...
> {code}
> PostgreSQL and MySQL support this;
> {code}
> ~:$psql -d postgres
> psql (9.5beta1)
> Type "help" for help.
> postgres=# select 1 AS "NAME";
> NAME
> --
> 1
> (1 row)
> mysql> select 1 AS "NAME";
> +--+
> | NAME |
> +--+
> |1 |
> +--+
> 1 row in set (0.00 sec)
> {code}
> Hive does not;
> {code}
> hive> select 1 AS "NAME";
> NoViableAltException(352@[87:7: ( ( ( KW_AS )? identifier ) | ( KW_AS LPAREN
> identifier ( COMMA identifier )* RPAREN ) )?])
> at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
> at org.antlr.runtime.DFA.predict(DFA.java:116)
> {code}
> Trivial though, it might some help for PostgreSQL/MySQL users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21674) Support double/single-quoted strings for alias names in SQL
[
https://issues.apache.org/jira/browse/SPARK-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134669#comment-16134669
]
Chin Han Yu commented on SPARK-21674:
-
Hi, I would like to work on this ticket.
Since it's common to use double quotes or single quotes to escape reserved word
as a column name to PostgreSQL/MySQL users, this improvement may be helpful.
> Support double/single-quoted strings for alias names in SQL
> ---
>
> Key: SPARK-21674
> URL: https://issues.apache.org/jira/browse/SPARK-21674
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.2.0
>Reporter: Takeshi Yamamuro
>Priority: Trivial
> Labels: starter
>
> In the master, users can't use single/double-quoted strings for alias names
> in SQL;
> {code}
> scala> sql("""SELECT 1 AS "NAME" """)
> org.apache.spark.sql.catalyst.parser.ParseException:
> extraneous input '"NAME"' expecting {, ',', 'FROM', 'WHERE', 'GROUP',
> 'ORDER', 'HAVING', 'LIMIT', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS',
> 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE'}(line 1, pos 12)
> == SQL ==
> SELECT 1 AS "NAME"
> ^^^
> at
> org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:217)
> at
> org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:114)
> ...
> {code}
> PostgreSQL and MySQL support this;
> {code}
> ~:$psql -d postgres
> psql (9.5beta1)
> Type "help" for help.
> postgres=# select 1 AS "NAME";
> NAME
> --
> 1
> (1 row)
> mysql> select 1 AS "NAME";
> +--+
> | NAME |
> +--+
> |1 |
> +--+
> 1 row in set (0.00 sec)
> {code}
> Hive does not;
> {code}
> hive> select 1 AS "NAME";
> NoViableAltException(352@[87:7: ( ( ( KW_AS )? identifier ) | ( KW_AS LPAREN
> identifier ( COMMA identifier )* RPAREN ) )?])
> at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
> at org.antlr.runtime.DFA.predict(DFA.java:116)
> {code}
> Trivial though, it might some help for PostgreSQL/MySQL users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21658) Adds the default None for value in na.replace in PySpark to match
[
https://issues.apache.org/jira/browse/SPARK-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125898#comment-16125898
]
Chin Han Yu commented on SPARK-21658:
-
[~hyukjin.kwon]
Hi, here is my JIRA account.
> Adds the default None for value in na.replace in PySpark to match
> -
>
> Key: SPARK-21658
> URL: https://issues.apache.org/jira/browse/SPARK-21658
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.2.0
>Reporter: Hyukjin Kwon
>Priority: Minor
> Labels: Starter
> Fix For: 2.3.0
>
>
> Looks {{na.replace}} missed the default value {{None}}.
> Both docs says they are aliases
> http://spark.apache.org/docs/2.2.0/api/python/pyspark.sql.html#pyspark.sql.DataFrame.replace
> http://spark.apache.org/docs/2.2.0/api/python/pyspark.sql.html#pyspark.sql.DataFrameNaFunctions.replace
> but the default values looks different, which ends up with:
> {code}
> >>> df = spark.createDataFrame([('Alice', 10, 80.0)])
> >>> df.replace({"Alice": "a"}).first()
> Row(_1=u'a', _2=10, _3=80.0)
> >>> df.na.replace({"Alice": "a"}).first()
> Traceback (most recent call last):
> File "", line 1, in
> TypeError: replace() takes at least 3 arguments (2 given)
> {code}
> To take the advantage of SPARK-19454, sounds we should match them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
