[
https://issues.apache.org/jira/browse/SPARK-54914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shubham_bhusate updated SPARK-54914:
------------------------------------
Description:
Enable the DROP operator in SQL pipe syntax to accept qualified column names
(e.g., table.column) in addition to unqualified column names.
When using the pipe syntax DROP operator to remove a field using a qualified
name, Spark throws a ParseException instead of dropping the specified field.
h4. Steps to Reproduce
* Create a temporary view:
spark.sql("""
Create or Replace temp view st as
Select 1 as i1, 2 as i2, 3 as i3
""")
* Attempt to drop the field using a qualified name with pipe syntax:
spark.sql("""
TABLE st
|> as s
|> Drop s.i1
""")
*Expected behaviour:*
It should have successfully dropped the column i1 from the table.
h4. Actual Behavior
Spark throws a parse error:
pyspark.errors.exceptions.captured.ParseException:
[PARSE_SYNTAX_ERROR] Syntax error at or near '.'. SQLSTATE: 42601 (line 4, pos
9)
was:Enable the DROP operator in SQL pipe syntax to accept qualified column
names (e.g., table.column) in addition to unqualified column names.
> Pipe Syntax - Qualified Column Names in DROP Operator
> -----------------------------------------------------
>
> Key: SPARK-54914
> URL: https://issues.apache.org/jira/browse/SPARK-54914
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: shubham_bhusate
> Priority: Major
> Labels: pull-request-available
>
> Enable the DROP operator in SQL pipe syntax to accept qualified column names
> (e.g., table.column) in addition to unqualified column names.
> When using the pipe syntax DROP operator to remove a field using a qualified
> name, Spark throws a ParseException instead of dropping the specified field.
> h4. Steps to Reproduce
> * Create a temporary view:
>
> spark.sql("""
> Create or Replace temp view st as
> Select 1 as i1, 2 as i2, 3 as i3
> """)
> * Attempt to drop the field using a qualified name with pipe syntax:
> spark.sql("""
> TABLE st
> |> as s
> |> Drop s.i1
> """)
>
>
>
> *Expected behaviour:*
> It should have successfully dropped the column i1 from the table.
> h4. Actual Behavior
> Spark throws a parse error:
>
> pyspark.errors.exceptions.captured.ParseException:
> [PARSE_SYNTAX_ERROR] Syntax error at or near '.'. SQLSTATE: 42601 (line 4,
> pos 9)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
