kbendick commented on a change in pull request #4396:
URL: https://github.com/apache/iceberg/pull/4396#discussion_r835665729
##########
File path:
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestCallStatementParser.java
##########
@@ -137,6 +139,28 @@ public void testCallParseError() {
() -> parser.parsePlan("CALL cat.system radish kebab"));
}
+ @Test
+ public void testCallStripsLeadingComments() throws ParseException {
+ // These comments are meant to look like those that systems like DBT would
add to statements.
+ List<String> callStatementsWithComments = Lists.newArrayList(
Review comment:
In fact, plain `spark.sql` can't handle multiple commands at once.
I tried this in a test, `sql("INSERT INTO TABLE %s VALUES (1, 'a'); INSERT
INTO TABLE %s VALUES(2, 'a');", tableName, tableName)`, and got the following
error message:
```
== SQL ==
INSERT INTO TABLE testhive.default.table VALUES (1, 'a'); INSERT INTO TABLE
testhive.default.table VALUES(2, 'a');
----------------------------------------------------------^^^
at
app//org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:266)
at
app//org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:127)
at
app//org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:51)
at
app//org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:77)
at
app//org.apache.spark.sql.catalyst.parser.extensions.IcebergSparkSqlExtensionsParser.parsePlan(IcebergSparkSqlExtensionsParser.scala:123)
```
So that's falling back to the delegate parser (Spark's own parser). But the
case of a single SQL command having comments at the beginning (or in the middle
or at the end) is covered by this change. Previously, it would have worked for
commands that are not covered by `isIcebergCommend` - e.g. anything starting
with `call` or some subset of `alter table` commands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
