Chen Zhang created SPARK-39380: ---------------------------------- Summary: Ignore comment syntax in dfs command Key: SPARK-39380 URL: https://issues.apache.org/jira/browse/SPARK-39380 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.2.1 Reporter: Chen Zhang
In version 3.2, Spark SQL ignores semicolons in comment syntax when splitting input lines. This behavior throws an unexpected dfs command error: (Related PR: SPARK-33100 SPARK-30049) mkdir testDfsCommand echo 'abc' > testDfsCommand/a---.txt ./spark-2.4.8-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/a---.txt;' # abc ./spark-2.4.8-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/*;' # abc ./spark-3.2.1-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/a---.txt;' # cat: `testDfsCommand/a---.txt;': No such file or directory # Command -cat testDfsCommand/a---.txt; failed with exit code = 1 ./spark-3.2.1-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/*;' # cat: `testDfsCommand/*;': No such file or directory # Command -cat testDfsCommand/*; failed with exit code = 1 -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org