gengliangwang commented on PR #46013: URL: https://github.com/apache/spark/pull/46013#issuecomment-2052657458
@yaooqinn enabling the ANSI SQL mode does address certain unreasonable SQL behaviors, such as integer overflow and division by zero, which could potentially disrupt users' workloads in a more understandable manner. ``` > select 2147483647 + 1; -2147483648 > select 1/0; NULL ``` This may break users' workload in a relatively reasonable way. However, toggling `spark.sql.ansi.enforceReservedKeywords` introduces a different set of challenges. For instance, attempting to query a table named "user" triggers a syntax error: ``` > select * from user; [PARSE_SYNTAX_ERROR] Syntax error at or near 'user'.(line 1, pos 14) == SQL == select * from user --------------^^^ ``` This could significantly disrupt existing workflows. Similarly, activating `spark.sql.ansi.doubleQuotedIdentifiers` results in unresolved column errors: ``` > select "hello world" [UNRESOLVED_COLUMN.WITHOUT_SUGGESTION] A column or function parameter with name `hello world` cannot be resolved. ; line 1 pos 7; 'Project ['hello world] +- OneRowRelation > select * from foo where str="some_value"; [UNRESOLVED_COLUMN.WITH_SUGGESTION] A column or function parameter with name `some_value` cannot be resolved. Did you mean one of the following? [`str`].; line 1 pos 28; 'Project [*] +- 'Filter (str#70 = 'some_value) +- SubqueryAlias spark_catalog.default.foo +- Relation spark_catalog.default.foo[str#70] parquet ``` This could cause frustration for users, especially considering the absence of clear benefits compared to the aforementioned ANSI SQL mode behaviors. While these small flags serve a purpose, they possess the potential to disrupt existing workloads with less substantial justification than the ANSI SQL mode adjustments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org