[
https://issues.apache.org/jira/browse/FLINK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17777035#comment-17777035
]
xiaogang zhou commented on FLINK-33249:
---------------------------------------
Hi [~martijnvisser]
https://issues.apache.org/jira/browse/CALCITE-6001 will improve CALCITE to omit
the charset from the generated literal when it is the default charset of the
DIALECT. maybe wait for the CALCITE future version and set the FLINK DIALECT to
use UTF-8
> comment should be parsed by StringLiteral() instead of SqlCharStringLiteral
> to avoid parsing failure
> ----------------------------------------------------------------------------------------------------
>
> Key: FLINK-33249
> URL: https://issues.apache.org/jira/browse/FLINK-33249
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Affects Versions: 1.17.1
> Reporter: xiaogang zhou
> Priority: Major
> Labels: pull-request-available
>
> this problem is also recorded in calcite
>
> https://issues.apache.org/jira/browse/CALCITE-6046
>
> Hi, I found this problem when I used below code to split SQL statements. the
> process is SQL string -> SqlNode -> SQL String
> {code:java}
> // code placeholder
> SqlParser.Config parserConfig = getCurrentSqlParserConfig(sqlDialect);
> SqlParser sqlParser = SqlParser.create(sqlContent, parserConfig);
> SqlNodeList sqlNodeList = sqlParser.parseStmtList();
> sqlParser.parse(sqlNodeList.get(0));{code}
> the Dialect/ SqlConformance is a costumed one:
> [https://github.com/apache/flink/blob/master/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/validate/FlinkSqlConformance.java]
>
>
> then I found below SQL
> {code:java}
> // code placeholder
> CREATE TABLE source (
> a BIGINT
> ) comment '测试test'
> WITH (
> 'connector' = 'test'
> ); {code}
> transformed to
> {code:java}
> // code placeholder
> CREATE TABLE `source` (
> `a` BIGINT
> )
> COMMENT u&'\5218\51eftest' WITH (
> 'connector' = 'test'
> ) {code}
>
> and the SQL parser template is like
> {code:java}
> // code placeholder
> SqlCreate SqlCreateTable(Span s, boolean replace, boolean isTemporary) :
> {
> final SqlParserPos startPos = s.pos();
> boolean ifNotExists = false;
> SqlIdentifier tableName;
> List<SqlTableConstraint> constraints = new
> ArrayList<SqlTableConstraint>();
> SqlWatermark watermark = null;
> SqlNodeList columnList = SqlNodeList.EMPTY;
> SqlCharStringLiteral comment = null;
> SqlTableLike tableLike = null;
> SqlNode asQuery = null;
> SqlNodeList propertyList = SqlNodeList.EMPTY;
> SqlNodeList partitionColumns = SqlNodeList.EMPTY;
> SqlParserPos pos = startPos;
> }
> {
> <TABLE>
> ifNotExists = IfNotExistsOpt()
> tableName = CompoundIdentifier()
> [
> <LPAREN> { pos = getPos(); TableCreationContext ctx = new
> TableCreationContext();}
> TableColumn(ctx)
> (
> <COMMA> TableColumn(ctx)
> )*
> {
> pos = pos.plus(getPos());
> columnList = new SqlNodeList(ctx.columnList, pos);
> constraints = ctx.constraints;
> watermark = ctx.watermark;
> }
> <RPAREN>
> ]
> [ <COMMENT> <QUOTED_STRING> {
> String p = SqlParserUtil.parseString(token.image);
> comment = SqlLiteral.createCharString(p, getPos());
> }]
> [
> <PARTITIONED> <BY>
> partitionColumns = ParenthesizedSimpleIdentifierList()
> ]
> [
> <WITH>
> propertyList = TableProperties()
> ]
> [
> <LIKE>
> tableLike = SqlTableLike(getPos())
> {
> return new SqlCreateTableLike(startPos.plus(getPos()),
> tableName,
> columnList,
> constraints,
> propertyList,
> partitionColumns,
> watermark,
> comment,
> tableLike,
> isTemporary,
> ifNotExists);
> }
> |
> <AS>
> asQuery = OrderedQueryOrExpr(ExprContext.ACCEPT_QUERY)
> {
> return new SqlCreateTableAs(startPos.plus(getPos()),
> tableName,
> columnList,
> constraints,
> propertyList,
> partitionColumns,
> watermark,
> comment,
> asQuery,
> isTemporary,
> ifNotExists);
> }
> ]
> {
> return new SqlCreateTable(startPos.plus(getPos()),
> tableName,
> columnList,
> constraints,
> propertyList,
> partitionColumns,
> watermark,
> comment,
> isTemporary,
> ifNotExists);
> }
> } {code}
> will give a exception :
> Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered
> "u&\'\\5218\\51eftest\'" at line 4, column 9.
> Was expecting:
> <QUOTED_STRING> ...
>
> so I think all the SqlCharStringLiteral should be replaced by StringLiteral()
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
