edmondop commented on PR #2610: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2610#issuecomment-5980944390
Agreed that a dialect-specific tokenizer is the right long-term fix for lexing. For Presto, `is_identifier_part` already covers it (#2611), since its grammar allows `:` in every identifier. This hook targets a different case: a dialect that only wants special syntax in table position, or wants the name split into parts, which the tokenizer can't decide without parser context. I'm open to working on the tokenizer side too. If useful, I can open an issue sketching it, for example a `Dialect` hook that can emit a token before the default tokenizer runs, with the same "return `None` to fall back" contract as `parse_prefix`. Would you merge this hook in the meantime, or rather wait for that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
