g4lb opened a new pull request, #3007:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/3007

   Dialects: none
   
   `Tokenizer::tokenize_word` collected the rest of an identifier into a 
temporary `String` with `peeking_take_while`, then copied it into the word. Now 
it appends the characters to the word directly. The tokens do not change.
   
   | Benchmark | Before | After | Change |
   |---|---:|---:|---:|
   | tokenize TPC-H Q1 (local) | 6.93 µs | 6.00 µs | -13.4% |
   | `parse_compound_chain/chain_20` | 4.43 µs | 4.20 µs | -5.2% |
   
   No other `sqlparser_bench` benchmark changed significantly. I used 10 
interleaved rounds of criterion and a Mann-Whitney test (Apple M5, rustc 
1.98.0).
   
   Tokens, ASTs, `Display` output and errors matched the old code on 1M SQL 
statements in 14 dialects. The statements came from the tests and fuzz seeds, 
also with random changes.
   
   AI assistance: an AI agent found and measured this change. I read the diff 
and the results.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to