Baymine opened a new issue, #66163: URL: https://github.com/apache/doris/issues/66163
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description `HiveTextFieldSplitter` (`be/src/format/text/text_reader.cpp`) splits every Hive text line on the hot scan path. Two avoidable costs exist: 1. Single-char separator, no escape char (the common case): the line is scanned one byte at a time in a `for + if` loop, even though `memchr` can locate the next separator with a vectorized library routine. 2. Multi-char separator (MultiDelimitSerDe): the KMP prefix ("next") table is recomputed from the separator on *every* line, although it depends only on the separator and can be built once. Both are pure micro-optimizations of an existing, behavior-preserving code path. ### Solution - Add a `memchr`-based fast path to `_split_field_single_char` for `escape_char == 0`; keep the byte-by-byte path (with `is_hive_text_separator_escaped`) for the escape case. - Precompute the KMP prefix table once in the `HiveTextFieldSplitter` constructor and reuse it in `_split_field_multi_char`. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
