twosom opened a new pull request, #15895: URL: https://github.com/apache/lucene/pull/15895
### Description This PR implements the `keepDecimalPoint` configuration option for `KoreanTokenizer` in the Nori module, as proposed in issue #15894 . The goal is to prevent numeric values containing decimal points (e.g., "10.1") from being fragmented into multiple tokens during the lattice construction phase. This ensures that the semantic integrity of measurements, version numbers, and economic data is preserved from the initial analysis stage. <!-- If this is your first contribution to Lucene, please make sure you have reviewed the contribution guide. https://github.com/apache/lucene/blob/main/CONTRIBUTING.md --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
