airborne12 commented on code in PR #67918:
URL: https://github.com/apache/doris/pull/67918#discussion_r4056695379
##########
fe/fe-core/src/main/java/org/apache/doris/indexpolicy/IndexPolicy.java:
##########
@@ -59,7 +59,8 @@ public class IndexPolicy implements Writable,
GsonPostProcessable {
public static final String PROP_TOKEN_FILTER = "token_filter";
public static final String PROP_CHAR_FILTER = "char_filter";
public static final Set<String> BUILTIN_TOKENIZERS = ImmutableSet.of(
- "empty", "ngram", "edge_ngram", "keyword", "standard",
"char_group", "basic", "icu", "pinyin");
+ "empty", "ngram", "edge_ngram", "keyword", "standard",
"char_group", "basic", "icu", "pinyin",
Review Comment:
Fixed in `191a43c4a77`.
`createIndexPolicy()` now performs the existing-policy lookup under the
write lock and honors `IF NOT EXISTS` before checking whether the requested
name is reserved by a built-in tokenizer. This keeps a replayed legacy
`ik_smart` or `ik_max_word` policy idempotent, while a fresh creation with
either reserved name is still rejected.
`PolicyValidatorTests.testIfNotExistsKeepsReplayedBuiltinTokenizerNameIdempotent`
covers the replayed-policy no-op and fresh-name rejection. The test failed
before the change because the existing `ik_smart` policy hit the built-in
conflict, then passed with the fix. The complete `PolicyValidatorTests` class
passed 38/38 after the master rebase, and `./build.sh --fe` completed
successfully with zero Checkstyle violations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
