github-actions[bot] commented on code in PR #67918:
URL: https://github.com/apache/doris/pull/67918#discussion_r4056763551
##########
fe/fe-core/src/main/java/org/apache/doris/indexpolicy/IndexPolicyMgr.java:
##########
@@ -140,7 +195,7 @@ private void
validateReferencedComponentsUsableLocked(String analyzerName, Index
return;
}
for (String tokenFilterName : tokenFilterNames.split(",\\s*")) {
- IndexPolicy tokenFilter =
nameToIndexPolicy.get(normalizeKey(tokenFilterName));
Review Comment:
[P1] Validate nested tokenizer references with the exact policy binding.
`validateReferencedComponentsUsableLocked()` still reads
`nameToIndexPolicy.get(normalizeKey(tokenizerName))`, even though this PR adds
exact-name bindings and the token-filter branch now calls
`getPolicyByNameLocked()`. After replaying case-distinct tokenizer policies
such as invalid `Foo` and valid `foo`, an analyzer persisted with
`tokenizer=Foo` can pass FE validation against `foo`, then BE's exact-first
`build_analyzer_config_from_policy()` selects `Foo` and fails or behaves
differently; the reverse collision falsely rejects the valid exact binding.
Resolve this tokenizer through `getPolicyByNameLocked(tokenizerName)` (and
preserve the expected TOKENIZER type check) and add replay/image coverage for
colliding valid/invalid references.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
