github-actions[bot] commented on code in PR #67918:
URL: https://github.com/apache/doris/pull/67918#discussion_r4079768031
##########
be/src/runtime/index_policy/index_policy_mgr.cpp:
##########
@@ -41,13 +43,72 @@ class SingleAnalyzerProvider final : public
segment_v2::inverted_index::Analyzer
const std::unordered_set<std::string> IndexPolicyMgr::BUILTIN_NORMALIZERS =
{"lowercase"};
-std::string IndexPolicyMgr::normalize_name(const std::string& name) {
+std::string IndexPolicyMgr::trim_name(const std::string& name) {
std::string result = name;
boost::algorithm::trim(result);
+ return result;
+}
+
+std::string IndexPolicyMgr::normalize_name(const std::string& name) {
+ std::string result = trim_name(name);
boost::algorithm::to_lower(result);
return result;
}
+const TIndexPolicy* IndexPolicyMgr::find_policy_by_name_locked(const
std::string& name) const {
+ const std::string exact_name = trim_name(name);
+ if (auto exact_it = _exact_name_to_id.find(exact_name); exact_it !=
_exact_name_to_id.end()) {
Review Comment:
[P1] Preserve the component binding used by existing postings across
upgrade. This is a residual beyond the earlier exact-binding thread: retaining
both logical policies cannot recover which normalized first-wins component the
base BE actually used to build a segment. The new collision test accepts the
arrival order where lowercase `ik_smart` (keyword) precedes `IK_SMART`
(standard), while `legacy_exact_analyzer` references `IK_SMART`. Under the base
hunk that order rejects the latter and writes `one two` as one keyword term;
after the supported BE-first upgrade, this line retains both and rebinds the
analyzer to the exact standard tokenizer, so `MATCH 'one'` no longer agrees
with the old postings and new segments also diverge. FE emits its full policy
`HashMap` without an ordering contract, and index metadata contains no resolved
component ID/config to recover what built it. Please preserve/version the
resolved binding for physical indexes or require a rebuild/migration, and add
an upgrade test that reads a base-manager segment after installing the new
manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
