bhabegger opened a new pull request, #3089: URL: https://github.com/apache/jackrabbit-oak/pull/3089
[OAK-12360](https://issues.apache.org/jira/browse/OAK-12360) ## Problem Today, an `oak:index` Lucene definition can declare a custom analyzer only under `analyzers/default`. Any other analyzer configured under `analyzers/<name>` is silently ignored — there's no way to apply a different tokenizer/stemmer/stopword-set to individual properties (e.g. a language-specific analyzer for one field while keeping the default for the rest of the index). ## Change Adds a new optional `analyzer` string property on a property definition: ``` indexRules/<nodeType>/properties/<propName>/analyzer = "<name>" ``` referencing a sibling node `analyzers/<name>`, alongside the existing `analyzers/default`. `LuceneIndexDefinition.createAnalyzer()` builds a `PerFieldAnalyzerWrapper` entry for every analyzed property with a resolving `analyzer` reference, keyed by that property's actual Lucene field name (`full:<pname>` for index format V2+, `<pname>` for legacy V1) — the same name `LuceneDocumentMaker` writes documents under, so index-time and query-time (both already resolve through `LuceneIndexDefinition.getAnalyzer()`) stay consistent automatically. Properties that don't set `analyzer` are completely unaffected — fully backward compatible, no feature toggle needed since the change is purely additive/opt-in. **Error handling:** a property's `analyzer` reference that doesn't resolve to an existing `analyzers/<name>` node logs a warning and falls back to the default analyzer for that property only, rather than failing the index build — consistent with the existing convention for other dangling references in `IndexDefinition` (e.g. an aggregate rule referencing a missing property). ## Known limitations (explicitly out of scope for this PR) * The aggregated `:fulltext` field (used by `CONTAINS(*, ...)`) collects raw text from every `nodeScopeIndex=true` property into one shared field, re-analyzed with a single analyzer — per-property analyzers can't differentiate text once merged into `:fulltext`. * Regular-expression property definitions — since the rule matches a different concrete property per node, there's no single property to attach a custom analyzer to. Both fall back to the default analyzer with a logged warning and are documented as known limitations in `oak-doc/src/site/markdown/query/lucene.md`. ## Out of scope * Elasticsearch provider (`oak-search-elastic`) — tracked separately. ## Testing New tests in `PropertyDefinitionTest` and `LuceneIndexDefinitionTest` cover: config parsing, the happy path (custom analyzer applied to the declared property's own field while a sibling property keeps the default), a dangling analyzer reference, regexp property definitions, and the `:fulltext` aggregate-field limitation. Full `oak-search` and `oak-lucene` module suites pass with no regressions. Marked as draft while an internal CI pipeline runs against the branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
