[jira] [Commented] (OAK-3495) Query engine: disable or restrict built-in full-text engine (backport to 1.0)
[ https://issues.apache.org/jira/browse/OAK-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954709#comment-14954709 ] Thomas Mueller commented on OAK-3495: - http://svn.apache.org/r1708313 > Query engine: disable or restrict built-in full-text engine (backport to 1.0) > - > > Key: OAK-3495 > URL: https://issues.apache.org/jira/browse/OAK-3495 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.0.23 > > > With Oak 1.0.x, the built-in full-text engine is used if no other full-text > index is available. This is different than in newer versions of Oak, where > the built-in engine is disabled in that case by default. > I think it makes sense to backport this to Oak 1.0.x, so that all versions > behave the same way. > This is to prevent out-of-memory problems if no full-text index is configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3495) Query engine: disable or restrict built-in full-text engine (backport to 1.0)
[ https://issues.apache.org/jira/browse/OAK-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954615#comment-14954615 ] Thomas Mueller commented on OAK-3495: - OK, thanks! I will commit it then. If the OOME problem persists, I will open a new issue. > Query engine: disable or restrict built-in full-text engine (backport to 1.0) > - > > Key: OAK-3495 > URL: https://issues.apache.org/jira/browse/OAK-3495 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.0.23 > > > With Oak 1.0.x, the built-in full-text engine is used if no other full-text > index is available. This is different than in newer versions of Oak, where > the built-in engine is disabled in that case by default. > I think it makes sense to backport this to Oak 1.0.x, so that all versions > behave the same way. > This is to prevent out-of-memory problems if no full-text index is configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3495) Query engine: disable or restrict built-in full-text engine (backport to 1.0)
[ https://issues.apache.org/jira/browse/OAK-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14950090#comment-14950090 ] Davide Giannella commented on OAK-3495: --- [~tmueller] patch looks good for what I can see and it's a pragmatic approach that should address the problem. In this way we restrict the options of OOM but we don't really prevent it. On a webscale someone could request several times concurrently a very very big string which would increase the memory usage. But I think that the use case is very very limited and the optimisations on Strings of the JVM and reusing the same object from the heap limit it to a probability of almost none. > Query engine: disable or restrict built-in full-text engine (backport to 1.0) > - > > Key: OAK-3495 > URL: https://issues.apache.org/jira/browse/OAK-3495 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.0.23 > > > With Oak 1.0.x, the built-in full-text engine is used if no other full-text > index is available. This is different than in newer versions of Oak, where > the built-in engine is disabled in that case by default. > I think it makes sense to backport this to Oak 1.0.x, so that all versions > behave the same way. > This is to prevent out-of-memory problems if no full-text index is configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3495) Query engine: disable or restrict built-in full-text engine (backport to 1.0)
[ https://issues.apache.org/jira/browse/OAK-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14948637#comment-14948637 ] Thomas Mueller commented on OAK-3495: - [~alex.parvulescu] and [~edivad], could you please review the above patch? It is really just to prevent OOME. First I thought we should also disable the built-in engine, but now I see it would require more changes. The above code should match the Oak 1.2 and newer behavior _if_ the built-in engine would be enabled there. This is the case for some of the unit tests. > Query engine: disable or restrict built-in full-text engine (backport to 1.0) > - > > Key: OAK-3495 > URL: https://issues.apache.org/jira/browse/OAK-3495 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.0.23 > > > With Oak 1.0.x, the built-in full-text engine is used if no other full-text > index is available. This is different than in newer versions of Oak, where > the built-in engine is disabled in that case by default. > I think it makes sense to backport this to Oak 1.0.x, so that all versions > behave the same way. > This is to prevent out-of-memory problems if no full-text index is configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3495) Query engine: disable or restrict built-in full-text engine (backport to 1.0)
[
https://issues.apache.org/jira/browse/OAK-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14948626#comment-14948626
]
Thomas Mueller commented on OAK-3495:
-
I found out that _disabling_ the built-in full-text engine would require
changing quite many tests (that means, also backport changes to the tests).
That would be possible, but would require more changes.
The following patch just _restricts_ the built-in full-text engine to String
only. That means binaries are not read. This should avoid out-of-memory. It is
the same code as used in Oak 1.2.x and newer.
{noformat}
oak-core
===
--- src/main/java/org/apache/jackrabbit/oak/query/ast/FullTextSearchImpl.java
(revision 1707457)
+++ src/main/java/org/apache/jackrabbit/oak/query/ast/FullTextSearchImpl.java
(working copy)
@@ -215,13 +215,23 @@
private static void appendString(StringBuilder buff, PropertyValue p) {
if (p.isArray()) {
-for (String v : p.getValue(STRINGS)) {
-buff.append(v).append(' ');
+if (p.getType() == Type.BINARIES) {
+// OAK-3495: don't try to load binaries as this would
+// run out of memory
+} else {
+for (String v : p.getValue(STRINGS)) {
+buff.append(v).append(' ');
+}
}
} else {
-buff.append(p.getValue(STRING)).append(' ');
+if (p.getType() == Type.BINARY) {
+// OAK-3495: don't try to load binaries as this would
+// run out of memory
+} else {
+buff.append(p.getValue(STRING)).append(' ');
+}
}
-}
+}
public void bindSelector(SourceImpl source) {
selector = source.getExistingSelector(selectorName);
{noformat}
> Query engine: disable or restrict built-in full-text engine (backport to 1.0)
> -
>
> Key: OAK-3495
> URL: https://issues.apache.org/jira/browse/OAK-3495
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
> Fix For: 1.0.23
>
>
> With Oak 1.0.x, the built-in full-text engine is used if no other full-text
> index is available. This is different than in newer versions of Oak, where
> the built-in engine is disabled in that case by default.
> I think it makes sense to backport this to Oak 1.0.x, so that all versions
> behave the same way.
> This is to prevent out-of-memory problems if no full-text index is configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
