[
https://issues.apache.org/jira/browse/OAK-9719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504474#comment-17504474
]
Henry Kuijpers commented on OAK-9719:
-------------------------------------
Thank you so much!
> Query read limit should be overridable through query option
> ------------------------------------------------------------
>
> Key: OAK-9719
> URL: https://issues.apache.org/jira/browse/OAK-9719
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
> Reporter: Henry Kuijpers
> Priority: Major
>
> When executing a query, it could happen that a query yields so many results
> (for example in the case of migration scripts), that it's causing a failure.
> Not while executing the query, but while iterating through the results of the
> query.
> We have a few migration scripts in our codebase that need to migrate content
> (such as CMS components, pages, ...). We also have, especially on production,
> quite a lot of content. Such scripts can easily find 100.000+ nodes and thus
> produce a resultset that is bigger than the "query read limit".
> This limit can currently be configured on system-level, either through a
> system property, or through OSGi configuration. QueryEngineSettingsService
> takes care of that.
> Raising this limit means raising the limit for the entire system. For every
> query that is executed. It would be ideal if we could configure this limit on
> the query level, for example through an option (like the options for
> traversal and for index tag selection). I would propose to add an option:
> "select * from ... option (readlimit 999999)"
> which would take precedence over the limit that is active in
> QueryEngineSettingsService. Then, it would be the responsibility of the
> developer who creates the query to specify the correct overridden limit (or
> not specify a limit at all, of course).
> Stacktrace of such a failing query, currently:
> {code:java}
> 10.03.2022 16:55:00.032 *WARN* [qtp881876674-5121]
> org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor
> Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path],
> [jcr:score], * from [cq:Page] as a where isdescendantnode(a, '/content') /*
> xpath: /jcr:root/content//element(*, cq:Page) */, path=/content//*)
> 10.03.2022 16:55:00.228 *ERROR* [qtp881876674-5121]
> com.day.crx.delite.impl.servlets.QueryServlet Exception while searching
> org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read
> or traversed more than 100000 nodes. To avoid affecting other tasks,
> processing was stopped.
> at
> org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
> [org.apache.jackrabbit.oak-lucene:1.22.9]
> at
> org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
> [org.apache.jackrabbit.oak-lucene:1.22.9]
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
> [org.apache.jackrabbit.oak-lucene:1.22.9]
> at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
> [org.apache.jackrabbit.oak-core:1.22.9]
> at
> org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
> [org.apache.jackrabbit.oak-jcr:1.22.9]
> at
> org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
> [org.apache.jackrabbit.oak-jcr:1.22.9]
> at
> org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
> [org.apache.jackrabbit.oak-jcr:1.22.9]
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
> [org.apache.jackrabbit.oak-jcr:1.22.9]
> at
> org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
> [org.apache.jackrabbit.oak-jcr:1.22.9]
> at
> org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
> [org.apache.jackrabbit.jackrabbit-jcr-commons:2.20.2]
> at
> org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
> [org.apache.jackrabbit.jackrabbit-jcr-commons:2.20.2]
> at
> org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
> [org.apache.jackrabbit.jackrabbit-jcr-commons:2.20.2]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
