[ https://issues.apache.org/jira/browse/SLING-11439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17601815#comment-17601815 ]
Julian Reschke commented on SLING-11439: ---------------------------------------- [~thomasm] re https://issues.apache.org/jira/browse/SLING-11439?focusedCommentId=17564308&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17564308: {noformat} This is not actually needed. Instead, we can iterate over the result, counting the entries in the loop itself, and stop when needed. E.g. lastEntry = ""; while (true) { it = query("select [sling:vanityPath], [sling:redirect], [sling:redirectStatus] " + "from [nt:base] where first([sling:vanityPath]) >= $lastEntry " + "order by first([sling:vanityPath]), [jcr:path]", lastEntry); for (int i = 0; it.hasNext(); i++) { String x = it.next().get("[sling:vanityPath]"); if (i > 1000 && !x.equals(lastEntry)) { // next page lastEntry = x; break; } ... remember x (in the Bloom filter) ... } } BTW we need "first(..)" because sling:vanityPath can be a multi-valued property. {noformat} That doesn't seem to work. In my test I see that org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException is thrown on getting the iterator for the query result: {noformat} 08.09.2022 14:19:38.945 *ERROR* [FelixDispatchQueue] org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl activate: Cannot access repository, failed setting up Mapping Support org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped. at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNextPossiblyDuplicate(ContentMirrorStoreStrategy.java:459) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNext(ContentMirrorStoreStrategy.java:402) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:488) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:338) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at com.google.common.collect.Iterators$5.next(Iterators.java:553) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.hasNext(FulltextIndex.java:419) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028] at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002] at org.apache.jackrabbit.oak.plugins.index.cursor.PathCursor.hasNext(PathCursor.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:452) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:529) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:522) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:933) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:960) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:207) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:241) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.QueryImpl.getRows(QueryImpl.java:631) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.query.ResultImpl$1.iterator(ResultImpl.java:72) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:177) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028] at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:174) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028] at org.apache.sling.jcr.resource.internal.helper.jcr.BasicQueryLanguageProvider.findResources(BasicQueryLanguageProvider.java:83) [org.apache.sling.jcr.resource:3.2.0] at org.apache.sling.resourceresolver.impl.providers.stateful.AuthenticatedResourceProvider.findResources(AuthenticatedResourceProvider.java:285) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT] at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.findResources(ResourceResolverControl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT] at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.findResources(ResourceResolverImpl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT] at org.apache.sling.resourceresolver.impl.mapping.MapEntries.queryAllVanityPaths(MapEntries.java:1282) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT] {noformat} So it's not possible just to read a "small" number of results, and then to restart the query. > resource resolver: fails to detect aborted vanity path query > ------------------------------------------------------------ > > Key: SLING-11439 > URL: https://issues.apache.org/jira/browse/SLING-11439 > Project: Sling > Issue Type: Bug > Components: ResourceResolver > Reporter: Julian Reschke > Priority: Major > > With the introduction of the Oak query limit, JCR queries may get aborted > when the result set size exceeds a certain value (currently by default > 100000). > However: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/604332e9be17378276685033bdbce54994dad8c1/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResourceIterator.java#L115-L134 > So Apache Sling JCR Resource's API hides that exception, and thus resource > resolver will happily startup with an incomplete cache. -- This message was sent by Atlassian Jira (v8.20.10#820010)