[jira] [Updated] (OAK-9091) oak-search-elastic: review query code
[ https://issues.apache.org/jira/browse/OAK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9091: -- Fix Version/s: 1.32.0 > oak-search-elastic: review query code > - > > Key: OAK-9091 > URL: https://issues.apache.org/jira/browse/OAK-9091 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: elastic-search, indexing >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.32.0 > > > The code to execute queries in the elastic module needs to be improved. > It has been inherited from the first prototype but has never been > reviewed/refactored. > Here is what needs to be done: > * ElasticsearchRowIterator [0] is the central point. We need to break it up > and separate the query building phase from the interpretation of the response > * Centralize how we resolve property names. This affects also mapping and > indexing phases > * We need to review how the queries are composed to increase performance. As > a rule of thumb, we should always create a BoolQuery and leverage as much as > possible the use of filters over matching queries. This allows us to use ES > caching efficiently > * Increase test coverage. Most of the testing in this area requires ES > integration. We should add more unit tests checking that input query plans > generate the expected ES requests. > [0] > [https://github.com/oak-indexing/jackrabbit-oak/blob/8cc86c4d142004bf2550cf2ac66400b54093209d/oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elasticsearch/query/ElasticsearchResultRowIterator.java] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9111) Default Analyzer for remote index / elastic search
[
https://issues.apache.org/jira/browse/OAK-9111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabrizio Fortino updated OAK-9111:
--
Fix Version/s: 1.32.0
> Default Analyzer for remote index / elastic search
> --
>
> Key: OAK-9111
> URL: https://issues.apache.org/jira/browse/OAK-9111
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing, search, search-elastic
>Reporter: Fabrizio Fortino
>Assignee: Fabrizio Fortino
>Priority: Major
> Fix For: 1.32.0
>
>
> Need to find out what needs to be done / if this is possible / how to do it /
> what are our options.
> The current default OakAnalyzer uses the following configuration:
> {noformat}
> TokenStreamComponents(
> WordDelimiterFilter(
> LowerCaseFilter(
> StandardTokenizer(
> Reader
> )
> ),
> GENERATE_WORD_PARTS
> | STEM_ENGLISH_POSSESSIVE
> | (indexOriginalTerm ? PRESERVE_ORIGINAL : 0)
> | GENERATE_NUMBER_PARTS
> )
> ){noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OAK-9109) Upgrade to Elasticsearch 7.7.1
[ https://issues.apache.org/jira/browse/OAK-9109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9109: -- Fix Version/s: 1.32.0 > Upgrade to Elasticsearch 7.7.1 > -- > > Key: OAK-9109 > URL: https://issues.apache.org/jira/browse/OAK-9109 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.32.0 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.7/release-notes-7.7.1.html|https://www.elastic.co/guide/en/elasticsearch/reference/7.7/release-notes-7.7.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9119) oak-search-elastic: support order by clause
[ https://issues.apache.org/jira/browse/OAK-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9119: -- Fix Version/s: 1.32.0 > oak-search-elastic: support order by clause > --- > > Key: OAK-9119 > URL: https://issues.apache.org/jira/browse/OAK-9119 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.32.0 > > > Support `order by` clause with the same lucene behaviour -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9123) Error: Document contains at least one immense term
[
https://issues.apache.org/jira/browse/OAK-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabrizio Fortino updated OAK-9123:
--
Fix Version/s: 1.32.0
> Error: Document contains at least one immense term
> --
>
> Key: OAK-9123
> URL: https://issues.apache.org/jira/browse/OAK-9123
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: elastic-search, indexing, search
>Reporter: Fabrizio Fortino
>Assignee: Fabrizio Fortino
>Priority: Major
> Fix For: 1.32.0
>
>
> {code:java}
> 11:35:09.400 [I/O dispatcher 1] ERROR o.a.j.o.p.i.e.i.ElasticIndexWriter -
> Bulk item with id /wikipedia/76/84/National Palace (Mexico) failed
> org.elasticsearch.ElasticsearchException: Elasticsearch exception
> [type=illegal_argument_exception, reason=Document contains at least one
> immense term in field="text.keyword" (whose UTF8 encoding is longer than the
> max length 32766), all of which were skipped. Please correct the analyzer to
> not produce such terms. The prefix of the first immense term is: '[123, 123,
> 73, 110, 102, 111, 98, 111, 120, 32, 104, 105, 115, 116, 111, 114, 105, 99,
> 32, 98, 117, 105, 108, 100, 105, 110, 103, 10, 124, 110]...', original
> message: bytes can be at most 32766 in length; got 33409]
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:496)
> at
> org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:407)
> at
> org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:138)
> at
> org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:196)
> at
> org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1888)
> at
> org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAsyncAndParseEntity$10(RestHighLevelClient.java:1676)
> at
> org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:1758)
> at
> org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:590)
> at org.elasticsearch.client.RestClient$1.completed(RestClient.java:333)
> at org.elasticsearch.client.RestClient$1.completed(RestClient.java:327)
> at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
> at
> org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
> at
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
> at
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
> at
> org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
> at
> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
> at
> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
> at
> org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.elasticsearch.ElasticsearchException: Elasticsearch exception
> [type=max_bytes_length_exceeded_exception, reason=bytes can be at most 32766
> in length; got 33409]
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:496)
> at
> org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:407)
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:437)
> ... 24 common frames omitted{code}
> This happens with huge keyword fields since Lucene doesn't allow terms with
> more than 32k bytes.
> See
> [https://discuss.elastic.co/t/error-document-contains-at-least-one-immense-term-in-field/66486]
> We have decided to always create keyword fields to remove the need to specify
> properties like ordered or facet. In this way every field can be sorted or
> used as facet.
> In this specific case the keyword field won't be needed at all but it would
> be hard to decide when include it or not. To solve this we are going to use
> `ignore_above=256` so huge keyword fields will be
[jira] [Updated] (OAK-9120) oak-search-elastic: integrate spellcheck with async iterator
[ https://issues.apache.org/jira/browse/OAK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9120: -- Fix Version/s: 1.32.0 > oak-search-elastic: integrate spellcheck with async iterator > > > Key: OAK-9120 > URL: https://issues.apache.org/jira/browse/OAK-9120 > Project: Jackrabbit Oak > Issue Type: Task > Components: indexing, search, search-elastic >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.32.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OAK-9185) AbstractAccessControlManager: improve refresh strategy of PermissionProvider
[
https://issues.apache.org/jira/browse/OAK-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Angela Schreiber reassigned OAK-9185:
-
Assignee: Angela Schreiber
> AbstractAccessControlManager: improve refresh strategy of PermissionProvider
>
>
> Key: OAK-9185
> URL: https://issues.apache.org/jira/browse/OAK-9185
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, jcr, security
>Reporter: Angela Schreiber
>Assignee: Angela Schreiber
>Priority: Major
> Labels: performance
>
> since {{AbstractAccessControlManager}} does not have access to the
> {{RefreshStrategy}} present in _oak-jcr_, it currently eagerly refresh the
> {{PermissionProvider}} used internally. [~asanso] reported that this may lead
> to performance issues, when item read operations are combined with frequent
> calls to {{AccessControlManager.hasPrivileges}} or
> {{AccessControlManager.getPrivileges}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (OAK-9189) Benchmark Results - Status Quo
Angela Schreiber created OAK-9189: - Summary: Benchmark Results - Status Quo Key: OAK-9189 URL: https://issues.apache.org/jira/browse/OAK-9189 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Angela Schreiber -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OAK-9186) Create Benchmark(s)
[ https://issues.apache.org/jira/browse/OAK-9186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Angela Schreiber reassigned OAK-9186: - Assignee: Angela Schreiber > Create Benchmark(s) > --- > > Key: OAK-9186 > URL: https://issues.apache.org/jira/browse/OAK-9186 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: benchmarks >Reporter: Angela Schreiber >Assignee: Angela Schreiber >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9184) Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
[
https://issues.apache.org/jira/browse/OAK-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Mueller updated OAK-9184:
Fix Version/s: 1.34.0
> Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
> ---
>
> Key: OAK-9184
> URL: https://issues.apache.org/jira/browse/OAK-9184
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Affects Versions: 1.22.4, 1.32.0, 1.8.23
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Major
> Fix For: 1.34.0
>
>
> We found one case of a very slow, possibly endless loop in this method.
> {noformat}
> java.lang.Thread.State: RUNNABLE at
> org.apache.lucene.search.TopScoreDocCollector$InOrderPagingScoreDocCollector.collect(TopScoreDocCollector.java:85)
> at org.apache.lucene.search.Scorer.score(Scorer.java:65) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448) at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:243) at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.loadDocs(LucenePropertyIndex.java:434)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:353)
> {noformat}
> Background: Many threads are waiting to lock here:
> {noformat}
> - waiting to lock <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:202)
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null;
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path); <<= synchronized method
> }
> }
> {noformat}
> And the thread that is holding that (synchronized) lock on IndexTracker is
> also waiting to get a write lock here:
> {noformat}
> "oak-lucene-992" #21559 daemon prio=1 os_prio=0 tid=0x7ed0280ad000
> nid=0x1be8 waiting on condition [0x7fb96c352000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7fbedad77210> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.close(IndexNodeManager.java:174)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.close(IndexTracker.java:104)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.update(IndexTracker.java:114)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> void close() throws IOException {
> lock.writeLock().lock(); <<== waiting here
> try {
> checkState(!closed);
> closed = true;
> } finally {
> lock.writeLock().unlock();
> }
> {noformat}
> It looks like a read lock is not released. The places where a read lock is
> acquired:
> {noformat}
> IndexNode acquire() {
> lock.readLock().lock();
> (released if the method returns null or throws an exception)
> private void release() {
> lock.readLock().unlock();
> }
> {noformat}
> acquire() is called in acquireIndexNode:
> {noformat}
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null; <<==
> here
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path);
> }
> }
> private synchronized IndexNode findIndexNode(String path) {
> // Retry the lookup from acquireIndexNode now that we're
> // synchronized. The acquire() call is guaranteed to
[jira] [Commented] (OAK-9184) Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
[
https://issues.apache.org/jira/browse/OAK-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186321#comment-17186321
]
Thomas Mueller commented on OAK-9184:
-
http://svn.apache.org/r1881271 (trunk)
Now need to backport to Oak 1.8
> Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
> ---
>
> Key: OAK-9184
> URL: https://issues.apache.org/jira/browse/OAK-9184
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Affects Versions: 1.22.4, 1.32.0, 1.8.23
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Major
>
> We found one case of a very slow, possibly endless loop in this method.
> {noformat}
> java.lang.Thread.State: RUNNABLE at
> org.apache.lucene.search.TopScoreDocCollector$InOrderPagingScoreDocCollector.collect(TopScoreDocCollector.java:85)
> at org.apache.lucene.search.Scorer.score(Scorer.java:65) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448) at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:243) at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.loadDocs(LucenePropertyIndex.java:434)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:353)
> {noformat}
> Background: Many threads are waiting to lock here:
> {noformat}
> - waiting to lock <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:202)
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null;
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path); <<= synchronized method
> }
> }
> {noformat}
> And the thread that is holding that (synchronized) lock on IndexTracker is
> also waiting to get a write lock here:
> {noformat}
> "oak-lucene-992" #21559 daemon prio=1 os_prio=0 tid=0x7ed0280ad000
> nid=0x1be8 waiting on condition [0x7fb96c352000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7fbedad77210> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.close(IndexNodeManager.java:174)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.close(IndexTracker.java:104)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.update(IndexTracker.java:114)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> void close() throws IOException {
> lock.writeLock().lock(); <<== waiting here
> try {
> checkState(!closed);
> closed = true;
> } finally {
> lock.writeLock().unlock();
> }
> {noformat}
> It looks like a read lock is not released. The places where a read lock is
> acquired:
> {noformat}
> IndexNode acquire() {
> lock.readLock().lock();
> (released if the method returns null or throws an exception)
> private void release() {
> lock.readLock().unlock();
> }
> {noformat}
> acquire() is called in acquireIndexNode:
> {noformat}
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null; <<==
> here
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path);
> }
> }
> private synchronized IndexNode findIndexNode(String path) {
> // Retry the lookup from acquireIndexNode now that we're
>
[jira] [Resolved] (OAK-9173) Oak-run indexing fails with "This map is closed"
[
https://issues.apache.org/jira/browse/OAK-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Mueller resolved OAK-9173.
-
Resolution: Fixed
> Oak-run indexing fails with "This map is closed"
>
>
> Key: OAK-9173
> URL: https://issues.apache.org/jira/browse/OAK-9173
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing, oak-run
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Major
> Fix For: 1.34.0
>
>
> Near the very end of the indexing, the log file contains the following error
> message
> {noformat}
> 06:21:13.704 [main] INFO o.a.j.oak.plugins.index.IndexUpdate - Reindexing
> Traversed #1337 /content/... [9179.83 nodes/s, 33047386.33 nodes/hr]
> (Elapsed 19.05 min, Expected 0.000 ns, Completed 99.95%)
> 06:21:14.524 [main] INFO o.a.j.o.i.i.d.f.l.PersistedLinkedList - Entries: 1
> map size: 1 file size: 9838592 bytes
> 06:21:17.463 [main] INFO o.a.j.o.i.i.d.f.l.PersistedLinkedList - Cache hits
> 467781832 misses 836251
> 06:21:17.463 [main] INFO o.a.j.o.i.i.d.f.l.PersistedLinkedList - Cache hits
> 467781980 misses 836251
> 06:21:24.400 [main] ERROR com.adobe.granite.indexing.tool.Main - Can't
> perform operation
> java.lang.IllegalStateException: This map is closed [1.4.194/4]
> at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:765)
> at org.h2.mvstore.MVMap.beforeWrite(MVMap.java:1044)
> at org.h2.mvstore.MVMap.remove(MVMap.java:542)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.linkedList.PersistedLinkedList.remove(PersistedLinkedList.java:140)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNextEntry(FlatFileStoreIterator.java:102)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNext(FlatFileStoreIterator.java:85)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNext(FlatFileStoreIterator.java:39)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.jackrabbit.oak.index.indexer.document.DocumentStoreIndexer.reindex(DocumentStoreIndexer.java:123)
> {noformat}
> The root cause seems to be:
> * The "main" iterator reaches the end, so that the map, and the file, is
> closed.
> * But then, a "clone of the iterator" (not sure if that's the right word) is
> still open and tries to continue.
> * This fails because the map is marked closed.
> We need to keep the map + file open for a longer time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OAK-9173) Oak-run indexing fails with "This map is closed"
[
https://issues.apache.org/jira/browse/OAK-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186314#comment-17186314
]
Thomas Mueller commented on OAK-9173:
-
Thanks [~reschke]! The test case should now be fixed in
http://svn.apache.org/r1881270
> Oak-run indexing fails with "This map is closed"
>
>
> Key: OAK-9173
> URL: https://issues.apache.org/jira/browse/OAK-9173
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing, oak-run
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Major
> Fix For: 1.34.0
>
>
> Near the very end of the indexing, the log file contains the following error
> message
> {noformat}
> 06:21:13.704 [main] INFO o.a.j.oak.plugins.index.IndexUpdate - Reindexing
> Traversed #1337 /content/... [9179.83 nodes/s, 33047386.33 nodes/hr]
> (Elapsed 19.05 min, Expected 0.000 ns, Completed 99.95%)
> 06:21:14.524 [main] INFO o.a.j.o.i.i.d.f.l.PersistedLinkedList - Entries: 1
> map size: 1 file size: 9838592 bytes
> 06:21:17.463 [main] INFO o.a.j.o.i.i.d.f.l.PersistedLinkedList - Cache hits
> 467781832 misses 836251
> 06:21:17.463 [main] INFO o.a.j.o.i.i.d.f.l.PersistedLinkedList - Cache hits
> 467781980 misses 836251
> 06:21:24.400 [main] ERROR com.adobe.granite.indexing.tool.Main - Can't
> perform operation
> java.lang.IllegalStateException: This map is closed [1.4.194/4]
> at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:765)
> at org.h2.mvstore.MVMap.beforeWrite(MVMap.java:1044)
> at org.h2.mvstore.MVMap.remove(MVMap.java:542)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.linkedList.PersistedLinkedList.remove(PersistedLinkedList.java:140)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNextEntry(FlatFileStoreIterator.java:102)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNext(FlatFileStoreIterator.java:85)
> at
> org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNext(FlatFileStoreIterator.java:39)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.jackrabbit.oak.index.indexer.document.DocumentStoreIndexer.reindex(DocumentStoreIndexer.java:123)
> {noformat}
> The root cause seems to be:
> * The "main" iterator reaches the end, so that the map, and the file, is
> closed.
> * But then, a "clone of the iterator" (not sure if that's the right word) is
> still open and tries to continue.
> * This fails because the map is marked closed.
> We need to keep the map + file open for a longer time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
